Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 2
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

BDWatchdogFaaS: A Tool for Monitoring and Analysis of Functions-as-a-Service in Cloud Environment

  • Framil de Amorín, Manuel
  • Carpente, Mario
  • Fraga, David
  • Enes, Jonatan
  • Expósito, Roberto R.
  • Taboada, Guillermo L.
  • Touriño, Juan
Cursos e Congresos , C-155, BDWatchdog is a framework to assist in the in-depth and real-time analysis of the execution of Big Data frameworks and applications. BDWatchdog was originally developed to monitor Hadoop ecosystems deployed on serverless containers, in order to detect bottlenecks and spot certain patterns that frameworks or applications may have. In thisworkwe shift the focus to monitoring serverless functions in the public cloud, by proposing an extension of BDWatchdog which captures, transforms and analyzes logs from both AWS Cloudwatch and Azure Application Insights, which store the logs from AWS Lambda and Azure Functions (respectively), the FaaS (Function-as-a-Service) solutions of the two main public cloud providers, AWS and Azure. The extension, called BDWatchdogFaaS, builds and stores a common model to both providers, allowing to consult, analyze and monitor function logs from AWS and Azure indistinctly. The transformation of logs into the common model is done by a FaaS of the corresponding provider, which in near real-time ingests, processes and sends the data to a common storage. In addition, the data is forwarded to a Power BI dashboard so that the serverless functions can be monitored easily, This work was funded by the Ministry of Science and Innovation of Spain (ref. PDC2021- 121309-I00/MCIN/AEI/10.13039/501100011033) and by the European Union “NextGenerationEU/ PRTR”. CITIC is funded by the Xunta de Galicia through the collaboration agreement between the Conseller´ıa de Cultura, Educaci´on, Formaci´on Profesional e Universidades and the Galician universities for the reinforcement of the research centres of the Galician University System (CIGUS)
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Serverless-like platform for container-based YARN clusters

  • Castellanos Rodríguez, Óscar
  • Expósito, Roberto R.
  • Enes, Jonatan
  • Taboada, Guillermo L.
  • Touriño, Juan
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract]: Serverless computing is an emerging paradigm that has gained a lot of relevance in recent years, as it allows users to consume computing resources without worrying about the underlying infrastructure and pay only for what they actually use. Most current services that implement this paradigm typically rely on the Function-as-a-Service (FaaS) model, which works perfectly for simple applications based on stateless functions triggered by specific events. However, these services are not designed to run more complex applications with intricate interactions, usually presenting a significant degree of configuration difficulty and/or low ability to customise the execution environment. They also tend to be designed for short and simple workloads, with some services even limiting their maximum runtime to just a few minutes. In this paper, we present a platform based on Hadoop YARN oriented to the execution of Big Data workloads in a containerised and serverless way, so that the resources allocated to such containers are automatically and dynamically scaled according to their actual usage. An experimental evaluation has been carried out to compare our serverless-like platform with a standard YARN deployment when executing Big Data workloads concurrently. Our results have shown experimental evidence of enhancing both performance and overall resource efficiency, providing runtime reductions and resource usage improvements of up to 41% and 50%, respectively., This work was supported by grants PID2019-104184RB-I00, PDC20 21-121309-I00, TED2021-129177B-I00 and PID2022-136435NB-I00, funded by the Ministry of Science and Innovation of Spain, MCIN/AEI/ 10.13039/501100011033 (PDC2021 and TED2021 also funded by ‘‘NextGenerationEU’’/PRTR, and PID2022 by ‘‘ERDF A way of making Europe’’, EU). It was also supported by Xunta de Galicia, Spain (predoctoral fellowship ED481A 2022/067). Funding for open access charge: Universidade da Coruña/CISUG., Xunta de Galicia; ED481A 2022/067