RUC. Repositorio da Universidade da Coruña
Publicación
Artículo científico (JournalArticle)
Serverless-like platform for container-based YARN clusters
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract]: Serverless computing is an emerging paradigm that has gained a lot of relevance in recent years, as it allows users to consume computing resources without worrying about the underlying infrastructure and pay only for what they actually use. Most current services that implement this paradigm typically rely on the Function-as-a-Service (FaaS) model, which works perfectly for simple applications based on stateless functions triggered by specific events. However, these services are not designed to run more complex applications with intricate interactions, usually presenting a significant degree of configuration difficulty and/or low ability to customise the execution environment. They also tend to be designed for short and simple workloads, with some services even limiting their maximum runtime to just a few minutes. In this paper, we present a platform based on Hadoop YARN oriented to the execution of Big Data workloads in a containerised and serverless way, so that the resources allocated to such containers are automatically and dynamically scaled according to their actual usage. An experimental evaluation has been carried out to compare our serverless-like platform with a standard YARN deployment when executing Big Data workloads concurrently. Our results have shown experimental evidence of enhancing both performance and overall resource efficiency, providing runtime reductions and resource usage improvements of up to 41% and 50%, respectively., This work was supported by grants PID2019-104184RB-I00, PDC20
21-121309-I00, TED2021-129177B-I00 and PID2022-136435NB-I00,
funded by the Ministry of Science and Innovation of Spain, MCIN/AEI/
10.13039/501100011033 (PDC2021 and TED2021 also funded by
‘‘NextGenerationEU’’/PRTR, and PID2022 by ‘‘ERDF A way of making
Europe’’, EU). It was also supported by Xunta de Galicia, Spain (predoctoral
fellowship ED481A 2022/067). Funding for open access charge:
Universidade da Coruña/CISUG., Xunta de Galicia; ED481A 2022/067
Proyecto:
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2019-104184RB-I00
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2022-136435NB-I00
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PDC2021-121309-I00
,
MICINN / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2024 TED2021-129177B-I00
HANDLE: http://hdl.handle.net/2183/36355
Ver en: http://hdl.handle.net/2183/36355