RUC. Repositorio da Universidade da Coruña
Publicación
Artículo científico (JournalArticle)
Population based metaheuristics in Spark: Towards a general framework using PSO as a case study
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract]: Over the years metaheuristics have been successfully applied to optimization problems in many real-world applications. The increasing complexity and scale of the problems addressed has posed new challenges to researchers in the field. The application of distributed metaheuristics is a common approach to speed up the time to solution or improve its quality by leveraging traditional parallel programming models on platforms like multicore processors or computer clusters. More recently, the emergence of Cloud Computing and new programming models and frameworks for Big Data has facilitated access to an unprecedented amount of computational resources, which led to a growing interest in optimization frameworks that support the development and execution of distributed metaheuristics taking advantage of this potential. In this paper, we present the current status of development of one such framework that aims to provide support for the application of distributed population-based metaheuristics to the global optimization of large-scale problems in Spark. The framework provides a reduced set of abstractions to represent the general structure of population-based metaheuristics as templates and strategies to particularize them into concrete metaheuristics, as well as other nice features like out of the box implementations of the most common distributed models, full configurability through a human-friendly format, and the possibility of rapid prototyping and testing metaheuristics in the Spark shell. To validate the approach, a template for Particle Swarm Optimization (PSO) was implemented as a proof of concept, which includes strategies for instantiating different variants of the algorithm, configurable topologies, and sequential and distributed execution models., This work was supported by MCIN/AEI/10.13039/501100011033 [grant numbers PID2019-104184RB-I00 , PID2022-136435NB-I00 and PID2020-117271RB-C22 (BIODYNAMICS) ], PID2022 also funded by “ ERDF A way of making Europe ”, EU. Also supported by Xunta de Galicia [grant number ED431C 2021/30 ]. Funding for open access charge: Universidade da Coruña/CISUG., Xunta de Galicia; ED431C 2021/30
Proyecto:
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2019-104184RB-I00
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2022-136435NB-I00
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-117271RB-C22
HANDLE: http://hdl.handle.net/2183/36167
Ver en: http://hdl.handle.net/2183/36167