Pasar al contenido principal

OPTIMIZACION DE MODELOS DEL SISTEMA TERRESTRE EN EL CAMINO A LA NUEVA GENERACION DE SISTEMAS "EXASCALE" DE ALTO RENDIMIENTO

PID2020-116324RA-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario BARCELONA SUPERCOMPUTING CENTER CENTRO NACIONAL DE SUPERCOMPUTACION

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Long-term evaluation of surface air pollution in CAMSRA and MERRA-2 global reanalyses over Europe (2003–2020)

  • Lacima, Aleksander
  • Petetin, Hervé
  • Soret, Albert
  • Bowdalo, Dene
  • Jorba, Oriol
  • Chen, Zhaoyue
  • Méndez Turrubiates, Raúl Fernando
  • Achebak, Hicham
  • Ballester, Joan
  • Pérez García-Pando, Carlos
Over the last century, our societies have experienced a sharp increase in urban population and fossil-fuelled transportation, turning air pollution into a critical issue. It is therefore key to accurately characterize the spatiotemporal variability of surface air pollution in order to understand its effects upon the environment, knowledge that can then be used to design effective pollution reduction policies. Global atmospheric composition reanalyses offer great capabilities towards this characterization through assimilation of satellite measurements. However, they generally do not integrate surface measurements and thus remain affected by significant biases at ground level. In this study, we thoroughly evaluate two global atmospheric composition reanalyses, the Copernicus Atmosphere Monitoring Service (CAMSRA) and the Modern-Era Retrospective Analysis for Research and Applications v2 (MERRA-2), between 2003 and 2020, against independent surface measurements of O3, NO2, CO, SO2 and particulate matter (PM; both PM10 and PM2.5) over the European continent. Overall, both reanalyses present significant and persistent biases for almost all examined pollutants. CAMSRA clearly outperforms MERRA-2 in capturing the spatiotemporal variability of most pollutants, as shown by generally lower biases (all pollutants except for PM2.5), lower errors (all pollutants) and higher correlations (all pollutants except SO2). CAMSRA also outperforms MERRA-2 in capturing the annual trends found in all pollutants (except for SO2). Overall, CAMSRA tends to perform best for O3 and CO, followed by NO2 and PM10, while poorer results are typically found for SO2 and PM2.5. Higher correlations are generally found in autumn and/or winter for reactive gases. Compared to MERRA-2, CAMSRA assimilates a wider range of satellite products which, while enhancing the performance of the reanalysis in the troposphere (as shown by other studies), has a limited impact on the surface. The biases found in both reanalyses are likely explained by a combination of factors, including errors in emission inventories and/or sinks, a lack of surface data assimilation, and their relatively coarse resolution. Our results highlight the current limitations of reanalyses to represent surface pollution, which limits their applicability for health and environmental impact studies. When applied to reanalysis data, bias-correction methodologies based on surface observations should help to constrain the spatiotemporal variability of surface pollution and its associated impacts., We acknowledge Red Temática ACTRIS España (CGL2017-90884-REDT), PRACE and RES for awarding us access to the MareNostrum Supercomputer in the Barcelona Supercomputing Center, and H2020 ACTRIS IMP(#871115). Last but not least, we gratefully acknowledge the outstanding work done by the Python development teams behind some specific libraries, including numpy, scipy, pandas, xarray, matplotlib and cartopy. This research has received funding from the European Research Council (ERC), in the frame of the EARLY-ADAPT project (https://early-adapt.eu/, last access: 15 December 2022), under the European Union’s Horizon 2020 research and innovation programme (grant no. 865564), as well as the MITIGATE project (project no. PID2020-116324RA I00/AEI/10.13039/501100011033) from the Agencia Estatal de Investigación (AEI). We also acknowledge support by the AXA Research Fund.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

The computational and energy cost of simulation and storage for climate science: lessons from CMIP6

  • Acosta Cobos, Mario César
  • Palomas Martínez, Sergi
  • Paronuzzi Ticco, Stella Valentina
  • Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
  • Biercamp, Joachim
  • Bretonnière, Pierre-Antoine|||0000-0002-3066-6685
  • Budich, Reinhard
  • Castrillo Melguizo, Miguel
  • Caubel, Arnaud
  • Doblas Reyes, Francisco
The Coupled Model Intercomparison Project (CMIP) is one of the biggest international efforts aimed at better understanding the past, present, and future of climate changes in a multi-model context. A total of 21 model intercomparison projects (MIPs) were endorsed in its sixth phase (CMIP6), which included 190 different experiments that were used to simulate 40 000 years and produced around 40 PB of data in total. This paper presents the main findings obtained from the CPMIP (the Computational Performance Model Intercomparison Project), a collection of a common set of metrics, specifically designed for assessing climate model performance. These metrics were exclusively collected from the production runs of experiments used in CMIP6 and primarily from institutions within the IS-ENES3 consortium. The document presents the full set of CPMIP metrics per institution and experiment, including a detailed analysis and discussion of each of the measurements. During the analysis, we found a positive correlation between the core hours needed, the complexity of the models, and the resolution used. Likewise, we show that between 5 %–15 % of the execution cost is spent in the coupling between independent components, and it only gets worse by increasing the number of resources. From the data, it is clear that queue times have a great impact on the actual speed achieved and have a huge variability across different institutions, ranging from none to up to 78 % execution overhead. Furthermore, our evaluation shows that the estimated carbon footprint of running such big simulations within the IS-ENES3 consortium is 1692 t of CO2 equivalent. As a result of the collection, we contribute to the creation of a comprehensive database for future community reference, establishing a benchmark for evaluation and facilitating the multi-model, multi-platform comparisons crucial for understanding climate modelling performance. Given the diverse range of applications, configurations, and hardware utilised, further work is required for the standardisation and formulation of general rules. The paper concludes with recommendations for future exercises aimed at addressing the encountered challenges which will facilitate more collections of a similar nature., This research has been supported by the Barcelona Supercomputing Center (grant nos. PID2020116324RA-I00 (OEMES) and H2020-GA-824084 (ISENES))., Peer Reviewed, Article signat per 22 autors/es: Mario C. Acosta 1, Sergi Palomas 1, Stella V. Paronuzzi Ticco 1, Gladys Utrera 1, Joachim Biercamp 2, Pierre-Antoine Bretonniere 1, Reinhard Budich 3, Miguel Castrillo 1, Arnaud Caubel 4, Francisco Doblas-Reyes 1,14, Italo Epicoco 5, Uwe Fladrich 6, Sylvie Joussaume 4, Alok Kumar Gupta 7, Bryan Lawrence 8, Philippe Le Sager 9, Grenville Lister 8, Marie-Pierre Moine 10, Jean-Christophe Rioual 11, Sophie Valcke 10, Niki Zadeh 12, and Venkatramani Balaji 13 / 1- Barcelona Supercomputing Center, Plaça d’Eusebi Güell, 1–3, 08034 Barcelona, Spain; 2- German Climate Computing Centre, Bundesstraße 45a, 20146 Hamburg, Germany; 3- Max Planck Institute, Hofgartenstr. 8, 80539 Munich, Germany; 4- Institut Pierre-Simon Laplace, 11 Bd d’Alembert, 78280 Guyancourt, France; 5- Euro-Mediterranean Center on Climate Change, Via della Libertà, 12, 30121 Venice, Italy; 6- Swedish Meteorological and Hydrological Institute, 601 76 Norrköping, Sweden; 7- Norwegian Research Centre, Nygårdsgaten 112, 5008 Bergen, Norway; 8- National Centre for Atmospheric Science, Fairbairn House, 71–75 Clarendon Rd, Woodhouse, Leeds LS2 9PH, United Kingdom; 9- Royal Netherlands Meteorological Institute, Utrechtseweg 297, 3731 GA De Bilt, the Netherlands; 10- European Center for Advanced Research and Training in Scientific Computing, 42 Av. Gaspard Coriolis, 31100 Toulouse, France; 11- Meteorological Office, Fitzroy Road, Exeter, Devon, EX1 3PB, United Kingdom; 12- National Oceanic and Atmospheric Administration, 1401 Constitution Avenue NW, Room 5128, Washington, DC 20230, USA; 13- High Meadows Environmental Institute, Princeton University, Guyot Hall, Room 129, Princeton, NJ 08544-1003, USA; 14- Catalan Institution for Research and Advanced Studies, Passeig Lluís Companys 23, 08010 Barcelona, Spain
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Evaluating the impact of task aggregation in workflows with shared resource environments: use case for the MONARCH application

  • Giménez de Castro Marciani, Manuel|||0000-0002-9852-3322
  • Castrillo Melguizo, Miguel
  • Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
  • Acosta Cobos, Mario César
  • de Paula Kinoshita, Bruno
  • Doblas Reyes, Francisco
High Performance Computing (HPC) is commonly employed to run high-impact Earth System Model (ESM) simulations, such as those for climate change. However, running workflows of ESM simulations on cutting-edge platforms can take long due to the congestion of the system and the lack of coordination between current HPC schedulers and workflow manager systems (WfMS). The Earth Sciences community has estimated the time in queue to be between 10 % to 20 % of the runtime in climate prediction experiments, the most time-consuming exercise. To address this issue, the developers of Autosubmit, a WfMS tailored for climate and air quality sciences, have developed wrappers to join multiple subsequent workflow tasks into a single submission. However, although wrappers are widely used in production for community models such as EC-Earth3, MONARCH, and Destination Earth simulations, to our knowledge, the benefits and potential drawbacks have never been rigorously evaluated. In addition, with portability in mind, the developers proposed to wrap depending on the entitlement of the user to the machine. In the widely utilized Slurm scheduler, this factor is called fair share. The objective of this paper is to quantify the impact of wrapping on queue time and understand its relationship with the fair share and the job's CPU and runtime request. To do this, we used a Slurm simulator to reproduce the behavior of the scheduler and, to recreate a representative usage of an HPC platform, we generated synthetic static workloads from data of the LUMI supercomputer and a dynamic workload from a past flagship HPC platform. As an example, we introduced jobs modeled after the MONARCH air quality application in these workloads, which we tracked their queue time. We found that, by simply joining tasks, the total runtime of the simulation reduces up to 7 %, and we have indications that this value is larger in reality. This saving translates to absolute terms in at least eight days less wasted in queue time for half of the simulations from the IS-ENES3 consortium of CMIP6 simulations. We also identified a high inverse correlation, of -0.87, between the queue time and the fair share factor., This work received support from grant CEX2021-001148-S-20-5 funded by MICIU/AEI/10.13039/501100011033 and by FSE+. Mario Acosta is supported from the Spanish National Research Council through OEMES (PID2020-116324RA-I00). This research has been supported by the Ministerio de Ciencia e Innovación (grant no. CEX2021-001148-S-20-5) and the Consejo Superior de Investigaciones Científicas (grant no. PID2020-116324RA-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reducing time and computing costs in EC-Earth: an automatic load-balancing approach for coupled Earth system models

  • Palomas Martínez, Sergi
  • Acosta Cobos, Mario César
  • Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
  • Tourigny, Etienne|||0000-0003-4628-1461
Earth system models (ESMs) are intricate models employed for simulating the Earth's climate, typically constructed from distinct independent components dedicated to simulate specific natural phenomena (such as atmosphere and ocean dynamics, atmospheric chemistry, land and ocean biosphere). In order to capture the interactions between these processes, ESMs utilize coupling libraries, which oversee the synchronization and field exchanges among independent developed codes typically operating in parallel as a multi-program, multiple data (MPMD) application. The performance achieved depends on the coupling approach, as well as on the number of parallel resources and scalability properties of each component. Determining the appropriate number of resources to use for each component in coupled ESMs is crucial for efficient utilization of the high-performance computing (HPC) infrastructures used in climate modeling. However, this task traditionally involves manual testing of multiple process allocations by trial and error, requiring significant time investment from researchers and making the process more error-prone, often resulting in a loss of application performance due to the complexity of the task. This paper introduces the automatic load-balance tool (auto-lb), a methodology and tool for determining the resource allocation to each component within coupled ESMs, aimed at improving the application's performance. Notably, this methodology is automatic and does not require expertise in HPC to improve the performance achieved by coupled ESMs. This is accomplished by minimizing the load imbalance: reducing each constituent's execution cost (core hours), as well as minimizing the core hours wasted resulting from the synchronizations between them, without penalizing the execution speed of the entire model. This optimization is achieved regardless of the scalability properties of each constituent and the complexity of their dependencies during the coupling. To achieve this, we designed a new performance metric called “fittingness” to assess the performance of coupled execution, evaluating the trade-off between parallel efficiency and application throughput. This metric is intended for scenarios where optimality can depend on various criteria and constraints. Aiming for maximum speed might not be desirable if it leads to a decrease in parallel efficiency and thus increases the computational costs of simulation. The methodology was tested across multiple experiments using the widely recognized European ESM, EC-Earth3. The results were compared with real operational configurations, such as those used for the Coupled Model Intercomparison Project Phase 6 (CMIP6) and European Climate Prediction project (EUCP), and validated on different HPC platforms. All of them suggest that the current approaches lead to performance loss, and that auto-lb can achieve better results in both execution speed and reduction of the core hours needed. When comparing to the EC-Earth standard-resolution CPMIP6 runs, we achieved a configuration 4.7 % faster while also reducing the core hours required by 1.3 %. Likewise, when compared to the EC-Earth high-resolution EUCP runs, the method presented showed an improvement of 34 % in the speed, with a 6.7 % reduction in the core hours consumed., This research was supported by the European High Performance Computing Joint Undertaking (EuroHPC JU) and the European Union (EU) through the ESiWACE3 project under grant agreement no. 101093054, as well as the ISENES3 project under grant agreement no. H2020-GA-824084. Additional funding was provided by the National Research Agency through the OEMES project (grant no. PID2020-116324RA-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Balancing EC-Earth3 improving the performance of EC-Earth CMIP6 configurations by minimizing the coupling cost

  • Acosta Cobos, Mario César
  • Palomas Martínez, Sergi
  • Tourigny, Etienne|||0000-0003-4628-1461
Earth System Models (ESMs) are complex systems used in weather and climate studies generally built from different independent components responsible for simulating a specific realm (ocean, atmosphere, biosphere, etc.). To replicate the interactions between these processes, ESMs typically use coupling libraries that manage the synchronization and field exchanges between the individual components, which run in parallel as a Multi-Program, Multiple-Data application. As ESMs get more complex (increase in resolution, number of components, configurations, etc.), achieving the best performance when running in High-performance Computing platforms has become increasingly challenging and of major concern. One of the critical bottlenecks is the load-imbalance, where the fastest components will have to wait for the slower ones. Finding the optimal number of processing elements to assign to each of the multiple independent constituents to minimize the performance loss due to synchronizations and maximize the overall parallel efficiency is impossible without the right performance metrics, methodology, and tools. This paper presents the results of balancing multiple Coupled Model Intercomparison Project phase 6 configurations for the EC-Earth3 ESM. We will show that intuitive approaches can lead to suboptimal resource allocations and propose new setups up to 25% fasters while reducing the computational cost by 72%. We prove that new methods are needed to deal with the load-balance of ESMs and hope that our study will serve as a guide to optimize any other coupled system., The research leading to these results has received co-funding from the National Research Agency through OEMES (PID2020-116324RA-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Optimized thread-block arrangement in a GPU implementation of a linear solver for atmospheric chemistry mechanisms

  • Guzmán Ruiz, Christian
  • Acosta Cobos, Mario César
  • Jorba Casellas, Oriol|||0000-0001-5872-0244
  • Cesar Galobardes, Eduardo
  • Dawson, Matthew
  • Oyarzun Altamirano, Guillermo|||0000-0001-9524-3782
  • García Pando, Carlos Pérez
  • Serradell Maronda, Kim|||0000-0001-8230-4347
Earth system models (ESM) demand significant hardware resources and energy consumption to solve atmospheric chemistry processes. Recent studies have shown improved performance from running these models on GPU accelerators. Nonetheless, there is room for improvement in exploiting even more GPU resources. This study proposes an optimized distribution of the chemical solver's computational load on the GPU, named Block-cells. Additionally, we evaluate different configurations for distributing the computational load in an NVIDIA GPU. We use the linear solver from the Chemistry Across Multiple Phases (CAMP) framework as our test bed. An intermediate-complexity chemical mechanism under typical atmospheric conditions is used. Results demonstrate a 35× speedup compared to the single-CPU thread reference case. Even using the full resources of the node (40 physical cores) on the reference case, the Block-cells version outperforms them by 50%. The Block-cells approach shows promise in alleviating the computational burden of chemical solvers on GPU architectures., This work was partially supported by funding from the Ministerio de Ciencia, Innovación y Universidades as part of the BROWNING project (RTI2018-099894-BI00 funded by MCIN AEI/10.13039/ 501100011033) and part of the CAROL project (PID2020-113614RBC21 funded by MCIN AEI/10.13039/501100011033), the Generalitat de Catalunya (2021-SGR-00574, 2021-SGR-01550 and 2021- SGR00785) and the AXA Research Fund through the AXA Chair on Sand and Dust Storms established at the Barcelona Supercomputing Center (BSC). Matthew Dawson has received funding from the European Union’s Horizon 2020 research and innovation program under Marie Sklodowska Curie grant agreement no. 747048. Mario Acosta has received funding from the National Research Agency through OEMES (PID2020-116324RA-I00). This paper expresses the opinions of the authors and not necessarily those of the funding commissions., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Reducing time and computing costs in EC-Earth: an automatic load-balancing approach for coupled Earth system models

  • Palomas Martínez, Sergi
  • Acosta Cobos, Mario César
  • Utrera Iglesias, Gladys Miriam
  • Tourigny, Etienne
Earth system models (ESMs) are intricate models employed for simulating the Earth's climate, typically constructed from distinct independent components dedicated to simulate specific natural phenomena (such as atmosphere and ocean dynamics, atmospheric chemistry, land and ocean biosphere). In order to capture the interactions between these processes, ESMs utilize coupling libraries, which oversee the synchronization and field exchanges among independent developed codes typically operating in parallel as a multi-program, multiple data (MPMD) application. The performance achieved depends on the coupling approach, as well as on the number of parallel resources and scalability properties of each component. Determining the appropriate number of resources to use for each component in coupled ESMs is crucial for efficient utilization of the high-performance computing (HPC) infrastructures used in climate modeling. However, this task traditionally involves manual testing of multiple process allocations by trial and error, requiring significant time investment from researchers and making the process more error-prone, often resulting in a loss of application performance due to the complexity of the task. This paper introduces the automatic load-balance tool (auto-lb), a methodology and tool for determining the resource allocation to each component within coupled ESMs, aimed at improving the application's performance. Notably, this methodology is automatic and does not require expertise in HPC to improve the performance achieved by coupled ESMs. This is accomplished by minimizing the load imbalance: reducing each constituent's execution cost (core hours), as well as minimizing the core hours wasted resulting from the synchronizations between them, without penalizing the execution speed of the entire model. This optimization is achieved regardless of the scalability properties of each constituent and the complexity of their dependencies during the coupling. To achieve this, we designed a new performance metric called “fittingness” to assess the performance of coupled execution, evaluating the trade-off between parallel efficiency and application throughput. This metric is intended for scenarios where optimality can depend on various criteria and constraints. Aiming for maximum speed might not be desirable if it leads to a decrease in parallel efficiency and thus increases the computational costs of simulation. The methodology was tested across multiple experiments using the widely recognized European ESM, EC-Earth3. The results were compared with real operational configurations, such as those used for the Coupled Model Intercomparison Project Phase 6 (CMIP6) and European Climate Prediction project (EUCP), and validated on different HPC platforms. All of them suggest that the current approaches lead to performance loss, and that auto-lb can achieve better results in both execution speed and reduction of the core hours needed. When comparing to the EC-Earth standard-resolution CPMIP6 runs, we achieved a configuration 4.7 % faster while also reducing the core hours required by 1.3 %. Likewise, when compared to the EC-Earth high-resolution EUCP runs, the method presented showed an improvement of 34 % in the speed, with a 6.7 % reduction in the core hours consumed., This research was supported by the European High Performance Computing Joint Undertaking (EuroHPC JU) and the European Union (EU) through the ESiWACE3 project under grant agreement no. 101093054, as well as the ISENES3 project under grant agreement no. H2020-GA-824084. Additional funding was provided by the National Research Agency through the OEMES project (grant no. PID2020-116324RA-I00)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The computational and energy cost of simulation and storage for climate science: lessons from CMIP6

  • Acosta Cobos, Mario César
  • Palomas Martínez, Sergi
  • Paronuzzi Ticco, Stella Valentina
  • Utrera Iglesias, Gladys Miriam
  • Biercamp, Joachim
  • Bretonnière, Pierre-Antoine
  • Budich, Reinhard
  • Castrillo Melguizo, Miguel
  • Caubel, Arnaud
  • Doblas Reyes, Francisco
The Coupled Model Intercomparison Project (CMIP) is one of the biggest international efforts aimed at better understanding the past, present, and future of climate changes in a multi-model context. A total of 21 model intercomparison projects (MIPs) were endorsed in its sixth phase (CMIP6), which included 190 different experiments that were used to simulate 40 000 years and produced around 40 PB of data in total. This paper presents the main findings obtained from the CPMIP (the Computational Performance Model Intercomparison Project), a collection of a common set of metrics, specifically designed for assessing climate model performance. These metrics were exclusively collected from the production runs of experiments used in CMIP6 and primarily from institutions within the IS-ENES3 consortium. The document presents the full set of CPMIP metrics per institution and experiment, including a detailed analysis and discussion of each of the measurements. During the analysis, we found a positive correlation between the core hours needed, the complexity of the models, and the resolution used. Likewise, we show that between 5 %–15 % of the execution cost is spent in the coupling between independent components, and it only gets worse by increasing the number of resources. From the data, it is clear that queue times have a great impact on the actual speed achieved and have a huge variability across different institutions, ranging from none to up to 78 % execution overhead. Furthermore, our evaluation shows that the estimated carbon footprint of running such big simulations within the IS-ENES3 consortium is 1692 t of CO2 equivalent. As a result of the collection, we contribute to the creation of a comprehensive database for future community reference, establishing a benchmark for evaluation and facilitating the multi-model, multi-platform comparisons crucial for understanding climate modelling performance. Given the diverse range of applications, configurations, and hardware utilised, further work is required for the standardisation and formulation of general rules. The paper concludes with recommendations for future exercises aimed at addressing the encountered challenges which will facilitate more collections of a similar nature., This research has been supported by the Barcelona Supercomputing Center (grant nos. PID2020116324RA-I00 (OEMES) and H2020-GA-824084 (ISENES))., Peer Reviewed, Article signat per 22 autors/es: Mario C. Acosta 1, Sergi Palomas 1, Stella V. Paronuzzi Ticco 1, Gladys Utrera 1, Joachim Biercamp 2, Pierre-Antoine Bretonniere 1, Reinhard Budich 3, Miguel Castrillo 1, Arnaud Caubel 4, Francisco Doblas-Reyes 1,14, Italo Epicoco 5, Uwe Fladrich 6, Sylvie Joussaume 4, Alok Kumar Gupta 7, Bryan Lawrence 8, Philippe Le Sager 9, Grenville Lister 8, Marie-Pierre Moine 10, Jean-Christophe Rioual 11, Sophie Valcke 10, Niki Zadeh 12, and Venkatramani Balaji 13 / 1- Barcelona Supercomputing Center, Plaça d’Eusebi Güell, 1–3, 08034 Barcelona, Spain; 2- German Climate Computing Centre, Bundesstraße 45a, 20146 Hamburg, Germany; 3- Max Planck Institute, Hofgartenstr. 8, 80539 Munich, Germany; 4- Institut Pierre-Simon Laplace, 11 Bd d’Alembert, 78280 Guyancourt, France; 5- Euro-Mediterranean Center on Climate Change, Via della Libertà, 12, 30121 Venice, Italy; 6- Swedish Meteorological and Hydrological Institute, 601 76 Norrköping, Sweden; 7- Norwegian Research Centre, Nygårdsgaten 112, 5008 Bergen, Norway; 8- National Centre for Atmospheric Science, Fairbairn House, 71–75 Clarendon Rd, Woodhouse, Leeds LS2 9PH, United Kingdom; 9- Royal Netherlands Meteorological Institute, Utrechtseweg 297, 3731 GA De Bilt, the Netherlands; 10- European Center for Advanced Research and Training in Scientific Computing, 42 Av. Gaspard Coriolis, 31100 Toulouse, France; 11- Meteorological Office, Fitzroy Road, Exeter, Devon, EX1 3PB, United Kingdom; 12- National Oceanic and Atmospheric Administration, 1401 Constitution Avenue NW, Room 5128, Washington, DC 20230, USA; 13- High Meadows Environmental Institute, Princeton University, Guyot Hall, Room 129, Princeton, NJ 08544-1003, USA; 14- Catalan Institution for Research and Advanced Studies, Passeig Lluís Companys 23, 08010 Barcelona, Spain, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Evaluating the impact of task aggregation in workflows with shared resource environments: use case for the MONARCH application

  • Giménez de Castro Marciani, Manuel
  • Castrillo Melguizo, Miguel
  • Utrera Iglesias, Gladys Miriam
  • Acosta Cobos, Mario César
  • de Paula Kinoshita, Bruno
  • Doblas Reyes, Francisco
High Performance Computing (HPC) is commonly employed to run high-impact Earth System Model (ESM) simulations, such as those for climate change. However, running workflows of ESM simulations on cutting-edge platforms can take long due to the congestion of the system and the lack of coordination between current HPC schedulers and workflow manager systems (WfMS). The Earth Sciences community has estimated the time in queue to be between 10 % to 20 % of the runtime in climate prediction experiments, the most time-consuming exercise. To address this issue, the developers of Autosubmit, a WfMS tailored for climate and air quality sciences, have developed wrappers to join multiple subsequent workflow tasks into a single submission. However, although wrappers are widely used in production for community models such as EC-Earth3, MONARCH, and Destination Earth simulations, to our knowledge, the benefits and potential drawbacks have never been rigorously evaluated. In addition, with portability in mind, the developers proposed to wrap depending on the entitlement of the user to the machine. In the widely utilized Slurm scheduler, this factor is called fair share. The objective of this paper is to quantify the impact of wrapping on queue time and understand its relationship with the fair share and the job's CPU and runtime request. To do this, we used a Slurm simulator to reproduce the behavior of the scheduler and, to recreate a representative usage of an HPC platform, we generated synthetic static workloads from data of the LUMI supercomputer and a dynamic workload from a past flagship HPC platform. As an example, we introduced jobs modeled after the MONARCH air quality application in these workloads, which we tracked their queue time. We found that, by simply joining tasks, the total runtime of the simulation reduces up to 7 %, and we have indications that this value is larger in reality. This saving translates to absolute terms in at least eight days less wasted in queue time for half of the simulations from the IS-ENES3 consortium of CMIP6 simulations. We also identified a high inverse correlation, of -0.87, between the queue time and the fair share factor., This work received support from grant CEX2021-001148-S-20-5 funded by MICIU/AEI/10.13039/501100011033 and by FSE+. Mario Acosta is supported from the Spanish National Research Council through OEMES (PID2020-116324RA-I00). This research has been supported by the Ministerio de Ciencia e Innovación (grant no. CEX2021-001148-S-20-5) and the Consejo Superior de Investigaciones Científicas (grant no. PID2020-116324RA-I00)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Long-term evaluation of surface air pollution in CAMSRA and MERRA-2 global reanalyses over Europe (2003–2020)

  • Lacima, Aleksander
  • Petetin, Hervé
  • Soret, Albert
  • Bowdalo, Dene
  • Jorba, Oriol
  • Chen, Zhaoyue
  • Méndez Turrubiates, Raúl Fernando
  • Achebak, Hicham
  • Ballester, Joan
  • Pérez García-Pando, Carlos
Over the last century, our societies have experienced a sharp increase in urban population and fossil-fuelled transportation, turning air pollution into a critical issue. It is therefore key to accurately characterize the spatiotemporal variability of surface air pollution in order to understand its effects upon the environment, knowledge that can then be used to design effective pollution reduction policies. Global atmospheric composition reanalyses offer great capabilities towards this characterization through assimilation of satellite measurements. However, they generally do not integrate surface measurements and thus remain affected by significant biases at ground level. In this study, we thoroughly evaluate two global atmospheric composition reanalyses, the Copernicus Atmosphere Monitoring Service (CAMSRA) and the Modern-Era Retrospective Analysis for Research and Applications v2 (MERRA-2), between 2003 and 2020, against independent surface measurements of O3, NO2, CO, SO2 and particulate matter (PM; both PM10 and PM2.5) over the European continent. Overall, both reanalyses present significant and persistent biases for almost all examined pollutants. CAMSRA clearly outperforms MERRA-2 in capturing the spatiotemporal variability of most pollutants, as shown by generally lower biases (all pollutants except for PM2.5), lower errors (all pollutants) and higher correlations (all pollutants except SO2). CAMSRA also outperforms MERRA-2 in capturing the annual trends found in all pollutants (except for SO2). Overall, CAMSRA tends to perform best for O3 and CO, followed by NO2 and PM10, while poorer results are typically found for SO2 and PM2.5. Higher correlations are generally found in autumn and/or winter for reactive gases. Compared to MERRA-2, CAMSRA assimilates a wider range of satellite products which, while enhancing the performance of the reanalysis in the troposphere (as shown by other studies), has a limited impact on the surface. The biases found in both reanalyses are likely explained by a combination of factors, including errors in emission inventories and/or sinks, a lack of surface data assimilation, and their relatively coarse resolution. Our results highlight the current limitations of reanalyses to represent surface pollution, which limits their applicability for health and environmental impact studies. When applied to reanalysis data, bias-correction methodologies based on surface observations should help to constrain the spatiotemporal variability of surface pollution and its associated impacts., We acknowledge Red Temática ACTRIS España (CGL2017-90884-REDT), PRACE and RES for awarding us access to the MareNostrum Supercomputer in the Barcelona Supercomputing Center, and H2020 ACTRIS IMP(#871115). Last but not least, we gratefully acknowledge the outstanding work done by the Python development teams behind some specific libraries, including numpy, scipy, pandas, xarray, matplotlib and cartopy. This research has received funding from the European Research Council (ERC), in the frame of the EARLY-ADAPT project (https://early-adapt.eu/, last access: 15 December 2022), under the European Union’s Horizon 2020 research and innovation programme (grant no. 865564), as well as the MITIGATE project (project no. PID2020-116324RA I00/AEI/10.13039/501100011033) from the Agencia Estatal de Investigación (AEI). We also acknowledge support by the AXA Research Fund.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Optimized thread-block arrangement in a GPU implementation of a linear solver for atmospheric chemistry mechanisms

  • Guzmán Ruiz, Christian
  • Acosta Cobos, Mario César
  • Jorba Casellas, Oriol
  • Cesar Galobardes, Eduardo
  • Dawson, Matthew
  • Oyarzun Altamirano, Guillermo
  • García Pando, Carlos Pérez
  • Serradell Maronda, Kim
Earth system models (ESM) demand significant hardware resources and energy consumption to solve atmospheric chemistry processes. Recent studies have shown improved performance from running these models on GPU accelerators. Nonetheless, there is room for improvement in exploiting even more GPU resources. This study proposes an optimized distribution of the chemical solver's computational load on the GPU, named Block-cells. Additionally, we evaluate different configurations for distributing the computational load in an NVIDIA GPU. We use the linear solver from the Chemistry Across Multiple Phases (CAMP) framework as our test bed. An intermediate-complexity chemical mechanism under typical atmospheric conditions is used. Results demonstrate a 35× speedup compared to the single-CPU thread reference case. Even using the full resources of the node (40 physical cores) on the reference case, the Block-cells version outperforms them by 50%. The Block-cells approach shows promise in alleviating the computational burden of chemical solvers on GPU architectures., This work was partially supported by funding from the Ministerio de Ciencia, Innovación y Universidades as part of the BROWNING project (RTI2018-099894-BI00 funded by MCIN AEI/10.13039/ 501100011033) and part of the CAROL project (PID2020-113614RBC21 funded by MCIN AEI/10.13039/501100011033), the Generalitat de Catalunya (2021-SGR-00574, 2021-SGR-01550 and 2021- SGR00785) and the AXA Research Fund through the AXA Chair on Sand and Dust Storms established at the Barcelona Supercomputing Center (BSC). Matthew Dawson has received funding from the European Union’s Horizon 2020 research and innovation program under Marie Sklodowska Curie grant agreement no. 747048. Mario Acosta has received funding from the National Research Agency through OEMES (PID2020-116324RA-I00). This paper expresses the opinions of the authors and not necessarily those of the funding commissions., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Balancing EC-Earth3 improving the performance of EC-Earth CMIP6 configurations by minimizing the coupling cost

  • Acosta Cobos, Mario César
  • Palomas Martínez, Sergi
  • Tourigny, Etienne
Earth System Models (ESMs) are complex systems used in weather and climate studies generally built from different independent components responsible for simulating a specific realm (ocean, atmosphere, biosphere, etc.). To replicate the interactions between these processes, ESMs typically use coupling libraries that manage the synchronization and field exchanges between the individual components, which run in parallel as a Multi-Program, Multiple-Data application. As ESMs get more complex (increase in resolution, number of components, configurations, etc.), achieving the best performance when running in High-performance Computing platforms has become increasingly challenging and of major concern. One of the critical bottlenecks is the load-imbalance, where the fastest components will have to wait for the slower ones. Finding the optimal number of processing elements to assign to each of the multiple independent constituents to minimize the performance loss due to synchronizations and maximize the overall parallel efficiency is impossible without the right performance metrics, methodology, and tools. This paper presents the results of balancing multiple Coupled Model Intercomparison Project phase 6 configurations for the EC-Earth3 ESM. We will show that intuitive approaches can lead to suboptimal resource allocations and propose new setups up to 25% fasters while reducing the computational cost by 72%. We prove that new methods are needed to deal with the load-balance of ESMs and hope that our study will serve as a guide to optimize any other coupled system., The research leading to these results has received co-funding from the National Research Agency through OEMES (PID2020-116324RA-I00)., Peer Reviewed, Postprint (published version)
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Optimized thread-block arrangement in a GPU implementation of a linear solver for atmospheric chemistry mechanisms

  • Guzman Ruiz, Christian|||0000-0003-3709-6483
  • Acosta, Mario C.|||0000-0001-7054-8168
  • Jorba, Oriol|||0000-0001-5872-0244
  • César Galobardes, Eduardo|||0000-0002-9729-8557
  • Dawson, Matthew
  • Oyarzun, Guillermo
  • Pérez García-Pando, Carlos|||0000-0002-4456-0697
  • Serradell, Kim|||0000-0001-8230-4347
Earth system models (ESM) demand significant hardware resources and energy consumption to solve atmospheric chemistry processes. Recent studies have shown improved performance from running these models on GPU accelerators. Nonetheless, there is room for improvement in exploiting even more GPU resources. This study proposes an optimized distribution of the chemical solver's computational load on the GPU, named Block-cells. Additionally, we evaluate different configurations for distributing the computational load in an NVIDIA GPU. We use the linear solver from the Chemistry Across Multiple Phases (CAMP) framework as our test bed. An intermediate-complexity chemical mechanism under typical atmospheric conditions is used. Results demonstrate a 35x speedup compared to the single-CPU thread reference case. Even using the full resources of the node (40 physical cores) on the reference case, the Block-cells version outperforms them by 50%. The Block-cells approach shows promise in alleviating the computational burden of chemical solvers on GPU architectures.