Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 11
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tintó, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel|||0000-0003-1826-623X
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco|||0000-0002-6622-4280
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN201453234C21R, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol
  • Castrillo Melguizo, Miguel
  • Acosta Cobos, Mario César
  • Mula Valls, Josep Oriol
  • Sánchez Lorente, Alícia
  • Serradell Maronda, Kim|||0000-0001-8230-4347
  • Cortés Fité, Ana
  • Doblas Reyes, Francisco
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge., The research leading to these results has received funding from the EU ESiWACE H2020 Framework Programme under grant agreement no. 675191, from the Severo Ochoa (SEV-2011-00067) programme of the Spanish Government and from the Ministerio de Economia y Competitividad under contract TIN2014-53234-C2-1R., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Replicability of the EC-Earth3 Earth system model under a change in computing environment

  • Massonnet, François
  • Menegoz, Martín
  • Acosta Cobos, Mario César
  • Yepes-Arbós, Xavier
  • Exarchou, Eleftheria|||0000-0002-1240-1368
  • Doblas Reyes, Francisco
Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solutions in the case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a “replicability” condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoff along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise., This research has been supported by the EU Commission – H2020 (grant nos. 727862: APPLICATE, 312979: IS-ENES2, 675191: ESiWACE, and 641727: PRIMAVERA) and the Spanish Ministerio de Economía y Competitividad (MINECO) (grant no. CGL2015-70353-R: HIATUS)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Crossing the chasm: how to develop weather and climate models for next generation computers?

  • Lawrence, Bryan N.
  • Rezny, Mike
  • Budich, Reinhard
  • Bauer, Peter
  • Behrens, Jörg
  • Carter, Mick
  • Deconinck, Willem
  • Ford, Rupert
  • Maynard, Christopher
  • Mullerworth, Steven
  • Osuna, Carlos
  • Porter, Andrew
  • Serradell, Kim
  • Valcke, Sophie
  • Wedi, Nils
  • Wilson, Simon
Weather and climate models are complex pieces of software which include many individual components, each of which is evolving under pressure to exploit advances in computing to enhance some combination of a range of possible improvements (higher spatio-temporal resolution, increased fidelity in terms of resolved processes, more quantification of uncertainty, etc.). However, after many years of a relatively stable computing environment with little choice in processing architecture or programming paradigm (basically X86 processors using MPI for parallelism), the existing menu of processor choices includes significant diversity, and more is on the horizon. This computational diversity, coupled with ever increasing software complexity, leads to the very real possibility that weather and climate modelling will arrive at a chasm which will separate scientific aspiration from our ability to develop and/or rapidly adapt codes to the available hardware. In this paper we review the hardware and software trends which are leading us towards this chasm, before describing current progress in addressing some of the tools which we may be able to use to bridge the chasm. This brief introduction to current tools and plans is followed by a discussion outlining the scientific requirements for quality model codes which have satisfactory performance and portability, while simultaneously supporting productive scientific evolution. We assert that the existing method of incremental model improvements employing small steps which adjust to the changing hardware environment is likely to be inadequate for crossing the chasm between aspiration and hardware at a satisfactory pace, in part because institutions cannot have all the relevant expertise in house. Instead, we outline a methodology based on large community efforts in engineering and standardisation, which will depend on identifying a taxonomy of key activities – perhaps based on existing efforts to develop domain-specific languages, identify common patterns in weather and climate codes, and develop community approaches to commonly needed tools and libraries – and then collaboratively building up those key components. Such a collaborative approach will depend on institutions, projects, and individuals adopting new interdependencies and ways of working., This paper reports European research funded by the following FW7 and H2020 research and innovation projects: IS-ENES2 under grant agreement 312979; ESCAPE under grant agreement no. 671627; and ESIWACE under grant agreement 675191. The authors acknowledge useful conversations with and input from Venkatramani Balaji, Terry Davies, Peter Fox, Rich Loft, Nigel Wood, and Andy Brown and the input of other participants at the “Crossing the Chasm” meeting, in particular Jean-Claude Andre, Joachim Biercamp, Antonio Cofiño, Marie-Alice Foujols, Sylvie Joussaume, Grenville Lister, Alastair Mckinstry, Annette Osprey, Øyvind Seland, and Manuel Vega., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tinto, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel|||0000-0003-1826-623X
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco|||0000-0002-6622-4280
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN-2014-53234-C2-1-R, Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tinto, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN-2014-53234-C2-1-R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Crossing the chasm: how to develop weather and climate models for next generation computers?

  • Lawrence, Bryan N.
  • Rezny, Mike
  • Budich, Reinhard
  • Bauer, Peter
  • Behrens, Jörg
  • Carter, Mick
  • Deconinck, Willem
  • Ford, Rupert
  • Maynard, Christopher
  • Mullerworth, Steven
  • Osuna, Carlos
  • Porter, Andrew
  • Serradell, Kim
  • Valcke, Sophie
  • Wedi, Nils
  • Wilson, Simon
Weather and climate models are complex pieces of software which include many individual components, each of which is evolving under pressure to exploit advances in computing to enhance some combination of a range of possible improvements (higher spatio-temporal resolution, increased fidelity in terms of resolved processes, more quantification of uncertainty, etc.). However, after many years of a relatively stable computing environment with little choice in processing architecture or programming paradigm (basically X86 processors using MPI for parallelism), the existing menu of processor choices includes significant diversity, and more is on the horizon. This computational diversity, coupled with ever increasing software complexity, leads to the very real possibility that weather and climate modelling will arrive at a chasm which will separate scientific aspiration from our ability to develop and/or rapidly adapt codes to the available hardware. In this paper we review the hardware and software trends which are leading us towards this chasm, before describing current progress in addressing some of the tools which we may be able to use to bridge the chasm. This brief introduction to current tools and plans is followed by a discussion outlining the scientific requirements for quality model codes which have satisfactory performance and portability, while simultaneously supporting productive scientific evolution. We assert that the existing method of incremental model improvements employing small steps which adjust to the changing hardware environment is likely to be inadequate for crossing the chasm between aspiration and hardware at a satisfactory pace, in part because institutions cannot have all the relevant expertise in house. Instead, we outline a methodology based on large community efforts in engineering and standardisation, which will depend on identifying a taxonomy of key activities – perhaps based on existing efforts to develop domain-specific languages, identify common patterns in weather and climate codes, and develop community approaches to commonly needed tools and libraries – and then collaboratively building up those key components. Such a collaborative approach will depend on institutions, projects, and individuals adopting new interdependencies and ways of working., This paper reports European research funded by the following FW7 and H2020 research and innovation projects: IS-ENES2 under grant agreement 312979; ESCAPE under grant agreement no. 671627; and ESIWACE under grant agreement 675191. The authors acknowledge useful conversations with and input from Venkatramani Balaji, Terry Davies, Peter Fox, Rich Loft, Nigel Wood, and Andy Brown and the input of other participants at the “Crossing the Chasm” meeting, in particular Jean-Claude Andre, Joachim Biercamp, Antonio Cofiño, Marie-Alice Foujols, Sylvie Joussaume, Grenville Lister, Alastair Mckinstry, Annette Osprey, Øyvind Seland, and Manuel Vega., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tintó, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN201453234C21R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Replicability of the EC-Earth3 Earth system model under a change in computing environment

  • Massonnet, François
  • Menegoz, Martín
  • Acosta Cobos, Mario César
  • Yepes-Arbós, Xavier
  • Exarchou, Eleftheria
  • Doblas Reyes, Francisco
Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solutions in the case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a “replicability” condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoff along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise., This research has been supported by the EU Commission – H2020 (grant nos. 727862: APPLICATE, 312979: IS-ENES2, 675191: ESiWACE, and 641727: PRIMAVERA) and the Spanish Ministerio de Economía y Competitividad (MINECO) (grant no. CGL2015-70353-R: HIATUS)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol
  • Castrillo Melguizo, Miguel
  • Acosta Cobos, Mario César
  • Mula Valls, Josep Oriol
  • Sánchez Lorente, Alícia
  • Serradell Maronda, Kim
  • Cortés Fité, Ana
  • Doblas Reyes, Francisco
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge., The research leading to these results has received funding from the EU ESiWACE H2020 Framework Programme under grant agreement no. 675191, from the Severo Ochoa (SEV-2011-00067) programme of the Spanish Government and from the Ministerio de Economia y Competitividad under contract TIN2014-53234-C2-1R., Peer Reviewed, Postprint (published version)
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol|||0000-0003-3182-3942
  • Castrillo, Miguel|||0000-0003-1826-623X
  • Acosta, Mario C.|||0000-0001-7054-8168
  • Mula-Valls, Oriol
  • Sanchez Lorente, Alicia
  • Serradell, Kim|||0000-0001-8230-4347
  • Cortés Fité, Ana|||0000-0003-1697-1293
  • Doblas-Reyes, Francisco J.|||0000-0002-6622-4280
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge.