Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

ALPI: Enhancing portability and interoperability of task-aware libraries

  • Sala Penadés, Kevin
  • Álvarez Robert, David
  • Peñacoba Veigas, Raúl
  • Arias Mallo, Rodrigo
  • Navarro Muñoz, Antoni
  • Roca Nonell, Aleix
  • Beltran Querol, Vicenç
Task-based programming models are a promising approach to exploiting complex distributed and heterogeneous systems. However, integrating different communication, offloading, and storage APIs within tasks poses performance and deadlock risks. Several Task-Aware libraries, such as TAMPI, TASIO, and TACUDA, have been developed to integrate blocking and non-blocking APIs within task-based programming models efficiently. In this paper, we introduce the Asynchronous Low-level Programming Interface (ALPI) to enable the interoperability and portability of Task-Aware libraries across various programming models and runtime systems. We have implemented ALPI in the Nanos6 and nOS-V runtimes, enhancing the integration of Task-Aware libraries with the OmpSs-2 and OpenMP programming models. This work is a step towards improving the composability of parallel programming models by supporting Task-Aware libraries across different runtime systems., This work was supported by the Spanish Ministry of Science and Innovation (grant PID2019-107255GB) and the Severo Ochoa Program (grant CEX2021-001148-S), both funded by MCIN/AEI/10.13039/501100011033. The Generalitat de Catalunya also supported this work via grant 2021-SGR-01007., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

DRAM errors and cosmic rays: space invaders or science fiction?

  • Boixaderas Coderch, Isaac
  • Amaya, Jorge
  • Moré Codina, Sergi
  • Bartolomé Rodríguez, Javier
  • Vicente Dorca, David
  • Unsal, Osman Sabri
  • Gizopoulos, Dimitris
  • Carpenter, Paul Matthew
  • Radojkovic, Petar
  • Ayguadé Parra, Eduard
It is widely accepted that cosmic rays are a plausible cause of DRAM errors in high-performance computing (HPC) systems, and various studies suggest that they could explain some aspects of the observed DRAM error behavior. However, this phenomenon is insufficiently studied in production environments. We analyze the correlations between cosmic rays and DRAM errors on two HPC clusters: a production supercomputer with server-class DDR3-1600 and a prototype with LPDDR3-1600 and no hardware error correction. Our error logs cover 2000 billion MB-hours for the MareNostrum 3 supercomputer and 135 million MB-hours for the Mont-Blanc prototype. Our analysis combines quantitative analysis, formal statistical methods and machine learning. We detect no indications that cosmic rays have any influence on the DRAM errors. To understand whether the findings are specific to systems under study, located at 100 meters above the sea level, the analysis should be repeated on other HPC clusters, especially the ones located on higher altitudes. Also, analysis can (and should) be applied to revisit and extend numerous previous studies which use cosmic rays as a hypothetical explanation for some aspects of the observed DRAM error behaviors., The research was supported by the Spanish Government through the contracts PID2019-107255GB-C21, PID2023-146511NB-I00 and CEX2021-001148-S, funded by MCIN/AEI/10.13039/501100011033. The work also received funding from the Department of Research and Universities of the Government of Catalonia (Code: 2021 SGR 00807, 2021 SGR 01007, 2021 SGR 01264). Paul Carpenter holds the Ramon y Cajal fellowship RYC2018-025628-I funded by MICIU/AEI/10.13039/501100011033 and ”ESF Investing in your future”. Support also came from the European Union’s Horizon Europe research and innovation programme under grant agreement No. 101097224 (REBECCA). Further financial assistance was provided by the Barcelona Zettascale Laboratory (reference REGAGE22e00058408992), which is financed by the Ministry for Digital Transformation and Public Services within the framework of the Resilience and Recovery Facility, and the European Union NextGenerationEU initiative. We acknowledge the NMDB database (www.nmdb.eu), founded under the European Union’s FP7 programme (contract no. 213007), for supplying part of the neutron monitor data., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Supporting OpenMP free-agents by leveraging nOS-V

  • Peñacoba Veigas, Raúl
  • Álvarez Robert, David
  • Arcila Larrea, Vincent A.
  • Navarro Muñoz, Antoni
  • Beltran Querol, Vicenç
The OpenMP 6.0 standard introduces the free-agents feature, which enables a more dynamic execution model to improve application malleability and resource utilization. With free-agents, idle threads within a parallel region can be dynamically reassigned to execute tasks from other parallel regions. However, integrating this feature into complex runtimes, such as LLVM’s libomp, presents significant challenges due to its departure from traditional threading models. This paper presents our implementation of free-agents within the LLVM libomp runtime, which significantly reduces the complexity of implementing free-agents by leveraging the nOS-V threading and tasking library. Moreover, nOS-V also provides support for Task-Aware (TA) libraries and the co-execution of applications. Our enhanced libomp runtime, ported on top of nOS-V (libompv), also leverages these features. With libompv, OpenMP tasks and threads from any parallel region can safely call TA libraries, and independent processes can be efficiently co-executed to improve resource utilization or mitigate load imbalance issues. Experimental results show that libompv with free-agents delivers better performance than the original libomp for load-imbalanced or co-executed applications, while introducing no additional overhead for workloads that do not benefit from these new features., This work was partially supported by the Generalitat de Catalunya (contract 2021-SGR-01007) and the Spanish Government through the Severo Ochoa Program (CEX2021-001148-S/MCIN/AEI/10.13039/501100011033), and it is part of the ST4HPC (PID2023-147979NB-C21) research project, funded by MICIU /AEI/10.13039 /501100011033 and by FEDER, UE. This paper is also cofinanced by the Barcelona Zettaescale Laboratory, promoted by the Ministry for Digital Transformation and the Civil Service, within the framework of the Recovery, Transformation and Resilience Plan - Funded by the European Union - NextGenerationEU., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Managing workflow malleability in urgent computing for earthquake alerts

  • Ejarque Artigas, Jorge
  • Monterrubio Velasco, Marisol
  • Bhihe, Cedric
  • Pienkowska, Marta
  • de la Puente, Josep
  • Badia Sala, Rosa Maria
When large earthquakes happen, first responders need fast and accurate information regarding their impact. UCIS4EQ is an urgent computing platform that estimates ground shaking based on high-performance parallel 3D simulations. In this work, we present the PyCOMPSs implementation of UCIS4EQ towards urgent high-performance computing with a particular focus on providing a malleability mechanism. This allows UCIS4EQ to scan for data updates during runtime and dynamically incorporate said data into its execution. This involves, in particular, leveraging several HPC jobs. The implementation has been validated with data from the Samos Izmir earthquake. The malleable version of the workflow can reduce the use of computational resources by 40% while reducing the variability of the results and keeping the same time-to-solution as a non-malleable run., This work has been supported by the Spanish Government (PID2019-107255GB) and by MCIN/AEI/ 10.13039/501100011033 (CEX2021-001148-S), by Generalitat de Catalunya (contract 2021-SGR-00412), and by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558 and by MCIN/AEI/ 10.13039/501100011033 and the European Union NextGenerationEU/ PRTR (project eFlows4HPC, PCI2021-121957)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The TEXTAROSSA project: Cool all the way down to the hardware

  • Filgueras Izquierdo, Antonio
  • Agosta, Giovanni
  • Aldinucci, Marco
  • Álvarez Martínez, Carlos
  • D'Ambra, Pasqua
  • De Haro Ruiz, Juan Miguel
  • Jiménez González, Daniel
  • Martorell Bofill, Xavier
  • Morais, Lucas Henrique
  • Vidal, Miquel
The TEXTAROSSA project aims to bridge the technology gaps that exascale computing systems will face in the near future in order to overcome their performance and energy efficiency challenges. This project provides solutions for improved energy efficiency and thermal control, seamless integration of heterogeneous accelerators in HPC multi-node platforms, and new arithmetic methods. Challenges are tacked through a co-design approach to heterogeneous HPC solutions, supported by the integration and extension of HW and SW IPs, programming models, and tools derived from European research., This work is supported by the (EuroHPC) TEXTAROSSA project G.A. n.956831, by the Spanish Government (Grants PCI2021-121964 - TEXTAROSSA, PID2019-107255GB-C21MCIN/AEI/10.13039/501100011033, and CEX2021-001148-S), by Generalitat de Catalunya (2021 SGR 01007), by the Italian National Resilience and Recovery Plan (PNRR) via the National Center for HPC, Big Data and Quantum Computing, by the Italian Departments of Excellence grant 2023-2027 Quantum Frontiers, by the Italian Ministry of Economic Development (MISE) project n. 2774 ”TEXTAROSSA”, the European Union via projects EuRyQa and QuantEra’s T-NISQ, and the German Federal Ministry of Education and Research (BMBF) via QRydDemo., Peer Reviewed, Article signat per 41 autors/es: Antonio Filgueras, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Giovanni Agosta, DEIB - Politecnico di Milano, Italy, Centro Interuniversitario Nazionale per l'Informatica (CINI), Italy; Marco Aldinucci, National Research Council (CNR), Italy, University of Torino, Italy; Carlos Álvarez, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Pasqua D’Ambra, National Research Council (CNR), Italy; Massimo Bernaschi, National Research Council (CNR), Italy; Andrea Biagioni, INFN, Sezione di Roma, Italy; Daniele Cattaneo, DEIB - Politecnico di Milano, Italy; Alessandro Celestini, National Research Council (CNR), Italy; Massimo Celino, IENEA, Italy; Carlotta Chiarini, INFN, Sezione di Roma, Italy, Università “Sapienza” di Roma, Italy; Francesca Lo Cicero, INFN, Sezione di Roma, Italy; Paolo Cretaro, INFN, Sezione di Roma, Italy; William Fornaciari, DEIB - Politecnico di Milano, Italy, National Research Council (CNR), Italy; Ottorino Frezza, INFN, Sezione di Roma, Italy; Andrea Galimberti, DEIB - Politecnico di Milano, Italy; Francesco Giacomini, INFN. CNAF., Italy; Juan Miguel de Haro Ruiz, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Francesco Iannone, IENEA, Italy; Daniel Jaschke, lnstitute for Complex Quantum Systems, Ulm University, Germany, INFN, Sezione di Padova, Italy, Dipartimento di Fisica e Astronomia “G. Galilei”, Università degli Studi di Padova, Italy; Daniel Jiménez-González, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Michal Kulczewski, Poznan Supercomputing and Networking Center, Poland; Alberto Leva, DEIB - Politecnico di Milano, Italy; Alessandro Lonardo, INFN, Sezione di Roma, Italy; Michele Martinelli, INFN, Sezione di Roma, Italy; Xavier Martorell, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Simone Montangero, Dipartimento di Fisica e Astronomia “G. Galilei”, Università degli Studi di Padova, Italy, Padua Quantum Technologies Research Center, Italy, INFN, Sezione di Padova, Italy, lnstitute for Complex Quantum Systems, Ulm University, Germany; Lucas Morais, Universitat Politècnica de Catalunya, Spain, Barcelona Supercomputing Center, Spain; Ariel Oleksiak, Poznan Supercomputing and Networking Center, Poland; Paolo Palazzari, IENEA, Italy; Luca Pontisso, INFN, Sezione di Roma, Italy; Federico Reghenzani, DEIB - Politecnico di Milano, Italy, National Research Council (CNR), Italy; Cristian Rossi, INFN, Sezione di Roma, Italy, Università “Sapienza” di Roma, Italy; Sergio Saponarat, University of Pisa, Italy, DEIB - Politecnico di Milano, Italy, Centro Interuniversitario Nazionale per l'Informatica (CINI), Italy; Carlo Saverio Lodi, DEIB - Politecnico di Milano, Italy; Francesco Simula, INFN, Sezione di Roma, Italy; Federico Terraneo, DEIB - Politecnico di Milano, Italy, Centro Interuniversitario Nazionale per l'Informatica (CINI), Italy; Piero Vicini, INFN, Sezione di Roma, Italy; Miquel Vidal, Barcelona Supercomputing Center, Spain; Davide Zoni, DEIB - Politecnico di Milano, Italy; Giuseppe Zummo, IENEA, Italy, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Makinote: An FPGA-based HW/SW platform for pre-silicon emulation of RISC-V designs

  • Perdomo Hourné, Elias Augusto
  • Kropotov, Alexander
  • Cano Ladino, Francelly Katherine
  • Ali Shah, Syed Zafar
  • Cervero García, Teresa
  • Martorell Bofill, Xavier
  • Salami, Behzad
Emulating chip functionality before silicon production is crucial, especially with the increasing prevalence of RISC-V-based designs. FPGAs are promising candidates for such purposes due to their high-speed and reconfigurable architecture. In this paper, we introduce our Makinote, an FPGA-based Cluster platform, hosted at Barcelona Supercomputing Center (BSC-CNS), which is composed of a large number of FPGAs (in total 96 AMD/Xilinx Alveo U55c) to emulate massive size RTL designs (up to 750M ASIC cells)1. In addition, we introduce our FPGA shell as a powerful tool to facilitate the utilization of such a large FPGA cluster with minimal effort needed by the designers. The proposed FPGA shell provides an easy-to-use interface for the RTL developers to rapidly port such design into several FPGAs by automatically connecting to the necessary ports, e.g., PCIe Gen4, DRAM (DDR4 and HBM), ETH10g/100g. Moreover, specific drivers for exploiting RISC-V based architectures are provided within the set of tools associated with the FPGA shell. We release the tool online for further extensions.2 We validate the efficiency of our hardware platform (i.e., FPGA cluster) and the software tool (i.e., FPGA Shell) by emulating a RISC-V processor and experimenting HPC Challenge application running on 32 FPGAs. Our results demonstrate that the performance improves by 8 times over the single-FPGA case., This work has received funding from the MEEP project (European High-Performance Computing Joint Undertaking (JU) under grant agreement No 946002), from ”Lenovo-BSC Contract-Framework (2022)”, from the Spanish Government (PID2019-107255GB-C21/AEI /10.13039/501100011033, CEX2021-001148-S funded by MCIN/AEI / 10.13039/501100011033), and from Generalitat de Catalunya (contract 2021-SGR-01007)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

nOS-V: Co-executing HPC applications using system-wide task scheduling

  • Álvarez Robert, David
  • Sala Penadés, Kevin
  • Beltran Querol, Vicenç
Future Exascale systems will feature massive parallelism, many-core processors and heterogeneous architectures. In this scenario, it is increasingly difficult for HPC applications to fully and efficiently utilize the resources in system nodes. Moreover, the increased parallelism exacerbates the effects of existing inefficiencies in current applications. Research has shown that co-scheduling applications to share system nodes instead of executing each application exclusively can increase resource utilization and efficiency. Nevertheless, the current oversubscription and co-location techniques to share nodes have several drawbacks which limit their applicability and make them very application-dependent.This paper presents co-execution through system-wide scheduling. Co-execution is a novel fine-grained technique to execute multiple HPC applications simultaneously on the same node, outperforming current state-of-the-art approaches. We implement this technique in nOS-V, a lightweight tasking library that supports co-execution through system-wide task scheduling. Moreover, nOS-V can be easily integrated with existing programming models, requiring no changes to user applications. We showcase how co-execution with nOS-V significantly reduces schedule makespan for several applications on different scenarios, outperforming prior node-sharing techniques., This work was supported by the Spanish Ministry of Science and Innovation (grant PID2019-107255GB) and the Severo Ochoa Program (grant CEX2021-001148-S), both funded by MCIN/AEI/10.13039/501100011033. The Generalitat de Catalunya also supported this work via grant 2021-SGR-01007., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

BerryBees: Breadth first search by bit-tensor-cores

  • Niu, Yuyao
  • Casas, Marc
Breadth First Search (BFS) plays a key role in computational science, networking, and artificial intelligence applications. Although the BFS approach has been extensively studied, particularly in its direction-optimized form, existing implementations still present three main issues: (1) high memory footprint; (2) the under-realized lightweight representations using bitmaps; and (3) the underuse of modern hardware such as Tensor Cores Units (TCUs). In this paper, we propose BerryBees, an efficient algebraic BFS algorithm that leverages the Matrix Multiply-Accumulate (MMA) instructions of TCUs. The novelty of BerryBees lies in (1) the Binarized Row Slice (BRS) format, which encodes the adjacency matrix by using bitmaps to represent non-empty row segments; and (2) a warp-level algorithm that leverages TCUs for accelerating both SpMV and SpMSpV operations for enhanced BFS performance. The experimental results on three latest NVIDIA GPUs show that BerryBees outperforms five state-of-the-art BFS methods: GAP, Gunrock, Enterprise, GSWITCH, and GraphBLAST, and delivers average speedups of 1.42 ×, 1.97×, 5.05×, 1.24×, and 3.74× (up to 9.99×, 13.66×, 114.07×, 13.97×, and 24.74×), respectively., This work has been partially supported by the European HiPEAC Network of Excellence, the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21 and PID2019-105660RBC22), and the Generalitat de Catalunya (contract 2021-SGR00864). Yuyao Niu is supported by Grant PRE2022-104335, funded by MICIU/AEI/10.13039/501100011033, and by the "ESF+" Grant CEX2021-001148-S-20-2, funded by MICIU/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Reinforcement learning-based adaptive mitigation of uncorrected DRAM errors in the field

  • Boixaderas Coderch, Isaac
  • Moré Codina, Sergi
  • Bartolomé Rodríguez, Javier
  • Vicente Dorca, David
  • Radojkovic, Petar
  • Carpenter, Paul Matthew
  • Ayguadé Parra, Eduard
Scaling to larger systems, with current levels of reliability, requires cost-effective methods to mitigate hardware failures. One of the main causes of hardware failure is an uncorrected error in memory, which terminates the current job and wastes all computation since the last checkpoint. This paper presents the first adaptive method for triggering uncorrected error mitigation. It uses a prediction approach that considers the likelihood of an uncorrected error and its current potential cost. The method is based on reinforcement learning, and the only user-defined parameters are the mitigation cost and whether the job can be restarted from a mitigation point. We evaluate our method using classical machine learning metrics together with a cost-benefit analysis, which compares the cost of mitigation actions with the benefits from mitigating some of the errors. On two years of production logs from the MareNostrum supercomputer, our method reduces lost compute time by 54% compared with no mitigation and is just 6% below the optimal Oracle method. All source code is open source., The work was supported by the Spanish Government, under the contracts PID2019-107255GB-C21 and CEX2021-001148-S funded by MCIN/AEI/ 10.13039/501100011033. The work also received funding from the Department of Research and Universities of the Government of Catalonia to the AccMem Research Group (Code: 2021 SGR 00807). Paul Carpenter holds the Ramon y Cajal fellowship RYC2018-025628-I funded by MICIU/AEI/10.13039/501100011033 and "ESF Investing in your future"., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Applying a task-based approach to distributed machine learning workflows

  • Vázquez Novoa, Fernando
  • Lezzi, Daniele
  • Lordan Gomis, Francesc
  • Baghdadi, Fatemeh
  • Cirillo, Davide
The growing demands across various scientific fields have led to a significant shift in applications that consume data at the edge of the computing continuum. These applications require unified programming models for the composition of components and coordinating the execution of computational workloads, including training machine learning (ML) models on distributed resources. Personalized healthcare often leverages data generated from wearable devices used to train ML models, can be benefited from distributed computing approaches. Specifically, stroke care can be greatly benefited from distributed ML with modifiable risk factors that can be monitored using wearable devices. In this work, we present an implementation that leverages distributed techniques for large-scale ML workflows using electrocardiogram (ECG) recordings for atrial fibrillation (AF) classification. The application was evaluated using the PhysioNet database, showcasing the potential of distributed, ML in stroke care, opening the way for future creation of more advanced models embedded in edge devices., Author Fernando Vázquez is supported by PRE2022- 104134 funded by MICIU/AEI /10.13039/501100011033 and by the FSE+. This work has been supported by the Spanish Government (PID2019-107255GB) and by MCIN/AEI /10.13039/501100011033 (CEX2021-001148-S), by Generalitat de Catalunya (contract 2021-SGR-00412), and by the European Commission through the Horizon Europe Research and Innovation program under Grant Agreement 101016577 (AI-SPRINT project)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol
  • Agulló López, Ferran
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Enhancing OmpSs-2 suspendable tasks by combining operating system and user-level threads with C++ coroutines

  • Cinca Roca, Arnau
  • Roca Nonell, Aleix
  • Sala Penadés, Kevin
  • Peñacoba Veigas, Raúl
  • Álvarez Robert, David
  • Beltran Querol, Vicenç
This paper explores three methods for implementing suspendable tasks within task-based programming models: OS threads (pthreads), User-Level Threads (ULTs), and C++ coroutines. We enhance the OmpSs-2 programming model, originally supporting suspendable tasks via pthreads, to also accommodate ULTs and C++ coroutines. This unified approach facilitates a comprehensive comparative analysis using various benchmarks that includes recursive fork-join and data-flow parallelization strategies. Additionally, we contrast these suspension methods with the Cilk and OpenMP task-based programming models, which, despite their efficiency, lack support for suspendable tasks. Key contributions of this study include the novel integration of C++20 coroutines into the OmpSs-2 programming model, which can be combined with pthreads or ULTs. Furthermore, we introduce a new Linux kernel syscall that accelerates pthread context switches by an order of magnitude, thus narrowing the performance gap between ULTs and pthreads in context-switch times from two to one order of magnitude. C++ Coroutines are the ideal solution for scenarios where many tasks are simultaneously in a suspended state, or the frequency of task suspension and resumption is high because they have the smallest memory footprint and minor contextswitch overhead. However, they are limited to C++ programs, do not support TLS, and only allow task suspension at top-level functions. Still, pthreads and ULTs can bring remarkable benefits where C++ Coroutines cannot. We conclude that combining the strengths of coroutines with pthreads or ULTs brings productivity and performance benefits for programming models., This work was partially supported by the Generalitat de Catalunya (contract 2021-SGR-01007) and the Spanish Government through the Severo Ochoa Program (CEX2021-001148-S/MCIN/AEI/10.13039/501100011033). It is also part of the research project PID2023-147979NB-C21, funded by MICIU/AEI (10.13039/501100011033) and co-financed by the European Regional Development Fund (FEDER, EU). Additionally, this work was developed in the context of the European PILOT project, which has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No. 101034126, with support from the European Union's Horizon 2020 research and innovation program and Spain, Italy, Switzerland, Germany, France, Greece, Sweden, Croatia, and Turkey. This paper is also co-financed by the Barcelona Zettaescale Laboratory (reference REGAGE22e00058408992), promoted by the Ministry for Digital Transformation and the Civil Service, within the framework of the Recovery, Transformation and Resilience Plan - Funded by the European Union - NextGenerationEU., Peer Reviewed, Postprint (author's final draft)