Pasar al contenido principal

BSC - COMPUTACION DE ALTAS PRESTACIONES VIII

PID2019-107255GB-C21

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2019
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario BARCELONA SUPERCOMPUTING CENTER CENTRO NACIONAL DE SUPERCOMPUTACION

Resultados relacionados

Resultados totales (Incluyendo duplicados): 75
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Punxa, A Python-Based RISC-V System Simulator for Education

  • Castells-Rufas, David|||0000-0002-7181-9705
  • Novo, David|||0000-0002-5510-4152
  • Martorell, Xavier|||0000-0002-0417-3430
The analysis and verification of the integration of hardware and software components is challenging. Traditional HDL simulators, which are often slow and provide low-level analysis tools, are not always effective for simulating the software of processor-based systems. This work presents an interactive co-simulation framework that enhances students' understanding of computer architecture concepts typically obscured by hardware intricacies. This framework facilitates a unified environment for iterative design, enabling detailed analysis and generation of reports and traces to identify system performance, bottlenecks, and find implementation errors. By prioritizing analysis features over performance, this framework serves as a valuable tool for educational purposes and comprehensive examination of the interactions between hardware and software.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

BIMSA, accelerating long sequence alignment using processing-in-memory

  • Alonso-Marín, Alejandro
  • Fernandez, Ivan
  • Aguado-Puig, Quim|||0000-0003-4871-3192
  • Gómez-Luna, Juan|||0000-0002-6514-1571
  • Marco-Sola, Santiago|||0000-0001-7951-3914
  • Mutlu, Onur
  • Moreto, Miquel|||0000-0002-9848-8758
Recent advances in sequencing technologies have stressed the critical role of sequence analysis algorithms and tools in genomics and healthcare research. In particular, sequence alignment is a fundamental building block in many sequence analysis pipelines and is frequently a performance bottleneck both in terms of execution time and memory usage. Classical sequence alignment algorithms are based on dynamic programming and often require quadratic time and memory with respect to the sequence length. As a result, classical sequence alignment algorithms fail to scale with increasing sequence lengths and quickly become memory-bound due to data-movement penalties. Processing-In-Memory (PIM) is an emerging architectural paradigm that seeks to accelerate memory-bound algorithms by bringing computation closer to the data to mitigate data-movement penalties. This work presents BIMSA (idirectional n-emory equence lignment), a PIM design and implementation for the state-of-the-art sequence alignment algorithm BiWFA (Bidirectional Wavefront Alignment), incorporating new hardware-aware optimizations for a production-ready PIM architecture (UPMEM). BIMSA supports aligning sequences up to 100K bases, exceeding the limitations of state-of-the-art PIM implementations. First, BIMSA achieves speedups up to 22.24× (11.95× on average) compared to state-of-the-art PIM-enabled implementations of sequence alignment algorithms. Second, achieves speedups up to 5.84× (2.83× on average) compared to the highest-performance multicore CPU implementation of BiWFA. Third, BIMSA exhibits linear scalability with the number of compute units in memory, enabling further performance improvements with upcoming PIM architectures equipped with more compute units and achieving speedups up to 9.56× (4.7× on average).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Leveraging HLS to Design a Versatile &amp, High-Performance Classic McEliece Accelerator

  • Kostalabros, Ioannis-Vatistas
  • Ribes, Jordi
  • Carril, Xavier
  • Farràs, Oriol
  • Moreto, Miquel
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] By harnessing fundamental quantum properties, a large-scale quantum computer could undermine currently deployed public-key algorithms. The post-quantum, code-based cryptosystem Classic McEliece (CM)addresses this security concern. However, its large public key size (up to 1.3 MB) poses various hardware implementation challenges. In this article, we focus on the high memory bandwidth requirements of the CM encoding function, in the context of heterogeneous CPU-FPGA devices. More concretely, we target the acceleration of public-key loading and processing from any globally shared or accelerator-private memory system. We present a novel and constant-time accelerator eEnc that exploits the elevated parallelization potential of FPGA devices to yield high-performance results. Our accelerator implements the encoding and the random error vector generation functions, which comprise the main computational load of Encapsulation. Two accelerator design variants are introduced, providing different hardware tradeoffs. Regarding intra-accelerator data communication, and unlike other state-of-the-art (SOTA) works, we combine a streaming protocol with task-level parallelization to remove the need to store the public key in accelerator-private memories. Our proposed design shows new record execution times over its SOTA counterparts, ranging on average from 3.5x up to 7.7x across the five security level parameter sets. Our end-to-end implementation in a Zynq SoC shows an average speedup of 2.2x compared to a 64-bit vectorized CM software-baseline. The elevated logic resource consumption, characteristic of HLS designs, can be readily adjusted with a performance tradeoff., This work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI 10.13039/501100011033 (contracts ACITHEC PID2021-124928NB-I00, PID2019-107255GB-C21 and "Ramon y Cajal" fellowship No. RYC2020-030685-I) , by the Generalitat de Catalunya (contract 2021-SGR-00763, 2021-SGR-00115 and 2023-146511NB-I00) , by the Agency for Management of University and Research Grants (AGAUR) of the Government of Catalonia under "Ajuts per a la contractacio de personal investigador novell" (fellowships No. 2019FI B01274 and 2024 FI-1 00520) , by the European Union within the framework of the ERDF of Catalonia 2014-2020 under the DRAC project [001-P-001723] , and by the project HERMES funded by the European NextGenerationEU/PRTR via INCIBE.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Parallel reduced-order modeling for digital twins using high-performance computing workflows

  • Ares de Parga, S.
  • Bravo, J. R.
  • Sibuet, N.
  • Hernandez, J. A.
  • Rossi, R.
  • Boschert, Stefan
  • Tatu, Cristian Catalin
  • Vázquez-Novoa, Fernando
  • Ejarque, Jorge
  • Badia, Rosa M.
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Tomás Domínguez, Andrés Enrique
[EN] The integration of reduced-order models with high-performance computing is critical for developing digital twins, particularly for real-time monitoring and predictive maintenance of industrial systems. This paper presents a comprehensive, high-performance computing-enabled workflow for developing and deploying projection-based reduced-order models for large-scale mechanical simulations. We use PyCOMPSs' parallel framework to efficiently execute reduced-order model training simulations, employing parallel singular value decomposition algorithms such as randomized singular value decomposition, Lanczos singular value decomposition, and full singular value decomposition based on tall-skinny QR. Moreover, we introduce a partitioned version of the hyperreduction scheme known as the Empirical Cubature Method to further enhance computational efficiency in projection-based reduced-order models for mechanical systems. Despite the widespread use of high-performance computing for projection-based reduced-order models, there is a significant lack of publications detailing comprehensive workflows for building and deploying end-to-end projection-based reduced-order models in high-performance computing environments. Our workflow is validated through a case study focusing on the thermal dynamics of a motor, a multiphysics problem involving convective heat transfer and mechanical components. The projection-based reduced-order model is designed to deliver a real-time prognosis tool that could enable rapid and safe motor restarts post-emergency shutdowns under different operating conditions, demonstrating its potential impact on the practice of simulations in engineering mechanics. To facilitate deployment, we use the High-Performance Computing Workflow as a Service strategy and Functional Mock-Up Units to ensure compatibility and ease of integration across high-performance computing, edge, and cloud environments. The outcomes illustrate the efficacy of combining projection-based reduced-order models and high-performance computing, establishing a precedent for scalable, real-time digital twin applications in computational mechanics across multiple industries., The authors acknowledge financial support from the Spanish Ministry of Economy and Competitiveness, through the Severo Ochoa Programme for Centres of Excellence in R&D (CEX2018-000797-S and CEX2021-001148-S) , as well as to the project PID2019-107255GB (BSC authors). This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway. This publication is part of the R&D project PCI2021-121944 and PCI2021-121957, financed by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR . Sebastian Ares de Parga Regalado acknowledges financial support from the Generalitat de Catalunya through the FI_SDUR-2021 grant (2021 FISDU 00142), which provided funding for his predoctoral training. BSC authors acknowledge financial support from the Departament de Recerca i Universitats de la Generalitat de Catalunya, research group MPiEDist (2021 SGR 00412). Author Fernando Vázquez is supported by PRE2022-104134 funded by MICIU/AEI /10.13039/501100011033 and by the FSE+.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Programming parallel dense matrix factorizations and inversion for new-generation NUMA architectures

  • Catalán, Sandra
  • Igual, Francisco D.
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We propose a methodology to address the programmability issues derived from the emergence of newgeneration shared-memory NUMA architectures. For this purpose, we employ dense matrix factorizations and matrix inversion (DMFI) as a use case, and we target two modern architectures (AMD Rome and Huawei Kunpeng 920) that exhibit configurable NUMA topologies. Our methodology pursues performance portability across different NUMA configurations by proposing multi-domain implementations for DMFI plus a hybrid task- and loop-level parallelization that configures multi-threaded executions to fix core-todata binding, exploiting locality at the expense of minor code modifications. In addition, we introduce a generalization of the multi-domain implementations for DMFI that offers support for virtually any NUMA topology in present and future architectures. Our experimentation on the two target architectures for three representative dense linear algebra operations validates the proposal, reveals insights on the necessity of adapting both the codes and their execution to improve data access locality, and reports performance across architectures and inter- and intra-socket NUMA configurations competitive with state-of-the-art message-passing implementations, maintaining the ease of development usually associated with shared-memory programming., This research was sponsored by project PID2019-107255GB of Ministerio de Ciencia, Innovacion y Universidades; project S2018/TCS-4423 of Comunidad de Madrid; project 2017-SGR-1414 of the Generalitat de Catalunya and the Madrid Government under the Multiannual Agreement with UCM in the line Pro-gram to Stimulate Research for Young Doctors in the context of the V PRICIT, project PR65/19-22445. This project has also re-ceived funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union's Horizon 2020 research and innovation programme, and Spain, Germany, France, Italy, Poland, Switzerland, Norway. The work is also supported by grants PID2020-113656RB-C22 and PID2021-126576NB-I00 of MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Accurately Measuring Contention in Mesh NoCs in Time-Sensitive Embedded Systems

  • Cardona, Jordi
  • Abella, Jaume
  • Mezzetti, Enrico
  • Cazorla, Francisco J.
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] The computing capacity demanded by embedded systems is on the rise as software implements more functionalities, ranging from best-effort entertainment functions to performance-guaranteed safety-related functions. Heterogeneous manycore processors, using wormhole mesh (wmesh) Network-on-Chips (NoCs) as the main communication means, and contention block among applications, are increasingly considered to deliver the required computing performance. Most research efforts on software timing analysis have focused on deriving bounds (estimates) to the contention that tasks can suffer when accessing wmesh NoCs. However, less effort has been devoted to an equally important problem, namely, accurately measuring the actual contention tasks generate each other on the wmesh which is instrumental during system validation to diagnose any software timing misbehavior and determine which tasks are particularly affected by contention on specific wmesh routers. In this article, we work on the foundations of contention measuring in wmesh NoCs and propose and explain the rationale of a golden metric, called task PairWise Contention (PWC). PWC allows ascribing the actual share of the contention a given task suffers in the wmesh to each of its co-runner tasks at packet level. We also introduce and formalize a Golden Reference Value (GRV) for PWC that specifically defines a criterion to fairly break down the contention suffered by a task among its co-runner tasks in the wmesh. Our evaluation shows that GRV effectively captures how contention occurs by identifying the actual core (task) causing contention and whether contention is caused by local or remote interference in the wmesh., This work has been supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GB-C21 funded by MCIN/AEI/10.13039/501100011033 and the European Research Council (ERC) under the EU's Horizon 2020 research and innovation programme (grant agreement No. 772773).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Cache-aware optimization of matrix multiplication and matrix factorizations on multicore processors

  • Martínez-Pérez, Héctor
  • Catalán, Sandra
  • Igual, Francisco D.
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] This paper advocates for a careful customization of the special general matrix multiplication (GEMM) kernels that are invoked from blocked routines for several relevant matrix factorizations in LAPACK, in order to improve their performance on modern multicore processors with hierarchical cache memories. To achieve this, we leverage a refined analytical model to dynamically tune the cache configuration parameters of GEMM for these kernels, taking into account the matrix operands' dimensions, in order to improve cache occupation. In addition, toward the same goal, we accommodate a flexible development of architecture-specific micro-kernels for GEMM that allows us to select the option that, depending on the operands' dimensions, ameliorates cache utilization. Our experiments for the LU and QR factorizations on two platforms, equipped with ARM (NVIDIA Carmel) and x86 (AMD EPYC) multi-core processors, demonstrate the benefits of this approach in terms of a better cache utilization and, in general, higher performance. Moreover, they also reveal the delicate balance between optimizing for multi-threaded parallelism versus cache usage as well as the positive effects of software prefetching., This work was supported by grants PID2020-113656RB-C22, PID2019-107255GB, PID2021-126576NB-I00 and PID2021-123627OB-C52 of MCIN/AEI/10.13039/501100011033, by "ERDF A way of making Europe", and 2021-SGR-01007 of the Generalitat de Catalunya. Hector Martinez is a postdoctoral fellow supported by the Consejeria de Transformacion Economica, Industria, Conocimiento y Universidades de la Junta de Andalucia. Sandra Catalan was supported by the grant RYC2021-033973- I, funded by MCIN/AEI/10.13039/501100011033 and the European Union "NextGenerationEU"/PRTR.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Fine-grain task-parallel algorithms for matrix factorizations and inversion on many-threaded CPUs

  • Catalán, Sandra
  • Herrero, José R.
  • Igual, Francisco D.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We extend a two-level task partitioning previously applied to the inversion of dense matrices via Gauss-Jordan elimination to the more challenging QR factorization as well as the initial orthogonal reduction to band form found in the singular value decomposition. Our new task-parallel algorithms leverage the tasking mechanism currently available in OpenMP to exploit "nested" task parallelism, with a first outer level that operates on matrix panels and a second inner level that processes the matrix either by mu$$ \mu $$-panels or by tiles, in order to expose a large number of independent tasks. We present a detailed performance analysis, including execution traces, which shows that the two-level refinement into fine grain tasks allows for an improved load balancing and delivers high performance on current general-purpose many-core processors (CPUs) from Intel and AMD., This research was sponsored by projects RTI2018-093684-B-I00, PID2019-107255GB and TIN2017-82972-R ofMinisterio de Ciencia, Innovación y Universidades; project S2018/TCS-4423 of Comunidad de Madrid; project 2017-SGR-1414 of the Generalitat de Catalunya and the Madrid Government under the Multiannual Agreement with UCM in the line Program to Stimulate Research for Young Doctors in the context of the V PRICIT, project PR65/19-22445.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Virtualization Technology for Dependable Embedded Systems: A Survey for Transportation and Industrial Domains

  • Pérez-Cerrolaza, Jon
  • Mezzetti, Enrico
  • Gonzalez-Eguiluz, Jon
  • Martín-Acosta, Miguel
  • Lefebvre, Charles-Alexis
  • Balbastre, Patricia|||0000-0001-9458-4083
  • Crespo, Alfons|||0000-0002-6606-7406
[EN] Next-generation transportation and industrial systems are characterized by a continuously increasing number of integrated software functionalities and computational performance requirements. Nevertheless, there is also a need to reduce the cost, number, size, power consumption, volume and weight of integrated dependable embedded systems. In this context, virtualization technology enables the reconciliation of these challenges supporting the integration of multiple software functionalities of mixed-criticality (real-time and non-real-time, safety and non-safety-related) on a single computing device. This survey provides an overview of virtualization solutions (hypervisors, microkernels, and separation kernels) and categorizes research contributions addressing virtualization, real-time, dependability, and domain-specific technological challenges., This work has been partially supported by European Research Council with Horizon 2020 (grant agreement No. 101082622), the Spanish Ministry of Science and Innovation (grant agreements No. PID2019-107255GB-C21, PID2021-124502OB-C41), and by grant PLEC2021-007609 funded by MICIU/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR .
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Hardware Acceleration for High-Volume Operations of CRYSTALS-Kyber and CRYSTALS-Dilithium

  • Carril, Xavier
  • Kardaris, Charalampos
  • Ribes-González, Jordi
  • Farràs, Oriol
  • Kostalabros, Vatistas
  • González-Jiménez, Joel Ulises
  • Moreto, Miquel
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] Many high-demand digital services need to perform several cryptographic operations, such as key exchange or security credentialing, in a concise amount of time. In turn, the security of some of these cryptographic schemes is threatened by advances in quantum computing, as quantum computer could break their security in the near future. Post-quantum cryptography (PQC) is an emerging field that studies cryptographic algorithms that resist such attacks. The National Institute of Standards and Technology (NIST) has selected the CRYSTALSKyber Key Encapsulation Mechanism and the CRYSTALS-Dilithium Digital Signature algorithm as primary PQC standards. In this article, we present field-programmable gate array (FPGA)-based hardware accelerators for high-volume operations of both schemes. We apply high-level synthesis (HLS) for hardware optimization, leveraging a batch processing approach to maximize the memory throughput and applying custom HLS logic to specific algorithmic components. Using reconfigurable FPGAs, we show that our hardware accelerators achieve speedups between 3x and 9x over software baseline implementations, even over ones leveraging CPU vector architectures. Furthermore, the methods used in this study can also be extended to the new CRYSTALS-based NIST FIPS drafts, ML-KEM and ML-DSA, with similar acceleration results., This work was part of the project (ISOLDE) , funded by MICIU/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR under Grant No. PCI2023-143372, and the European Union's Horizon Europe Programme under project Chips Joint Undertaking (JU) under Grant No. 101112274. At the same time, this work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contract PID2019-107255GB-C21) , and by the Government of Catalonia (contract 2021-SGR-763) .X. Carril has the support of the predoctoral programme AGAUR-FI grants (2024 FI-1 00520) Joan Oro from the Secretariat for Universities and Research of the Department of Research and Universities of the Generalitat de Catalunya and theEuropean Social Fund Plus. J. Ribes-Gonzalez and O. Farras are supported by grant 2021-SGR-00115 from the Government of Catalonia, by the project HERMES, funded by INCIBE and by the European Union NextGeneration EU/PRTR, and the project ACITHEC PID2021-124928NB-I00, funded by MCIN/AEI/10.13039/501100011033. C. Hernandez was supported by "Ramony Cajal" fellowship No. RYC2020-030685-I. V. Kostalabros was supported by the Agency for Management of Universityand Research Grants (AGAUR) of the Government of Catalonia under "Ajuts per a la contractacio de personal investigadornovell" fellowship No. 2019FIB01274.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Hunting for open clusters in Gaia EDR3: 628 new open clusters found with OCfinder

  • Castro-Ginard, Alfred
  • Jordi, Carme
  • Luri, Xavier
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Casamiquela, Laia
  • Anders, Friedrich
  • Balaguer-Nuñez, Lola
  • Badia, Rosa M.
[Context] The improvements in the precision of the published data in Gaia EDR3 with respect to Gaia DR2, particularly for parallaxes and proper motions, offer the opportunity to increase the number of known open clusters in the Milky Way by detecting farther and fainter objects that have thus far gone unnoticed. [Aims] Our aim is to continue to complete the open cluster census in the Milky Way with the detection of new stellar groups in the Galactic disc. We use Gaia EDR3 up to magnitude G = 18 mag, increasing the magnitude limit and therefore the search volume explored in one unit with respect to our previous studies. [Methods] We used the OCfinder method to search for new open clusters in Gaia EDR3 using a big data environment. As a first step, OCfinder identified stellar statistical overdensities in five-dimensional astrometric space (position, parallax, and proper motions) using the DBSCAN clustering algorithm. Then, these overdensities were classified into random statistical overdensities or real physical open clusters using a deep artificial neural network trained on well-characterised G, GBP - GRP colour-magnitude diagrams. [Results] We report the discovery of 628 new open clusters within the Galactic disc, with most of them being located beyond 1 kpc from the Sun. From the estimation of ages, distances, and line-of-sight extinctions of these open clusters, we see that young clusters align following the Galactic spiral arms while older ones are dispersed in the Galactic disc. Furthermore, we find that most open clusters are located at low Galactic altitudes with the exception of a few groups older than 1 Gyr. [Conclusions] We show the success of the OCfinder method leading to the discovery of a total of 1274 open clusters (joining the discoveries here with the previous ones based on Gaia DR2), which represents almost 50% of the known population. Our ability to perform big data searches on a large volume of the Galactic disc, together with the higher precision in Gaia EDR3, enable us to keep completing the census with the discovery of new open clusters., This work was (partially) funded by the Spanish MICIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe” by the “European Union” through grant RTI2018-095076-B-C21, and the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M. A.C.G. acknowledges Spanish Ministry FPI fellowship n. BES-2016-078499. This work has been partially supported by the Spanish Government (PID2019-107255GB), by Generalitat de Catalunya (contract 2014-SGR-1051).
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Towards automatic model specialization for edge video analytics

  • Rivas Barragan, Daniel
  • Guim Bernat, Francesc
  • Polo Bardés, Jorda
  • Silva, Pubudu Madhawa
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Carrera Pérez, David|||0000-0003-4898-3424
The number of cameras deployed to the edge of the network increases by the day, while emerging use cases, such as smart cities or autonomous driving, also grow to expect images to be analyzed in real-time by increasingly accurate and complex neural networks. Unfortunately, state-of-the-art accuracy comes at a computational cost rarely available in the edge cloud. At the same time, due to strict latency constraints and the vast amount of bandwidth edge cameras generate, we can no longer rely on offloading the task to a centralized cloud. Consequently, there is a need for a meeting point between the resource-constrained edge cloud and accurate real-time video analytics. If state-of-the-art models are too expensive to run on the edge, and lightweight models are not accurate enough for the use cases in the edge, one solution is to demand less from the lightweight model and specialize it in a narrower scope of the problem, a technique known as model specialization. By specializing a model to the context of a single camera, we can boost its accuracy while keeping its computational cost constant. However, this also involves one training per camera, which quickly becomes unfeasible unless the entire process is fully automated. In this paper, we present and evaluate COVA (Contextually Optimized Video Analytics), a framework to assist in the automatic specialization of models for video analytics in edge cloud cameras. COVA aims to automatically improve the accuracy of lightweight models by specializing them to the context to which they will be deployed. Moreover, we discuss and analyze each step involved in the process to understand the different trade-offs that each one entails. Using COVA, we demonstrate that the whole pipeline can be effectively automated by leveraging large neural networks used as teachers whose predictions are used to train and specialize lightweight neural networks. Results show that COVA can automatically improve pre-trained models by an average of 21% mAP on the different scenes of the VIRAT dataset., This work has been partially supported by the Spanish Government (contract PID2019-107255GB) and by Generalitat de Catalunya, Spain (contract 2014-SGR-1051)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Scanflow: A multi-graph framework for machine learning workflow management, supervision, and debugging

  • Bravo Rocca, Gusseppe Jesus|||0000-0001-6824-1124
  • Liu, Peini|||0000-0003-0058-8732
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Dholakia, Ajay
  • Ellison, David
  • Falkanger, Jeffrey
  • Hodak, Miroslav
Machine Learning (ML) is more than just training models, the whole workflow must be considered. Once deployed, a ML model needs to be watched and constantly supervised and debugged to guarantee its validity and robustness in unexpected situations. Debugging in ML aims to identify (and address) the model weaknesses in not trivial contexts. Several techniques have been proposed to identify different types of model weaknesses, such as bias in classification, model decay, adversarial attacks, etc., yet there is not a generic framework that allows them to work in a collaborative, modular, portable, iterative way and, more importantly, flexible enough to allow both human- and machine-driven techniques. In this paper, we propose a novel containerized directed graph framework to support and accelerate end-to-end ML workflow management, supervision, and debugging. The framework allows defining and deploying ML workflows in containers, tracking their metadata, checking their behavior in production, and improving the models by using both learned and human-provided knowledge. We demonstrate these capabilities by integrating in the framework two hybrid systems to detect data drift distribution which identify the samples that are far from the latent space of the original distribution, ask for human intervention, and whether retrain the model or wrap it with a filter to remove the noise of corrupted data at inference time. We test these systems on MNIST-C, CIFAR-10-C, and FashionMNIST-C datasets, obtaining promising accuracy results with the help of human involvement., This work was partially supported by Lenovo as part of Framework Contract Lenovo-BSC 2020, by the Spanish Government under contract PID2019-107255 GB, and by the Generalitat de Catalunya, Spain under contract 2017-SGR-1414 and under grant 2020 FI-B 00257., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

XeroZerox: Analysis and Optimization of GPU Memory Management for High-Integrity Autonomous Systems

  • Calderón, Alejandro J.
  • Kosmidis, Leonidas
  • Nicolás, Carlos F.
  • Cazorla Almeida, Francisco Javier
Autonomous systems require high-performance processing capabilities, which demand the use of powerful accelerators such as GPUs. However, the use of GPUs in critical systems presents several challenges, since GPU programming models rely on explicit dynamic memory management. Traditionally, dynamic memory allocation in such systems is restricted to certain controlled scenarios, which require programs to be rewritten so that all the required memory is allocated at the beginning of the program and released at its end. However, many GPU applications do not follow this approach. Moreover, when dynamic memory allocation is used, it is critical to compute the exact amount of memory that will be used as well as to minimize it, to guarantee that it fits in the physical system memory. In this paper we present XeroZerox, an open-source tool that automatically converts the traditional dynamic memory allocations of GPU applications into allocations served from a centralized and optimally sized memory pool, which is managed in a way better suited for critical systems. XeroZerox allows legacy GPU applications to be used in a critical setup without rewriting them, minimizing at the same time their memory consumption and memory management runtime overhead., Spanish Ministry of Economy and Competitiveness (Grant Number: IJC-2020-045931-I and PID2019-107255GB-C21) Department of Research and Universities of the Government of Catalonia (Grant Number: 2021 SGR 00637) European Union Horizon Europe Programme (Grant Number: 101082622), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Boosting HPC data analysis performance with the ParSoDA-Py library

  • Belcastro, Loris
  • Giampà, Salvatore
  • Marozzo, Fabrizio
  • Talia, Domenico
  • Trunfio, Paolo
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Ejarque, Jorge|||0000-0003-4725-5097
  • Mammadli, Nihad
Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications., This work has been partially funded by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under Grant agreement No 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021-121957), project eFlows4HPC. It has also been supported by the Spanish Government (PID2019-107255GB) and by the Departament de Recerca i Universitats de la Generalitat de Catalunya to the Research Group MPiEDist (2021 SGR 00412) We also acknowledge financial support from “National Centre for HPC, Big Data and Quantum Computing," CN00000013 - CUP H23C22000360005, and from “FAIR - Future Artificial Intelligence Research" Project - CUP H23C22000860006., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

O(n) key–value sort with active compute memory

  • Esmaili Dokht, Pouya
  • Guiot Cusido, Miquel
  • Radojkovic, Petar
  • Martorell Bofill, Xavier|||0000-0002-0417-3430
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
  • Adlard, Jason
  • Amato, Paolo
  • Sforzin, Marco
We propose the Active Compute Memory (ACM), a near-memory-processing architecture capable of performing key–value sort directly in the DRAM. In the ACM architecture, sort is merely the writing of data into memory with one addressing protocol (perspective) and reading it back with different perspective. The first perspective is conventional, based on the data address; the second perspective is the sorted order. The ACM requires additional tables to store the meta-data and moderate control logic enhancements that can be implemented directly in the DRAM silicon. By these modest enhancements to DRAM, ACM exploits the parallelism inherently available in the row buffer to enable sort with O ( n ) complexity. This leads to an order of magnitude improvement in ACM performance and energy compared to conventional O ( n log n ) CPU-centric sort algorithms. The ACM also shows superior performance compared to other near-memory sort accelerators. This is because the ACM processing is done near the row buffer and it exploits much lower memory access latency, higher bandwidth and wider parallel processing. The sort operation covered in this paper is just an example of an address management operation that can be efficiently implemented directly in the DRAM silicon. We release as an open source the simulation infrastructure for the ACM performance and energy modeling. We would encourage the community to use it, adapt it to other PIM proposals, and share their own evaluations., This work was funded by the Collaboration Agreement between Micron Technology, Inc. and BSC. The work was also supported by the Spanish Government, under the contracts PID2019-107255GB-C21 and CEX2021-001148-S funded by MCIN/AEI/ 10.13039/501100011033. The work also received funding from the Department of Research and Universities of the Government of Catalonia to the AccMem Research Group (Code: 2021 SGR 00807) and project DEEP-SEA funded by European Union’s Horizon 2020 research and innovation programme , under Grant Agreement no.955606 and the Spanish Government, under the contract PCI2021-121958 funded by MCIN/AEI/ 10.13039/501100011033 and “European Union NextGenerationEU/PRTR”., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Dynamic load balance of chemical source term evaluation in high-fidelity combustion simulations

  • Ramirez Miranda, Guillem
  • Mira Martínez, Daniel
  • Pérez Sánchez, Eduardo Javier|||0000-0002-5056-415X
  • Surapaneni, Anurag
  • Borrell Pol, Ricard
  • Houzeaux, Guillaume|||0000-0002-2592-1426
  • Garcia Gasulla, Marta|||0000-0003-3682-9905
This paper presents a load balancing strategy for reaction rate evaluation and chemistry integration in reacting flow simulations. The large disparity in scales during combustion introduces stiffness in the numerical integration of the PDEs and generates load imbalance during the parallel execution. The strategy is based on the use of the DLB library to redistribute the computing resources at node level, lending additional CPU-cores to higher loaded MPI processes. This approach does not require explicit data transfer and is activated automatically at runtime. Two chemistry descriptions, detailed and reduced, are evaluated on two different configurations: laminar counterflow flame and a turbulent swirl-stabilized flame. For single-node calculations, speedups of 2.3x and 7x are obtained for the detailed and reduced chemistry, respectively. Results on multi-node runs also show that DLB improves the performance of the pure-MPI code similar to single node runs. It is shown DLB can get performance improvements in both detailed and reduced chemistry calculations., This work has been supported by the Spanish Government (PID2019-107255GB), by Generalitat de Catalunya (contract 2014-SGR-1051), and by the European POP CoE (GA No 824080) and EXCELLERAT project GA No 823691., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

GenArchBench: A genomics benchmark suite for arm HPC processors

  • López Villellas, Lorien
  • Langarita Benítez, Rubén|||0000-0002-6080-724X
  • Badouh, Asaf
  • Soria Pardos, Víctor
  • Aguado Puig, Quim|||0000-0003-4871-3192
  • López Paradís, Guillem
  • Doblas Font, Max|||0000-0002-8909-3033
  • Setoain, Javier
  • Kim, Chulho
  • Ono, Makoto
  • Armejach Sanosa, Adrià|||0000-0003-2869-668X
  • Marco Sola, Santiago|||0000-0001-7951-3914
  • Alastruey Benedé, Jesús
  • Ibáñez Marín, Pablo
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Arm usage has substantially grown in the High-Performance Computing (HPC) community. Japanese supercomputer Fugaku, powered by Arm-based A64FX processors, held the top position on the Top500 list between June 2020 and June 2022, currently sitting in the fourth position. The recently released 7th generation of Amazon EC2 instances for compute-intensive workloads (C7 g) is also powered by Arm Graviton3 processors. Projects like European Mont-Blanc and U.S. DOE/NNSA Astra are further examples of Arm irruption in HPC. In parallel, over the last decade, the rapid improvement of genomic sequencing technologies and the exponential growth of sequencing data has placed a significant bottleneck on the computational side. While most genomics applications have been thoroughly tested and optimized for x86 systems, just a few are prepared to perform efficiently on Arm machines. Moreover, these applications do not exploit the newly introduced Scalable Vector Extensions (SVE). This paper presents GenArchBench, the first genome analysis benchmark suite targeting Arm architectures. We have selected computationally demanding kernels from the most widely used tools in genome data analysis and ported them to Arm-based A64FX and Graviton3 processors. Overall, the GenArch benchmark suite comprises 13 multi-core kernels from critical stages of widely-used genome analysis pipelines, including base-calling, read mapping, variant calling, and genome assembly. Our benchmark suite includes different input data sets per kernel (small and large), each with a corresponding regression test to verify the correctness of each execution automatically. Moreover, the porting features the usage of the novel Arm SVE instructions, algorithmic and code optimizations, and the exploitation of Arm-optimized libraries. We present the optimizations implemented in each kernel and a detailed performance evaluation and comparison of their performance on four different HPC machines (i.e., A64FX, Graviton3, Intel Xeon Skylake Platinum, and AMD EPYC Rome). Overall, the experimental evaluation shows that Graviton3 outperforms other machines on average. Moreover, we observed that the performance of the A64FX is significantly constrained by its small memory hierarchy and latencies. Additionally, as proof of concept, we study the performance of a production-ready tool that exploits two of the ported and optimized genomic kernels., This work has been partially supported by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21, PID2019-105660RB-C21, PID2022136454NB-C22, and TED2021-132634A-I00), by the Generalitat de Catalunya, Spain (contract 2021-SGR-763), by the Gobierno de Aragón (T58_23R research group), by the European Union NextGenerationEU/ PRTR, and by Lenovo BSC Contract-Framework Contract (2020)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

On Neural Networks Redundancy and Diversity for Their Use in Safety-Critical Systems

  • Brando, Axel
  • Serra, Isabel
  • Mezzetti, Enrico|||0000-0002-1886-2931
  • Cazorla Almeida, Francisco Javier
  • Perez Cerrolaza, Jon
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
An increasing number of critical functionalities integrated in embedded critical systems rely on deep learning (DL) technology. This article summarizes certain key aspects of DL’s intrinsic stochastic and training-data-dependent nature that are at odds with current domain-specific functional safety standards. We exemplify how redundancy and diversity of neural networks can help to reconcile DL technology and functional safety requirements., The research leading to these results has received funding from the European Research Council (ERC) grant agreement No. 772773 (SuPerCom), the Horizon Europe Programme under the SAFEXPLAIN Project (www.safexplain.eu), grant agreement num.101069595, and the Spanish Ministry of Science and Innovation under grant PID2019-107255GBC21/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Dynamic resource allocation for efficient parallel CFD simulations

  • Houzeaux, Guillaume|||0000-0002-2592-1426
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Borrell Pol, Ricard
  • Dosimont, Damien
  • Ejarque Artigas, Jorge
  • Garcia Gasulla, Marta|||0000-0003-3682-9905
  • López Herrero, Víctor
CFD users of supercomputers usually resort to rule-of-thumb methods to select the number of subdomains (partitions) when relying on MPI-based parallelization. One common approach is to set a minimum number of elements or cells per subdomain, under which the parallel efficiency of the code is “known” to fall below a subjective level, say 80%. The situation is even worse when the user is not aware of the “good” practices for the given code and a huge amount of resources can thus be wasted. This work presents an elastic computing methodology that adapts at runtime the resources allocated to a simulation automatically. The criterion to control the required resources is based on a runtime measure of the communication efficiency of the execution. According to some analytical estimates, the resources are then expanded or reduced to fulfill this criterion and eventually execute an efficient simulation., This work has been supported by the Spanish Government (PID2019- 107255GB); by Generalitat de Catalunya (contract 2014-SGR-1051); by the European Commission H2020 project POP CoE (GA n. 824080); by the European Commission H2020 project CompBioMed CoE (GA n. 823712) and by the European Commission and the EuroHPC JU under contract 955558 (eFlows4HPC project)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Storage-heterogeneity aware task-based programming models to optimize I/O intensive applications

  • Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502
  • Ejarque Artigas, Jorge
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
Task-based programming models have enabled the optimized execution of the computation workloads of applications. These programming models can take advantage of large-scale distributed infrastructures by allowing the parallel and distributed execution of applications in high-level work components called tasks. Nevertheless, in the era of Big Data and Exascale, the amount of data produced by modern scientific applications has already surpassed terabytes and is rapidly increasing. Hence, I/O performance became the bottleneck to overcome in order to achieve more total performance improvement. New storage technologies offer higher bandwidth and faster solutions than traditional Parallel File Systems (PFS). Such storage devices are deployed in modern day infrastructures to boost I/O performance by offering a fast layer that absorbs the generated data. Therefore, it is necessary for any programming model targeting more performance to manage this heterogeneity and take advantage of it to improve the I/O performance of applications. Towards this goal, we propose in this paper a set of programming model capabilities that we refer to as Storage-Heterogeneity Awareness. Such capabilities include: (i) abstracting the heterogeneity of storage systems, and (ii) optimizing I/O performance by supporting dedicated I/O schedulers and an automatic data flushing technique. The evaluation section of this paper presents the performance results of different applications on the MareNostrum CTE-Power heterogeneous storage cluster. Our experiments demonstrate that a storage-heterogeneity aware programming model can achieve up to almost 5x I/O performance speedup and 48% total time improvement compared to the reference PFS-based usage of the execution infrastructure., This work is partially supported by the European Union through the Horizon 2020 research and innovation programme under contracts 721865 (EXPERTISE Project) by the Spanish Government (PID2019-107255GB) and the Generalitat de Catalunya (contract 2014-SGR-1051)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Vector extensions in COTS processors to increase guaranteed performance in real-time systems

  • Pujol Torramorell, Roger
  • Jorba Jorba, Josep
  • Tabani, Hamid
  • Kosmidis, Leonidas
  • Mezzetti, Enrico|||0000-0002-1886-2931
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Cazorla Almeida, Francisco Javier
The need for increased application performance in high-integrity systems like those in avionics is on the rise as software continues to implement more complex functionalities. The prevalent computing solution for future high-integrity embedded products are multi-processors systems-on-chip (MPSoC) processors. MPSoCs include CPU multicores that enable improving performance via thread-level parallelism. MPSoCs also include generic accelerators (GPUs) and application-specific accelerators. However, the data processing approach (DPA) required to exploit each of these underlying parallel hardware blocks carries several open challenges to enable the safe deployment in high-integrity domains. The main challenges include the qualification of its associated runtime system and the difficulties in analyzing programs deploying the DPA with out-of-the-box timing analysis and code coverage tools. In this work, we perform a thorough analysis of vector extensions (VExt) in current COTS processors for high-integrity systems. We show that VExt prevent many of the challenges arising with parallel programming models and GPUs. Unlike other DPAs, VExt require no runtime support, prevent by design race conditions that might arise with parallel programming models, and have minimum impact on the software ecosystem enabling the use of existing code coverage and timing analysis tools. We develop vectorized versions of neural network kernels and show that the NVIDIA Xavier VExt provide a reasonable increase in guaranteed application performance of up to 2.7x. Our analysis contends that VExt are the DPA approach with arguably the fastest path for adoption in high-integrity systems., This work has received funding from the the European Research Council (ERC) grant agreement No. 772773 (SuPerCom) and the Spanish Ministry of Science and Innovation (AEI/10.13039/501100011033) under grants PID2019-107255GB-C21 and IJC2020-045931-I., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

QuickEd: high-performance exact sequence alignment based on bound-and-align

  • Doblas Font, Max|||0000-0002-8909-3033
  • Lostes Cazorla, Óscar|||0000-0001-5166-3806
  • Aguado Puig, Quim|||0000-0003-4871-3192
  • Iñiguez Rodríguez, Cristian
  • Moretó Planas, Miquel|||0000-0002-9848-8758
  • Marco Sola, Santiago|||0000-0001-7951-3914
Motivation: Pairwise sequence alignment is a core component of multiple sequencing-data analysis tools. Recent advancements in sequencing technologies have enabled the generation of longer sequences at a much lower price. Thus, long-read sequencing technologies have become increasingly popular in sequencing-based studies. However, classical sequence analysis algorithms face significant scalability challenges when aligning long sequences. As a result, several heuristic methods have been developed to improve performance at the expense of accuracy, as they often fail to produce the optimal alignment. Results: This paper introduces QuickEd, a sequence alignment algorithm based on a bound-and-align strategy. First, QuickEd effectively bounds the maximum alignment-score using efficient heuristic strategies. Then, QuickEd utilizes this bound to reduce the computations required to produce the optimal alignment. Compared to Oðn2Þ complexity of traditional dynamic programming algorithms, QuickEd’s bound-and-align strategy achieves OðnbsÞ complexity, where n is the sequence length and bs is an estimated upper bound of the alignment-score between the sequences. As a result, QuickEd is consistently faster than other state-of-the-art implementations, such as Edlib and BiWFA, achieving performance speedups of 4:2-5:9× and 3:8-4:4×, respectively, aligning long and noisy datasets. In addition, QuickEd maintains a stable memory footprint below 35 MB while aligning sequences up to 1 Mbp. Availability and implementation: QuickEd code and documentation are publicly available at https://github.com/maxdoblas/QuickEd., This work was partially funded by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 and FSEþ [PID2019-107255GB-C21, PID2020-113614RBC21, TED2021-132634A-I00, and PRE2021-101059 to Q. A.-P.], the Spanish Ministry of Economy, Industry, and Competitiveness through an FPU fellowship [FPU20-04076 to M.D.], the Spanish Ministry, Ministerio para la Transformación Digital y de la Función Pública, and the European Union – NextGenerationEU through the Cátedra Chip UPC project [TSI-069100-2023-0015], Lenovo-BSC Contract-Framework Contract (2020), and the Generalitat de Catalunya GenCat-DIUiE (GRR) [2017-SGR-313, 2017- SGR-1328, 2017-SGR-1414, 2021-SGR-00763, and 2021- SGR-00574]., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Extreme-scale workflows: a perspective from the JLESC international community

  • Yildiz, Orcun
  • Gueroudji, Amal
  • Bigot, Julien
  • Raffin, Bruno
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Peterka, Tom
The Joint Laboratory for Extreme-Scale Computing (JLESC) focuses on software challenges in high-performance computing systems to meet the needs of today’s science campaigns, which often require large resources, consist of multiple tasks, and generate vast amounts of data. In this context, extreme-scale workflows have been the key factor in enabling scientific discoveries by helping scientists automate the dependencies and data exchanges between workflow tasks, instead of managing those manually. In this paper we present representative extreme-scale workflows and feature workflow systems developed by JLESC participating institutions. We present lessons learned while developing these tools, alongside with the open challenges and future research directions in the field of extreme-scale workflows., This research is supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under contract number DE-AC02- 06CH11357, program manager Margaret Lentz. The BSC author acknowledges the support of the Spanish Government (PID2019-107255GB) and by MCIN/AEI /10.13039 /501100011033 (CEX2021-001148-S), by the Departament de Recerca i Universitats de la Generalitat de Catalunya to the Research Group MPiEDist (2021 SGR 00412). The same author acknowledges the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No. 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021-121957), project eFlows4HPC. The INRIA author acknowledges the European Union’s Horizon research and innovation programme under grant agreement No 101144014., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Hardware acceleration for high-volume operations of CRYSTALS-Kyber and CRYSTALS-Dilithium

  • Carril Gil, Xavier|||0009-0008-8392-7579
  • Kardaris, Charalampos
  • Ribes González, Jordi
  • Farràs Ventura, Oriol
  • Hernández Luz, Carles
  • Kostalabros, Vatistas
  • González Jiménez, Joel Ulises
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Many high-demand digital services need to perform several cryptographic operations, such as key exchange or security credentialing, in a concise amount of time. In turn, the security of some of these cryptographic schemes is threatened by advances in quantum computing, as quantum computer could break their security in the near future. Post-Quantum Cryptography (PQC) is an emerging field that studies cryptographic algorithms that resist such attacks. The National Institute of Standards and Technology (NIST) has selected the CRYSTALS-Kyber Key Encapsulation Mechanism and the CRYSTALSDilithium Digital Signature algorithm as primary PQC standards. In this paper, we present FPGA-based hardware accelerators for high-volume operations of both schemes. We apply High-Level Synthesis (HLS) for hardware optimization, leveraging a batch processing approach to maximize the memory throughput, and applying custom HLS logic to specific algorithmic components. Using reconfigurable field-programmable gate arrays (FPGAs), we show that our hardware accelerators achieve speedups between 3x and 9x over software baseline implementations, even over ones leveraging CPU vector architectures. Furthermore, the methods used in this study can also be extended to the new CRYSTALS-based NIST FIPS drafts, ML-KEM and ML-DSA, with similar acceleration results., This work was part of the project (ISOLDE), funded by MICIU/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR under Grant No. PCI2023-143372, and the European Union’s Horizon Europe Programme under project Chips Joint Undertaking (JU) under Grant No. 101112274. At the same time, this work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI/ 10.13039/501100011033 (contract PID2019-107255GB-C21), and by the Government of Catalonia (contract 2021-SGR-763). X. Carril has the support of the predoctoral programme AGAUR-FI grants (2024 FI-1 00520) Joan Oró from the Secretariat for Universities and Research of the Department of Research and Universities of the Generalitat de Catalunya and the European Social Fund Plus. J. Ribes-González and O. Farràs are supported by grant 2021-SGR-00115 from the Government of Catalonia, by the project HERMES, funded by INCIBE and by the European Union NextGeneration EU/PRTR, and the project ACITHEC PID2021-124928NB-I00, funded by MCIN/AEI/10.13039/501100011033. C. Hernandez was supported by “Ramón y Cajal” fellowship No. RYC2020-030685-I. V. Kostalabros was supported by the Agency for Management of University and Research Grants (AGAUR) of the Government of Catalonia under “Ajuts per a la contractació de personal investigador novell” fellowship No. 2019FI B01274., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

SafeDE: A low-cost hardware solution to enforce diverse redundancy in multicores

  • Bas Jalón, Francisco
  • Alcaide Portet, Sergi|||0000-0001-8561-7795
  • Cabo Pitarch, Guillem
  • Benedicte Illescas, Pedro|||0000-0003-1670-7783
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
Failure risk must be tiny in high-integrity systems, such as those in cars, satellites and aircraft. Hence, safety measures must be deployed to avoid a single fault leading to a failure. Redundancy has been often used to address this concern, but it has been proven insufficient if a single fault can cause the same error in all redundant elements, which defeats the purpose of redundancy for error detection. Hence, to avoid this scenario, diversity is implemented along with redundancy, being lockstep execution the most popular diverse redundancy solution for computing cores. However, classic lockstep solutions have non-negligible limitations if implemented in hardware (e.g., half of the cores can only be used for redundant execution and are not even visible at user level), or in software (e.g., the software loop to enforce staggering is long and costs performance). This paper tackles the limitations of classic lockstep solutions by providing an extended analysis and evaluation of SafeDE, a Diversity Enforcement hardware module combining the short loop to enforce diversity of hardware solutions, and the nonintrusiveness of software solutions. Hence, cores can operate in lockstep mode efficiently or run independent tasks. In this paper, we present SafeDE and its rationale, its application to N-modular systems, its hardware and software integration, and an evaluation showing its performance and area efficiency, and its behavior in the presence of faults., This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 871467, and in part by the Spanish Ministry of Science and Innovation under Grant PID2019-107255GB-C21/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Adaptive power shifting for power-constrained heterogeneous systems

  • Ortega Carrasco, Cristobal
  • Alvarez Martí, Lluc
  • Buyuktosunoglu, Alper
  • Bertran Monfort, Ramon
  • Rosedahl, Todd Jon
  • Bose, Pradip
  • Moretó Planas, Miquel|||0000-0002-9848-8758
The number and heterogeneity of compute devices, even within a single compute node, has been steadily on the rise. Since all systems must operate under a power cap, the number of discrete devices that can run simultaneously at their highest frequency is limited by the globally-imposed power cap. Current systems incorporate a centralized power management unit that statically controls the distribution of power among the devices within the node. However, such static distribution policies are unaware of the dynamic utilization profile across the devices, which leads to unfair power allocations that end up degrading system throughput performance. The problem is particularly acute in the presence of heterogeneity since type-specific performance-boost capabilities cannot be leveraged via utilization-agnostic static power allocations. This paper proposes Adaptive Power Shifting for multi-accelerator heterogeneous systems (APS), a technique that leverages system utilization information to dynamically allocate and re-distribute power budgets across multiple discrete devices. Democratizing the power allocation based on dynamic needs results in dramatic speedup over a need-agnostic static allocation. We use APS in a real OpenPOWER compute node with 2 CPUs and 4 GPUs to demonstrate the value of on-demand, equitable power allocations. Overall, the proposed solution increases performance with respect to two state-of-the-art techniques by up to 14.9% and 13.8%., This work has been partially supported by the European Union’s Horizon 2020 research and innovation program under the Mont-Blanc 2020 project (grant agreement 779877), by the Spanish Ministry of Science and Innovation (contract PID2019-107255GB-C22), by Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272) and by the IBM/BSC Deep Learning Center initiative. Ll. Alvarez has been supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under the Juan de la Cierva Formacion fellowship No. FJCI-2016- 30984. M. Moreto has been supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal fellowship No. RYC-2016-21104., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Porting and optimizing BWA-MEM2 using the Fujitsu A64FX processor

  • Langarita Benítez, Rubén|||0000-0002-6080-724X
  • Armejach Sanosa, Adrià|||0000-0003-2869-668X
  • Ibáñez Marín, Pablo
  • Alastruey Benedé, Jesús
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Sequence alignment pipelines for human genomes are an emerging workload that will dominate in the precision medicine field. BWA-MEM2 is a tool widely used in the scientific community to perform read mapping studies. In this paper, we port BWA-MEM2 to the AArch64 architecture using the ARMv8-A specification, and we compare the resulting version against an Intel Skylake system both in performance and in energy-to-solution. The porting effort entails numerous code modifications, since BWA-MEM2 implements certain kernels using x86_64 specific intrinsics, e.g., AVX-512. To adapt this code we use the recently introduced Arm's Scalable Vector Extensions (SVE). More specifically, we use Fujitsu's A64FX processor, the first to implement SVE. The A64FX powers the Fugaku Supercomputer that led the Top500 ranking from June 2020 to November 2021. After porting BWA-MEM2 we define and implement a number of optimizations to improve performance in the A64FX target architecture. We show that while the A64FX performance is lower than that of the Skylake system, A64FX delivers 11.6% better energy-to-solution on average. All the code used for this article is available at https://gitlab.bsc.es/rlangari/bwa-a64fx, This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (contracts PID2019-107255GB-C21 / AEI /10.13039/501100011033 and PID2019-105660RB-C21 / AEI / 10.13039/501100011033), Gobierno de Aragon (T5820R research group), the Generalitat de Catalunya (contracts 2017-SGR-1328 and 2017-SGR1414), and the European Union’s Horizon 2020 research and innovation program (Mont-Blanc 2020 project, grant agreement 779877). Finally, A. Armejach and M. Moreto have been partially supported by the Spanish Ministry of Economy, Industry and Competitiveness under Juan de la Cierva fellowship no. IJCI-2017-33945 and Ramon y Cajal fellowship no. RYC-2016-21104, respectively., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Accurate and efficient constrained molecular dynamics of polymers using Newton's method and special purpose code

  • López Villellas, Lorién
  • Kjelgaard Mikkelsen, Carl Christian
  • Galano Frutos, Juan José
  • Marco Sola, Santiago|||0000-0001-7951-3914
  • Alastruey Benedé, Jesús
  • Ibáñez Marín, Pablo Enrique
  • Moretó Planas, Miquel|||0000-0002-9848-8758
  • Sancho Sanz, Javier
  • García Risueño, Pablo
In molecular dynamics simulations we can often increase the time step by imposing constraints on bond lengths and bond angles. This allows us to extend the length of the time interval and therefore the range of physical phenomena that we can afford to simulate. We examine the existing algorithms and software for solving nonlinear constraint equations in parallel and we explain why it is necessary to advance the state-of-the-art. We present ILVES-PC, a new algorithm for imposing bond constraints on proteins accurately and efficiently. It solves the same system of differential algebraic equations as the celebrated SHAKE algorithm, but ILVES-PC solves the nonlinear constraint equations using Newton's method rather than the nonlinear Gauss-Seidel method. Moreover, ILVES-PC solves the necessary linear systems using a specialized linear solver that exploits the structure of the protein. ILVES-PC can rapidly solve constraint equations as accurately as the hardware will allow. The run-time of ILVES-PC is proportional to the number of constraints. We have integrated ILVES-PC into GROMACS and simulated proteins of different sizes. Compared with SHAKE, we have achieved speedups of up to 4.9× in single-threaded executions and up to 76× in shared-memory multi-threaded executions. Moreover, ILVES-PC is more accurate than P-LINCS algorithm. Our work is a proof-of-concept of the utility of software designed specifically for the simulation of polymers., This work has been partially supported by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21, PID2019-105660RB-C21, and PID2019-107293GB-I00), by the Generalitat de Catalunya (contract 2017-SGR-1328), by the Gobierno de Aragón (E45_20R T58_20R research groups), and by Lenovo-BSC Contract-Framework Contract (2020). Santiago Marco was supported by the Agencia Estatal de Investigación (Spain) under Juan de la Cierva fellowship grant IJC2020-045916-I. Miquel Moretó was partially supported by the Agencia Estatal de Investigación (Spain) under Ramón y Cajal fellowship number RYC-2016-21104. Carl Christian Kjelgaard Mikkelsen is supported by eSSENCE, a collaborative e-Science programme funded by the Swedish Research Council within the framework of the strategic research areas designated by the Swedish Government., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Leveraging HLS to design a versatile & high-performance Classic McEliece accelerator

  • Kostalabros, Vatistas
  • Ribes González, Jordi
  • Carril Gil, Xavier|||0009-0008-8392-7579
  • Farràs Ventura, Oriol
  • Hernández Luz, Carles
  • Moretó Planas, Miquel|||0000-0002-9848-8758
By harnessing fundamental quantum properties, a large-scale quantum computer could undermine currently deployed public-key algorithms. The post-quantum, code-based cryptosystem Classic McEliece (CM) addresses this security concern. However, its large public key size (up to 1.3 MB) poses various hardware implementation challenges. In this article, we focus on the high memory bandwidth requirements of the CM encoding function, in the context of heterogeneous CPU-FPGA devices. More concretely, we target the acceleration of public-key loading and processing from any globally shared or accelerator-private memory system. We present a novel and constant-time accelerator eEnc that exploits the elevated parallelization potential of FPGA devices to yield high-performance results. Our accelerator implements the encoding and the random error vector generation functions, which comprise the main computational load of Encapsulation. Two accelerator design variants are introduced, providing different hardware tradeoffs. Regarding intra-accelerator data communication, and unlike other state-of-the-art (SOTA) works, we combine a streaming protocol with task-level parallelization to remove the need to store the public key in accelerator-private memories. Our proposed design shows new record execution times over its SOTA counterparts, ranging on average from 3.5× up to 7.7× across the five security level parameter sets. Our end-to-end implementation in a Zynq SoC shows an average speedup of 2.2× compared to a 64-bit vectorized CM software-baseline. The elevated logic resource consumption, characteristic of HLS designs, can be readily adjusted with a performance tradeoff., This work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Scienceand Innovation MCIN/AEI 10.13039/501100011033 (contracts ACITHEC PID2021-124928NB-I00, PID2019-107255GB-C21and ”Ramón y Cajal” fellowship No. RYC2020-030685-I), by the Generalitat de Catalunya (contract 2021-SGR-00763, 2021-SGR-00115 and 2023-146511NB-I00), by the Agency for Management of University and Research Grants (AGAUR) of theGovernment of Catalonia under ”Ajuts per a la contractació de personal investigador novell” (fellowships No. 2019FI B01274and 2024 FI-1 00520), by the European Union within the framework of the ERDF of Catalonia 2014–2020 under the DRACproject [001-P-001723], and by the project HERMES funded by the European NextGenerationEU/PRTR via INCIBE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Hunting for open clusters in Gaia EDR3: 628 new open clusters found with OCfinder

  • Castro Ginard, Alfred
  • Jordi Nebot, Carme
  • Luri Carrascoso, Xavier
  • Cantat Gaudin, Tristan
  • Carrasco Martínez, Josep Manel
  • Casamiquela Floriach, Laia
  • Anders, Friedrich
  • Balaguer Núñez, Lola
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
Context. The improvements in the precision of the published data in Gaia EDR3 with respect to Gaia DR2, particularly for parallaxes and proper motions, offer the opportunity to increase the number of known open clusters in the Milky Way by detecting farther and fainter objects that have thus far gone unnoticed. Aims. Our aim is to continue to complete the open cluster census in the Milky Way with the detection of new stellar groups in the Galactic disc. We use Gaia EDR3 up to magnitude G = 18 mag, increasing the magnitude limit and therefore the search volume explored in one unit with respect to our previous studies. Methods. We used the OCfinder method to search for new open clusters in Gaia EDR3 using a big data environment. As a first step, OCfinder identified stellar statistical overdensities in five-dimensional astrometric space (position, parallax, and proper motions) using the DBSCAN clustering algorithm. Then, these overdensities were classified into random statistical overdensities or real physical open clusters using a deep artificial neural network trained on well-characterised G, GBP – GRP colour-magnitude diagrams. Results. We report the discovery of 628 new open clusters within the Galactic disc, with most of them being located beyond 1 kpc from the Sun. From the estimation of ages, distances, and line-of-sight extinctions of these open clusters, we see that young clusters align following the Galactic spiral arms while older ones are dispersed in the Galactic disc. Furthermore, we find that most open clusters are located at low Galactic altitudes with the exception of a few groups older than 1 Gyr. Conclusions. We show the success of the OCfinder method leading to the discovery of a total of 1274 open clusters (joining the discoveries here with the previous ones based on Gaia DR2), which represents almost 50% of the known population. Our ability to perform big data searches on a large volume of the Galactic disc, together with the higher precision in Gaia EDR3, enable us to keep completing the census with the discovery of new open clusters., This work was (partially) funded by the Spanish MICIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe” by the “European Union” through grant RTI2018-095076-B-C21, and the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M. A.C.G. acknowledges Spanish Ministry FPI fellowship n. BES-2016-078499. This work has been partially supported by the Spanish Government (PID2019-107255GB), by Generalitat de Catalunya (contract 2014-SGR-1051)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

XeroZerox: Analysis and Optimization of GPU Memory Management for High-Integrity Autonomous Systems

  • Calderon, Alejandro J.
  • Kosmidis, Leonidas
  • Nicolás, Carlos F.
  • Cazorla Almeida, Francisco Javier
Autonomous systems require high-performance processing capabilities, which demand the use of powerful accelerators such as GPUs. However, the use of GPUs in critical systems presents several challenges, since GPU programming models rely on explicit dynamic memory management. Traditionally, dynamic memory allocation in such systems is restricted to certain controlled scenarios, which require programs to be rewritten so that all the required memory is allocated at the beginning of the program and released at its end. However, many GPU applications do not follow this approach. Moreover, when dynamic memory allocation is used, it is critical to compute the exact amount of memory that will be used as well as to minimize it, to guarantee that it fits in the physical system memory. In this paper we present XeroZerox, an open-source tool that automatically converts the traditional dynamic memory allocations of GPU applications into allocations served from a centralized and optimally sized memory pool, which is managed in a way better suited for critical systems. XeroZerox allows legacy GPU applications to be used in a critical setup without rewriting them, minimizing at the same time their memory consumption and memory management runtime overhead., European Union’s Horizon Europe Program through the Modular Model-Based Design and Testing for Applications in Satellites (METASAT) (Grant Number: 101082622) Spanish Ministry of Economy and Competitiveness (Spanish State Research Agency/Agencia Española de Investigación (Grant Number: PID2019-107255GB-C21 and IJC-2020-045931-I) Department of Research and Universities of the Government of Catalonia with a Grant to the Computer Architecture and Operating Systems (CAOS) Research Group (Grant Number: 2021 SGR 00637), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

DDS: integrating data analytics transformations in task-based workflows [version 1, peer review: 1 approved, 2 approved with reservations]

  • Mammadli, Nihad
  • Ejarque Artigas, Jorge
  • Álvarez Cid-Fuentes, Javier
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
High-performance data analytics (HPDA) is a current trend in e-science research that aims to integrate traditional HPC with recent data analytic frameworks. Most of the work done in this field has focused on improving data analytic frameworks by implementing their engines on top of HPC technologies such as Message Passing Interface. However, there is a lack of integration from an application development perspective. HPC workflows have their own parallel programming models, while data analytic (DA) algorithms are mainly implemented using data transformations and executed with frameworks like Spark. Task-based programming models (TBPMs) are a very efficient approach for implementing HPC workflows. Data analytic transformations can also be decomposed as a set of tasks and implemented with a task-based programming model. In this paper, we present a methodology to develop HPDA applications on top of TBPMs that allow developers to combine HPC workflows and data analytic transformations seamlessly. A prototype of this approach has been implemented on top of the PyCOMPSs task- based programming model to validate two aspects: HPDA applications can be seamlessly developed and have better performance than Spark. We compare our results using different programs. Finally, we conclude with the idea of integrating DA into HPC applications and evaluation of our method against Spark., This research was financially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 780622; and the Spanish Government (PID2019-107255GB), Generalitat de Catalunya (2014-SGR-1051)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Block size estimation for data partitioning in HPC applications using machine learning techniques

  • Cantini, Riccardo
  • Marozzo, Fabrizio
  • Orsino, Alessio
  • Talia, Domenico
  • Trunfio, Paolo
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Ejarque Artigas, Jorge
  • Vázquez-Novoa, Fernando
The extensive use of HPC infrastructures and frameworks for running data-intensive applications has led to a growing interest in data partitioning techniques and strategies. In fact, application performance can be heavily affected by how data are partitioned, which in turn depends on the selected size for data blocks, i.e. the block size. Therefore, finding an effective partitioning, i.e. a suitable block size, is a key strategy to speed-up parallel data-intensive applications and increase scalability. This paper describes a methodology, namely BLEST-ML (BLock size ESTimation through Machine Learning), for block size estimation that relies on supervised machine learning techniques. The proposed methodology was evaluated by designing an implementation tailored to dislib, a distributed computing library highly focused on machine learning algorithms built on top of the PyCOMPSs framework. We assessed the effectiveness of the provided implementation through an extensive experimental evaluation considering different algorithms from dislib, datasets, and infrastructures, including the MareNostrum 4 supercomputer. The results we obtained show the ability of BLEST-ML to efficiently determine a suitable way to split a given dataset, thus providing a proof of its applicability to enable the efficient execution of data-parallel applications in high performance environments., This work has been partially supported by the European Commission through the Horizon 2020 Research and Innovation program and the EuroHPC JU under contract 955558 (eFlows4HPC project) and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021‑121957 and CEX2021‑ 001148‑S) and by the Spanish Government (PID2019‑107255GB), and Generalitat de Catalunya (contract 2021‑SGR‑00412). We also acknowledge financial support from “National Centre for HPC, Big Data and Quantum Computing”, CN00000013 ‑ CUP H23C22000360005, and from “FAIR ‑ Future Artificial Intelligence Research” project ‑ CUP H23C22000860006., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Parallel reduced-order modeling for digital twins using high-performance computing workflows

  • Ares de Parga Regalado, Sebastian|||0000-0001-5709-4683
  • Bravo Martínez, José Raúl|||0000-0002-4465-7536
  • Sibuet Ruiz, Nicolás|||0000-0003-2968-3653
  • Hernández Ortega, Joaquín Alberto|||0000-0001-9334-4002
  • Rossi, Riccardo|||0000-0003-0528-7074
  • Boschert, Stefan
  • Quintana Ortí, Enrique Salvador
  • Tomás Domínguez, Andrés Enrique
  • Tatu, Cristian Catalin
  • Vázquez Novoa, Fernando
  • Ejarque Artigas, Jorge
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
The integration of reduced-order models with high-performance computing is critical for developing digital twins, particularly for real-time monitoring and predictive maintenance of industrial systems. This paper presents a comprehensive, high-performance computing-enabled workflow for developing and deploying projection-based reduced-order models for large-scale mechanical simulations. We use PyCOMPSs’ parallel framework to efficiently execute reduced-order model training simulations, employing parallel singular value decomposition algorithms such as randomized singular value decomposition, Lanczos singular value decomposition, and full singular value decomposition based on tall-skinny QR. Moreover, we introduce a partitioned version of the hyperreduction scheme known as the Empirical Cubature Method to further enhance computational efficiency in projection-based reduced-order models for mechanical systems. Despite the widespread use of high-performance computing for projection-based reduced-order models, there is a significant lack of publications detailing comprehensive workflows for building and deploying end-to-end projection-based reduced-order models in high-performance computing environments. Our workflow is validated through a case study focusing on the thermal dynamics of a motor, a multiphysics problem involving convective heat transfer and mechanical components. The projection-based reduced-order model is designed to deliver a real-time prognosis tool that could enable rapid and safe motor restarts post-emergency shutdowns under different operating conditions, demonstrating its potential impact on the practice of simulations in engineering mechanics. To facilitate deployment, we use the High-Performance Computing Workflow as a Service strategy and Functional Mock-Up Units to ensure compatibility and ease of integration across high-performance computing, edge, and cloud environments. The outcomes illustrate the efficacy of combining projection-based reduced-order models and high-performance computing, establishing a precedent for scalable, real-time digital twin applications in computational mechanics across multiple industries., The authors acknowledge financial support from the Spanish Ministry of Economy and Competitiveness, through the “Severo Ochoa Programme for Centres of Excellence in R&D” (CEX2018-000797-S and CEX2021-001148-S)”, as well as to the project PID2019-107255GB (BSC authors). This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway. This publication is part of the R&D project PCI2021-121944 and PCI2021-121957, financed by MCIN/AEI/10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR”. Sebastian Ares de Parga Regalado acknowledges financial support from the Generalitat de Catalunya through the FI_SDUR-2021 grant (2021 FISDU 00142), which provided funding for his predoctoral training. BSC authors acknowledge financial support from the Departament de Recerca i Universitats de la Generalitat de Catalunya, research group MPiEDist (2021 SGR 00412). Author Fernando Vázquez is supported by PRE2022-104134 funded by MICIU/AEI /10.13039/501100011033 and by the FSE+., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Portability and scalability evaluation of large-scale statistical modeling and prediction software through HPC-ready containers

  • Abdulah, Sameh
  • Ejarque Artigas, Jorge
  • Marzouk, Omar
  • Ltaief, Hatem
  • Sun, Ying
  • Genton, Marc G.
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Keyes, David E.
HPC-based applications often have complex workflows with many software dependencies that hinder their portability on contemporary HPC architectures. In addition, these applications often require extraordinary efforts to deploy and execute at performance potential on new HPC systems, while the users expert in these applications generally have less expertise in HPC and related technologies. This paper provides a dynamic solution that facilitates containerization for transferring HPC software onto diverse parallel systems. The study relies on the HPC Workflow as a Service (HPCWaaS) paradigm proposed by the EuroHPC eFlows4HPC project. It offers to deploy workflows through containers tailored for any of a number of specific HPC systems. Traditional container image creation tools rely on OS system packages compiled for generic architecture families (x86_64, amd64, ppc64, …) and specific MPI or GPU runtime library versions. The containerization solution proposed in this paper leverages HPC Builders such as Spack or Easybuild and multi-platform builders such as buildx to create a service for automating the creation of container images for the software specific to each hardware architecture, aiming to sustain the overall performance of the software. We assess the efficiency of our proposed solution for porting the geostatistics ExaGeoStat software on various parallel systems while preserving the computational performance. The results show that the performance of the generated images is comparable with the native execution of the software on the same architectures. On the distributed-memory system, the containerized version can scale up to 256 nodes without impacting performance., This work has been supported by King Abdullah University of Science and Technology (KAUST), Saudi Arabia, the Spanish Government (contract PID2019-107255GB), by the Generalitat de Catalunya (contract 2017-SGR-01414), and by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (project eFlows4HPC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

WFA-FPGA: An efficient accelerator of the wavefront algorithm for short and long read genomics alignment

  • Haghi, Abbas
  • Marco Sola, Santiago|||0000-0001-7951-3914
  • Álvarez Martí, Lluc|||0000-0003-0506-8867
  • Diamantopoulos, Dionysios
  • Hagleitner, Christoph
  • Moretó Planas, Miquel|||0000-0002-9848-8758
In the last years, advances in genome sequencing technologies have enabled the proliferation of genomic applications that guide personalized medicine. These applications have an enormous computational cost due to the large amount of genomic data they process. The first step in many of these applications consists in aligning DNA reads against a reference genome. Very recently, the wavefront alignment (WFA) algorithm has been introduced, significantly reducing the execution time of the read alignment process. This paper presents the first FPGA-based hardware/software co-designed accelerator of such relevant algorithm. Compared to the reference WFA CPU-only implementation, the proposed accelerator achieves performance speedups of up to 13.5 while consuming up to 14.6 less energy when aligning short reads. When aligning long reads, the proposed accelerator achieves speedups of up to 9.9 while consuming up to 10.9 less energy., This work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21 and TED2021-132634A-I00), by the Generalitat de Catalunya (contract 2021-SGR-00763), by the European Union within the framework of the ERDF of Catalonia 2014-2020 under the DRAC project [001-P-001723], by the European NextGenerationEU/PRTR, by the Lenovo-BSC Contract-Framework (2022), and by the IBM/BSC Deep Learning Center initiative. M. Moreto has been partially supported by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal fellowship No. RYC-2016-21104., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

OpenPiton4HPC: optimizing OpenPiton towards high performance manycores

  • Leyva Santes, Neiel Israel
  • Monemi, Alireza
  • Oliete Escuín, Noelia
  • López Paradís, Guillem
  • Abancens Calvo, Xabier
  • Balkind, Jonathan
  • Vallejo Gutiérrez, Enrique
  • Moretó Planas, Miquel|||0000-0002-9848-8758
  • Alvarez Martí, Lluc
In recent years, numerous multicore RISC-V platforms have emerged. Development frameworks such as OpenPiton are employed in designs that aim to scale to a large number of cores. While OpenPiton presents a large flexibility, supporting different requirements and processing cores, some of its design decisions result in designs that are not optimized for High-Performance Computing (HPC) requirements. This work presents OpenPiton4HPC, an extension and optimization of OpenPiton for high-performance manycores. The key contributions are enabling multiple memory controllers, supporting router bypassing and NoC concentration, adding support for configurable cache sizes and cache block sizes, and allowing configurable bus widths in the NoC and in the cache SRAMs. On a 64-core manycore architecture, these new features and optimizations provide a geometric mean speedup of 7.2x compared to the OpenPiton baseline., This work was supported in part by the European HiPEAC Network of Excellence; in part by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 under Contract PID2019-107255GB-C21, Contract PID2022-136454NBC21, and Contract TED2021-131176B-I00; in part by the Spanish Ministry of Economy and Digital Transformation under Contract TSI069200-2023-0011; in part by the Generalitat de Catalunya under Contract 2021-SGR-00763; in part by Lenovo-BSC Contract-Framework Contract under Grant 2022; and in part by the BZL project is funded by the Ministerio de Transformación Digital y de la Función Pública by the Plan de Recuperación, Transformación y Resiliencia-financed by the European Union-NextGenerationEU. The work of Guillem LópezParadís was supported by the Generalitat de Catalunya through a FI Fellowship under Grant 2021FI-B00994., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Main sources of variability and non-determinism in AD software: taxonomy and prospects to handle them

  • Alcón Doganoc, Miguel
  • Brando Guillaumes, Axel
  • Mezzetti, Enrico|||0000-0002-1886-2931
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Cazorla Almeida, Francisco Javier
Safety standards in domains like automotive and avionics seek for deterministic execution (lack of jittery behavior) as a stepping stone to build a certification argument on the correct timing behavior of the system. However, the use of artificial-intelligence (AI) software in safety-critical systems carries several built-in and derivative sources of non-determinism that are at odds with safety standard determinism requirements. In this work we analyze the main sources of non-determinism of autonomous driving (AD) software, as highly representative and compelling example of the use of AI software, deep neural networks (DNN) in particular, in critical embedded systems. Paradoxically, DNN-based software in its inference phase—once the NN structure and weights have been fixed—turns out to consist mainly in matrix multiplications, which are inherently quite time deterministic. Our work focuses on sources of variability and non-determinism in AD software, covering algorithmic elements of AD software, low-level software and hardware computing platform, and data-flow constraints among AD modules. As final contribution of our work, which mainly focuses on problem identification, we develop some prospects on the information and metrics needed to better understand and control the unpredictability and non-determinism of AD software., This work has been supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GBC21/AEI/10.13039/501100011033, and the European Research Council (ERC) grant agreement No. 772773 (SuPerCom)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

At-scale evaluation of weight clustering to enable energy-efficient object detection

  • Caro, Martí
  • Tabani, Habani
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
Accelerators implementing Deep Neural Networks (DNNs) for image-based object detection operate on large volumes of data due to fetching images and neural network parameters, especially if they need to process video streams, hence with high power dissipation and bandwidth requirements to fetch all those data. While some solutions exist to mitigate power and bandwidth demands for data fetching, they are often assessed in the context of limited evaluations with a scale much smaller than that of the target application, which challenges finding the best tradeoff in practice. This paper sets up the infrastructure to assess at-scale a key power and bandwidth optimization – weight clustering – for You Only Look Once v3 (YOLOv3), a neural network-based object detection system, using videos of real driving conditions. Our assessment shows that accelerators such as systolic arrays with an Output Stationary architecture turn out to be a highly effective solution combined with weight clustering. In particular, applying weight clustering independently per neural network layer, and using between 32 (5-bit) and 256 (8-bit) weights allows achieving an accuracy close to that of the original YOLOv3 weights (32-bit weights). Such bit-count reduction of the weights allows shaving bandwidth requirements down to 30%–40% of the original requirements, and reduces energy consumption down to 45%. This is based on the fact that (i) energy due to multiply-and-accumulate operations is much smaller than DRAM data fetching, and (ii) designing accelerators appropriately may make that most of the data fetched corresponds to neural network weights, where clustering can be applied. Overall, our at-scale assessment provides key results to architect camera-based object detection accelerators by putting together a real-life application (YOLOv3), and real driving videos, in a unified setup so that trends observed are reliable., The DRAC project under grant 001-P-001723 is co-financed by the European Union Regional Development Fund within the framework of the ERDF Operational Program of Catalonia 2014-2020 with a grant of 50% of total eligible cost. This work has also been partially sup- ported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GB- C21/AEI/10.13039/501100011033., Peer Reviewed
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Accurate and efficient constrained molecular dynamics of polymers using Newton's method and special purpose code

  • López-Villellas, L.
  • Kjelgaard Mikkelsen C. Ch.
  • Galano-Frutos, J. J.
  • Marco-Sola, S.
  • Alastruey-Benedé, J.
  • Ibáñez, P.
  • Moretó, M.
  • Sancho, J.
  • García-Risueño, P.
In molecular dynamics simulations we can often increase the time step by imposing constraints on bond lengths and bond angles. This allows us to extend the length of the time interval and therefore the range of physical phenomena that we can afford to simulate. We examine the existing algorithms and software for solving nonlinear constraint equations in parallel and we explain why it is necessary to advance the state-of-the-art. We present ILVES-PC, a new algorithm for imposing bond constraints on proteins accurately and efficiently. It solves the same system of differential algebraic equations as the celebrated SHAKE algorithm, but ILVES-PC solves the nonlinear constraint equations using Newton’s method rather than the nonlinear Gauss-Seidel method. Moreover, ILVES-PC solves the necessary linear systems using a specialized linear solver that exploits the structure of the protein. ILVES-PC can rapidly solve constraint equations as accurately as the hardware will allow. The run-time of ILVES-PC is proportional to the number of constraints. We have integrated ILVES-PC into GROMACS and simulated proteins of different sizes. Compared with SHAKE, we have achieved speedups of up to 4.9× in single-threaded executions and up to 76× in shared-memory multi-threaded executions. Moreover, ILVES-PC is more accurate than P-LINCS algorithm. Our work is a proof-of-concept of the utility of software designed specifically for the simulation of polymers.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Porting and optimizing BWA-MEM2 using the Fujitsu A64FX processor

  • Langarita, Rubén
  • Armejach, Adrià
  • Ibáñez, Pablo
  • Alastruey-Benedé, Jesús
  • Moretó, Miquel
Sequence alignment pipelines for human genomes are an emerging workload that will dominate in the precision medicine field. BWA-MEM2 is a tool widely used in the scientific community to perform read mapping studies. In this paper, we port BWA-MEM2 to the AArch64 architecture using the ARMv8-A specification, and we compare the resulting version against an Intel Skylake system both in performance and in energy-to-solution. The porting effort entails numerous code modifications, since BWA-MEM2 implements certain kernels using x86 64 specific intrinsics, e.g., AVX-512. To adapt this code we use the recently introduced Arm’s Scalable Vector Extensions (SVE). More specifically, we use Fujitsu’s A64FX processor, the first to implement SVE. The A64FX powers the Fugaku Supercomputer that led the Top500 ranking from June 2020 to November 2021. After porting BWA-MEM2 we define and implement a number of optimizations to improve performance in the A64FX target architecture. We show that while the A64FX performance is lower than that of the Skylake system, A64FX delivers 11.6% better energy-to-solution on average. All the code used for this article is available at https://gitlab.bsc.es/rlangari/bwa-a64fx.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

GenArchBench: A genomics benchmark suite for arm HPC processors

  • López-Villellas, Lorién
  • Langarita-Benítez, Rubén
  • Badouh, Asaf
  • Soria-Pardos, Víctor
  • Aguado-Puig, Quim
  • López-Paradís, Guillem
  • Doblas, Max
  • Setoain, Javier
  • Kim, Chulho
  • Ono, Makoto
  • Armejach, Adrià
  • Marco-Sola, Santiago
  • Alastruey-Benedé, Jesús
  • Ibáñez, Pablo
  • Moretó, Miquel
Arm usage has substantially grown in the High-Performance Computing (HPC) community. Japanese supercomputer Fugaku, powered by Arm-based A64FX processors, held the top position on the Top500 list between June 2020 and June 2022, currently sitting in the fourth position. The recently released 7th generation of Amazon EC2 instances for compute-intensive workloads (C7 g) is also powered by Arm Graviton3 processors. Projects like European Mont-Blanc and U.S. DOE/NNSA Astra are further examples of Arm irruption in HPC. In parallel, over the last decade, the rapid improvement of genomic sequencing technologies and the exponential growth of sequencing data has placed a significant bottleneck on the computational side. While most genomics applications have been thoroughly tested and optimized for x86 systems, just a few are prepared to perform efficiently on Arm machines. Moreover, these applications do not exploit the newly introduced Scalable Vector Extensions (SVE). This paper presents GenArchBench, the first genome analysis benchmark suite targeting Arm architectures. We have selected computationally demanding kernels from the most widely used tools in genome data analysis and ported them to Arm-based A64FX and Graviton3 processors. Overall, the GenArch benchmark suite comprises 13 multi-core kernels from critical stages of widely-used genome analysis pipelines, including base-calling, read mapping, variant calling, and genome assembly. Our benchmark suite includes different input data sets per kernel (small and large), each with a corresponding regression test to verify the correctness of each execution automatically. Moreover, the porting features the usage of the novel Arm SVE instructions, algorithmic and code optimizations, and the exploitation of Arm-optimized libraries. We present the optimizations implemented in each kernel and a detailed performance evaluation and comparison of their performance on four different HPC machines (i.e., A64FX, Graviton3, Intel Xeon Skylake Platinum, and AMD EPYC Rome). Overall, the experimental evaluation shows that Graviton3 outperforms other machines on average. Moreover, we observed that the performance of the A64FX is significantly constrained by its small memory hierarchy and latencies. Additionally, as proof of concept, we study the performance of a production-ready tool that exploits two of the ported and optimized genomic kernels.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

OpenPiton4HPC: optimizing OpenPiton towards high performance manycores

  • Leyva Santes, Neiel Israel
  • Monemi, Alireza
  • Oliete Escuín, Noelia
  • López Paradís, Guillem
  • Abancens Calvo, Xabier
  • Balkind, Jonathan
  • Vallejo Gutiérrez, Enrique
  • Moretó Planas, Miquel
  • Alvarez Martí, Lluc
In recent years, numerous multicore RISC-V platforms have emerged. Development frameworks such as OpenPiton are employed in designs that aim to scale to a large number of cores. While OpenPiton presents a large flexibility, supporting different requirements and processing cores, some of its design decisions result in designs that are not optimized for High-Performance Computing (HPC) requirements. This work presents OpenPiton4HPC, an extension and optimization of OpenPiton for high-performance manycores. The key contributions are enabling multiple memory controllers, supporting router bypassing and NoC concentration, adding support for configurable cache sizes and cache block sizes, and allowing configurable bus widths in the NoC and in the cache SRAMs. On a 64-core manycore architecture, these new features and optimizations provide a geometric mean speedup of 7.2x compared to the OpenPiton baseline., This work was supported in part by the European HiPEAC Network of Excellence; in part by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 under Contract PID2019-107255GB-C21, Contract PID2022-136454NBC21, and Contract TED2021-131176B-I00; in part by the Spanish Ministry of Economy and Digital Transformation under Contract TSI069200-2023-0011; in part by the Generalitat de Catalunya under Contract 2021-SGR-00763; in part by Lenovo-BSC Contract-Framework Contract under Grant 2022; and in part by the BZL project is funded by the Ministerio de Transformación Digital y de la Función Pública by the Plan de Recuperación, Transformación y Resiliencia-financed by the European Union-NextGenerationEU. The work of Guillem LópezParadís was supported by the Generalitat de Catalunya through a FI Fellowship under Grant 2021FI-B00994., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

SafeDE: A low-cost hardware solution to enforce diverse redundancy in multicores

  • Bas Jalón, Francisco
  • Alcaide Portet, Sergi
  • Cabo Pitarch, Guillem
  • Benedicte Illescas, Pedro
  • Abella Ferrer, Jaume
Failure risk must be tiny in high-integrity systems, such as those in cars, satellites and aircraft. Hence, safety measures must be deployed to avoid a single fault leading to a failure. Redundancy has been often used to address this concern, but it has been proven insufficient if a single fault can cause the same error in all redundant elements, which defeats the purpose of redundancy for error detection. Hence, to avoid this scenario, diversity is implemented along with redundancy, being lockstep execution the most popular diverse redundancy solution for computing cores. However, classic lockstep solutions have non-negligible limitations if implemented in hardware (e.g., half of the cores can only be used for redundant execution and are not even visible at user level), or in software (e.g., the software loop to enforce staggering is long and costs performance). This paper tackles the limitations of classic lockstep solutions by providing an extended analysis and evaluation of SafeDE, a Diversity Enforcement hardware module combining the short loop to enforce diversity of hardware solutions, and the nonintrusiveness of software solutions. Hence, cores can operate in lockstep mode efficiently or run independent tasks. In this paper, we present SafeDE and its rationale, its application to N-modular systems, its hardware and software integration, and an evaluation showing its performance and area efficiency, and its behavior in the presence of faults., This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 871467, and in part by the Spanish Ministry of Science and Innovation under Grant PID2019-107255GB-C21/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Boosting HPC data analysis performance with the ParSoDA-Py library

  • Belcastro, Loris
  • Giampà, Salvatore
  • Marozzo, Fabrizio
  • Talia, Domenico
  • Trunfio, Paolo
  • Badia Sala, Rosa Maria
  • Ejarque, Jorge
  • Mammadli, Nihad
Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications., This work has been partially funded by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under Grant agreement No 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021-121957), project eFlows4HPC. It has also been supported by the Spanish Government (PID2019-107255GB) and by the Departament de Recerca i Universitats de la Generalitat de Catalunya to the Research Group MPiEDist (2021 SGR 00412) We also acknowledge financial support from “National Centre for HPC, Big Data and Quantum Computing," CN00000013 - CUP H23C22000360005, and from “FAIR - Future Artificial Intelligence Research" Project - CUP H23C22000860006., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Portability and scalability evaluation of large-scale statistical modeling and prediction software through HPC-ready containers

  • Abdulah, Sameh
  • Ejarque Artigas, Jorge
  • Marzouk, Omar
  • Ltaief, Hatem
  • Sun, Ying
  • Genton, Marc G.
  • Badia Sala, Rosa Maria
  • Keyes, David E.
HPC-based applications often have complex workflows with many software dependencies that hinder their portability on contemporary HPC architectures. In addition, these applications often require extraordinary efforts to deploy and execute at performance potential on new HPC systems, while the users expert in these applications generally have less expertise in HPC and related technologies. This paper provides a dynamic solution that facilitates containerization for transferring HPC software onto diverse parallel systems. The study relies on the HPC Workflow as a Service (HPCWaaS) paradigm proposed by the EuroHPC eFlows4HPC project. It offers to deploy workflows through containers tailored for any of a number of specific HPC systems. Traditional container image creation tools rely on OS system packages compiled for generic architecture families (x86_64, amd64, ppc64, …) and specific MPI or GPU runtime library versions. The containerization solution proposed in this paper leverages HPC Builders such as Spack or Easybuild and multi-platform builders such as buildx to create a service for automating the creation of container images for the software specific to each hardware architecture, aiming to sustain the overall performance of the software. We assess the efficiency of our proposed solution for porting the geostatistics ExaGeoStat software on various parallel systems while preserving the computational performance. The results show that the performance of the generated images is comparable with the native execution of the software on the same architectures. On the distributed-memory system, the containerized version can scale up to 256 nodes without impacting performance., This work has been supported by King Abdullah University of Science and Technology (KAUST), Saudi Arabia, the Spanish Government (contract PID2019-107255GB), by the Generalitat de Catalunya (contract 2017-SGR-01414), and by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (project eFlows4HPC)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

At-scale evaluation of weight clustering to enable energy-efficient object detection

  • Caro, Martí
  • Tabani, Habani
  • Abella Ferrer, Jaume
Accelerators implementing Deep Neural Networks (DNNs) for image-based object detection operate on large volumes of data due to fetching images and neural network parameters, especially if they need to process video streams, hence with high power dissipation and bandwidth requirements to fetch all those data. While some solutions exist to mitigate power and bandwidth demands for data fetching, they are often assessed in the context of limited evaluations with a scale much smaller than that of the target application, which challenges finding the best tradeoff in practice. This paper sets up the infrastructure to assess at-scale a key power and bandwidth optimization – weight clustering – for You Only Look Once v3 (YOLOv3), a neural network-based object detection system, using videos of real driving conditions. Our assessment shows that accelerators such as systolic arrays with an Output Stationary architecture turn out to be a highly effective solution combined with weight clustering. In particular, applying weight clustering independently per neural network layer, and using between 32 (5-bit) and 256 (8-bit) weights allows achieving an accuracy close to that of the original YOLOv3 weights (32-bit weights). Such bit-count reduction of the weights allows shaving bandwidth requirements down to 30%–40% of the original requirements, and reduces energy consumption down to 45%. This is based on the fact that (i) energy due to multiply-and-accumulate operations is much smaller than DRAM data fetching, and (ii) designing accelerators appropriately may make that most of the data fetched corresponds to neural network weights, where clustering can be applied. Overall, our at-scale assessment provides key results to architect camera-based object detection accelerators by putting together a real-life application (YOLOv3), and real driving videos, in a unified setup so that trends observed are reliable., The DRAC project under grant 001-P-001723 is co-financed by the European Union Regional Development Fund within the framework of the ERDF Operational Program of Catalonia 2014-2020 with a grant of 50% of total eligible cost. This work has also been partially sup- ported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GB- C21/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

SpChar: characterizing the sparse puzzle via decision trees

  • Sgherzi, Francesco
  • Siracusa, Marco
  • Fernández Vega, Ivan
  • Armejach Sanosa, Adrià
  • Moretó Planas, Miquel
Sparse matrix computation is crucial in various modern applications, including large-scale graph analytics, deep learning, and recommender systems. The performance of sparse kernels varies greatly depending on the structure of the input matrix, making it difficult to gain a comprehensive understanding of sparse computation and its relationship to inputs, algorithms, and target machine architecture. Despite extensive research on certain sparse kernels, such as Sparse Matrix-Vector Multiplication (SpMV), the overall family of sparse algorithms has yet to be investigated as a whole. This paper introduces SpChar, a workload characterization methodology for general sparse computation. SpChar employs tree-based models to identify the most relevant hardware and input characteristics, starting from hardware and input-related metrics gathered from Performance Monitoring Counters (PMCs) and matrices. Our analysis enables the creation of a characterization loop that facilitates the optimization of sparse computation by mapping the impact of architectural features to inputs and algorithmic choices. We apply SpChar to more than 600 matrices from the SuiteSparse Matrix collection and three state-of-the-art Arm Central Processing Units (CPUs) to determine the critical hardware and software characteristics that affect sparse computation. In our analysis, we determine that the biggest limiting factors for high-performance sparse computation are (1) the latency of the memory system, (2) the pipeline flush overhead resulting from branch misprediction, and (3) the poor reuse of cached elements. Additionally, we propose software and hardware optimizations that designers can implement to create a platform suitable for sparse computation. We then investigate these optimizations using the gem5 simulator to achieve a significant speedup of up to 2.63× compared to a CPU where the optimizations are not applied., This research was supported by: the Spanish Ministry of Science and Innovation (MCIN) through contracts PID2019-107255GB-C21, PID2019-107255GBC21 (MCIN/AEI/10.13039/501100011033) and the European Processor Initiative (EPI) under grant agreement No. 101036168. M. Siracusa has been partially supported by an FI fellowship (2022FI B 00969). Finally, A. Armejach is a Serra Hunter Fellow, Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Scanflow: A multi-graph framework for machine learning workflow management, supervision, and debugging

  • Bravo Rocca, Gusseppe Jesus
  • Liu, Peini
  • Guitart Fernández, Jordi
  • Dholakia, Ajay
  • Ellison, David
  • Falkanger, Jeffrey
  • Hodak, Miroslav
Machine Learning (ML) is more than just training models, the whole workflow must be considered. Once deployed, a ML model needs to be watched and constantly supervised and debugged to guarantee its validity and robustness in unexpected situations. Debugging in ML aims to identify (and address) the model weaknesses in not trivial contexts. Several techniques have been proposed to identify different types of model weaknesses, such as bias in classification, model decay, adversarial attacks, etc., yet there is not a generic framework that allows them to work in a collaborative, modular, portable, iterative way and, more importantly, flexible enough to allow both human- and machine-driven techniques. In this paper, we propose a novel containerized directed graph framework to support and accelerate end-to-end ML workflow management, supervision, and debugging. The framework allows defining and deploying ML workflows in containers, tracking their metadata, checking their behavior in production, and improving the models by using both learned and human-provided knowledge. We demonstrate these capabilities by integrating in the framework two hybrid systems to detect data drift distribution which identify the samples that are far from the latent space of the original distribution, ask for human intervention, and whether retrain the model or wrap it with a filter to remove the noise of corrupted data at inference time. We test these systems on MNIST-C, CIFAR-10-C, and FashionMNIST-C datasets, obtaining promising accuracy results with the help of human involvement., This work was partially supported by Lenovo as part of Framework Contract Lenovo-BSC 2020, by the Spanish Government under contract PID2019-107255 GB, and by the Generalitat de Catalunya, Spain under contract 2017-SGR-1414 and under grant 2020 FI-B 00257., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

O(n) key–value sort with active compute memory

  • Esmaili Dokht, Pouya
  • Guiot Cusido, Miquel
  • Radojkovic, Petar
  • Martorell Bofill, Xavier
  • Ayguadé Parra, Eduard
  • Labarta Mancho, Jesús José
  • Adlard, Jason
  • Amato, Paolo
  • Sforzin, Marco
We propose the Active Compute Memory (ACM), a near-memory-processing architecture capable of performing key–value sort directly in the DRAM. In the ACM architecture, sort is merely the writing of data into memory with one addressing protocol (perspective) and reading it back with different perspective. The first perspective is conventional, based on the data address; the second perspective is the sorted order. The ACM requires additional tables to store the meta-data and moderate control logic enhancements that can be implemented directly in the DRAM silicon. By these modest enhancements to DRAM, ACM exploits the parallelism inherently available in the row buffer to enable sort with O ( n ) complexity. This leads to an order of magnitude improvement in ACM performance and energy compared to conventional O ( n log n ) CPU-centric sort algorithms. The ACM also shows superior performance compared to other near-memory sort accelerators. This is because the ACM processing is done near the row buffer and it exploits much lower memory access latency, higher bandwidth and wider parallel processing. The sort operation covered in this paper is just an example of an address management operation that can be efficiently implemented directly in the DRAM silicon. We release as an open source the simulation infrastructure for the ACM performance and energy modeling. We would encourage the community to use it, adapt it to other PIM proposals, and share their own evaluations., This work was funded by the Collaboration Agreement between Micron Technology, Inc. and BSC. The work was also supported by the Spanish Government, under the contracts PID2019-107255GB-C21 and CEX2021-001148-S funded by MCIN/AEI/ 10.13039/501100011033. The work also received funding from the Department of Research and Universities of the Government of Catalonia to the AccMem Research Group (Code: 2021 SGR 00807) and project DEEP-SEA funded by European Union’s Horizon 2020 research and innovation programme , under Grant Agreement no.955606 and the Spanish Government, under the contract PCI2021-121958 funded by MCIN/AEI/ 10.13039/501100011033 and “European Union NextGenerationEU/PRTR”., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

XeroZerox: Analysis and Optimization of GPU Memory Management for High-Integrity Autonomous Systems

  • Calderon, Alejandro J.
  • Kosmidis, Leonidas
  • Nicolás, Carlos F.
  • Cazorla Almeida, Francisco Javier
Autonomous systems require high-performance processing capabilities, which demand the use of powerful accelerators such as GPUs. However, the use of GPUs in critical systems presents several challenges, since GPU programming models rely on explicit dynamic memory management. Traditionally, dynamic memory allocation in such systems is restricted to certain controlled scenarios, which require programs to be rewritten so that all the required memory is allocated at the beginning of the program and released at its end. However, many GPU applications do not follow this approach. Moreover, when dynamic memory allocation is used, it is critical to compute the exact amount of memory that will be used as well as to minimize it, to guarantee that it fits in the physical system memory. In this paper we present XeroZerox, an open-source tool that automatically converts the traditional dynamic memory allocations of GPU applications into allocations served from a centralized and optimally sized memory pool, which is managed in a way better suited for critical systems. XeroZerox allows legacy GPU applications to be used in a critical setup without rewriting them, minimizing at the same time their memory consumption and memory management runtime overhead., European Union’s Horizon Europe Program through the Modular Model-Based Design and Testing for Applications in Satellites (METASAT) (Grant Number: 101082622) Spanish Ministry of Economy and Competitiveness (Spanish State Research Agency/Agencia Española de Investigación (Grant Number: PID2019-107255GB-C21 and IJC-2020-045931-I) Department of Research and Universities of the Government of Catalonia with a Grant to the Computer Architecture and Operating Systems (CAOS) Research Group (Grant Number: 2021 SGR 00637), Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Adaptive power shifting for power-constrained heterogeneous systems

  • Ortega Carrasco, Cristobal
  • Alvarez Martí, Lluc
  • Buyuktosunoglu, Alper
  • Bertran Monfort, Ramon
  • Rosedahl, Todd Jon
  • Bose, Pradip
  • Moretó Planas, Miquel
The number and heterogeneity of compute devices, even within a single compute node, has been steadily on the rise. Since all systems must operate under a power cap, the number of discrete devices that can run simultaneously at their highest frequency is limited by the globally-imposed power cap. Current systems incorporate a centralized power management unit that statically controls the distribution of power among the devices within the node. However, such static distribution policies are unaware of the dynamic utilization profile across the devices, which leads to unfair power allocations that end up degrading system throughput performance. The problem is particularly acute in the presence of heterogeneity since type-specific performance-boost capabilities cannot be leveraged via utilization-agnostic static power allocations. This paper proposes Adaptive Power Shifting for multi-accelerator heterogeneous systems (APS), a technique that leverages system utilization information to dynamically allocate and re-distribute power budgets across multiple discrete devices. Democratizing the power allocation based on dynamic needs results in dramatic speedup over a need-agnostic static allocation. We use APS in a real OpenPOWER compute node with 2 CPUs and 4 GPUs to demonstrate the value of on-demand, equitable power allocations. Overall, the proposed solution increases performance with respect to two state-of-the-art techniques by up to 14.9% and 13.8%., This work has been partially supported by the European Union’s Horizon 2020 research and innovation program under the Mont-Blanc 2020 project (grant agreement 779877), by the Spanish Ministry of Science and Innovation (contract PID2019-107255GB-C22), by Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272) and by the IBM/BSC Deep Learning Center initiative. Ll. Alvarez has been supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under the Juan de la Cierva Formacion fellowship No. FJCI-2016- 30984. M. Moreto has been supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal fellowship No. RYC-2016-21104., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Accurate and efficient constrained molecular dynamics of polymers using Newton's method and special purpose code

  • López Villellas, Lorién
  • Kjelgaard Mikkelsen, Carl Christian
  • Galano Frutos, Juan José
  • Marco Sola, Santiago
  • Alastruey Benedé, Jesús
  • Ibáñez Marín, Pablo Enrique
  • Moretó Planas, Miquel
  • Sancho Sanz, Javier
  • García Risueño, Pablo
In molecular dynamics simulations we can often increase the time step by imposing constraints on bond lengths and bond angles. This allows us to extend the length of the time interval and therefore the range of physical phenomena that we can afford to simulate. We examine the existing algorithms and software for solving nonlinear constraint equations in parallel and we explain why it is necessary to advance the state-of-the-art. We present ILVES-PC, a new algorithm for imposing bond constraints on proteins accurately and efficiently. It solves the same system of differential algebraic equations as the celebrated SHAKE algorithm, but ILVES-PC solves the nonlinear constraint equations using Newton's method rather than the nonlinear Gauss-Seidel method. Moreover, ILVES-PC solves the necessary linear systems using a specialized linear solver that exploits the structure of the protein. ILVES-PC can rapidly solve constraint equations as accurately as the hardware will allow. The run-time of ILVES-PC is proportional to the number of constraints. We have integrated ILVES-PC into GROMACS and simulated proteins of different sizes. Compared with SHAKE, we have achieved speedups of up to 4.9× in single-threaded executions and up to 76× in shared-memory multi-threaded executions. Moreover, ILVES-PC is more accurate than P-LINCS algorithm. Our work is a proof-of-concept of the utility of software designed specifically for the simulation of polymers., This work has been partially supported by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21, PID2019-105660RB-C21, and PID2019-107293GB-I00), by the Generalitat de Catalunya (contract 2017-SGR-1328), by the Gobierno de Aragón (E45_20R T58_20R research groups), and by Lenovo-BSC Contract-Framework Contract (2020). Santiago Marco was supported by the Agencia Estatal de Investigación (Spain) under Juan de la Cierva fellowship grant IJC2020-045916-I. Miquel Moretó was partially supported by the Agencia Estatal de Investigación (Spain) under Ramón y Cajal fellowship number RYC-2016-21104. Carl Christian Kjelgaard Mikkelsen is supported by eSSENCE, a collaborative e-Science programme funded by the Swedish Research Council within the framework of the strategic research areas designated by the Swedish Government., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

On Neural Networks Redundancy and Diversity for Their Use in Safety-Critical Systems

  • Brando, Axel
  • Serra, Isabel
  • Mezzetti, Enrico
  • Cazorla Almeida, Francisco Javier
  • Perez Cerrolaza, Jon
  • Abella Ferrer, Jaume
An increasing number of critical functionalities integrated in embedded critical systems rely on deep learning (DL) technology. This article summarizes certain key aspects of DL’s intrinsic stochastic and training-data-dependent nature that are at odds with current domain-specific functional safety standards. We exemplify how redundancy and diversity of neural networks can help to reconcile DL technology and functional safety requirements., The research leading to these results has received funding from the European Research Council (ERC) grant agreement No. 772773 (SuPerCom), the Horizon Europe Programme under the SAFEXPLAIN Project (www.safexplain.eu), grant agreement num.101069595, and the Spanish Ministry of Science and Innovation under grant PID2019-107255GBC21/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

QuickEd: high-performance exact sequence alignment based on bound-and-align

  • Doblas Font, Max
  • Lostes Cazorla, Óscar
  • Aguado Puig, Quim
  • Iñiguez Rodríguez, Cristian
  • Moretó Planas, Miquel
  • Marco Sola, Santiago
Motivation: Pairwise sequence alignment is a core component of multiple sequencing-data analysis tools. Recent advancements in sequencing technologies have enabled the generation of longer sequences at a much lower price. Thus, long-read sequencing technologies have become increasingly popular in sequencing-based studies. However, classical sequence analysis algorithms face significant scalability challenges when aligning long sequences. As a result, several heuristic methods have been developed to improve performance at the expense of accuracy, as they often fail to produce the optimal alignment. Results: This paper introduces QuickEd, a sequence alignment algorithm based on a bound-and-align strategy. First, QuickEd effectively bounds the maximum alignment-score using efficient heuristic strategies. Then, QuickEd utilizes this bound to reduce the computations required to produce the optimal alignment. Compared to Oðn2Þ complexity of traditional dynamic programming algorithms, QuickEd’s bound-and-align strategy achieves OðnbsÞ complexity, where n is the sequence length and bs is an estimated upper bound of the alignment-score between the sequences. As a result, QuickEd is consistently faster than other state-of-the-art implementations, such as Edlib and BiWFA, achieving performance speedups of 4:2-5:9× and 3:8-4:4×, respectively, aligning long and noisy datasets. In addition, QuickEd maintains a stable memory footprint below 35 MB while aligning sequences up to 1 Mbp. Availability and implementation: QuickEd code and documentation are publicly available at https://github.com/maxdoblas/QuickEd., This work was partially funded by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 and FSEþ [PID2019-107255GB-C21, PID2020-113614RBC21, TED2021-132634A-I00, and PRE2021-101059 to Q. A.-P.], the Spanish Ministry of Economy, Industry, and Competitiveness through an FPU fellowship [FPU20-04076 to M.D.], the Spanish Ministry, Ministerio para la Transformación Digital y de la Función Pública, and the European Union – NextGenerationEU through the Cátedra Chip UPC project [TSI-069100-2023-0015], Lenovo-BSC Contract-Framework Contract (2020), and the Generalitat de Catalunya GenCat-DIUiE (GRR) [2017-SGR-313, 2017- SGR-1328, 2017-SGR-1414, 2021-SGR-00763, and 2021- SGR-00574]., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Vector extensions in COTS processors to increase guaranteed performance in real-time systems

  • Pujol Torramorell, Roger
  • Jorba Jorba, Josep
  • Tabani, Hamid
  • Kosmidis, Leonidas
  • Mezzetti, Enrico
  • Abella Ferrer, Jaume
  • Cazorla Almeida, Francisco Javier
The need for increased application performance in high-integrity systems like those in avionics is on the rise as software continues to implement more complex functionalities. The prevalent computing solution for future high-integrity embedded products are multi-processors systems-on-chip (MPSoC) processors. MPSoCs include CPU multicores that enable improving performance via thread-level parallelism. MPSoCs also include generic accelerators (GPUs) and application-specific accelerators. However, the data processing approach (DPA) required to exploit each of these underlying parallel hardware blocks carries several open challenges to enable the safe deployment in high-integrity domains. The main challenges include the qualification of its associated runtime system and the difficulties in analyzing programs deploying the DPA with out-of-the-box timing analysis and code coverage tools. In this work, we perform a thorough analysis of vector extensions (VExt) in current COTS processors for high-integrity systems. We show that VExt prevent many of the challenges arising with parallel programming models and GPUs. Unlike other DPAs, VExt require no runtime support, prevent by design race conditions that might arise with parallel programming models, and have minimum impact on the software ecosystem enabling the use of existing code coverage and timing analysis tools. We develop vectorized versions of neural network kernels and show that the NVIDIA Xavier VExt provide a reasonable increase in guaranteed application performance of up to 2.7x. Our analysis contends that VExt are the DPA approach with arguably the fastest path for adoption in high-integrity systems., This work has received funding from the the European Research Council (ERC) grant agreement No. 772773 (SuPerCom) and the Spanish Ministry of Science and Innovation (AEI/10.13039/501100011033) under grants PID2019-107255GB-C21 and IJC2020-045931-I., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

GenArchBench: A genomics benchmark suite for arm HPC processors

  • López Villellas, Lorien
  • Langarita Benítez, Rubén
  • Badouh, Asaf
  • Soria Pardos, Víctor
  • Aguado Puig, Quim
  • López Paradís, Guillem
  • Doblas Font, Max
  • Setoain, Javier
  • Kim, Chulho
  • Ono, Makoto
  • Armejach Sanosa, Adrià
  • Marco Sola, Santiago
  • Alastruey Benedé, Jesús
  • Ibáñez Marín, Pablo
  • Moretó Planas, Miquel
Arm usage has substantially grown in the High-Performance Computing (HPC) community. Japanese supercomputer Fugaku, powered by Arm-based A64FX processors, held the top position on the Top500 list between June 2020 and June 2022, currently sitting in the fourth position. The recently released 7th generation of Amazon EC2 instances for compute-intensive workloads (C7 g) is also powered by Arm Graviton3 processors. Projects like European Mont-Blanc and U.S. DOE/NNSA Astra are further examples of Arm irruption in HPC. In parallel, over the last decade, the rapid improvement of genomic sequencing technologies and the exponential growth of sequencing data has placed a significant bottleneck on the computational side. While most genomics applications have been thoroughly tested and optimized for x86 systems, just a few are prepared to perform efficiently on Arm machines. Moreover, these applications do not exploit the newly introduced Scalable Vector Extensions (SVE). This paper presents GenArchBench, the first genome analysis benchmark suite targeting Arm architectures. We have selected computationally demanding kernels from the most widely used tools in genome data analysis and ported them to Arm-based A64FX and Graviton3 processors. Overall, the GenArch benchmark suite comprises 13 multi-core kernels from critical stages of widely-used genome analysis pipelines, including base-calling, read mapping, variant calling, and genome assembly. Our benchmark suite includes different input data sets per kernel (small and large), each with a corresponding regression test to verify the correctness of each execution automatically. Moreover, the porting features the usage of the novel Arm SVE instructions, algorithmic and code optimizations, and the exploitation of Arm-optimized libraries. We present the optimizations implemented in each kernel and a detailed performance evaluation and comparison of their performance on four different HPC machines (i.e., A64FX, Graviton3, Intel Xeon Skylake Platinum, and AMD EPYC Rome). Overall, the experimental evaluation shows that Graviton3 outperforms other machines on average. Moreover, we observed that the performance of the A64FX is significantly constrained by its small memory hierarchy and latencies. Additionally, as proof of concept, we study the performance of a production-ready tool that exploits two of the ported and optimized genomic kernels., This work has been partially supported by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21, PID2019-105660RB-C21, PID2022136454NB-C22, and TED2021-132634A-I00), by the Generalitat de Catalunya, Spain (contract 2021-SGR-763), by the Gobierno de Aragón (T58_23R research group), by the European Union NextGenerationEU/ PRTR, and by Lenovo BSC Contract-Framework Contract (2020)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

DDS: integrating data analytics transformations in task-based workflows [version 1; peer review: 1 approved, 2 approved with reservations]

  • Mammadli, Nihad
  • Ejarque Artigas, Jorge
  • Álvarez Cid-Fuentes, Javier
  • Badia Sala, Rosa Maria
High-performance data analytics (HPDA) is a current trend in e-science research that aims to integrate traditional HPC with recent data analytic frameworks. Most of the work done in this field has focused on improving data analytic frameworks by implementing their engines on top of HPC technologies such as Message Passing Interface. However, there is a lack of integration from an application development perspective. HPC workflows have their own parallel programming models, while data analytic (DA) algorithms are mainly implemented using data transformations and executed with frameworks like Spark. Task-based programming models (TBPMs) are a very efficient approach for implementing HPC workflows. Data analytic transformations can also be decomposed as a set of tasks and implemented with a task-based programming model. In this paper, we present a methodology to develop HPDA applications on top of TBPMs that allow developers to combine HPC workflows and data analytic transformations seamlessly. A prototype of this approach has been implemented on top of the PyCOMPSs task- based programming model to validate two aspects: HPDA applications can be seamlessly developed and have better performance than Spark. We compare our results using different programs. Finally, we conclude with the idea of integrating DA into HPC applications and evaluation of our method against Spark., This research was financially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 780622; and the Spanish Government (PID2019-107255GB), Generalitat de Catalunya (2014-SGR-1051)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

WFA-FPGA: An efficient accelerator of the wavefront algorithm for short and long read genomics alignment

  • Haghi, Abbas
  • Marco Sola, Santiago
  • Álvarez Martí, Lluc
  • Diamantopoulos, Dionysios
  • Hagleitner, Christoph
  • Moretó Planas, Miquel
In the last years, advances in genome sequencing technologies have enabled the proliferation of genomic applications that guide personalized medicine. These applications have an enormous computational cost due to the large amount of genomic data they process. The first step in many of these applications consists in aligning DNA reads against a reference genome. Very recently, the wavefront alignment (WFA) algorithm has been introduced, significantly reducing the execution time of the read alignment process. This paper presents the first FPGA-based hardware/software co-designed accelerator of such relevant algorithm. Compared to the reference WFA CPU-only implementation, the proposed accelerator achieves performance speedups of up to 13.5 while consuming up to 14.6 less energy when aligning short reads. When aligning long reads, the proposed accelerator achieves speedups of up to 9.9 while consuming up to 10.9 less energy., This work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 (contracts PID2019-107255GB-C21 and TED2021-132634A-I00), by the Generalitat de Catalunya (contract 2021-SGR-00763), by the European Union within the framework of the ERDF of Catalonia 2014-2020 under the DRAC project [001-P-001723], by the European NextGenerationEU/PRTR, by the Lenovo-BSC Contract-Framework (2022), and by the IBM/BSC Deep Learning Center initiative. M. Moreto has been partially supported by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal fellowship No. RYC-2016-21104., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Parallel reduced-order modeling for digital twins using high-performance computing workflows

  • Ares de Parga Regalado, Sebastian
  • Bravo Martínez, José Raúl
  • Sibuet Ruiz, Nicolás
  • Hernández Ortega, Joaquín Alberto
  • Rossi, Riccardo
  • Boschert, Stefan
  • Quintana Ortí, Enrique Salvador
  • Tomás Domínguez, Andrés Enrique
  • Tatu, Cristian Catalin
  • Vázquez Novoa, Fernando
  • Ejarque Artigas, Jorge
  • Badia Sala, Rosa Maria
The integration of reduced-order models with high-performance computing is critical for developing digital twins, particularly for real-time monitoring and predictive maintenance of industrial systems. This paper presents a comprehensive, high-performance computing-enabled workflow for developing and deploying projection-based reduced-order models for large-scale mechanical simulations. We use PyCOMPSs’ parallel framework to efficiently execute reduced-order model training simulations, employing parallel singular value decomposition algorithms such as randomized singular value decomposition, Lanczos singular value decomposition, and full singular value decomposition based on tall-skinny QR. Moreover, we introduce a partitioned version of the hyperreduction scheme known as the Empirical Cubature Method to further enhance computational efficiency in projection-based reduced-order models for mechanical systems. Despite the widespread use of high-performance computing for projection-based reduced-order models, there is a significant lack of publications detailing comprehensive workflows for building and deploying end-to-end projection-based reduced-order models in high-performance computing environments. Our workflow is validated through a case study focusing on the thermal dynamics of a motor, a multiphysics problem involving convective heat transfer and mechanical components. The projection-based reduced-order model is designed to deliver a real-time prognosis tool that could enable rapid and safe motor restarts post-emergency shutdowns under different operating conditions, demonstrating its potential impact on the practice of simulations in engineering mechanics. To facilitate deployment, we use the High-Performance Computing Workflow as a Service strategy and Functional Mock-Up Units to ensure compatibility and ease of integration across high-performance computing, edge, and cloud environments. The outcomes illustrate the efficacy of combining projection-based reduced-order models and high-performance computing, establishing a precedent for scalable, real-time digital twin applications in computational mechanics across multiple industries., The authors acknowledge financial support from the Spanish Ministry of Economy and Competitiveness, through the “Severo Ochoa Programme for Centres of Excellence in R&D” (CEX2018-000797-S and CEX2021-001148-S)”, as well as to the project PID2019-107255GB (BSC authors). This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway. This publication is part of the R&D project PCI2021-121944 and PCI2021-121957, financed by MCIN/AEI/10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR”. Sebastian Ares de Parga Regalado acknowledges financial support from the Generalitat de Catalunya through the FI_SDUR-2021 grant (2021 FISDU 00142), which provided funding for his predoctoral training. BSC authors acknowledge financial support from the Departament de Recerca i Universitats de la Generalitat de Catalunya, research group MPiEDist (2021 SGR 00412). Author Fernando Vázquez is supported by PRE2022-104134 funded by MICIU/AEI /10.13039/501100011033 and by the FSE+., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Main sources of variability and non-determinism in AD software: taxonomy and prospects to handle them

  • Alcón Doganoc, Miguel
  • Brando Guillaumes, Axel
  • Mezzetti, Enrico
  • Abella Ferrer, Jaume
  • Cazorla Almeida, Francisco Javier
Safety standards in domains like automotive and avionics seek for deterministic execution (lack of jittery behavior) as a stepping stone to build a certification argument on the correct timing behavior of the system. However, the use of artificial-intelligence (AI) software in safety-critical systems carries several built-in and derivative sources of non-determinism that are at odds with safety standard determinism requirements. In this work we analyze the main sources of non-determinism of autonomous driving (AD) software, as highly representative and compelling example of the use of AI software, deep neural networks (DNN) in particular, in critical embedded systems. Paradoxically, DNN-based software in its inference phase—once the NN structure and weights have been fixed—turns out to consist mainly in matrix multiplications, which are inherently quite time deterministic. Our work focuses on sources of variability and non-determinism in AD software, covering algorithmic elements of AD software, low-level software and hardware computing platform, and data-flow constraints among AD modules. As final contribution of our work, which mainly focuses on problem identification, we develop some prospects on the information and metrics needed to better understand and control the unpredictability and non-determinism of AD software., This work has been supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GBC21/AEI/10.13039/501100011033, and the European Research Council (ERC) grant agreement No. 772773 (SuPerCom)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Hunting for open clusters in Gaia EDR3: 628 new open clusters found with OCfinder

  • Castro Ginard, Alfred
  • Jordi Nebot, Carme
  • Luri Carrascoso, Xavier
  • Cantat Gaudin, Tristan
  • Carrasco Martínez, Josep Manel
  • Casamiquela Floriach, Laia
  • Anders, Friedrich
  • Balaguer Núñez, Lola
  • Badia Sala, Rosa Maria
Context. The improvements in the precision of the published data in Gaia EDR3 with respect to Gaia DR2, particularly for parallaxes and proper motions, offer the opportunity to increase the number of known open clusters in the Milky Way by detecting farther and fainter objects that have thus far gone unnoticed. Aims. Our aim is to continue to complete the open cluster census in the Milky Way with the detection of new stellar groups in the Galactic disc. We use Gaia EDR3 up to magnitude G = 18 mag, increasing the magnitude limit and therefore the search volume explored in one unit with respect to our previous studies. Methods. We used the OCfinder method to search for new open clusters in Gaia EDR3 using a big data environment. As a first step, OCfinder identified stellar statistical overdensities in five-dimensional astrometric space (position, parallax, and proper motions) using the DBSCAN clustering algorithm. Then, these overdensities were classified into random statistical overdensities or real physical open clusters using a deep artificial neural network trained on well-characterised G, GBP – GRP colour-magnitude diagrams. Results. We report the discovery of 628 new open clusters within the Galactic disc, with most of them being located beyond 1 kpc from the Sun. From the estimation of ages, distances, and line-of-sight extinctions of these open clusters, we see that young clusters align following the Galactic spiral arms while older ones are dispersed in the Galactic disc. Furthermore, we find that most open clusters are located at low Galactic altitudes with the exception of a few groups older than 1 Gyr. Conclusions. We show the success of the OCfinder method leading to the discovery of a total of 1274 open clusters (joining the discoveries here with the previous ones based on Gaia DR2), which represents almost 50% of the known population. Our ability to perform big data searches on a large volume of the Galactic disc, together with the higher precision in Gaia EDR3, enable us to keep completing the census with the discovery of new open clusters., This work was (partially) funded by the Spanish MICIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe” by the “European Union” through grant RTI2018-095076-B-C21, and the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M. A.C.G. acknowledges Spanish Ministry FPI fellowship n. BES-2016-078499. This work has been partially supported by the Spanish Government (PID2019-107255GB), by Generalitat de Catalunya (contract 2014-SGR-1051)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

An automotive case study on the limits of approximation for object detection

  • Caro Roca, Martí
  • Tabani, Hamid
  • Abella Ferrer, Jaume
  • Moll Echeto, Francisco de Borja
  • Morancho Llena, Enrique
  • Canal Corretger, Ramon
  • Altet Sanahujes, Josep
  • Calomarde Palomino, Antonio
  • Cazorla Almeida, Francisco Javier
  • Rubio Romano, Antonio
  • Fontova Muste, Pau
  • Fornt Mas, Jordi
The accuracy of camera-based object detection (CBOD) built upon deep learning is often evaluated against the real objects in frames only. However, such simplistic evaluation ignores the fact that many unimportant objects are small, distant, or background, and hence, their misdetections have less impact than those for closer, larger, and foreground objects in domains such as autonomous driving. Moreover, sporadic misdetections are irrelevant since confidence on detections is typically averaged across consecutive frames, and detection devices (e.g. cameras, LiDARs) are often redundant, thus providing fault tolerance. This paper exploits such intrinsic fault tolerance of the CBOD process, and assesses in an automotive case study to what extent CBOD can tolerate approximation coming from multiple sources such as lower precision arithmetic, approximate arithmetic units, and even random faults due to, for instance, low voltage operation. We show that the accuracy impact of those sources of approximation is within 1% of the baseline even when considering the three approximate domains simultaneously, and hence, multiple sources of approximation can be exploited to build highly efficient accelerators for CBOD in cars., This work is partially funded by the DRAC project, which is co-financed by the European Union Regional Development Fund within the framework of the ERDF Operational Program of Catalonia 2014–2020 with a grant of 50% of total cost eligible. This work has also been partially supported by the Spanish Ministry of Science and Innovation under grant PID2019-107255GB., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Block size estimation for data partitioning in HPC applications using machine learning techniques

  • Cantini, Riccardo
  • Marozzo, Fabrizio
  • Orsino, Alessio
  • Talia, Domenico
  • Trunfio, Paolo
  • Badia Sala, Rosa Maria
  • Ejarque Artigas, Jorge
  • Vázquez-Novoa, Fernando
The extensive use of HPC infrastructures and frameworks for running data-intensive applications has led to a growing interest in data partitioning techniques and strategies. In fact, application performance can be heavily affected by how data are partitioned, which in turn depends on the selected size for data blocks, i.e. the block size. Therefore, finding an effective partitioning, i.e. a suitable block size, is a key strategy to speed-up parallel data-intensive applications and increase scalability. This paper describes a methodology, namely BLEST-ML (BLock size ESTimation through Machine Learning), for block size estimation that relies on supervised machine learning techniques. The proposed methodology was evaluated by designing an implementation tailored to dislib, a distributed computing library highly focused on machine learning algorithms built on top of the PyCOMPSs framework. We assessed the effectiveness of the provided implementation through an extensive experimental evaluation considering different algorithms from dislib, datasets, and infrastructures, including the MareNostrum 4 supercomputer. The results we obtained show the ability of BLEST-ML to efficiently determine a suitable way to split a given dataset, thus providing a proof of its applicability to enable the efficient execution of data-parallel applications in high performance environments., This work has been partially supported by the European Commission through the Horizon 2020 Research and Innovation program and the EuroHPC JU under contract 955558 (eFlows4HPC project) and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021‑121957 and CEX2021‑ 001148‑S) and by the Spanish Government (PID2019‑107255GB), and Generalitat de Catalunya (contract 2021‑SGR‑00412). We also acknowledge financial support from “National Centre for HPC, Big Data and Quantum Computing”, CN00000013 ‑ CUP H23C22000360005, and from “FAIR ‑ Future Artificial Intelligence Research” project ‑ CUP H23C22000860006., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Automatizing the creation of specialized high-performance computing containers

  • Ejarque Artigas, Jorge
  • Badia Sala, Rosa Maria
With Exascale computing already here, supercomputers are systems every time larger, more complex, and heterogeneous. While expert system administrators can install and deploy applications in the systems correctly, this is something that general users can not usually do. The eFlows4HPC project aims to provide methodologies and tools to enable the use and reuse of application workflows. One of the aspects that the project focuses on is simplifying the application deployment in large and complex systems. The approach uses containers, not generic ones, but containers tailored for each target High-Performance Computing (HPC) system. This paper presents the Container Image Creation service developed in the framework of the project and experimentation based on project applications. We compare the performance of the specialized containers against generic containers and against a native installation. The results show that in almost all cases, the specialized containers outperform the generic ones (up to 2× faster), and in all cases, the performance is the same as with the native installation., This work has been supported by the Spanish Government (PID2019-107255GB) and by MCIN/AEI /10.13039/501100011033 (CEX2021-001148-S), by Generalitat de Catalunya (contract 2021-SGR-00412), and by the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (project eFlows4HPC)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Porting and optimizing BWA-MEM2 using the Fujitsu A64FX processor

  • Langarita Benítez, Rubén
  • Armejach Sanosa, Adrià
  • Ibáñez Marín, Pablo
  • Alastruey Benedé, Jesús
  • Moretó Planas, Miquel
Sequence alignment pipelines for human genomes are an emerging workload that will dominate in the precision medicine field. BWA-MEM2 is a tool widely used in the scientific community to perform read mapping studies. In this paper, we port BWA-MEM2 to the AArch64 architecture using the ARMv8-A specification, and we compare the resulting version against an Intel Skylake system both in performance and in energy-to-solution. The porting effort entails numerous code modifications, since BWA-MEM2 implements certain kernels using x86_64 specific intrinsics, e.g., AVX-512. To adapt this code we use the recently introduced Arm's Scalable Vector Extensions (SVE). More specifically, we use Fujitsu's A64FX processor, the first to implement SVE. The A64FX powers the Fugaku Supercomputer that led the Top500 ranking from June 2020 to November 2021. After porting BWA-MEM2 we define and implement a number of optimizations to improve performance in the A64FX target architecture. We show that while the A64FX performance is lower than that of the Skylake system, A64FX delivers 11.6% better energy-to-solution on average. All the code used for this article is available at https://gitlab.bsc.es/rlangari/bwa-a64fx, This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (contracts PID2019-107255GB-C21 / AEI /10.13039/501100011033 and PID2019-105660RB-C21 / AEI / 10.13039/501100011033), Gobierno de Aragon (T5820R research group), the Generalitat de Catalunya (contracts 2017-SGR-1328 and 2017-SGR1414), and the European Union’s Horizon 2020 research and innovation program (Mont-Blanc 2020 project, grant agreement 779877). Finally, A. Armejach and M. Moreto have been partially supported by the Spanish Ministry of Economy, Industry and Competitiveness under Juan de la Cierva fellowship no. IJCI-2017-33945 and Ramon y Cajal fellowship no. RYC-2016-21104, respectively., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Hardware acceleration for high-volume operations of CRYSTALS-Kyber and CRYSTALS-Dilithium

  • Carril Gil, Xavier
  • Kardaris, Charalampos
  • Ribes González, Jordi
  • Farràs Ventura, Oriol
  • Hernández Luz, Carles
  • Kostalabros, Vatistas
  • González Jiménez, Joel Ulises
  • Moretó Planas, Miquel
Many high-demand digital services need to perform several cryptographic operations, such as key exchange or security credentialing, in a concise amount of time. In turn, the security of some of these cryptographic schemes is threatened by advances in quantum computing, as quantum computer could break their security in the near future. Post-Quantum Cryptography (PQC) is an emerging field that studies cryptographic algorithms that resist such attacks. The National Institute of Standards and Technology (NIST) has selected the CRYSTALS-Kyber Key Encapsulation Mechanism and the CRYSTALSDilithium Digital Signature algorithm as primary PQC standards. In this paper, we present FPGA-based hardware accelerators for high-volume operations of both schemes. We apply High-Level Synthesis (HLS) for hardware optimization, leveraging a batch processing approach to maximize the memory throughput, and applying custom HLS logic to specific algorithmic components. Using reconfigurable field-programmable gate arrays (FPGAs), we show that our hardware accelerators achieve speedups between 3x and 9x over software baseline implementations, even over ones leveraging CPU vector architectures. Furthermore, the methods used in this study can also be extended to the new CRYSTALS-based NIST FIPS drafts, ML-KEM and ML-DSA, with similar acceleration results., This work was part of the project (ISOLDE), funded by MICIU/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR under Grant No. PCI2023-143372, and the European Union’s Horizon Europe Programme under project Chips Joint Undertaking (JU) under Grant No. 101112274. At the same time, this work has been partially supported by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation MCIN/AEI/ 10.13039/501100011033 (contract PID2019-107255GB-C21), and by the Government of Catalonia (contract 2021-SGR-763). X. Carril has the support of the predoctoral programme AGAUR-FI grants (2024 FI-1 00520) Joan Oró from the Secretariat for Universities and Research of the Department of Research and Universities of the Generalitat de Catalunya and the European Social Fund Plus. J. Ribes-González and O. Farràs are supported by grant 2021-SGR-00115 from the Government of Catalonia, by the project HERMES, funded by INCIBE and by the European Union NextGeneration EU/PRTR, and the project ACITHEC PID2021-124928NB-I00, funded by MCIN/AEI/10.13039/501100011033. C. Hernandez was supported by “Ramón y Cajal” fellowship No. RYC2020-030685-I. V. Kostalabros was supported by the Agency for Management of University and Research Grants (AGAUR) of the Government of Catalonia under “Ajuts per a la contractació de personal investigador novell” fellowship No. 2019FI B01274., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

XeroZerox: Analysis and Optimization of GPU Memory Management for High-Integrity Autonomous Systems

  • Calderón, Alejandro J.
  • Kosmidis, Leonidas
  • Nicolás, Carlos F.
  • Cazorla Almeida, Francisco Javier
Autonomous systems require high-performance processing capabilities, which demand the use of powerful accelerators such as GPUs. However, the use of GPUs in critical systems presents several challenges, since GPU programming models rely on explicit dynamic memory management. Traditionally, dynamic memory allocation in such systems is restricted to certain controlled scenarios, which require programs to be rewritten so that all the required memory is allocated at the beginning of the program and released at its end. However, many GPU applications do not follow this approach. Moreover, when dynamic memory allocation is used, it is critical to compute the exact amount of memory that will be used as well as to minimize it, to guarantee that it fits in the physical system memory. In this paper we present XeroZerox, an open-source tool that automatically converts the traditional dynamic memory allocations of GPU applications into allocations served from a centralized and optimally sized memory pool, which is managed in a way better suited for critical systems. XeroZerox allows legacy GPU applications to be used in a critical setup without rewriting them, minimizing at the same time their memory consumption and memory management runtime overhead., Spanish Ministry of Economy and Competitiveness (Grant Number: IJC-2020-045931-I and PID2019-107255GB-C21) Department of Research and Universities of the Government of Catalonia (Grant Number: 2021 SGR 00637) European Union Horizon Europe Programme (Grant Number: 101082622), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Dynamic resource allocation for efficient parallel CFD simulations

  • Houzeaux, Guillaume
  • Badia Sala, Rosa Maria
  • Borrell Pol, Ricard
  • Dosimont, Damien
  • Ejarque Artigas, Jorge
  • Garcia Gasulla, Marta
  • López Herrero, Víctor
CFD users of supercomputers usually resort to rule-of-thumb methods to select the number of subdomains (partitions) when relying on MPI-based parallelization. One common approach is to set a minimum number of elements or cells per subdomain, under which the parallel efficiency of the code is “known” to fall below a subjective level, say 80%. The situation is even worse when the user is not aware of the “good” practices for the given code and a huge amount of resources can thus be wasted. This work presents an elastic computing methodology that adapts at runtime the resources allocated to a simulation automatically. The criterion to control the required resources is based on a runtime measure of the communication efficiency of the execution. According to some analytical estimates, the resources are then expanded or reduced to fulfill this criterion and eventually execute an efficient simulation., This work has been supported by the Spanish Government (PID2019- 107255GB); by Generalitat de Catalunya (contract 2014-SGR-1051); by the European Commission H2020 project POP CoE (GA n. 824080); by the European Commission H2020 project CompBioMed CoE (GA n. 823712) and by the European Commission and the EuroHPC JU under contract 955558 (eFlows4HPC project)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Dynamic load balance of chemical source term evaluation in high-fidelity combustion simulations

  • Ramirez Miranda, Guillem
  • Mira Martínez, Daniel
  • Pérez Sánchez, Eduardo Javier
  • Surapaneni, Anurag
  • Borrell Pol, Ricard
  • Houzeaux, Guillaume
  • Garcia Gasulla, Marta
This paper presents a load balancing strategy for reaction rate evaluation and chemistry integration in reacting flow simulations. The large disparity in scales during combustion introduces stiffness in the numerical integration of the PDEs and generates load imbalance during the parallel execution. The strategy is based on the use of the DLB library to redistribute the computing resources at node level, lending additional CPU-cores to higher loaded MPI processes. This approach does not require explicit data transfer and is activated automatically at runtime. Two chemistry descriptions, detailed and reduced, are evaluated on two different configurations: laminar counterflow flame and a turbulent swirl-stabilized flame. For single-node calculations, speedups of 2.3x and 7x are obtained for the detailed and reduced chemistry, respectively. Results on multi-node runs also show that DLB improves the performance of the pure-MPI code similar to single node runs. It is shown DLB can get performance improvements in both detailed and reduced chemistry calculations., This work has been supported by the Spanish Government (PID2019-107255GB), by Generalitat de Catalunya (contract 2014-SGR-1051), and by the European POP CoE (GA No 824080) and EXCELLERAT project GA No 823691., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Extreme-scale workflows: a perspective from the JLESC international community

  • Yildiz, Orcun
  • Gueroudji, Amal
  • Bigot, Julien
  • Raffin, Bruno
  • Badia Sala, Rosa Maria
  • Peterka, Tom
The Joint Laboratory for Extreme-Scale Computing (JLESC) focuses on software challenges in high-performance computing systems to meet the needs of today’s science campaigns, which often require large resources, consist of multiple tasks, and generate vast amounts of data. In this context, extreme-scale workflows have been the key factor in enabling scientific discoveries by helping scientists automate the dependencies and data exchanges between workflow tasks, instead of managing those manually. In this paper we present representative extreme-scale workflows and feature workflow systems developed by JLESC participating institutions. We present lessons learned while developing these tools, alongside with the open challenges and future research directions in the field of extreme-scale workflows., This research is supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, under contract number DE-AC02- 06CH11357, program manager Margaret Lentz. The BSC author acknowledges the support of the Spanish Government (PID2019-107255GB) and by MCIN/AEI /10.13039 /501100011033 (CEX2021-001148-S), by the Departament de Recerca i Universitats de la Generalitat de Catalunya to the Research Group MPiEDist (2021 SGR 00412). The same author acknowledges the European Commission’s Horizon 2020 Framework program and the European High-Performance Computing Joint Undertaking (JU) under grant agreement No. 955558 and by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021-121957), project eFlows4HPC. The INRIA author acknowledges the European Union’s Horizon research and innovation programme under grant agreement No 101144014., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Towards automatic model specialization for edge video analytics

  • Rivas Barragan, Daniel
  • Guim Bernat, Francesc
  • Polo Bardés, Jorda
  • Silva, Pubudu Madhawa
  • Berral García, Josep Lluís
  • Carrera Pérez, David
The number of cameras deployed to the edge of the network increases by the day, while emerging use cases, such as smart cities or autonomous driving, also grow to expect images to be analyzed in real-time by increasingly accurate and complex neural networks. Unfortunately, state-of-the-art accuracy comes at a computational cost rarely available in the edge cloud. At the same time, due to strict latency constraints and the vast amount of bandwidth edge cameras generate, we can no longer rely on offloading the task to a centralized cloud. Consequently, there is a need for a meeting point between the resource-constrained edge cloud and accurate real-time video analytics. If state-of-the-art models are too expensive to run on the edge, and lightweight models are not accurate enough for the use cases in the edge, one solution is to demand less from the lightweight model and specialize it in a narrower scope of the problem, a technique known as model specialization. By specializing a model to the context of a single camera, we can boost its accuracy while keeping its computational cost constant. However, this also involves one training per camera, which quickly becomes unfeasible unless the entire process is fully automated. In this paper, we present and evaluate COVA (Contextually Optimized Video Analytics), a framework to assist in the automatic specialization of models for video analytics in edge cloud cameras. COVA aims to automatically improve the accuracy of lightweight models by specializing them to the context to which they will be deployed. Moreover, we discuss and analyze each step involved in the process to understand the different trade-offs that each one entails. Using COVA, we demonstrate that the whole pipeline can be effectively automated by leveraging large neural networks used as teachers whose predictions are used to train and specialize lightweight neural networks. Results show that COVA can automatically improve pre-trained models by an average of 21% mAP on the different scenes of the VIRAT dataset., This work has been partially supported by the Spanish Government (contract PID2019-107255GB) and by Generalitat de Catalunya, Spain (contract 2014-SGR-1051)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Storage-heterogeneity aware task-based programming models to optimize I/O intensive applications

  • Elshazly, Hatem Mohamed Abdelfattah Eid
  • Ejarque Artigas, Jorge
  • Badia Sala, Rosa Maria
Task-based programming models have enabled the optimized execution of the computation workloads of applications. These programming models can take advantage of large-scale distributed infrastructures by allowing the parallel and distributed execution of applications in high-level work components called tasks. Nevertheless, in the era of Big Data and Exascale, the amount of data produced by modern scientific applications has already surpassed terabytes and is rapidly increasing. Hence, I/O performance became the bottleneck to overcome in order to achieve more total performance improvement. New storage technologies offer higher bandwidth and faster solutions than traditional Parallel File Systems (PFS). Such storage devices are deployed in modern day infrastructures to boost I/O performance by offering a fast layer that absorbs the generated data. Therefore, it is necessary for any programming model targeting more performance to manage this heterogeneity and take advantage of it to improve the I/O performance of applications. Towards this goal, we propose in this paper a set of programming model capabilities that we refer to as Storage-Heterogeneity Awareness. Such capabilities include: (i) abstracting the heterogeneity of storage systems, and (ii) optimizing I/O performance by supporting dedicated I/O schedulers and an automatic data flushing technique. The evaluation section of this paper presents the performance results of different applications on the MareNostrum CTE-Power heterogeneous storage cluster. Our experiments demonstrate that a storage-heterogeneity aware programming model can achieve up to almost 5x I/O performance speedup and 48% total time improvement compared to the reference PFS-based usage of the execution infrastructure., This work is partially supported by the European Union through the Horizon 2020 research and innovation programme under contracts 721865 (EXPERTISE Project) by the Spanish Government (PID2019-107255GB) and the Generalitat de Catalunya (contract 2014-SGR-1051)., Peer Reviewed, Postprint (author's final draft)
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Optimal gap-affine alignment in O (s) space

  • Marco-Sola, Santiago|||0000-0001-7951-3914
  • Eizenga, Jordan|||0000-0001-8345-8356
  • Guarracino, Andrea|||0000-0001-9744-131X
  • Paten, Benedict|||0000-0001-8863-3539
  • Garrison, Erik|||0000-0003-3821-631X
  • Moreto, Miquel|||0000-0002-9848-8758
Pairwise sequence alignment remains a fundamental problem in computational biology and bioinformatics. Recent advances in genomics and sequencing technologies demand faster and scalable algorithms that can cope with the ever-increasing sequence lengths. Classical pairwise alignment algorithms based on dynamic programming are strongly limited by quadratic requirements in time and memory. The recently proposed wavefront alignment algorithm (WFA) introduced an efficient algorithm to perform exact gap-affine alignment in time, where s is the optimal score and n is the sequence length. Notwithstanding these bounds, WFA's memory requirements become computationally impractical for genome-scale alignments, leading to a need for further improvement. In this article, we present the bidirectional WFA algorithm, the first gap-affine algorithm capable of computing optimal alignments in memory while retaining WFA's time complexity of . As a result, this work improves the lowest known memory bound to compute gap-affine alignments. In practice, our implementation never requires more than a few hundred MBs aligning noisy Oxford Nanopore Technologies reads up to 1 Mbp long while maintaining competitive execution times. All code is publicly available at . Supplementary data are available at Bioinformatics online.