Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Agulló López, Ferran|||0000-0002-7276-2472
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Beyond parameter averaging in model aggregation

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Hauberg, Søren
  • Moreno Muñoz, Pablo
The success of foundation models is strongly linked to scale, which has reinforced the interest in federated learning. With the prohibitive cost of training a large language model (LLM) in mind, little attention has been placed on reusing pre-trained models in collaborative training settings. Self-supervision has also played an important role in this success, but its emphasis has been primarily on data. This paper leverages Bayesian principles to bring self-supervision into the model aggregation toolbox. It introduces self-supervised Fisher merging, a framework that successfully merges models in parameter space without re-visiting data, opening a new door in model reusability. Experimental results build the foundation of our method on tractable linear models, and highlight its potential on aggregating neural networks., This work was supported by a research grant (42062) from VILLUM FONDEN. This project has also received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 757360). Our work was funded in part by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Sciences (MLSS) with reference number NNF20OC0062606. This work has been partially financed by the EU-H2020 programme under grant agreement EU-H2020 GA.952179 and grant agreement EU-HORIZON GA.101095717. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

SECURED for health: scaling up privacy to enable the integration of the european health data space

  • Regazzoni, Francesco
  • Acs, Gergely
  • Aszalos, Albert Zoltan
  • Avgerinos, Christos
  • Bakalos, Nikolaos
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Bos, Joppe W.
  • Brohet, Marco
  • Castillo Sanz, Andrés G.
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
In this paper, we present the SECURED project funded in part by the European Union (EU), Grant Agreement no. 10109571. Views and opinions expressed are those of the authors and do not necessarily reflect those of the EU or the Health and Digital Executive Agency. Neither the EU nor the granting authority are responsible for them., aimed at improving privacy-preserving processing of data in the health domain. The technologies developed in the project will be demonstrated in four health-related use cases and with the involvement of SME's selected through an open funding call., Funded in part by the European Union (EU), Grant Agreement no. 10109571., Peer Reviewed, Article signat per 41 autors/es: Francesco Regazzoni ∗ xviii , Gergely Acs xii , Albert Zoltan Aszalos x, Christos Avgerinos xv , Nikolaos Bakalos xvi , Josep Ll. Berral ††, Joppe W. Bos §, Marco Brohet *, Andres G. Castillo Sanz ‡, Gareth T. Davies §, Stefanos Florescu ∥, Pierre-Elisee Flory ¶, Alberto Gutierrez-Torre ††, Evangelos Haleplidis xi , Alice Heliou ¶, Sotirios Ioannidis †, Alexander Islam El-Kadyxi, 14 , Katarzyna Kapusta ¶, Konstantina Karagianni xi , Pieter Kruizinga ∥, Kyrian Maat ∗, Zoltan Adam Mann ∗, Kalliopi Mastoraki †, SeoJeong Moon §, Maja Nisevic ∗∗, Balazs Pejo xii , Kostas Papagiannopoulos ∗, Vassilis Paliuras xi xix , Paolo Palmieri xiii , Francesca Palumbo ‡‡, Juan Carlos Perez Baun xvii, Peter Pollner x, Eduard Porta-Pardo xiv , Luca Pulina ‡‡, Muhammad Ali Siddiqi ∥, Daniela Spajic ∗∗, Christos Strydis ∥, Georgios Tasopoulos ∗, Vincent Thouvenot ¶, Christos Tselios xi , Apostolos P. Fournaris xi / ∗ University of Amsterdam (NL), † Circular Economy Foundation (BE), ‡ FHUNJ (ES), § NXP (BE), ¶ Thales (FR), ∥ Erasmus Medical Center (NL), ∗∗ KU Leuven (BE), †† Barcelona Supercomputing Center (ES), ‡‡ University of Sassari (IT), x Semmelweis University (HU), xi Industrial Systems Institute/R.C. ATHENA (GR), xii BME (HU), xiii University College Cork (IE), xv Catalink (CY), xvi ICCS (GR), xvii EVIDEN (ES), xiv Josep Carreras Leukaemia Research Institute (ES), xviii Universita della Svizzera italiana (CH), xix University of Patras (GR)
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

HealthMesh: An architectural framework for federated healthcare data management

  • Bisquert Parés, Aniol
  • Hmimou Ham Man, Achraf|||0009-0004-0448-3406
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Romero Moral, Óscar|||0000-0001-6350-8328
Recently, significant milestones have been achieved in the field of healthcare data analysis. However, alongside these accomplishments, substantial data-related challenges have emerged in the domain of big data management. Modern healthcare projects are no more dealing with a single data repository but many heterogeneous ones and must overcome data variety, privacy and governance issues. Yet, current solutions face a privacy-decentralization trade-off. To address this dual challenge, we introduce HealthMesh, a novel layered architectural framework based on the Data Mesh principles, providing a domaindecentralised paradigm. In addition, the framework incorporates a Semantic Data Model which establishes robust governance, enables interoperability and guarantees policy compliance for all the data assets. To demonstrate the capabilities of the proposed approach, we provide an illustrative example inspired by the use case of the INCISIVE project for breast cancer analytics. Overall, this work makes a significant contribution on collecting key challenges, identifying actors and providing a set of components and guidelines for establishing a holistic framework for the complex field of healthcare data management., The work reported here was supported by the EU-H2020 programme under GA.952179 INCISIVE, Horizon Europe Programme under GA.101095717 (SECURED) and GA.101135513 (CYCLOPS), the Spanish Research State Agency (MICINN/AEI, ERDF/FEDER) under GA.MCIN/AEI/10.13039/501100011033/FEDER (UE DALEST), the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/501100011033 (DOGO4ML), and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478 (CROMAI)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Towards Pareto optimal throughput in small language model serving

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Zhu, Yue
  • Wang, Chen
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models (SLMs) offers new opportunities for resource-constrained users, who now are able to serve small models with cutting-edge performance. In this paper, we present a set of experiments designed to benchmark SLM inference at performance and energy levels. Our analysis provides a new perspective in serving, highlighting that the small memory footprint of SLMs allows for reaching the Pareto-optimal throughput within the resource capacity of a single accelerator. In this regard, we present an initial set of findings demonstrating how model replication can effectively improve resource utilization for serving SLMs., This work has been partially financed by grant agreement EUHORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021- SGR-00478, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/ 10.13039/ 501100011033/ FEDER, UE., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

SECURED for health: scaling up privacy to enable the integration of the european health data space

  • Regazzoni, Francesco
  • Acs, Gergely
  • Aszalos, Albert Zoltan
  • Avgerinos, Christos
  • Bakalos, Nikolaos
  • Berral García, Josep Lluís
  • Bos, Joppe W.
  • Brohet, Marco
  • Castillo Sanz, Andrés G.
  • Gutiérrez Torre, Alberto
In this paper, we present the SECURED project funded in part by the European Union (EU), Grant Agreement no. 10109571. Views and opinions expressed are those of the authors and do not necessarily reflect those of the EU or the Health and Digital Executive Agency. Neither the EU nor the granting authority are responsible for them., aimed at improving privacy-preserving processing of data in the health domain. The technologies developed in the project will be demonstrated in four health-related use cases and with the involvement of SME's selected through an open funding call., Funded in part by the European Union (EU), Grant Agreement no. 10109571., Peer Reviewed, Article signat per 41 autors/es: Francesco Regazzoni ∗ xviii , Gergely Acs xii , Albert Zoltan Aszalos x, Christos Avgerinos xv , Nikolaos Bakalos xvi , Josep Ll. Berral ††, Joppe W. Bos §, Marco Brohet *, Andres G. Castillo Sanz ‡, Gareth T. Davies §, Stefanos Florescu ∥, Pierre-Elisee Flory ¶, Alberto Gutierrez-Torre ††, Evangelos Haleplidis xi , Alice Heliou ¶, Sotirios Ioannidis †, Alexander Islam El-Kadyxi, 14 , Katarzyna Kapusta ¶, Konstantina Karagianni xi , Pieter Kruizinga ∥, Kyrian Maat ∗, Zoltan Adam Mann ∗, Kalliopi Mastoraki †, SeoJeong Moon §, Maja Nisevic ∗∗, Balazs Pejo xii , Kostas Papagiannopoulos ∗, Vassilis Paliuras xi xix , Paolo Palmieri xiii , Francesca Palumbo ‡‡, Juan Carlos Perez Baun xvii, Peter Pollner x, Eduard Porta-Pardo xiv , Luca Pulina ‡‡, Muhammad Ali Siddiqi ∥, Daniela Spajic ∗∗, Christos Strydis ∥, Georgios Tasopoulos ∗, Vincent Thouvenot ¶, Christos Tselios xi , Apostolos P. Fournaris xi / ∗ University of Amsterdam (NL), † Circular Economy Foundation (BE), ‡ FHUNJ (ES), § NXP (BE), ¶ Thales (FR), ∥ Erasmus Medical Center (NL), ∗∗ KU Leuven (BE), †† Barcelona Supercomputing Center (ES), ‡‡ University of Sassari (IT), x Semmelweis University (HU), xi Industrial Systems Institute/R.C. ATHENA (GR), xii BME (HU), xiii University College Cork (IE), xv Catalink (CY), xvi ICCS (GR), xvii EVIDEN (ES), xiv Josep Carreras Leukaemia Research Institute (ES), xviii Universita della Svizzera italiana (CH), xix University of Patras (GR), Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Beyond parameter averaging in model aggregation

  • Garcia Recasens, Pol
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
  • Hauberg, Søren
  • Moreno Muñoz, Pablo
The success of foundation models is strongly linked to scale, which has reinforced the interest in federated learning. With the prohibitive cost of training a large language model (LLM) in mind, little attention has been placed on reusing pre-trained models in collaborative training settings. Self-supervision has also played an important role in this success, but its emphasis has been primarily on data. This paper leverages Bayesian principles to bring self-supervision into the model aggregation toolbox. It introduces self-supervised Fisher merging, a framework that successfully merges models in parameter space without re-visiting data, opening a new door in model reusability. Experimental results build the foundation of our method on tractable linear models, and highlight its potential on aggregating neural networks., This work was supported by a research grant (42062) from VILLUM FONDEN. This project has also received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement 757360). Our work was funded in part by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Sciences (MLSS) with reference number NNF20OC0062606. This work has been partially financed by the EU-H2020 programme under grant agreement EU-H2020 GA.952179 and grant agreement EU-HORIZON GA.101095717. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards Pareto optimal throughput in small language model serving

  • Garcia Recasens, Pol
  • Zhu, Yue
  • Wang, Chen
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models (SLMs) offers new opportunities for resource-constrained users, who now are able to serve small models with cutting-edge performance. In this paper, we present a set of experiments designed to benchmark SLM inference at performance and energy levels. Our analysis provides a new perspective in serving, highlighting that the small memory footprint of SLMs allows for reaching the Pareto-optimal throughput within the resource capacity of a single accelerator. In this regard, we present an initial set of findings demonstrating how model replication can effectively improve resource utilization for serving SLMs., This work has been partially financed by grant agreement EUHORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021- SGR-00478, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/ 10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol
  • Agulló López, Ferran
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

HealthMesh: An architectural framework for federated healthcare data management

  • Bisquert Parés, Aniol
  • Hmimou Ham Man, Achraf
  • Berral García, Josep Lluís
  • Gutiérrez Torre, Alberto
  • Romero Moral, Óscar
Recently, significant milestones have been achieved in the field of healthcare data analysis. However, alongside these accomplishments, substantial data-related challenges have emerged in the domain of big data management. Modern healthcare projects are no more dealing with a single data repository but many heterogeneous ones and must overcome data variety, privacy and governance issues. Yet, current solutions face a privacy-decentralization trade-off. To address this dual challenge, we introduce HealthMesh, a novel layered architectural framework based on the Data Mesh principles, providing a domaindecentralised paradigm. In addition, the framework incorporates a Semantic Data Model which establishes robust governance, enables interoperability and guarantees policy compliance for all the data assets. To demonstrate the capabilities of the proposed approach, we provide an illustrative example inspired by the use case of the INCISIVE project for breast cancer analytics. Overall, this work makes a significant contribution on collecting key challenges, identifying actors and providing a set of components and guidelines for establishing a holistic framework for the complex field of healthcare data management., The work reported here was supported by the EU-H2020 programme under GA.952179 INCISIVE, Horizon Europe Programme under GA.101095717 (SECURED) and GA.101135513 (CYCLOPS), the Spanish Research State Agency (MICINN/AEI, ERDF/FEDER) under GA.MCIN/AEI/10.13039/501100011033/FEDER (UE DALEST), the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/501100011033 (DOGO4ML), and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478 (CROMAI)., Peer Reviewed, Postprint (published version)