Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 2
Encontrada(s) 1 página(s)

Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol
  • Agulló López, Ferran
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Towards generalizable Federated Learning in medical imaging: a real-world case study on mammography data

  • Tzortzis, Ioannis N.
  • Gutiérrez Torre, Alberto
  • Sykiotis, Stavros
  • Agulló López, Ferran
  • Bakalos, Nikolaos
  • Doulamis, Anastasios
  • Doulamis, Nikolaos
  • Berral García, Josep Lluís
Federated Learning has been rapidly gaining in popularity in medical applications, due to the increased privacy offered, since medical data doesn't need to leave the hospitals' premises for AI model training. However, a direct translation of a classic experiment to a federated one is not always straightforward. In this work, we delve into the intricacies of federated learning for a breast cancer classification tool. We compare classic model training with a federated variant, and highlight the adaptations that need to be taken care of to ensure the equivalence between the two. Specifically, we introduce the Breast Area Detection tool as an essential component of the pre-processing pipeline to enhance the robustness of Federated Learning by offering data harmonization. On top of that, we present an end-to-end Federated Learning framework that is effective for real-world data and scenarios. Among the three real-world hospitals involved in the experimental procedure, the proposed framework significantly improves performance at the first hospital, providing consistent results similar to those achieved in the classic approach. Experimental results demonstrate that the interventions introduced improved model performance by approximately 35%, aligning federated learning and centralized model performance., This paper is funded by the EU funded projects SECURED “Scaling Up Secure Processing, Anonymization And Generation Of Health Data For EU Cross Border Collaborative Research And Innovation” with grant agreement no. 101095717 and INCISIVE “A multimodal AI-based toolbox and an interoperable health imaging repository for the empowerment of imaging analysis related to the diagnosis, prediction and follow-up of cancer” with grant agreement no. 952179. This paper has been partially financed by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, 2021- SGR-00478, UE, also Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478., Peer Reviewed, Postprint (published version)