Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 4
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Data-driven optimization of GPU efficiency for distributed LLM-adapter serving

  • Agulló López, Ferran|||0000-0002-7276-2472
  • Oliveras Torra, Joan
  • Wang, Chen
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently. While prior work has largely focused on latency and throughput optimization, minimizing GPU resource requirements through near-peak utilization remains largely underexplored. This paper presents a data-driven pipeline that, for a given workload, computes an adapter placement that serves the workload with the minimum number of GPUs while avoiding request starvation and GPU memory errors. To that end, the approach identifies the maximum feasible throughput attainable on each GPU by leveraging accurate performance predictions learned from real serving behavior. The proposed pipeline integrates three components: (i) a Digital Twin (DT) tailored to LLM-adapter serving, (ii) a distilled machine learning (ML) model trained on DT-generated data, and (iii) a greedy placement algorithm that exploits ML-based performance estimates to maximize GPU efficiency. The DT emulates real system dynamics with high fidelity, achieving below 5% throughput estimation error while executing up to 90×faster than full LLM benchmarking across both predictable and unpredictable workloads. The learned ML models further accelerate performance estimation with marginal accuracy degradation, enabling scalable optimization. Experimental results demonstrate that the pipeline substantially improves GPU efficiency, reducing the number of GPUs required to sustain target workloads by 60% on average across the evaluated scenarios. Beyond GPU efficiency, the pipeline can be adapted to alternative objectives, such as latency minimization, highlighting its versatility for future large-scale LLM serving infrastructures., This work has been partially financed by the EU-HORIZON MSCA programme under grant agreement GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2024-160996OB-I00, MICIU/AEI/10.13039/501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Enhancing the output of time series forecasting algorithms for cloud resource provisioning

  • Agulló López, Ferran|||0000-0002-7276-2472
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Forecasting the resource consumption of workloads is a frequent approach in the cloud provisioning field. Ideally, such predictions allow obtaining a more accurate scheduling and management of resources in a computing cluster. However, the current approaches fail to properly forecast the future consumption in areas where sudden increases of consumption are present, i.e., spikes. Even, commonly employed metrics lack the ability to properly evaluate sharp behaviours in the traces. This may generate resource starvation problems in the running workloads and decreases the Quality of Service (QoS) provided to external users. To address this issue, we propose two strategies that modify the outputs of forecasting algorithms without changing the algorithms’ internals. The new outputs considerably enhance the prediction of sudden increases, duplicating the F1 score metric in average for all tested algorithms. This improvement in the handling of spikes comes with an increased over-provision of resources. Nevertheless, the proposed strategies give the user an easy way to control this trade-off between predicting spikes and the amount of over-provision. The user can decide which is the right balance that better fits the requirements of its specific scenario. Furthermore, we propose a new evaluation methodology that better assesses the behaviour of forecasting algorithms in cloud traces, especially focused on the performance around increases of consumption, and we give insights on the reasons behind the predictions of the algorithms with the application of explainability techniques. The code repository of this work can be accessed through GitHub at this link https://github.com/FerranAgulloLopez/ResourceForecasting., This work has been supported by the EU’s Horizon research and innovation programme under grant agreements HORIZON GA.101092646 (CloudSkin), HORIZON MSCA GA.101086248 (CloudStars), and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID-2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033, UE (DALEST), Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3 and the Generalitat de Catalunya (AGAUR), Spain 2021-SGR-00478 (CROMAI)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Data-driven optimization of GPU efficiency for distributed LLM-adapter serving

  • Agulló López, Ferran
  • Oliveras Torra, Joan
  • Wang, Chen
  • Gutiérrez Torre, Alberto
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
Large Language Model (LLM) adapters enable low-cost model specialization, but introduce complex caching and scheduling challenges in distributed serving systems where hundreds of adapters must be hosted concurrently. While prior work has largely focused on latency and throughput optimization, minimizing GPU resource requirements through near-peak utilization remains largely underexplored. This paper presents a data-driven pipeline that, for a given workload, computes an adapter placement that serves the workload with the minimum number of GPUs while avoiding request starvation and GPU memory errors. To that end, the approach identifies the maximum feasible throughput attainable on each GPU by leveraging accurate performance predictions learned from real serving behavior. The proposed pipeline integrates three components: (i) a Digital Twin (DT) tailored to LLM-adapter serving, (ii) a distilled machine learning (ML) model trained on DT-generated data, and (iii) a greedy placement algorithm that exploits ML-based performance estimates to maximize GPU efficiency. The DT emulates real system dynamics with high fidelity, achieving below 5% throughput estimation error while executing up to 90×faster than full LLM benchmarking across both predictable and unpredictable workloads. The learned ML models further accelerate performance estimation with marginal accuracy degradation, enabling scalable optimization. Experimental results demonstrate that the pipeline substantially improves GPU efficiency, reducing the number of GPUs required to sustain target workloads by 60% on average across the evaluated scenarios. Beyond GPU efficiency, the pipeline can be adapted to alternative objectives, such as latency minimization, highlighting its versatility for future large-scale LLM serving infrastructures., This work has been partially financed by the EU-HORIZON MSCA programme under grant agreement GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2024-160996OB-I00, MICIU/AEI/10.13039/501100011033/FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Enhancing the output of time series forecasting algorithms for cloud resource provisioning

  • Agulló López, Ferran
  • Gutiérrez Torre, Alberto
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
Forecasting the resource consumption of workloads is a frequent approach in the cloud provisioning field. Ideally, such predictions allow obtaining a more accurate scheduling and management of resources in a computing cluster. However, the current approaches fail to properly forecast the future consumption in areas where sudden increases of consumption are present, i.e., spikes. Even, commonly employed metrics lack the ability to properly evaluate sharp behaviours in the traces. This may generate resource starvation problems in the running workloads and decreases the Quality of Service (QoS) provided to external users. To address this issue, we propose two strategies that modify the outputs of forecasting algorithms without changing the algorithms’ internals. The new outputs considerably enhance the prediction of sudden increases, duplicating the F1 score metric in average for all tested algorithms. This improvement in the handling of spikes comes with an increased over-provision of resources. Nevertheless, the proposed strategies give the user an easy way to control this trade-off between predicting spikes and the amount of over-provision. The user can decide which is the right balance that better fits the requirements of its specific scenario. Furthermore, we propose a new evaluation methodology that better assesses the behaviour of forecasting algorithms in cloud traces, especially focused on the performance around increases of consumption, and we give insights on the reasons behind the predictions of the algorithms with the application of explainability techniques. The code repository of this work can be accessed through GitHub at this link https://github.com/FerranAgulloLopez/ResourceForecasting., This work has been supported by the EU’s Horizon research and innovation programme under grant agreements HORIZON GA.101092646 (CloudSkin), HORIZON MSCA GA.101086248 (CloudStars), and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID-2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033, UE (DALEST), Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3 and the Generalitat de Catalunya (AGAUR), Spain 2021-SGR-00478 (CROMAI)., Peer Reviewed, Postprint (published version)