Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 34
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Agulló López, Ferran|||0000-0002-7276-2472
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Mobility usecase: intelligent service migration in cloud-edge continuum

  • Liu, Peini|||0000-0003-0058-8732
  • Oliveras Torra, Joan
  • Palacín Marfil, Marc
  • Dalgitsis, Michail|||0000-0001-8660-0813
  • Serrano, María A.
  • Datsika, Eftychia
  • Antonopoulos, Angelos|||0000-0002-3546-1080
  • Santaella Sánchez, Javier
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Nou Castell, Ramon|||0000-0003-0464-1473
As industries increasingly embrace digital and intelligent transformation, enterprises face significant challenges in the containerization upgrades for their Artificial Intelligence(AI) applications and dynamic service migration and management in Cloud-Edge continuum(CEC). This paper presents CloudSkin, an innovative platform designed to realize streamlined, seamless and intelligent service migration in Cloud-Edge Continuum by integrating advanced containerization techniques and AI-driven orchestration capabilities. Our approach, leveraging intelligent algorithms for service migration, can seamlessly transit services between cloud and edge environments, ensuring optimised resource allocation and reducing service latency to assure quality of service(QoS). CloudSkin has been enabled in a Mobility Usecase, empowering Cellnex businesses undergoing digital transformation to achieve higher operational efficiency. The experimental results show that compared to traditional reactive service migration, using intelligent proactive service migration can provide better migration detection, improving F1-Score up to 23.5%, and reducing 28.9% the service running time where the service latency violates SLA., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Dexter: a performance-cost efficient resource allocation manager for serverless data analytics

  • Nestorov, Anna Maria
  • Marrón Vida, Diego
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Wang, Chen
  • Misale, Claudia
  • Youssef, Alaa
  • Carrera Pérez, David|||0000-0003-4898-3424
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Leveraging serverless platforms for the efficient execution of distributed data analytics frameworks, such as Apache Spark [ 3], has gained substantial interest since early 2022. The elasticity, free-of-management, and on-demand scalability of serverless have motivated the effort in deploying distributed data analytics applications to serverless platforms. However, effectively auto-scaling resources for such complex workloads so that we can fully benefit from the resource elasticity of serverless remains challenging. Mis-configuration can result in severe performance and cost issues arising from resource under- and over-provisioning. In this paper, we present Dexter, a robust resource allocation manager dynamically allocating resources at a fine-grained level to guarantee performance-cost efficiency (optimizing total runtime cost). Dexter is novel in combining predictive and reactive strategies that fully leverage the elasticity of serverless to enhance the performance-cost efficiency for workflow executions. Unlike blackbox ML models, Dexter quickly reaches a sufficiently good solution, prioritizing simplicity, generality, and ease of understanding. Our experimental evaluation shows that, compared with the default serverless Spark resource allocation that dynamically requests exponentially more executors to accommodate pending tasks, our solution achieves a cost reduction of up to 4.65×, while improving performance-cost efficiency up to 3.50×. Dexter also enables a substantial resource saving, demanding up to 5.75× fewer resources., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

On the relation between open project-based learning in undergraduate computer science education and contemporary technological trends

  • Tous Liesa, Rubén|||0000-0002-1409-5843
  • Freitag, Fèlix|||0000-0001-5438-479X
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Amidst rapid and constant technological change, keeping IT higher education curricula up to date is becoming increasingly challenging. For a decade, the course”Project on Information Technologies” within the undergraduate computer science studies offered by a major technical university has been pursuing an effort of continuous curriculum adaptation and active learning practices based on an open project-based learning (PBL) methodology. This article researches the implications of employing an open-ended PBL approach, with a specific focus on the alignment of acquired skills with the trends in the professional landscape. Our analysis has identified a strong correlation between the technologies utilized in the projects and the contemporary technological trends in the areas of global technological focus, programming languages, server-side technologies, database management systems, and DevOps-related tools. For the study, we analyzed empirical data gathered from over 100 projects involving more than 400 students who enrolled in this reference course, belonging to the last year of the IT higher education programme in the last 10 years. The results suggest the open project-based design as a teaching means in the student’s study plan for fostering the student the learning of prevailing current practical technologies., This work is partially supported by the Span-ish Ministry of Science under contracts PID2019-107255GB, PID2019-106774RB-C21, PID2021-126248OB-I00 (MCIN/AEI/10.13039/501100011033/FEDER), PDC2023-145809-I00 (PDC/AEI/10.13039/501100011033), the Recovery and Resilience Facility of the European Union, and by the SGR programme of the Catalan Government under contracts 2021-SGR-00478 and 2021-SGR-01059., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

In-context bias propagation in LLM-based tabular data generation

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Halimi, Anisa
  • Fraser, Kieran
Large Language Models (LLMs) are increasingly used for synthetic tabular data generation through in-context learning (ICL), offering a practical solution for data augmentation in data scarce scenarios. While prior work has shown the potential of LLMs to improve downstream task performance through augmenting underrepresented groups, these benefits often assume access to a subset of unbiased in-context examples, representative of the real dataset. In real-world settings, however, data is frequently noisy and demographically skewed. In this paper, we systematically study how statistical biases within in-context examples propagate to the distribution of synthetic tabular data, showing that even mild in-context biases lead to global statistical distortions. We further introduce an adversarial scenario where a malicious contributor can inject bias into the synthetic dataset via a subset of in-context examples, ultimately compromising the fairness of downstream classifiers for a targeted and protected subgroup. Finally, we evaluate mitigation strategies based on preprocessing in-context examples, demonstrating that while such interventions can attenuate disparity, the inherent sensitivity of LLMs to adversarial prompts remains a persistent challenge. Our findings highlight a critical new vulnerability in LLM-based data generation pipelines within sensitive domains, This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE. Kieran Fraser and Anisa Halimi were partly supported by the Innovative Health Initiative Joint Undertaking (IHI JU) under grant agreement No. 101172997– SEARCH., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Feature engineering for agents: an adaptive cognitive architecture for interpretable ML monitoring

  • Bravo Rocca, Gusseppe Jesus|||0000-0001-6824-1124
  • Liu, Peini|||0000-0003-0058-8732
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Carrillo Larco, Rodrigo M.
  • Dholakia, Ajay
  • Ellison, David
Monitoring Machine Learning (ML) models in production environments is crucial, yet traditional approaches often yield verbose, low-interpretability outputs that hinder effective decision-making. We propose a cognitive architecture for ML monitoring that applies feature engineering principles to agents based on Large Language Models (LLMs), significantly enhancing the interpretability of monitoring outputs. Central to our approach is a Decision Procedure module that simulates feature engineering through three key steps: Refactor, Break Down, and Compile. The Refactor step improves data representation to better capture feature semantics, allowing the LLM to focus on salient aspects of the monitoring data while reducing noise and irrelevant information. Break Down decomposes complex information for detailed analysis, and Compile integrates sub-insights into clear, interpretable outputs. This process leads to a more deterministic planning approach, reducing dependence on LLM-generated planning, which can sometimes be inconsistent and overly general. The combination of feature engineering-driven planning and selective LLM utilization results in a robust decision support system, capable of providing highly interpretable and actionable insights. Experiments using multiple LLMs demonstrate the efficacy of our approach, achieving significantly higher accuracy compared to various baselines across several domains., This work was partially supported by Lenovo and Intel® as part of the Lenovo AI Innovators University Research program, by Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under contracts PID2019- 107255GB-C22 and PID2021-126248OB-I00, and by the Generalitat de Catalunya (AGAUR) under contract 2021-SGR-00478., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Time-quality tradeoff of MuseHash query processing performance

  • Pegia, Maria
  • Agulló López, Ferran|||0000-0002-7276-2472
  • Moumtzidou, Anastasia
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Jónsson, Björn Þór
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Nowadays, large quantities of multimedia data are generated by various applications on smartphones, drones and other devices. Facilitating retrieval from these multimedia collections requires (a) effective media representation and (b) efficient indexing and query processing approaches. Recently, the MuseHash approach was proposed, which can effectively represent a variety of modalities, improving on previous hashing-based approaches. However, the interaction of the MuseHash approach with existing indexing and query processing approaches has not been considered. This paper provides the first systematic evaluation of the time-quality tradeoff that arises when MuseHash media representation is combined with state-of-the-art approximate nearest-neighbour indexes along multi-core and GPU processing., The work reported here was supported by the EU’s Horizon 2020 research and innovation programme under grant agreements H2020-101070250 CALLISTO, H2020-101070262 WATERVERSE, and H2020-101080090 ALLIES, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, Severo Ochoa Center of Excellence CEX2021-001148-S-20-3 and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Integrating reliability into intent-driven orchestration on kubernetes-based edge nodes

  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Aguilera Luzón, David|||0009-0001-1302-9000
  • Liu, Peini|||0000-0003-0058-8732
  • Nou Castell, Ramon|||0000-0003-0464-1473
  • Serrano, María A.
  • Antonopoulos, Angelos|||0000-0002-3546-1080
  • Santaella Sánchez, Javier
  • Divan Koller, Mario José
Placing workloads across the Edge and Cloud can be performed through containerization. However, conditions in Edge nodes are very different from Cloud, as environmental factors such as temperature, humidity, voltage provisioning or dust, heavily affect execution performance and device health. Reliability is an important factor when deciding to deploy such load onto Edge devices, indicating their capability to achieve a desired Quality of Service (QoS). For this, research on Edge computing must focus on how to monitor, estimate and manage devices, in a distributed, autonomous and reliable manner. Our current efforts on performance analysis for devices under "wild conditions" are moving towards integrating reliability into orchestrator systems. Here we present our vision and roadmap for expanding Cloud orchestration towards the Edge with technologies capable of providing knowledge about node environmental conditions and mitigate its impact. Through Out-Of-Band telemetry, we can retrieve temperature and power consumption variables from node components, indicating its health and estimating its reliability given external stress factors. In particular, using intent-based orchestration for containerized platforms, such factors can be used for enforcing reliability as a key-performance indicator. The current work in progress focuses on industrial and commercial scenarios, e.g., Edge computing for urban mobility, with road-side nodes performing AI-based Video-Analytics, exposed to uncontrolled weather conditions. The principal objective is to achieve an Edge network orchestration that takes into account node health in an automatic manner for reducing operational costs such as energy consumption, device repair and replacement, while maintaining QoS in the Edge., This work is financed by the INTEL UFunding #14780, the EU-HORIZON and MSCA programmes under GA.101092646 and GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Experience replay and zero-shot clustering for continual learning in diabetic retinopathy detection

  • Bravo Rocca, Gusseppe Jesus|||0000-0001-6824-1124
  • Liu, Peini|||0000-0003-0058-8732
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Dholakia, Ajay
  • Ellison, David
  • Carrillo Larco, Rodrigo M.
We present an approach to mitigate catastrophic forgetting in Continual Learning (CL), focusing on domain incremental scenarios in medical imaging. Our method leverages Large Language Models (LLMs) to generate task-agnostic descriptions from multimodal inputs, enabling zero-shot clustering of tasks without supervision. This clustering underpins an enhanced Experience Replay (ER) strategy, strategically sampling data points to refresh the model’s memory while preserving privacy. By incrementally updating a multi-head classifier using only data embeddings, our approach maintains both efficiency and data confidentiality. Evaluated on a challenging diabetic retinopathy dataset, our method demonstrates significant improvements over traditional CL techniques, including Elastic Weight Consolidation (EWC), Gradient Episodic Memory (GEM), and Learning Without Forgetting (LWF). Extensive experiments across Multi-Layer Perceptron (MLP), Residual, and Attention architectures show consistent performance gains (up to 3.1% in Average Mean Class Accuracy) and reduced forgetting, with only 6% computational overhead. These results highlight our approach’s potential for privacy-preserving, efficient CL in sensitive domains like healthcare, offering a promising direction for developing adaptive AI systems that can learn continuously while respecting data privacy constraints., This work was partially supported by Lenovo and Intel as part of the Lenovo AI Innovators University Research program, by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID2019-107255GB-C22 and PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, and by the Generalitat de Catalunya under contract 2021-SGR-00478.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Data-Connector: an agent-based framework for autonomous ML-based smart management in cloud-edge continuum

  • Liu, Peini|||0000-0003-0058-8732
  • Oliveras Torra, Joan
  • Palacín Marfil, Marc
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Nou Castell, Ramon|||0000-0003-0464-1473
Machine Learning (ML) is becoming pervasive and integrated into different kinds of intelligent applications, and the collaborative Cloud-Edge continuum has been introduced as an emerging trend to support their adoption into use cases. However, managing these ML applications in the CloudEdge continuum is challenging due to the ML application's dynamic resource usage with different user loads and Cloud and Edge's dynamic resource availability. We envision machine learning methods that can be used for smart management in this dynamic environment, but how to deploy and utilize them for the adaptation scenario in Cloud-Edge continuum is unknown. This paper proposes an agent-based framework to enable autonomous smart management mechanisms that can be broadly enabled in diverse adaptation scenarios. The agent acts as a data-connector1 1https://github.com/bsc-scanflow/data-connector, connecting different sources of data, utilizing ML models for decision-making and triggering adaptations in Cloud-edge platforms. The case study shows the feasibility of our proposed data-connector for smart migration of an ML workload in the Cloud-edge continuum. The result shows that the smart migration-enabled Cloud-edge scenario has 11.9% ML application prediction time better than the Cloud scenario without migration. Moreover, with minimal customization, the data connector agent can be adapted for more use cases., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A data-driven ML approach for maximizing performance in LLM-adapter serving

  • Agulló López, Ferran|||0000-0002-7276-2472
  • Oliveras Torra, Joan
  • Wang, Chen
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds or thousands of these adapters on a single GPU allows request aggregation, increasing throughput, but may also cause request starvation if GPU memory limits are exceeded. To address this issue, this study focuses on determining the joint configuration of concurrent and parallel adapters that maximizes GPU throughput without inducing starvation, given heterogeneous adapter and traffic properties. We propose a data-driven ML approach leveraging interpretable models to tackle this caching problem and introduce the first Digital Twin capable of reproducing an LLM-adapter serving system, enabling efficient training data generation. Experiments with the vLLM framework and LoRA adapters show that the Digital Twin reproduces throughput within 5.1% of real results, while the ML approach predicts optimal numbers of concurrent and parallel adapters with an error of at most 7.2% under heterogeneous, real-world workloads., This work has been partially financed by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCAGA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR)under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

CloudSkin: ML-based smart management for the cloud-edge continuum

  • Liu, Peini|||0000-0003-0058-8732
  • Nestorov, Anna Maria
  • Palacín Marfil, Marc
  • Nou Castell, Ramon|||0000-0003-0464-1473
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
  • Berral García, Josep Lluís|||0000-0003-3037-3580
The CloudSkin project embraces ML-based autonomous management technologies, introducing the Learning Plane (LP), which provides services and methods to embed intelligence in Cloud-Edge Continuum management layers. By leveraging AI, the continuum can achieve user demands for resource, privacy, resilience, and energy efficiency requirements, in an adaptive way. As a first Proof-of-Concept, we envision a smart orchestrator to drive a 5G-powered automotive usecase, focusing on a mobility scenario where the LP optimises resources for AI data and applications, on a transversal Edge environment., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, EU-HORIZON GA.101092644, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

GreenWise: intelligent application migration for containerized machine learning services in the computing continuum

  • Liu, Peini|||0000-0003-0058-8732
  • Santos, José
  • De Turck, Filip
  • Guitart Fernández, Jordi|||0000-0003-0751-3100
Machine Learning (ML) services require both responsiveness and efficiency. In the dynamic Computing Continuum (CC), service migration has emerged as an important strategy to optimize service performance, improve power efficiency, or reduce operational costs. A basis enabler of the service migration in the CC is containerization and container orchestration, however, dynamically moving the service among Cloud and Edge servers based on several uncertainties is still challenging. In this paper, we present GreenWise, a framework for intelligent service migration for containerized ML services in the heterogeneous CC. GreenWise extends the monitoring agents to continuously monitor power consumption and performance metrics across diverse layers and sources in real-time, providing a holistic view of states. Leveraging Reinforcement Learning (RL), it enables Power-Performance aware strategies to achieve near-optimal online migration decisions under dynamic conditions. We perform GreenWise on top of a Kubernetes-based CC platform, implementing agents for intelligent migration for containerizedML services. Experimental results demonstrate that GreenWise achieves effective trade-offs between performance and power efficiency. Our proposed Power-Latency-Aware (MaskPPO) migration strategy outperforms Random and Round-Robin baselines 159.2% and 13.8% in a real cluster. These results highlight GreenWise’s potential for sustainable and intelligent ML service migration in the dynamic CC., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON CLOUDSKIN GA.101092646, EU-HORIZON SPIRIT GA.101070672, EU-HORIZON MSCA CLOUDSTARS GA.101086248; by the Spanish Ministry of Science (MICINN) PID2021-126248OB-I00, MCIN/AEI/10.13039/501100011033/FEDER, UE and PID2024-160996OB-I00; and by the Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478. José Santos is funded by the Research Foundation Flanders (FWO), grant numbers 1299323N and 1253226N., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A reliability score for video-analytics edge-IoT devices in urban environments

  • Garcia Boris, Marçal
  • Oliveras Torra, Joan
  • Aguilera Luzón, David|||0009-0001-1302-9000
  • Liu, Peini|||0000-0003-0058-8732
  • Divan Koller, Mario José
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Performing video-analytics in Edge and IoT devices towards urban mobility allows computing near-data, providing fast and autonomous management from streets and roads. Thanks to advances in artificial intelligence methods, real-time classification of camera-provided image feeds can be performed in low-power devices to be deployed next to urban signaling and city services. However, devices deployed “in the wild” are affected by environmental factors such as temperature, either ambient or retained by its container, humidity or voltage changes, coming from sensorized urban furniture. Deciding where to deploy video-analytics workload requires a minimal device performance reliability, assuring that load can be processed in-place, before moving it towards the Fog/Cloud or other nearby devices. Here we present a simplistic factorization of a reliability Score to be quickly computed in Edge nodes, towards orchestrating workload according to the node surrounding environment. The score takes into account ambient temperature and humidity, also received voltage. The scoring is evaluated performing video-analytics workloads on real devices under physical stress, observing the performance degradation related to the external and internal factors applied onto the device. From the study, we observe how the different factors affect the functioning mode and quality of service, aside of indicating an estimation of the reliability of the urban-placed devices., This work is financed by the INTEL UFunding #14780, the EU-HORIZON and MSCA programmes under GA.101092646 and GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

HealthMesh: An architectural framework for federated healthcare data management

  • Bisquert Parés, Aniol
  • Hmimou Ham Man, Achraf|||0009-0004-0448-3406
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Romero Moral, Óscar|||0000-0001-6350-8328
Recently, significant milestones have been achieved in the field of healthcare data analysis. However, alongside these accomplishments, substantial data-related challenges have emerged in the domain of big data management. Modern healthcare projects are no more dealing with a single data repository but many heterogeneous ones and must overcome data variety, privacy and governance issues. Yet, current solutions face a privacy-decentralization trade-off. To address this dual challenge, we introduce HealthMesh, a novel layered architectural framework based on the Data Mesh principles, providing a domaindecentralised paradigm. In addition, the framework incorporates a Semantic Data Model which establishes robust governance, enables interoperability and guarantees policy compliance for all the data assets. To demonstrate the capabilities of the proposed approach, we provide an illustrative example inspired by the use case of the INCISIVE project for breast cancer analytics. Overall, this work makes a significant contribution on collecting key challenges, identifying actors and providing a set of components and guidelines for establishing a holistic framework for the complex field of healthcare data management., The work reported here was supported by the EU-H2020 programme under GA.952179 INCISIVE, Horizon Europe Programme under GA.101095717 (SECURED) and GA.101135513 (CYCLOPS), the Spanish Research State Agency (MICINN/AEI, ERDF/FEDER) under GA.MCIN/AEI/10.13039/501100011033/FEDER (UE DALEST), the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/501100011033 (DOGO4ML), and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478 (CROMAI)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

MScheduler: leveraging spot instances for high-performance reservoir simulation in the cloud

  • Albuquerque Portella, Felipe|||0000-0003-1245-7465
  • Estrela, Paulo J.B.
  • Malini, Renzo Q.
  • Teylo, Luan
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Drummond, Lúcia M.A.
Petroleum reservoir simulation uses computer models to predict fluid flow in porous media, aiding to forecast oil production. Engineers execute numerous simulations with different geological realizations to refine the accuracy of the model. These experiments require considerable computational resources, which are not always available within the on-premises infrastructure. Commercial public cloud platforms can offer many advantages, such as virtually unlimited scalability and payper-use pricing. This paper introduces MSCHEDULER, a meta scheduler framework for reservoir simulations at Petrobras, a Brazilian energy company. It efficiently executes jobs in the cloud, utilizing spot Virtual Machines (VMs) to reduce costs and ensure job completion even with VM termination. Contributions include a novel methodology for reservoir simulation checkpointing, a cost-based scheduler, and an analysis of the strategy using real production jobs from Petrobras., This work was funded by Petroleo Brasileiro S.A. – Petrobras, by Project Universal/CNPq no 404087/2021-3 and CNE/FAPERJ no E-26/201.012/2022(271103). Also by Spanish Ministry of Science (MICINN, AEI, ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, and Generalitat de Catalunya (AGAUR) 2021-SGR-00478., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Towards Pareto optimal throughput in small language model serving

  • Garcia Recasens, Pol|||0009-0008-5535-0012
  • Zhu, Yue
  • Wang, Chen
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Berral García, Josep Lluís|||0000-0003-3037-3580
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models (SLMs) offers new opportunities for resource-constrained users, who now are able to serve small models with cutting-edge performance. In this paper, we present a set of experiments designed to benchmark SLM inference at performance and energy levels. Our analysis provides a new perspective in serving, highlighting that the small memory footprint of SLMs allows for reaching the Pareto-optimal throughput within the resource capacity of a single accelerator. In this regard, we present an initial set of findings demonstrating how model replication can effectively improve resource utilization for serving SLMs., This work has been partially financed by grant agreement EUHORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021- SGR-00478, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/ 10.13039/ 501100011033/ FEDER, UE., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

GreenWise: intelligent application migration for containerized machine learning services in the computing continuum

  • Liu, Peini
  • Santos, José
  • De Turck, Filip
  • Guitart Fernández, Jordi
Machine Learning (ML) services require both responsiveness and efficiency. In the dynamic Computing Continuum (CC), service migration has emerged as an important strategy to optimize service performance, improve power efficiency, or reduce operational costs. A basis enabler of the service migration in the CC is containerization and container orchestration, however, dynamically moving the service among Cloud and Edge servers based on several uncertainties is still challenging. In this paper, we present GreenWise, a framework for intelligent service migration for containerized ML services in the heterogeneous CC. GreenWise extends the monitoring agents to continuously monitor power consumption and performance metrics across diverse layers and sources in real-time, providing a holistic view of states. Leveraging Reinforcement Learning (RL), it enables Power-Performance aware strategies to achieve near-optimal online migration decisions under dynamic conditions. We perform GreenWise on top of a Kubernetes-based CC platform, implementing agents for intelligent migration for containerizedML services. Experimental results demonstrate that GreenWise achieves effective trade-offs between performance and power efficiency. Our proposed Power-Latency-Aware (MaskPPO) migration strategy outperforms Random and Round-Robin baselines 159.2% and 13.8% in a real cluster. These results highlight GreenWise’s potential for sustainable and intelligent ML service migration in the dynamic CC., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON CLOUDSKIN GA.101092646, EU-HORIZON SPIRIT GA.101070672, EU-HORIZON MSCA CLOUDSTARS GA.101086248; by the Spanish Ministry of Science (MICINN) PID2021-126248OB-I00, MCIN/AEI/10.13039/501100011033/FEDER, UE and PID2024-160996OB-I00; and by the Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478. José Santos is funded by the Research Foundation Flanders (FWO), grant numbers 1299323N and 1253226N., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Experience replay and zero-shot clustering for continual learning in diabetic retinopathy detection

  • Bravo Rocca, Gusseppe Jesus
  • Liu, Peini
  • Guitart Fernández, Jordi
  • Dholakia, Ajay
  • Ellison, David
  • Carrillo Larco, Rodrigo M.
We present an approach to mitigate catastrophic forgetting in Continual Learning (CL), focusing on domain incremental scenarios in medical imaging. Our method leverages Large Language Models (LLMs) to generate task-agnostic descriptions from multimodal inputs, enabling zero-shot clustering of tasks without supervision. This clustering underpins an enhanced Experience Replay (ER) strategy, strategically sampling data points to refresh the model’s memory while preserving privacy. By incrementally updating a multi-head classifier using only data embeddings, our approach maintains both efficiency and data confidentiality. Evaluated on a challenging diabetic retinopathy dataset, our method demonstrates significant improvements over traditional CL techniques, including Elastic Weight Consolidation (EWC), Gradient Episodic Memory (GEM), and Learning Without Forgetting (LWF). Extensive experiments across Multi-Layer Perceptron (MLP), Residual, and Attention architectures show consistent performance gains (up to 3.1% in Average Mean Class Accuracy) and reduced forgetting, with only 6% computational overhead. These results highlight our approach’s potential for privacy-preserving, efficient CL in sensitive domains like healthcare, offering a promising direction for developing adaptive AI systems that can learn continuously while respecting data privacy constraints., This work was partially supported by Lenovo and Intel as part of the Lenovo AI Innovators University Research program, by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID2019-107255GB-C22 and PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, and by the Generalitat de Catalunya under contract 2021-SGR-00478., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

On the relation between open project-based learning in undergraduate computer science education and contemporary technological trends

  • Tous Liesa, Rubén
  • Freitag, Fèlix
  • Berral García, Josep Lluís
Amidst rapid and constant technological change, keeping IT higher education curricula up to date is becoming increasingly challenging. For a decade, the course”Project on Information Technologies” within the undergraduate computer science studies offered by a major technical university has been pursuing an effort of continuous curriculum adaptation and active learning practices based on an open project-based learning (PBL) methodology. This article researches the implications of employing an open-ended PBL approach, with a specific focus on the alignment of acquired skills with the trends in the professional landscape. Our analysis has identified a strong correlation between the technologies utilized in the projects and the contemporary technological trends in the areas of global technological focus, programming languages, server-side technologies, database management systems, and DevOps-related tools. For the study, we analyzed empirical data gathered from over 100 projects involving more than 400 students who enrolled in this reference course, belonging to the last year of the IT higher education programme in the last 10 years. The results suggest the open project-based design as a teaching means in the student’s study plan for fostering the student the learning of prevailing current practical technologies., This work is partially supported by the Span-ish Ministry of Science under contracts PID2019-107255GB, PID2019-106774RB-C21, PID2021-126248OB-I00 (MCIN/AEI/10.13039/501100011033/FEDER), PDC2023-145809-I00 (PDC/AEI/10.13039/501100011033), the Recovery and Resilience Facility of the European Union, and by the SGR programme of the Catalan Government under contracts 2021-SGR-00478 and 2021-SGR-01059., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Feature engineering for agents: an adaptive cognitive architecture for interpretable ML monitoring

  • Bravo Rocca, Gusseppe Jesus
  • Liu, Peini
  • Guitart Fernández, Jordi
  • Carrillo Larco, Rodrigo M.
  • Dholakia, Ajay
  • Ellison, David
Monitoring Machine Learning (ML) models in production environments is crucial, yet traditional approaches often yield verbose, low-interpretability outputs that hinder effective decision-making. We propose a cognitive architecture for ML monitoring that applies feature engineering principles to agents based on Large Language Models (LLMs), significantly enhancing the interpretability of monitoring outputs. Central to our approach is a Decision Procedure module that simulates feature engineering through three key steps: Refactor, Break Down, and Compile. The Refactor step improves data representation to better capture feature semantics, allowing the LLM to focus on salient aspects of the monitoring data while reducing noise and irrelevant information. Break Down decomposes complex information for detailed analysis, and Compile integrates sub-insights into clear, interpretable outputs. This process leads to a more deterministic planning approach, reducing dependence on LLM-generated planning, which can sometimes be inconsistent and overly general. The combination of feature engineering-driven planning and selective LLM utilization results in a robust decision support system, capable of providing highly interpretable and actionable insights. Experiments using multiple LLMs demonstrate the efficacy of our approach, achieving significantly higher accuracy compared to various baselines across several domains., This work was partially supported by Lenovo and Intel® as part of the Lenovo AI Innovators University Research program, by Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under contracts PID2019- 107255GB-C22 and PID2021-126248OB-I00, and by the Generalitat de Catalunya (AGAUR) under contract 2021-SGR-00478., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

MScheduler: leveraging spot instances for high-performance reservoir simulation in the cloud

  • Albuquerque Portella, Felipe
  • Estrela, Paulo J.B.
  • Malini, Renzo Q.
  • Teylo, Luan
  • Berral García, Josep Lluís
  • Drummond, Lúcia M.A.
Petroleum reservoir simulation uses computer models to predict fluid flow in porous media, aiding to forecast oil production. Engineers execute numerous simulations with different geological realizations to refine the accuracy of the model. These experiments require considerable computational resources, which are not always available within the on-premises infrastructure. Commercial public cloud platforms can offer many advantages, such as virtually unlimited scalability and payper-use pricing. This paper introduces MSCHEDULER, a meta scheduler framework for reservoir simulations at Petrobras, a Brazilian energy company. It efficiently executes jobs in the cloud, utilizing spot Virtual Machines (VMs) to reduce costs and ensure job completion even with VM termination. Contributions include a novel methodology for reservoir simulation checkpointing, a cost-based scheduler, and an analysis of the strategy using real production jobs from Petrobras., This work was funded by Petroleo Brasileiro S.A. – Petrobras, by Project Universal/CNPq no 404087/2021-3 and CNE/FAPERJ no E-26/201.012/2022(271103). Also by Spanish Ministry of Science (MICINN, AEI, ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, and Generalitat de Catalunya (AGAUR) 2021-SGR-00478., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Integrating reliability into intent-driven orchestration on kubernetes-based edge nodes

  • Berral García, Josep Lluís
  • Aguilera Luzón, David
  • Liu, Peini
  • Nou Castell, Ramon
  • Serrano, María A.
  • Antonopoulos, Angelos
  • Santaella Sánchez, Javier
  • Divan Koller, Mario José
Placing workloads across the Edge and Cloud can be performed through containerization. However, conditions in Edge nodes are very different from Cloud, as environmental factors such as temperature, humidity, voltage provisioning or dust, heavily affect execution performance and device health. Reliability is an important factor when deciding to deploy such load onto Edge devices, indicating their capability to achieve a desired Quality of Service (QoS). For this, research on Edge computing must focus on how to monitor, estimate and manage devices, in a distributed, autonomous and reliable manner. Our current efforts on performance analysis for devices under "wild conditions" are moving towards integrating reliability into orchestrator systems. Here we present our vision and roadmap for expanding Cloud orchestration towards the Edge with technologies capable of providing knowledge about node environmental conditions and mitigate its impact. Through Out-Of-Band telemetry, we can retrieve temperature and power consumption variables from node components, indicating its health and estimating its reliability given external stress factors. In particular, using intent-based orchestration for containerized platforms, such factors can be used for enforcing reliability as a key-performance indicator. The current work in progress focuses on industrial and commercial scenarios, e.g., Edge computing for urban mobility, with road-side nodes performing AI-based Video-Analytics, exposed to uncontrolled weather conditions. The principal objective is to achieve an Edge network orchestration that takes into account node health in an automatic manner for reducing operational costs such as energy consumption, device repair and replacement, while maintaining QoS in the Edge., This work is financed by the INTEL UFunding #14780, the EU-HORIZON and MSCA programmes under GA.101092646 and GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A data-driven ML approach for maximizing performance in LLM-adapter serving

  • Agulló López, Ferran
  • Oliveras Torra, Joan
  • Wang, Chen
  • Gutiérrez Torre, Alberto
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds or thousands of these adapters on a single GPU allows request aggregation, increasing throughput, but may also cause request starvation if GPU memory limits are exceeded. To address this issue, this study focuses on determining the joint configuration of concurrent and parallel adapters that maximizes GPU throughput without inducing starvation, given heterogeneous adapter and traffic properties. We propose a data-driven ML approach leveraging interpretable models to tackle this caching problem and introduce the first Digital Twin capable of reproducing an LLM-adapter serving system, enabling efficient training data generation. Experiments with the vLLM framework and LoRA adapters show that the Digital Twin reproduces throughput within 5.1% of real results, while the ML approach predicts optimal numbers of concurrent and parallel adapters with an error of at most 7.2% under heterogeneous, real-world workloads., This work has been partially financed by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCAGA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR)under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Data-Connector: an agent-based framework for autonomous ML-based smart management in cloud-edge continuum

  • Liu, Peini
  • Oliveras Torra, Joan
  • Palacín Marfil, Marc
  • Guitart Fernández, Jordi
  • Berral García, Josep Lluís
  • Nou Castell, Ramon
Machine Learning (ML) is becoming pervasive and integrated into different kinds of intelligent applications, and the collaborative Cloud-Edge continuum has been introduced as an emerging trend to support their adoption into use cases. However, managing these ML applications in the CloudEdge continuum is challenging due to the ML application's dynamic resource usage with different user loads and Cloud and Edge's dynamic resource availability. We envision machine learning methods that can be used for smart management in this dynamic environment, but how to deploy and utilize them for the adaptation scenario in Cloud-Edge continuum is unknown. This paper proposes an agent-based framework to enable autonomous smart management mechanisms that can be broadly enabled in diverse adaptation scenarios. The agent acts as a data-connector1 1https://github.com/bsc-scanflow/data-connector, connecting different sources of data, utilizing ML models for decision-making and triggering adaptations in Cloud-edge platforms. The case study shows the feasibility of our proposed data-connector for smart migration of an ML workload in the Cloud-edge continuum. The result shows that the smart migration-enabled Cloud-edge scenario has 11.9% ML application prediction time better than the Cloud scenario without migration. Moreover, with minimal customization, the data connector agent can be adapted for more use cases., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

In-context bias propagation in LLM-based tabular data generation

  • Garcia Recasens, Pol
  • Gutiérrez Torre, Alberto
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
  • Halimi, Anisa
  • Fraser, Kieran
Large Language Models (LLMs) are increasingly used for synthetic tabular data generation through in-context learning (ICL), offering a practical solution for data augmentation in data scarce scenarios. While prior work has shown the potential of LLMs to improve downstream task performance through augmenting underrepresented groups, these benefits often assume access to a subset of unbiased in-context examples, representative of the real dataset. In real-world settings, however, data is frequently noisy and demographically skewed. In this paper, we systematically study how statistical biases within in-context examples propagate to the distribution of synthetic tabular data, showing that even mild in-context biases lead to global statistical distortions. We further introduce an adversarial scenario where a malicious contributor can inject bias into the synthetic dataset via a subset of in-context examples, ultimately compromising the fairness of downstream classifiers for a targeted and protected subgroup. Finally, we evaluate mitigation strategies based on preprocessing in-context examples, demonstrating that while such interventions can attenuate disparity, the inherent sensitivity of LLMs to adversarial prompts remains a persistent challenge. Our findings highlight a critical new vulnerability in LLM-based data generation pipelines within sensitive domains, This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE. Kieran Fraser and Anisa Halimi were partly supported by the Innovative Health Initiative Joint Undertaking (IHI JU) under grant agreement No. 101172997– SEARCH., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mobility usecase: intelligent service migration in cloud-edge continuum

  • Liu, Peini
  • Oliveras Torra, Joan
  • Palacín Marfil, Marc
  • Dalgitsis, Michail
  • Serrano, María A.
  • Datsika, Eftychia
  • Antonopoulos, Angelos
  • Santaella Sánchez, Javier
  • Guitart Fernández, Jordi
  • Berral García, Josep Lluís
  • Nou Castell, Ramon
As industries increasingly embrace digital and intelligent transformation, enterprises face significant challenges in the containerization upgrades for their Artificial Intelligence(AI) applications and dynamic service migration and management in Cloud-Edge continuum(CEC). This paper presents CloudSkin, an innovative platform designed to realize streamlined, seamless and intelligent service migration in Cloud-Edge Continuum by integrating advanced containerization techniques and AI-driven orchestration capabilities. Our approach, leveraging intelligent algorithms for service migration, can seamlessly transit services between cloud and edge environments, ensuring optimised resource allocation and reducing service latency to assure quality of service(QoS). CloudSkin has been enabled in a Mobility Usecase, empowering Cellnex businesses undergoing digital transformation to achieve higher operational efficiency. The experimental results show that compared to traditional reactive service migration, using intelligent proactive service migration can provide better migration detection, improving F1-Score up to 23.5%, and reducing 28.9% the service running time where the service latency violates SLA., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

CloudSkin: ML-based smart management for the cloud-edge continuum

  • Liu, Peini
  • Nestorov, Anna Maria
  • Palacín Marfil, Marc
  • Nou Castell, Ramon
  • Guitart Fernández, Jordi
  • Berral García, Josep Lluís
The CloudSkin project embraces ML-based autonomous management technologies, introducing the Learning Plane (LP), which provides services and methods to embed intelligence in Cloud-Edge Continuum management layers. By leveraging AI, the continuum can achieve user demands for resource, privacy, resilience, and energy efficiency requirements, in an adaptive way. As a first Proof-of-Concept, we envision a smart orchestrator to drive a 5G-powered automotive usecase, focusing on a mobility scenario where the LP optimises resources for AI data and applications, on a transversal Edge environment., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, EU-HORIZON GA.101092644, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A reliability score for video-analytics edge-IoT devices in urban environments

  • Garcia Boris, Marçal
  • Oliveras Torra, Joan
  • Aguilera Luzón, David
  • Liu, Peini
  • Divan Koller, Mario José
  • Berral García, Josep Lluís
Performing video-analytics in Edge and IoT devices towards urban mobility allows computing near-data, providing fast and autonomous management from streets and roads. Thanks to advances in artificial intelligence methods, real-time classification of camera-provided image feeds can be performed in low-power devices to be deployed next to urban signaling and city services. However, devices deployed “in the wild” are affected by environmental factors such as temperature, either ambient or retained by its container, humidity or voltage changes, coming from sensorized urban furniture. Deciding where to deploy video-analytics workload requires a minimal device performance reliability, assuring that load can be processed in-place, before moving it towards the Fog/Cloud or other nearby devices. Here we present a simplistic factorization of a reliability Score to be quickly computed in Edge nodes, towards orchestrating workload according to the node surrounding environment. The score takes into account ambient temperature and humidity, also received voltage. The scoring is evaluated performing video-analytics workloads on real devices under physical stress, observing the performance degradation related to the external and internal factors applied onto the device. From the study, we observe how the different factors affect the functioning mode and quality of service, aside of indicating an estimation of the reliability of the urban-placed devices., This work is financed by the INTEL UFunding #14780, the EU-HORIZON and MSCA programmes under GA.101092646 and GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards Pareto optimal throughput in small language model serving

  • Garcia Recasens, Pol
  • Zhu, Yue
  • Wang, Chen
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Youssef, Alaa
  • Torres Viñals, Jordi
  • Berral García, Josep Lluís
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models (SLMs) offers new opportunities for resource-constrained users, who now are able to serve small models with cutting-edge performance. In this paper, we present a set of experiments designed to benchmark SLM inference at performance and energy levels. Our analysis provides a new perspective in serving, highlighting that the small memory footprint of SLMs allows for reaching the Pareto-optimal throughput within the resource capacity of a single accelerator. In this regard, we present an initial set of findings demonstrating how model replication can effectively improve resource utilization for serving SLMs., This work has been partially financed by grant agreement EUHORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021- SGR-00478, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OB-I00, MCIN/AEI/ 10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mind the memory gap: unveiling GPU bottlenecks in large-batch LLM inference

  • Garcia Recasens, Pol
  • Agulló López, Ferran
  • Zhu, Yue
  • Wang, Changyang
  • Lee, Eun Kyung
  • Tardieu, Olivier
  • Torres Tomás, Juan José
  • Berral García, Josep Lluís
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference. While batching is commonly used to increase throughput, per-formance gains plateau beyond a certain batch size, especially with smaller models, a phenomenon that existing literature typically explains as a shift to the compute-bound regime. In this paper, through an in-depth GPU-level analysis, we reveal that large-batch inference remains memory-bound, with most GPU compute capabilities underutilized due to DRAM bandwidth saturation as the primary bottleneck. To address this, we propose a Batching Configuration Advisor (BCA) that optimizes memory allocation, reducing GPU memory requirements with minimal impact on throughput. The freed memory and underutilized GPU compute capabilities can then be leveraged by concurrent workloads. Specifically, we use model replication to improve serving throughput and GPU utilization. Our findings challenge conventional assumptions about LLM inference, offering new in-sights and practical strategies for improving resource utilization, particularly for smaller language models., This work has been partially financed by grant agreement EU-HORIZON GA.101095717 and by the EU-HORIZON MSCA programme under grant agreement EU-HORIZON MSCA GA.101086248. Also, it has been partially financed by Generalitat de Catalunya (AGAUR) under grant agreement 2021-SGR-00478, by Severo Ochoa Center of Excellence CEX-2021-001148-S-20-3, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreement PID2021-126248OBI00, MCIN/AEI/10.13039/ 501100011033/ FEDER, UE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Time-quality tradeoff of MuseHash query processing performance

  • Pegia, Maria
  • Agulló López, Ferran
  • Moumtzidou, Anastasia
  • Gutiérrez Torre, Alberto
  • Jónsson, Björn Þór
  • Berral García, Josep Lluís
  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Nowadays, large quantities of multimedia data are generated by various applications on smartphones, drones and other devices. Facilitating retrieval from these multimedia collections requires (a) effective media representation and (b) efficient indexing and query processing approaches. Recently, the MuseHash approach was proposed, which can effectively represent a variety of modalities, improving on previous hashing-based approaches. However, the interaction of the MuseHash approach with existing indexing and query processing approaches has not been considered. This paper provides the first systematic evaluation of the time-quality tradeoff that arises when MuseHash media representation is combined with state-of-the-art approximate nearest-neighbour indexes along multi-core and GPU processing., The work reported here was supported by the EU’s Horizon 2020 research and innovation programme under grant agreements H2020-101070250 CALLISTO, H2020-101070262 WATERVERSE, and H2020-101080090 ALLIES, and by the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) under grant agreements PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE, Severo Ochoa Center of Excellence CEX2021-001148-S-20-3 and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Dexter: a performance-cost efficient resource allocation manager for serverless data analytics

  • Nestorov, Anna Maria
  • Marrón Vida, Diego
  • Gutiérrez Torre, Alberto
  • Wang, Chen
  • Misale, Claudia
  • Youssef, Alaa
  • Carrera Pérez, David
  • Berral García, Josep Lluís
Leveraging serverless platforms for the efficient execution of distributed data analytics frameworks, such as Apache Spark [ 3], has gained substantial interest since early 2022. The elasticity, free-of-management, and on-demand scalability of serverless have motivated the effort in deploying distributed data analytics applications to serverless platforms. However, effectively auto-scaling resources for such complex workloads so that we can fully benefit from the resource elasticity of serverless remains challenging. Mis-configuration can result in severe performance and cost issues arising from resource under- and over-provisioning. In this paper, we present Dexter, a robust resource allocation manager dynamically allocating resources at a fine-grained level to guarantee performance-cost efficiency (optimizing total runtime cost). Dexter is novel in combining predictive and reactive strategies that fully leverage the elasticity of serverless to enhance the performance-cost efficiency for workflow executions. Unlike blackbox ML models, Dexter quickly reaches a sufficiently good solution, prioritizing simplicity, generality, and ease of understanding. Our experimental evaluation shows that, compared with the default serverless Spark resource allocation that dynamically requests exponentially more executors to accommodate pending tasks, our solution achieves a cost reduction of up to 4.65×, while improving performance-cost efficiency up to 3.50×. Dexter also enables a substantial resource saving, demanding up to 5.75× fewer resources., This work is financed by the EU-HORIZON programme under grant agreements EU-HORIZON GA.101092646, EU-HORIZON MSCA GA.101086248, by Generalitat de Catalunya (AGAUR) GA.2021-SGR-00478, and the Spanish Ministry of Science (MICINN), the Research State Agency (AEI) and European Regional Development Funds (ERDF/FEDER) PID2021-126248OB-I00, MCIN/AEI/10.13039/ 501100011033/FEDER, UE., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

HealthMesh: An architectural framework for federated healthcare data management

  • Bisquert Parés, Aniol
  • Hmimou Ham Man, Achraf
  • Berral García, Josep Lluís
  • Gutiérrez Torre, Alberto
  • Romero Moral, Óscar
Recently, significant milestones have been achieved in the field of healthcare data analysis. However, alongside these accomplishments, substantial data-related challenges have emerged in the domain of big data management. Modern healthcare projects are no more dealing with a single data repository but many heterogeneous ones and must overcome data variety, privacy and governance issues. Yet, current solutions face a privacy-decentralization trade-off. To address this dual challenge, we introduce HealthMesh, a novel layered architectural framework based on the Data Mesh principles, providing a domaindecentralised paradigm. In addition, the framework incorporates a Semantic Data Model which establishes robust governance, enables interoperability and guarantees policy compliance for all the data assets. To demonstrate the capabilities of the proposed approach, we provide an illustrative example inspired by the use case of the INCISIVE project for breast cancer analytics. Overall, this work makes a significant contribution on collecting key challenges, identifying actors and providing a set of components and guidelines for establishing a holistic framework for the complex field of healthcare data management., The work reported here was supported by the EU-H2020 programme under GA.952179 INCISIVE, Horizon Europe Programme under GA.101095717 (SECURED) and GA.101135513 (CYCLOPS), the Spanish Research State Agency (MICINN/AEI, ERDF/FEDER) under GA.MCIN/AEI/10.13039/501100011033/FEDER (UE DALEST), the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/501100011033 (DOGO4ML), and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478 (CROMAI)., Peer Reviewed, Postprint (published version)