Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 31
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

AI4EOSC: A federated cloud platform for Artificial Intelligence in scientific research

  • Heredia, Ignacio
  • López Garcia, Álvaro
  • Aguilar Gómez, Fernando
  • Alibabaei, Khadijeh
  • Berberi, Lisana
  • Castro, Pedro
  • Costantini, Alessandro
  • David, Mário
  • Díez, Jaime
  • Dlugolinsky, Stefan
  • Donvito, Giacinto
  • Duda, Leonhard
  • Esteban-Sanchis, Borja
  • Fernandez Tobías, Saúl
  • Heredia Canales, Andrés
  • Kozlov, Valentin
  • Machado, João
  • San Martín, Daniel
  • Nguyen, Giang
  • Obregón Ruiz, Marta
  • Plóciennik, Marcin
  • Rebolledo Ruiz, Susana
  • Sáinz-Pardo Díaz, Judith
  • eleng, Martin
  • Tran, Viet
  • Aguirre-Ramírez, Diego Alejandro
  • Alarcón-Marín, Caterina
  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Langarita-Benítez, Sergio
  • Moltó, Germán|||0000-0002-8049-253X
  • Rodríguez-Benítez, Vicente|||0000-0002-7335-3849
[EN] The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard machine learning operations (MLOps) tools and platforms and the unique requirements of modern and Open Science, particularly regarding the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. This paper presents AI4EOSC, a federated, open-source platform designed to operationalize the full AI/ML life-cycle within the European Open Science Cloud (EOSC) ecosystem. Our methodology tackles the fragmentation of distributed research infrastructures by integrating a modular and distributed architecture comprising an AI development platform, a serverless AI-as-a-Service layer, and a federated orchestration model that is able to integrate heterogeneous computing and storage resources from distributed e-infrastructures. AI4EOSC also introduces a ¿FAIR-by-design¿ approach that enforces metadata standardization (via MLDCAT-AP) and W3C PROV-compliant provenance tracking through a platform-integrated CI/CD pipeline. The added value of AI4EOSC is demonstrated through the delivery of a diverse set of community installations, which show consistent and seamless deployment across heterogeneous cloud providers. These installations are validated by a set of scientific cases, showing how our work reduces the manual burden on researchers while ensuring high levels of reproducibility and interoperability and providing a unified environment for the development, training, and production of AI/ML models in the EOSC., The authors acknowledge Marcos Lloret Iglesias, the AI4EOSC project manager, for his constant support and help towards bringing the project to fruition. The authors acknowledge funding and support from the AI4EOSC project Artificial Intelligence for the European Open Science Cloud , that has received funding from the European Union s Horizon Europe research and innovation programme under grant agreement number 101058593. ALG also acknowledges the support from the Consejería de Educación, Formación Profesional y Universidades of the Gobierno de Cantabria via the Actividad estructural para el desarrollo de la investigación del Instituto de Física de Cantabria project.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Rescheduling serverless workloads across the cloud-to-edge continuum

  • Risco, Sebastián
  • Alarcón-Marín, Caterina
  • Langarita-Benítez, Sergio
  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Moltó, Germán|||0000-0002-8049-253X
[EN] Serverless computing was a breakthrough in Cloud computing due to its high elasticity capabilities and fine-grained pay-per-use model offered by the main public Cloud providers. Meanwhile, open-source serverless platforms supporting the FaaS (Function as a Service) model allow users to take advantage of many of their benefits while operating on the on-premises platforms of organizations. This opens the possibility to deploy and exploit them on the different layers of the cloud-to-edge continuum, either on IoT (Internet of Things) devices located at the Edge (i.e. next to data acquisition devices), in on-premises clusters closer to the data sources (i.e. Fog computing) or directly on the Cloud. This paper presents two strategies to mitigate the overload that disparate data ingestion rates may cause in low-powered devices at the Edge or Fog layers. To this end, it is proposed to delegate and reschedule serverless jobs between the different layers of the cloud-to-edge continuum using an open-source platform for event-driven file processing. To demonstrate the performance of these strategies, a use case for fire detection is proposed that includes processing in the Fog via minified Kubernetes clusters located near the Edge, in the private Cloud via on-premises elastic clusters and, finally, in the public Cloud by using the AWS (Amazon Web Services) Lambda FaaS service. The results indicate that these strategies can mitigate overloads in use cases involving processing across the cloud-to-edge continuum by coordinating several layers of computing resources., Grant PID2020-113126RB-I00 funded by MCIN/AEI/10.13039/501100011033. Project PDC2021-120844-I00 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. This work was supported by the project AI-SPRINT "AI in Secure Privacy-Preserving Computing Continuum" that has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant 101016577. This work was also supported by the project AI4EOSC "Artificial Intelligence for the European Open Science Cloud" that has received funding from the European Union's Horizon Europe Research and Innovation Programme under Grant 101058593.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Tesis de maestría (MasterThesis)

Portal for the deployment and execution of AI inference models on serverless platforms, Portal for the deployment and execution of AI inference models on serverless platforms, Portal web per al desplegament i execució de models d&apos, inferència de IA sobre plataformes serverless

  • Reinoso Hernandez, Santiago Andres
[ES] Este proyecto se centra en integrar nuevas capacidades en un portal web serverless basado en la arquitectura JamStack, desde el portal se permitirá desplegar y ejecutar modelos de IA encapsulados contenedores de Docker. El enfoque JamStack permite la creación y distribución eficiente de sitios web estáticos, lo que resulta en una experiencia de usuario fluida y veloz. La aplicación se compone de un frontend desarrollado con el framework de JavaScript Angular, en colaboración con el equipo del proyecto europeo AI4EOSC. Se busca extender el portal para simplificar el uso de los modelos de inferencia de IA, haciendo uso de OSCAR, una plataforma de código abierto para la ejecución dirigida por eventos de aplicaciones computacionalmente intensivas. Para la integración con OSCAR, se desarrolla un cliente utilizando Node.js y Typescript. Este cliente permitirá realizar solicitudes a los servicios de OSCAR a través de su API, así como validar y transformar los datos antes de enviarlos al servicio correspondiente de OSCAR. La creación de un servicio de OSCAR desde el portal web es simple, rápida e intuitiva para el usuario. El cliente de TypeScript actúa como una dependencia dentro del portal web, realizando validaciones de datos y generando un nuevo servicio de OSCAR en un clúster especificado. Como resultado, el usuario obtiene un modelo de IA desplegado como un servicio de OSCAR. Una vez desplegado, el servicio de OSCAR puede probar fácilmente desde el portal web seleccionando la opción "Try me". Esto desencadena una invocación que ejecuta el modelo de IA dentro de un contenedor Docker en la plataforma OSCAR. Después de recibir la respuesta, el portal web ha sido mejorado para reaccionar dinámicamente a esta, mostrándola de forma amigable para el usuario. En cuanto a los modelos de inferencia de IA, el proyecto AI4EOSC ofrece una innovadora funcionalidad al permitir a los usuarios acceder a un marketplace existente de imágenes Docker, con una amplia variedad de modelos disponibles para probar. El acceso a este marketplace es libre y brinda a los usuarios la posibilidad de seleccionar y utilizar los modelos que mejor se adapten a sus necesidades, lo que otorga una flexibilidad y personalización en el proceso de inferencia. Este trabajo académico proporcionará una valiosa ventaja en el trabajo de científicos, investigadores y académicos al ofrecer acceso a una amplia gama de modelos de inferencia, eficiencia en el despliegue, adaptabilidad, escalabilidad y la oportunidad de colaborar con otros expertos en el campo. Estos beneficios combinados contribuirán a mejorar la calidad de sus investigaciones y proyectos en inteligencia artificial, [EN] This project focuses on integrating new capabilities into a serverless web portal based on the JamStack architecture, allowing deployment and running AI models encapsulated in Docker containers. The JamStack approach enables an efficient creation and distribution of static websites, resulting in a fluent and fast user experience. The application is composed by a frontend developed with the Angular JavaScript framework, together with the European AI4EOSC project team. It seeks extend the portal in order to simplify the use of AI inference models, using OSCAR, an open-source platform for the event-driven execution of applications that are highly computationally intensive. For the integration with OSCAR, a Node.js and Typescript client is developed. This client will allow requests to OSCAR services through its API, as well as validate and transform the data before sending it to the corresponding OSCAR service. The creation of an OSCAR's service using the web portal is simple, fast and intuitive for the user. The TypeScript client acts as a dependency inside the web portal, performing data validations and generating a new OSCAR service in a specified cluster. As a result, the user gets an AI model deployed as an OSCAR service. Once deployed, the OSCAR service can be easily tested from the web portal using the "Try me" option. This triggers an invocation that runs the AI model inside a Docker container on the OSCAR platform. After receiving the response, the web portal has been enhanced to dynamically react to it, displaying it in a user-friendly way. Regarding AI inference models, the AI4EOSC project offers innovative functionality allowing users to access an existing marketplace of Docker images, with a large variety of models available for test. Access to this marketplace is free and gives users the ability to select and use the models that best suit to their needs, providing flexibility and customization in the inference process. This academic work will provide a valuable advantage in the work of scientists, researchers, and academics by offering access to a wide range of inference models, efficiency in deployment, adaptability, scalability and the opportunity to collaborate with experts in the field. These combined benefits will contribute to improving the quality of your research and projects in artificial intelligence.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Machine learning operations landscape: platforms and tools

  • Berberi, Lisana
  • Kozlov, Valentin
  • Nguyen, Giang
  • Sáinz-Pardo Díaz, Judith
  • Tran, Viet
  • López García, Álvaro
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Moltó, Germán|||0000-0002-8049-253X
[EN] As the field of machine learning advances, managing and monitoring intelligent models in production, also known as machine learning operations (MLOps), has become essential. Organizations are increasingly adopting artificial intelligence as a strategic tool, thus increasing the need for reliable, and scalable MLOps platforms. Consequently, every aspect of the machine learning life cycle, from workflow orchestration to performance monitoring, presents both challenges and opportunities that require sophisticated, flexible, and scalable technological solutions. This research addresses this demand by providing a comprehensive assessment framework of MLOps platforms highlighting the key features necessary for a robust MLOps solution. The paper examines 16 MLOps tools widely used, which revolve around capabilities within AI infrastructure management, including but not limited to experiment tracking, model deployment, and model inference. Our three-step evaluation framework starts with a feature analysis of the MLOps platforms, then GitHub stars growth assessment for adoption and prominence, and finally, a weighted scoring method to single out the most influential platforms. From this process, we derive valuable insights into the essential components of effective MLOps systems and provide a decision-making flowchart that simplifies platform selection. This framework provides hands-on guidance for organizations looking to initiate or enhance their MLOps strategies, whether they require an end-end solutions or specialized tools., This work is funded by the European Union through the Horizon Europe AI4EOSC project under Grant Number 101058593.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Best practices for AI-based image analysis applications in aquatic sciences: The iMagine case study

  • Azmi, Elnaz
  • Alibabaei, Khadijeh
  • Kozlov, Valentin
  • Tjerk, Krijger
  • Accarino, Gabriele
  • Ayata, Sakina-Dorothée
  • De Carlo, Marco Mariano
  • Decrop, Wout
  • Elia, Donatello
  • Fiore, Sandro
  • Francescangeli, Marco
  • Irisson, Jean-Olivier
  • Rune Lagaisse
  • Martin Laviale
  • Antoine Lebeaud
  • Carolin Leluschko
  • Enoc Martínez
  • Igor Ruiz Atake
  • Antonio Augusto Sepp Neves
  • Damian Smyth
  • Jesús Soriano-González
  • Muhammad Arabi Tayyab
  • Vanessa Tosello
  • Álvaro López García
  • Dick Schaap
  • Gergely Sipos
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Moltó, Germán|||0000-0002-8049-253X
[EN] The iMagine project is an EU-funded initiative led by the EGI Foundation. One of the objectives of this project is to provide an AI platform that leverages AI-powered tools to improve the processing and analysis of imaging data from marine and freshwater ecosystems, contributing to the study of the health of oceans, seas, coasts, and inland waters. Connected to the European Open Science Cloud (EOSC), iMagine supports the development, training, and deployment of AI models by collaborating with twelve use cases across diverse aquatic science fields. This collaboration fosters valuable insights and accelerates scientific progress by refining existing solutions in data acquisition, preprocessing, and model deployment. The platform offers trained models as a service, integrating AI tools for image annotation, ensuring the creation of high-quality datasets that comply with FAIR principles. Through these methodologies, iMagine enhances consistency, enabling researchers to efficiently publish and share data in repositories., The project iMagine receives funding from the European Union s Horizon Europe research and innovation program under grant agreement number 101058625. The AI4OS software stack is developed in the AI4EOSC project, which receives funding from the European Union s Horizon Europe research and innovation program under grant agreement number 101058593. The use case Freshwater Diatoms Identification described in Section 13.10 receives funding from ANR, France (ANR-20-THIA-0010).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Landscape of machine learning evolution: privacy-preserving federated learning frameworks and tools

  • Nguyen, Giang
  • Sáinz-Pardo Díaz, Judith
  • Berberi, Lisana
  • Lytvyn, Oleksandr
  • Kozlov, Valentin
  • Tran, Viet
  • López García, Álvaro
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Moltó, Germán|||0000-0002-8049-253X
[EN] Machine learning is one of the most widely used technologies in the field of Artificial Intelligence. As machine learning applications become increasingly ubiquitous, concerns about data privacy and security have also grown. The work in this paper presents a broad theoretical landscape concerning the evolution of machine learning and deep learning from centralized to distributed learning, first in relation to privacy-preserving machine learning and secondly in the area of privacy-enhancing technologies. It provides a comprehensive landscape of the synergy between distributed machine learning and privacy-enhancing technologies, with federated learning being one of the most prominent architectures. Various distributed learning approaches to privacy-aware techniques are structured in a review, followed by an in-depth description of relevant frameworks and libraries, more particularly in the context of federated learning. The paper also highlights the need for data protection and privacy addressed from different approaches, key findings in the field concerning AI applications, and advances in the development of related tools and techniques., Open access funding provided by The Ministry of Education, Science, Research and Sport of the Slovak Republic in cooperation with Centre for Scientific and Technical Information of the Slovak Republic. This work is funded by the European Union through the Horizon Europe AI4EOSC project under grant number 101058593.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Efficient and scalable covariate drift detection in machine learning systems with serverless computing

  • Céspedes Sisniega, Jaime
  • López García, Álvaro
  • Rodríguez-Benítez, Vicente|||0000-0002-7335-3849
  • Moltó, Germán|||0000-0002-8049-253X
[EN] As machine learning models are increasingly deployed in production, robust monitoring and detection of concept and covariate drift become critical. This paper addresses the gap in the widespread adoption of drift detection techniques by proposing a serverless-based approach for batch covariate drift detection in ML systems. Leveraging the open-source OSCAR framework and the open-source Frouros drift detection library, we develop a set of services that enable parallel execution of two key components: the ML inference pipeline and the batch covariate drift detection pipeline. To this end, our proposal takes advantage of the elasticity and efficiency of serverless computing for ML pipelines, including scalability, cost-effectiveness, and seamless integration with existing infrastructure. We evaluate this approach through an edge ML use case, showcasing its operation on a simulated batch covariate drift scenario. Our research highlights the importance of integrating drift detection as a fundamental requirement in developing robust and trustworthy AI systems and encourages the adoption of these techniques in ML deployment pipelines. In this way, organizations can proactively identify and mitigate the adverse effects of covariate drift while capitalizing on the benefits offered by serverless computing., This work was supported by the project AI4EOSC "Artificial Intelligence for the European Open Science Cloud" that has received funding from the European Union's Horizon Europe Research and Innovation Programme under Grant 101058593. JCS and ALG acknowledge the funding from the Agencia Estatal de Investigacion, Unidad de Excelencia Maria de Maeztu, ref. MDM-2017-0765. GM and VR would like to acknowledge Grant PID2020-113126RB-I00 funded by MICIU/AEI/10.13039/501100011033 and also project PDC2021-120844-I00 funded by MICIU/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Infrastructure Manager: A TOSCA-Based Orchestrator for the Computing Continuum

  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Moltó, Germán|||0000-0002-8049-253X
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Blanquer Espert, Ignacio|||0000-0003-1692-8922
[EN] The edge-to-cloud continuum involves heterogeneous computing resources, including low-power physical devices, Virtual Machines (VMs) in cloud management platforms and serverless computing services based on the FaaS (Functions as a Service) model. This requires novel strategies to describe and efficiently deploy complex applications that execute across the computing continuum. To this end, this paper introduces the developments in the Infrastructure Manager (IM), an open-source TOSCA-based orchestrator to provision and configure virtualized computing resources from a wide range of cloud platforms. By supplementing TOSCA with additional types, the IM can also provision from FaaS platforms across the computing continuum by leveraging public cloud services such as AWS Lambda and on-premises serverless platforms, such as OSCAR. This allows event-driven data-processing applications across multiple computing platforms and architectures. The evolution of the Infrastructure Manager is described to accommodate the definition in TOSCA of complex applications that span across the computing continuum and their automated provisioning and configuration using Infrastructure as Code (IaC) approaches. Its effectiveness is assessed through a real use case involving a machine-learning classifier application for assisting in the early diagnosis of Rheumatic Heart Disease (RHD). The results show that the new developments enable the IM to efficiently deploy complete application architectures described in TOSCA across the computing continuum, from VMs to FaaS services., Grant PID2020-113126RB-I00 funded by MCIN/AEI/10.13039/501100011033. Also, Project PDC2021- 120844-I00 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. This work was partially supported by the project AI-SPRINT AI in Secure Privacy-Preserving Computing Continuum which has received funding from the European Union s Horizon 2020 Research and Innovation Programme under Grant 101016577 and by the AI4EOSC Artificial Intelligence for the European Open Science Cloud that has received funding from the European Union s Horizon Europe Research and Innovation Programme under Grant 101058593. Finally, Amanda wants to thank the support received by the Universitat Politècnica de València for the project Extensión de modelos de computaci¿n en cloud en entornos confiables y altamente distribuidos , from the PAID-10-20 postdoctoral program.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Study of the performance and scalability of federated learning for medical imaging with intermittent clients

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
Federated learning is a data decentralization privacy-preserving technique used to perform machine or deep learning in a secure way. In this paper we present theoretical aspects about federated learning, such as the presentation of an aggregation operator, different types of federated learning, and issues to be taken into account in relation to the distribution of data from the clients, together with the exhaustive analysis of a use case where the number of clients varies. Specifically, a use case of medical image analysis is proposed, using chest X-ray images obtained from an open data repository. In addition to the advantages related to privacy, improvements in predictions (in terms of accuracy, loss and area under the curve) and reduction of execution times will be studied with respect to the classical case (the centralized approach). Different clients will be simulated from the training data, selected in an unbalanced manner. The results of considering three or ten clients are exposed and compared between them and against the centralized case. Two different problems related to intermittent clients are discussed, together with two approaches to be followed for each of them. Specifically, this type of problems may occur because in a real scenario some clients may leave the training, and others enter it, and on the other hand because of client technical or connectivity problems. Finally, improvements and future work in the field are proposed., The authors acknowledge the funding through the European Union - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI + Salud Global) and the support from the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A Python library to check the level of anonymity of a dataset

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
Openly sharing data with sensitive attributes and privacy restrictions is a challenging task. In this document we present the implementation of pyCANON, a Python library and command line interface (CLI) to check and assess the level of anonymity of a dataset through some of the most common anonymization techniques: k-anonymity, (α,k)-anonymity, ℓ-diversity, entropy ℓ-diversity, recursive (c,ℓ)-diversity, t-closeness, basic β-likeness, enhanced β-likeness and δ-disclosure privacy. For the case of more than one sensitive attribute, two approaches are proposed for evaluating these techniques. The main strength of this library is to obtain a full report of the parameters that are fulfilled for each of the techniques mentioned above, with the unique requirement of the set of quasi-identifiers and sensitive attributes. The methods implemented are presented together with the attacks they prevent, the description of the library, examples of the different functions’ usage, as well as the impact and the possible applications that can be developed. Finally, some possible aspects to be incorporated in future updates are proposed., The authors acknowledge the funding through the European Union - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI + Salud Global) and the support from the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

pyCANON

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
pyCANON is a Python library and CLI to assess the values of the parameters associated with the most common privacy-preserving techniques.-- Language: Python., Last update 18 May 2023., pyCANON is a Python library and CLI to assess the values of the parameters associated with the most common privacy-preserving techniques. It can be used to know the level of anonymity of a dataset (and thus publish or share it while being aware of the risks involved). Nine different techniques are used for this purpose. Customized reports in JSON and PDF formats can be easy obtained., Funding through the European Union - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI + Salud Global) and the support from the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Comparison of machine learning models applied on anonymized data with different techniques

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
Anonymization techniques based on obfuscating the quasi-identifiers by means of value generalization hierarchies are widely used to achieve preset levels of privacy. To prevent different types of attacks against database privacy it is necessary to apply several anonymization techniques beyond the classical k-anonymity or ℓ-diversity. However, the application of these methods is directly connected to a reduction of their utility in prediction and decision making tasks. In this work we study four classical machine learning methods currently used for classification purposes in order to analyze the results as a function of the anonymization techniques applied and the parameters selected for each of them. The performance of these models is studied when varying the value of k for k-anonymity and additional tools such as ℓ-diversity, t-closeness and δ-disclosure privacy are also deployed on the well-known adult dataset., The authors would like to thank the funding through the European Union - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI+Salud Global) and the support from the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., No
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Deep learning based soft-sensor for continuous chlorophyll estimation on decentralized data

  • Sáinz-Pardo Díaz, Judith
  • Castrillo, María
  • López-García, Álvaro
Monitoring the concentration of pigments like chlorophyll (Chl) in water-bodies is a key task to contribute to their conservation. However, with the existing sensor technology, measurement in real-time and with enough frequency to ensure proper risk management is not completely feasible. In this work, with the concept of data-driven soft-sensing, three hydrophysical features are used together with three meteorological ones to estimate the concentration of Chl in two tributaries of the River Thames. Data driven models, specifically neural networks, are used with three learning approaches: individual, centralized and federated. Data reduction scenarios are proposed in order to analyze the performance of each approach when less data is available. The best results in the training are usually obtained with the individual approach. However, the federated learning provides better generalization ability. It was also observed that in most of the cases the results of the federated learning approach improve those of the centralized one., The authors acknowledge the support from the project AI4EOSC ‘Artificial Intelligence for the European Open Science Cloud’ that has received funding from the European Union's Horizon Europe research and innovation program under grant agreement number 101058593, as well as the support from Universidad de Cantabria and Consejeria de Universidades, Igualdad, Cultura y Deporte del Gobierno de Cantabria via the ‘Instrumentacion y Ciencia de Datos para Sondear la Naturaleza del Universo’ project., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Enhancing radiomics and Deep Learning systems through the standardization of medical imaging workflows

  • Cobo Cano, Miriam
  • Menéndez Fernández-Miranda, Pablo
  • Bastarrika, Gorka
  • Lloret Iglesias, Lara
Recent advances in computer-aided diagnosis, treatment response and prognosis in radiomics and deep learning challenge radiology with requirements for world-wide methodological standards for labeling, preprocessing and image acquisition protocols. The adoption of these standards in the clinical workflows is a necessary step towards generalization and interoperability of radiomics and artificial intelligence algorithms in medical imaging., M. C. would like to acknowledge the support received by the Ministry of Education of Spain (FPU grant, reference FPU21-04458). The authors would like to acknowledge the support from the project AI4EOSC ‘‘Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Frouros: An open-source Python library for drift detection in machine learning systems

  • Céspedes Sisniega, Jaime
  • López-García, Álvaro
Frouros is an open-source Python library capable of detecting drift in machine learning systems. It provides a combination of classical and more recent algorithms for drift detection, covering both concept and data drift. We have designed it to be compatible with any machine learning framework and easily adaptable to real-world use cases. The library is developed following best development and continuous integration practices to ensure ease of maintenance and extensibility., The authors acknowledge the funding from the Agencia Estatal de Investigación, Spain, Unidad de Excelencia María de Maeztu, Spain, ref. MDM-2017-0765 and the support from the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” which has received funding from the European Union’s Horizon Europe Research and Innovation Programme under grant agreement 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Making Federated Learning Accessible to Scientists: The AI4EOSC Approach

  • Sáinz-Pardo Díaz, Judith
  • Heredia Canales, Andrés
  • Heredia, Ignacio
  • Tran, Viet
  • Nguyen, Giang
  • Alibabaei, Khadijeh
  • Obregón Ruiz, Marta
  • Rebolledo Ruiz, Susana
  • López-García, Álvaro
Trabajo presentado a: Conference: IH&MMSEC '24: ACM Workshop on Information Hiding and Multimedia Security Baiona Spain June 24 - 26, 2024., Access to computing resources is a critical requirement for researchers in a wide diversity of areas. This has become even more important with the rise of artificial intelligence techniques through the training of machine learning and deep learning models. In this sense, the AI4EOSC project aims to respond to this need by delivering an enhanced set of advanced services and tools for the development of artificial intelligence, machine and deep models, such as federated learning, in the European Open Science Cloud (EOSC). Federated learning is a technology in the field of privacy-preserving machine learning techniques that has revolutionized the current state of the art, evolving from classical centralized approaches to allow training models in a decentralized way, without sharing raw data. In this work, we present the production implementation of a federated learning system based on the Flower framework that allows users, without a technological background, to exploit this technique, performing federated learning training within the AI4EOSC platform. The objective is to be able to train this type of architecture in an intuitive way; for this purpose, a user-friendly dashboard has been implemented, whose development will be reviewed. The frameworks and technologies used for this implementation will be exposed together with an example of use from scratch, in order to demonstrate the use of this functionality of the platform. Finally, two scenarios concerning client availability are analyzed., The authors would like to thank the funding through the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Efficient and scalable covariate drift detection in machine learning systems with serverless computing

  • Céspedes Sisniega, Jaime
  • Rodríguez Rodríguez, Vicente
  • Moltó, Germán
  • López-García, Álvaro
As machine learning models are increasingly deployed in production, robust monitoring and detection of concept and covariate drift become critical. This paper addresses the gap in the widespread adoption of drift detection techniques by proposing a serverless-based approach for batch covariate drift detection in ML systems. Leveraging the open-source OSCAR framework and the open-source Frouros drift detection library, we develop a set of services that enable parallel execution of two key components: the ML inference pipeline and the batch covariate drift detection pipeline. To this end, our proposal takes advantage of the elasticity and efficiency of serverless computing for ML pipelines, including scalability, cost-effectiveness, and seamless integration with existing infrastructure. We evaluate this approach through an edge ML use case, showcasing its operation on a simulated batch covariate drift scenario. Our research highlights the importance of integrating drift detection as a fundamental requirement in developing robust and trustworthy AI systems and encourages the adoption of these techniques in ML deployment pipelines. In this way, organizations can proactively identify and mitigate the adverse effects of covariate drift while capitalizing on the benefits offered by serverless computing., This work was supported by the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant 101058593. JCS and ALG acknowledge the funding from the Agencia Estatal de Investigación, Unidad de Excelencia María de Maeztu, ref. MDM-2017-0765. GM and VR would like to acknowledge Grant PID2020-113126RB-I00 funded by MICIU/AEI/10.13039/501100011033 and also project PDC2021-120844-I00 funded by MICIU/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Network security AIOps for online stream data monitoring

  • Nguyen, Giang
  • Dlugolinsky, Stefan
  • Tran, Viet
  • López-García, Álvaro
In cybersecurity, live production data for predictive analysis pose a significant challenge due to the inherently secure nature of the domain. Although there are publicly available, synthesized, and artificially generated datasets, authentic scenarios are rarely encountered. For anomaly-based detection, the dynamic definition of thresholds has gained importance and attention in detecting abnormalities and preventing malicious activities. Unlike conventional threshold-based methods, deep learning data modeling provides a more nuanced perspective on network monitoring. This enables security systems to continually refine and adapt to the evolving situation in streaming data online, which is also our goal. Furthermore, our work in this paper contributes significantly to AIOps research, particularly through the deployment of our intelligent module that cooperates within a monitoring system in production. Our work addresses a crucial gap in the security research landscape toward more practical and effective secure strategies., This work is funded by the European Union through the Horizon Europe AI4EOSC project under grant number 101058593., Open access funding provided by The Ministry of Education, Science, Research and Sport of the Slovak Republic in cooperation with Centre for Scientific and Technical Information of the Slovak Republic., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A retrospective study of deep learning generalization across two centers and multiple models of X-ray devices using COVID-19 chest-X rays

  • Menéndez Fernández-Miranda, Pablo
  • Marqués Fraguela, Enrique
  • Álvarez de Linera-Alperi, Marta
  • Cobo Cano, Miriam
  • Pérez del Barrio, Amaia
  • Rodríguez González, David
  • Vega, Jose Antonio
  • Lloret Iglesias, Lara
Generalization of deep learning (DL) algorithms is critical for the secure implementation of computer-aided diagnosis systems in clinical practice. However, broad generalization remains to be a challenge in machine learning. This research aims to identify and study potential factors that can affect the internal validation and generalization of DL networks, namely the institution where the images come from, the image processing applied by the X-ray device, and the type of response function of the X-ray device. For these purposes, a pre-trained convolutional neural network (CNN) (VGG16) was trained three times for classifying COVID-19 and control chest radiographs with the same hyperparameters, but using different combinations of data acquired in two institutions by three different X-ray device manufacturers. Regarding internal validation, the addition of images from an external institution to the training set did not modify the algorithm's internal performance, however, the inclusion of images acquired by a device from a different manufacturer decreased the performance up to 8% (p < 0.05). In contrast, generalization across institutions and X-ray devices with the same type of response function was achieved. Nonetheless, generalization was not observed across devices with different types of response function. This factor was the key impediment to achieving broad generalization in our research, followed by the device's image-processing and the inter-institutional differences, which both reduced generalization performance to 18.9% (p < 0.05), and 9.8% (p < 0.05), respectively. Finally, clustering analysis with features extracted by the CNN was performed, revealing a substantial dependence of feature values extracted by the pre-trained CNN on the X-ray device which acquired the images., This research work was funded by the European Commission—NextGenerationEU (Regulation EU 2020/2094), through CSIC Global Health Platform (PTI+ Salud Global), 202050E107. The authors would like to acknowledge the support from the project AI4EOSC ‘‘Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation program under grant agreement number 101058593. The authors would like to thank the institutions that participated in this study, the patients who made this research possible by providing their images, the radiologists who generously labeled the images, and CSIC (Spanish National Research Council) Global Health Platform. M.C. acknowledges the support received by the Ministry of Education of Spain (FPU grant, reference FPU21-04458)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Personalized federated learning for improving radar based precipitation nowcasting on heterogeneous areas

  • Sáinz-Pardo Díaz, Judith
  • Castrillo, María
  • Bartok, Juraj
  • Heredia, Ignacio
  • Ondík, Irina Malkin
  • Martynovskyi, Ivan
  • Alibabaei, Khadijeh
  • Berberi, Lisana
  • Kozlov, Valentin
  • López-García, Álvaro
The increasing generation of data in different areas of life, such as the environment, highlights the need to explore new techniques for processing and exploiting data for useful purposes. In this context, artificial intelligence techniques, especially through deep learning models, are key tools to be used on the large amount of data that can be obtained, for example, from weather radars. In many cases, the information collected by these radars is not open, or belongs to different institutions, thus needing to deal with the distributed nature of this data. In this work, the applicability of a personalized federated learning architecture, which has been called adapFL, on distributed weather radar images is addressed. To this end, given a single available radar covering 400 km in diameter, the captured images are divided in such a way that they are disjointly distributed into four different federated clients. The results obtained with adapFL are analyzed in each zone, as well as in a central area covering part of the surface of each of the previously distributed areas. The ultimate goal of this work is to study the generalization capability of this type of learning technique for its extrapolation to use cases in which a representative number of radars is available, whose data can not be centralized due to technical, legal or administrative concerns. The results of this preliminary study indicate that the performance obtained in each zone with the adapFL approach allows improving the results of the federated learning approach, the individual deep learning models and the classical Continuity Tracking Radar Echoes by Correlation approach., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature., This work has been funded by the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

An Open Source Python Library for Anonymizing Sensitive Data

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
Open science is a fundamental pillar to promote scientific progress and collaboration, based on the principles of open data, open source and open access. However, the requirements for publishing and sharing open data are in many cases difficult to meet in compliance with strict data protection regulations. Consequently, researchers need to rely on proven methods that allow them to anonymize their data without sharing it with third parties. To this end, this paper presents the implementation of a Python library for the anonymization of sensitive tabular data. This framework provides users with a wide range of anonymization methods that can be applied on the given dataset, including the set of identifiers, quasi-identifiers, generalization hierarchies and allowed level of suppression, along with the sensitive attribute and the level of anonymity required. The library has been implemented following best practices for integration and continuous development, as well as the use of workflows to test code coverage based on unit and functional tests., The authors would like to thank the funding through the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593 and from the SIESTA project “Secure Interactive Environments for Sensitive daTa Analytics”, funded by the European Union (Horizon Europe) under grant agreement number 101131957., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Landscape of machine learning evolution: privacy-preserving federated learning frameworks and tools

  • Nguyen, Giang
  • Sáinz-Pardo Díaz, Judith
  • Calatrava, Amanda
  • Berberi, Lisana
  • Lytvyn, Oleksandr
  • Kozlov, Valentin
  • Tran, Viet
  • Moltó, Germán
  • López-García, Álvaro
Machine learning is one of the most widely used technologies in the field of Artificial Intelligence. As machine learning applications become increasingly ubiquitous, concerns about data privacy and security have also grown. The work in this paper presents a broad theoretical landscape concerning the evolution of machine learning and deep learning from centralized to distributed learning, first in relation to privacy-preserving machine learning and secondly in the area of privacy-enhancing technologies. It provides a comprehensive landscape of the synergy between distributed machine learning and privacy-enhancing technologies, with federated learning being one of the most prominent architectures. Various distributed learning approaches to privacy-aware techniques are structured in a review, followed by an in-depth description of relevant frameworks and libraries, more particularly in the context of federated learning. The paper also highlights the need for data protection and privacy addressed from different approaches, key findings in the field concerning AI applications, and advances in the development of related tools and techniques., Open access funding provided by The Ministry of Education, Science, Research and Sport of the Slovak Republic in cooperation with Centre for Scientific and Technical Information of the Slovak Republic., This work is funded by the European Union through the Horizon Europe AI4EOSC project under grant number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Machine learning operations landscape: platforms and tools

  • Berberi, Lisana
  • Kozlov, Valentin
  • Nguyen, Giang
  • Sáinz-Pardo Díaz, Judith
  • Calatrava, Amanda
  • Moltó, Germán
  • Tran, Viet
  • López-García, Álvaro
As the field of machine learning advances, managing and monitoring intelligent models in production, also known as machine learning operations (MLOps), has become essential. Organizations are increasingly adopting artificial intelligence as a strategic tool, thus increasing the need for reliable, and scalable MLOps platforms. Consequently, every aspect of the machine learning life cycle, from workflow orchestration to performance monitoring, presents both challenges and opportunities that require sophisticated, flexible, and scalable technological solutions. This research addresses this demand by providing a comprehensive assessment framework of MLOps platforms highlighting the key features necessary for a robust MLOps solution. The paper examines 16 MLOps tools widely used, which revolve around capabilities within AI infrastructure management, including but not limited to experiment tracking, model deployment, and model inference. Our three-step evaluation framework starts with a feature analysis of the MLOps platforms, then GitHub stars growth assessment for adoption and prominence, and finally, a weighted scoring method to single out the most influential platforms. From this process, we derive valuable insights into the essential components of effective MLOps systems and provide a decision-making flowchart that simplifies platform selection. This framework provides hands-on guidance for organizations looking to initiate or enhance their MLOps strategies, whether they require an end-end solutions or specialized tools., Open Access funding enabled and organized by Projekt DEAL., This work is funded by the European Union through the Horizon Europe AI4EOSC project under Grant Number 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Rescheduling serverless workloads across the cloud-to-edge continuum

  • Risco, Sebastián
  • Alarcón, Caterina
  • Langarita, Sergio
  • Caballer, Miguel
  • Moltó, Germán
Serverless computing was a breakthrough in Cloud computing due to its high elasticity capabilities and fine-grained pay-per-use model offered by the main public Cloud providers. Meanwhile, open-source serverless platforms supporting the FaaS (Function as a Service) model allow users to take advantage of many of their benefits while operating on the on-premises platforms of organizations. This opens the possibility to deploy and exploit them on the different layers of the cloud-to-edge continuum, either on IoT (Internet of Things) devices located at the Edge (i.e. next to data acquisition devices), in on-premises clusters closer to the data sources (i.e. Fog computing) or directly on the Cloud. This paper presents two strategies to mitigate the overload that disparate data ingestion rates may cause in low-powered devices at the Edge or Fog layers. To this end, it is proposed to delegate and reschedule serverless jobs between the different layers of the cloud-to-edge continuum using an open-source platform for event-driven file processing. To demonstrate the performance of these strategies, a use case for fire detection is proposed that includes processing in the Fog via minified Kubernetes clusters located near the Edge, in the private Cloud via on-premises elastic clusters and, finally, in the public Cloud by using the AWS (Amazon Web Services) Lambda FaaS service. The results indicate that these strategies can mitigate overloads in use cases involving processing across the cloud-to-edge continuum by coordinating several layers of computing resources., Grant PID2020-113126RB-I00 funded by MCIN/AEI/10.13039/501100011033. Project PDC2021-120844-I00 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. This work was supported by the project AI-SPRINT “AI in Secure Privacy-Preserving Computing Continuum” that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant 101016577. This work was also supported by the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe Research and Innovation Programme under Grant 101058593., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Best practices for AI-based image analysis applications in aquatic sciences: The iMagine case study

  • Azmi, Elnaz
  • Alibabaei, Khadijeh
  • Moltó, Germán
  • Kozlov, Valentin
  • Calatrava, Amanda
  • López-García, Álvaro
The implementations and datasets of use cases are all publicly available at the following addresses: • The tools available on the iMagine platform as a part of the AI4OS software stack: https://github.com/ai4os • The code sources of applications and models implemented through the iMagine use cases: https://github.com/ai4os-hub • The ready to use applications of the use cases on the iMagine Marketplace: https://dashboard.cloud.imagine-ai.eu/marketplace • The training datasets of the iMagine use cases: https://zenodo.org/communities/imagine-project/records?q=&f=resource_type%3Adataset, E. Azmi et al., The iMagine project is an EU-funded initiative led by the EGI Foundation. One of the objectives of this project is to provide an AI platform that leverages AI-powered tools to improve the processing and analysis of imaging data from marine and freshwater ecosystems, contributing to the study of the health of oceans, seas, coasts, and inland waters. Connected to the European Open Science Cloud (EOSC), iMagine supports the development, training, and deployment of AI models by collaborating with twelve use cases across diverse aquatic science fields. This collaboration fosters valuable insights and accelerates scientific progress by refining existing solutions in data acquisition, preprocessing, and model deployment. The platform offers trained models as a service, integrating AI tools for image annotation, ensuring the creation of high-quality datasets that comply with FAIR principles. Through these methodologies, iMagine enhances consistency, enabling researchers to efficiently publish and share data in repositories., Beyond its AI tools, iMagine places a strong emphasis on deep learning models, such as convolutional neural networks, for tasks like image classification, object detection, and segmentation, tailored to the unique requirements of aquatic sciences. It also provides robust performance evaluation tools, including experiment tracking, while tackling challenges such as AI model drift and data biases to ensure research reproducibility and transparency. The platform enables users to develop, train, share, and deploy AI models within a flexible environment that integrates with federated cloud and high-performance computing infrastructures, using Docker containers for smooth execution. Additionally, iMagine fosters collaboration with projects like AI4EOSC and Blue-Cloud, and Research Infrastructures such as EMSO and SeaDataNet, expanding its impact on the scientific community., This paper summarizes the key lessons and best practices learned in the iMagine project through the full process of AI-based aquatic image analysis, from data preparation and annotation to model deployment and evaluation. The paper therefore helps aquatic scientists advance their AI-driven image analysis approaches., The project iMagine receives funding from the European Union’s Horizon Europe research and innovation program under grant agreement number 101058625. The AI4OS software stack is developed in the AI4EOSC project, which receives funding from the European Union’s Horizon Europe research and innovation program under grant agreement number 101058593. The use case “Freshwater Diatoms Identification” described in Section 13.10 receives funding from ANR , France (ANR-20-THIA-0010)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Enhancing the convergence of federated learning aggregation strategies with limited data

  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
Trabajo presentado a la 3rd IEEE International Conference on Federated Learning Technologies and Applications (FLTA25), celebrada en Dubrovnik, Croatia del 14 al 17 de octubre de 2025., The development of deep learning techniques is a leading field applied to cases in which medical data is used, particularly in cases of image diagnosis. This type of data has privacy and legal restrictions that in many cases prevent it from being processed from central servers. However, in this area collaboration between different research centers, in order to create models as robust as possible, trained with the largest quantity and diversity of data available, is a critical point to be taken into account. In this sense, the application of privacy aware distributed architectures, such as federated learning arises. When applying this type of architecture, the server aggregates the different local models trained with the data of each data owner to build a global model. This point is critical and therefore it is fundamental to analyze different ways of aggregation according to the use case, taking into account the distribution of the clients, the characteristics of the model, etc. In this paper we propose a novel aggregation strategy and we apply it to a use case of cerebral magnetic resonance image classification. In this use case the aggregation function proposed manages to improve the convergence obtained over the rounds of the federated learning process in relation to different aggregation strategies classically implemented and applied., We would like to thank the funding and support from the AI4EOSC project and from the EOSC-SIESTA project (funded by the European Union (Horizon Europe) under grant agreement numbers 101058593 and 101131957 respectively)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Exploring federated learning for thermal urban feature segmentation - A comparison of centralized and decentralized approaches

  • Duda, Leonhard
  • Alibabaei, Khadijeh
  • Kozlov, Valentin
  • Berberi, Lisana
  • Sáinz-Pardo Díaz, Judith
  • López-García, Álvaro
L. Duda et al., The 25th International Conference on Computational Science and Its Applications (ICCSA 2025) has been held on June 30- July 3, 2025 in collaboration with Galatasaray University, Istanbul, Türkiye., Federated Learning (FL) is an approach for training a shared Machine Learning (ML) model with distributed training data and multiple participants. FL allows bypassing limitations of the traditional Centralized Machine Learning CL if data cannot be shared or stored centrally due to privacy or technical restrictions -- the participants train the model locally with their training data and do not need to share it among the other participants. This paper investigates the practical implementation and effectiveness of FL in a real-world scenario, specifically focusing on unmanned aerial vehicle (UAV)-based thermal images for common thermal feature detection in urban environments. The distributed nature of the data arises naturally and makes it suitable for FL applications, as images captured in two German cities are available. This application presents unique challenges due to non-identical distribution and feature characteristics of data captured at both locations. The study makes several key contributions by evaluating FL algorithms in real deployment scenarios rather than simulation. We compare several FL approaches with a centralized learning baseline across key performance metrics such as model accuracy, training time, communication overhead, and energy usage. This paper also explores various FL workflows, comparing client-controlled workflows and server-controlled workflows. The findings of this work serve as a valuable reference for understanding the practical application and limitations of the FL methods in segmentation tasks in UAV-based imaging., This work is supported by the AI4EOSC project, which receives funding from the European Union’s Horizon Europe 2022 research and innovation programme under agreement 101058593. Additionally, computational resources were provided by the HoreKa supercomputer, funded by the Ministry of Science, Research and the Arts Baden-Wurttemberg and the Federal Ministry of Education and Research, and by the Helmholtz Association Initiative and Networking Fund through the Helmholtz AI platform grant and the HAICORE@KIT partition., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Robust and explainable deep learning for computed tomography-based diagnostic systems, Aprendizaje profundo explicable y robusto para sistemas de diagnóstico basados en tomografía computarizada

  • Cobo Cano, Miriam
Memoria presentada para acceder al título de Doctor en Ciencia y Tecnología por la Universidad de Cantabria., [ES] La adopción clínica de la inteligencia artificial está limitada por problemas relacionados con los datos, los algoritmos y la falta de validación clínica rigurosa. Esta tesis se centra en mejorar los sistemas de diagnóstico basados en tomografía computarizada con técnicas de aprendizaje profundo explicables y robustas. En la primera parte se presentan los fundamentos del aprendizaje profundo, los principios físicos de las imágenes médicas, junto con las limitaciones actuales (estandarización, interoperabilidad, reproducibilidad). La segunda parte profundiza en dos aplicaciones concretas de aprendizaje profundo, el pronóstico de hemorragias intracraneales y el diagnóstico precoz del cáncer de pulmón. La inteligencia artificial en salud depende de colaboraciones interdisciplinarias y validaciones externas de alta calidad para identificar necesidades clínicas reales, avanzar en la medicina personalizada, además de garantizar una traslación clínica segura y fiable., [EN] The clinical adoption of artificial intelligence is hindered by challenges related to the data, the algorithms, and lack of rigorous clinical validation. This thesis is focused on enhancing computed tomography-based diagnostic systems with robust and explainable deep learning techniques. The first part presents the foundations of deep learning, the fundamentals of physics in medical imaging, together with current limitations to the clinical adoption of artificial intelligence (standardization, interoperability, reproducibility). The second part delves into two specific applications of deep learning, intracranial hemorrhage prognosis and lung cancer early diagnosis. Artificial intelligence in healthcare relies on interdisciplinary collaborations and high quality external validation to identify real clinical needs, advance personalized medicine, in addition to ensuring safe and reliable clinical translation., This work was funded by the Ministry of Education of Spain through the FPU (Training programme for Academic Staff) grant with reference FPU21-04458, and the FPU grant for research stays EST24/00533, as well as the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud”, that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593. Funding for the collaboration with CIMA-CUN has also been allocated from the Lung Ambition Alliance Prize for Research on Lung Cancer Early Detection (01/11/2021-01/11/2024)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Metric-privacy-inspired noise calibration in federated learning: Improving convergence and preventing client inference attacks

  • Sáinz-Pardo Díaz, Judith
  • Athanasiou, Andreas
  • Jung, Kangsoo
  • Palamidessi, Catuscia
  • López-García, Álvaro
Federated learning (FL) enables the training of a global model across multiple data owners (clients) without sharing raw data. This distributed architecture is orchestrated by a central server that aggregates the local models from the clients. In cases where the server is trusted but not all network nodes, differential privacy (DP) can be used to privatize the aggregated model by adding noise. However, this may affect convergence across the FL rounds. In this work, we build on the notion of metric-privacy as a design principle to calibrate the noise added by the server under a global-DP setting, with the objective of mitigating its impact on the convergence of the aggregated model. We do not enforce metric-privacy as a formal guarantee, but rather use it to guide noise calibration. We compare our approach with vanilla FL and global-DP by analyzing the impact on six aggregation strategies and applying it to a medical imaging use case, simulating different scenarios with homogeneous and non-i.i.d. clients. Finally, we introduce the client inference attack (CIA), where a semi-honest client tries to find whether another client participated in the training and study how it can be mitigated using DP and metric-aware noise calibration. Our experiments show that metric-privacy aware noise calibration strategy improves the accuracy compared to standard DP in all the scenarios analyzed, while achieving a comparable success rate against CIA. These results indicate that metric-privacy inspired noise calibration can deliver a superior utility-privacy trade-off in medical-imaging federated settings., Judith Sáinz-Pardo Díaz and Álvaro López García would like to thank the funding through the project AI4EOSC “Artificial Intelligence for the European Open Science Cloud” that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593 and from the SIESTA project “Secure Interactive Environments for Sensitive daTa Analytics”, funded by the European Union (Horizon Europe) under grant agreement number 101131957. The work of Andreas Athanasiou is supported by the project RECITALS, funded by the EU (grant agreement 101168490) and the project CRYPTECS, funded by the ANR (project number ANR-20-CYAL-0006) and by the BMBF (project number 16KIS1439). The work of Kangsoo Jung is supported by the project ELSA, funded by the Horizon Europe Framework (with project number 101070617). The work of Catuscia Palamidessi is supported by the project HYPATIA, funded by the ERC (grant agreement number 835294)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

AI4EOSC: A federated cloud platform for Artificial Intelligence in scientific research

  • Heredia, Ignacio
  • López-García, Álvaro
  • Aguilar, Fernando
  • Aguirre, Diego
  • Alarcón Marín, Caterina
  • Alibabaei, Khadijeh
  • Berberi, Lisana
  • Caballer, Miguel
  • Calatrava, Amanda
  • Castro, Pedro
  • Costantini, Alessandro
  • David, Mario
  • Díez, Jaime
  • Dlugolinsky, Stefan
  • Donvito, Giacinto
  • Duda, Leonhard
  • Esteban, Borja
  • Fernandez Tobías, Saúl
  • Heredia Canales, Andrés
  • Kozlov, Valentin
  • Langarita, Sergio
  • Machado, João
  • San-Martín, Daniel
  • Moltó, Germán
  • Nguyen, Giang
  • Obregón Ruiz, Marta
  • Plociennik, Marcin
  • Rebolledo Ruiz, Susana
  • Rodríguez Rodríguez, Vicente
  • Sáinz-Pardo Díaz, Judith
  • Šeleng, Martin
  • Tran, Viet
The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard machine learning operations (MLOps) tools and platforms and the unique requirements of modern and Open Science, particularly regarding the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. This paper presents AI4EOSC, a federated, open-source platform designed to operationalize the full AI/ML life-cycle within the European Open Science Cloud (EOSC) ecosystem. Our methodology tackles the fragmentation of distributed research infrastructures by integrating a modular and distributed architecture comprising an AI development platform, a serverless AI-as-a-Service layer, and a federated orchestration model that is able to integrate heterogeneous computing and storage resources from distributed e-infrastructures. AI4EOSC also introduces a “FAIR-by-design” approach that enforces metadata standardization (via MLDCAT-AP) and W3C PROV-compliant provenance tracking through a platform-integrated CI/CD pipeline. The added value of AI4EOSC is demonstrated through the delivery of a diverse set of community installations, which show consistent and seamless deployment across heterogeneous cloud providers. These installations are validated by a set of scientific cases, showing how our work reduces the manual burden on researchers while ensuring high levels of reproducibility and interoperability and providing a unified environment for the development, training, and production of AI/ML models in the EOSC., The authors acknowledge Marcos Lloret Iglesias, the AI4EOSC project manager, for his constant support and help towards bringing the project to fruition. The authors acknowledge funding and support from the AI4EOSC project “Artificial Intelligence for the European Open Science Cloud”, that has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement number 101058593. ALG also acknowledges the support from the Consejería de Educación, Formación Profesional y Universidades of the Gobierno de Cantabria via the “Actividad estructural para el desarrollo de la investigación del Instituto de Física de Cantabria” project., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Interpretable deep learning for rotator cuff calcific tendinopathy diagnosis: a multi-center study

  • Miranda Bautista, Juan
  • Llorente Peris, Javier
  • Ybáñez Carrillo, Fernando
  • Lloret Iglesias, Lara
  • Vega, Jose Antonio
  • Menéndez Fernández-Miranda, Pablo
The reliable deployment of artificial intelligence systems in medical imaging requires high diagnostic performance, robustness and interpretability. In this study, we developed and evaluated two automated frameworks for binary classification of shoulder radiographs (XRs) using deep learning (DL) and hybrid DL–machine learning (ML) approaches. A convolutional neural network (CNN) based on a fine-tuned VGG19 architecture was trained end-to-end on a large, balanced dataset of 4,268 shoulder XRs. In parallel, hybrid models were constructed by extracting deep feature representations from the trained network and combining them with traditional ML classifiers. Model performance was evaluated on independent internal (n = 480) and external (n = 308) validation sets. Both approaches achieved high discriminative performance. Paired comparison of Receiver Operating Characteristic (ROC) curves using the DeLong test revealed no statistically significant differences between the end-to-end CNN and the hybrid CNN–ML pipeline for either internal validation (AUC 0.956 vs. 0.961) or external generalization (AUC 0.940 vs. 0.942). Model interpretability was assessed using Grad-CAM and SHAP values. Our results suggest that while both frameworks are robust, the end-to-end DL approach offers a more streamlined workflow and more direct visual explainability via saliency maps. These findings support the potential of AI-based tools for shoulder XR analysis; however, prospective real-world validation, assessment under routine prevalence conditions, and direct comparison with human readers are still needed before clinical integration can be established., This work was supported by the AI4EOSC project (“Artificial Intelligence for the European Open Science Cloud”), which has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No. 101058593., Peer reviewed