Pasar al contenido principal

HACIA UN CICLO DE VIDA AUTOMATIZADO DE DATOS CENTRADO EN LA IA

PID2023-152841OA-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2023
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 18
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Operationalizing and automating data validation in data spaces

  • Hmimou Ham Man, Achraf|||0009-0004-0448-3406
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Data spaces have recently emerged as an innovative paradigm for cross-organizational data sharing. These decentralized environments require sophisticated data governance protocols to ensure compliance with data standards, roles and policies. While current policy-based solutions address enforcement of data access control and usage rights, they lack mechanisms for automated data validation -essential for ensuring data quality for collaborative analytics. To address this gap, we present a knowledge graph-based framework to automate data validation inline with data policies. This framework relies on the concept of policy checkers, which represent high-level and technology-agnostic data validation plans that can be dynamically translated into technology-specific user defined functions (UDFs) for compliance checking. Importantly, the usage of knowledge graphs to describe the policy checkers enhances the transparency and traceability of data validation processes, while the two-stage process (technology-agnostic policy checkers and technology-specific UDFs) accommodate data validation on multimodal data. We accompany the description of our approach with a proof of concept that demonstrates the feasibility of this solution in real data spaces., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

CAPS: Cost-Aware ML Pipeline Selection

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Simitsis, Alkis
We present CAPS, a novel system for cost-aware pipeline selection in automated machine learning (AutoML). CAPS’ approach is orthogonal to the search strategies used by existing AutoML frameworks, enabling seamless integration with them. This integration benefits AutoML systems by reducing computational waste—time spent evaluating inefficient pipelines or exceeding time or memory constraints—through lightweight performance and cost estimation. To incorporate cost-awareness into AutoML, CAPS represents a set of pipelines as a directed hypergraph, estimates execution times for each function, and formulates the optimal subset selection as a constrained prize-collecting optimization problem. CAPS employs a simple yet effective greedy algorithm to approximate this optimal subset. Our evaluation shows that when CAPS prioritizes cost, it reduces waste up to 4×—saving up to 23 hours in 50 hours of pipeline evaluation. When CAPS balances the trade-off between performance and cost, it consistently achieves performance improvements of 5% to 15% over the state of the art within the same time budget across datasets. It also demonstrates that CAPS is compatible with complementary efficiency techniques such as early stopping and warm-starting, achieving speedups of up to 4.7×., This work has been partially supported by the H2020-MSCA-ITN2020 DEDS project (GA 955895), the EU Horizon Europe research and innovation programmes DataGEMS (GA 101188416), WiseFood (GA 101181895), and CREXDATA (GA 101092749), MLEvol (PID2024-156019OB-I00/AEI/10.13039/501100011033) , TALC (PID2023-152841/OA-I00/AEI/10.13039/501100011033) and by ERDF, EU. The computation infrastructure has been provided by the Consortium des Équipements de Calcul Intensif (CÉCI), funded by the Fonds de la Recherche Scientifique de Belgique (F.R.S.-FNRS) under Grant No. 2.5020.11 and by the Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Capturing and anticipating user intents in data analytics via knowledge graphs

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
In today’s data-driven world, the ability to extract meaningful information from data is becoming essential for businesses, organizations and researchers. For this purpose, a wide range of tools and systems exists addressing data-related tasks, from data integration, preprocessing and modeling, to the interpretation and evaluation of the results. As data continues to grow in volume and complexity, there is an increasing need for advanced yet user-friendly tools, such as intelligent discovery assistants (IDAs) or automated machine learning (AutoML) systems, that facilitate the user’s interaction with data. This enables non-expert users to effectively leverage powerful data analytics techniques. However, the use of these tools still requires non-trivial user input that cannot be anticipated from the analytical problem’s data alone, but must be tailored to each individual user and their specific intents. To this end, this work explores the use of Knowledge Graphs (KG) as a foundational representation for capturing complex analytics workflows, as well as information about the users, their intents and their feedback, in order to facilitate user interaction with IDAs or AutoML tools. This is achieved through established techniques from recommender systems, in particular a link prediction approach based on KG embeddings and graph neural networks. Experimental results show that the proposed method effectively captures the graph structure and produces meaningful suggestions for users. To demonstrate the feasibility of the approach, a working prototype is presented., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OAI00/AEI/10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Supporting data discovery tasks at scale with FREYJA

  • Maynou Yelamos, Marc|||0009-0004-3908-4392
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
Data lakes, massive repositories of heterogeneous data, were popularized by implementing a load-first model-later approach to data integration in contrast to the traditional model-first loadlater implemented in data warehouses. Yet, such settings create new challenges related to a) how to effectively discover relevant data and automatically find meaningful relationships; and b) how to do so efficiently at scale. In order to address them, we present Freyja, a data discovery system designed to navigate through vasts amounts of data and support users on finding joinable pairs of attributes. Freyja is based on the definition of a novel similarity measure, designed to automatically discovery semantically-relevant relationships. The adopted metric is tailored to data lake contexts, where datasets are denormalized and data from several domains are stored. In order to optimize and scale the computation of such metric, Freyja adopts a learning approach relying on profiles, succinct and lightweight representations of the underlying characteristics of the data attributes. Freyja leverages analytical databases to efficiently compute column profiles. In this demo, we demonstrate how Freyja, which is offered as a Python library, supports data augmentation tasks (i.e., augmenting training datasets with new features to improve the performance of a machine learning model)., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00 / AEI/10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Towards fair machine learning using many-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. B. Bilalli is partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/501100011033 (TALC). G. Bontempi is supported by the Service Public de Wallonie Recherche, Spain under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Enhancing data discovery with contextual pre-filtering

  • Flores Herrera, Javier de Jesús|||0000-0002-2998-9962
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
In this paper, entity contextual pre-filtering is proposed to refine dataset relevance assessment and streamline data discovery. Heterogeneous Graph Neural Networks are used to exploit the local context embedded within graph-based schemas. The proposed pre-filtering approach is versatile and does not rely on any specific similarity metric, making it applicable to a wide range of data discovery methods. The proposed technique increases data discovery precision by reducing false positives and identifying significant data relationships. This method has been empirically validated across a variety of real-world datasets to improve data discovery efficiency and accuracy., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00 / AEI / 10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

A framework for continual knowledge graph embedding

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Knowledge Graph Embeddings (KGEs) are widely used to support downstream tasks over Knowledge Graphs (KGs) by representing entities and relations in a low-dimensional vector space. However, real-world KGs are not static, as new facts are continuously added, requiring old embeddings to be updated and new embeddings to be learned without forgetting previously acquired information. Existing KGE libraries focus on the static KG scenario. Moreover, while recent works on Continual Knowledge Graph Embedding (CKGE) propose strategies to address this problem, they rely on standalone frameworks and only support simple KGE models, such as TransE, limiting their potential. We extended the KGE library PyKEEN to enable continual learning in KGs. Our extension (i) expands any unimodal KGE model in PyKEEN with new entities and relations, (ii) introduces an informed initialization method to improve the addition of new embeddings, (iii) incorporates CKGE-oriented evaluation metrics to assess both performance and knowledge forgetting and acquisition, and (iv) offers an intuitive training function with dashboard support, enabling easy experimentation and tracking of the evolution of embeddings across updates. A repository containing the scripts, an interactive notebook tutorial, and an accompanying video are available., This work is supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/50110 0011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Improving continual knowledge graph embedding via informed initialization

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Knowledge Graphs (KGs) represent knowledge as structured graphs of entities and their relationships. Knowledge Graph Embeddings (KGEs) enable learning and inference over KGs by representing their entities and relations as continuous vectors. Many KGs are frequently updated, requiring their KGEs to adapt to these changes. Continual learning methods for KGEs address this by incorporating new embeddings while updating existing ones to account for the new facts without retraining from scratch. A necessary step in these methods is the initialization of new embeddings, which has an important impact both on the accuracy of the resulting embeddings, and in the time required for their training. Current continual learning methods either use a random initialization as in non-continual scenarios, ignoring relevant information in existing embeddings, or rely on a model-dependent initialization, which is only compatible with specific KGE training models. We propose a novel informed embedding initialization strategy that can be seamlessly integrated into existing continual learning methods for KGEs. Our approach leverages the KG schema along with previously learned embeddings to generate semantically informed initial representations at every continual learning step, enhancing knowledge acquisition while reducing catastrophic forgetting in the final embeddings obtained. Experimental results show that our strategy improves the predictive performance of the resulting KGEs and enhances knowledge retention. Moreover, it accelerates the learning process, reducing the number of epochs, and thus the time required to incrementally learn new embeddings. Finally, our evaluation demonstrates that these benefits are consistent regardless of the KGE learning model used., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Freyja: Efficient join discovery in data lakes

  • Maynou Yelamos, Marc|||0009-0004-3908-4392
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Panadero Palenzuela, Raquel
  • Flores Herrera, Javier de Jesús|||0000-0002-2998-9962
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Queralt Calafat, Anna|||0000-0003-2782-2955
We study the problem of efficiently computing rankings of joinable attributes in data lakes. Traditional set-overlap measures produce numerous false positives in this scenario, while modern, more accurate Table Representation Learning (TRL) techniques incur prohibitive computational costs. In contrast to the state-of-the-art, we adopt a novel notion of join quality tailored to data lakes relying on a metric that combines multiset Jaccard and cardinality proportion. The proposed metric merges the best of both worlds by leveraging syntactic measures while achieving accuracy scores comparable to those of TRL approaches. Generating rankings of joinable pairs is highly scalable at both preparation and query time, since we train a general-purpose predictive model. Predictions are based on data profiles, succinct and efficiently computed representations of dataset characteristics. Our experiments show that our system, Freyja, matches and improves upon, the results obtained by the state-of-the-art while reducing execution costs by orders of magnitude., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovacion under project PID2023-152841OA-I00 / AEI/10.13039/501100011033 (TALC). Anna Queralt is a Serra Hunter Fellow., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Offloading artificial intelligence workloads across the computing continuum by means of active storage systems

  • Barceló Cuerda, Alex|||0000-0002-0877-5738
  • Cajas Ordoñez, Sebastián A.
  • Samanta, Jaydeep
  • Suárez Cetrulo, Andrés L.
  • Ghosh, Romila
  • Simón Carbajo, Ricardo
  • Queralt Calafat, Anna|||0000-0003-2782-2955
The increasing demand for artificial intelligence (AI) workloads across diverse computing environments has driven the need for more efficient data management strategies. Traditional cloud-based architectures struggle to handle the sheer volume and velocity of AI-driven data, leading to inefficiencies in storage, computation, and data movement. This paper explores the integration of active storage systems within the computing continuum to optimize AI workload distribution. By embedding computation directly into storage architectures, active storage is able to reduce data transfer overhead, enhancing performance and improving resource utilization. Other existing frameworks and architectures offer mechanisms to distribute certain AI processes across distributed environments; however, they lack the flexibility and adaptability that the continuum requires, both regarding the heterogeneity of devices and the rapid-changing algorithms and models being used by domain experts and researchers. This article proposes a software architecture aimed at seamlessly distributing AI workloads across the computing continuum, and presents its implementation using mainstream Python libraries and dataClay, an active storage platform. The evaluation shows the benefits and trade-offs regarding memory consumption, storage requirements, training times, and execution efficiency across different devices. Experimental results demonstrate that the process of offloading workloads through active storage significantly improves memory efficiency and training speeds while maintaining accuracy. Our findings highlight the potential of active storage to revolutionize AI workload management, making distributed AI deployments more scalable and resource-efficient with a very low entry barrier for domain experts and application developers., This work was partially supported by the projects PID2019107255GB-C21 and PID2023-147979NB-C21, funded by MCIN/AEI/ 10.13039 / 501100011033, and by FEDER, UE. This work was partially supported by the project “Towards a functional continuum operating system (ICOS)” funded by the European Commission under Project code/Grant Number 101070177 through the HORIZON EU program. This work was partially supported by the project “Open CloudEdgeIoT Platform Uptake in Large Scale Cross-Domain Pilots (O-CEI)” funded by the European Commission under Project code/Grant Number 101189589 through the HORIZON EUprogram. Anna Queralt is a Serra-Hunter Fellow and has been partially supported by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2023-152841OAI00 / AEI/10.13039/501100011033 (TALC)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Freyja: Efficient join discovery in data lakes

  • Maynou Yelamos, Marc
  • Nadal Francesch, Sergi
  • Panadero Palenzuela, Raquel
  • Flores Herrera, Javier de Jesús
  • Romero Moral, Óscar
  • Queralt Calafat, Anna
We study the problem of efficiently computing rankings of joinable attributes in data lakes. Traditional set-overlap measures produce numerous false positives in this scenario, while modern, more accurate Table Representation Learning (TRL) techniques incur prohibitive computational costs. In contrast to the state-of-the-art, we adopt a novel notion of join quality tailored to data lakes relying on a metric that combines multiset Jaccard and cardinality proportion. The proposed metric merges the best of both worlds by leveraging syntactic measures while achieving accuracy scores comparable to those of TRL approaches. Generating rankings of joinable pairs is highly scalable at both preparation and query time, since we train a general-purpose predictive model. Predictions are based on data profiles, succinct and efficiently computed representations of dataset characteristics. Our experiments show that our system, Freyja, matches and improves upon, the results obtained by the state-of-the-art while reducing execution costs by orders of magnitude., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovacion under project PID2023-152841OA-I00 / AEI/10.13039/501100011033 (TALC). Anna Queralt is a Serra Hunter Fellow., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Offloading artificial intelligence workloads across the computing continuum by means of active storage systems

  • Barceló Cuerda, Alex
  • Cajas Ordoñez, Sebastián A.
  • Samanta, Jaydeep
  • Suárez Cetrulo, Andrés L.
  • Ghosh, Romila
  • Simón Carbajo, Ricardo
  • Queralt Calafat, Anna
The increasing demand for artificial intelligence (AI) workloads across diverse computing environments has driven the need for more efficient data management strategies. Traditional cloud-based architectures struggle to handle the sheer volume and velocity of AI-driven data, leading to inefficiencies in storage, computation, and data movement. This paper explores the integration of active storage systems within the computing continuum to optimize AI workload distribution. By embedding computation directly into storage architectures, active storage is able to reduce data transfer overhead, enhancing performance and improving resource utilization. Other existing frameworks and architectures offer mechanisms to distribute certain AI processes across distributed environments; however, they lack the flexibility and adaptability that the continuum requires, both regarding the heterogeneity of devices and the rapid-changing algorithms and models being used by domain experts and researchers. This article proposes a software architecture aimed at seamlessly distributing AI workloads across the computing continuum, and presents its implementation using mainstream Python libraries and dataClay, an active storage platform. The evaluation shows the benefits and trade-offs regarding memory consumption, storage requirements, training times, and execution efficiency across different devices. Experimental results demonstrate that the process of offloading workloads through active storage significantly improves memory efficiency and training speeds while maintaining accuracy. Our findings highlight the potential of active storage to revolutionize AI workload management, making distributed AI deployments more scalable and resource-efficient with a very low entry barrier for domain experts and application developers., This work was partially supported by the projects PID2019107255GB-C21 and PID2023-147979NB-C21, funded by MCIN/AEI/ 10.13039 / 501100011033, and by FEDER, UE. This work was partially supported by the project “Towards a functional continuum operating system (ICOS)” funded by the European Commission under Project code/Grant Number 101070177 through the HORIZON EU program. This work was partially supported by the project “Open CloudEdgeIoT Platform Uptake in Large Scale Cross-Domain Pilots (O-CEI)” funded by the European Commission under Project code/Grant Number 101189589 through the HORIZON EUprogram. Anna Queralt is a Serra-Hunter Fellow and has been partially supported by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2023-152841OAI00 / AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Operationalizing and automating data validation in data spaces

  • Hmimou Ham Man, Achraf
  • Jovanovic, Petar
  • Nadal Francesch, Sergi
  • Romero Moral, Óscar
  • Queralt Calafat, Anna
Data spaces have recently emerged as an innovative paradigm for cross-organizational data sharing. These decentralized environments require sophisticated data governance protocols to ensure compliance with data standards, roles and policies. While current policy-based solutions address enforcement of data access control and usage rights, they lack mechanisms for automated data validation -essential for ensuring data quality for collaborative analytics. To address this gap, we present a knowledge graph-based framework to automate data validation inline with data policies. This framework relies on the concept of policy checkers, which represent high-level and technology-agnostic data validation plans that can be dynamically translated into technology-specific user defined functions (UDFs) for compliance checking. Importantly, the usage of knowledge graphs to describe the policy checkers enhances the transparency and traceability of data validation processes, while the two-stage process (technology-agnostic policy checkers and technology-specific UDFs) accommodate data validation on multimodal data. We accompany the description of our approach with a proof of concept that demonstrates the feasibility of this solution in real data spaces., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Enhancing data discovery with contextual pre-filtering

  • Flores Herrera, Javier de Jesús
  • Nadal Francesch, Sergi
  • Romero Moral, Óscar
In this paper, entity contextual pre-filtering is proposed to refine dataset relevance assessment and streamline data discovery. Heterogeneous Graph Neural Networks are used to exploit the local context embedded within graph-based schemas. The proposed pre-filtering approach is versatile and does not rely on any specific similarity metric, making it applicable to a wide range of data discovery methods. The proposed technique increases data discovery precision by reducing false positives and identifying significant data relationships. This method has been empirically validated across a variety of real-world datasets to improve data discovery efficiency and accuracy., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00 / AEI / 10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Supporting data discovery tasks at scale with FREYJA

  • Maynou Yelamos, Marc
  • Nadal Francesch, Sergi
Data lakes, massive repositories of heterogeneous data, were popularized by implementing a load-first model-later approach to data integration in contrast to the traditional model-first loadlater implemented in data warehouses. Yet, such settings create new challenges related to a) how to effectively discover relevant data and automatically find meaningful relationships; and b) how to do so efficiently at scale. In order to address them, we present Freyja, a data discovery system designed to navigate through vasts amounts of data and support users on finding joinable pairs of attributes. Freyja is based on the definition of a novel similarity measure, designed to automatically discovery semantically-relevant relationships. The adopted metric is tailored to data lake contexts, where datasets are denormalized and data from several domains are stored. In order to optimize and scale the computation of such metric, Freyja adopts a learning approach relying on profiles, succinct and lightweight representations of the underlying characteristics of the data attributes. Freyja leverages analytical databases to efficiently compute column profiles. In this demo, we demonstrate how Freyja, which is offered as a Python library, supports data augmentation tasks (i.e., augmenting training datasets with new features to improve the performance of a machine learning model)., This work has been partly supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00 / AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Improving continual knowledge graph embedding via informed initialization

  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
Knowledge Graphs (KGs) represent knowledge as structured graphs of entities and their relationships. Knowledge Graph Embeddings (KGEs) enable learning and inference over KGs by representing their entities and relations as continuous vectors. Many KGs are frequently updated, requiring their KGEs to adapt to these changes. Continual learning methods for KGEs address this by incorporating new embeddings while updating existing ones to account for the new facts without retraining from scratch. A necessary step in these methods is the initialization of new embeddings, which has an important impact both on the accuracy of the resulting embeddings, and in the time required for their training. Current continual learning methods either use a random initialization as in non-continual scenarios, ignoring relevant information in existing embeddings, or rely on a model-dependent initialization, which is only compatible with specific KGE training models. We propose a novel informed embedding initialization strategy that can be seamlessly integrated into existing continual learning methods for KGEs. Our approach leverages the KG schema along with previously learned embeddings to generate semantically informed initial representations at every continual learning step, enhancing knowledge acquisition while reducing catastrophic forgetting in the final embeddings obtained. Experimental results show that our strategy improves the predictive performance of the resulting KGEs and enhances knowledge retention. Moreover, it accelerates the learning process, reducing the number of epochs, and thus the time required to incrementally learn new embeddings. Finally, our evaluation demonstrates that these benefits are consistent regardless of the KGE learning model used., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Capturing and anticipating user intents in data analytics via knowledge graphs

  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
In today’s data-driven world, the ability to extract meaningful information from data is becoming essential for businesses, organizations and researchers. For this purpose, a wide range of tools and systems exists addressing data-related tasks, from data integration, preprocessing and modeling, to the interpretation and evaluation of the results. As data continues to grow in volume and complexity, there is an increasing need for advanced yet user-friendly tools, such as intelligent discovery assistants (IDAs) or automated machine learning (AutoML) systems, that facilitate the user’s interaction with data. This enables non-expert users to effectively leverage powerful data analytics techniques. However, the use of these tools still requires non-trivial user input that cannot be anticipated from the analytical problem’s data alone, but must be tailored to each individual user and their specific intents. To this end, this work explores the use of Knowledge Graphs (KG) as a foundational representation for capturing complex analytics workflows, as well as information about the users, their intents and their feedback, in order to facilitate user interaction with IDAs or AutoML tools. This is achieved through established techniques from recommender systems, in particular a link prediction approach based on KG embeddings and graph neural networks. Experimental results show that the proposed method effectively captures the graph structure and produces meaningful suggestions for users. To demonstrate the feasibility of the approach, a working prototype is presented., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OAI00/AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Towards fair machine learning using many-objective feature selection

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. B. Bilalli is partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/501100011033 (TALC). G. Bontempi is supported by the Service Public de Wallonie Recherche, Spain under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed, Postprint (published version)