Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 28
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Capturing analytical intents from text

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Dimic, Miona|||0009-0008-1869-3585
  • Bilalli, Besim|||0000-0002-0575-2389
The ability to extract valuable information from data is crucial for organizations and individuals who want to remain competitive in a constantly evolving data-driven environment. However, some of them lack the skills required to appropriately leverage the existing data analytics tools and methods. This problem is aggravated when the users are domain-experts but completely unfamiliar with data analytics terminology, as existing assistant tools, such as AutoML or Intelligent Discovery Assistants, require them to state their analytical intent (i.e., the type of data analysis they want to perform). To address this problem, we propose to capture the underlying analytical intent from textual problem descriptions by leveraging Large Language Models (LLMs). To this end, we propose a hierarchical categorization of analytical intents, along with a data collection methodology to obtain analytical problem descriptions for all of them in order to validate different approaches that aim to extract such intents from text. Next, we compare the performance of state-of-the-art approaches with LLMs, and then study the performance of different LLMs based on their characteristics and the impact of the source of validation data. Finally, we develop a prototype to showcase how our method could interact with existing AutoML systems., Gerard Pons is supported by the EU’s Horizon Programme call, under Grant Agreement No. 101093164 (ExtremeXP), and Besim Bilalli is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia i Innovación under the funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Evaluating quality of disparate data sources: A discord-driven approach

  • Akter, Yeasmin Ara|||0000-0002-9865-0700
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Sagi, Tomer
  • Hose, Katja
Among other measures of data quality, determining the reliability of conflicting values from different sources is especially challenging. Traditional data fusion approaches often infer correct values in simple cases, but struggle to handle variations in data granularity (such as differences in temporal, spatial, or categorical aggregations) and offer limited insight into the nature of disagreements. Thus, we propose a new source evaluation approach for numerical attributes that measures discordance (i.e., the extent to which sources differ from each other). Unlike existing methods that focus solely on point estimation, we allow both fine-grained and coarse-grained analysis, allowing more sophisticated data quality assessments. We employ a linear programming solver that transparently adapts to any data alignment expressed in a set of operators resembling relational algebra. Extensive experiments on real-world datasets demonstrate that our method generalizes existing truth discovery techniques measuring differences with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and can adapt to diverse and complex scenarios., Y. A. Akter is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). A. Abelló and P. Jovanovic are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Hyppo: efficient discovery and execution of data science pipelines in collaborative environments

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
In exploratory data science and machine learning (ML), developing an effective and efficient solution involves the exploration of numerous pipelines per dataset, considering various combinations of data preprocessing, feature selection, model selection, evaluation metrics, etc. This is an iterative process of trial and error, where a pipeline is revised and refined until a satisfactory level of performance is achieved. Such a process can be substantially improved by leveraging collaboration opportunities and automated pipeline optimization. We demonstrate Hyppo, a scalable collaboration system designed for interactive discovery and automated optimization, resulting in a system that supports evaluating and sharing pipelines with minimal overhead. Hyppo takes advantage of numerous optimization possibilities as exploiting equivalences among tasks, sharing computations, artifact materialization, and reuse from past executions to derive better execution plans. We showcase Hyppo with open data and pipelines from popular public code repositories and catalogs. Demo video: https://youtu.be/iilPV_xtXwM, This work has been partially supported by the H2020-MSCA-ITN-2020 DEDS (GA.955895), the European Union’s Horizon Europe Research and Innovation programmes DataGEMS (GA.101188 416), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191 RB-I00/AEI/10.13039/50 1100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Capturing and anticipating user intents in data analytics via knowledge graphs

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
In today’s data-driven world, the ability to extract meaningful information from data is becoming essential for businesses, organizations and researchers. For this purpose, a wide range of tools and systems exists addressing data-related tasks, from data integration, preprocessing and modeling, to the interpretation and evaluation of the results. As data continues to grow in volume and complexity, there is an increasing need for advanced yet user-friendly tools, such as intelligent discovery assistants (IDAs) or automated machine learning (AutoML) systems, that facilitate the user’s interaction with data. This enables non-expert users to effectively leverage powerful data analytics techniques. However, the use of these tools still requires non-trivial user input that cannot be anticipated from the analytical problem’s data alone, but must be tailored to each individual user and their specific intents. To this end, this work explores the use of Knowledge Graphs (KG) as a foundational representation for capturing complex analytics workflows, as well as information about the users, their intents and their feedback, in order to facilitate user interaction with IDAs or AutoML tools. This is achieved through established techniques from recommender systems, in particular a link prediction approach based on KG embeddings and graph neural networks. Experimental results show that the proposed method effectively captures the graph structure and produces meaningful suggestions for users. To demonstrate the feasibility of the approach, a working prototype is presented., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OAI00/AEI/10.13039/501100011033 (TALC)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Discovery of semantic non-syntactic joins

  • Maynou Yelamos, Marc|||0009-0004-3908-4392
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
Data discovery is an essential step in the data integration pipeline involving finding datasets whose combined information provides relevant insights. Discovering joinable attributes requires assessing the closeness of the semantic concepts that two attributes represent, which is highly sensitive and dependent on the chosen similarity metric. The state of the art commonly approaches this task from a syntactic perspective, this is, performing comparisons based on the data values or on direct transformations (e.g., via hash functions). These approaches suffice when the two sets of instances share the same syntactic representation, but fail to detect cases in which the same semantic concept is represented by different sets of values, which we refer as semantic non-syntactic joins. This is a relevant problem in data lake scenarios, when the underlying datasets present high heterogeneity and lack of standardization. To that end, in this paper, we propose an empirical approach to detect semantic non-syntactic joins, which leverages, simultaneously, syntactic and semantic measurements of the data. We demonstrate that our approach is effective in detecting such kind of joins., Marc Maynou is supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP), and Sergi Nadal is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Towards continuous experiment-driven MLOps

  • Rajenthiram, Keerthiga
  • Abdullah, Milad
  • Gerostathopoulos, Ilias
  • Hnetynka, Petr
  • Bureš, Tomáš
  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Despite advancements in MLOps and AutoML, ML development still remains challenging for data scientists. First, there is poor support for and limited control over optimizing and evolving ML models. Second, there is lack of efficient mechanisms for continuous evolution of ML models which would leverage the knowledge gained in previous optimizations of the same or different models. We propose an experiment-driven MLOps approach which tackles these problems. Our approach relies on the concept of an experiment, which embodies a fully controllable optimization process. It introduces full traceability and repeatability to the optimization process, allows humans to be in full control of it, and enables continuous improvement of the ML system. Importantly, it also establishes knowledge, which is carried over and built across a series of experiments and allows for improving the efficiency of experimentation over time. We demonstrate our approach through its realization and application in the ExtremeXp11https://extremexp.eu/ project (Horizon Europe)., This work has been partially supported by the EU project ExtremeXP grant agreement 101093164, by Charles University institutional funding SVV 260698, and by the Spanish Ministerio de Ciencia e Innovación under project PID2020- 117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Mitigating data sparsity in integrated data through text conceptualization

  • Rahman, Md Ataur|||0000-0002-9092-8771
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Sacharidis, Dimitris
We study the data sparsity problem for data generated from an integration system. We approach the problem from a textual information extraction perspective and propose to conceptualize external documents using the concepts in the integrated schema. We present THOR, a novel system that, unlike related approaches, neither relies on complex rules nor models trained with large annotated corpus, but on the integrated data and its schema without the need for human annotations. An extensive evaluation on the text conceptualization task demonstrates the superiority of our approach in terms of F1-score, effort and use of resources over the state-of-the-art language models., Md Ataur Rahman is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). Sergi Nadal and Oscar Romero are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Reproducible experiments for generating pre-processing pipelines for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim|||0000-0002-0575-2389
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Silva Coira, Fernando
  • de Bernardo Roca, Guillermo
This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Diaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied., This work is partially supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP) and the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the project/funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Gestió de dades massives

  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
Aquest article pretén donar una visió general del que és la gestió de dades massives, la seva problemàtica i com s’han d’abordar les solucions. La principal dificultat és que no existeix una solució genèrica i s’ha de construir a mida de cada organització., Aquest treball està parcialment finançat per la Comissió Europea sota l’acord 101093164, corresponent al projecte ExtremeXP., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

There is no data science without data governance: a proposal based on knowledge graphs

  • Bilalli, Besim|||0000-0002-0575-2389
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
Data Science and data-driven Artificial Intelligence are here to stay and they are expected to further transform the current global economy. From a technical point of view, there is an overall agreement that disciplines based on data require to combine data engineering and data analysis skills, but the fact is that data engineering is nowadays trailing and catching up with the rapid changes in the data analysis landscape. To unleash the real power of data, data-centric systems must be professionalized, i.e., operationalized and systematized, so that repetitive, time-consuming and error-prone tasks are automated. To such end, we propose our vision on next generation data governance for data-centric systems based on knowledge graphs. We claim that without the knowledge embedded in the data governance layer, Data Science will not unleash its potential., This work is supported by the Horizon Europe Programme under GA.101135513 (CYCLOPS) and GA. 101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 / AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Knowledge graphs for enhancing large language models in entity disambiguation

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities’ classes in a KG to gradually prune the candidate space as well as the entities’ descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG’s semantic expressivity on the ED performance., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML). Anna Queralt is a Serra Húnter Fellow., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos|||0000-0001-7015-9517
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Improving continual knowledge graph embedding via informed initialization

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Knowledge Graphs (KGs) represent knowledge as structured graphs of entities and their relationships. Knowledge Graph Embeddings (KGEs) enable learning and inference over KGs by representing their entities and relations as continuous vectors. Many KGs are frequently updated, requiring their KGEs to adapt to these changes. Continual learning methods for KGEs address this by incorporating new embeddings while updating existing ones to account for the new facts without retraining from scratch. A necessary step in these methods is the initialization of new embeddings, which has an important impact both on the accuracy of the resulting embeddings, and in the time required for their training. Current continual learning methods either use a random initialization as in non-continual scenarios, ignoring relevant information in existing embeddings, or rely on a model-dependent initialization, which is only compatible with specific KGE training models. We propose a novel informed embedding initialization strategy that can be seamlessly integrated into existing continual learning methods for KGEs. Our approach leverages the KG schema along with previously learned embeddings to generate semantically informed initial representations at every continual learning step, enhancing knowledge acquisition while reducing catastrophic forgetting in the final embeddings obtained. Experimental results show that our strategy improves the predictive performance of the resulting KGEs and enhances knowledge retention. Moreover, it accelerates the learning process, reducing the number of epochs, and thus the time required to incrementally learn new embeddings. Finally, our evaluation demonstrates that these benefits are consistent regardless of the KGE learning model used., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Discovery of semantic non-syntactic joins

  • Maynou Yelamos, Marc
  • Nadal Francesch, Sergi
Data discovery is an essential step in the data integration pipeline involving finding datasets whose combined information provides relevant insights. Discovering joinable attributes requires assessing the closeness of the semantic concepts that two attributes represent, which is highly sensitive and dependent on the chosen similarity metric. The state of the art commonly approaches this task from a syntactic perspective, this is, performing comparisons based on the data values or on direct transformations (e.g., via hash functions). These approaches suffice when the two sets of instances share the same syntactic representation, but fail to detect cases in which the same semantic concept is represented by different sets of values, which we refer as semantic non-syntactic joins. This is a relevant problem in data lake scenarios, when the underlying datasets present high heterogeneity and lack of standardization. To that end, in this paper, we propose an empirical approach to detect semantic non-syntactic joins, which leverages, simultaneously, syntactic and semantic measurements of the data. We demonstrate that our approach is effective in detecting such kind of joins., Marc Maynou is supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP), and Sergi Nadal is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

There is no data science without data governance: a proposal based on knowledge graphs

  • Bilalli, Besim
  • Jovanovic, Petar
  • Nadal Francesch, Sergi
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
Data Science and data-driven Artificial Intelligence are here to stay and they are expected to further transform the current global economy. From a technical point of view, there is an overall agreement that disciplines based on data require to combine data engineering and data analysis skills, but the fact is that data engineering is nowadays trailing and catching up with the rapid changes in the data analysis landscape. To unleash the real power of data, data-centric systems must be professionalized, i.e., operationalized and systematized, so that repetitive, time-consuming and error-prone tasks are automated. To such end, we propose our vision on next generation data governance for data-centric systems based on knowledge graphs. We claim that without the knowledge embedded in the data governance layer, Data Science will not unleash its potential., This work is supported by the Horizon Europe Programme under GA.101135513 (CYCLOPS) and GA. 101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 / AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards continuous experiment-driven MLOps

  • Rajenthiram, Keerthiga
  • Abdullah, Milad
  • Gerostathopoulos, Ilias
  • Hnetynka, Petr
  • Bureš, Tomáš
  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
Despite advancements in MLOps and AutoML, ML development still remains challenging for data scientists. First, there is poor support for and limited control over optimizing and evolving ML models. Second, there is lack of efficient mechanisms for continuous evolution of ML models which would leverage the knowledge gained in previous optimizations of the same or different models. We propose an experiment-driven MLOps approach which tackles these problems. Our approach relies on the concept of an experiment, which embodies a fully controllable optimization process. It introduces full traceability and repeatability to the optimization process, allows humans to be in full control of it, and enables continuous improvement of the ML system. Importantly, it also establishes knowledge, which is carried over and built across a series of experiments and allows for improving the efficiency of experimentation over time. We demonstrate our approach through its realization and application in the ExtremeXp11https://extremexp.eu/ project (Horizon Europe)., This work has been partially supported by the EU project ExtremeXP grant agreement 101093164, by Charles University institutional funding SVV 260698, and by the Spanish Ministerio de Ciencia e Innovación under project PID2020- 117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto
  • Nadal Francesch, Sergi
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Reproducible experiments for generating pre-processing pipelines for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim
  • Abelló Gamazo, Alberto
  • Silva Coira, Fernando
  • de Bernardo Roca, Guillermo
This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Diaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied., This work is partially supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP) and the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the project/funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Knowledge graphs for enhancing large language models in entity disambiguation

  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities’ classes in a KG to gradually prune the candidate space as well as the entities’ descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG’s semantic expressivity on the ED performance., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML). Anna Queralt is a Serra Húnter Fellow., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Gestió de dades massives

  • Abelló Gamazo, Alberto
  • Bilalli, Besim
Aquest article pretén donar una visió general del que és la gestió de dades massives, la seva problemàtica i com s’han d’abordar les solucions. La principal dificultat és que no existeix una solució genèrica i s’ha de construir a mida de cada organització., Aquest treball està parcialment finançat per la Comissió Europea sota l’acord 101093164, corresponent al projecte ExtremeXP., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Improving continual knowledge graph embedding via informed initialization

  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
Knowledge Graphs (KGs) represent knowledge as structured graphs of entities and their relationships. Knowledge Graph Embeddings (KGEs) enable learning and inference over KGs by representing their entities and relations as continuous vectors. Many KGs are frequently updated, requiring their KGEs to adapt to these changes. Continual learning methods for KGEs address this by incorporating new embeddings while updating existing ones to account for the new facts without retraining from scratch. A necessary step in these methods is the initialization of new embeddings, which has an important impact both on the accuracy of the resulting embeddings, and in the time required for their training. Current continual learning methods either use a random initialization as in non-continual scenarios, ignoring relevant information in existing embeddings, or rely on a model-dependent initialization, which is only compatible with specific KGE training models. We propose a novel informed embedding initialization strategy that can be seamlessly integrated into existing continual learning methods for KGEs. Our approach leverages the KG schema along with previously learned embeddings to generate semantically informed initial representations at every continual learning step, enhancing knowledge acquisition while reducing catastrophic forgetting in the final embeddings obtained. Experimental results show that our strategy improves the predictive performance of the resulting KGEs and enhances knowledge retention. Moreover, it accelerates the learning process, reducing the number of epochs, and thus the time required to incrementally learn new embeddings. Finally, our evaluation demonstrates that these benefits are consistent regardless of the KGE learning model used., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/ AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Capturing analytical intents from text

  • Pons Recasens, Gerard
  • Dimic, Miona
  • Bilalli, Besim
The ability to extract valuable information from data is crucial for organizations and individuals who want to remain competitive in a constantly evolving data-driven environment. However, some of them lack the skills required to appropriately leverage the existing data analytics tools and methods. This problem is aggravated when the users are domain-experts but completely unfamiliar with data analytics terminology, as existing assistant tools, such as AutoML or Intelligent Discovery Assistants, require them to state their analytical intent (i.e., the type of data analysis they want to perform). To address this problem, we propose to capture the underlying analytical intent from textual problem descriptions by leveraging Large Language Models (LLMs). To this end, we propose a hierarchical categorization of analytical intents, along with a data collection methodology to obtain analytical problem descriptions for all of them in order to validate different approaches that aim to extract such intents from text. Next, we compare the performance of state-of-the-art approaches with LLMs, and then study the performance of different LLMs based on their characteristics and the impact of the source of validation data. Finally, we develop a prototype to showcase how our method could interact with existing AutoML systems., Gerard Pons is supported by the EU’s Horizon Programme call, under Grant Agreement No. 101093164 (ExtremeXP), and Besim Bilalli is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia i Innovación under the funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Evaluating quality of disparate data sources: A discord-driven approach

  • Akter, Yeasmin Ara
  • Abelló Gamazo, Alberto
  • Jovanovic, Petar
  • Sagi, Tomer
  • Hose, Katja
Among other measures of data quality, determining the reliability of conflicting values from different sources is especially challenging. Traditional data fusion approaches often infer correct values in simple cases, but struggle to handle variations in data granularity (such as differences in temporal, spatial, or categorical aggregations) and offer limited insight into the nature of disagreements. Thus, we propose a new source evaluation approach for numerical attributes that measures discordance (i.e., the extent to which sources differ from each other). Unlike existing methods that focus solely on point estimation, we allow both fine-grained and coarse-grained analysis, allowing more sophisticated data quality assessments. We employ a linear programming solver that transparently adapts to any data alignment expressed in a set of operators resembling relational algebra. Extensive experiments on real-world datasets demonstrate that our method generalizes existing truth discovery techniques measuring differences with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and can adapt to diverse and complex scenarios., Y. A. Akter is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). A. Abelló and P. Jovanovic are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Capturing and anticipating user intents in data analytics via knowledge graphs

  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
In today’s data-driven world, the ability to extract meaningful information from data is becoming essential for businesses, organizations and researchers. For this purpose, a wide range of tools and systems exists addressing data-related tasks, from data integration, preprocessing and modeling, to the interpretation and evaluation of the results. As data continues to grow in volume and complexity, there is an increasing need for advanced yet user-friendly tools, such as intelligent discovery assistants (IDAs) or automated machine learning (AutoML) systems, that facilitate the user’s interaction with data. This enables non-expert users to effectively leverage powerful data analytics techniques. However, the use of these tools still requires non-trivial user input that cannot be anticipated from the analytical problem’s data alone, but must be tailored to each individual user and their specific intents. To this end, this work explores the use of Knowledge Graphs (KG) as a foundational representation for capturing complex analytics workflows, as well as information about the users, their intents and their feedback, in order to facilitate user interaction with IDAs or AutoML tools. This is achieved through established techniques from recommender systems, in particular a link prediction approach based on KG embeddings and graph neural networks. Experimental results show that the proposed method effectively captures the graph structure and produces meaningful suggestions for users. To demonstrate the feasibility of the approach, a working prototype is presented., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OAI00/AEI/10.13039/501100011033 (TALC)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Hyppo: efficient discovery and execution of data science pipelines in collaborative environments

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto
  • Nadal Francesch, Sergi
In exploratory data science and machine learning (ML), developing an effective and efficient solution involves the exploration of numerous pipelines per dataset, considering various combinations of data preprocessing, feature selection, model selection, evaluation metrics, etc. This is an iterative process of trial and error, where a pipeline is revised and refined until a satisfactory level of performance is achieved. Such a process can be substantially improved by leveraging collaboration opportunities and automated pipeline optimization. We demonstrate Hyppo, a scalable collaboration system designed for interactive discovery and automated optimization, resulting in a system that supports evaluating and sharing pipelines with minimal overhead. Hyppo takes advantage of numerous optimization possibilities as exploiting equivalences among tasks, sharing computations, artifact materialization, and reuse from past executions to derive better execution plans. We showcase Hyppo with open data and pipelines from popular public code repositories and catalogs. Demo video: https://youtu.be/iilPV_xtXwM, This work has been partially supported by the H2020-MSCA-ITN-2020 DEDS (GA.955895), the European Union’s Horizon Europe Research and Innovation programmes DataGEMS (GA.101188 416), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191 RB-I00/AEI/10.13039/50 1100011033 (DOGO4ML)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mitigating data sparsity in integrated data through text conceptualization

  • Rahman, Md Ataur
  • Nadal Francesch, Sergi
  • Romero Moral, Óscar
  • Sacharidis, Dimitris
We study the data sparsity problem for data generated from an integration system. We approach the problem from a textual information extraction perspective and propose to conceptualize external documents using the concepts in the integrated schema. We present THOR, a novel system that, unlike related approaches, neither relies on complex rules nor models trained with large annotated corpus, but on the integrated data and its schema without the need for human annotations. An extensive evaluation on the text conceptualization task demonstrates the superiority of our approach in terms of F1-score, effort and use of resources over the state-of-the-art language models., Md Ataur Rahman is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). Sergi Nadal and Oscar Romero are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed, Postprint (author's final draft)