Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 25
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Evaluating quality of disparate data sources: A discord-driven approach

  • Akter, Yeasmin Ara|||0000-0002-9865-0700
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Sagi, Tomer
  • Hose, Katja
Among other measures of data quality, determining the reliability of conflicting values from different sources is especially challenging. Traditional data fusion approaches often infer correct values in simple cases, but struggle to handle variations in data granularity (such as differences in temporal, spatial, or categorical aggregations) and offer limited insight into the nature of disagreements. Thus, we propose a new source evaluation approach for numerical attributes that measures discordance (i.e., the extent to which sources differ from each other). Unlike existing methods that focus solely on point estimation, we allow both fine-grained and coarse-grained analysis, allowing more sophisticated data quality assessments. We employ a linear programming solver that transparently adapts to any data alignment expressed in a set of operators resembling relational algebra. Extensive experiments on real-world datasets demonstrate that our method generalizes existing truth discovery techniques measuring differences with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and can adapt to diverse and complex scenarios., Y. A. Akter is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). A. Abelló and P. Jovanovic are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

CAPS: Cost-Aware ML Pipeline Selection

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Simitsis, Alkis
We present CAPS, a novel system for cost-aware pipeline selection in automated machine learning (AutoML). CAPS’ approach is orthogonal to the search strategies used by existing AutoML frameworks, enabling seamless integration with them. This integration benefits AutoML systems by reducing computational waste—time spent evaluating inefficient pipelines or exceeding time or memory constraints—through lightweight performance and cost estimation. To incorporate cost-awareness into AutoML, CAPS represents a set of pipelines as a directed hypergraph, estimates execution times for each function, and formulates the optimal subset selection as a constrained prize-collecting optimization problem. CAPS employs a simple yet effective greedy algorithm to approximate this optimal subset. Our evaluation shows that when CAPS prioritizes cost, it reduces waste up to 4×—saving up to 23 hours in 50 hours of pipeline evaluation. When CAPS balances the trade-off between performance and cost, it consistently achieves performance improvements of 5% to 15% over the state of the art within the same time budget across datasets. It also demonstrates that CAPS is compatible with complementary efficiency techniques such as early stopping and warm-starting, achieving speedups of up to 4.7×., This work has been partially supported by the H2020-MSCA-ITN2020 DEDS project (GA 955895), the EU Horizon Europe research and innovation programmes DataGEMS (GA 101188416), WiseFood (GA 101181895), and CREXDATA (GA 101092749), MLEvol (PID2024-156019OB-I00/AEI/10.13039/501100011033) , TALC (PID2023-152841/OA-I00/AEI/10.13039/501100011033) and by ERDF, EU. The computation infrastructure has been provided by the Consortium des Équipements de Calcul Intensif (CÉCI), funded by the Fonds de la Recherche Scientifique de Belgique (F.R.S.-FNRS) under Grant No. 2.5020.11 and by the Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Hyppo: efficient discovery and execution of data science pipelines in collaborative environments

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
In exploratory data science and machine learning (ML), developing an effective and efficient solution involves the exploration of numerous pipelines per dataset, considering various combinations of data preprocessing, feature selection, model selection, evaluation metrics, etc. This is an iterative process of trial and error, where a pipeline is revised and refined until a satisfactory level of performance is achieved. Such a process can be substantially improved by leveraging collaboration opportunities and automated pipeline optimization. We demonstrate Hyppo, a scalable collaboration system designed for interactive discovery and automated optimization, resulting in a system that supports evaluating and sharing pipelines with minimal overhead. Hyppo takes advantage of numerous optimization possibilities as exploiting equivalences among tasks, sharing computations, artifact materialization, and reuse from past executions to derive better execution plans. We showcase Hyppo with open data and pipelines from popular public code repositories and catalogs. Demo video: https://youtu.be/iilPV_xtXwM, This work has been partially supported by the H2020-MSCA-ITN-2020 DEDS (GA.955895), the European Union’s Horizon Europe Research and Innovation programmes DataGEMS (GA.101188 416), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191 RB-I00/AEI/10.13039/50 1100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Towards fair machine learning using many-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. B. Bilalli is partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/501100011033 (TALC). G. Bontempi is supported by the Service Public de Wallonie Recherche, Spain under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

On many-objective feature selection and the need for interpretability

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
Big data comes with the challenge of containing irrelevant and redundant information (i.e., features). Given that a single objective cannot fully capture a feature’s relevance, a Many-Objective Feature Selection (MOFS) approach able to accommodate various relevant perspectives is preferred for identifying the most appropriate features in a given context. However, MOFS produces a large set of solutions whose interpretability has been largely overlooked. First, we demonstrate the relevance of MOFS and establish its necessity by considering up to six objectives using a genetic algorithm and Naive Bayes on ten datasets for classification tasks. Then, we propose a novel methodology to improve the interpretability of MOFS results in order to support the data scientist in selecting the subset of features pertinent to their use case. Our methodology is instantiated as an intuitive and interactive dashboard that provides insights into the results beyond the pure numerical representation of the objectives being considered and evaluated with 50 participants. The outcome shows that it addresses the need for a methodological approach and comprehensive visualization to achieve interoperability., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Workload placement on heterogeneous CPU-GPU systems

  • Nogueira Lobo de Carvalho, Marcos|||0000-0001-7015-9517
  • Simitsis, Alkis
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
The popularity of heterogeneous CPU-GPU processing has increased considerably in recent years. To efficiently utilize heterogeneous resources, data processing systems depend on an appropriate workload placement strategy to assign the right amount of compute to the right processor. However, finding an optimal placement strategy is not trivial due to various complex and conflicting tradeoffs related to the characteristics of processors, the nature of the workload, and data locality. In addition, placement decisions impact workload runtime and performance cost, and also depend on the availability of potentially different implementations for CPUs and GPUs, which adds extra complexity in such heterogeneous environments. In this tutorial, we review and compare state-of-the-art strategies for workload placement on heterogeneous CPU-GPU architectures, along with runtime prediction techniques and methods to support multi-device code. We also discuss open issues and identify potentially promising future research directions., This work has been partially supported by the H2020-MSCAITN2020 DEDS (GA.955895), and the EU-HORIZON programmes CREXDATA (GA.101092749) and FAIR-CORE4EOSC (GA.101057264) and the Spanish MICIU DOGO4ML (PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Mitigating data sparsity in integrated data through text conceptualization

  • Rahman, Md Ataur|||0000-0002-9092-8771
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Sacharidis, Dimitris
We study the data sparsity problem for data generated from an integration system. We approach the problem from a textual information extraction perspective and propose to conceptualize external documents using the concepts in the integrated schema. We present THOR, a novel system that, unlike related approaches, neither relies on complex rules nor models trained with large annotated corpus, but on the integrated data and its schema without the need for human annotations. An extensive evaluation on the text conceptualization task demonstrates the superiority of our approach in terms of F1-score, effort and use of resources over the state-of-the-art language models., Md Ataur Rahman is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). Sergi Nadal and Oscar Romero are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A data-science pipeline to enable the interpretability of many-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
Many-Objective Feature Selection (MOFS) approaches use four or more objectives to determine the relevance of a subset of features in a supervised learning task. As a consequence, MOFS typically returns a large set of non-dominated solutions, which have to be assessed by the data scientist in order to proceed with the final choice. Given the multi-variate nature of the assessment, which may include objectives (e.g., fairness) unrelated to predictive accuracy, this step is often not straightforward and suffers from the lack of existing tools. For instance, it is common to make use of a tabular presentation of the solutions, which provides little information about the trade-offs and the relationships between objectives over the set of solutions. Adopting a GA-based MOFS with six objectives (number of selected features, balanced accuracy, F1-Score, variance inflation factor, statistical parity, and equalised odds) for two feature selection tasks, this paper illustrates the complex challenge of assessing MOFS results and the need for a methodology to aid and justify the final choice of a solution., The project leading to this publication has received funding from the European Union’s Horizon 2020 research and innovation programme (G.A. No. 955895). A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under G.A. No. 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Wrapper methods for multi-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
The ongoing data boom has democratized the use of data for improved decision-making. Beyond gathering voluminous data, preprocessing the data is crucial to ensure that their most rele- vant aspects are considered during the analysis. Feature Selection (FS) is one integral step in data preprocessing for reducing data dimensionality and preserving the most relevant features of the data. FS can be done by inspecting inherent associations among the features in the data (filter methods) or using the model per- formance of a concrete learning algorithm (wrapper methods). In this work, we extensively evaluate a set of FS methods on 32 datasets and measure their effect on model performance, stability, scalability and memory usage. The results re-establish the superiority of wrapper methods over filter methods in model performance. We further investigate the unique role of wrapper methods in multi-objective FS with a focus on two traditional metrics - accuracy and Area Under the ROC Curve (AUC). On model performance, our experiments showed that optimizing for both metrics simultaneously, rather than using a single metric, led to improvements in the accuracy and AUC trade-off1 up to 5% and 10%, respectively., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 955895). Besim Bilalli is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union-Next Generation EU, under the project FJC 2021-046606-I/AEI/10.13039/501100011033. Gianluca Bontempi was supported by Service Public de Wallonie Recherche undergrant n°2010235–ARIAC by DIGITALWALLONIA4.AI., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Impact of filter feature selection on classification: an empirical study

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
The high-dimensionality of Big Data poses challenges in data understanding and visualization. Furthermore, it leads to lengthy model building times in data analysis and poor generalization for machine learning models. Consequently, there is a need for feature selection, which allows identifying the more relevant part of the data to improve the data analysis (e.g., building simpler and more understandable models with reduced training time and improved model performance). This study aims to (i) characterize the factors (i.e., dataset characteristics) that influence the performance of feature selection methods, and (ii) assess the impact of feature selection on the training time and accuracy of binary and multiclass classification problems. As a result, we propose a systematic method to select representative datasets (i.e., considering the distributions of several dataset characteristics) in a given repository. Next, we provide an empirical study of the impact of eight feature selection methods on Naive Bayes (NB), Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), and Multilayer Perceptron (MLP) classification algorithms using 32 real-world datasets and a relative performance measure. We observed that feature selection is more effective in reducing training time (e.g., up to 60% for LDA classifiers) than improving classification accuracy (e.g., up to 5%). Furthermore, we observed that feature selection gives slight accuracy improvement for binary classification (i.e., up to 5%), while it mostly leads to accuracy degradation for multiclass classification. Although none of the studied feature selection methods is best in all cases, for multiclass classification, we observed that correlation based and minimum redundancy maximum relevance feature selection methods gave the best results in accuracy. Through statistical testing, we found LDA and MLP to benefit more in accuracy improvement after feature selection than KNN and NB., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 955895)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos|||0000-0001-7015-9517
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Finding relevant information in big datasets with ML

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
Due to the abundance of data, noisy, irrelevant, or redundant features often need to be identified and discarded. Feature selection is a collection of methods used to ensure that only relevant data are used for a data analysis task. Extracting and using only useful data for analysis promotes model understanding and performance and reduces the model training time and variance, i.e., overfitting. There is an abundance of methods for feature selection, and they can be categorised by various perspectives and are applicable to differing use cases. In this tutorial, we introduce the feature selection problem and present it from three perspectives of categorisation: search strategy, model reliance, and relevance definition. Furthermore, we propose a guideline for the use of the various methods. Lastly, we discuss current challenges and opportunities for research on feature selection., The project leading to this publication has received funding from 2020 research and innovation programme (grant agreement No 955895)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Impact of filter feature selection on classification: an empirical study

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
The high-dimensionality of Big Data poses challenges in data understanding and visualization. Furthermore, it leads to lengthy model building times in data analysis and poor generalization for machine learning models. Consequently, there is a need for feature selection, which allows identifying the more relevant part of the data to improve the data analysis (e.g., building simpler and more understandable models with reduced training time and improved model performance). This study aims to (i) characterize the factors (i.e., dataset characteristics) that influence the performance of feature selection methods, and (ii) assess the impact of feature selection on the training time and accuracy of binary and multiclass classification problems. As a result, we propose a systematic method to select representative datasets (i.e., considering the distributions of several dataset characteristics) in a given repository. Next, we provide an empirical study of the impact of eight feature selection methods on Naive Bayes (NB), Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), and Multilayer Perceptron (MLP) classification algorithms using 32 real-world datasets and a relative performance measure. We observed that feature selection is more effective in reducing training time (e.g., up to 60% for LDA classifiers) than improving classification accuracy (e.g., up to 5%). Furthermore, we observed that feature selection gives slight accuracy improvement for binary classification (i.e., up to 5%), while it mostly leads to accuracy degradation for multiclass classification. Although none of the studied feature selection methods is best in all cases, for multiclass classification, we observed that correlation based and minimum redundancy maximum relevance feature selection methods gave the best results in accuracy. Through statistical testing, we found LDA and MLP to benefit more in accuracy improvement after feature selection than KNN and NB., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 955895)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto
  • Nadal Francesch, Sergi
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Wrapper methods for multi-objective feature selection

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
The ongoing data boom has democratized the use of data for improved decision-making. Beyond gathering voluminous data, preprocessing the data is crucial to ensure that their most rele- vant aspects are considered during the analysis. Feature Selection (FS) is one integral step in data preprocessing for reducing data dimensionality and preserving the most relevant features of the data. FS can be done by inspecting inherent associations among the features in the data (filter methods) or using the model per- formance of a concrete learning algorithm (wrapper methods). In this work, we extensively evaluate a set of FS methods on 32 datasets and measure their effect on model performance, stability, scalability and memory usage. The results re-establish the superiority of wrapper methods over filter methods in model performance. We further investigate the unique role of wrapper methods in multi-objective FS with a focus on two traditional metrics - accuracy and Area Under the ROC Curve (AUC). On model performance, our experiments showed that optimizing for both metrics simultaneously, rather than using a single metric, led to improvements in the accuracy and AUC trade-off1 up to 5% and 10%, respectively., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 955895). Besim Bilalli is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union-Next Generation EU, under the project FJC 2021-046606-I/AEI/10.13039/501100011033. Gianluca Bontempi was supported by Service Public de Wallonie Recherche undergrant n°2010235–ARIAC by DIGITALWALLONIA4.AI., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A data-science pipeline to enable the interpretability of many-objective feature selection

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
Many-Objective Feature Selection (MOFS) approaches use four or more objectives to determine the relevance of a subset of features in a supervised learning task. As a consequence, MOFS typically returns a large set of non-dominated solutions, which have to be assessed by the data scientist in order to proceed with the final choice. Given the multi-variate nature of the assessment, which may include objectives (e.g., fairness) unrelated to predictive accuracy, this step is often not straightforward and suffers from the lack of existing tools. For instance, it is common to make use of a tabular presentation of the solutions, which provides little information about the trade-offs and the relationships between objectives over the set of solutions. Adopting a GA-based MOFS with six objectives (number of selected features, balanced accuracy, F1-Score, variance inflation factor, statistical parity, and equalised odds) for two feature selection tasks, this paper illustrates the complex challenge of assessing MOFS results and the need for a methodology to aid and justify the final choice of a solution., The project leading to this publication has received funding from the European Union’s Horizon 2020 research and innovation programme (G.A. No. 955895). A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under G.A. No. 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Evaluating quality of disparate data sources: A discord-driven approach

  • Akter, Yeasmin Ara
  • Abelló Gamazo, Alberto
  • Jovanovic, Petar
  • Sagi, Tomer
  • Hose, Katja
Among other measures of data quality, determining the reliability of conflicting values from different sources is especially challenging. Traditional data fusion approaches often infer correct values in simple cases, but struggle to handle variations in data granularity (such as differences in temporal, spatial, or categorical aggregations) and offer limited insight into the nature of disagreements. Thus, we propose a new source evaluation approach for numerical attributes that measures discordance (i.e., the extent to which sources differ from each other). Unlike existing methods that focus solely on point estimation, we allow both fine-grained and coarse-grained analysis, allowing more sophisticated data quality assessments. We employ a linear programming solver that transparently adapts to any data alignment expressed in a set of operators resembling relational algebra. Extensive experiments on real-world datasets demonstrate that our method generalizes existing truth discovery techniques measuring differences with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and can adapt to diverse and complex scenarios., Y. A. Akter is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). A. Abelló and P. Jovanovic are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Hyppo: efficient discovery and execution of data science pipelines in collaborative environments

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto
  • Nadal Francesch, Sergi
In exploratory data science and machine learning (ML), developing an effective and efficient solution involves the exploration of numerous pipelines per dataset, considering various combinations of data preprocessing, feature selection, model selection, evaluation metrics, etc. This is an iterative process of trial and error, where a pipeline is revised and refined until a satisfactory level of performance is achieved. Such a process can be substantially improved by leveraging collaboration opportunities and automated pipeline optimization. We demonstrate Hyppo, a scalable collaboration system designed for interactive discovery and automated optimization, resulting in a system that supports evaluating and sharing pipelines with minimal overhead. Hyppo takes advantage of numerous optimization possibilities as exploiting equivalences among tasks, sharing computations, artifact materialization, and reuse from past executions to derive better execution plans. We showcase Hyppo with open data and pipelines from popular public code repositories and catalogs. Demo video: https://youtu.be/iilPV_xtXwM, This work has been partially supported by the H2020-MSCA-ITN-2020 DEDS (GA.955895), the European Union’s Horizon Europe Research and Innovation programmes DataGEMS (GA.101188 416), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191 RB-I00/AEI/10.13039/50 1100011033 (DOGO4ML)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Mitigating data sparsity in integrated data through text conceptualization

  • Rahman, Md Ataur
  • Nadal Francesch, Sergi
  • Romero Moral, Óscar
  • Sacharidis, Dimitris
We study the data sparsity problem for data generated from an integration system. We approach the problem from a textual information extraction perspective and propose to conceptualize external documents using the concepts in the integrated schema. We present THOR, a novel system that, unlike related approaches, neither relies on complex rules nor models trained with large annotated corpus, but on the integrated data and its schema without the need for human annotations. An extensive evaluation on the text conceptualization task demonstrates the superiority of our approach in terms of F1-score, effort and use of resources over the state-of-the-art language models., Md Ataur Rahman is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). Sergi Nadal and Oscar Romero are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Finding relevant information in big datasets with ML

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
Due to the abundance of data, noisy, irrelevant, or redundant features often need to be identified and discarded. Feature selection is a collection of methods used to ensure that only relevant data are used for a data analysis task. Extracting and using only useful data for analysis promotes model understanding and performance and reduces the model training time and variance, i.e., overfitting. There is an abundance of methods for feature selection, and they can be categorised by various perspectives and are applicable to differing use cases. In this tutorial, we introduce the feature selection problem and present it from three perspectives of categorisation: search strategy, model reliance, and relevance definition. Furthermore, we propose a guideline for the use of the various methods. Lastly, we discuss current challenges and opportunities for research on feature selection., The project leading to this publication has received funding from 2020 research and innovation programme (grant agreement No 955895)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

On many-objective feature selection and the need for interpretability

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
Big data comes with the challenge of containing irrelevant and redundant information (i.e., features). Given that a single objective cannot fully capture a feature’s relevance, a Many-Objective Feature Selection (MOFS) approach able to accommodate various relevant perspectives is preferred for identifying the most appropriate features in a given context. However, MOFS produces a large set of solutions whose interpretability has been largely overlooked. First, we demonstrate the relevance of MOFS and establish its necessity by considering up to six objectives using a genetic algorithm and Naive Bayes on ten datasets for classification tasks. Then, we propose a novel methodology to improve the interpretability of MOFS results in order to support the data scientist in selecting the subset of features pertinent to their use case. Our methodology is instantiated as an intuitive and interactive dashboard that provides insights into the results beyond the pure numerical representation of the objectives being considered and evaluated with 50 participants. The outcome shows that it addresses the need for a methodological approach and comprehensive visualization to achieve interoperability., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Towards fair machine learning using many-objective feature selection

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. B. Bilalli is partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/501100011033 (TALC). G. Bontempi is supported by the Service Public de Wallonie Recherche, Spain under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Workload placement on heterogeneous CPU-GPU systems

  • Nogueira Lobo de Carvalho, Marcos
  • Simitsis, Alkis
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
The popularity of heterogeneous CPU-GPU processing has increased considerably in recent years. To efficiently utilize heterogeneous resources, data processing systems depend on an appropriate workload placement strategy to assign the right amount of compute to the right processor. However, finding an optimal placement strategy is not trivial due to various complex and conflicting tradeoffs related to the characteristics of processors, the nature of the workload, and data locality. In addition, placement decisions impact workload runtime and performance cost, and also depend on the availability of potentially different implementations for CPUs and GPUs, which adds extra complexity in such heterogeneous environments. In this tutorial, we review and compare state-of-the-art strategies for workload placement on heterogeneous CPU-GPU architectures, along with runtime prediction techniques and methods to support multi-device code. We also discuss open issues and identify potentially promising future research directions., This work has been partially supported by the H2020-MSCAITN2020 DEDS (GA.955895), and the EU-HORIZON programmes CREXDATA (GA.101092749) and FAIR-CORE4EOSC (GA.101057264) and the Spanish MICIU DOGO4ML (PID2020-117191RB-I00)., Peer Reviewed, Postprint (published version)