Pasar al contenido principal

DESARROLLO, OPERATIVA Y GOBERNANZA DE DATOS PARA SISTEMAS SOFTWARE BASADOS EN APRENDIZAJE AUTOMATICO

PID2020-117191RB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 139
Encontrada(s) 2 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

The knowledge graph lifecycle in NTT DATA

  • Flores Herrera, Javier de Jesús|||0000-0002-2998-9962
  • Jamin, Emmanuel
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
The Semantic Business Unit (SEMBU) in NTT DATA aims to increase the semantic interoper ability and accessibility of European institutions’ data projects by following Linked Open Data (LOD) principles to build controlled vocabularies and produce Knowledge Graphs (KGs). One of its most notable projects revolves around the CORDIS portal1, which publishes information about research and innovation projects funded by the European Commission. SEMBU pursues two main goals: (i) expose semantic data related to CORDIS via a SPARQL endpoint that facilitates access and reuse of quality scientific-related data, and (ii) design an efficient, incremental, and automated KG lifecycle to be used as a reference in other data projects. To that end, we have adopted state-of-the-art semantic technologies to support the creation and management of the KG with the goal of centralizing knowledge and providing an overall view of data assets that improve data governance, maintenance, and external interaction by data consumers. We have also identified some of their limitations which are tackled via an industrial PhD. This paper reports our experience, the obstacles, and proposals for generating and maintaining the CORDIS KG., This work was partly funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 (DOGO4ML). Javier Flores is supported by contract 2020-DI-027 of the Industrial Doctorate Program of the Government of Catalonia and CONACYT’s scholarship. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I /AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Capturing analytical intents from text

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Dimic, Miona|||0009-0008-1869-3585
  • Bilalli, Besim|||0000-0002-0575-2389
The ability to extract valuable information from data is crucial for organizations and individuals who want to remain competitive in a constantly evolving data-driven environment. However, some of them lack the skills required to appropriately leverage the existing data analytics tools and methods. This problem is aggravated when the users are domain-experts but completely unfamiliar with data analytics terminology, as existing assistant tools, such as AutoML or Intelligent Discovery Assistants, require them to state their analytical intent (i.e., the type of data analysis they want to perform). To address this problem, we propose to capture the underlying analytical intent from textual problem descriptions by leveraging Large Language Models (LLMs). To this end, we propose a hierarchical categorization of analytical intents, along with a data collection methodology to obtain analytical problem descriptions for all of them in order to validate different approaches that aim to extract such intents from text. Next, we compare the performance of state-of-the-art approaches with LLMs, and then study the performance of different LLMs based on their characteristics and the impact of the source of validation data. Finally, we develop a prototype to showcase how our method could interact with existing AutoML systems., Gerard Pons is supported by the EU’s Horizon Programme call, under Grant Agreement No. 101093164 (ExtremeXP), and Besim Bilalli is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia i Innovación under the funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Evaluating quality of disparate data sources: A discord-driven approach

  • Akter, Yeasmin Ara|||0000-0002-9865-0700
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Sagi, Tomer
  • Hose, Katja
Among other measures of data quality, determining the reliability of conflicting values from different sources is especially challenging. Traditional data fusion approaches often infer correct values in simple cases, but struggle to handle variations in data granularity (such as differences in temporal, spatial, or categorical aggregations) and offer limited insight into the nature of disagreements. Thus, we propose a new source evaluation approach for numerical attributes that measures discordance (i.e., the extent to which sources differ from each other). Unlike existing methods that focus solely on point estimation, we allow both fine-grained and coarse-grained analysis, allowing more sophisticated data quality assessments. We employ a linear programming solver that transparently adapts to any data alignment expressed in a set of operators resembling relational algebra. Extensive experiments on real-world datasets demonstrate that our method generalizes existing truth discovery techniques measuring differences with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and can adapt to diverse and complex scenarios., Y. A. Akter is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). A. Abelló and P. Jovanovic are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Multi-agent debate strategies to enhance requirements engineering with large language models

  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Marco Gómez, Jordi|||0000-0002-0078-7929
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Context: Large Language Model (LLM) agents are becoming widely used for various Requirements Engineering (RE) tasks. Research on improving their accuracy mainly focuses on prompt engineering, model fine-tuning, and retrieval augmented generation. However, these methods often treat models as isolated black boxes - relying on single-pass outputs without iterative refinement or collaboration, limiting robustness and adaptability. Objective: We propose that, just as human debates enhance accuracy and reduce bias in RE tasks by incorporating diverse perspectives, different LLM agents debating and collaborating may achieve similar improvements. Our goal is to investigate whether Multi-Agent Debate (MAD) strategies can enhance RE performance. Method: We conducted a systematic study of existing MAD strategies across various domains to identify their key characteristics. To assess their applicability in RE, we implemented and tested a preliminary MAD-based framework for RE classification. Results: Our study identified and categorized several MAD strategies, leading to a taxonomy outlining their core attributes. Our preliminary evaluation demonstrated the feasibility of applying MAD to RE classification. Conclusions: MAD presents a promising approach for improving LLM accuracy in RE tasks. This study provides a foundational understanding of MAD strategies, offering insights for future research and refinements in RE applications., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745; and by the Spanish Ministerio de Ciencia e Innovación under project funding scheme PID2020-117191RB-I00/ AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Improved management of issue dependencies in issue trackers of large collaborative projects

  • Raatikainen, Mikko
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Lüders, Clara Marie
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Myllyaho, Lalli
  • Kettunen, Elina
  • Marco Gómez, Jordi|||0000-0002-0078-7929
  • Tiihonen, Juha
  • Halonen, Mikko
  • Männistö, Tomi
Issue trackers, such as Jira, have become the prevalent collaborative tools in software engineering for managing issues, such as requirements, development tasks, and software bugs. However, issue trackers inherently focus on the lifecycle of single issues, although issues have and express dependencies on other issues that constitute issue dependency networks in large complex collaborative projects. The objective of this study is to develop supportive solutions for the improved management of dependent issues in an issue tracker. This study follows the Design Science methodology, consisting of eliciting drawbacks and constructing and evaluating a solution and system. The study was carried out in the context of The Qt Company's Jira, which exemplifies an actively used, almost two-decade-old issue tracker with over 100,000 issues. The drawbacks capture how users operate with issue trackers to handle issue information in large, collaborative, and long-lived projects. The basis of the solution is to keep issues and dependencies as separate objects and automatically construct an issue graph. Dependency detections complement the issue graph by proposing missing dependencies, while consistency checks and diagnoses identify conflicting issue priorities and release assignments. Jira's plugin and service-based system architecture realize the functional and quality concerns of the system implementation. We show how to adopt the intelligent supporting techniques of an issue tracker in a complex use context and a large data-set. The solution considers an integrated and holistic system view, practical applicability and utility, and the practical characteristics of issue data, such as inherent incompleteness., The work presented in this paper has been conducted within the scope of the Horizon 2020 project OpenReq, which is supported by the European Union under Grant Nr. 732463. We are grateful for the provision of the Finnish computing infrastructure to carry out the tests (persistent identifier urn:nbn:fi:research-infras-2016072533). This paper has been funded by the Spanish Ministerio de Ciencia e Innovacionúnder project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

RE-Miner: Mining mobile user reviews with feature extraction and emotion classification

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Tiessler Aguirre, Max
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Bertolín Rico, Irene
In the context of app stores, user reviews are pivotal on supporting multiple requirements engineering tasks. Among these, feature extraction and emotion classification play a crucial role in requirements prioritization, feedback gathering and release planning. Empirical evaluation of these techniques is impeded by data collection complexities and a lack of reproducible methods and available tools. Furthermore, existing studies often focus on isolated tasks, hindering a comprehensive analysis of user perceptions. This paper introduces RE-Miner, a work-in-progress tool integrating multiple feature extraction and emotion classification innovative methods, enabling a detailed analysis of feature-oriented user feedback. RE-Miner comprises a web-based service for task integration and comparison, and a web application for persistent storage and analytical visualization of reviews. As a result, RE-Miner provides a platform for seamless integration, replication, and comparison of review mining techniques, fostering advancements in feature extraction and emotion classification understanding for requirements engineering. A demo of the tool is showcased here: https://youtu.be/PFNCbborPuU., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Quality measurement in agile and rapid software development: A systematic mapping

  • López Cuesta, Lidia|||0000-0002-6901-9223
  • Burgués Illa, Xavier|||0000-0001-6974-9886
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Vollmer, Anna Maria
  • Behutiye, Woubshet
  • Karhapää, Pertti
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Rodríguez, Pilar
  • Oivo, Markku
Context: In despite of agile and rapid software development (ARSD) being researched and applied extensively, managing quality requirements (QRs) are still challenging. As ARSD processes produce a large amount of data, measurement has become a strategy to facilitate QR management. Objective: This study aims to survey the literature related to QR management through metrics in ARSD, focusing on: bibliometrics, QR metrics, and quality-related indicators used in quality management. Method: The study design includes the definition of research questions, selection criteria, and snowballing as search strategy. Results: We selected 61 primary studies (2001-2019). Despite a large body of knowledge and standards, there is no consensus regarding QR measurement. Terminology is varying as are the measuring models. However, seemingly different measurement models do contain similarities. Conclusion: The industrial relevance of the primary studies shows that practitioners have a need to improve quality measurement. Our collection of measures and data sources can serve as a starting point for practitioners to include quality measurement into their decision-making processes. Researchers could benefit from the identified similarities to start building a common framework for quality measurement. In addition, this could help researchers identify what quality aspects need more focus, e.g., security and usability with few metrics reported., This work has been funded by the European Union’s Horizon 2020 research and innovation program through the Q-Rapids project (grant no. 732253). This research was also partially supported by the Spanish Ministerio de Economía, Industria y Competitividad through the DOGO4ML project (grant PID2020-117191RB-I00). Silverio Martínez-Fernández worked in Fraunhofer IESE before January 2020., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Which design decisions in AI-enabled mobile applications contribute to greener AI?

  • Creus Castanyer, Roger
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Background: The construction, evolution and usage of complex artificial intelligence (AI) models demand expensive computational resources. While currently available high-performance computing environments support well this complexity, the deployment of AI models in mobile devices, which is an increasing trend, is challenging. Mobile applications consist of environments with low computational resources and hence imply limitations in the design decisions during the AI-enabled software engineering lifecycle that balance the trade-off between the accuracy and the complexity of the mobile applications. Objective: Our objective is to systematically assess the trade-off between accuracy and complexity when deploying complex AI models (e.g. neural networks) to mobile devices, which have an implicit resource limitation. We aim to cover (i) the impact of the design decisions on the achievement of high-accuracy and low resource-consumption implementations; and (ii) the validation of profiling tools for systematically promoting greener AI. In this way, we aim to provide a quantitative analysis of the performance of AI-enabled applications in operation with respect to their design decisions. Method: This confirmatory registered report consists of a plan to conduct an empirical study to quantify the implications of the design decisions on AI-enabled applications performance and to report experiences of the end-to-end AI-enabled software engineering lifecycle. Concretely, we will implement both image-based and language-based neural networks in mobile applications to solve multiple image classification and text classification problems on different benchmark datasets. For that, we make use of the Android Studio and Unity3D frameworks for building mobile applications that work over Android; and TensorFlow Lite and Open Neural Network Exchange (ONNX) file formats for deploying the AI models in the applications. Overall, we plan to model the accuracy and complexity of AI-enabled applications in operation with respect to their design decisions and will provide tools for allowing practitioners to gain consciousness of the quantitative relationship between the design decisions and the green characteristics of study. Additionally, we will provide experiences in the end-to-end AI-enabled software lifecycle and discuss the challenges found, tools and practices for practitioners. Finally, we will provide an open-source data repository following the ESEM open science practices and containing all the experimentation, analysis and reports in our study., This work has been partially supported by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00) and by the the "Beatriz Galindo" Spanish Program BEAGAL18/00064., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

T-FREX: a transformer-based feature extraction method from mobile app reviews

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Miaschi, Alessio
  • Dell’Orletta, Felice
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Mobile app reviews are a large-scale data source for software-related knowledge generation activities, including software maintenance, evolution and feedback analysis. Effective extraction of features (i.e., functionalities or characteristics) from these reviews is key to support analysis on the acceptance of these features, identification of relevant new feature requests and prioritization of feature development, among others. Traditional methods focus on syntactic pattern-based approaches, typically context-agnostic, evaluated on a closed set of apps, difficult to replicate and limited to a reduced set and domain of apps. Mean-while, the pervasiveness of Large Language Models (LLMs) based on the Transformer architecture in software engineering tasks lays the groundwork for empirical evaluation of the performance of these models to support feature extraction. In this study, we present T-FREX, a Transformer-based, fully automatic approach for mobile app review feature extraction. First, we collect a set of ground truth features from users in a real crowdsourced software recommendation platform and transfer them automatically into a dataset of app reviews. Then, we use this newly created dataset to fine-tune multiple LLMs on a named entity recognition task under different data configurations. We assess the performance of T- FREX with respect to this ground truth, and we complement our analysis by comparing T- FREX with a baseline method from the field. Finally, we assess the quality of new features predicted by T- FREX through an external human evaluation. Results show that T- FREX outperforms on average the traditional syntactic-based method, especially when discovering new features from a domain for which the model has been fine-tuned., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovacion under project I funding scheme PID2020-117191RB-100 I AEI/10.13039/501100011033. Alessio Miaschi and Felice Dell’ Orletta have also been supported by the PNRR project FAIR - Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Eris: efficiently measuring discord in multidimensional sources

  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Cheney, James
Data integration is a classical problem in databases, typically decomposed into schema matching, entity matching and data fusion. To solve the latter, it is mostly assumed that ground truth can be determined. However, in general, the data gathering processes in the different sources are imperfect and cannot provide an accurate merging of values. Thus, in the absence of ways to determine ground truth, it is important to at least quantify how far from being internally consistent a dataset is. Hence, we propose definitions of concordant data and define a discordance metric as a way of measuring disagreement to improve decision-making based on trustworthiness. We define the discord measurement problem of numerical attributes in which given a set of uncertain raw observations or aggregate results (such as case/hospitalization/death data relevant to COVID-19) and information on the alignment of different conceptualizations of the same reality (e.g., granularities or units), we wish to assess whether the different sources are concordant, or if not, use the discordance metric to quantify how discordant they are. We also define a set of algebraic operators to describe the alignments of different data sources with correctness guarantees, together with two alternative relational database implementations that reduce the problem to linear or quadratic programming. These are evaluated against both COVID-19 and synthetic data, and our experimental results show that discordance measurement can be performed efficiently in realistic situations., The work of A. Abelló has been done under project PID2020-117191RB-I00 funded by MCIN/AEI/ 10.13039/501100011033. The work of J. Cheney was supported by ERC Consolidator Grant Skye (grant number 682315). Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

DOGO4ML: Development, operation and data governance for ML-based software systems

  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
  • Bilalli, Besim|||0000-0002-0575-2389
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
Machine Learning based Software Systems (MLSS) are becoming increasingly pervasive in today’s society and can be found in virtually every domain. Building MLSS is challenging due to their interdisciplinary nature. MLSS engineering encompasses multiple disciplines, of which Data Engineering and Software Engineering appear as most relevant. The DOGO4ML project aims at reconciling these two disciplines for providing a holistic end-to-end framework to develop, operate and govern MLSS and their data. It proposes to combine and intertwine two software cycles: the DataOps and the DevOps lifecycles. The DataOps lifecycle manages the complexity of dealing with the big data needed by ML models, while the DevOps lifecycle is in charge of building the system that embeds these models. In this paper, we present the main vision and goals of the project as well as its expected contributions and outcomes. Although the project is in its initial stage, the progress of the research undertaken so far is detailed., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Software analytics tools: an intentional view

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Software analytic tools consume big amounts of data coming from either (or both) the software development process or the system usage and aggregate them into indicators which are rendered to different types of stakeholders, also offering them a portfolio of techniques and capabilities such as what-if analysis, prediction and alerts. Precisely, the variety of stakeholders and the different goals they pursue justifies the convenience of performing an intentional analysis of the use of software analytics tools. With this aim, we first enumerate the different stakeholders and identify their intentional relationships with software analytics tools in the form of dependencies. Then, we focus on one particular stakeholder, namely the requirements engineer, and identify further intentional elements represented in a strategic rationale model. The resulting model provides an abstract view of the domain which may help stakeholders when deciding on the adoption of software analytic tools in their particular context., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Operationalizing and automating data governance

  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Bilalli, Besim|||0000-0002-0575-2389
  • Romero Moral, Óscar|||0000-0001-6350-8328
The ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens or hundreds of continuously evolving data sources produce semi-structured data. To overcome this challenge, we develop a framework for operationalizing and automating data governance. For the first, we propose a zoned data lake architecture and a set of data governance processes that allow the systematic ingestion, transformation and integration of data from heterogeneous sources, in order to make them readily available for business users. For the second, we propose a set of metadata artifacts that allow the automatic execution of data governance processes, addressing a wide range of data management challenges. We showcase the usefulness of the proposed approach using a real world use case, stemming from the collaborative project with the World Health Organization for the management and analysis of data about Neglected Tropical Diseases. Overall, this work contributes on facilitating organizations the adoption of data-driven strategies into a cohesive framework operationalizing and automating data governance., This work was partly supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

MApp-KG: Mobile app knowledge graph for document-based feature knowledge generation

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Mobile app repositories serve as large-scale crowdsourced information systems used for various document-based software engineering tasks, leveraging product descriptions, user reviews, and other natural language documents. Particularly, feature extraction (i.e., identifying functionalities or capabilities of a mobile app mentioned in these documents) is key for product recommendation, topic modelling, and feedback analysis. However, researchers often face domain-specific challenges in mining these repositories, including the integration of heterogeneous data sources, large-scale data collection, normalization and ground-truth generation for feature-oriented tasks. In this paper, we introduce MApp-KG, a combination of software resources and data artefacts in the field of mobile app repositories aimed at supporting feature-oriented knowledge generation tasks. Our contribution provides a framework for automatically constructing a knowledge graph that models a domain-specific catalog of natural language documents related to mobile applications. We distribute MApp-KG through a public triplestore, enabling its immediate use for future research and replication of our findings., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Forecasting software indicators: an industry-academia collaboration

  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Manzano Aguilar, Martí|||0000-0001-7944-7232
  • Abherve, Antonin
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Context: Nowadays software-development organizations are urged to exploit their data for empowering their decision-making processes. Such data may be used to monitor the status of meaningful software indicators (e.g., software quality, productivity and on-time delivery) that are relevant for their decision-making processes. Forecasting the values of such indicators may provide evidence of a potentially high risk or opportunity that could help to anticipate actions accordingly. Most of the existing forecasting proposals in software engineering use open-source data rather than data from industrial projects. Therefore, there is a lack of evidence on how these proposals ft the particular needs of a software-development organization and how they can be automated into the organization’s infrastructure. Objective: To enable software indicators´ forecasting in a software-development organization (Modeliosoft). Method: We designed an industry-academia collaboration based on Action Design Research (ADR) to address Modeliosoft’s forecasting challenges. Results A tool-supported method called FOSI (Forecasting Of Software Indicators) for enabling forecasting in Modeliosoft. We obtained positive results regarding its suitability and technical feasibility in a pilot project of the organization. In addition, we provide details and refections on the potential usefulness of the method for addressing similar feld problems. Conclusions: The procedures and results detailed in this paper are valuable to: 1) address Modeliosoft’s forecasting challenges 2) inspire other software-development organizations on how to deal with similar problems and even reuse some procedures and software support tools resulted from this work, 3) promote the win-win benefts of industry-academia collaborations., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been partially funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00/AEI/10.13039/ 50 501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Hyppo: efficient discovery and execution of data science pipelines in collaborative environments

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
In exploratory data science and machine learning (ML), developing an effective and efficient solution involves the exploration of numerous pipelines per dataset, considering various combinations of data preprocessing, feature selection, model selection, evaluation metrics, etc. This is an iterative process of trial and error, where a pipeline is revised and refined until a satisfactory level of performance is achieved. Such a process can be substantially improved by leveraging collaboration opportunities and automated pipeline optimization. We demonstrate Hyppo, a scalable collaboration system designed for interactive discovery and automated optimization, resulting in a system that supports evaluating and sharing pipelines with minimal overhead. Hyppo takes advantage of numerous optimization possibilities as exploiting equivalences among tasks, sharing computations, artifact materialization, and reuse from past executions to derive better execution plans. We showcase Hyppo with open data and pipelines from popular public code repositories and catalogs. Demo video: https://youtu.be/iilPV_xtXwM, This work has been partially supported by the H2020-MSCA-ITN-2020 DEDS (GA.955895), the European Union’s Horizon Europe Research and Innovation programmes DataGEMS (GA.101188 416), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191 RB-I00/AEI/10.13039/50 1100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reprogramming heritage: an approach for the automatization in the adaptative reuse of buildings

  • Domènech Rodríguez, Marta|||0000-0002-6152-679X
  • López López, David|||0000-0002-3544-618X
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Cornadó Bardón, Còssima|||0000-0003-0553-7176
This article introduces a methodology for a novel data-driven computational model aimed at aiding public administrations in managing and evaluating the adaptative reuse of buildings while tackling ecological and digital challenges. Drawing from the 2030 Agenda for Sustainable Development, the study underscores the significance of innovative approaches in harnessing the economic potential of data. Focusing on Barcelona’s Ciutat Vella district, the research selects five historic public buildings for analysis, strategically positioned to spur local entrepreneurship and counteract tourism dominance. Through an extensive literature review, the article identifies a gap in computational models for building adaptative reuse and proposes a methodological framework that integrates data collection, processing, and computational modelling, underscored by GIS technology and open data sources. The proposed methodology for a computational algorithm aims to systematise spatial characteristics, assess programmatic needs, and optimise building usage, while addressing challenges such as data integration and quality assurance. Ultimately, the research presents a pioneering approach to building adaptative reuse, aimed at fostering sustainable urban development and offering replicable insights applicable to similar challenges in other cities., Cohabitem Barcelona: cinc casos d’estudi a Ciutat Vella per a la reutilització urbana i el foment de models productius innovadors, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Chatbots4Mobile: Feature-oriented knowledge base generation using natural language

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Chatbots4Mobile is a research project from the GESSI research group (UPC-BarcelonaTech) which aims at designing and developing a task oriented, knowledge based conversational agent to support mobile users in the process of managing and integrating the functionalities exposed by their own application portfolio. To support the design of the required knowledge base, the project focuses on the application of Natural Language Processing (NLP) techniques to infer extended knowledge about the features exposed by a subset of mobile applications, including feature extraction from app-related documents, syntactic and semantic similarity analysis between features, and intent/entity classification focused on functionality identification from user requests. As next steps, we are focusing on the evaluation of embedded linguistic knowledge in large language models, as well as the application of granular sentiment analysis techniques to discern biased documents and process sentiment-based user feedback., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

How and why false denial constraints are discovered

  • Martín Garcia, Albert|||0009-0003-9967-9331
  • Cunha de Almeida, Eduardo
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Denial Constraints (DCs) are a flexible formalism to express many types of data rules, making them a widely adopted tool for many applications. This flexibility led to the development of numerous algorithms to automatically discover DCs directly from data. However, few studies have been conducted on the quality of the discovered DCs. We experimentally quantify the lack of quality in the results obtained by state-of-the-art algorithms, showing how the proportion of discovered DCs that are false is rarely below 95%. We hypothesize that the common source of these erroneous DCs stems from the adoption of the current DC validity definition. We use a statistical approach to explain the mechanism leading to these results, and propose a redefinition of DC validity properties to avoid the acceptance of false DCs. We validate this redefinition experimentally, showing that it exclusively accepts true constraints of the data, and is reliable enough to discover DCs missed by domain experts. Additionally, we provide curated sets of golden DCs for each dataset used in our study, those generated by domain experts and those discovered using our approach., This work is supported by the Horizon Europe Programme under GA.101135513 (CyclOps) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/ 501100011033 (DOGO4ML). Anna Queralt is a Serra-Húnter fellow. E. Almeida is funded by the CNPQ grants 302909/2022-2 and 444192/2024-7. Albert Martin is funded by the predoctoral program AGAUR-FI grants (2025 FI-1 00967) Joan Oró, which is backed by the Secretariat of Universities and Research of the Department of Research and Universities of the Generalitat of Catalonia, as well as the European Social Plus Fund., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Estudio, informe, memoria (Report)

SOPCOM: A tool to manage software process components for situational method engineering

  • Costal Costa, Dolors|||0000-0002-7340-0414
  • Quer, Carme|||0000-0002-9000-6371
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Situational Method Engineering (SME) principles and techniques have been proposed to promote the modularization of method knowledge in the form of reusable method components and their composition into new situation specific methods. In this paper, we present SOPCOM, a tool aimed at supporting the creation and management of an SME method components (“chunks”) catalogue, and the composition of these components into methods., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project /funding scheme PID2020-117191RB-I00 /AEI/10.13039/501100011033.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Using metrics for code smells of ML pipelines

  • Costal Costa, Dolors|||0000-0002-7340-0414
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Rey Juárez, Santiago del|||0000-0003-4979-414X
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. https://dx.doi.org/0.1109/ICSA-C63560.2024.00055, ML pipelines, as key components of ML systems, shall be developed following quality assurance techniques. Unfortunately, it is often the case in which they present maintainability issues, due to the experimental nature of data collection and ML model construction. In this work, we perform a first evaluation of a set of metrics, proposed in previous research, for measuring the presence of code smells related to maintainability, in ML pipeline application examples. Moreover, we provide the lessons learnt and insights gained from this evaluation., This work has been funded by the National Spanish Program for Research Aimed at the Challenges of Society 2020. The project and funding identifiers are PID2020-117191RB-I00/ AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Impact of ML optimization tactics on greener pre-trained ML models

  • González Álvarez, Alexandra|||0009-0003-7634-0343
  • Castaño Fernández, Joel|||0009-0005-6385-9255
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
Machine Learning (ML)-based solutions have currently surpassed human performance in tasks like image classification, visual reasoning, and English understanding. However, this advancement comes at the cost of increasing energy consumption. Traditionally, ML projects have prioritized accuracy over energy, creating a gap in energy consumption during model inference. This study aims to (i) understand image classification datasets and pre-trained models, which is essential for the subsequent analyses, (ii) improve inference efficiency by comparing optimized and non-optimized models, and (iii) assess the economic impact of the optimizations. We conduct a controlled experiment to evaluate the impact of various PyTorch optimization techniques (dynamic quantization, , local pruning, and global pruning) on 42 Hugging Face models for image classification. The metrics examined include GPU utilization, power and energy consumption, accuracy, time, computational complexity, and economic costs. The models are repeatedly evaluated to quantify the effects of these optimization techniques. Dynamic quantization demonstrates inference time and energy consumption reductions, making it suitable for large-scale systems. balances accuracy and energy. Local pruning shows no positive impact on accuracy, and global pruning’s longer optimization times impact economic costs. This study highlights the role of software engineering tactics in achieving greener ML models, offering guidelines for practitioners to make informed decisions on optimization methods that align with sustainability goals., This work has been partially funded by the Spanish research projects GAISSA (TED2021-130923B-I00 by MCIN/ AEI/10.13039/ 501100011033) and DOGO4ML (ref. PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Web API evolution patterns: A usage-driven approach

  • Koçi, Rediana|||0000-0003-3367-950X
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
As the use of Application Programming Interfaces (APIs) is increasingly growing, their evolution becomes more challenging in terms of the service provided according to consumers' needs. In this paper, we address the role of consumers' needs in WAPIs evolution and introduce a process mining pattern-based method to support providers in WAPIs evolution by analyzing and understanding consumers' behavior, imprinted in WAPI usage logs. We take the position that WAPIs' evolution should be mainly usage-based, i.e., the way consumers use them should be one of the main drivers of their changes. We start by characterizing the structural relationships between endpoints, and next, we summarize these relationships into a set of behavioral patterns (i.e., usage patterns whose occurrences indicate specific consumers' behavior like repetitive or consecutive calls), that can potentially imply the need for changes (e.g., creating new parameters for endpoints, merging endpoints). We analyze the logs and extract several metrics for the endpoints and their relationships, to then detect the patterns. We apply our method in two real-world WAPIs from different domains, education, and health, respectively the WAPI of Barcelona School of Informatics at the Polytechnic University of Catalonia (Facultat d'Informàtica de Barcelona, FIB, UPC), and District Health Information Software 2 (DHIS2) WAPI. The feedback from consumers and providers of these WAPIs proved the effectiveness of the detected patterns and confirmed the promising potential of our approach., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Energy efficiency of training neural network architectures: an empirical study

  • Xu, Yinlena
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Martínez Martínez, Matías-Sebastián|||0000-0002-2945-866X
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
The evaluation of Deep Learning (DL) models has traditionally focused on criteria such as accuracy, F1 score, and related measures. The increasing availability of high computational power environments allows the creation of deeper and more complex models. However, the computations needed to train such models entail a large carbon footprint. In this work, we study the relations between DL model architectures and their environmental impact in terms of energy consumed and CO2 emissions produced during training by means of an empirical study using Deep Convolutional Neural Networks. Concretely, we study: (i) the impact of the architecture and the location where the computations are hosted on the energy consumption and emissions produced; (ii) the trade-off between accuracy and energy efficiency; and (iii) the difference on the method of measurement of the energy consumed using software-based and hardware-based tools., This work has been supported by the Spanish project PID2020-117191RB-I00 funded by MCIN/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Goal-oriented models for teaching and understanding data structures

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Ruiz, Marcela
Most computer science curricula include a compulsory course on data structures. Students are prone to memorise facts about data structures instead of understanding the essence of underlying concepts. This can be explained by the fact that learning the basics of each data structure, the difference with each other, and the adequacy of each of them to the most appropriate context of use, is far from trivial. This paper explores the idea of providing adequate levels of abstractions to describe data structures from an intentional point of view. Our hypothesis is that adopting a goal-oriented perspective could emphasise the main goals of each data structure, its qualities, and its relationships with the potential context of use. Following this hypothesis, in this paper we present the use of iStar2.0 to teach and understand data structures. We conducted a comparative quasi-experiment with undergraduate students to evaluate the effectiveness of the approach. Significant results show the great potential of goal modeling for teaching technical courses like data structures. We conclude this paper by reflecting on further teaching and conceptual modeling research to be conducted in this field., This work has been partially supported by the by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00), the Digitalization Initiative of the Canton of Zü-rich (DIZH), and ZHAW Digital., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Towards fair machine learning using many-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. B. Bilalli is partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2023-152841OA-I00/AEI/10.13039/501100011033 (TALC). G. Bontempi is supported by the Service Public de Wallonie Recherche, Spain under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Discovery of semantic non-syntactic joins

  • Maynou Yelamos, Marc|||0009-0004-3908-4392
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
Data discovery is an essential step in the data integration pipeline involving finding datasets whose combined information provides relevant insights. Discovering joinable attributes requires assessing the closeness of the semantic concepts that two attributes represent, which is highly sensitive and dependent on the chosen similarity metric. The state of the art commonly approaches this task from a syntactic perspective, this is, performing comparisons based on the data values or on direct transformations (e.g., via hash functions). These approaches suffice when the two sets of instances share the same syntactic representation, but fail to detect cases in which the same semantic concept is represented by different sets of values, which we refer as semantic non-syntactic joins. This is a relevant problem in data lake scenarios, when the underlying datasets present high heterogeneity and lack of standardization. To that end, in this paper, we propose an empirical approach to detect semantic non-syntactic joins, which leverages, simultaneously, syntactic and semantic measurements of the data. We demonstrate that our approach is effective in detecting such kind of joins., Marc Maynou is supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP), and Sergi Nadal is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

On many-objective feature selection and the need for interpretability

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
Big data comes with the challenge of containing irrelevant and redundant information (i.e., features). Given that a single objective cannot fully capture a feature’s relevance, a Many-Objective Feature Selection (MOFS) approach able to accommodate various relevant perspectives is preferred for identifying the most appropriate features in a given context. However, MOFS produces a large set of solutions whose interpretability has been largely overlooked. First, we demonstrate the relevance of MOFS and establish its necessity by considering up to six objectives using a genetic algorithm and Naive Bayes on ten datasets for classification tasks. Then, we propose a novel methodology to improve the interpretability of MOFS results in order to support the data scientist in selecting the subset of features pertinent to their use case. Our methodology is instantiated as an intuitive and interactive dashboard that provides insights into the results beyond the pure numerical representation of the objectives being considered and evaluated with 50 participants. The outcome shows that it addresses the need for a methodological approach and comprehensive visualization to achieve interoperability., The project leading to this publication has received funding from the European Commission under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 955895). Moreover, A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under grant nr 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Measuring discord among multidimensional data sources

  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Cheney, James
Data integration is a classical problem in databases, typically decomposed into schema matching, entity matching and record merging. To solve the latter, it is mostly assumed that ground truth can be determined, either as master data or from user feedback. However, in many cases, this is not the case because firstly the merging processes cannot be accurate enough, and also the data gathering processes in the different sources are simply imperfect and cannot provide high quality data. Instead of enforcing consistency, we propose to evaluate how concordant or discordant sources are as a measure of trustworthiness (the more discordant are the sources, the less we can trust their data). Thus, we define the discord measurement problem in which given a set of uncertain raw observations or aggregate results (such as case/hospitalization/death data relevant to COVID-19) and information on the alignment of different data (for example, cases and deaths), we wish to assess whether the different sources are concordant, or if not, measure how discordant they are., The work of Alberto Abelló has been done under project PID2020- 117191RB-I00 funded by MCIN/ AEI /10.13039/501100011033. The work of James Cheney was supported by ERC Consolidator Grant Skye (grant number 682315)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Mining app reviews for user feedback analysis in requirements engineering: a project report

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Tiessler Aguirre, Max
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Mining app reviews has emerged as a valuable practice in requirements engineering, providing insights into feature usage trends, user satisfaction, and emerging software issues. While recent advances in natural language processing have enhanced review analysis, challenges persist in feature extraction, sentiment ambiguity, and the scalability of automated methods, among others. This project report presents our research efforts in app review mining, focusing on methodological, software-based, and data-driven contributions. We explore both supervised and unsupervised learning approaches, leveraging large language models for key tasks such as feature identification, competition analysis, and emotion extraction. Additionally, we develop open-source tools and datasets to support reproducibility and adoption of our methods. Our findings highlight the potential of large language models in automating user feedback analysis while identifying gaps that require further research, particularly in addressing model reliability and evaluation challenges., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Inclusion and exclusion criteria in software engineering tertiary studies: A systematic mapping and emerging framework

  • Costal Costa, Dolors|||0000-0002-7340-0414
  • Farré Tost, Carles|||0000-0001-5814-3782
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Quer, Carme|||0000-0002-9000-6371
Context: Tertiary studies in software engineering (TS@SE) are widely used to synthesise evidence on a research topic systematically. As part of their protocol, TS@SE define inclusion and exclusion criteria (IC/EC) aimed at selecting those secondary studies (SS) to be included in the analysis. Aims: To provide a state of the art on the definition and application of IC/EC in TS@SE, and from the results of this analysis, we outline an emerging framework, TSICEC, to be used by SE researchers. Method: To provide the state of the art, we conducted a systematic mapping (SM) combining automatic search and snowballing over the body of SE scientific literature, which led to 50 papers after application of our own IC/EC. The extracted data was synthesised using content analysis. The results were used to define a first version of TSICEC. Results: The SM resulted in a coding schema, and a thorough analysis of the selected papers on the basis of this coding. Our TSICEC framework includes guidelines for the definition of IC/EC in TS@SE. Conclusion: This paper is a step forward establishing a foundation for researchers in two ways. As authors, understanding the different possibilities to define IC/EC and apply them to select SS. As readers, having an instrument to understand the methodological rigor upon which TS@SE may claim their findings., This work has been partially supported by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Insights on the use of software design principles in machine learning pipelines

  • López Cuesta, Lidia|||0000-0002-6901-9223
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
The rapid expansion of Artificial Intelligence has driven a surge in the development of Machine Learning (ML) pipelines, essential for constructing and maintaining ML models. Despite the growing recognition of the importance of high-quality ML pipelines, there is limited actionable guidance on improving pipelines quality at the implementation level. The Object-Oriented (OO) paradigm and software design principles have been used in software engineering to enhance software system quality. In this context, this work explores the coverage of existing ML pipelines in relation to OO concepts and software design principles. Our findings reveal a limited use of OO mechanisms and minimal adherence to software design principles. Specifically, issues such as poor code organization and deficient levels of coupling and cohesion were observed. Our results can shed some light on the development of strategies to overcome these eficiencies for improving ML pipelines quality., This work was supported by the “Spanish Ministerio de Ciencia e Innovación”, under project PID2020-117191RB-I00, funding scheme AEI/https://doi.org/10.13039/ 501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Workload placement on heterogeneous CPU-GPU systems

  • Nogueira Lobo de Carvalho, Marcos|||0000-0001-7015-9517
  • Simitsis, Alkis
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
The popularity of heterogeneous CPU-GPU processing has increased considerably in recent years. To efficiently utilize heterogeneous resources, data processing systems depend on an appropriate workload placement strategy to assign the right amount of compute to the right processor. However, finding an optimal placement strategy is not trivial due to various complex and conflicting tradeoffs related to the characteristics of processors, the nature of the workload, and data locality. In addition, placement decisions impact workload runtime and performance cost, and also depend on the availability of potentially different implementations for CPUs and GPUs, which adds extra complexity in such heterogeneous environments. In this tutorial, we review and compare state-of-the-art strategies for workload placement on heterogeneous CPU-GPU architectures, along with runtime prediction techniques and methods to support multi-device code. We also discuss open issues and identify potentially promising future research directions., This work has been partially supported by the H2020-MSCAITN2020 DEDS (GA.955895), and the EU-HORIZON programmes CREXDATA (GA.101092749) and FAIR-CORE4EOSC (GA.101057264) and the Spanish MICIU DOGO4ML (PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Adaptive task-oriented chatbots using feature-based knowledge bases

  • Campàs Gené, Carla
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Task-oriented chatbots relying on a knowledge base for domain-specific content exploitation have been largely addressed in research and industry applications. Despite this, multiple challenges remain to be fully conquered, including adaptive knowledge mechanisms, personalization for user-specific demands, and composite intent resolution. To address these challenges, in this paper, we present a work-in-progress summary of a task-oriented, knowledge-based chatbot in the field of mobile software ecosystems. The chatbot is designed to assist users in the combined use of multiple features from different applications. The proposed knowledge base and the machine learning pipeline supporting the chatbot technical core are designed to: (i) effectively use user context, (ii) process runtime feedback, (iii) use user historical data, and (iv) automatically infer slot values and dependent actions. With this report, we expect to lay the groundwork for future development stages and user validation studies., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Comprehensive assessment of open source software ecosystem health

  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Müller Cejas, Carlos Guillermo
  • Marco Gómez, Jordi|||0000-0002-0078-7929
  • Fernández Montes, Pablo
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Ruiz Cortés, Antonio
Recent surveys expose that the use of Open Source Software (OSS) is increasingly becoming a need for organizations in their development projects. However, deciding a proper OSS to be adopted or to contribute to its development is a complex and error-prone task. Analyzing the OSS ecosystem (OSSECO) health may help providing information about: (1) the OSS itself (number of commits, days after the last release, etc.); and (2) their main actors (number of contributors, partners, etc.). There exist proposals that go further and provide aggregated high-level indicators (e.g. visibility as an aggregation of number of community events, number of partners, and other metrics). Nevertheless, there is a lack of useful OSSECO analysis tools to ease the decision making on which OSSECO has the health required by a potential OSS adopter or contributor. In this work, we provide OSS-CARE (OSSeCo heAlthy monitoR and analysEr), an OSS-independent, fully automatic, and real-time framework to assess OSSECO’s health. OSS-CARE supports defining the ecosystem health objectives of potential OSS adopters, OSS contributors, and even OSS managers to inspect their provided health. These objectives are defined based on a well-established model characterizing health metrics that can be potentially aggregated by using a Bayesian network technique. Moreover, the integrated monitoring and analysis components perform an automated assessment of OSSECO’s health by checking the fulfillment of the required health objectives. Furthermore, the result is shown in an appealing dashboard that may ease the complex decision making of which OSS to choose., This work has been supported by the Spanish project PID2020-117191RB-I00 funded by MCIN/AEI/10.13039/501100011033; grants RTI2018-101204-B-C21, RTI2018-101204-B-C22, PID2021-126227NB-C21, PID2021-126227NB-C22 funded by MCIN/AEI/10.13039/501100011033 and ‘‘ERDF a way of making Europe’’; grants PYC20 RE 084 US, P18-FR-2895, US-1264651, US-1381595 funded by Junta de Andalucıa/ERDF, UE, Spain; and FPU19/00666 funded by MCIN/AEI/10.13039/501100011033 and by ‘‘ESF Investing in your future’’., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Data-driven agile requirements elicitation through the lenses of situational method engineering

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Henriksson, Aron
  • Ralyté, Jolita
  • Zdravkovic, Jelena
Ubiquitous digitalization has led to the continuous generation of large amounts of digital data, both in organizations and in society at large. In the requirements engineering community, there has been a growing interest in considering digital data as new sources for requirements elicitation, in addition to stake-holders. The volume, dynamics, and variety of data makes iterative requirements elicitation increasingly continuous, but also unstructured and complex, which current agile methods are unable to consider and manage in a systematic and efficient manner. There is also the need to support software evolution by enabling a synergy of stakeholder-driven requirements elicitation and management with data-driven approaches. In this study, we propose extension of agile requirements elicitation by applying situational method engineering. The research is grounded on two studies in the business domains of video games and online banking., The work presented in this paper is partially funded by the DOGO4ML Spanish research project, PID2020-117191RB-I00., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Estudio, informe, memoria (Report)

Application of SME in the DOGO4ML project: an example

  • Costal Costa, Dolors|||0000-0002-7340-0414
  • Quer, Carme|||0000-0002-9000-6371
The discipline of Situational Method Engineering (SME) [HRA14] provides theory and guidance to build methods that consider the specific situation of the organization or project at hand. This is facilitated by the modularity of the methods that are defined in terms of autonomous and interoperable building blocks (method components), which represent process fragments. These fragments can be assembled (composed) to define the intended method. The DOGO4ML project (https://dogo4ml.upc.edu/en) focuses on obtaining a holistic end-to-end framework to develop, operate and govern Machine Learning (ML) software systems and their data. To address the challenges imposed by this end, the DOGO4ML project uses SME to define its proposals in terms of method chunks to conform a library such that its elements can be combined into more general processes. This report illustrates the application of SME in the DOGO4ML project by means of an application example., This report illustrates the application of Situational Method Engineering in the DOGO4ML project by means of an application example. The application example is based on a proposal for the measurement and treatment of ML pipelines code smells., This work has been funded by the National Spanish Program for Research Aimed at the Challenges of Society 2020. The project and funding identifiers are PID2020-117191RB-I00/ AEI/10.13039/501100011033.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Towards continuous experiment-driven MLOps

  • Rajenthiram, Keerthiga
  • Abdullah, Milad
  • Gerostathopoulos, Ilias
  • Hnetynka, Petr
  • Bureš, Tomáš
  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Despite advancements in MLOps and AutoML, ML development still remains challenging for data scientists. First, there is poor support for and limited control over optimizing and evolving ML models. Second, there is lack of efficient mechanisms for continuous evolution of ML models which would leverage the knowledge gained in previous optimizations of the same or different models. We propose an experiment-driven MLOps approach which tackles these problems. Our approach relies on the concept of an experiment, which embodies a fully controllable optimization process. It introduces full traceability and repeatability to the optimization process, allows humans to be in full control of it, and enables continuous improvement of the ML system. Importantly, it also establishes knowledge, which is carried over and built across a series of experiments and allows for improving the efficiency of experimentation over time. We demonstrate our approach through its realization and application in the ExtremeXp11https://extremexp.eu/ project (Horizon Europe)., This work has been partially supported by the EU project ExtremeXP grant agreement 101093164, by Charles University institutional funding SVV 260698, and by the Spanish Ministerio de Ciencia e Innovación under project PID2020- 117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Mitigating data sparsity in integrated data through text conceptualization

  • Rahman, Md Ataur|||0000-0002-9092-8771
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Sacharidis, Dimitris
We study the data sparsity problem for data generated from an integration system. We approach the problem from a textual information extraction perspective and propose to conceptualize external documents using the concepts in the integrated schema. We present THOR, a novel system that, unlike related approaches, neither relies on complex rules nor models trained with large annotated corpus, but on the integrated data and its schema without the need for human annotations. An extensive evaluation on the text conceptualization task demonstrates the superiority of our approach in terms of F1-score, effort and use of resources over the state-of-the-art language models., Md Ataur Rahman is funded by the EC Horizon 2020 research and innovation programme (DEDS: grant agreement No 955895). Sergi Nadal and Oscar Romero are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML) and the EC Horizon Europe programme (ExtremeXP: grant agreement No 101093164)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

A data-science pipeline to enable the interpretability of many-objective feature selection

  • Njoku, Uchechukwu Fortune|||0000-0002-2599-9645
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Bilalli, Besim|||0000-0002-0575-2389
  • Bontempi, Gianluca
Many-Objective Feature Selection (MOFS) approaches use four or more objectives to determine the relevance of a subset of features in a supervised learning task. As a consequence, MOFS typically returns a large set of non-dominated solutions, which have to be assessed by the data scientist in order to proceed with the final choice. Given the multi-variate nature of the assessment, which may include objectives (e.g., fairness) unrelated to predictive accuracy, this step is often not straightforward and suffers from the lack of existing tools. For instance, it is common to make use of a tabular presentation of the solutions, which provides little information about the trade-offs and the relationships between objectives over the set of solutions. Adopting a GA-based MOFS with six objectives (number of selected features, balanced accuracy, F1-Score, variance inflation factor, statistical parity, and equalised odds) for two feature selection tasks, this paper illustrates the complex challenge of assessing MOFS results and the need for a methodology to aid and justify the final choice of a solution., The project leading to this publication has received funding from the European Union’s Horizon 2020 research and innovation programme (G.A. No. 955895). A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under G.A. No. 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Mobile feature-oriented knowledge base generation using knowledge graphs

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Knowledge bases are centralized repositories used for developing knowledge-oriented information systems. They are essential for adaptive, specialized knowledge in dialogue systems, supporting up-to-date domain-specific discussions with users. However, designing large-scale knowledge bases presents multiple challenges in data collection and knowledge exploitation. Knowledge graphs provide various research opportunities by integrating decentralized data and generating advanced knowledge. Our contribution presents a knowledge base in the form of a knowledge graph for extended knowledge generation for mobile apps and features extracted from related natural language documents. Our work encompasses the knowledge graph completion and deductive and inductive knowledge requests. We evaluated the effectiveness and performance of these knowledge strategies, which can be used as on-demand knowledge requests used by third-party software systems., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Leveraging requirements elicitation through software requirement patterns and LLMs

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Gnesi, Stefania
  • Paccosi, Federico
  • Quer, Carme|||0000-0002-9000-6371
  • Semini, Laura
Software requirement patterns (SRPs) is one of the many techniques that contribute to requirements elicitation. At this respect, the emergence of large language models (LLMs) opens the door to cost-effective strategies to create and use SRPs. Still, the stochastic nature of LLMs threatens the inherent quality of requirements reuse and consequently, that of the elicitation process. [Question/problem] In this scientific evaluation paper, we investigate whether and how LLMs can be used in order to create an SRP catalogue and elicit requirements from it. [Principal ideas/results] SRPs can be effectively extracted by querying an LLM through appropriate prompts, but still expert assessment is key in order to deliver the best results. LLM-driven generation of questions to stakeholders for eliciting requirements from these SRPs is feasible but suffers from deficiencies such as excessive number of repetitions and out of scope requirements. [Contribution] We show that (1) LLMs can be embedded into the requirements elicitation process through a pattern instantiation-based strategy, but at the same time (2) the current state of LLM technologies requires expert assessment at a large extent., This paper has been partially funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RBI00/AEI/10.13039/501100011033, by the Italian MIUR, under project PRIN 2022 STENDHAL and PNRR Project Securing sOftware Platforms.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

A data quality framework for graph-based virtual data integration systems

  • Li, Yalei
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Romero Moral, Óscar|||0000-0001-6350-8328
Data Quality (DQ) plays a critical role in data integration. Up to now, DQ has mostly been addressed from a single database perspective. Popular DQ frameworks rely on Integrity Constraints (IC) to enforce valid application semantics, which lead to the Denial Constraint (DC) formalism which models a broad range of ICs in real-world applications. Yet, current approaches are rather monolithic, considering a single database and do not suit data integration scenarios. In this paper, we address DQ for data integration systems. Specifically, we extend virtual data integration systems to elicit DCs from disparate data sources to be integrated, using DC-related state-of-the-art, and propagate them to the integrated schema (global DCs). Then, we propose a method to manage global DCs and identify (i) minimal DCs and (ii) potential clashes between them., This work was partly supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reproducible experiments for generating pre-processing pipelines for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim|||0000-0002-0575-2389
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
  • Silva Coira, Fernando
  • de Bernardo Roca, Guillermo
This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Diaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied., This work is partially supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP) and the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the project/funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

What about emotions? Guiding fine-grained emotion extraction from mobile app reviews

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Tiessler Aguirre, Max
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Opinion mining plays a vital role in analysing user feedback and extracting insights from textual data. While most research focuses on sentiment polarity (e.g., positive, negative, neutral), fine-grained emotion classification in app reviews remains underexplored. Fine-grained emotion classification is thus needed to better understand users’ affective responses and support downstream tasks such as feature-emotion analysis, user-oriented release planning, and issue triaging. This paper addresses this gap by identifying and addressing the challenges and limitations in fine-grained emotion analysis in the context of app reviews. Our study adapts Plutchik’s emotion taxonomy to app reviews by developing a structured annotation framework and dataset. Through an iterative human annotation process, we define clear annotation guidelines and document key challenges in emotion classification. Additionally, we evaluate the feasibility of automating emotion annotation using large language models, assessing their cost-effectiveness and agreement with human-labelled data. Our findings reveal that while large language models significantly reduce manual effort and maintain substantial agreement with human annotators, full automation remains challenging due to the complexity of emotional interpretation. This work contributes to opinion mining in requirements engineering by providing structured guidelines, an annotated dataset, and insights for developing automated pipelines to capture the complexity of emotions in app reviews., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745. This paper has been funded by the Spanish Ministerio de Ciencia e Innovacióin under project/funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

The state-of-practice in requirements specification: an extended interview study at 12 companies

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Palomares Bonache, Cristina|||0000-0003-4722-5584
  • Quer, Carme|||0000-0002-9000-6371
  • Chatzipetrou, Panagiota
  • Gorschek, Tony
Requirements specifcation is a core activity in the requirements engineering phase of a software development project. Researchers have contributed extensively to the feld of requirements specifcation, but the extent to which their proposals have been adopted in practice remains unclear. We gathered evidence about the state of practice in requirements specifcation by focussing on the artefacts used in this activity, the application of templates or guidelines, how requirements are structured in the specifcation document, what tools practitioners use to specify requirements, and what challenges they face. We conducted an interview-based survey study involving 24 practitioners from 12 diferent Swedish IT companies. We recorded the interviews and analysed these recordings, primarily by using qualitative methods. Natural language constitutes the main specifcation artefact but is usually accompanied by some other type of instrument. Most requirements specifcations use templates or guidelines, although they seldom follow any fxed standard. Requirements are always structured in the document according to the main functionalities of the system or to project areas or system parts. Diferent types of tools, including MS Ofce tools, are used, either individually or combined, in the compilation of requirements specifcations. We also note that challenges related to the use of natural language (dealing with ambiguity, inconsistency, and incompleteness) are the most frequent challenges that practitioners face in the compilation of requirements specifcations. These fndings are contextualized in terms of demographic factors related to the individual interviewees, the organization they are afliated with, and the project they selected to discuss during our interviews. A number of our fndings have been previously reported in related studies. These fndings show that, in spite of the large number of notations, models and tools proposed from academia for improving requirements specifcation, practitioners still mainly rely on plain natural language and general-purpose tool support. We expect more empirical studies in this area in order to better understand the reason of this low adoption of research results., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00/AEI/., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Architectural decisions in AI-based systems: an ontological view

  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
Architecting AI-based systems entails making some decisions that are particular to this type of systems. Therefore, it becomes necessary to gather all necessary knowledge to inform such decisions, and to articulate this knowledge in a form that facilitates knowledge transfer among different AI projects. In this exploratory paper, we first present the main results of a literature survey in the area, and then we propose a preliminary ontology for architectural decision making, which we exemplify using a subset of the papers selected in the literature review. In the discussion, we remark on the variety of decision types and system contexts, highlighting the need to further investigate the current state of research and practice in this area. Besides, we summarize our plans to move along this research area by widening the literature review and incorporating more AI-related concepts to this first version of the ontology., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Capítulo o parte de un libro (BookPart)

Domain-independent reference architectures and standards

  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
To remain competitive, organizations are challenged to make informed design decisions in order to construct software systems with similar architectural needs. They use reference architectures to achieve interoperability of (parts of) their software, standardization of software systems among multiple actors, and faster development with templates and guidelines for designing. In this chapter, we give an overview of domain-independent reference architectures and standards available in the gray literature for practitioners, classified by technology (cloud computing, big data, Internet of Things, and artificial intelligence) and analyzed based on their characteristics (e.g., purpose and contents). The discussed reference architectures are either published by standardization organizations such as ISO/IEC and NIST or fostered by large companies such as Microsoft and IBM., This work has been partially funded by the “Beatriz Galindo” Spanish Program BEAGAL18/00064 and by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Web API change-proneness prediction

  • Koçi, Rediana|||0000-0003-3367-950X
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
Change-proneness of software artifacts has been mainly related to the design characteristics and their previous history of changes. While these two aspects are essential and contribute significantly to the prediction, they leave out a critical factor: how the artifacts are being used. In the context of web APIs, consumers represent one of the main drivers of the change. Therefore, we propose a methodology for predicting the change-proneness of web API endpoint interfaces, taking into account not only design and change history but also their usage. Since the evolution of web APIs is and should be usage-driven, the way consumers use an API affects the future changes implemented by providers. Consequently, consumers' usage behavior contains essential information that contributes to identifying endpoints that are more prone to change. By considering the reasons behind changes, we introduce a set of metrics comprising design and usage aspects to be used as variables in prediction. To demonstrate the usefulness of the approach we perform an initial evaluation using a real-world web API. We quantify the introduced metrics using web API documentation, code, and usage logs in order to build a classifier able to predict with 82 % accuracy if an endpoint will change based on its design, history of changes, and usage characteristics., This paper has been funded by the Spanish Ministerio de Ciencia e Innovacion under project/funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Merging datasets for emotion analysis

  • de Arriba Serra, Ariadna
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Context. Applying sentiment analysis is in general a laborious task. Furthermore, if we add the task of getting a good quality dataset with balanced distribution and enough samples, the job becomes more complicated. Objective. We want to find out whether merging compatible datasets improves emotion analysis based on machine learning (ML) techniques, compared to the original, individual datasets. Method. We obtained two datasets with Covid-19-related tweets written in Spanish, and then built from them two new datasets combining the original ones with different consolidation of balance. We analyzed the results according to precision, recall, F1-score and accuracy. Results. The results obtained show that merging two datasets can improve the performance of ML models, particularly the F1-score, when the merging process follows a strategy that optimizes the balance of the resulting dataset. Conclusions. Merging two datasets can improve the performance of ML models for emotion analysis, whilst saving resources for labeling training data. This might be especially useful for several software engineering activities that leverage on ML-based emotion analysis techniques., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Replication in requirements engineering: the NLP for RE case

  • Abualhaija, Sallam
  • Aydemir, Fatma Basak
  • Dalpiaz, Fabiano
  • Dell'Anna, Davide
  • Ferrari, Alessio
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Fucci, Davide
Natural language processing (NLP) techniques have been widely applied in the requirements engineering (RE) field to support tasks such as classification and ambiguity detection. Despite its empirical vocation, RE research has given limited attention to replication of NLP for RE studies. Replication is hampered by several factors, including the context specificity of the studies, the heterogeneity of the tasks involving NLP, the tasks’ inherent hairiness, and, in turn, the heterogeneous reporting structure. To address these issues, we propose a new artifact, referred to as ID-Card, whose goal is to provide a structured summary of research papers emphasizing replication-relevant information. We construct the ID-Card through a structured, iterative process based on design science. In this article: (i) we report on hands-on experiences of replication; (ii) we review the state-of-the-art and extract replication-relevant information: (iii) we identify, through focus groups, challenges across two typical dimensions of replication: data annotation and tool reconstruction; and (iv) we present the concept and structure of the ID-Card to mitigate the identified challenges. This study aims to create awareness of replication in NLP for RE. We propose an ID-Card that is intended to foster study replication but can also be used in other contexts, e.g., for educational purposes., This article was partially supported by the following projects and grants: Italian MUR–PRIN 2020TL3X8X project T-LADIES (Typeful Language Adaptation for Dynamic, Interacting and Evolving Systems); EU Project CODECS GA 101060179, by the MOST – Sustainable Mobility National Research Center and received funding from the European Union Next- Generation EU (PIANO NAZIONALE DI RIPRESA E RESILIENZA (PNRR) – MISSIONE 4 COMPONENTE 2, INVESTIMENTO 1.4 – D.D. 1033 17/06/2022, CN00000023); KKS foundation through the S.E.R.T. Research Profile project at Blekinge Institute of Technology; Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RBI00/ AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Use and misuse of the term "Experiment" in mining software repositories research

  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
  • Turhan, Burak
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Juristo Juzgado, Natalia
The significant momentum and importance of Mining Software Repositories (MSR) in Software Engineering (SE) has fostered new opportunities and challenges for extensive empirical research. However, MSR researchers seem to struggle to characterize the empirical methods they use into the existing empirical SE body of knowledge. This is especially the case of MSR experiments. To provide evidence on the special characteristics of MSR experiments and their differences with experiments traditionally acknowledged in SE so far, we elicited the hallmarks that differentiate an experiment from other types of empirical studies and characterized the hallmarks and types of experiments in MSR. We analyzed MSR literature obtained from a small-scale systematic mapping study to assess the use of the term experiment in MSR. We found that 19% of the papers claiming to be an experiment are indeed not an experiment at all but also observational studies, so they use the term in a misleading way. From the remaining 81% of the papers, only one of them refers to a genuine controlled experiment while the others stand for experiments with limited control. MSR researchers tend to overlook such limitations, compromising the interpretation of the results of their studies. We provide recommendations and insights to support the improvement of MSR experiments., This work has been partially supported by the Spanish project: MCI PID2020-117191RB-I00., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Metrics for code smells of ML pipelines

  • Costal Costa, Dolors|||0000-0002-7340-0414
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
ML pipelines, as key components of ML systems, shall be developed following quality assurance techniques. Unfortunately, it is often the case in which they present maintainability issues, due to the experimentatal nature of data collection and ML model construction. To address this problem, this work in progress proposes initial metrics to measure the presence of code smells in ML pipelines. These metrics reflect good software engineering practices for code quality of ML pipelines., This work was supported by the “Spanish Ministerio de Ciencia e Innovación” under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

There is no data science without data governance: a proposal based on knowledge graphs

  • Bilalli, Besim|||0000-0002-0575-2389
  • Jovanovic, Petar|||0000-0003-4635-6646
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
Data Science and data-driven Artificial Intelligence are here to stay and they are expected to further transform the current global economy. From a technical point of view, there is an overall agreement that disciplines based on data require to combine data engineering and data analysis skills, but the fact is that data engineering is nowadays trailing and catching up with the rapid changes in the data analysis landscape. To unleash the real power of data, data-centric systems must be professionalized, i.e., operationalized and systematized, so that repetitive, time-consuming and error-prone tasks are automated. To such end, we propose our vision on next generation data governance for data-centric systems based on knowledge graphs. We claim that without the knowledge embedded in the data governance layer, Data Science will not unleash its potential., This work is supported by the Horizon Europe Programme under GA.101135513 (CYCLOPS) and GA. 101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 / AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Knowledge graphs for enhancing large language models in entity disambiguation

  • Pons Recasens, Gerard|||0000-0003-2225-3255
  • Bilalli, Besim|||0000-0002-0575-2389
  • Queralt Calafat, Anna|||0000-0003-2782-2955
Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities’ classes in a KG to gradually prune the candidate space as well as the entities’ descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG’s semantic expressivity on the ED performance., This work is supported by the Horizon Europe Programme under GA.101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML). Anna Queralt is a Serra Húnter Fellow., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Software-based dialogue systems: Survey, taxonomy and challenges

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
The use of natural language interfaces in the field of human-computer interaction is undergoing intense study through dedicated scientific and industrial research. The latest contributions in the field, including deep learning approaches like recurrent neural networks, the potential of context-aware strategies and user-centred design approaches, have brought back the attention of the community to software-based dialogue systems, generally known as conversational agents or chatbots. Nonetheless, and given the novelty of the field, a generic, context-independent overview on the current state of research of conversational agents covering all research perspectives involved is missing. Motivated by this context, this paper reports a survey of the current state of research of conversational agents through a systematic literature review of secondary studies. The conducted research is designed to develop an exhaustive perspective through a clear presentation of the aggregated knowledge published by recent literature within a variety of domains, research focuses and contexts. As a result, this research proposes a holistic taxonomy of the different dimensions involved in the conversational agents’ field, which is expected to help researchers and to lay the groundwork for future research in the field of natural language interfaces., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. The corresponding author gratefully acknowledges the Universitat Politècnica de Catalunya and Banco Santander for the inancial support of his predoctoral grant FPI-UPC. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Applying transfer learning to sentiment analysis in social media

  • de Arriba Serra, Ariadna
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Context: Sentiment analysis is an NLP technique that can be used to automatically obtain the sentiment of a crowd of end-users regarding a software application. However, applying sentiment analysis is a difficult task, especially considering the need of obtaining enough good quality data for training a Machine Learning (ML) model. To address this challenge, transfer learning can help us save time and get better performance results with a limited amount of data. Objective: In this paper, we aim at identifying to which degree transfer learning improves the results of sentiment analysis of messages shared by end-users in social media. Method: We propose a tool-supported framework able to monitor and analyze the sentiment of tweets with different ML models and settings. Using the proposed framework, we apply transfer learning and conduct a set of experiments with multiple datasets. Results: The performance of different ML models with transfer learning from different datasets are obtained and discussed, showing how different factors affect the results, and discussing how they have to be considered when applying transfer learning., This work has been partially supported by the Spanish project DOGO4ML (contract PID2020-117191RB-I00)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Incremental schema integration for data wrangling via knowledge graphs

  • Flores Herrera, Javier de Jesús|||0000-0002-2998-9962
  • Rabbani, Kashif
  • Nadal Francesch, Sergi|||0000-0002-8565-952X
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Jamin, Emmanuel
  • Dasiopoulou, Stamatia
Virtual data integration is the current approach to go for data wrangling in data-driven decision-making. In this paper, we focus on automating schema integration, which extracts a homogenised representation of the data source schemata and integrates them into a global schema to enable virtual data integration. Schema integration requires a set of well-known constructs: the data source schemata and wrappers, a global integrated schema and the mappings between them. Based on them, virtual data integration systems enable fast and on-demand data exploration via query rewriting. Unfortunately, the generation of such constructs is currently performed in a largely manual manner, hindering its feasibility in real scenarios. This becomes aggravated when dealing with heterogeneous and evolving data sources. To overcome these issues, we propose a fully-fledged semi-automatic and incremental approach grounded on knowledge graphs to generate the required schema integration constructs in four main steps: bootstrapping, schema matching, schema integration, and generation of system-specific constructs. We also present NextiaDI, a tool implementing our approach. Finally, a comprehensive evaluation is presented to scrutinize our approach., This work was partly supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00, and D3M project, funded by the Spanish Agencia Estatal de Investigación (AEI) under project PDC2021-121195-I00. Javier Flores is supported by contract 2020-DI-027 of the Industrial Doctorate Program of the Government of Catalonia and Consejo Nacional de Ciencia y Tecnología (CONACYT, Mexico). Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union – NextGenerationEU, under project FJC2020-045809-I., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos|||0000-0001-7015-9517
  • Queralt Calafat, Anna|||0000-0003-2782-2955
  • Romero Moral, Óscar|||0000-0001-6350-8328
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

HealthMesh: An architectural framework for federated healthcare data management

  • Bisquert Parés, Aniol
  • Hmimou Ham Man, Achraf|||0009-0004-0448-3406
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Gutiérrez Torre, Alberto|||0000-0002-5548-3359
  • Romero Moral, Óscar|||0000-0001-6350-8328
Recently, significant milestones have been achieved in the field of healthcare data analysis. However, alongside these accomplishments, substantial data-related challenges have emerged in the domain of big data management. Modern healthcare projects are no more dealing with a single data repository but many heterogeneous ones and must overcome data variety, privacy and governance issues. Yet, current solutions face a privacy-decentralization trade-off. To address this dual challenge, we introduce HealthMesh, a novel layered architectural framework based on the Data Mesh principles, providing a domaindecentralised paradigm. In addition, the framework incorporates a Semantic Data Model which establishes robust governance, enables interoperability and guarantees policy compliance for all the data assets. To demonstrate the capabilities of the proposed approach, we provide an illustrative example inspired by the use case of the INCISIVE project for breast cancer analytics. Overall, this work makes a significant contribution on collecting key challenges, identifying actors and providing a set of components and guidelines for establishing a holistic framework for the complex field of healthcare data management., The work reported here was supported by the EU-H2020 programme under GA.952179 INCISIVE, Horizon Europe Programme under GA.101095717 (SECURED) and GA.101135513 (CYCLOPS), the Spanish Research State Agency (MICINN/AEI, ERDF/FEDER) under GA.MCIN/AEI/10.13039/501100011033/FEDER (UE DALEST), the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00 / AEI/10.13039/501100011033 (DOGO4ML), and the Generalitat de Catalunya (AGAUR) 2021-SGR-00478 (CROMAI)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

TrustML: A Python package for computing the trustworthiness of ML models

  • Manzano Aguilar, Martí|||0000-0001-7944-7232
  • Ayala Martínez, Claudia Patricia|||0000-0002-6262-3698
  • Gómez Seoane, Cristina|||0000-0002-3872-0439
Artificial Intelligence has become increasingly important for aiding human decision making in various fields such as healthcare, automotive industry, education, and economy. Hence, in the last years, the interest of researchers and practitioners on developing and deploying Machine Learning (ML) models that are trustworthy is increasing. In this paper, we present TrustML, a simple to use, extensible and yet comprehensive Python package that allows specifying and computing the trustworthiness of ML models meant to conform AI systems. This package may be used for evaluating ML model's trustworthiness either during their development process and in production environments to support the continuous monitoring of their trustworthiness, benefiting taking correction actions when needed., This work was supported by “Agència de Gestió d’Ajuts Universitaris i de Recerca (Generalitat de Catalunya)” under grant agreement 2020 FI SDUR 00279, and by the “Spanish Ministerio de Ciencia e Innovación” under project / funding scheme PID2020–117191RB-I00 / AEI/ 10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Leveraging encoder-only large language models for mobile app review feature extraction

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Miaschi, Alessio
  • Dell’Orletta, Felice
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Mobile app review analysis presents unique challenges due to the low quality, subjective bias, and noisy content of user-generated documents. Extracting features from these reviews is essential for tasks such as feature prioritization and sentiment analysis, but it remains a challenging task. Meanwhile, encoder-only models based on the Transformer architecture have shown promising results for classification and information extraction tasks for multiple software engineering processes. This study explores the hypothesis that encoder-only large language models can enhance feature extraction from mobile app reviews. By leveraging crowdsourced annotations from an industrial context, we redefine feature extraction as a supervised token classification task. Our approach includes extending the pre-training of these models with a large corpus of user reviews to improve contextual understanding and employing instance selection techniques to optimize model fine-tuning. Empirical evaluations demonstrate that these methods improve the precision and recall of extracted features and enhance performance efficiency. Key contributions include a novel approach to feature extraction, annotated datasets, extended pre-trained models, and an instance selection mechanism for cost-effective fine-tuning. This research provides practical methods and empirical evidence in applying large language models to natural language processing tasks within mobile app reviews, offering improved performance in feature extraction., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033. This paper has been also supported by FAIR - Future AI Research (PE00000013) project under the NRRP MUR program funded by the NextGenerationEU., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Software engineering for AI-based systems: A survey

  • Martínez Fernández, Silverio Juan|||0000-0001-9928-133X
  • Bogner, Justus
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Siebert, Julien
  • Trendowicz, Adam
  • Vollmer, Anna Maria
  • Wagner, Stefan
AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image-, speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state-of-the-art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula., This work has been partially funded by the “Beatriz Galindo” Spanish Program BEAGAL18/00064 and by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

RE-Miner 2.0: a holistic framework for mining mobile application reviews

  • Tiessler Aguirre, Max
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
In the domain of application stores and marketplaces, user reviews are crucial for supporting multiple requirements engineering tasks. Feature extraction, emotion classification, topic analysis, review type identification, and polarity analysis are key components in requirements prioritization, feedback gathering, and release planning. Empirical evaluation of these techniques is challenging due to data collection complexities and a lack of reproducible methods and available tools. Furthermore, existing studies often focus on isolated tasks, hindering a comprehensive analysis of user perceptions. This paper introduces RE-Miner 2.0, a work-in-progress tool that integrates multiple data extraction and analysis methods in a distributed environment (RE-Miner Ecosystem), enabling a multidimensional and detailed analysis of user feedback. It offers a web-based service for task integration and comparison, supported by persistent storage and a web application that allows analytical visualization of reviews. As a result, RE-Miner 2.0 provides a platform for task integration, replication, and comparison of review mining techniques. Bringing advancements in deep review analysis for requirements engineering. A demo of the tool is showcased here: https://www.youtube.com/watch?v=a11bHSCYqqM., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Data pre-processing pipeline generation for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim|||0000-0002-0575-2389
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
Data pre-processing plays a key role in a data analytics process (e.g., applying a classification algorithm on a predictive task). It encompasses a broad range of activities that span from correcting errors to selecting the most relevant features for the analysis phase. There is no clear evidence, or rules defined, on how pre-processing transformations impact the final results of the analysis. The problem is exacerbated when transformations are combined into pre-processing pipeline prototypes. Data scientists cannot easily foresee the impact of pipeline prototypes and hence require a method to discriminate between them and find the most relevant ones (e.g., with highest positive impact) for their study at hand. Once found, these prototypes can be instantiated and optimized e.g., using Bayesian Optimization. In this work, we study the impact of transformations when chained together into prototypes, and the impact of transformations when instantiated via various operators. We develop and scrutinize a generic method that allows to generate pre-processing pipelines, as a step towards AutoETL. We make use of rules that enable the construction of prototypes (i.e., define the order of transformations), and rules that guide the instantiation of the transformations inside the prototypes (i.e., define the operator for each transformation). The optimization of our effective pipeline prototypes provide results that compared to an exhaustive search, get 90% of the predictive accuracy in the median, but with a time cost that is 24 times smaller., This work was supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project/ funding scheme PID2020-117191RB-I00/AEI/10.13039/50110001 1033. We thank University of Bologna for issuing a grant for author’s research stay at Universitat Politècnica de Catalunya., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Impact study of incremental NoSQL refactoring in SkyServer database

  • Gallinucci, Enrico
  • Golfarelli, Matteo
  • Radwan, Wafaa
  • Zarate Calderón, Gabriel
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
Data modeling in NoSQL databases is notoriously complex and driven by multiple and possibly conflicting requirements. Researchers have proposed methodologies to optimize schema design of a given domain for a given workload; however, due to the agile environment in which NoSQL databases are usually employed, both domain and workload are frequently subject to changes and evolution — possibly neutralizing the benefits of optimization. When this happens, the benefits of a new optimal schema design must be weighed against the costs of migrating the data. In this work, we empirically show the benefits of schema redesign in a real publicly available database. In particular, we identify multiple snapshots (in terms of domain extension and querying workload) in the 20+ years evolution of SkyServer, demonstrate how NoSQL schema optimization at a given time can later backfire, and evaluate the conditions under which data migration becomes beneficial. This takes us to define the foundations and challenges of a framework for continuous NoSQL database refactoring, with the goal of helping DBAs and data engineers decide if, when, and how a NoSQL database should be reconsidered to restore schema design optimality; special attention is paid to the optimization of data migration tasks scheduling., This work has been partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Impact study of NoSQL refactoring in SkyServer database

  • Gallinucci, Enrico
  • Golfarelli, Matteo
  • Radwan, Wafaa
  • Zarate Calderón, Gabriel
  • Abelló Gamazo, Alberto|||0000-0002-3223-2186
Data modeling in NoSQL databases is notoriously complex and driven by multiple and possibly conflicting requirements. Researchers have proposed methodologies to optimize schema design of a given domain for a given workload; however, due to the agile environment in which NoSQL databases are usually employed, both domain and workload are frequently subject to changes and evolution- possibly neutralizing the benefits of optimization. When this happens, the benefits of a new optimal schema design must be weighed against the costs of migrating the data. In this work, we empirically show the benefits of schema redesign in a real publicly available database. In particular, we identify multiple snapshots (in terms of domain extension and querying workload) in the 20+ years evolution of SkyServer, demonstrate how NoSQL schema optimization at a given time can later backfire, and evaluate the conditions under which data migration becomes beneficial. This takes us to define the foundations and challenges of a framework for continuous NoSQL database refactoring, with the goal of helping DBAs and data engineers decide if, when, and how a NoSQL database should be reconsidered to restore schema design optimality., This work has been partially supported by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

TransFeatEx: a NLP pipeline for feature extraction

  • Gallego Marfa, Agustí
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Mobile app stores provide centralized access to a large data set of mobile app related natural language textual data, including developer’s documentation (e.g., descriptions, changelogs) and user-generated data (e.g., user reviews). Motivated by this context, multiple studies have focused on data-driven elicitation processes for the automatic extraction of the set of features exposed by a catalogue of applications and the inferred, extended knowledge that can be derived from this information. Moreover, with the emerging and generalization of large language models, traditional linguistic-based approaches can be significantly improved by the potential of the knowledge embedded in this kind of models. In this paper, we present TransFeatEx, a NLP-based feature extraction pipeline that combines the use of a RoBERTa-based model with the application of consolidated syntactic and semantic techniques. The pipeline is designed as a customizable, standalone service to be used either as a playground, experimentation tool or as a software component to be embedded into a third-party software system for batch-processing large document corpora. An example of a demo plan is showcased here: https://youtu.be/gfFyi_i_uTw, With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Unveiling competition dynamics in mobile app markets through user reviews

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Gervasi, Vincenzo
  • Marco Gómez, Jordi|||0000-0002-0078-7929
[Context and motivation] User reviews published in mobile app repositories are essential for understanding user satisfaction and engagement within a specific market segment. [Question/problem] Manual analysis of reviews is impractical due to the large data volume, and automated analysis faces challenges like data synthesis and reporting. This complicates the task for app providers in identifying patterns and significant events, especially in assessing the influence of competitor apps. Furthermore, review-based research is mostly limited to a single app or a single app provider, excluding potential competition analysis. Consequently, there is an open research challenge in leveraging user reviews to support cross-app analysis within a specific market segment. [Principal ideas/results] Following a case-study research method in the microblogging app market, we introduce an automatic, novel approach to support mobile app market analysis. Our approach leverages quantitative metrics and event detection techniques based on newly published user reviews. Significant events are proactively identified and summarized by comparing metric deviations with historical baseline indicators within the lifecycle of a mobile app. [Contribution] Results from our case study show empirical evidence of the detection of relevant events within the selected market segment, including software- or release-based events, contextual events and the emergence of new competitors., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RBI00/AEI/10.13039/501100011033., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Inclusion and exclusion criteria in software engineering tertiary studies: A systematic mapping and emerging framework

  • Costa, Dolors
  • Farré Tost, Carles
  • Franch Gutiérrez, Javier
  • Quer, Carme
Context: Tertiary studies in software engineering (TS@SE) are widely used to synthesise evidence on a research topic systematically. As part of their protocol, TS@SE define inclusion and exclusion criteria (IC/EC) aimed at selecting those secondary studies (SS) to be included in the analysis. Aims: To provide a state of the art on the definition and application of IC/EC in TS@SE, and from the results of this analysis, we outline an emerging framework, TSICEC, to be used by SE researchers. Method: To provide the state of the art, we conducted a systematic mapping (SM) combining automatic search and snowballing over the body of SE scientific literature, which led to 50 papers after application of our own IC/EC. The extracted data was synthesised using content analysis. The results were used to define a first version of TSICEC. Results: The SM resulted in a coding schema, and a thorough analysis of the selected papers on the basis of this coding. Our TSICEC framework includes guidelines for the definition of IC/EC in TS@SE. Conclusion: This paper is a step forward establishing a foundation for researchers in two ways. As authors, understanding the different possibilities to define IC/EC and apply them to select SS. As readers, having an instrument to understand the methodological rigor upon which TS@SE may claim their findings., This work has been partially supported by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Discovery of semantic non-syntactic joins

  • Maynou Yelamos, Marc
  • Nadal Francesch, Sergi
Data discovery is an essential step in the data integration pipeline involving finding datasets whose combined information provides relevant insights. Discovering joinable attributes requires assessing the closeness of the semantic concepts that two attributes represent, which is highly sensitive and dependent on the chosen similarity metric. The state of the art commonly approaches this task from a syntactic perspective, this is, performing comparisons based on the data values or on direct transformations (e.g., via hash functions). These approaches suffice when the two sets of instances share the same syntactic representation, but fail to detect cases in which the same semantic concept is represented by different sets of values, which we refer as semantic non-syntactic joins. This is a relevant problem in data lake scenarios, when the underlying datasets present high heterogeneity and lack of standardization. To that end, in this paper, we propose an empirical approach to detect semantic non-syntactic joins, which leverages, simultaneously, syntactic and semantic measurements of the data. We demonstrate that our approach is effective in detecting such kind of joins., Marc Maynou is supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP), and Sergi Nadal is partially supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación

SOPCOM: A tool to manage software process components for situational method engineering

  • Costa, Dolors
  • Quer, Carme
  • Franch Gutiérrez, Javier
Situational Method Engineering (SME) principles and techniques have been proposed to promote the modularization of method knowledge in the form of reusable method components and their composition into new situation specific methods. In this paper, we present SOPCOM, a tool aimed at supporting the creation and management of an SME method components (“chunks”) catalogue, and the composition of these components into methods., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project /funding scheme PID2020-117191RB-I00 /AEI/10.13039/501100011033., Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Operationalizing and automating data governance

  • Nadal Francesch, Sergi
  • Jovanovic, Petar
  • Bilalli, Besim
  • Romero Moral, Óscar
The ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens or hundreds of continuously evolving data sources produce semi-structured data. To overcome this challenge, we develop a framework for operationalizing and automating data governance. For the first, we propose a zoned data lake architecture and a set of data governance processes that allow the systematic ingestion, transformation and integration of data from heterogeneous sources, in order to make them readily available for business users. For the second, we propose a set of metadata artifacts that allow the automatic execution of data governance processes, addressing a wide range of data management challenges. We showcase the usefulness of the proposed approach using a real world use case, stemming from the collaborative project with the World Health Organization for the management and analysis of data about Neglected Tropical Diseases. Overall, this work contributes on facilitating organizations the adoption of data-driven strategies into a cohesive framework operationalizing and automating data governance., This work was partly supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I/AEI/10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Insights on the use of software design principles in machine learning pipelines

  • López Cuesta, Lidia
  • Gómez Seoane, Cristina
  • Ayala Martínez, Claudia Patricia
The rapid expansion of Artificial Intelligence has driven a surge in the development of Machine Learning (ML) pipelines, essential for constructing and maintaining ML models. Despite the growing recognition of the importance of high-quality ML pipelines, there is limited actionable guidance on improving pipelines quality at the implementation level. The Object-Oriented (OO) paradigm and software design principles have been used in software engineering to enhance software system quality. In this context, this work explores the coverage of existing ML pipelines in relation to OO concepts and software design principles. Our findings reveal a limited use of OO mechanisms and minimal adherence to software design principles. Specifically, issues such as poor code organization and deficient levels of coupling and cohesion were observed. Our results can shed some light on the development of strategies to overcome these eficiencies for improving ML pipelines quality., This work was supported by the “Spanish Ministerio de Ciencia e Innovación”, under project PID2020-117191RB-I00, funding scheme AEI/https://doi.org/10.13039/ 501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

T-FREX: a transformer-based feature extraction method from mobile app reviews

  • Motger de la Encarnación, Joaquim
  • Miaschi, Alessio
  • Dell’Orletta, Felice
  • Franch Gutiérrez, Javier
  • Marco Gómez, Jordi
Mobile app reviews are a large-scale data source for software-related knowledge generation activities, including software maintenance, evolution and feedback analysis. Effective extraction of features (i.e., functionalities or characteristics) from these reviews is key to support analysis on the acceptance of these features, identification of relevant new feature requests and prioritization of feature development, among others. Traditional methods focus on syntactic pattern-based approaches, typically context-agnostic, evaluated on a closed set of apps, difficult to replicate and limited to a reduced set and domain of apps. Mean-while, the pervasiveness of Large Language Models (LLMs) based on the Transformer architecture in software engineering tasks lays the groundwork for empirical evaluation of the performance of these models to support feature extraction. In this study, we present T-FREX, a Transformer-based, fully automatic approach for mobile app review feature extraction. First, we collect a set of ground truth features from users in a real crowdsourced software recommendation platform and transfer them automatically into a dataset of app reviews. Then, we use this newly created dataset to fine-tune multiple LLMs on a named entity recognition task under different data configurations. We assess the performance of T- FREX with respect to this ground truth, and we complement our analysis by comparing T- FREX with a baseline method from the field. Finally, we assess the quality of new features predicted by T- FREX through an external human evaluation. Results show that T- FREX outperforms on average the traditional syntactic-based method, especially when discovering new features from a domain for which the model has been fine-tuned., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovacion under project I funding scheme PID2020-117191RB-100 I AEI/10.13039/501100011033. Alessio Miaschi and Felice Dell’ Orletta have also been supported by the PNRR project FAIR - Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

There is no data science without data governance: a proposal based on knowledge graphs

  • Bilalli, Besim
  • Jovanovic, Petar
  • Nadal Francesch, Sergi
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
Data Science and data-driven Artificial Intelligence are here to stay and they are expected to further transform the current global economy. From a technical point of view, there is an overall agreement that disciplines based on data require to combine data engineering and data analysis skills, but the fact is that data engineering is nowadays trailing and catching up with the rapid changes in the data analysis landscape. To unleash the real power of data, data-centric systems must be professionalized, i.e., operationalized and systematized, so that repetitive, time-consuming and error-prone tasks are automated. To such end, we propose our vision on next generation data governance for data-centric systems based on knowledge graphs. We claim that without the knowledge embedded in the data governance layer, Data Science will not unleash its potential., This work is supported by the Horizon Europe Programme under GA.101135513 (CYCLOPS) and GA. 101093164 (ExtremeXP) and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 / AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Web API change-proneness prediction

  • Koçi, Rediana
  • Franch Gutiérrez, Javier
  • Jovanovic, Petar
  • Abelló Gamazo, Alberto
Change-proneness of software artifacts has been mainly related to the design characteristics and their previous history of changes. While these two aspects are essential and contribute significantly to the prediction, they leave out a critical factor: how the artifacts are being used. In the context of web APIs, consumers represent one of the main drivers of the change. Therefore, we propose a methodology for predicting the change-proneness of web API endpoint interfaces, taking into account not only design and change history but also their usage. Since the evolution of web APIs is and should be usage-driven, the way consumers use an API affects the future changes implemented by providers. Consequently, consumers' usage behavior contains essential information that contributes to identifying endpoints that are more prone to change. By considering the reasons behind changes, we introduce a set of metrics comprising design and usage aspects to be used as variables in prediction. To demonstrate the usefulness of the approach we perform an initial evaluation using a real-world web API. We quantify the introduced metrics using web API documentation, code, and usage logs in order to build a classifier able to predict with 82 % accuracy if an endpoint will change based on its design, history of changes, and usage characteristics., This paper has been funded by the Spanish Ministerio de Ciencia e Innovacion under project/funding scheme PID2020-117191RB-I00/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Towards continuous experiment-driven MLOps

  • Rajenthiram, Keerthiga
  • Abdullah, Milad
  • Gerostathopoulos, Ilias
  • Hnetynka, Petr
  • Bureš, Tomáš
  • Pons Recasens, Gerard
  • Bilalli, Besim
  • Queralt Calafat, Anna
Despite advancements in MLOps and AutoML, ML development still remains challenging for data scientists. First, there is poor support for and limited control over optimizing and evolving ML models. Second, there is lack of efficient mechanisms for continuous evolution of ML models which would leverage the knowledge gained in previous optimizations of the same or different models. We propose an experiment-driven MLOps approach which tackles these problems. Our approach relies on the concept of an experiment, which embodies a fully controllable optimization process. It introduces full traceability and repeatability to the optimization process, allows humans to be in full control of it, and enables continuous improvement of the ML system. Importantly, it also establishes knowledge, which is carried over and built across a series of experiments and allows for improving the efficiency of experimentation over time. We demonstrate our approach through its realization and application in the ExtremeXp11https://extremexp.eu/ project (Horizon Europe)., This work has been partially supported by the EU project ExtremeXP grant agreement 101093164, by Charles University institutional funding SVV 260698, and by the Spanish Ministerio de Ciencia e Innovación under project PID2020- 117191RB-I00/ AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Mobile feature-oriented knowledge base generation using knowledge graphs

  • Motger de la Encarnación, Joaquim
  • Franch Gutiérrez, Javier
  • Marco Gómez, Jordi
Knowledge bases are centralized repositories used for developing knowledge-oriented information systems. They are essential for adaptive, specialized knowledge in dialogue systems, supporting up-to-date domain-specific discussions with users. However, designing large-scale knowledge bases presents multiple challenges in data collection and knowledge exploitation. Knowledge graphs provide various research opportunities by integrating decentralized data and generating advanced knowledge. Our contribution presents a knowledge base in the form of a knowledge graph for extended knowledge generation for mobile apps and features extracted from related natural language documents. Our work encompasses the knowledge graph completion and deductive and inductive knowledge requests. We evaluated the effectiveness and performance of these knowledge strategies, which can be used as on-demand knowledge requests used by third-party software systems., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Goal-oriented models for teaching and understanding data structures

  • Franch Gutiérrez, Javier
  • Ruiz, Marcela
Most computer science curricula include a compulsory course on data structures. Students are prone to memorise facts about data structures instead of understanding the essence of underlying concepts. This can be explained by the fact that learning the basics of each data structure, the difference with each other, and the adequacy of each of them to the most appropriate context of use, is far from trivial. This paper explores the idea of providing adequate levels of abstractions to describe data structures from an intentional point of view. Our hypothesis is that adopting a goal-oriented perspective could emphasise the main goals of each data structure, its qualities, and its relationships with the potential context of use. Following this hypothesis, in this paper we present the use of iStar2.0 to teach and understand data structures. We conducted a comparative quasi-experiment with undergraduate students to evaluate the effectiveness of the approach. Significant results show the great potential of goal modeling for teaching technical courses like data structures. We conclude this paper by reflecting on further teaching and conceptual modeling research to be conducted in this field., This work has been partially supported by the by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00), the Digitalization Initiative of the Canton of Zü-rich (DIZH), and ZHAW Digital., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Leveraging requirements elicitation through software requirement patterns and LLMs

  • Franch Gutiérrez, Javier
  • Gnesi, Stefania
  • Paccosi, Federico
  • Quer, Carme
  • Semini, Laura
Software requirement patterns (SRPs) is one of the many techniques that contribute to requirements elicitation. At this respect, the emergence of large language models (LLMs) opens the door to cost-effective strategies to create and use SRPs. Still, the stochastic nature of LLMs threatens the inherent quality of requirements reuse and consequently, that of the elicitation process. [Question/problem] In this scientific evaluation paper, we investigate whether and how LLMs can be used in order to create an SRP catalogue and elicit requirements from it. [Principal ideas/results] SRPs can be effectively extracted by querying an LLM through appropriate prompts, but still expert assessment is key in order to deliver the best results. LLM-driven generation of questions to stakeholders for eliciting requirements from these SRPs is feasible but suffers from deficiencies such as excessive number of repetitions and out of scope requirements. [Contribution] We show that (1) LLMs can be embedded into the requirements elicitation process through a pattern instantiation-based strategy, but at the same time (2) the current state of LLM technologies requires expert assessment at a large extent., This paper has been partially funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RBI00/AEI/10.13039/501100011033, by the Italian MIUR, under project PRIN 2022 STENDHAL and PNRR Project Securing sOftware Platforms., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Hyppo: using equivalences to optimize pipelines in exploratory machine learning

  • Kontaxakis, Antonios
  • Sacharidis, Dimitris
  • Simitsis, Alkis
  • Abelló Gamazo, Alberto
  • Nadal Francesch, Sergi
We present HYPPO, a novel system to optimize pipelines encountered in exploratory machine learning. HYPPO exploits alternative computational paths of artifacts from past executions to derive better execution plans while reusing materialized artifacts. Adding alternative computations introduces new challenges for exploratory machine learning regarding workload representation, system architecture, and optimal execution plan generation. To this end, we present a novel workload representation based on directed hypergraphs, and we formulate the problem of discovering the optimal execution plan as a search problem over directed hypergraphs and that of selecting artifacts to materialize as an optimization problem. A thorough experimental evaluation shows that HYPPO results in plans that are typically one order (up to two orders) of magnitude faster and cheaper than the non-optimized pipeline and considerably (up to one order of magnitude) faster and cheaper than plans generated by the state of the art when materializing artifacts is possible. Lastly, our evaluation reveals that HYPPO reduces the cost by 3–4× even when materialization cannot be exploited., This work has been partially supported by the H2020-MSCAITN-2020 DEDS (GA.955895), the EU-HORIZON programmes FAIR-CORE4EOSC (GA.101057264), CREXDATA (GA.101092749), ExtremeXP (GA.101093164), and the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033 (DOGO4ML)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Software analytics tools: an intentional view

  • Franch Gutiérrez, Javier
Software analytic tools consume big amounts of data coming from either (or both) the software development process or the system usage and aggregate them into indicators which are rendered to different types of stakeholders, also offering them a portfolio of techniques and capabilities such as what-if analysis, prediction and alerts. Precisely, the variety of stakeholders and the different goals they pursue justifies the convenience of performing an intentional analysis of the use of software analytics tools. With this aim, we first enumerate the different stakeholders and identify their intentional relationships with software analytics tools in the form of dependencies. Then, we focus on one particular stakeholder, namely the requirements engineer, and identify further intentional elements represented in a strategic rationale model. The resulting model provides an abstract view of the domain which may help stakeholders when deciding on the adoption of software analytic tools in their particular context., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación

Application of SME in the DOGO4ML project: an example

  • Costa, Dolors
  • Quer, Carme
The discipline of Situational Method Engineering (SME) [HRA14] provides theory and guidance to build methods that consider the specific situation of the organization or project at hand. This is facilitated by the modularity of the methods that are defined in terms of autonomous and interoperable building blocks (method components), which represent process fragments. These fragments can be assembled (composed) to define the intended method. The DOGO4ML project (https://dogo4ml.upc.edu/en) focuses on obtaining a holistic end-to-end framework to develop, operate and govern Machine Learning (ML) software systems and their data. To address the challenges imposed by this end, the DOGO4ML project uses SME to define its proposals in terms of method chunks to conform a library such that its elements can be combined into more general processes. This report illustrates the application of SME in the DOGO4ML project by means of an application example., This report illustrates the application of Situational Method Engineering in the DOGO4ML project by means of an application example. The application example is based on a proposal for the measurement and treatment of ML pipelines code smells., This work has been funded by the National Spanish Program for Research Aimed at the Challenges of Society 2020. The project and funding identifiers are PID2020-117191RB-I00/ AEI/10.13039/501100011033., Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Reproducible experiments for generating pre-processing pipelines for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim
  • Abelló Gamazo, Alberto
  • Silva Coira, Fernando
  • de Bernardo Roca, Guillermo
This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Diaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied., This work is partially supported by the EU’s Horizon Programme call, under Grant Agreements No. 101093164 (ExtremeXP) and the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under the project/funding scheme PID2020-117191RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Data pre-processing pipeline generation for AutoETL

  • Giovanelli, Joseph
  • Bilalli, Besim
  • Abelló Gamazo, Alberto
Data pre-processing plays a key role in a data analytics process (e.g., applying a classification algorithm on a predictive task). It encompasses a broad range of activities that span from correcting errors to selecting the most relevant features for the analysis phase. There is no clear evidence, or rules defined, on how pre-processing transformations impact the final results of the analysis. The problem is exacerbated when transformations are combined into pre-processing pipeline prototypes. Data scientists cannot easily foresee the impact of pipeline prototypes and hence require a method to discriminate between them and find the most relevant ones (e.g., with highest positive impact) for their study at hand. Once found, these prototypes can be instantiated and optimized e.g., using Bayesian Optimization. In this work, we study the impact of transformations when chained together into prototypes, and the impact of transformations when instantiated via various operators. We develop and scrutinize a generic method that allows to generate pre-processing pipelines, as a step towards AutoETL. We make use of rules that enable the construction of prototypes (i.e., define the order of transformations), and rules that guide the instantiation of the transformations inside the prototypes (i.e., define the operator for each transformation). The optimization of our effective pipeline prototypes provide results that compared to an exhaustive search, get 90% of the predictive accuracy in the median, but with a time cost that is 24 times smaller., This work was supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project/ funding scheme PID2020-117191RB-I00/AEI/10.13039/50110001 1033. We thank University of Bologna for issuing a grant for author’s research stay at Universitat Politècnica de Catalunya., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The state-of-practice in requirements specification: an extended interview study at 12 companies

  • Franch Gutiérrez, Javier
  • Palomares Bonache, Cristina
  • Quer, Carme
  • Chatzipetrou, Panagiota
  • Gorschek, Tony
Requirements specifcation is a core activity in the requirements engineering phase of a software development project. Researchers have contributed extensively to the feld of requirements specifcation, but the extent to which their proposals have been adopted in practice remains unclear. We gathered evidence about the state of practice in requirements specifcation by focussing on the artefacts used in this activity, the application of templates or guidelines, how requirements are structured in the specifcation document, what tools practitioners use to specify requirements, and what challenges they face. We conducted an interview-based survey study involving 24 practitioners from 12 diferent Swedish IT companies. We recorded the interviews and analysed these recordings, primarily by using qualitative methods. Natural language constitutes the main specifcation artefact but is usually accompanied by some other type of instrument. Most requirements specifcations use templates or guidelines, although they seldom follow any fxed standard. Requirements are always structured in the document according to the main functionalities of the system or to project areas or system parts. Diferent types of tools, including MS Ofce tools, are used, either individually or combined, in the compilation of requirements specifcations. We also note that challenges related to the use of natural language (dealing with ambiguity, inconsistency, and incompleteness) are the most frequent challenges that practitioners face in the compilation of requirements specifcations. These fndings are contextualized in terms of demographic factors related to the individual interviewees, the organization they are afliated with, and the project they selected to discuss during our interviews. A number of our fndings have been previously reported in related studies. These fndings show that, in spite of the large number of notations, models and tools proposed from academia for improving requirements specifcation, practitioners still mainly rely on plain natural language and general-purpose tool support. We expect more empirical studies in this area in order to better understand the reason of this low adoption of research results., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00/AEI/., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Measuring discord among multidimensional data sources

  • Abelló Gamazo, Alberto
  • Cheney, James
Data integration is a classical problem in databases, typically decomposed into schema matching, entity matching and record merging. To solve the latter, it is mostly assumed that ground truth can be determined, either as master data or from user feedback. However, in many cases, this is not the case because firstly the merging processes cannot be accurate enough, and also the data gathering processes in the different sources are simply imperfect and cannot provide high quality data. Instead of enforcing consistency, we propose to evaluate how concordant or discordant sources are as a measure of trustworthiness (the more discordant are the sources, the less we can trust their data). Thus, we define the discord measurement problem in which given a set of uncertain raw observations or aggregate results (such as case/hospitalization/death data relevant to COVID-19) and information on the alignment of different data (for example, cases and deaths), we wish to assess whether the different sources are concordant, or if not, measure how discordant they are., The work of Alberto Abelló has been done under project PID2020- 117191RB-I00 funded by MCIN/ AEI /10.13039/501100011033. The work of James Cheney was supported by ERC Consolidator Grant Skye (grant number 682315)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The knowledge graph lifecycle in NTT DATA

  • Flores Herrera, Javier de Jesús
  • Jamin, Emmanuel
  • Nadal Francesch, Sergi
  • Romero Moral, Óscar
The Semantic Business Unit (SEMBU) in NTT DATA aims to increase the semantic interoper ability and accessibility of European institutions’ data projects by following Linked Open Data (LOD) principles to build controlled vocabularies and produce Knowledge Graphs (KGs). One of its most notable projects revolves around the CORDIS portal1, which publishes information about research and innovation projects funded by the European Commission. SEMBU pursues two main goals: (i) expose semantic data related to CORDIS via a SPARQL endpoint that facilitates access and reuse of quality scientific-related data, and (ii) design an efficient, incremental, and automated KG lifecycle to be used as a reference in other data projects. To that end, we have adopted state-of-the-art semantic technologies to support the creation and management of the KG with the goal of centralizing knowledge and providing an overall view of data assets that improve data governance, maintenance, and external interaction by data consumers. We have also identified some of their limitations which are tackled via an industrial PhD. This paper reports our experience, the obstacles, and proposals for generating and maintaining the CORDIS KG., This work was partly funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00 (DOGO4ML). Javier Flores is supported by contract 2020-DI-027 of the Industrial Doctorate Program of the Government of Catalonia and CONACYT’s scholarship. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I /AEI/10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Architectural decisions in AI-based systems: an ontological view

  • Franch Gutiérrez, Javier
  • Martínez Fernández, Silverio Juan
  • Ayala Martínez, Claudia Patricia
  • Gómez Seoane, Cristina
Architecting AI-based systems entails making some decisions that are particular to this type of systems. Therefore, it becomes necessary to gather all necessary knowledge to inform such decisions, and to articulate this knowledge in a form that facilitates knowledge transfer among different AI projects. In this exploratory paper, we first present the main results of a literature survey in the area, and then we propose a preliminary ontology for architectural decision making, which we exemplify using a subset of the papers selected in the literature review. In the discussion, we remark on the variety of decision types and system contexts, highlighting the need to further investigate the current state of research and practice in this area. Besides, we summarize our plans to move along this research area by widening the literature review and incorporating more AI-related concepts to this first version of the ontology., This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Applying transfer learning to sentiment analysis in social media

  • de Arriba Serra, Ariadna
  • Oriol Hilari, Marc
  • Franch Gutiérrez, Javier
Context: Sentiment analysis is an NLP technique that can be used to automatically obtain the sentiment of a crowd of end-users regarding a software application. However, applying sentiment analysis is a difficult task, especially considering the need of obtaining enough good quality data for training a Machine Learning (ML) model. To address this challenge, transfer learning can help us save time and get better performance results with a limited amount of data. Objective: In this paper, we aim at identifying to which degree transfer learning improves the results of sentiment analysis of messages shared by end-users in social media. Method: We propose a tool-supported framework able to monitor and analyze the sentiment of tweets with different ML models and settings. Using the proposed framework, we apply transfer learning and conduct a set of experiments with multiple datasets. Results: The performance of different ML models with transfer learning from different datasets are obtained and discussed, showing how different factors affect the results, and discussing how they have to be considered when applying transfer learning., This work has been partially supported by the Spanish project DOGO4ML (contract PID2020-117191RB-I00)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Using metrics for code smells of ML pipelines

  • Costa, Dolors
  • Gómez Seoane, Cristina
  • Rey Juárez, Santiago del
  • Martínez Fernández, Silverio Juan
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. https://dx.doi.org/0.1109/ICSA-C63560.2024.00055, ML pipelines, as key components of ML systems, shall be developed following quality assurance techniques. Unfortunately, it is often the case in which they present maintainability issues, due to the experimental nature of data collection and ML model construction. In this work, we perform a first evaluation of a set of metrics, proposed in previous research, for measuring the presence of code smells related to maintainability, in ML pipeline application examples. Moreover, we provide the lessons learnt and insights gained from this evaluation., This work has been funded by the National Spanish Program for Research Aimed at the Challenges of Society 2020. The project and funding identifiers are PID2020-117191RB-I00/ AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

MApp-KG: Mobile app knowledge graph for document-based feature knowledge generation

  • Motger de la Encarnación, Joaquim
  • Franch Gutiérrez, Javier
  • Marco Gómez, Jordi
Mobile app repositories serve as large-scale crowdsourced information systems used for various document-based software engineering tasks, leveraging product descriptions, user reviews, and other natural language documents. Particularly, feature extraction (i.e., identifying functionalities or capabilities of a mobile app mentioned in these documents) is key for product recommendation, topic modelling, and feedback analysis. However, researchers often face domain-specific challenges in mining these repositories, including the integration of heterogeneous data sources, large-scale data collection, normalization and ground-truth generation for feature-oriented tasks. In this paper, we introduce MApp-KG, a combination of software resources and data artefacts in the field of mobile app repositories aimed at supporting feature-oriented knowledge generation tasks. Our contribution provides a framework for automatically constructing a knowledge graph that models a domain-specific catalog of natural language documents related to mobile applications. We distribute MApp-KG through a public triplestore, enabling its immediate use for future research and replication of our findings., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Forecasting software indicators: an industry-academia collaboration

  • Ayala Martínez, Claudia Patricia
  • Gómez Seoane, Cristina
  • Manzano Aguilar, Martí
  • Abherve, Antonin
  • Franch Gutiérrez, Javier
Context: Nowadays software-development organizations are urged to exploit their data for empowering their decision-making processes. Such data may be used to monitor the status of meaningful software indicators (e.g., software quality, productivity and on-time delivery) that are relevant for their decision-making processes. Forecasting the values of such indicators may provide evidence of a potentially high risk or opportunity that could help to anticipate actions accordingly. Most of the existing forecasting proposals in software engineering use open-source data rather than data from industrial projects. Therefore, there is a lack of evidence on how these proposals ft the particular needs of a software-development organization and how they can be automated into the organization’s infrastructure. Objective: To enable software indicators´ forecasting in a software-development organization (Modeliosoft). Method: We designed an industry-academia collaboration based on Action Design Research (ADR) to address Modeliosoft’s forecasting challenges. Results A tool-supported method called FOSI (Forecasting Of Software Indicators) for enabling forecasting in Modeliosoft. We obtained positive results regarding its suitability and technical feasibility in a pilot project of the organization. In addition, we provide details and refections on the potential usefulness of the method for addressing similar feld problems. Conclusions: The procedures and results detailed in this paper are valuable to: 1) address Modeliosoft’s forecasting challenges 2) inspire other software-development organizations on how to deal with similar problems and even reuse some procedures and software support tools resulted from this work, 3) promote the win-win benefts of industry-academia collaborations., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been partially funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RB-I00/AEI/10.13039/ 50 501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Data-driven agile requirements elicitation through the lenses of situational method engineering

  • Franch Gutiérrez, Javier
  • Henriksson, Aron
  • Ralyté, Jolita
  • Zdravkovic, Jelena
Ubiquitous digitalization has led to the continuous generation of large amounts of digital data, both in organizations and in society at large. In the requirements engineering community, there has been a growing interest in considering digital data as new sources for requirements elicitation, in addition to stake-holders. The volume, dynamics, and variety of data makes iterative requirements elicitation increasingly continuous, but also unstructured and complex, which current agile methods are unable to consider and manage in a systematic and efficient manner. There is also the need to support software evolution by enabling a synergy of stakeholder-driven requirements elicitation and management with data-driven approaches. In this study, we propose extension of agile requirements elicitation by applying situational method engineering. The research is grounded on two studies in the business domains of video games and online banking., The work presented in this paper is partially funded by the DOGO4ML Spanish research project, PID2020-117191RB-I00., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Performance analysis of distributed GPU-accelerated task-based workflows

  • Nogueira Lobo de Carvalho, Marcos
  • Queralt Calafat, Anna
  • Romero Moral, Óscar
  • Simitsis, Alkis
  • Tatu, Cristian
  • Badia Sala, Rosa Maria
We present an empirical approach to identify the key factors affecting the execution performance of task-based workflows on a High Performance Computing (HPC) infrastructure composed of heterogeneous CPU-GPU clusters. Our results reveal that the execution performance in distributed GPU-accelerated task-based workflows highly depends on several interrelated factors regarding the task algorithm, dataset, resources, and system employed. In addition, our analysis identifies key correlations among these factors, presents novel observations, and offers guidelines toward designing an automated method to handle task-based workflows in modern, high-compute capacity, CPU-GPU engines., This work has been partially supported by DEDS (H2020-MSCAITN2020) with grant agreement No. 955895, the EU-HORIZON programme CREXDATA under GA.101092749, the EU-HORIZON programme FAIR-CORE4EOSC under GA.101057264, the EUHORIZON programme EXTREMEXP under GA.101093164, the Spanish Government projects PID2019-107255GB and PID2020117191RB-I00/AEI/10.13039/501100011033andMCIN/AEI/10.13039 /501100011033 (CEX2021-001148-S), and by the Departament de Recerca i Universitats de la Generalitat de Catalunya (2021 SGR 00412, MPiEDist)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A data-science pipeline to enable the interpretability of many-objective feature selection

  • Njoku, Uchechukwu Fortune
  • Abelló Gamazo, Alberto
  • Bilalli, Besim
  • Bontempi, Gianluca
Many-Objective Feature Selection (MOFS) approaches use four or more objectives to determine the relevance of a subset of features in a supervised learning task. As a consequence, MOFS typically returns a large set of non-dominated solutions, which have to be assessed by the data scientist in order to proceed with the final choice. Given the multi-variate nature of the assessment, which may include objectives (e.g., fairness) unrelated to predictive accuracy, this step is often not straightforward and suffers from the lack of existing tools. For instance, it is common to make use of a tabular presentation of the solutions, which provides little information about the trade-offs and the relationships between objectives over the set of solutions. Adopting a GA-based MOFS with six objectives (number of selected features, balanced accuracy, F1-Score, variance inflation factor, statistical parity, and equalised odds) for two feature selection tasks, this paper illustrates the complex challenge of assessing MOFS results and the need for a methodology to aid and justify the final choice of a solution., The project leading to this publication has received funding from the European Union’s Horizon 2020 research and innovation programme (G.A. No. 955895). A. Abelló and B. Bilalli are funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RBI00, funding scheme AEI/10.13039/501100011033. G. Bontempi is supported by the Service Public de Wallonie Recherche under G.A. No. 2010235–ARIAC by DigitalWallonia4.ai. Computational resources have been provided by the Consortium des Équipements de Calcul Intensif, funded by the Fonds de la Recherche Scientifique de Belgique (G.A. No. 2.5020.11) and Walloon Region., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Unveiling competition dynamics in mobile app markets through user reviews

  • Motger de la Encarnación, Joaquim
  • Franch Gutiérrez, Javier
  • Gervasi, Vincenzo
  • Marco Gómez, Jordi
[Context and motivation] User reviews published in mobile app repositories are essential for understanding user satisfaction and engagement within a specific market segment. [Question/problem] Manual analysis of reviews is impractical due to the large data volume, and automated analysis faces challenges like data synthesis and reporting. This complicates the task for app providers in identifying patterns and significant events, especially in assessing the influence of competitor apps. Furthermore, review-based research is mostly limited to a single app or a single app provider, excluding potential competition analysis. Consequently, there is an open research challenge in leveraging user reviews to support cross-app analysis within a specific market segment. [Principal ideas/results] Following a case-study research method in the microblogging app market, we introduce an automatic, novel approach to support mobile app market analysis. Our approach leverages quantitative metrics and event detection techniques based on newly published user reviews. Significant events are proactively identified and summarized by comparing metric deviations with historical baseline indicators within the lifecycle of a mobile app. [Contribution] Results from our case study show empirical evidence of the detection of relevant events within the selected market segment, including software- or release-based events, contextual events and the emergence of new competitors., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project/funding scheme PID2020-117191RBI00/AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Chatbots4Mobile: Feature-oriented knowledge base generation using natural language

  • Motger de la Encarnación, Joaquim
  • Franch Gutiérrez, Javier
  • Marco Gómez, Jordi
Chatbots4Mobile is a research project from the GESSI research group (UPC-BarcelonaTech) which aims at designing and developing a task oriented, knowledge based conversational agent to support mobile users in the process of managing and integrating the functionalities exposed by their own application portfolio. To support the design of the required knowledge base, the project focuses on the application of Natural Language Processing (NLP) techniques to infer extended knowledge about the features exposed by a subset of mobile applications, including feature extraction from app-related documents, syntactic and semantic similarity analysis between features, and intent/entity classification focused on functionality identification from user requests. As next steps, we are focusing on the evaluation of embedded linguistic knowledge in large language models, as well as the application of granular sentiment analysis techniques to discern biased documents and process sentiment-based user feedback., With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (published version)