Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Multi-agent debate strategies to enhance requirements engineering with large language models

  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Marco Gómez, Jordi|||0000-0002-0078-7929
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
Context: Large Language Model (LLM) agents are becoming widely used for various Requirements Engineering (RE) tasks. Research on improving their accuracy mainly focuses on prompt engineering, model fine-tuning, and retrieval augmented generation. However, these methods often treat models as isolated black boxes - relying on single-pass outputs without iterative refinement or collaboration, limiting robustness and adaptability. Objective: We propose that, just as human debates enhance accuracy and reduce bias in RE tasks by incorporating diverse perspectives, different LLM agents debating and collaborating may achieve similar improvements. Our goal is to investigate whether Multi-Agent Debate (MAD) strategies can enhance RE performance. Method: We conducted a systematic study of existing MAD strategies across various domains to identify their key characteristics. To assess their applicability in RE, we implemented and tested a preliminary MAD-based framework for RE classification. Results: Our study identified and categorized several MAD strategies, leading to a taxonomy outlining their core attributes. Our preliminary evaluation demonstrated the feasibility of applying MAD to RE classification. Conclusions: MAD presents a promising approach for improving LLM accuracy in RE tasks. This study provides a foundational understanding of MAD strategies, offering insights for future research and refinements in RE applications., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745; and by the Spanish Ministerio de Ciencia e Innovación under project funding scheme PID2020-117191RB-I00/ AEI/10.13039/501100011033., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Multivocal literature reviews in emergent research areas: An experience report

  • López Cuesta, Lidia|||0000-0002-6901-9223
  • Farré Tost, Carles|||0000-0001-5814-3782
  • Akbarilalaei, Razieh|||0009-0004-6865-3347
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
In this experience report, we reflect on the process of conductinga Multivocal Literature Review (MLR) on the use of GenerativeArtificial Intelligence agents or multi-agent systems in the contextof project management within Agile software development. We ex-amine the challenge of balancing timeliness with evidence maturitywhen analyzing a rapidly evolving field characterized by preprintsand grey literature. We outline our research protocol, key designchoices, challenges faced during the initial phases of the review(search, study selection, and quality assessment), and strategiesfor maintaining rigor alongside agility. Our main contribution isa set of practical lessons derived from this early stage of the pro-cess, focusing on ensuring construct stability, handling preprints,dual-track quality assurance, and adapting the MLR process to sup-port an iterative approach. These insights aim to assist researchersplanning secondary studies in similarly dynamic research areas., This work has been supported by funding from the HIVEMIND project. Horizon Europe call HORIZON-CL4-2024 DIGITAL-EMERGING-01 under Grant Agreement Number 101189745., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

What about emotions? Guiding fine-grained emotion extraction from mobile app reviews

  • Motger de la Encarnación, Joaquim|||0000-0002-4896-7515
  • Oriol Hilari, Marc|||0000-0003-1928-7024
  • Tiessler Aguirre, Max
  • Franch Gutiérrez, Javier|||0000-0001-9733-8830
  • Marco Gómez, Jordi|||0000-0002-0078-7929
Opinion mining plays a vital role in analysing user feedback and extracting insights from textual data. While most research focuses on sentiment polarity (e.g., positive, negative, neutral), fine-grained emotion classification in app reviews remains underexplored. Fine-grained emotion classification is thus needed to better understand users’ affective responses and support downstream tasks such as feature-emotion analysis, user-oriented release planning, and issue triaging. This paper addresses this gap by identifying and addressing the challenges and limitations in fine-grained emotion analysis in the context of app reviews. Our study adapts Plutchik’s emotion taxonomy to app reviews by developing a structured annotation framework and dataset. Through an iterative human annotation process, we define clear annotation guidelines and document key challenges in emotion classification. Additionally, we evaluate the feasibility of automating emotion annotation using large language models, assessing their cost-effectiveness and agreement with human-labelled data. Our findings reveal that while large language models significantly reduce manual effort and maintain substantial agreement with human annotators, full automation remains challenging due to the complexity of emotional interpretation. This work contributes to opinion mining in requirements engineering by providing structured guidelines, an annotated dataset, and insights for developing automated pipelines to capture the complexity of emotions in app reviews., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745. This paper has been funded by the Spanish Ministerio de Ciencia e Innovacióin under project/funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Multi-agent debate strategies to enhance requirements engineering with large language models

  • Oriol Hilari, Marc
  • Motger de la Encarnación, Joaquim
  • Marco Gómez, Jordi
  • Franch Gutiérrez, Javier
Context: Large Language Model (LLM) agents are becoming widely used for various Requirements Engineering (RE) tasks. Research on improving their accuracy mainly focuses on prompt engineering, model fine-tuning, and retrieval augmented generation. However, these methods often treat models as isolated black boxes - relying on single-pass outputs without iterative refinement or collaboration, limiting robustness and adaptability. Objective: We propose that, just as human debates enhance accuracy and reduce bias in RE tasks by incorporating diverse perspectives, different LLM agents debating and collaborating may achieve similar improvements. Our goal is to investigate whether Multi-Agent Debate (MAD) strategies can enhance RE performance. Method: We conducted a systematic study of existing MAD strategies across various domains to identify their key characteristics. To assess their applicability in RE, we implemented and tested a preliminary MAD-based framework for RE classification. Results: Our study identified and categorized several MAD strategies, leading to a taxonomy outlining their core attributes. Our preliminary evaluation demonstrated the feasibility of applying MAD to RE classification. Conclusions: MAD presents a promising approach for improving LLM accuracy in RE tasks. This study provides a foundational understanding of MAD strategies, offering insights for future research and refinements in RE applications., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745; and by the Spanish Ministerio de Ciencia e Innovación under project funding scheme PID2020-117191RB-I00/ AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

What about emotions? Guiding fine-grained emotion extraction from mobile app reviews

  • Motger de la Encarnación, Joaquim
  • Oriol Hilari, Marc
  • Tiessler Aguirre, Max
  • Franch Gutiérrez, Javier
  • Marco Gómez, Jordi
Opinion mining plays a vital role in analysing user feedback and extracting insights from textual data. While most research focuses on sentiment polarity (e.g., positive, negative, neutral), fine-grained emotion classification in app reviews remains underexplored. Fine-grained emotion classification is thus needed to better understand users’ affective responses and support downstream tasks such as feature-emotion analysis, user-oriented release planning, and issue triaging. This paper addresses this gap by identifying and addressing the challenges and limitations in fine-grained emotion analysis in the context of app reviews. Our study adapts Plutchik’s emotion taxonomy to app reviews by developing a structured annotation framework and dataset. Through an iterative human annotation process, we define clear annotation guidelines and document key challenges in emotion classification. Additionally, we evaluate the feasibility of automating emotion annotation using large language models, assessing their cost-effectiveness and agreement with human-labelled data. Our findings reveal that while large language models significantly reduce manual effort and maintain substantial agreement with human annotators, full automation remains challenging due to the complexity of emotional interpretation. This work contributes to opinion mining in requirements engineering by providing structured guidelines, an annotated dataset, and insights for developing automated pipelines to capture the complexity of emotions in app reviews., This work has been supported by funding from the HIVEMIND project – Horizon Europe call HORIZON-CL4-2024- DIGITAL-EMERGING-01 under Grant Agreement Number 101189745. This paper has been funded by the Spanish Ministerio de Ciencia e Innovacióin under project/funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033., Peer Reviewed, Postprint (author's final draft)