Pasar al contenido principal

ASISTENTES EVOLUTIVOS INTELIGENTES PARA INICIAR LINEAS DE PRODUCTO SOFTWARE

RTI2018-096411-B-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2018
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario FUNDACION UNIVERSIDAD SAN JORGE

Resultados relacionados

Resultados totales (Incluyendo duplicados): 19
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Traceability Link Recovery between Requirements and Models using an Evolutionary Algorithm Guided by a Learning to Rank Algorithm: Train control and management case

  • Marcén, Ana C.
  • Lapeña, Raúl
  • Pastor López, Oscar|||0000-0002-1320-8471
  • Cetina Englada, Carlos
[EN] Traceability Link Recovery (TLR) has been a topic of interest for many years within the software engineering community. In recent years, TLR has been attracting more attention, becoming the subject of both fundamental and applied research. However, there still exists a large gap between the actual needs of industry on one hand and the solutions published through academic research on the other. In this work, we propose a novel approach, named Evolutionary Learning to Rank for Traceability Link Recovery (TLR-ELtoR). TLR-ELtoR recovers traceability links between a requirement and a model through the combination of evolutionary computation and machine learning techniques, generating as a result a ranking of model fragments that can realize the requirement. TLR-ELtoR was evaluated in a real-world case study in the railway domain, comparing its outcomes with five TLR approaches (Information Retrieval, Linguistic Rule-based, Feedforward Neural Network, Recurrent Neural Network, and Learning to Rank). The results show that TLR-ELtoR achieved the best results for most performance indicators, providing a mean precision value of 59.91%, a recall value of 78.95%, a combined F-measure of 62.50%, and a MCC value of 0.64. The statistical analysis of the results assesses the magnitude of the improvement, and the discussion presents why TLR-ELtoR achieves better results than the baselines., This work has been developed with the financial support of the Spanish State Research Agency and the Generalitat Valenciana under the projects DataME TIN2016-80811-P, ALPS RT12018-096411-B-100, ACIF/2018/171 and PROMETEO/2018/176, and co-financed with ERDF.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the Benefits of Model-Driven Development

  • Domingo, África
  • Echeverría, Jorge
  • Pastor López, Oscar|||0000-0002-1320-8471
  • Cetina Englada, Carlos
[EN] Researchers have been evaluating the benefits of Model-Driven Development (MDD) for more than a decade now. Although some works suggest that MDD decreases development time, other works limit MDD benefits to academic exercises and to developers without experience. To clarify the benefits of MDD, we present the results of our experiment, which compares MDD and Code-centric Development (CcD) in terms of correctness, efficiency, and satisfaction. Our experiment achieves fidelity to real-world settings because the tasks are taken from real-world video game development, and the subjects use domain frameworks as they are used in real-world developments. Our results challenge previous ideas that limit the benefits of MDD to academic exercises and to developers without experience. Furthermore, our results also suggest that understanding the benefits of MDD might require researchers to rethink their experiments to include the social part of software development., Partially supported by MINECO under the Project ALPS (RTI2018-096411-B-I00)
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Utilizing Automatic Query Reformulations as Genetic Operations to Improve Feature Location in Software Models

  • Pérez, Francisca
  • Cetina Englada, Carlos
  • Ziadi, Tewfik
[EN] In the combination of Model-Driven Engineering (MDE) and Search-Based Software Engineering (SBSE), genetic operations are one of the key ingredients. Our work proposes a novel adaptation of automatic query reformulations as genetic operations that leverage the latent semantics of software models (the cornerstone artefact of MDE). We analyze the impact of these reformulation operations in a real-world industrial case study of feature location in models. As baselines, we use: 1) the widespread single-point crossover plus random mutation; and 2) mask crossover plus random mutation, which is the best performer for feature location in models. We also perform a statistical analysis to provide quantitative evidence of the impact of the results and to show that this impact is significant. Our reformulation operations improve the results of the best baseline by 37.73 percent in recall and 14.08 percent in precision. These results are relevant for the task of feature location in models (one of the main activities performed during software maintenance and evolution). Furthermore, given that the only requirement to apply our approach is term availability in models, our work opens a new research direction to improve more tasks in MDE such as bug location or requirements traceability., The work of Francisca Perez and Carlos Cetina was mainly done during their stay at the LIP6 of Sorbonne University. This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00). We also thank the ITEA3 15010 REVaMP2 Project.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Topic modeling for feature location in software models: Studying both code generation and interpreted models

  • Pérez, Francisca
  • Cetina Englada, Carlos
  • Lapeña, Raúl
  • Marcén, Ana C.
[EN] Context: In the last 20 years, the research community has increased its attention to the use of topic modeling for software maintenance and evolution tasks in code. Topic modeling is a popular and promising information retrieval technique that represents topics by word probabilities. Latent Dirichlet Allocation (LDA) is one of the most popular topic modeling methods. However, the use of topic modeling in model-driven software development has been largely neglected. Since software models have less noise (implementation details) than software code, software models might be well-suited for topic modeling. Objective: This paper presents our LDA-guided evolutionary approach for feature location in software models. Specifically, we consider two types of software models: models for code generation and interpreted model. Method: We evaluate our approach considering two real-world industrial case studies: code-generation models for train control software, and interpreted models for a commercial video game. To study the impact on the results, we compare our approach for feature location in models against random search and a baseline based on Latent Semantic Indexing, which is a popular information retrieval technique. In addition, we perform a statistical analysis of the results to show that this impact is significant. We also discuss the results in terms of the following aspects: data sparsity, implementation complexity, calibration, and stability. Results: Our approach significantly outperforms the baseline in terms of recall, precision and F-measure when it comes to interpreted models. This is not the case for code-generation models. Conclusions: Our analysis of the results uncovers a recommendation towards results improvement. We also show that calibration approaches can be transferred from code to models. The findings of our work with regards to the compensation of instability have the potential to help not only feature location in models, but also in code., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) , Spain through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

On the influence of model fragment properties on a machine learning-based approach for feature location

  • Ballarin, Manuel
  • Marcén, Ana C.
  • Pelechano Ferragud, Vicente|||0000-0003-1090-230X
  • Cetina Englada, Carlos
[EN] Context: Leveraging machine learning techniques to address feature location on models has been gaining attention. Machine learning techniques empower software product companies to take advantage of the knowledge and the experience to improve the performance of the feature location process. Most of the machine learning-based works for feature location on models report the machine learning techniques and the tuning parameters in detail. However, these works focus on the size and the distribution of the data sets, neglecting the properties of their contents. Objective: In this paper, we analyze the influence of three model fragment properties (density, multiplicity, and dispersion) on a machine learning-based approach for feature location. Method: The analysis of these properties is based on an industrial case provided by CAF, a worldwide provider of railway solutions. The test cases were evaluated through a machine learning technique that uses different subsets of a knowledge base to learn how to locate unknown features. Results: Results show that the density and dispersion properties have a direct impact on the results. In our case study, the model fragments with extra-small density values achieve results with up to 43% more precision, 41% more recall, 42% more F-measure, and 0.53 more Matthews Correlation Coefficient (MCC) than the model fragments with other density values. On the other hand, the model fragments with extra-small and small dispersion values achieve results with up to 53% more precision, 52% more recall, 52% more F-measure, and 0.57 more MCC than the model fragments with other dispersion values. Conclusions: The analysis of the results shows that both density and dispersion properties significantly influence the results. These results can serve not only to improve the reports by means of the model fragment properties, but also to be able to compare machine learning-based feature location approaches fairly improving the feature location results., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO), Spain through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018096411-B-I00). We also thank the ITEA3 15010 REVaMP2 Project and ACIF/2018/171.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Leveraging execution traces to enhance traceability links recovery in BPMN models

  • Lapeña, Raúl
  • Pérez, Francisca
  • Pastor López, Oscar|||0000-0002-1320-8471
  • Cetina Englada, Carlos
[EN] Context: Traceability Links Recovery has been a topic of interest for many years, resulting in techniques that perform traceability based on the linguistic clues of the software artifacts under study. However, BPMN models tend to present an overall lack of linguistic clues when compared to code-based artifacts or code generation models. Hence, TLR becomes a harder task when performed among requirements and BPMN models.Objective: This paper proposes a novel approach, called METRA, that leverages the execution traces of BPMN to expand the BPMN models. The expansion of the BPMN models enhances their linguistic clues, bridging the language between BPMN models and other software artifacts, and improving the TLR process between requirements and BPMN models.Methods: The proposed approach is evaluated through a real-world industrial case study, comparing its outcomes against two state-of-the-art baselines, TLR and LORE. The paper also evaluates the combination of METRA with LORE against the rest of the approaches, including standalone METRA. The evaluation process generates a report of measurements (precision, recall, f-measure, and MCC), over which a statistical analysis is conducted.Results: Results show that approaches based on METRA maintain the excellent precision results obtained by baseline approaches (74.2% for METRA, 78.8% for METRA+LORE), whilst also improving the recall results from the unacceptable values obtained by the baselines to good values (72.4% for METRA, 73.9% for METRA+LORE). Moreover, according to the statistical analysis, the differences in the results obtained by the evaluated approaches are statistically significant. Conclusions: This paper opens a novel field of work in TLR by analyzing the improvement of the TLR process through the inclusion of linguistic clues present in execution traces, and discusses ideas for further research that can delve into this promising direction explored by our work., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the project ALPS (RTI2018-096411-B-I00) and in part by the Gobierno de Aragon (Spain) (Research Group S05_20D) .
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

On the influence of architectural languages on requirements traceability

  • Ballarin-Naya, Manuel
  • Arcega, Lorena
  • Pelechano Ferragud, Vicente|||0000-0003-1090-230X
  • Cetina Englada, Carlos
[EN] Today, a considerable number of Architectural Languages (ALs) have been proposed for specifying and analyzing the architecture of software systems. Despite the popularity of different ALs, how ALs influence software system maintainability has not received much attention. One of the most important tasks in software maintenance is requirements traceability. Requirements traceability establishes links between requirements and other software artifacts, facilitating system maintenance. In this paper, we analyze the influence of ALs on requirements traceability. Taking into account the ALs used by the industry, we analyze how ALs influence traceability among requirements and architecture models. We conducted an evaluation with our industrial partner CAF. The results show significant differences in AL performance. We also analyze the results in terms of AL concepts, requirements model elements, and AL type in order to understand the performance differences. General-Purpose/Research Languages achieve the best results for all of the performance indicators, providing a mean precision value of 0.51, a recall value of 0.38, a combined F-measure of 0.40, and an Matthews Correlation Coefficient value of 0.33. Those ALs that influence engineers to use more generic and domain-independent terms to specify their architectures obtain the best results during requirements traceability. Our results have the potential to help AL designers to improve their languages and also to help practitioners make a more informed decision about whether or not a given AL meets their traceability needs., Secretaria de Estado de Investigacion, Desarrollo e Innovacion, Grant/Award Number: RTI2018-096411-B-I00
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Empowering the Human as the Fitness Function in Search-Based Model-Driven Engineering

  • Pérez, Francisca
  • Cetina Englada, Carlos
  • Font, Jaime
  • Arcega, Lorena
[EN] In Search-Based Software Engineering, more than 100 works have involved the human in the search process to obtain better results. However, the case where the human completely replaces the fitness function remains neglected. There is a good reason for that; no matter how intelligent the human is, humans cannot assess millions of candidate solutions as heuristics do. In this work, we study the influence of using the Human as the Fitness Function (HaFF) on the quality of the results. To do that, we focus on Search-Based Model-Driven Engineering (SBMDE) because inspecting models should require less human effort than inspecting code thanks to the abstraction of models. Therefore, we analyze the impact of HaFF in a real-world industrial case study of feature location in models. Furthermore, we also consider a reformulation operation (replacement) in the evaluation because a recent work reported that this operation significantly reduces the number of iterations required in comparison to the widespread crossover and mutation operations. The combination of HaFF and the reformulation operation (HaFF_R) improves the results of the best baseline by 0.15% in recall and 14.26% in precision. Analyzing the results, we learned how to better leverage HaFF_R, which increased the improvement with regard to the best baseline to 1.15% in recall and 20.05% in precision. HaFF_R significantly improves precision because humans are immune to the main limitations of the baselines: vocabulary mismatch and tacit knowledge. A focus group confirmed the acceptance of HaFF. These results are relevant for SBMDE because feature location is one of the main activities performed during maintenance and evolution. Our results, and what we learned from them, can also motivate and help other researchers to explore the benefits of HaFF. In fact, we provide a guideline that further discusses how to apply HaFF to other software engineering problems., This work was supported in part by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National RthornDthorni Plan and ERDF funds under the Project ALPS under Grant RTI2018-096411-B-I00, and in part by the Gobierno de Aragon (Spain) (Research Group S05_20D).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

A fine-grained requirement traceability evolutionary algorithm: Kromaia, a commercial video game case study

  • Blasco, Daniel
  • Cetina Englada, Carlos
  • Pastor López, Oscar|||0000-0002-1320-8471
[EN] Context:Commercial video games usually feature an extensive source code and requirements that are related to code lines from multiple methods. Traceability is vital in terms of maintenance and content update, so it is necessary to explore such search spaces properly. Objective:This work presents and evaluates CODFREL (Code Fragment-based Requirement Location), our approach to fine-grained requirement traceability, which lies in an evolutionary algorithm and includes encoding and genetic operators to manipulate code fragments that are built from source code lines. We compare it with a baseline approach (Regular-LSI) by configuring both approaches with different granularities (code lines / complete methods). Method:We evaluated our approach and Regular-LSI in the Kromaia video game case study, which is a commercial video game released on PC and PlayStation 4. The approaches are configured with method and code line granularity and work on 20 requirements that are provided by the development company. Our approach and Regular-LSI calculate similarities between requirements and code fragments or methods to propose possible solutions and, in the case of CODFREL, to guide the evolutionary algorithm. Results:The results, which compare code line and method granularity configurations of CODFREL with different granularity configurations of Regular-LSI, show that our approach outperforms Regular-LSI in precision and recall, with values that are 26 and 8 times better, respectively, even though it does not achieve the optimal solutions. We make an open-source implementation of CODFREL available. Conclusions:Since our approach takes into consideration key issues like the source code size in commercial video games and the requirement dispersion, it provides better starting points than Regular-LSI in the search for solution candidates for the requirements. However, the results and the influence of domain-specific language on them show that more explicit knowledge is required to improve such results., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R + D + i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Tesis doctoral (DoctoralThesis)

Towards Search-based Game Software Engineering

  • Blasco Latorre, Daniel
[ES] Los videojuegos son proyectos multidisciplinares que implican, en buena medida, el desarrollo de software. Esta tesis trata la faceta del desarrollo de videojuegos relativa al software mediante la Ingeniería del Software basada en Búsqueda (SBSE, Search-based Software Engineering). El objetivo específico de este trabajo es valerse de las características de los videojuegos en pro de una Ingeniería del Software de Videojuegos basada en Búsqueda (SBGSE, Search-based Game Software Engineering), incluyendo el uso de simulaciones de videojuegos para guiar búsquedas, codificación de granularidad fina y operaciones genéticas de mejora. Las aproximaciones propuestas superan a las de referencia en mantenimiento (trazabilidad de requisitos) y creación de contenido (generación de NPCs). El mantenimiento y la creación de contenido son, a menudo, tareas esenciales para garantizar la retención de usuarios por medio de actualizaciones o expansiones. Además, esta investigación aborda la necesidad de estudios de caso industriales. Esta tesis presenta un compendio que incluye tres artículos realizados durante el proceso de investigación y publicados en revistas académicas, con resultados que muestran que las aproximaciones de la Ingeniería del Software de Videojuegos basada en Búsqueda (SBGSE, Search-based Game Software Engineering) pueden mejorar la calidad de las soluciones generadas, así como reducir el tiempo necesario para producirlas., [CA] Els videojocs són projectes multidisciplinaris que impliquen, en bona part, el desenvolupament de software. Aquesta tesi tracta la faceta del desenvolupament de videojocs relativa al software mitjançant l'Enginyeria del Software basada en Cerca (SBSE, Search-based Software Engineering). L'objectiu específic d'aquest treball és valdre's de les característiques dels videojocs en pro d'una Enginyeria del Software de Videojocs basada en Cerca (SBGSE, Search-based Game Software Engineering), incloent-hi l'ús de simulacions de videojocs per a guiar cerques, codificació de granularitat fina i operacions genètiques de millora. Les aproximacions proposades superen a les de referència en manteniment (traçabilitat de requisits) i creació de contingut (generació de NPCs). El manteniment i la creació de contingut són, sovint, tasques essencials per a garantir la retenció d'usuaris per mitjà d'actualitzacions o expansions. A més, aquesta investigació aborda la necessitat d'estudis de cas industrials. Aquesta tesi presenta un compendi que inclou tres articles realitzats durant el procés d'investigació i publicats en revistes acadèmiques, amb resultats que mostren que les aproximacions de l'Enginyeria del Software de Videojocs basada en Cerca (SBGSE, Search-based Game Software Engineering) poden millorar la qualitat de les solucions generades, així com reduir el temps necessari per a produir-les., [EN] Video games are multidisciplinary projects which involve software development to a significant extent. This thesis tackles the software aspect of video game development through Search-based Engineering. Specifically, the objective of this work is to leverage the characteristics of video games towards Search-based Game Software Engineering, including the use of video game simulations to guide the search, a fine-grained encoding, and improvement genetic operations. The approaches proposed outperform the baselines in maintenance (requirement traceability) and content creation (NPC generation) tasks. Maintenance and content creation are often essential tasks to ensure player retention by means of updates or expansions. In addition, this research addresses the need for industrial case studies. This thesis presents a compendium that includes three papers produced through the research and published in academic journals, with results that show that Search-based Game Software Engineering approaches can provide improved solutions, in terms of quality and time cost., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Artículo científico (JournalArticle)

A Fine-Grained Requirement Traceability Evolutionary Algorithm: Kromaia, a Commercial Video Game Case Study

  • Blasco Latorre, Daniel
  • Cetina, Carlos
  • Pastor, Óscar
Context: Commercial video games usually feature an extensive source code and requirements that are related to code lines from multiple methods. Traceability is vital in terms of maintenance and content update, so it is necessary to explore such search spaces properly. Objective: This work presents and evaluates CODFREL (Code Fragment-based Requirement Location), our approach to ne-grained requirement traceability, which lies in an evolutionary algorithm and includes encoding and genetic operators to manipulate code fragments that are built from source code lines. We compare it with a baseline approach (Regular-LSI) by con guring both approaches with di erent granularities (code lines / complete methods). Method: We evaluated our approach and Regular-LSI in the Kromaia video game case study, which is a commercial video game released on PC and PlayStation 4. The approaches are con gured with method and code line granularity and work on 20 requirements that are provided by the development company. Our approach and Regular-LSI calculate similarities between requirements and code fragments or methods to propose possible solutions and, in the case of CODFREL, to guide the evolutionary algorithm. Results: The results, which compare code line and method granularity con gurations of CODFREL with di erent granularity con gurations of Regular-LSI, show that our approach outperforms Regular-LSI in precision and recall, with values that are 26 and 8 times better, respectively, even though it does not achieve the optimal solutions. We make an open-source implementation of CODFREL available. Conclusions: Since our approach takes into consideration key issues like the source code size in commercial video games and the requirement dispersion, it provides better starting points than Regular-LSI in the search for solution candidates for the requirements. However, the results and the infuence of domain-speci c language on them show that more explicit knowledge is required to improve such results., This work has been partially supported by theMinistry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Artículo científico (JournalArticle)

Evaluating the Influence of Scope on Feature Location

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
Context: Feature Location (FL) is a widespread technique that is used to maintain and evolve a software product. FL is also helpful in reengineering a family of software products into a Software Product Line (SPL). Despite the popularity of FL, there is no study that evaluates the influence of scope (single product or product family) when engineers perform FL. Objective: The goal of this paper is to compare the performance, productivity, and perceived difficulty of manual FL when scope changes from a single product to a product family. Method: We conducted a crossover experiment to compare the performance, productivity, and perceived difficulty of manual FL when scope changes. The experimental objects are extracted from a real-world SPL that uses a Domain-Specific Language to generate the firmware of its products. Results: Performance and productivity decrease significantly when engineers locate features in a product family regardless of their experience. For these variables the impact of the FL Scope is medium–large. On contrast, for perceived difficulty, the magnitude of the difference is moderate and is not significant. Conclusions: While performance and productivity decrease significantly when engineers locate features in a product family, the difficulty they perceive does not predict the significant worsening of the results. Our work also identifies strengths and weaknesses in FL. This can help in developing better FL approaches and test cases for evaluation., Partially supported by MINECO, Spain under the Project ALPS (RTI2018-096411-B-I00)
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Artículo científico (JournalArticle)

Evaluating the Influence of Scope on Feature Location

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
Context: Feature Location (FL) is a widespread technique that is used to maintain and evolve a software product. FL is also helpful in reengineering a family of software products into a Software Product Line (SPL). Despite the popularity of FL, there is no study that evaluates the influence of scope (single product or product family) when engineers perform FL. Objective: The goal of this paper is to compare the performance, productivity, and perceived difficulty of manual FL when scope changes from a single product to a product family. Method: We conducted a crossover experiment to compare the performance, productivity, and perceived difficulty of manual FL when scope changes. The experimental objects are extracted from a real-world SPL that uses a Domain-Specific Language to generate the firmware of its products. Results: Performance and productivity decrease significantly when engineers locate features in a product family regardless of their experience. For these variables the impact of the FL Scope is medium–large. On contrast, for perceived difficulty, the magnitude of the difference is moderate and is not significant. Conclusions: While performance and productivity decrease significantly when engineers locate features in a product family, the difficulty they perceive does not predict the significant worsening of the results. Our work also identifies strengths and weaknesses in FL. This can help in developing better FL approaches and test cases for evaluation., Partially supported by MINECO, Spain under the Project ALPS (RTI2018-096411-B-I00)
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the Benefits of Model-Driven Development

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
Researchers have been evaluating the benefits of Model-Driven Development (MDD) for more than a decade now. Although some works suggest that MDD decreases development time, other works limit MDD benefits to academic exercises and to developers without experience. To clarify the benefits of MDD, we present the results of our experiment, which compares MDD and Code-centric Development (CcD) in terms of correctness, efficiency, and satisfaction. Our experiment achieves fidelity to real-world settings because the tasks are taken from real-world video game development, and the subjects use domain frameworks as they are used in real-world developments. Our results challenge previous ideas that limit the benefits of MDD to academic exercises and to developers without experience. Furthermore, our results also suggest that understanding the benefits of MDD might require researchers to rethink their experiments to include the social part of software development., Partially supported by MINECO under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Artículo científico (JournalArticle)

Empowering the human as the fitness function in search-based model-driven engineering

  • Pérez Pérez, Francisca
  • Font, Jaime
  • Arcega, Lorena
  • Cetina, Carlos
In Search-Based Software Engineering, more than 100 works have involved the human in the search process to obtain better results. However, the case where the human completely replaces the fitness function remains neglected. There is a good reason for that; no matter how intelligent the human is, humans cannot assess millions of candidate solutions as heuristics do. In this work, we study the influence of using the Human as the Fitness Function (HaFF) on the quality of the results. To do that, we focus on Search-Based Model-Driven Engineering (SBMDE) because inspecting models should require less human effort than inspecting code thanks to the abstraction of models. Therefore, we analyze the impact of HaFF in a real-world industrial case study of feature location in models. Furthermore, we also consider a reformulation operation (replacement) in the evaluation because a recent work reported that this operation significantly reduces the number of iterations required in comparison to the widespread crossover and mutation operations. The combination of HaFF and the reformulation operation (HaFF R) improves the results of the best baseline by 0.15% in recall and 14.26% in precision. Analyzing the results, we learned how to better leverage HaFF R, which increased the improvement with regard to the best baseline to 1.15% in recall and 20.05% in precision. HaFF R significantly improves precision because humans are immune to the main limitations of the baselines: vocabulary mismatch and tacit knowledge. A focus group confirmed the acceptance of HaFF. These results are relevant for SBMDE because feature location is one of the main activities performed during maintenance and evolution. Our results, and what we learned from them, can also motivate and help other researchers to explore the benefits of HaFF. In fact, we provide a guideline that further discusses how to apply HaFF to other software engineering problems., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the Benefits of Model-Driven Development

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
Researchers have been evaluating the benefits of Model-Driven Development (MDD) for more than a decade now. Although some works suggest that MDD decreases development time, other works limit MDD benefits to academic exercises and to developers without experience. To clarify the benefits of MDD, we present the results of our experiment, which compares MDD and Code-centric Development (CcD) in terms of correctness, efficiency, and satisfaction. Our experiment achieves fidelity to real-world settings because the tasks are taken from real-world video game development, and the subjects use domain frameworks as they are used in real-world developments. Our results challenge previous ideas that limit the benefits of MDD to academic exercises and to developers without experience. Furthermore, our results also suggest that understanding the benefits of MDD might require researchers to rethink their experiments to include the social part of software development., Partially supported by MINECO under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Artículo científico (JournalArticle)

Empowering the human as the fitness function in search-based model-driven engineering

  • Pérez Pérez, Francisca
  • Font, Jaime
  • Arcega, Lorena
  • Cetina, Carlos
In Search-Based Software Engineering, more than 100 works have involved the human in the search process to obtain better results. However, the case where the human completely replaces the fitness function remains neglected. There is a good reason for that; no matter how intelligent the human is, humans cannot assess millions of candidate solutions as heuristics do. In this work, we study the influence of using the Human as the Fitness Function (HaFF) on the quality of the results. To do that, we focus on Search-Based Model-Driven Engineering (SBMDE) because inspecting models should require less human effort than inspecting code thanks to the abstraction of models. Therefore, we analyze the impact of HaFF in a real-world industrial case study of feature location in models. Furthermore, we also consider a reformulation operation (replacement) in the evaluation because a recent work reported that this operation significantly reduces the number of iterations required in comparison to the widespread crossover and mutation operations. The combination of HaFF and the reformulation operation (HaFF R) improves the results of the best baseline by 0.15% in recall and 14.26% in precision. Analyzing the results, we learned how to better leverage HaFF R, which increased the improvement with regard to the best baseline to 1.15% in recall and 20.05% in precision. HaFF R significantly improves precision because humans are immune to the main limitations of the baselines: vocabulary mismatch and tacit knowledge. A focus group confirmed the acceptance of HaFF. These results are relevant for SBMDE because feature location is one of the main activities performed during maintenance and evolution. Our results, and what we learned from them, can also motivate and help other researchers to explore the benefits of HaFF. In fact, we provide a guideline that further discusses how to apply HaFF to other software engineering problems., This work has been partially supported by the Ministry of Economy and Competitiveness (MINECO) through the Spanish National R+D+i Plan and ERDF funds under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Contribución de congreso (ConferenceOutput)

Comparing UML-based and DSL-based Modeling from Subjective and Objective Perspectives

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
In the last two decades, researchers have conducted several empirical evaluations, involving thousands of subjects, to understand the use of models in software development. The results of these evaluations show that most of the subjects make informal use of the models, which is known as ‘modeling as sketch’. In this paper, we present an experiment that compares UML-based and DSL-based modeling when subjects model a part of a commercial video game. In the comparison, we have used objective and subjective measures, in contrast to other works that focus either on objective measures to evaluate modeling performance or on subjective measures to analyze modeling styles. Our results reveal that subjects underestimate the potential of their own models. Our finding is relevant for the design of future evaluations and for the teaching and adoption of modeling. If users correctly assess their models, they might leverage their potential as programs., Partially supported by MINECO under the Project ALPS (RTI2018-096411-B-I00).
R-USJ. Repositorio Institucional de la Universidad San Jorge
Publicación Contribución de congreso (ConferenceOutput)

Comparing UML-based and DSL-based Modeling from Subjective and Objective Perspectives

  • Domingo Montes, África
  • Echeverría Ochoa, Jorge
  • Pastor, Óscar
  • Cetina, Carlos
In the last two decades, researchers have conducted several empirical evaluations, involving thousands of subjects, to understand the use of models in software development. The results of these evaluations show that most of the subjects make informal use of the models, which is known as ‘modeling as sketch’. In this paper, we present an experiment that compares UML-based and DSL-based modeling when subjects model a part of a commercial video game. In the comparison, we have used objective and subjective measures, in contrast to other works that focus either on objective measures to evaluate modeling performance or on subjective measures to analyze modeling styles. Our results reveal that subjects underestimate the potential of their own models. Our finding is relevant for the design of future evaluations and for the teaching and adoption of modeling. If users correctly assess their models, they might leverage their potential as programs., Partially supported by MINECO under the Project ALPS (RTI2018-096411-B-I00).