Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

30 Years of Automated GUI Testing: A Bibliometric Analysis

  • Rodríguez-Valdés, Olivia
  • Aho, Pekka
  • Vos, Tanja Ernestina|||0000-0002-6003-9113
  • Marín, Beatriz|||0000-0001-8025-0023
[EN] Context: Over the last 30 years, GUIs have changed considerably, becoming everyday part of our lives through smart phones and other devices. More complex GUIs and multitude of platforms have increased the challenges when testing software through the GUI. Objective: To visualise how the field of automated GUI testing has evolved by studying the growth of the field; types of publications; influential events, papers and authors; collaboration among authors; and trends on GUI testing. Method: To conduct a bibliometric analysis of automated GUI testing by performing a systematic search of primary studies in Scopus from 1990 to 2020. Results: 744 publications were selected as primary studies. The majority of them were conference papers, the most cited paper was published on 2013, and the most published author has 53 papers. Conclusions: Automated GUI testing has continuously grown. Keywords show that testing applied to mobile interfaces will be the trend in next years, along with the integration of Artificial Intelligence and automated exploration techniques., We thank Fernando Pastor for his valuable contribution. This research has been funded by DECODER (decoder-project.eu), iv4XR (iv4xrproject.eu), and IVVES (ivves.weebly.com) projects.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

DECODER - DEveloper COmpanion for Documented and annotatEd code Reference

  • Torres Bosch, Maria Victoria|||0000-0002-2039-2174
  • Pelechano Ferragud, Vicente|||0000-0003-1090-230X
  • Gil Pascual, Miriam
Software is everywhere and the productivity of Software Engineers has increased radically with the advent of new specifications, design and programming paradigms and languages. The main objective of the DECODER project is to introduce radical solutions to increase productivity by increasing the abstraction level, at specification stage, using requirements engineering techniques to integrate more complete specifications into the development process, and formal methods to reduce the time and efforts for integration testing. DECODER project will develop a methodology and tools to improve the productivity of the software development process for medium-criticality applications in the domains of IoT, Cloud Computing, and Operating Systems by combining Natural Language Processing techniques, modelling techniques and Formal Methods. A radical improvement is expected from the management and transformation of informal data into material (herein called knowledge ) that can be assimilated by any party involved in a development process. The project expects an average benefit of 20% in terms of efforts on several use cases belonging to the beforehand mentioned domains and will provide recommendations on how to generalize the approach to other medium-critical domains., This work has been developed with the financial support of the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 824231 and the Spanish State Research Agency under the project TIN2017-84094-R and co-financed with ERDF.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Software Knowledge Representation to Understand Software Systems

  • Torres Bosch, Maria Victoria|||0000-0002-2039-2174
  • Pelechano Ferragud, Vicente|||0000-0003-1090-230X
  • Gil Pascual, Miriam
A software development process involves numerous persons, including customers, domain experts, software engineers, managers, evaluators and certifiers. Together, they produce some software that satisfies its requirements and its quality criteria at a certain point in time. This software contains faults and flaws of different levels of severity and at different phases of its production (specification, design, etc.) so maintenance is needed in order to correct it. Perfective and adaptive maintenance is also needed to cope with changes in the environment or with new requirements, e.g. new functionalities. In this work, we introduce the Persistent Knowledge Monitor (PKM), which is being developed within the DECODER H2020 project for handling (i.e. storing, retrieving, merging and checking for consistency) all kinds of knowledge and information related to a software project. The PKM will be part of a platform capable of taking advantage of all the artefacts available in a software ecosystem, not only the source code, but also its version control system, abstract specifications, informal documents or reports, etc. for representing the software knowledge and improving the workflow of software developers., This work has been developed with the financial support of the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 824231 and the Spanish State Research Agency under the project TIN2017-84094-R and co-financed with ERDF.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Deploying TESTAR to Enable Remote Testing in an Industrial CI Pipeline: A Case-Based Evaluation

  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
  • Vos, Tanja Ernestina|||0000-0002-6003-9113
  • Aho, Pekka
  • Torres Boigues, Ismael
  • Calás Blasco, Ernesto
  • Martínez Martínez, Hector
[EN] Companies are facing constant pressure towards shorter release cycles while still maintaining a high level of quality. Agile development, continuous integration and testing are commonly used quality assurance techniques applied in industry. Increasing the level of test automation is a key ingredient to address the short release cycles. Testing at the graphical user interface (GUI) level is challenging to automate, and therefore many companies still do this manually. To help find solutions for better GUI test automation, academics are researching scriptless GUI testing to complement the script-based approach. In order to better match industrial problems with academic results, more academia-industry collaborations for case-based evaluations are needed. This paper describes such an initiative to improve, transfer and integrate an academic scriptless GUI testing tool TESTAR into the CI pipeline of a Spanish company Prodevelop. The paper describes the steps taken, the outcome, the challenges, and some lessons learned for successful industry-academia collaboration., This work has been funded through the ITEA3 TESTOMAT project (www.testomatproject.eu), the EU H2020 DECODER project (www.decoderproject.eu), the EU H2020 iv4XR project (iv4xr-project.eu) and the ITEA3 IVVES project (ivves.weebly.com).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Applying Scriptless Test Automation on Web Applications from the Financial Sector

  • Aho, Pekka
  • Buijs, Govert
  • Akin, Abdurrahman
  • Senturk, Serafettin
  • de Gouw, Stijn
  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
  • Vos, Tanja Ernestina|||0000-0002-6003-9113
[EN] This industry showcase presents experiences on application of testar, an open source tool for scriptless testing through graphical user interface (GUI), to the web applications of Kuveyt T¿urk Participation Bank in Turkey. Kuveyt T¿urk Bank uses Selenium and Appium for regression testing of mobile and internet banking, but the maintenance cost of the test scripts is increasing day by day. Therefore, scriptless GUI testing with testar was evaluated. To provide better support for testing web-based applications, testar was extended with Selenium WebDriver integration, JavaScript support, and other new features. Results show that testar detects GUI elements much better after the improvements, and it was able to find 2 relevant errors that were not identified by existing scripted test cases., This work has been partially funded by ITEA3 TESTOMAT Project, ITEA3 IVVES project7 and EU H2020 DECODER projec
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

DECODER - DEveloper COmpanion for Documented and annotatEd code Reference

  • Gil Pascual, Miriam
  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
  • Torres Bosch, Maria Victoria|||0000-0002-2039-2174
  • Vos, Tanja Ernestina|||0000-0002-6003-9113
This work has been developed with the financial support of the European Union's Horizon 2020 research and innovation programme under grant agreement No. 824231
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Extracting Knowledge from Software Artefacts to Assist Software Project Stakeholders

  • Gil Pascual, Miriam
  • Torres Bosch, Maria Victoria|||0000-0002-2039-2174
  • Albert Albiol, Manuela|||0000-0003-3747-400X
  • Pelechano Ferragud, Vicente|||0000-0003-1090-230X
[EN] Software development methods should foster the exploitation of artefacts from existing code bases in order to improve software development productivity. These artefacts are commonly stored in repositories from which extracting knowledge is very difficult for several reasons, i.e., the stored data is represented in a wide variety of formats or is not usually linked properly to all the related artefacts. In this work, we address the challenge of extracting knowledge from different artefacts that can be produced within a software project. To this end, we present a Persistent Knowledge Monitor (PKM) for handling several kinds of knowledge and information related to a software project. The PKM bases on the JSON format to structure and store the different artefacts. By using a common representation format, we are able to extract knowledge more easily. Also, we provide a query language for searching and reasoning on the stored data., This work has been developed with the financial support of the European Union's Horizon 2020 research and innovation programme under grant agreement No. 824231 and the Spanish State Research Agency under the project TIN2017-84094-R and co-financed with ERDF
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Tesis doctoral (DoctoralThesis)

Improving the QA Testing Process of Desktop, Web, Mobile, and XR Applications through Scriptless Testing using the TESTAR Tool

  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
[ES] El software es un pilar fundamental en sectores como la banca, los videojuegos y la salud. Las pruebas de software son el método más empleado para garantizar la calidad y fiabilidad de estos sistemas. Al someter los sistemas a pruebas rigurosas y recopilar evidencias, este proceso permite tomar decisiones informadas sobre la calidad del producto. Los testers humanos son fundamentales en las pruebas de software, aportando intuición, creatividad y comprensión profunda. No obstante, los esfuerzos de los testers no deben centrarse en la ejecución manual y repetitiva de interacciones con el software, ya que ello implica una carga de trabajo excesiva, frustración y fatiga, lo que genera falta de atención y limitación en el tiempo de realización de pruebas. Scripted testing es un enfoque para automatizar pruebas de software con scripts secuenciales y oráculos de prueba, reduciendo el esfuerzo manual y ayudando a mantener la calidad del software al detectar fallos de regresión. Sin embargo, la necesidad de mantener un número incontrolable de scripts, requeridos para sistemas grandes y complejos, hace que las pruebas basadas en scripts resulten insuficientes. Scriptless testing es un enfoque automatizado complementario que no depende de scripts, sino que explora el software de manera dinámica mediante acciones no secuenciales. Este enfoque introduce aleatoriedad, lo que permite descubrir estados y fallos no identificados por pruebas manuales o con scripts. TESTAR es una herramienta de automatización de pruebas sin scripts de código abierto que ha demostrado ser eficaz en la cobertura y detección de fallos al probar aplicaciones de escritorio y web industriales, a través de la Interfaz Gráfica de Usuario (GUI). La arquitectura modular de TESTAR permite seguir mejorando la herramienta para: (1) conectarse con otros sistemas software complejos, (2) incorporar algoritmos exploratorios para mejorar la efectividad y eficiencia de las pruebas, (3) integrar diverso de oráculos de prueba para cubrir una amplia variedad de características y detectar sus fallos, y (4) ser validada en proyectos industriales y de código abierto bien conocidos. En esta tesis, hemos colaborado con socios industriales y proyectos de código abierto para mejorar TESTAR, integrando marcos tecnológicos para conectar con aplicaciones de escritorio, web, móvil y de Realidad Extendida (XR); añadiendo algoritmos exploratorios para mejorar la efectividad y eficiencia de los mecanismos de selección de acciones; e integrando nuevos oráculos de prueba para la detección automatizada de fallos. Estas nuevas extensiones se han evaluado con proyectos industriales y de código abierto bien establecidos. Nuestros resultados empíricos demuestran una mejora significativa en las capacidades de TESTAR. Primero, TESTAR puede conectarse y probar aplicaciones de escritorio, web, móvil y juegos de XR. Segundo, la implementación de procesos de inferencia de modelos de estado, un enfoque de inferencia distribuida, reglas de selección de acciones basadas en gramáticas y un algoritmo interactivo para juegos, mejoran significativamente la efectividad y la eficiencia de la exploración sin scripts. Tercero, la integración de oráculos de prueba y detección de cambios delta permiten a TESTAR detectar diversos tipos de fallos en el software e identificar cambios en la GUI entre versiones delta de la misma aplicación. Cuarto, TESTAR ha demostrado ser una solución complementaria a enfoques de pruebas manuales y scripts en la industria. Finalmente, hemos realizado una colaboración implementando un enfoque de Desarrollo Guiado por Comportamiento (BDD) para automatizar pruebas de regresión de un juego XR industrial. Esta tesis concluye que usar una herramienta scriptless como TESTAR puede mejorar significativamente el proceso de aseguramiento de la calidad (QA) en aplicaciones de escritorio, web, móviles y XR, al aumentar la cobertura y detectar fallos inesperados., [CA] El programari és un pilar fonamental en sectors com la banca, els videojocs i la salut. Les proves de programari són el procés més utilitzat per assegurar la qualitat i la fiabilitat d'estos sistemes. Mitjançant proves rigoroses sobre els sistemes i la recopilació d'evidències, este procés permet prendre decisions informades sobre la qualitat d'un producte de programari. Els testers humans són fonamental en les proves de programari perquè aporten intuïció, creativitat i comprensió profunda. No obstant això, els seus esforços no haurien de destinar-se a l'execució d'interaccions manuals i repetitives amb el programari, ja que això genera càrregues de treball elevades, frustració i fatiga, cosa que comporta una falta d'atenció i de temps per realitzar les proves. Scripted testing és un enfocament per automatitzar l'execució de proves creant scripts seqüencials i oracles de prova, reduint l'esforç manual i ajudant a mantindre la qualitat del programari detectant fallades de regressió. No obstant això, la necessitat de mantindre un nombre incontrolable de scripts, per a sistemes grans i complexos, fan que la prova amb scripts siga un enfocament insuficient. Scriptless testing és un enfocament automatitzat complementari que no es basa en scripts, sinó que explora dinàmicament el programari amb accions no seqüencials. Este enfocament introduïx aleatorietat, la qual ajuda a descobrir estats i falles no identificats per les proves manuals o amb scripts. TESTAR és una eina d'automatització de proves sense scripts de codi obert que ha demostrat efectivitat en la cobertura i detecció de falles provant aplicacions industrials d'escriptori i web a través de la Interfície Gràfica d'Usuari (GUI). L'arquitectura modular de TESTAR permet a l'eina de millorar per a: (1) connectar-se amb altres sistemes de programari complexos, (2) incorporar algoritmes exploratoris per millorar l'efectivitat i l'eficiència de les proves, (3) integrar diversos oracles de prova per cobrir una àmplia varietat de característiques o descobrir els seus errors, i (4) ser validada en projectes industrials i de codi obert coneguts. En esta tesi, hem col·laborat amb socis industrials i projectes de codi obert per millorar TESTAR, integrant marcs tecnològics per connectar amb aplicacions d'escriptori, web, mòbils i de Realitat Estesa (XR); afegint algoritmes exploratoris per millorar l'efectivitat i l'eficiència dels mecanismes de selecció d'accions; i integrant nous oracles de prova per a la detecció automatitzada de fallades. Estes noves extensions s'han avaluat amb projectes industrials i de codi obert ben establits. Els nostres resultats empírics demostren una millora significativa en les capacitats de TESTAR. Primer, TESTAR pot connectar-se i provar aplicacions d'escriptori, web, mòbils i jocs de XR. En segon lloc, la implementació de processos d'inferència de models d'estat, un enfocament d'inferència distribuïda, regles de selecció d'accions basades en gramàtiques i un algoritme interactiu per a jocs milloren significativament l'efectivitat i l'eficiència de l'exploració sense scripts. En tercer lloc, la integració d'oracles de prova i la detecció de canvis delta permeten a TESTAR detectar diversos tipus de fallades en el programari i identificar canvis en la GUI entre versions delta de la mateixa aplicació. En quart lloc, TESTAR ha demostrat ser una solució complementària als enfocaments de proves manuals i amb scripts en la indústria. Finalment, hem col·laborat implementant un enfocament de Desenvolupament Guiat per Comportament (BDD) per automatitzar proves de regressió d'un joc XR industrial. Esta tesi conclou que utilitzar una eina scriptless com TESTAR pot millorar significativament el procés d'assegurament de la qualitat (QA) de les aplicacions d'escriptori, web, mòbils i XR augmentant la cobertura i detectant fallades inesperades., [EN] Software is a fundamental pillar in sectors such as banking, video games, and healthcare. Software testing is the most commonly used method to ensure the quality and reliability of these systems. By rigorously exercising software systems and collecting evidence, testing enables informed decisions about a software product's quality. Human testers play a vital role in software testing, bringing intuition, creativity, and a deep understanding of real-world contexts. However, testers' efforts should not be misdirected to execute manual and repetitive software interactions, as this leads to excessive workload, frustration, and fatigue, resulting in decreased attention and limited testing time. Scripted testing automates testing through sequential scripts and test oracles, reducing manual effort and helping maintain software quality by detecting unintended regression failures. However, the need to maintain these scripts and the unmanageable number of scripts required for large and complex systems make scripted testing an insufficient approach. Scriptless testing is a complementary automated approach that does not rely on scripts but dynamically explores software with non-sequential actions. This approach introduces randomness, which helps to uncover states and failures not identified by manual or scripted testing. TESTAR is an open-source scriptless testing tool that has demonstrated its coverage and fault detection effectiveness when testing industrial desktop and web applications through the Graphical User Interface (GUI). The modular architecture of TESTAR enables the tool for further improvement to: (1) connect with other complex software systems, (2) incorporate additional exploratory algorithms for test effectiveness and efficiency, (3) integrate a diverse set of test oracles to cover a wide variety of features or discover its faults, and (4) be validated with industrial and well-known open-source projects. In this thesis, we conducted several research collaborations with industrial partners and open-source projects to improve TESTAR. We extended its interaction capabilities by integrating frameworks to test desktop, web, mobile, and eXtended Reality (XR) applications, added exploratory algorithms to improve the test effectiveness and efficiency of action selection mechanisms, and integrated a new set of test oracles for automated software assurance and fault detection. These novel extensions were evaluated with industrial applications and well-established open-source projects. Our empirical results demonstrate a significant improvement in TESTAR capabilities. First, TESTAR can successfully connect to and test various desktop, web, mobile, and XR game applications. Second, a state model inference processes, a distributed inference approach, grammar-based action selection rules, and an interactive algorithm for navigating game environments help significantly improve the effectiveness and efficiency of scriptless exploration. Third, the integration of online test oracles and offline delta change detection oracles enabled TESTAR to detect diverse types of software failures and identify and highlight GUI changes between delta versions of the same software application. Fourth, TESTAR has proved to be complementary to manual and scripted testing approaches in industrial organizations. Finally, we additionally collaborated to implement a Behavior-Driven Development (BDD) approach for automating regression tests of an XR industrial game system. This thesis concludes that using a scriptless testing tool like TESTAR can significantly improve the Quality Assurance (QA) testing process of desktop, web, mobile, and XR applications by increasing coverage and detecting unexpected failures., This thesis has been developed at the Universitat Politècnica de València and has been funded by various European research projects: ITEA3 TESTOMAT project (TESTOMAT 16032), H2020 DEveloper COmpanion for Documented and annotatEd code Reference (DECODER 824231), H2020 Intelligent Verification/Validation for Extended Reality Based Systems (IV4XR 856716), ITEA3 Industrial-grade Verification and Validation of Evolving Systems (IVVES 18022), Erasmus+ European Innovation Alliance for Testing Education (ENACTEST 101055874), NWO OTP project Automated Unobtrusive Techniques for LINK- ing requirements and testing in agile software development (AUTOLINK 19521).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Evaluating TESTAR&apos, s effectiveness through code coverage

  • van der Brugge, Aaron
  • Aho, Pekka
  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
  • Marín, Beatriz|||0000-0001-8025-0023
  • Vos, Tanja Ernestina|||0000-0002-6003-9113
[EN] Testing is of paramount importance in assuring the quality of software products. Nevertheless, it is not easy to judge which techniques or tools are the most effective. A commonly used surrogate metric to evaluate the effectiveness of testing tools is code coverage, which has been widely used for unit and integration testing. However, for GUI testing approaches, this metric has not been sufficiently investigated. To fill this gap, we run experiments with the TESTAR tool, a scriptless testing tool that automatically generates test cases at the Graphical User Interface (GUI) level. In the experiment, we analyze and compare the obtained code coverage when using four different action selection mechanisms (ASMs) in TESTAR that are used to test three SUTs, This research has been funded by the following projects: H2020 EU project DECODER (www.decoder-project.eu), H2020 EU project iv4XR (www.iv4xrproject.eu) and ITEA project IVVES (www.ivves.eu).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

testar - scriptless testing through graphical user interface

  • Vos, Tanja Ernestina|||0000-0002-6003-9113
  • Pastor-Ricós, Fernando|||0000-0002-5790-193X
  • Aho, Pekka
  • Rodríguez-Valdés, Olivia
  • Mulders, Ad
[EN] Covering all the possible paths of the graphical user interface (GUI) with test scripts would take too much effort and result in serious maintenance issues. We propose complementing scripted testing with scriptless test automation using the open-source TESTAR tool. This paper gives a comprehensive overview of TESTAR and its latest extensions together with the ongoing and future research. With this paper, we hope we can help and encourage other researchers to use TESTAR for their GUI testing-related research and pave the way for an international research agenda in GUI testing built upon stable and open-source infrastructure. © 2021 The Authors. Software Testing, Verification & Reliability published by John Wiley & Sons Ltd., This work has been funded by the following projects: FITTEST (FP7 Information and Communication Technologies, 257574), SHIP (EACEA/A2/UHB/CL 554187), PERTEST (TIN2013-46928-C3-1-R), TESTOMAT (ITEA3, 16032), DECODER (H2020, 824231), iv4XR (H2020, 856716), TAILOR (H2020, 952215) and IVVES (ITEA3, 18022)