Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

A systematic review on cloud testing

  • Bertolino, Antonia
  • De Angelis, Guglielmo
  • Gallego, Micael
  • García Gutiérrez, Boni
  • Gortázar, Francisco
  • Lonetti, Francesca
  • Marchetti, Eda
A systematic literature review is presented that surveyed the topic of cloud testing over the period (2012-2017). Cloud testing can refer either to testing cloud-based systems (testing of the cloud), or to leveraging the cloud for testing purposes (testing in the cloud): both approaches (and their combination into testing of the cloud in the cloud) have drawn research interest. An extensive paper search was conducted by both automated query of popular digital libraries and snowballing, which resulted into the final selection of 147 primary studies. Along the survey a framework has been incrementally derived that classifies cloud testing research along six main areas and their topics. The paper includes a detailed analysis of the selected primary studies to identify trends and gaps, as well as an extensive report of the state of art as it emerges by answering the identified Research Questions. We find that cloud testing is an active research field, although not all topics have received so far enough attention, and conclude by presenting the most relevant open research challenges for each area of the classification framework., This paper describes research work mostly undertaken in the context of the European Project H2020 731535: ElasTest. This work has also been partially supported by: the Italian MIUR PRIN 2015 Project: GAUSS; the Regional Government of Madrid (CM) under project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER; and the Spanish Government under project LERNIM (RTC-2016-4674-7) cofunded by the Ministry of Economy and Competitiveness, FEDER & AEI.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

Practical evaluation of VMAF perceptual video quality for WebRTC applications

  • García Gutiérrez, Boni
  • López-Fernández, Luis
  • Gortázar, Francisco
  • Gallego, Micael
WebRTC is the umbrella term for several emergent technologies aimed to exchange real-time media in the Web. Like other media-related services, the perceived quality of WebRTC communication can be measured using Quality of Experience (QoE) indicators. QoE assessment methods can be classified as subjective (users' evaluation scores) or objective (models computed as a function of different parameters). In this paper, we focus on VMAF (Video Multi-method Assessment Fusion), which is an emergent full-reference objective video quality assessment model developed by Netflix. VMAF is typically used to assess video streaming services. This paper evaluates the use of VMAF in a different type of application: WebRTC. To that aim, we present a practical use case built on the top of well-known open source technologies, such as JUnit, Selenium, Docker, and FFmpeg. In addition to VMAF, we also calculate other objective QoE video metrics such as Visual Information Fidelity in the pixel domain (VIFp), Structural Similarity (SSIM), or Peak Signal-to-Noise Ratio (PSNR) applied to a WebRTC communication in different network conditions in terms of packet loss. Finally, we compare these objective results with a subjective evaluation using a Mean Opinion Score (MOS) scale to the same WebRTC streams. As a result, we found a strong correlation of the subjective video quality perceived in WebRTC video calls with the objective results computed with VMAF and VIFp in comparison with SSIM and PSNR and their variants., This work has been supported by the European Commission under project ElasTest (H2020-ICT-10-2016, GA-731535); by the Regional Government of Madrid (CM) under project EDGEDATA-CM (P2018/TCS-4499) cofunded by FSE and FEDER; by the Spanish Government under projects LERNIM (RTC-2016-4674-7) and BugBirth (RTI2018-101963-B-I00) cofunded by the Ministry of Economy and Competitiveness, FEDER and AEI.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

Kurento: The Swiss Army Knife of WebRTC Media Servers

  • García Gutiérrez, Boni
  • López-Fernández, Luis
  • Gallego, Micael
  • Gortázar, Francisco
In this article we introduce Kurento, an open source WebRTC media server and a set of client APIs intended to simplify the development of applications with rich media capabilities for the Web and smartphone platforms. Kurento features include group communications, transcoding, recording, mixing, broadcasting and routing of audiovisual flows, but also provides advanced media processing capabilities such as computer vision and augmented reality. It is based on a modular architecture, which makes it possible for developers to extend and customize its native capabilities with third-party media processing algorithms. Thanks to all of this, Kurento can be a powerful tool for Web developers who may find natural programming with its Java and JavaScript APIs following the traditional three-tiered Web development model.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

Understanding and estimating quality of experience in WebRTC applications

  • García Gutiérrez, Boni
  • Gallego, Micael
  • Gortázar, Francisco
  • Bertolino, Antonia
WebRTC comprises a set of technologies and standards that provide real-time communication with web browsers, simplifying the embedding of voice and video communication in web applications and mobile devices. The perceived quality of WebRTC communication can be measured using quality of experience (QoE) indicators. QoE is defined as the degree of delight or annoyance of the user with an application or service. This paper is focused on the QoE assessment of WebRTC-based applications and its contribution is threefold. First, an analysis of how WebRTC topologies affect the quality perceived by users is provided. Second, a group of Key Performance Indicators for estimating the QoE of WebRTC users is proposed. Finally, a systematic survey of the literature on QoE assessment in the WebRTC arena is presented.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Capítulo o parte de un libro (BookPart)

ElasTest, an open-source platform to ease end-to-end testing

  • García Gutiérrez, Boni
  • Gallego, Micael
  • Gortázar, Francisco
  • López, Luis
The demand for larger and more interconnected software systems is constantly increasing, but the ability of developers to satisfy this demand is not evolving accordingly. The most limiting factor is software Verification and Validation (V&V), which typically requires very costly and complex testing processes. The objective of the ElasTest project is to significantly improve the efficiency and effectiveness of the testing process and, with it, the overall quality of large software systems. To that aim, ElasTest provides an integrated solution for end-to-end test automation along the development life cycle, including test case management, deployment, instrumentation, and monitoring for different kind of applications, including web and mobile., This work has been supported by the European Commission under project ElasTest (H2020-ICT-10-2016, GA-731535); by the Regional Government of Madrid (CM) under project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER; and Spanish Government under project LERNIM (RTC-2016-4674-7) cofunded by the Ministry of Economy and Competitiveness, FEDER & AEI.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

Assessment of QoE for video and audio in WebRTC applications using full-reference models

  • García Gutiérrez, Boni
  • Gortázar, Francisco
  • Gallego, Micael
  • Hines, Andrew
WebRTC is a set of standard technologies that allows exchanging video and audio in real time on the Web. As with other media-related applications, the user-perceived audiovisual quality can be estimated using Quality of Experience (QoE) measurements. This paper analyses the behavior of different objective Full-Reference (FR) models for video and audio in WebRTC applications. FR models calculate the video and audio quality by comparing some original media reference with the degraded signal. To compute these models, we have created an open-source benchmark in which different types of reference media inputs are sent browser to browser while simulating different kinds of network conditions in terms of packet loss and jitter. Our benchmark provides recording capabilities of the impairment WebRTC streams. Then, we use different existing FR metrics for video (VMAF, VIFp, SSIM, MS-SSIM, PSNR, PSNR-HVS, and PSNR-HVS-M) and audio (PESQ, ViSQOL, and POLQA) recordings together with their references. Moreover, we use the same recordings to carry out a subjective analysis in which real users rate the video and audio quality using a Mean Opinion Score (MOS). Finally, we calculate the correlations between the objective and subjective results to find the objective models that better correspond with the subjective outcome, which is considered the ground truth QoE. We find that some of the studied objective models, such as VMAF, VIFp, and POLQA, show a strong correlation with the subjective results in packet loss scenarios., This work has been supported by the European Commission under project ElasTest (H2020-ICT-10-2016, GA-731535); by the Regional Government of Madrid (CM) under project EDGEDATA-CM (P2018/TCS-4499) cofunded by FSE and FEDER; by the Spanish Government under projects LERNIM (RTC-2016-4674-7) and BugBirth (RTI2018-101963-B-I00) cofunded by the Ministry of Economy and Competitiveness, FEDER, and AEI; and by Science Foundation Ireland (SFI) cofunded under the European Regional Development Fund under grant number 12/RC/2289_P2 and grant number SFI/12/RC/2077.
e-Archivo. Repositorio Institucional de la Universidad Carlos III de Madrid
Publicación Artículo científico (JournalArticle)

WebRTC testing: challenges and practical solutions

  • García Gutiérrez, Boni
  • Gortázar, Francisco
  • López-Fernández, Luis
  • Gallego, Micael
  • París, Miguel
WebRTC comprises a set of novel technologies and standards that provide Real-Time Communication on Web browsers. WebRTC makes simple the embedding of voice and video communications in all types of applications. However, releasing those applications to production is still very challenging due to the complexity of their testing. Validating a WebRTC service requires assessing many functional (e.g. signaling logic, media connectivity, etc.) and non-functional (e.g. quality of experience, interoperability, scalability, etc.) properties on large, complex, distributed and heterogeneous systems that spawn across client devices, networks and cloud infrastructures. In this article, we present a novel methodology and an associated tool for doing it at scale and in an automated way. Our strategy is based on a blackbox end-to-end approach through which we use an automated containerized cloud environment for instrumenting Web browser clients, which benchmark the SUT (system under test), and fake clients, that load it. Through these benchmarks, we obtain, in a reliable and statistically significant way, both network-dependent QoS (Quality of Service) metrics and media-dependent QoE (Quality of Experience) indicators. These are fed, at a second stage, to a number of testing assertions that validate the appropriateness of the functional and non-functional properties of the SUT under controlled and configurable load and fail conditions. To finish, we illustrate our experiences using such tool and methodology in the context of the Kurento open source software project and conclude that they are suitable for validating large and complex WebRTC systems at scale.
Archivo Digital UPM
Publicación Tesis doctoral (DoctoralThesis)

Tools for the Detection and Analysis of Potentially Unwanted Programs

Archivo Digital UPM
Publicación Tesis doctoral (DoctoralThesis)

A Systematic Empirical Analysis of Unwanted Software Abuse, Prevalence, Distribution, and Economics

Archivo Digital UPM
Publicación Tesis doctoral (DoctoralThesis)

Lineage Inference of Packed Malware using Binary Code Similarity

RUO. Repositorio Institucional de la Universidad de Oviedo
Publicación Artículo científico (JournalArticle)

RETORCH: an approach for resource-aware orchestration of end-to-end test cases

  • Augusto Alonso, Cristian|||0000-0001-6140-1375
  • Morán Barbón, Jesús|||0000-0002-7544-3901
  • Bertolino, A.
  • Riva Álvarez, Claudio A. de la|||0000-0001-5592-9683
  • Tuya González, Pablo Javier|||0000-0002-1091-934X
Continuous integration practice mandates to continuously introduce incremental changes into code, but doing so may introduce new faults too. These faults could be detected automatically through regression testing, but this practice becomes prohibitive as the cost of executing the tests grows. This problem is preponderant in end-to-end testing where the whole system is requested for test execution. However, some of these test cases could be executed with fewer resources (e.g., memory, web services, computation, Cloud instances, among others), by deploying only the subsystems needed by each test. This paper is focused on the optimization of the resources employed in end-to-end testing by means of a resource-aware test orchestration technique in the context of continuous integration practices in the Cloud. The RETORCH approach proposes a novel way to identify the resources required by end-to-end test cases and to use this information to group together those tests requiring equivalent resources. Besides, the approach proposes to deploy the grouped tests in isolated and elastic environments, so that their execution can be scheduled in parallel on several machines. RETORCH is exemplified with a real-world application, and its performance evaluation shows promising savings in terms of resource usage and time., Este trabajo fue financiado en parte por el Ministerio de Economía y Competitividad de España en el marco del proyecto TestEAMoS (TIN2016-76956-C3-1-R) y fondos FEDER y por el Proyecto Europeo ElasTest en el programa de investigación e innovación Horizonte 2020 (GA No. 731535).
RUO. Repositorio Institucional de la Universidad de Oviedo
Publicación Contribución de congreso (ConferenceOutput)

Debugging flaky tests on web applications

  • Morán Barbón, Jesús|||0000-0002-7544-3901
  • Augusto Alonso, Cristian|||0000-0001-6140-1375
  • Bertolino, A.
  • Riva Álvarez, Claudio A. de la|||0000-0001-5592-9683
  • García Tuya, José Florentino
International Conference on Web Information Systems and Technologies, WEBIST (15th. 2019. Vienna, Austria), This work was supported in part by the Spanish Ministry of Economy and Competitiveness under TestEAMoS (TIN2016-76956-C3-1-R) project and ERDF funds, and by the European Project ElasTest in the Horizon 2020 research and innovation program (GA No. 731535).
RUO. Repositorio Institucional de la Universidad de Oviedo
Publicación Artículo científico (JournalArticle)

FlakyLoc: flakiness localization for reliable test suites in web applications

  • Morán Barbón, Jesús|||0000-0002-7544-3901
  • Augusto Alonso, Cristian|||0000-0001-6140-1375
  • Bertolino, Antonia
  • Riva Álvarez, Claudio A. de la|||0000-0001-5592-9683
  • Tuya González, Pablo Javier|||0000-0002-1091-934X
Web application testing is a great challenge due to the management of complex asynchronous communications, the concurrency between the clients-servers, and the heterogeneity of resources employed. It is difficult to ensure that a test case is re-running in the same conditions because it can be executed in undesirable ways according to several environmental factors that are not easy to fine-grain control such as network bottlenecks, memory issues or screen resolution. These environmental factors can cause flakiness, which occurs when the same test case sometimes obtains one test outcome and other times another outcome in the same application due to the execution of environmental factors. The tester usually stops relying on flaky test cases because their outcome varies during the re-executions. To fix and reduce the flakiness it is very important to locate and understand which environmental factors cause the flakiness. This paper is focused on the localization of the root cause of flakiness in web applications based on the characterization of the different environmental factors that are not controlled during testing. The root cause of flakiness is located by means of spectrum-based localization techniques that analyse the test execution under different combinations of the environmental factors that can trigger the flakiness. This technique is evaluated with an educational web platform called FullTeaching. As a result, our technique was able to locate automatically the root cause of flakiness and provide enough information to both understand it and fix it., Este trabajo fue financiado en parte por el Ministerio de Economía y Competitividad de España bajo proyecto TestEAMoS (TIN2016-76956-C3-1-R) y proyecto POLOLAS (TIN2016- 76956-C3-2-R), fondos FEDER, y por el Proyecto Europeo ElasTest en el Horizonte 2020 programa de investigación e innovación (GA No. 731535).