Skip to main content

NUEVOS DESAFIOS EN COMPUTACION DE ALTAS PRESTACIONES: DESDE ARQUITECTURAS HASTA APLICACIONES (II)

TIN2016-75845-P

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2016
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSIDADE DA CORUÑA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 36
Encontrada(s) 1 página(s)

Investigo. Repositorio Institucional de la Universidade de Vigo
Publication Artículo científico (JournalArticle)

Toxo: a library for calculating penetrance tables of high-order epistasis models

  • Ponte-Fernández, Christian
  • González-Domínguez, Jorge
  • Carvajal Rodríguez, Antonio
  • Martín, María J.
Background: Epistasis is defined as the interaction between different genes when expressing a specific phenotype. The most common way to characterize an epistatic relationship is using a penetrance table, which contains the probability of expressing the phenotype under study given a particular allele combination. Available simulators can only create penetrance tables for well-known epistasis models involving a small number of genes and under a large number of limitations., Results: Toxo is a MATLAB library designed to calculate penetrance tables of epistasis models of any interaction order which resemble real data more closely. The user specifies the desired heritability (or prevalence) and the program maximizes the table’s prevalence (or heritability) according to the input epistatic model boundaries., Conclusions: Toxo extends the capabilities of existing simulators that define epistasis using penetrance tables. These tables can be directly used as input for software simulators such as GAMETES so that they are able to generate data samples with larger interactions and more realistic prevalences/heritabilities., Ministerio de Economía, Industria y Competitividad | Ref. CGL2016-75482-P, Ministerio de Economía, Industria y Competitividad | Ref. TIN2016-75845-P, Xunta de Galicia | Ref. ED431C2016-037, Xunta de Galicia | Ref. ED431G2019/01, Xunta de Galicia | Ref. ED431C 2017/04, Ministerio de Educación, Cultura y Deporte | Ref. FPU16/01333
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

A Software Cache Autotuning Strategy for Dataflow Computing with UPC++ DepSpawn

  • Fraguela, Basilio B.
  • Andrade, Diego
This is the accepted version of the following article: B. B. Fraguela, D. Andrade. A software cache autotuning strategy for dataflow computing with UPC++ DepSpawn. Computational and Mathematical Methods, 3(6), e1148. November 2021, which has been published in final form at http://dx.doi.org/10.1002/cmm4.1148. This article may be used for noncommercial purposes in accordance with the Wiley Self-Archiving Policy [http://www.wileyauthors.com/self-archiving]., [Abstract] Dataflow computing allows to start computations as soon as all their dependencies are satisfied. This is particularly useful in applications with irregular or complex patterns of dependencies which would otherwise involve either coarse grain synchronizations which would degrade performance, or high programming costs. A recent proposal for the easy development of performant dataflow algorithms in hybrid shared/distributed memory systems is UPC++ DepSpawn. Among the many techniques it applies to provide good performance is a software cache that minimizes the communications among the processes involved. In this article we provide the details of the implementation and operation of this cache and we present an autotuning strategy that simplifies its usage by freeing the user from having to estimate an adequate size for this cache. Rather, the runtime is now able to define reasonably sized caches that provide near optimal behavior., This research was funded by the Ministry of Science and Innovation of Spain (TIN2016-75845-P and PID2019-104184RB-I00, AEI/FEDER/EU, 10.13039/501100011033), and by the Xunta de Galicia co-funded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2017/04). The authors acknowledge also the support from the Centro Singular de Investigación de Galicia “CITIC,” funded by Xunta de Galicia and the European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant ED431G 2019/01. They also acknowledge the Centro de Supercomputación de Galicia (CESGA) for the use of its computers, Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Guiding the Optimization of Parallel Codes on Multicores Using an Analytical Cache Model

  • Andrade, Diego
  • Fraguela, Basilio B.
  • Doallo, Ramón
Versión final aceptada de: https://doi.org/10.1007/978-3-319-93713-7_32, This is a post-peer-review, pre-copyedit version of an article published in Lecture Notes on Computer Science (ICCS 2018 proceedings). The final authenticated version is available online at: http://dx.doi.org/10.1007/978-3-319-93713-7_32, [Abstract]: Cache performance is particularly hard to predict in modern multicore processors as several threads can be concurrently in execution, and private cache levels are combined with shared ones. This paper presents an analytical model able to evaluate the cache performance of the whole cache hierarchy for parallel applications in less than one second taking as input their source code and the cache configuration. While the model does not tackle some advanced hardware features, it can help optimizers to make reasonably good decisions in a very short time. This is supported by an evaluation based on two modern architectures and three different case studies, in which the model predictions differ on average just 5.05% from the results of a detailed hardware simulator and correctly guide different optimization decisions., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds (80%) of the EU (TIN2016-75845-P), and by the Government of Galicia (Xunta de Galicia) co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2017/04) as well as under the Centro Singular de Investigación de Galicia accreditation 2016-2019 (ED431G/01). We also acknowledge the Centro de Supercomputación de Galicia (CESGA) for the use of their computers., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

SeQual: Big Data Tool to Perform Quality Control and Data Preprocessing of Large NGS Datasets

  • Expósito, Roberto R.
  • Galego Torreiro, Roi
  • González-Domínguez, Jorge
[Abstract] This paper presents SeQual, a scalable tool to efficiently perform quality control of large genomic datasets. Our tool currently supports more than 30 different operations (e.g., filtering, trimming, formatting) that can be applied to DNA/RNA reads in FASTQ/FASTA formats to improve subsequent downstream analyses, while providing a simple and user-friendly graphical interface for non-expert users. Furthermore, SeQual takes full advantage of Big Data technologies to process massive datasets on distributed-memory systems such as clusters by relying on the open-source Apache Spark cluster computing framework. Our scalable Spark-based implementation allows to reduce the runtime from more than three hours to less than 20 minutes when processing a paired-end dataset with 251 million reads per input file on an 8-node multi-core cluster., 10.13039/501100004837-Ministry of Science and Innovation of Spain (Grant Number: TIN2016-75845-P and PID2019-104184RB-I00) 10.13039/501100004837-AEI/FEDER/EU (Grant Number: 10.13039/501100011033) 10.13039/501100010801-Xunta de Galicia and FEDER funds (Centro de Investigación de Galicia accreditation 2019–2022 and the Consolidation Program of Competitive Reference Groups) (Grant Number: ED431G 2019/01 and ED431C 2017/04), Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2017/04
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Big data storage technologies: a case study for web-based LiDAR visualization

  • Deibe, David
  • Amor, Margarita
  • Doallo, Ramón
Presented at: 2018 IEEE International Conference on Big Data (Big Data), Seattle, 10 December 2018 through 13 December, This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/BigData.2018.8622589, [Abstract]: Big data technologies have been growing up quickly during past years. New storage and computing solutions appear while those already established in the market are improved with new features and better performance. Along with this growth also rises the number of applications and fields where the inclusion of big data technologies provides a large number of benefits, from the reduction in computational costs and economic resources to the improvement in the quality of the services provided which has a direct impact on the customers satisfaction. LiDAR (Light Detection and Ranging) data processing is one of the topics that could benefit from the adoption of these kind of technologies due to the massive datasets that are being gathered nowadays, with applications in archaeology, geography, geology or forestry, among many others. An efficient management of this volume of data becomes a key point especially in visualization, computing and analytic processes. In this paper, we analyse how web applications for the visualization of LiDAR data can benefit from the adoption of big data storage technologies, as well as the advantages and disadvantages that may determine the choice of one of them., This research has been supported by the Government of Galicia (Xunta de Galicia) under the Consolidation Programme of Competitive Reference Groups, co-founded by ERDF funds from the EU [Ref. ED431C 2017/04]; under the Consolidation Programme of Competitive Research Units, co-founded by ERDF funds from the EU [Ref. R2016/037]; by Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016/2019) and the European Union (European Regional Development Fund, ERDF) under Grant Ref. ED431G/01; and by the Ministry of Economy and Competitiveness of Spain and ERDF funds from the EU [TIN2016-75845-P]. The LiDAR datasets used in this article belong to: LiDAR-PNOA data repository. Provided by c Instituto Geográfico Nacional de España [33]. PG&E Diablo Canyon Power Plant (DCPP): San Simeon, CA Central Coast and PG&E Diablo Canyon Power Plant (DCPP): Los Osos, CA Central Coast. This material is based on LiDAR Point Cloud Data Distribution and Processing services provided by the OpenTopography Facility with support from the National Science Foundation under NSF Award Numbers 1226353 & 1225810 [34]., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01, Xunta de Galicia; R2016/037, United States. National Science Foundation; 1226353, United States. National Science Foundation; 1225810
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Evaluation of Existing Methods for High-Order Epistasis Detection

  • Ponte-Fernández, Christian
  • González-Domínguez, Jorge
  • Carvajal-Rodriguez, Antonio
  • Martín, María J.
[Abstract] Finding epistatic interactions among loci when expressing a phenotype is a widely employed strategy to understand the genetic architecture of complex traits in GWAS. The abundance of methods dedicated to the same purpose, however, makes it increasingly difficult for scientists to decide which method is more suitable for their studies. This work compares the different epistasis detection methods published during the last decade in terms of runtime, detection power and type I error rate, with a special emphasis on high-order interactions. Results show that in terms of detection power, the only methods that perform well across all experiments are the exhaustive methods, although their computational cost may be prohibitive in large-scale studies. Regarding non-exhaustive methods, not one could consistently find epistasis interactions when marginal effects are absent. If marginal effects are present, there are methods that perform well for high-order interactions, such as BADTrees, FDHE-IW, SingleMI or SNPHarvester. As for false-positive control, only SNPHarvester, FDHE-IW and DCHE show good results. The study concludes that there is no single epistasis detection method to recommend in all scenarios. Authors should prioritize exhaustive methods when sufficient computational resources are available considering the data set size, and resort to non-exhaustive methods when the analysis time is prohibitive., This research was funded by the Ministry of Economy and Competitiveness of Spain (CGL2016-75482-P), Ministry of Science and Innovation of Spain (TIN2016-75845-P and PID2019-104184RB-I00, AEI/FEDER/EU, 10.13039/501100011033), the Xunta de Galicia (Grupo de Referencia Competitiva, ED431C2016-037), the Xunta de Galicia and FEDER funds of the EU (Centro de Investigación de Galicia accreditation 2019-2022, ref. ED431G2019/01), Consolidation Program of Competitive Reference Groups (ED431C 2017/04)and the FPU Program of the Ministry of Education of Spain (FPU16/01333). The authors would like to thank the Supercomputing Center of Galicia (CESGA)for providing access to its supercomputing facilities and assisting with the problems encountered, as well as all of the authors contacted for assistance while using their epistasis detection applications., Xunta de Galicia; ED431C2016-037, Xunta de Galicia; ED431G2019/01, Xunta de Galicia; ED431C 2017/04
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Parallel feature selection for distributed-memory clusters

  • González-Domínguez, Jorge
  • Bolón-Canedo, Verónica
  • Freire, Borja
  • Touriño, Juan
Versión final aceptada de: https://doi.org/10.1016/j.ins.2019.01.050, This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/ licenses/by-nc-nd/4.0/. This version of the article: González-Domínguez, J. et al. (2019) ‘Parallel feature selection for distributed-memory clusters’, has been accepted for publication in Information Sciences, 496, pp. 399–409. The Version of Record is available online at: https://doi.org/10.1016/j.ins.2019.01.050, [Abstract]: Feature selection is nowadays an extremely important data mining stage in the field of machine learning due to the appearance of problems of high dimensionality. In the literature there are numerous feature selection methods, mRMR (minimum-Redundancy-Maximum-Relevance) being one of the most widely used. However, although it achieves good results in selecting relevant features, it is impractical for datasets with thousands of features. A possible solution to this limitation is the use of the fast-mRMR method, a greedy optimization of the mRMR algorithm that improves both scalability and efficiency. In this work we present fast-mRMR-MPI, a novel hybrid parallel implementation that uses MPI and OpenMP to accelerate feature selection on distributed-memory clusters. Our performance evaluation on two different systems using five representative input datasets shows that fast-mRMR-MPI is significantly faster than fast-mRMR while providing the same results. As an example, our tool needs less than one minute to select 200 features of a dataset with more than four million features and 16,000 samples on a cluster with 32 nodes (768 cores in total), while the sequential fast-mRMR required more than eight hours. Moreover, fast-mRMR-MPI distributes data so that it is able to exploit the memory available on different nodes of a cluster and then complete analyses that fail on a single node due to memory constraints. Our tool is publicly available at https://github.com/borjaf696/Fast-mRMR., This research has been partially funded by projects TIN2016-75845-P and TIN-2015-65069-C2-1-R of the Ministry of Economy, Industry and Competitiveness of Spain, as well as by Xunta de Galicia projects ED431D R2016/045 and GRC2014/035, all of them partially funded by FEDER funds of the European Union. We gratefully thank CESGA for providing access to the Finis Terrae II supercomputer., Xunta de Galicia; ED431D R2016/045, Xunta de Galicia; GRC2014/035
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Local Rollback for Resilient Mpi Applications With Application-Level Checkpointing and Message Logging

  • Losada, Nuria
  • Bosilca, George
  • Bouteiller, Aurelien
  • González, Patricia
  • Martín, María J.
[Abstract] The resilience approach generally used in high-performance computing (HPC) relies on coordinated checkpoint/restart, a global rollback of all the processes that are running the application. However, in many instances, the failure has a more localized scope and its impact is usually restricted to a subset of the resources being used. Thus, a global rollback would result in unnecessary overhead and energy consumption, since all processes, including those unaffected by the failure, discard their state and roll back to the last checkpoint to repeat computations that were already done. The User Level Failure Mitigation (ULFM) interface – the last proposal for the inclusion of resilience features in the Message Passing Interface (MPI) standard – enables the deployment of more flexible recovery strategies, including localized recovery. This work proposes a local rollback approach that can be generally applied to Single Program, Multiple Data (SPMD) applications by combining ULFM, the ComPiler for Portable Checkpointing (CPPC) tool, and the Open MPI VProtocol system-level message logging component. Only failed processes are recovered from the last checkpoint, while consistency before further progress in the execution is achieved through a two-level message logging process. To further optimize this approach point-to-point communications are logged by the Open MPI VProtocol component, while collective communications are optimally logged at the application level—thereby decoupling the logging protocol from the particular collective implementation. This spatially coordinated protocol applied by CPPC reduces the log size, the log memory requirements and overall the resilience impact on the applications., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (Projects TIN2016-75845-P and the predoctoral grants of Nuria Losada ref. BES-2014-068066 and ref. EEBB-I-17-12005); by EU under the COST Program Action IC1305 Network for Sustainable Ultrascale Computing (NESUS) and a HiPEAC Collaboration Grant and by the Galician Government (Xunta de Galicia) under the Consolidation Program of Competitive Research (ref. ED431C 2017/04). We gratefully thank Galicia Supercomputing Center for providing access to the FinisTerrae-II supercomputer. This material is also based upon work supported by the US National Science Foundation, Office of Advanced Cyberinfrastructure , under Grants No. #1664142 and #1339763, Xunta de Galicia; ED431C 2017/04, US National Science Foundation, Office of Advanced Cyberinfrastructure; 1664142, US National Science Foundation, Office of Advanced Cyberinfrastructure; 1339763
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Heterogeneous distributed computing based on high-level abstractions

  • Viñas Buceta, Moisés
  • Fraguela, Basilio B.
  • Andrade, Diego
  • Doallo, Ramón
Versión final aceptada de: https://doi.org/10.1002/cpe.4664, This is the peer reviewed version of the following article: Viñas, M, Fraguela, BB, Andrade, D, Doallo, R. Heterogeneous distributed computing based on high level abstractions. Concurrency Computat Pract Exper. 2018; 30:e4664., which has been published in final form at https://doi.org/10.1002/cpe.4664. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited., [Abstract]: The rise of heterogeneous systems has given place to great challenges for users as they involve new concepts, restrictions, and frameworks. Their exploitation is further complicated in the context of distributed memory systems, which require the usage of additional different programming paradigms and tools. In this paper, we propose a novel approach to program heterogeneous clusters that is based on high-level abstractions such as tiles and hierarchical decomposition combined with the powerful APIs that data types and embedded languages can provide in languages such as C++. Rather than building our proposal from scratch, we have implemented it as a natural integration of the existing Hierarchically Tiled Arrays (HTA) and Heterogeneous Programming Library (HPL) projects, ie, the first one being focused on distributed computing and the second one on heterogeneous processing. The result, called Heterogeneous Hierarchically Tiled Arrays (H2TA), is very intuitive and easy to use thanks to the global view of the data and the single-threaded view of the execution that it provides at cluster level together with the transparency it provides with respect to the management of the heterogeneous devices. An evaluation comparing our proposal with MPI-based implementations shows its large programmability advantages and the reasonable overhead incurred., This research was supported by the Ministerio de Economía y Competitividad under project TIN2016-75845-P; and the Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia (Centro Singular de Investigación de Galicia [accreditation 2016-2019] and Consolidación y Estructuración de Unidades de Investigación Competitivas: Grupo de Arquitectura de Computadores de la Universidad de A Coruña [UDC/GI-000265]); and the European Union (European Regional Development Fund) under grants ED431G/01 and ED431C 2017/04., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; UDC/GI-000265
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Toxo: A Library for Calculating Penetrance Tables of High-Order Epistasis Models

  • Ponte-Fernández, Christian
  • González-Domínguez, Jorge
  • Carvajal-Rodriguez, Antonio
  • Martín, María J.
[Abstract] Background Epistasis is defined as the interaction between different genes when expressing a specific phenotype. The most common way to characterize an epistatic relationship is using a penetrance table, which contains the probability of expressing the phenotype under study given a particular allele combination. Available simulators can only create penetrance tables for well-known epistasis models involving a small number of genes and under a large number of limitations. Results Toxo is a MATLAB library designed to calculate penetrance tables of epistasis models of any interaction order which resemble real data more closely. The user specifies the desired heritability (or prevalence) and the program maximizes the table’s prevalence (or heritability) according to the input epistatic model boundaries. Conclusions Toxo extends the capabilities of existing simulators that define epistasis using penetrance tables. These tables can be directly used as input for software simulators such as GAMETES so that they are able to generate data samples with larger interactions and more realistic prevalences/heritabilities., This research was supported by the Ministry of Economy and Competitiveness of Spain (CGL2016-75482-P), the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (TIN2016-75845-P), the Xunta de Galicia (Grupo de Referencia Competitiva, ED431C2016-037), the Xunta de Galicia and FEDER funds of the EU (Centro de Investigación de Galicia accreditation 2019-2022, ref. ED431G2019/01), Consolidation Program of Competitive Research (ED431C 2017/04) and the FPU Program of the Ministry of Education of Spain (FPU16/01333), Xunta de Galicia; ED431C2016-037, Xunta de Galicia; ED431G2019/01, Xunta de Galicia; ED431C 2017/04
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Supporting multi-resolution out-of-core rendering of massive LiDAR point clouds through non-redundant data structures

  • Deibe, David
  • Amor, Margarita
  • Doallo, Ramón
This is an Accepted Manuscript of an article published by Taylor & Francis in INTERNATIONAL JOURNAL OF GEOGRAPHICAL INFORMATION SCIENCE on 28 Nov 2018, available at: https://doi.org/10.1080/13658816.2018.1549734, [Abstract]: In recent years, the evolution and improvement of LiDAR (Light Detection and Ranging) hardware has increased the quality and quantity of the gathered data, making the storage, processing and management thereof particularly challenging. In this work we present a novel, multi-resolution, out-of-core technique, used for web-based visualization and implemented through a non-redundant, data point organization method, which we call Hierarchically Layered Tiles (HLT), and a tree-like structure called Tile Grid Partitioning Tree (TGPT). The design of these elements is mainly focused on attaining very low levels of memory consumption, disk storage usage and network traffic on both, client and server-side, while delivering high-performance interactive visualization of massive LiDAR point clouds (up to 28 billion points) on multiplatform environments (mobile devices or desktop computers). HLT and TGPT were incorporated and tested in ViLMA (Visualization for LiDAR data using a Multi-resolution Approach), our own web-based visualization software specially designed to work with massive LiDAR point clouds., This research was supported by Xunta de Galicia under the Consolidation Programme of Competitive Reference Groups, co-founded by ERDF funds from the EU [Ref. ED431C 2017/04]; Consolidation Programme of Competitive Research Units, co-founded by ERDF funds from the EU [Ref. R2016/037]; Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016/2019) and the European Union (European Regional Development Fund, ERDF) under Grant [Ref. ED431G/01]; and the Ministry of Economy and Competitiveness of Spain and ERDF funds from the EU [TIN2016-75845-P]., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; R2016/037, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

An automatic optimizer for heterogeneous devices

  • Fernández-Fabeiro, Jorge
  • Andrade, Diego
  • Fraguela, Basilio B.
  • Doallo, Ramón
Versión final aceptada de: https://doi.org/10.1016/j.future.2020.01.018, This version of the article: Fernández-Fabeiro, J., Andrade, D., Fraguela, B. B., & Doallo, R. (2020). 'An automaticoptimizer for heterogeneous devices' has been accepted for publication in: Future Generation Computer Systems, 106, 572–584. The Version of Record is available online at: https://doi.org/10.1016/j.future.2020.01.018 ., [Abstract]: Codes written in a naive way seldom effectively exploit the computing resources, while writing optimized codes is usually a complex task that requires certain levels of expertise. This problem is further increased in the presence of heterogeneous devices, which present more tunable parameters than regular CPUs and high sensitivity to the optimization decisions taken. Furthermore, portability is an added concern given the wide variety of accelerators available. This paper tackles this problem adding an automatic optimizer to a library that already provides an easy and portable way to program heterogeneous devices, the Heterogeneous Programming Library (HPL). Our optimizer takes as input a simple version of a code and then tunes it for the device where it is going to be executed by performing the most usual set of optimizations applicable in heterogeneous devices. These optimizations are parametrized using a set of optimization parameters that need to be tuned for the device. The HPL library has also been equipped with an autotuner that can be used to this purpose. The effectiveness of the autotuner and the optimizer has been tested on several codes and devices. The results show that the combination of the autotuner and the optimizer make the tested codes 16 times faster on average than the original codes written by the programmer., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds (80%) of the EU (TIN2016-75845-P), and by the Government of Galicia (Xunta de Galicia, Spain) co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2017/04) as well as under Xunta de Galicia and FEDER funds of the EU (Centro de Investigación de Galicia accreditation 2019–2022, ref. ED431G2019/01), Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G2019/01
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Enabling Hardware Affinity in JVM-Based Applications: A Case Study for Big Data

  • Expósito, Roberto R.
  • Veiga, Jorge
  • Touriño, Juan
This version of the article has been accepted for publication, after peer review and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-030-50371-0_3., [Abstract]: Java has been the backbone of Big Data processing for more than a decade due to its interesting features such as object orientation, cross-platform portability and good programming productivity. In fact, most popular Big Data frameworks such as Hadoop and Spark are implemented in Java or using other languages designed to run on the Java Virtual Machine (JVM) such as Scala. However, modern computing hardware is increasingly complex, featuring multiple processing cores aggregated into one or more CPUs that are usually organized as a Non-Uniform Memory Access (NUMA) architecture. The platform-independent features of the JVM come at the cost of hardware abstraction, which makes it more difficult for Big Data developers to take advantage of hardware-aware optimizations based on managing CPU or NUMA affinities. In this paper we introduce jhwloc, a Java library for easily managing such affinities in JVM-based applications and gathering information about the underlying hardware topology. To demonstrate the functionality and benefits of our proposal, we have extended Flame-MR, our Java-based MapReduce framework, to provide support for setting CPU affinities through jhwloc. The experimental evaluation using representative Big Data workloads has shown that performance can be improved by up to 17% when efficiently exploiting the hardware. jhwloc is publicly available to download at https://github.com/rreye/jhwloc., This work was supported by the Ministry of Economy, Industry and Competitiveness of Spain and FEDER funds of the European Union [ref. TIN2016-75845-P (AEI/FEDER/EU)]; and by Xunta de Galicia and FEDER funds [Centro de Investigación de Galicia accreditation 2019-2022, ref. ED431G2019/01, Consolidation Program of Competitive Reference Groups, ref. ED431C2017/04]., Xunta de Galicia; ED431G2019/01, Xunta de Galicia; ED431C2017/04
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Spark Implementation of the Enhanced Scatter Search Metaheuristic: Methodology and Assessment

  • Pardo, Xoán C.
  • Argüeso-Alejandro, Pablo
  • González, Patricia
  • Banga, Julio R.
  • Doallo, Ramón
This ACCEPTED VERSION of the article: X.C. Pardo, P. Argüeso-Alejandro, P. González, J.R. Banga, R. Doallo (2020). 'Spark Implementation of the Enhanced Scatter Search Metaheuristic: Methodology and Assessment’ has been accepted for publication in: Swarm and Evolutionary Computation (ISSN 2210-6502), 59:100748. The Version of Record is available online at https://doi.org/10.1016/j.swevo.2020.100748., [Abstract] Optimization problems arise nowadays in all disciplines, not only in the scientific area but also in the field of engineering or economics, and in many others. Currently, challenging optimization problems require solution methods that consume a significant amount of computational resources. The application of High-Performance Computing techniques is a common approach to obtain efficient implementations in traditional parallel computing systems. However, more recent approaches are exploring distributed programming frameworks developed in recent years to achieve efficient computations on clusters and cloud systems. In this paper we present a parallel implementation of the enhanced Scatter Search metaheuristic using Spark. The parallel program was obtained as a particularization of a general software framework we developed to support different realisations of the Scatter Search metaheuristic. The aim of this paper is to provide helpful guidance to readers interested in applying, or developing their own, parallel metaheuristics to solve challenging problems in the Cloud. With the twofold objective of demonstrating the potential of the parallelization with Spark and also of studying the factors that influence the performance of the solution, the proposal has been thoroughly evaluated on two different platforms, a cluster and a cloud platform, using a representative set of parameter estimation problems in the field of Computational Systems Biology., This work was supported by the Ministry of Science and Innovation of Spain (DPI2017-82896-C2-2-R, TIN2016-75845-P and PID2019-104184RB-I00, AEI/FEDER/UE, 10.13039/501100011033); and by Xunta de Galicia and FEDER funds (Centro de Investigación de Galicia accreditation 2019-2022, ref. ED431G 2019/01, as well as Consolidation Program of Competitive Reference Groups, ref. ED431C 2017/04), Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2017/04
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico antes de ser publicado, versión del autor (Preprint)

A Portable and Adaptable Fault Tolerance Solution for Heterogeneous Applications

  • Losada, Nuria
  • Fraguela, Basilio B.
  • González, Patricia
  • Martín, María J.
[Abstract] Heterogeneous systems have increased their popularity in recent years due to the high performance and reduced energy consumption capabilities provided by using devices such as GPUs or Xeon Phi accelerators. This paper proposes a checkpoint-based fault tolerance solution for heterogeneous applications, allowing them to survive fail-stop failures in the host CPU or in any of the accelerators used. Besides, applications can be restarted changing the host CPU and/or the accelerator device architecture, and adapting the computation to the number of devices available during recovery. The proposed solution is built combining CPPC (ComPiler for Portable Checkpointing), an application-level checkpointing tool, and HPL (Heterogeneous Programming Library), a library that facilitates the development of OpenCL-based applications. Experimental results show the low overhead introduced by the proposal and prove its portability and adaptability benefits., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (Projects TIN2013-42148-P, TIN2016-75845-P and the predoctoral Grant of Nuria Losada Ref. BES-2014-068066), by EU under the COST Program Action IC1305, Network for Sustainable Ultrascale Computing (NESUS), and by the Galician Government (Xunta de Galicia) and FEDER funds of the EU under the Consolidation Program of Competitive Research (Ref. GRC2013/055), Xunta de Galicia; GRC 2013/055
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Accelerating binary biclustering on platforms with CUDA-enabled GPUs

  • González-Domínguez, Jorge
  • Expósito, Roberto R.
© 2018 Elsevier B.V. All rights reserved. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/bync-nd/4.0/. This version of the article has been accepted for publication in Information Sciences. The Version of Record is available online at https://doi.org/10.1016/j.ins.2018.05.025, This is a version of: J. González-Domínguez and R. R. Expósito, "Accelerating binary biclustering on platforms with CUDA-enabled GPUs", Information Sciences, Vol. 496, Sept. 2019, pp. 317-325, https://doi.org/10.1016/j.ins.2018.05.025, [Abstract]: Data mining is nowadays essential in many scientific fields to extract valuable information from large input datasets and transform it into an understandable structure. For instance, biclustering techniques are very useful in identifying subsets of two-dimensional data where both rows and columns are correlated. However, some biclustering techniques have become extremely time-consuming when processing very large datasets, which nowadays prevents their use in many areas of research and industry (such as bioinformatics) that have experienced an explosive growth on the amount of available data. In this work we present CUBiBit, a tool that accelerates the search for relevant biclusters on binary data by exploiting the computational capabilities of CUDA-enabled GPUs as well as the several CPU cores available in most current systems. The experimental evaluation has shown that CUBiBit is up to 116 times faster than the fastest state-of-the-art tool, BiBit, in a system with two Intel Sandy Bridge processors (16 CPU cores) and three NVIDIA K20 GPUs. CUBiBit is publicly available to download from https://sourceforge.net/projects/cubibit, This work was supported by the Ministry of Economy, Industry and Competitiveness of Spain and FEDER funds of the European Union [grant TIN2016-75845-P (AEI/FEDER/UE)], as well as by Xunta de Galicia (Centro Singular de Investigacion de Galicia accreditation 2016-2019, ref. EDG431G/01)., Xunta de Galicia; EDG431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

ScalaParBiBit: Scaling the Binary Biclustering in Distributed-Memory Systems

  • Fraguela, Basilio B.
  • Andrade, Diego
  • González-Domínguez, Jorge
[Abstract] Biclustering is a data mining technique that allows us to find groups of rows and columns that are highly correlated in a 2D dataset. Although there exist several software applications to perform biclustering, most of them suffer from a high computational complexity which prevents their use in large datasets. In this work we present ScalaParBiBit, a parallel tool to find biclusters on binary data, quite common in many research fields such as text mining, marketing or bioinformatics. ScalaParBiBit takes advantage of the special characteristics of these binary datasets, as well as of an efficient parallel implementation and algorithm, to accelerate the biclustering procedure in distributed-memory systems. The experimental evaluation proves that our tool is significantly faster and more scalable that the state-of-the-art tool ParBiBit in a cluster with 32 nodes and 768 cores. Our tool together with its reference manual are freely available at https://github.com/fraguela/ScalaParBiBit., This research was supported by the Ministry of Science and Innovation of Spain (TIN2016-75845-P and PID2019-104184RB-I00, AEI/FEDER/EU, 10.13039/ 501100011033), and by the Xunta de Galicia co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2017/04). We acknowledge also the support from the Centro Singular de Investigación de Galicia “CITIC”, funded by Xunta de Galicia and the European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant ED431G 2019/01. We also acknowledge the Centro de Supercomputación de Galicia (CESGA) for the usage of their resources, Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Big Data Geospatial Processing for Massive Aerial LiDAR Datasets

  • Deibe, David
  • Amor, Margarita
  • Doallo, Ramón
[Abstract] For years, Light Detection and Ranging (LiDAR) technology has been considered as a challenge when it comes to developing efficient software to handle the extremely large volumes of data this surveying method is able to collect. In contexts such as this, big data technologies have been providing powerful solutions for distributed storage and computing. In this work, a big data approach on geospatial processing for massive aerial LiDAR point clouds is presented. By using Cassandra and Spark, our proposal is intended to support the execution of any kind of heavy time-consuming process; nonetheless, as an initial case of study, we have focused on fast ground-only rasters obtention to generate digital terrain models (DTMs) from massive LiDAR datasets. Filtered clouds obtained from the isolated processing of adjacent zones may exhibit errors located on the boundaries of the zones in the form of misclassified points. Usually, this type of error is corrected through manual or semi-automatic procedures. In this work, we also present an automated strategy for correcting errors of this type, improving the quality of the classification process and the DTMs obtained while minimizing user intervention. The autonomous nature of all computing stages, along with the low processing times achieved, opens the possibility of considering the system as a highly scalable service-oriented solution for on-demand DTM generation or any other geospatial process. Said solution would be a highly useful and unique service for many users in the LiDAR field, and one which could get near to real-time processing with appropriate computational resources., This research was supported by the Government of Galicia (Xunta de Galicia) under the Consolidation Programme of Competitive Reference Groups, co-founded by ERDF funds from the EU [Ref. ED431C 2017/04]; under the Consolidation Programme of Competitive Research Units, co-founded by ERDF funds from the EU [Ref. R2016/037]; by Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016/2019) and the European Union (European Regional Development Fund, ERDF) under [Grant Ref. ED431G/01]; and by the Ministry of Economy and Competitiveness of Spain and ERDF funds from the EU [TIN2016-75845-P], Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

A Fast Solver for Large Tridiagonal Systems on Multi-Core Processors (Lass Library)

  • Valero-Lara, Pedro
  • Andrade, Diego
  • Sirvent, Raül
  • Labarta, Jesús
  • Fraguela, Basilio B.
  • Doallo, Ramón
[Abstract]: Many problems of industrial and scientific interest require the solving of tridiagonal linear systems. This paper presents several implementations for the parallel solving of large tridiagonal systems on multi-core architectures, using the OmpSs programming model. The strategy used for the parallelization is based on the combination of two different existing algorithms, PCR and Thomas. The Thomas algorithm, which cannot be parallelized, requires the fewest number of floating point operations. The PCR algorithm is the most popular parallel method, but it is more computationally expensive than Thomas. The method proposed in this paper starts applying the PCR algorithm to break down one large tridiagonal system into a set of smaller and independent ones. In a second step, these independent systems are concurrently solved using Thomas. The paper also contains an analytical study of which is the best point to switch from PCR to Thomas. Also, the paper addresses the main performance issues of combining PCR and Thomas proposing a set of alternative implementations, some of them even imply algorithmic changes. The performance evaluation shows that the best implementation achieves a peak speedup of 4 with respect to the Intel MKL counterpart routine and 2.5 with respect to a single-threaded Thomas., This work was supported in part by the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreements Human Brain Project SGA1 and Human Brain Project SGA2 under Grant 720270 and Grant 785907, in part by the Spanish Ministry of Economy and Competitiveness under the Project Computación de Altas Prestaciones VII under Grant TIN2015-65316-P, in part by the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya, under project MPEXPAR: Models de Programació i Entorns d’Execució Paralůlels under Grant 2014-SGR-1051, in part by the Juan de la Cierva under Grant IJCI-2017-33511, in part by the Fujitsu under the Barcelona Supercomputing Center-Fujitsu Joint Project: Math Libraries Migration and Optimization, in part by the Ministerio de Economía, Industria y Competitividad of Spain, in part by the Fondo Europeo de Desarrollo Regional Funds of the European Union under Grant TIN2016-75845-P, and in part by the Xunta de Galicia co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups under Grant ED431C 2017/04, and in part by the Centro Singular de Investigación de Galicia accreditatión 2016-2019 under Grant ED431G/01., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01, Generalitat de Catalunya; 2014-SGR-1051
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Hardware Implementation of Statecharts for FPGA-based Control in Scientific Facilities

  • Cereijo García, Javier
  • Osorio, Roberto
Date of Conference: 20-22 Nov. 2019; Conference Location: Bilbao, Spain, [Abstract] The problem of generating complex synchronization patterns using automated tools is addressed in this paper. This work was originally motivated by the need of fast and jitter free synchronization in scientific facilities, where a large number of sensors and actuators must be controlled at the right time in a variety of situations. Programmable processors cannot meet the real-time requirements, forcing to use dedicated circuits to produce and transmit the control signals. Designing application specific hardware by hand is a slow and error-prone task. Hence, a set of tools is required that allow specifying the control systems in a clear and efficient way and producing synthesizable HDL (hardware description language) code in an automated manner. Statechart diagrams have been selected as the input method, and this work focuses on how to translate those diagrams into HDL code. We present a tool that analyzes a Statecharts specification and implements the required control systems using FPGAs. A number of solutions are provided to deal with multiple triggering events and concurrent super-states. Also, an alternative microprogrammed implementation is proposed., This work was funded in part by the Ministry of Economy and Competitiveness of Spain, Project TIN2016-75845-P (AEI/FEDER, UE), Xunta de Galicia and FEDER funds of the EU under the Consolidation Program of Competitive Reference Groups (ED431C 2017/04), and under the Centro Singular de Investigaci ´on de Galicia accreditation 2016-2019 (ED431G/01), Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

CUDA-JMI: Acceleration of feature selection on heterogeneous systems

  • González-Domínguez, Jorge
  • Expósito, Roberto R.
  • Bolón-Canedo, Verónica
©2019 Elsevier B.V. All rights reserved. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/bync-nd/4.0/. This version of the article has been accepted for publication in Future Generation Computer Systems. The Version of Record is available online at https://doi.org/10.1016/j.future.2019.08.031, Versión final aceptada de: J. González-Domínguez, R. R. Expósito, and V. Bolón-Canedo, "CUDA-JMI: Acceleration of feature selection on heterogeneous systemss", Future Generation Computer Systems, Vol. 102, pp. 426-436, Jan. 2020, https://doi.org/10.1016/j.future.2019.08.031, [Abstract]: Feature selection is a crucial step nowadays in machine learning and data analytics to remove irrelevant and redundant characteristics and thus to provide fast and reliable analyses. Many research works have focused on developing new methods that increase the global relevance of the subset of selected features while reducing the redundancy of information. However, those methods that select features with high relevance and low redundancy are extremely time-consuming when processing large datasets. In this work we present CUDA-JMI, a tool based on Joint Mutual Information that accelerates feature selection by exploiting the computational capabilities of modern heterogeneous systems that contain several CPU cores and GPU devices. The experimental evaluation has been carried out in three systems with different type and amount of CPUs and GPUs using five publicly available datasets from different fields. These results show that CUDA-JMI is significantly faster than its original sequential counterpart for all systems and input datasets. For instance, the runtime of CUDA-JMI is up to 52 times faster than an existing sequential JMI-based implementation in a machine with 24 CPU cores and two NVIDIA M60 boards (four GPUs). CUDA-JMI is publicly available to download from https://sourceforge.net/projects/cuda-jmi, This research has been partially funded by projects TIN2016-75845-P and TIN-2015-65069-C2-1-R of the Ministry of Economy, Industry and Competitiveness of Spain, as well as by Xunta de Galicia, Spain projects ED431D R2016/045, ED431G/01 and GRC2014/035, all of them partially funded by FEDER, Spain funds of the European Union., Xunta de Galicia; ED431D R2016/045, Xunta de Galicia; ED431G/01, Xunta de Galicia; GRC2014/035
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Real-time resource scaling platform for Big Data workloads on serverless environments

  • Enes, Jonatan
  • Expósito, Roberto R.
  • Touriño, Juan
Versión final aceptada de: https://doi.org/10.1016/j.future.2019.11.037, This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-ncnd/ 4.0/. This version of the article: Enes, J., Expósito, R. R., & Touriño, J. (2020). 'Real-time resource scaling platform for Big Data workloads on serverless environments', has been accepted for publication in.: Future Generation Computer Systems, 105, 361–379. The Version of Record is available online at: https://doi.org/10.1016/j.future.2019.11.037., The serverless execution paradigm is becoming an increasingly popular option when workloads are to be deployed in an abstracted way, more specifically, without specifying any infrastructure requirements. Currently, such workloads are typically comprised of small programs or even a series of single functions used as event triggers or to process a data stream. Other applications that may also fit on a serverless scenario are stateless services that may need to seamlessly scale in terms of resources, such as a web server. Although several commercial serverless services are available (e.g., Amazon Lambda), their use cases are mostly limited to the execution of functions or scripts that can be adapted to predefined templates or specifications. However, current research efforts point out that it is interesting for the serverless paradigm to evolve from single functions and support more flexible infrastructure units such as operating-system-level virtualization in the form of containers. In this paper we present a novel platform to automatically scale container resources in real time, while they are running, and without any need for reboots. This platform is evaluated using Big Data workloads, both batch and streaming, as representative examples of applications that could be initially regarded as unsuitable for the serverless paradigm considering the currently available services. The results show how our serverless platform can improve the CPU utilization by up to 77% with an execution time overhead of only 6%, while remaining scalable when using a 32-container cluster., This work was supported by the Ministry of Economy, Industry and Competitiveness of Spain and FEDER funds of the European Union (project TIN2016-75845-P, AEI/FEDER/EU), the FPU Program of the Ministry of Education, Spain (grant FPU15/03381) and by Xunta de Galicia, Spain (Centro Singular de Investigación de Galicia accreditation 2016–2019, ref. ED431G/01). We also gratefully acknowledge CESGA for providing access to the Big Data infrastructure, and also sincerely thank Dr. Javier López Cacheiro for his technical support to perform some of the experiments. Other experiments presented in this paper were carried out using the Grid’5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several universities as well as other organizations., Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Accelerating the HyperLogLog Cardinality Estimation Algorithm

  • Bozkus, Cem
  • Fraguela, Basilio B.
[Abstract] In recent years, vast amounts of data of different kinds, from pictures and videos from our cameras to software logs from sensor networks and Internet routers operating day and night, are being generated. This has led to new big data problems, which require new algorithms to handle these large volumes of data and as a result are very computationally demanding because of the volumes to process. In this paper, we parallelize one of these new algorithms, namely, the HyperLogLog algorithm, which estimates the number of different items in a large data set with minimal memory usage, as it lowers the typical memory usage of this type of calculation from 𝑂(𝑛) to 𝑂(1). We have implemented parallelizations based on OpenMP and OpenCL and evaluated them in a standard multicore system, an Intel Xeon Phi, and two GPUs from different vendors. The results obtained in our experiments, in which we reach a speedup of 88.6 with respect to an optimized sequential implementation, are very positive, particularly taking into account the need to run this kind of algorithm on large amounts of data., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds (80%) of the EU (Projects TIN2013-42148-P and TIN2016-75845-P) as well as by the Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016–2019) and the European Union (European Regional Development Fund, ERDF) under Grant Ref. ED431G/01, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Efficient Solving of Scan Primitive on Multi-GPU Systems

  • Pérez Diéguez, Adrián
  • Amor, Margarita
  • Doallo, Ramón
  • Nukada, Akira
  • Matsuoka, Satoshi
This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/IPDPS.2018.00089, Presented at: 32nd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2018, Vancouver, 21-25 May 2018, [Abstract]: GPUs fulfill high computation demands, but it is necessary to develop code carefully, selecting algorithms well suited to the GPU architecture and applying different optimizations. This article presents a GPU-suitable algorithm and a tuning strategy for performing the scan primitive over large problem sizes in CUDA. This tuning strategy defines different performance premises to find the GPU execution parameters that maximize performance. Taking these premises into consideration, we easily develop the kernels using CUDA skeletons to ensure efficiency and portability. Based on this, we describe an optimal proposal analyzed over different multiple GPU environments, the first multiple-GPU batch scan proposal to the best of our knowledge. The resulting implementations outperform other well-known libraries in most cases, such as CUDPP, ModernGPU, Thrust, CUB and LightScan., This work was cofunded by the Government of Galicia and ERDF funds from the EU, under the Consolidation Programme of Competitive Reference Groups [ED431C 2017/04] and Competitive Research Units [R2014/049 and R2016/037]; by the Ministry of Economy and Competitiveness of Spain and ERDF funds [TIN2016-75845-P]; and by the Ministry of Education of Spain (FPU14/02801)., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; R2014/049, Xunta de Galicia; R2016/037
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Simulating the Network Activity of Modern Manycores

  • Horro, M.
  • Rodríguez, Gabriel
  • Touriño, Juan
[Abstract]: Manycore architectures are one of the most promising candidates to reach the exascale. However, the increase in the number of cores on a single die exacerbates the memory wall problem. Modern manycore architectures integrate increasingly complex and heterogeneous memory systems to work around the memory bottleneck while increasing computational power. The Intel Mesh Interconnect architecture is the latest interconnect designed by Intel for its HPC product lines. Processors are organized in a rectangular network-on-chip (NoC), connected to several different memory interfaces, and using a distributed directory to guarantee coherent memory accesses. Since the traffic on the NoC is completely opaque to the programmer, simulation tools are needed to understand the performance trade-offs of code optimizations. Recently featured in Intel's Xeon Scalable lines, this interconnect was first included in the Knights Landing (KNL), a manycore processor with up to 72 cores. This work analyzes the behavior of the Intel Mesh Interconnect through the KNL architecture, proposing ways to discover the physical layout of its logical components. We have designed and developed an extension to the Tejas memory system simulator to replicate and study the low-level data traffic of the processor network. The reliability and accuracy of the proposed simulator is assessed using several state-of-the-art sequential and parallel benchmarks, and a particular Intel Mesh Interconnect-focused locality optimization is proposed and studied using the simulator and a real KNL system., This research was supported by the Ministry of Economy, Industry and Competitiveness of Spain, Project TIN2016-75845-P (AEI/FEDER/EU) and by the Ministry of Education under Grant FPU16/00816.
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Parallel prefix operations on GPU: tridiagonal system solvers and scan operators

  • Pérez Diéguez, Adrián
  • Amor, Margarita
  • Doallo, Ramón
This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/s11227-018-2676-z, [Abstract]: Modern GPUs can achieve high computing power at low cost, but still requires much time and effort. Tridiagonal system and scan solvers are one example of widely used algorithms which can take advantage of these devices. In this article, one tridiagonal system solver and two scan primitive operators are implemented on CUDA GPUs. To do so, a tuning strategy based on three phases is developed. Additionally, a performance analysis is performed for two different CUDA GPU architectures, resulting in a huge improvement with respect to the state of the art., This work is supported by the Ministry of Economy and Competitiveness of Spain, TIN2016-75845-P (AEI/FEDER, UE), by the Galician Government and FEDER funds under the Consolidation Program of Competitive Reference Groups (GRC2013-055) as well as under the Consolidation Programme of Competitive Research Units [Ref. R2014/049 and Ref. R2016/037]; and by the FPU Program of the Ministry of Education of Spain (FPU14/02801)., Xunta de Galicia; GRC2013/055, Xunta de Galicia; R2014/049, Xunta de Galicia; R2016/037
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Comparison of Hardwired and Microprogrammed Statechart Implementations

  • Cereijo García, Javier
  • Osorio, Roberto
[Abstract]: In scientific facilities such as particle accelerators, fast and jitter-free synchronization is required in order to trigger a large number of actuators at the right time in a variety of situations. The behaviour of the control systems and subsystems may be specified by using statechart diagrams, which expand the capabilities of finite state machines allowing concurrency, a hierarchy of states, and history. Hence, there is a need of tools for synthesizing those diagrams so that a new control configuration may be deployed in a short time and an error-free manner in the required environments. In this work, we present a tool that analyses the specification of a variant of the State Chart XML (SCXML) standard tailored to hardware systems and produces a hardware description language (HDL) code suited to implement the required control systems using FPGAs. A number of solutions are provided to deal with the specific features of statecharts, such as multiple triggering events and concurrent super-states. We also present a microprogrammed architecture able to implement statecharts defined as firmware. Finally, we compare the advantages of each strategy in terms of usability, resource usage, and performance, and their applicability to a specific facility is evaluated., This research was funded in part by the Ministry of Science and Innovation of Spain (project TIN2016-75845-P, AEI/FEDER/EU), Xunta de Galicia and FEDER funds of the EU under the Consolidation Program of Competitive Reference Groups (ED431C 2017/04), and under the Centro de Investigación de Galicia accreditation 2019-2022 (ED431G 2019/01)., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

SMusket: Spark-based DNA error correction on distributed-memory systems

  • Expósito, Roberto R.
  • González-Domínguez, Jorge
  • Touriño, Juan
©2020 Elsevier B.V. All rights reserved. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/bync-nd/4.0/. This version of the article has been accepted for publication in Future Generation Computer Systems. The Version of Record is available online at https://doi.org/10.1016/j.future.2019.10.038, This is the accepted version of: R. R. Expósito, J. González-Domínguez, and J. Touriño, "SMusket: Sparkbased DNA error correction on distributed-memory systems", Future Generation Computer Systems, vol. 111, pp. 698-713, 2020, https://doi.org/10.1016/j.future.2019.10.038, [Abstract]: Next-Generation Sequencing (NGS) technologies have revolutionized genomics research over the last decade, bringing new opportunities for scientists to perform groundbreaking biological studies. Error correction in NGS datasets is considered an important preprocessing step in many workflows as sequencing errors can severely affect the quality of downstream analysis. Although current error correction approaches provide reasonably high accuracies, their computational cost can be still unacceptable when processing large datasets. In this paper we propose SparkMusket (SMusket), a Big Data tool built upon the open-source Apache Spark cluster computing framework to boost the performance of Musket, one of the most widely adopted and top-performing multithreaded correctors. Our tool efficiently exploits Spark features to implement a scalable error correction algorithm intended for distributed-memory systems built using commodity hardware. The experimental evaluation on a 16-node cluster using four publicly available datasets has shown that SMusket is up to 15.3 times faster than previous state-of-the-art MPI-based tools, also providing a maximum speedup of 29.8 over its multithreaded counterpart. SMusket is publicly available under an open-source license at https://github.com/rreye/smusket, This work was supported by the Ministry of Economy, Industry and Competitiveness of Spain and FEDER, Spain funds of the European Union (project TIN2016-75845-P, AEI/FEDER/EU); and by Xunta de Galicia, Spain (projects ED431G/01 and ED431C 2017/04)., Xunta de galicia; ED431G/01, Xunta de Galicia; ED431C 2017/04
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Portable and efficient FFT and DCT algorithms with the Heterogeneous Butterfly Processing Library

  • Vázquez Pardo, Sergio
  • Amor, Margarita
  • Fraguela, Basilio B.
Versión final aceptada de: https://doi.org/10.1016/j.jpdc.2018.11.011, This version of the article: Vázquez, S., Amor, M., Fraguela, B. B. (2019). 'Portable and efficient FFT and DCT algorithms with the heterogeneous butterfly processing library', has been accepted for publication in Journal of Parallel and Distributed Computing, 125, 135–146. The Version of Record is available online at https://doi.org/10.1016/j.jpdc.2018.11.011., [Abstract]: The existence of a wide variety of computing devices with very different properties makes essential the development of software that is not only portable among them, but which also adapts to the properties of each platform. In this paper, we present the Heterogeneous Butterfly Processing Library (HBPL), which provides optimized portable kernels for problems of small sizes that allow using orthogonal transform algorithms such as the FFT and DCT on different accelerators and regular CPUs. Our library is implemented on the OpenCL standard, which provides portability on a large number of platforms. Furthermore, high performance is achieved on a wide range of devices by exploiting run-time code generation and metaprogramming guided by a parametrization strategy. An exhaustive evaluation on different platforms shows that our proposal obtains competitive or better performance than related libraries., This research has received financial support from the Ministerio de Economía y Competitividad of Spain and European Regional Development Fund (ERDF) funds (80%) of the EU (TIN2016-75845-P), by the Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia co-founded by European Regional Development Fund (ERDF) funds under the Consolidation Programme of Competitive Reference Groups (Ref. ED431C 2017/04) and the Consolidation Programme of Competitive Research Units (Ref. R2014/049 and Ref. R2016/037) as well as by the Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016–2019) and the European Union (European Regional Development Fund, ERDF) under Grant Ref. ED431G/01., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01, Xunta de Galicia; R2014/049, Xunta de Galicia; R2016/037
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Easy Dataflow Programming in Clusters with UPC++ DepSpawn

  • Fraguela, Basilio B.
  • Andrade, Diego
Versión final aceptada de: https://doi.org/10.1109/TPDS.2018.2884716, This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/TPDS.2018.2884716, [Abstract]: The Partitioned Global Address Space (PGAS) programming model is one of the most relevant proposals to improve the ability of developers to exploit distributed memory systems. However, despite its important advantages with respect to the traditional message-passing paradigm, PGAS has not been yet widely adopted. We think that PGAS libraries are more promising than languages because they avoid the requirement to (re)write the applications using them, with the implied uncertainties related to portability and interoperability with the vast amount of APIs and libraries that exist for widespread languages. Nevertheless, the need to embed these libraries within a host language can limit their expressiveness and very useful features can be missing. This paper contributes to the advance of PGAS by enabling the simple development of arbitrarily complex task-parallel codes following a dataflow approach on top of the PGAS UPC++ library, implemented in C++. In addition, our proposal, called UPC++ DepSpawn, relies on an optimized multithreaded runtime that provides very competitive performance, as our experimental evaluation shows., This research was supported by the Ministerio de Economía, Industria y Competitividad of Spain and FEDER funds of the EU (TIN2016-75845-P), and by the Xunta de Galicia co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2017/04) as well as under the Centro Singular de Investigación de Galicia accreditation 2016-2019 (ED431G/01). We also acknowledge the Centro de Supercomputación de Galicia (CESGA) for the use of their computers., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Truncated SIMD Multiplier Architecture for Approximate Computing in Low-Power Programmable Processors

  • Osorio, Roberto
  • Rodríguez, Gabriel
[Abstract]: Approximate computing has been exploited for many years in application-specific architectures. Recently, it has also been proposed for low-power programmable processors. However, this poses some challenges as, in a microprocessor, the energy consumed by fetching and decoding an instruction may be significantly higher than that of the execution itself. Therefore, approximate computing would be advisable only for those instructions, in which the execution stage is significantly expensive in terms of energy consumption. In this paper, we present new architectures for truncated SIMD multipliers able to calculate signed and unsigned products from 8 × 8 to 64× 64 bits. Next, we analyze the precision loss incurred by truncation for all product sizes. We implement accurate and truncated architectures for both scalar and SIMD products and find that truncation allows area savings of up to 27%. The proposed design is experimentally evaluated in different scenarios, showing potential energy savings ranging from 29% to 42%. Finally, this paper analyzes the overall convenience of introducing truncated SIMD architectures with respect to accurate SIMD and scalar architectures., This work was supported in part by the Ministry of Economy and Competitiveness of Spain under Project TIN2016-75845-P (AEI/FEDER, UE), in part by the Xunta de Galicia and FEDER Funds of the EU under the Consolidation Program of Competitive Reference Groups under Grant ED431C 2017/04, and in part by the Centro Singular de Investigación de Galicia Accreditation 2016 2019 under Grant ED431G/01., Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico antes de ser publicado, versión del autor (Preprint)

High Productivity Multi-device Exploitation with the Heterogeneous Programming Library

  • Viñas Buceta, Moisés
  • Fraguela, Basilio B.
  • Andrade, Diego
  • Doallo, Ramón
[Abstract] Heterogeneous devices require much more work from programmers than traditional CPUs, particularly when there are several of them, as each one has its own memory space. Multidevice applications require to distribute kernel executions and, even worse, arrays portions that must be kept coherent among the di_erent device memories and the host memory. In addition, when devices with di_erent characteristics participate in a computation, optimally distributing the work among them is not trivial. In this paper we extend an existing framework for the programming of accelerators called Heterogeneous Programming Library (HPL) with three kinds of improvements that facilitate these tasks. The _rst two ones are the ability to de_ne subarrays and subkernels, which distribute kernels on di_erent devices. The last one is a convenient extension of the subkernel mechanism to distribute computations among heterogeneous devices seeking the best work balance among them. This last contribution includes two analytical models that have proved to automatically provide very good work distributions. Our experiments also show the large programmability advantages of our approach and the negligible overhead incurred., This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds (80%) of the EU (Projects TIN2013-42148-P and TIN2016-75845-P), by the Galician Government (consolidation program of competitive reference groups GRC2013/055), the EU under the COST Program Action IC1305 and by the Network for Sustainable Ultrascale Computing (NESUS), Xunta de Galicia; GRC2013/055, European Cooperation in Science and Technology; IC1305
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

ParBiBit: Parallel tool for binary biclustering on modern distributed-memory systems

  • González-Domínguez, Jorge
  • Expósito, Roberto R.
[Abstract]: Biclustering techniques are gaining attention in the analysis of large-scale datasets as they identify two-dimensional submatrices where both rows and columns are correlated. In this work we present ParBiBit, a parallel tool to accelerate the search of interesting biclusters on binary datasets, which are very popular on different fields such as genetics, marketing or text mining. It is based on the state-of-the-art sequential Java tool BiBit, which has been proved accurate by several studies, especially on scenarios that result on many large biclusters. ParBiBit uses the same methodology as BiBit (grouping the binary information into patterns) and provides the same results. Nevertheless, our tool significantly improves performance thanks to an efficient implementation based on C++11 that includes support for threads and MPI processes in order to exploit the compute capabilities of modern distributed-memory systems, which provide several multicore CPU nodes interconnected through a network. Our performance evaluation with 18 representative input datasets on two different eight-node systems shows that our tool is significantly faster than the original BiBit. Source code in C++ and MPI running on Linux systems as well as a reference manual are available at https://sourceforge.net/projects/parbibit/., This work was supported by the Ministry of Economy, Industry and Competitiveness of Spain and FEDER funds of the European Union [grant TIN2016-75845-P (AEI/FEDER/UE)], as well as by Xunta de Galicia (Centro Singular de Investigacion de Galicia accreditation 2016-2019, ref. EDG431G/01)., Xunta de Galicia; EDG431G/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Tree Partitioning Reduction: A New Parallel Partition Method for Solving Tridiagonal Systems

  • Pérez Diéguez, Adrián
  • Amor, Margarita
  • Doallo, Ramón
© 2019 Copyright held by the owner/author(s). This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Mathematical Software, https://doi.org/10.1145/3328731, This work was cofunded by the Government of Galicia and ERDF funds from the EU, under the Consolidation Programme of Competitive Reference Groups [ED431C 2017/04], by the Ministry of Economy and Competitiveness of Spain and ERDF funds [TIN2016-75845-P], and by the Ministry of Education of Spain (FPU14/02801). Additionally, it has been also supported by the Xunta de Galicia (Centro Singular de Investigación de Galicia accreditation 2016-2019) and ERDF funds [ED7431G/01], Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED7431G/01
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy

  • Penas, David R.
  • González, Patricia
  • Egea, José A.
  • Doallo, Ramón
  • Banga, Julio R.
[Background]: The development of large-scale kinetic models is one of the current key issues in computational systems biology and bioinformatics. Here we consider the problem of parameter estimation in nonlinear dynamic models. Global optimization methods can be used to solve this type of problems but the associated computational cost is very large. Moreover, many of these methods need the tuning of a number of adjustable search parameters, requiring a number of initial exploratory runs and therefore further increasing the computation times. Here we present a novel parallel method, self-adaptive cooperative enhanced scatter search (saCeSS), to accelerate the solution of this class of problems. The method is based on the scatter search optimization metaheuristic and incorporates several key new mechanisms: (i) asynchronous cooperation between parallel processes, (ii) coarse and fine-grained parallelism, and (iii) self-tuning strategies., [Results]: The performance and robustness of saCeSS is illustrated by solving a set of challenging parameter estimation problems, including medium and large-scale kinetic models of the bacterium E. coli, bakerés yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The results consistently show that saCeSS is a robust and efficient method, allowing very significant reduction of computation times with respect to several previous state of the art methods (from days to minutes, in several cases) even when only a small number of processors is used., [Conclusions]: The new parallel cooperative method presented here allows the solution of medium and large scale parameter estimation problems in reasonable computation times and with small hardware requirements. Further, the method includes self-tuning mechanisms which facilitate its use by non-experts. We believe that this new method can play a key role in the development of large-scale and even whole-cell dynamic models., This research received financial support from the Spanish Ministerio de Economía y Competitividad (and the FEDER) through the “MultiScales” project (subprojects DPI2011-28112-C04-03, DPI2011-28112-C04-04), “SYNBIOFACTORY” (DPI2014-55276-C5-2-R), TIN2013-42148-P and TIN2016- 75845-P, and from the Galician Government under the Consolidation Program of Competitive Research Units (Networks ref. R2014/041 and ref. R2016/045, and competitive reference groups GRC2013/055). DRP acknowledges financial support from the MICINN-FPI programme., We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI)., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards Ad Hoc Recovery for Soft Errors

  • Losada, Nuria
  • Bautista-Gomez, Leonardo
  • Keller, Kai
  • Unsal, Osman
The coming exascale era is a great opportunity for high performance computing (HPC) applications. However, high failure rates on these systems will hazard the successful completion of their execution. Bit-flip errors in dynamic random access memory (DRAM) account for a noticeable share of the failures in supercomputers. Hardware mechanisms, such as error correcting code (ECC), can detect and correct single-bit errors and can detect some multi-bit errors while others can go undiscovered. Unfortunately, detected multi-bit errors will most of the time force the termination of the application and lead to a global restart. Thus, other strategies at the software level are needed to tolerate these type of faults more efficiently and to avoid a global restart. In this work, we extend the FTI checkpointing library to facilitate the implementation of custom recovery strategies for MPI applications, minimizing the overhead introduced when coping with soft errors. The new functionalities are evaluated by implementing local forward recovery on three HPC benchmarks with different reliability requirements. Our results demonstrate a reduction on the recovery times by up to 14%., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 708566 (DURO). This research is also supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (Projects TIN2016-75845-P and the predoctoral grant of Nuria Losada ref. BES-2014-068066), and by the Galician Government (Xunta de Galicia) under the Consolidation Program of Competitive Research (ref. ED431C 2017/04)., Peer Reviewed, Postprint (author's final draft)