Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 26
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publication Capítulo o parte de un libro (BookPart)

Teaching parallel programming in interdisciplinary studies

  • César Galobardes, Eduardo|||0000-0002-9729-8557
  • Cortés Fité, Ana|||0000-0003-1697-1293
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Margalef, Tomàs|||0000-0001-6384-7389
  • Moure, Juan C.|||0000-0001-6697-0331
  • Sikora, Anna|||0000-0003-0090-4109
  • Suppi, Remo|||0000-0002-0373-8292
Nowadays many fields of science and engineering are evolving by the joint contribution of complementary fields. Computer science, and especially high performance computing, has become a key factor in the development of many research fields, establishing a new paradigm called computational science. Researchers and professionals from many different fields require a knowledge of high performance computing, including parallel programming, to develop a fruitful work in their particular field. So, at Universitat Autònoma of Barcelona, an interdisciplinary master on Modeling for science and engineering was started 5 years ago to provide a deep knowledge on the application of modeling and simulation to graduate students on different fields (Mathematics, Physics, Chemistry, Engineering, Geology, etc.). In this master, Parallel Programming appears as a compulsory subject, because it is a key topic for them. The concepts learnt in parallel programming must be applied to real applications. Therefore, a subject on Applied Modelling and Simulation has also been included. In this paper, the experience on teaching parallel programming in such interdisciplinary master is shown. © Springer International Publishing Switzerland 2015. Ministerio de Economía y Competitividad.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Introducing computational thinking, parallel programming and performance engineering in interdisciplinary studies

  • César Galobardes, Eduardo|||0000-0002-9729-8557
  • Cortés Fité, Ana|||0000-0003-1697-1293
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Margalef, Tomàs|||0000-0001-6384-7389
  • Sikora, Anna|||0000-0003-0090-4109
  • Suppi, Remo|||0000-0002-0373-8292
Nowadays, many fields of science and engineering are evolving through the joint contribution of complementary fields. Computer science, and especially High Performance Computing, has become a key factor in the development of many research fields, establishing a new paradigm called computational science. Researchers and professionals from many different fields require knowledge of High Performance Computing, including parallel programming, to develop fruitful and efficient work in their particular field. Therefore, at Universitat Autònoma of Barcelona (Spain), an interdisciplinary Master on "Modeling for Science and Engineering" was started 5 years ago to provide a thorough knowledge of the application of modeling and simulation to graduate students in different fields (Mathematics, Physics, Chemistry, Engineering, Geology, etc.). In this Master's degree, "Parallel Programming" appears as a compulsory subject because it is a key topic for them. The concepts learned in this subject must be applied to real applications. Therefore, a complementary subject on "Applied Modeling and Simulation" has also been included. It is very important to show the students how to analyze their particular problems, think about them from a computational perspective and consider the related performance issues. So, in this paper, the methodology and the experience in introducing computational thinking, parallel programming and performance engineering in this interdisciplinary Master's degree are shown. This overall approach has been refined through the Master's life, leading to excellent academic results and improving the industry and students appraisal of this programme.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

The choice of universal primers and the characteristics of the species mixture determine when DNA metabarcoding can be quantitative

  • Piñol, Josep|||0000-0002-4067-3301
  • Senar Rosell, Miquel|||0000-0002-0316-5420
  • Symondson, William O.C.
DNA metabarcoding is a technique used to survey biodiversity in many ecological settings, but there are doubts about whether it can provide quantitative results, that is, the proportions of each species in the mixture as opposed to a species list. While there are several experimental studies that report quantitative metabarcoding results, there are a similar number that fail to do so. Here, we provide the rationale to understand under what circumstances the technique can be quantitative. In essence, we simulate a mixture of DNA of S species with a defined initial abundance distribution. In the simulated PCR, each species increases its concentration following a certain amplification efficiency. The final DNA concentration will reflect the initial one when the efficiency is similar for all species; otherwise, the initial and final DNA concentrations would be poorly related. Although there are many known factors that modulate amplification efficiency, we focused on the number of primer-template mismatches, arguably the most important one. We used 15 common primers pairs targeting the mitochondrial COI region and the mitogenomes of ca. 1,200 insect species. The results showed that some primers pairs produced quantitative results under most circumstances, whereas some other primers failed to do so. In conclusion, depending on the primer pair used in the PCR amplification and on the characteristics of the mixture analysed (i.e., high species richness, low evenness), DNA metabarcoding can provide a quantitative estimate of the relative abundances of different species.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

A performance comparison of data and memory allocation strategies for sequence aligners on NUMA architectures

  • Lenis, Josefina|||0000-0002-0766-3021
  • Senar Rosell, Miquel Àngel|||0000-0002-0316-5420
Over the last several years, many sequence alignment tools have appeared and become popular for the fast evolution of next generation sequencing technologies. Obviously, researchers that use such tools are interested in getting maximum performance when they execute them in modern infrastructures. Today's NUMA (Non-uniform memory access) architectures present major challenges in getting such applications to achieve good scalability as more processors/cores are used. The memory system in NUMA systems shows a high complexity and may be the main cause for the loss of an application's performance. The existence of several memory banks in NUMA systems implies a logical increase in latency associated with the accesses of a given processor to a remote bank. This phenomenon is usually attenuated by the application of strategies that tend to increase the locality of memory accesses. However, NUMA systems may also suffer from contention problems that can occur when concurrent accesses are concentrated on a reduced number of banks. Sequence alignment tools use large data structures to contain reference genomes to which all reads are aligned. Therefore, these tools are very sensitive to performance problems related to the memory system. The main goal of this study is to explore the trade-offs between data locality and data dispersion in NUMA systems. We have performed experiments with several popular sequence alignment tools on two widely available NUMA systems to assess the performance of different memory allocation policies and data partitioning strategies. We find that there is not one method that is best in all cases. However, we conclude that memory interleaving is the memory allocation strategy that provides the best performance when a large number of processors and memory banks are used. In the case of data partitioning, the best results are usually obtained when the number of partitions used is greater, sometimes combined with an interleave policy.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

From Wet-Lab to Variations: Concordance and Speed of Bioinformatics Pipelines for Whole Genome and Whole Exome Sequencing

  • Laurie, Steven, 1973-
  • Fernández Callejo, Marcos
  • Marco Sola, Santiago
  • Trotta, Jean-Remi
  • Camps-Puchadas, Jordi
  • Chacón, Alejandro
  • Espinosa, Antonio
  • Gut, Marta
  • Gut, Ivo Glynne
  • Heath, Simon
  • Beltran, Sergi
As whole genome sequencing becomes cheaper and faster, it will progressively substitute targeted next-generation sequencing as standard practice in research and diagnostics. However, computing cost–performance ratio is not advancing at an equivalent rate. Therefore, it is essential to evaluate the robustness of the variant detection process taking into account the computing resources required. We have benchmarked six combinations of state-of-the-art read aligners (BWA-MEM and GEM3) and variant callers (FreeBayes, GATK HaplotypeCaller, SAMtools) on whole genome and whole exome sequencing data from the NA12878 human sample. Results have been compared between them and against the NIST Genome in a Bottle (GIAB) variants reference dataset. We report differences in speed of up to 20 times in some steps of the process and have observed that SNV, and to a lesser extent InDel, detection is highly consistent in 70% of the genome. SNV, and especially InDel, detection is less reliable in 20% of the genome, and almost unfeasible in the remaining 10%. These findings will aid in choosing the appropriate tools bearing in mind objectives, workload, and computing infrastructure available., Contract Grant Sponsors: Spanish Ministry of Economy and Competitiveness; Generalitat de Catalunya; European Regional Development Fund (ERDF); RD-Connect Project (EC FP7/2007-2013 #305444); ELIXIR-EXCELERATE (EC H2020 #676559); MICINN (TIN2014-53234- C2-1-R).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Approaching long genomic regions and large recombination rates with msParSm as an alternative to MaCS

  • Montemuiño, Carlos
  • Espinosa, Antonio
  • Moure, Juan C.
  • Vera, Gonzalo
  • Hernández, Porfidio
  • Ramos-Onsins, Sebastian E.
The msParSm application is an evolution of msPar, the parallel version of the coalescent simulation program ms, which removes the limitation for simulating long stretches of DNA sequences with large recombination rates, without compromising the accuracy of the standard coalescence. This work introduces msParSm, describes its significant performance improvements over msPar and its shared memory parallelization details, and shows how it can get better, if not similar, execution times than MaCS. Two case studies with different mutation rates were analyzed, one approximating the human average and the other approximating the Drosophila melanogaster average. Source code is available at https://github.com/cmontemuino/msparsm., This work has been supported by projects (number: AGL2013-41834-R and TIN2014-53234-C2-1-R) of Spanish Ministerio de Ciencia y Tecnologia., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Improving bioinformatics analysis of large sequence datasets parallelizing tools for population genomics

  • Navarro, Javier
  • Vera, Gonzalo
  • Ramos-Onsins, Sebastian E.
  • Hernández, Porfidio
Trabajo presentado al 4th International Workshop on Parallelism in Bioinformatics (euro-Par), celebrado del 22 al 26 de agosto de 2016 en Grenoble, France., Next-generation sequencing (NGS) technologies initiated a revolution in genomics, producing massive amounts of biological data and the consequent need for adapting current computing infrastructures. Multiple alignment of genomes, analysis of variants or phylogenetic tree construction, with quadratic polynomial complexity in the best case are tools that can take days or weeks to complete in conventional computers. Most of these analysis, involving several tools integrated in workflows, present the possibility of dividing the computational load in independent tasks allowing parallel execution. Determining adequate load balancing, data partitioning, granularity and I/O tuning are key factors for achieving suitable speedups. In this paper we present a coarse-grain parallelization of GH caller (Genotype/Haplotype caller), a tool used in population genomics workflows that performs a probabilistic identification process to account for the frequency of variants present between population individuals. It implements a master-worker model, using the standard Message Passing Interface (MPI), and concurrently and iteratively distributes the data among the available worker processes by mapping subsets of data and leaving the orchestration to the master process. Our results show a performance gain factor of 260x using 64 processes and additional optimizations with regard to the initial non-parallelized version., This work was supported by Ministerio de Ciencia y Tecnología (Spain) under project number TIN2014-53234-C2-1-R, and Ministerio de Economía y Competitividad (grant AGL2013-41834-R) to S.E.R.-O., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Optimized Next-Generation Sequencing Genotype-Haplotype Calling for Genome Variability Analysis

  • Navarro, Javier
  • Nevado, Bruno
  • Hernández, Porfidio
  • Vera, Gonzalo
  • Ramos-Onsins, Sebastián E.
The accurate estimation of nucleotide variability using next-generation sequencing data is challenged by the high number of sequencing errors produced by new sequencing technologies, especially for nonmodel species, where reference sequences may not be available and the read depth may be low due to limited budgets. The most popular single-nucleotide polymorphism (SNP) callers are designed to obtain a high SNP recovery and low false discovery rate but are not designed to account appropriately the frequency of the variants. Instead, algorithms designed to account for the frequency of SNPs give precise results for estimating the levels and the patterns of variability. These algorithms are focused on the unbiased estimation of the variability and not on the high recovery of SNPs. Here, we implemented a fast and optimized parallel algorithm that includes the method developed by Roesti et al and Lynch, which estimates the genotype of each individual at each site, considering the possibility to call both bases from the genotype, a single one or none. This algorithm does not consider the reference and therefore is independent of biases related to the reference nucleotide specified. The pipeline starts from a BAM file converted to pileup or mpileup format and the software outputs a FASTA file. The new program not only reduces the running times but also, given the improved use of resources, it allows its usage with smaller computers and large parallel computers, expanding its benefits to a wider range of researchers. The output file can be analyzed using software for population genetics analysis, such as the R library PopGenome, the software VariScan, and the program mstatspop for analysis considering positions with missing data., This work has been supported by Ministerio de Ciencia y Tecnología (Spain) under project number TIN2014-53234-C2-1-R and Ministerio de Economía y Competitividad (grant AGL2016-78709-R). The authors also acknowledge the support of the Spanish Ministry of Economy and Competitivity for the Center of Excellence Severo Ochoa 2016-2019 (SEV-2015-0533) grant awarded to the Center for Research in Agricultural Genomics and by the CERCA Programme/Generalitat de Catalunya and the European Regional Development Fund (ERDF)., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tintó, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel|||0000-0003-1826-623X
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco|||0000-0002-6622-4280
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN201453234C21R, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol
  • Castrillo Melguizo, Miguel
  • Acosta Cobos, Mario César
  • Mula Valls, Josep Oriol
  • Sánchez Lorente, Alícia
  • Serradell Maronda, Kim|||0000-0001-8230-4347
  • Cortés Fité, Ana
  • Doblas Reyes, Francisco
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge., The research leading to these results has received funding from the EU ESiWACE H2020 Framework Programme under grant agreement no. 675191, from the Severo Ochoa (SEV-2011-00067) programme of the Spanish Government and from the Ministerio de Economia y Competitividad under contract TIN2014-53234-C2-1R., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Colorectal tumour simulation using agent based modelling and high performance computing

  • Kang, Guiyeom
  • Márquez, Claudio|||0000-0002-5491-2836
  • Barat, Ana
  • Byrne, Annette T.
  • Prehn, Jochen H.M.
  • Sorribes, Joan
  • César, Eduardo
450,000 European citizens are diagnosed every year with colorectal cancer (CRC) and more than 230,000 succumb to the disease annually. For this reason, significant resources are dedicated to the identification of more effective therapies for this disease. However, classical assessment techniques for these treatments are slow and costly. Consequently, systems biology researchers at the Royal College of Surgeons in Ireland (RCSI) are developing computational agent-based models simulating tumour growth and treatment responses with the objective of speeding up the therapeutic development process while, at the same time, producing a tool for adapting treatments to patient-specific characteristics. However, the model complexity and the high number of agents to be simulated require a thorough optimisation of the process in order to execute realistic simulations of tumour growth on currently available platforms. We propose to apply the most advanced HPC techniques to achieve the efficient and realistic simulation of a virtual tissue model that mimics tumour growth or regression in space and time. These techniques combine extensions of the previously developed agent-based simulation software platform (FLAME) with autotuning capabilities and optimisation strategies for the current tumour model. Development of such a platform could advance the development of novel therapeutic approaches for the treatment of CRC which can also be applied other solid tumours., This work has been partially supported by MICINN-Spain under contract TIN2011- 28689-C02-01 and TIN2014-53234-C2-1-R and GenCat-DIUiE(GRR) 2014-SGR-576. This research was also funded by the European Community’s Framework Programme Seven (FP7) Programme under contract No. 278981 680 AngioPredict and supported by the DJEI/DES/SFI/HEA Irish Centre for High- End Computing (ICHEC)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

From Wet-Lab to Variations: Concordance and Speed of Bioinformatics Pipelines for Whole Genome and Whole Exome Sequencing

  • Laurie, Steven, 1973-
  • Fernández Callejo, Marcos
  • Marco Sola, Santiago
  • Trotta, Jean-Remi
  • Camps-Puchadas, Jordi
  • Chacón, Alejandro
  • Espinosa, Antonio
  • Gut, Marta
  • Gut, Ivo Glynne
  • Heath, Simon
  • Beltran, Sergi
As whole genome sequencing becomes cheaper and faster, it will progressively substitute targeted next-generation sequencing as standard practice in research and diagnostics. However, computing cost–performance ratio is not advancing at an equivalent rate. Therefore, it is essential to evaluate the robustness of the variant detection process taking into account the computing resources required. We have benchmarked six combinations of state-of-the-art read aligners (BWA-MEM and GEM3) and variant callers (FreeBayes, GATK HaplotypeCaller, SAMtools) on whole genome and whole exome sequencing data from the NA12878 human sample. Results have been compared between them and against the NIST Genome in a Bottle (GIAB) variants reference dataset. We report differences in speed of up to 20 times in some steps of the process and have observed that SNV, and to a lesser extent InDel, detection is highly consistent in 70% of the genome. SNV, and especially InDel, detection is less reliable in 20% of the genome, and almost unfeasible in the remaining 10%. These findings will aid in choosing the appropriate tools bearing in mind objectives, workload, and computing infrastructure available., Contract Grant Sponsors: Spanish Ministry of Economy and Competitiveness; Generalitat de Catalunya; European Regional Development Fund (ERDF); RD-Connect Project (EC FP7/2007-2013 #305444); ELIXIR-EXCELERATE (EC H2020 #676559); MICINN (TIN2014-53234- C2-1-R).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Colorectal tumour simulation using agent based modelling and high performance computing

  • Kang, Guiyeom
  • Márquez, Claudio
  • Barat, Ana
  • Byrne, Annette T.
  • Prehn, Jochen H.M.
  • Sorribes, Joan
  • César, Eduardo
450,000 European citizens are diagnosed every year with colorectal cancer (CRC) and more than 230,000 succumb to the disease annually. For this reason, significant resources are dedicated to the identification of more effective therapies for this disease. However, classical assessment techniques for these treatments are slow and costly. Consequently, systems biology researchers at the Royal College of Surgeons in Ireland (RCSI) are developing computational agent-based models simulating tumour growth and treatment responses with the objective of speeding up the therapeutic development process while, at the same time, producing a tool for adapting treatments to patient-specific characteristics. However, the model complexity and the high number of agents to be simulated require a thorough optimisation of the process in order to execute realistic simulations of tumour growth on currently available platforms. We propose to apply the most advanced HPC techniques to achieve the efficient and realistic simulation of a virtual tissue model that mimics tumour growth or regression in space and time. These techniques combine extensions of the previously developed agent-based simulation software platform (FLAME) with autotuning capabilities and optimisation strategies for the current tumour model. Development of such a platform could advance the development of novel therapeutic approaches for the treatment of CRC which can also be applied other solid tumours., This work has been partially supported by MICINN-Spain under contract TIN2011- 28689-C02-01 and TIN2014-53234-C2-1-R and GenCat-DIUiE(GRR) 2014-SGR-576. This research was also funded by the European Community’s Framework Programme Seven (FP7) Programme under contract No. 278981 680 AngioPredict and supported by the DJEI/DES/SFI/HEA Irish Centre for High- End Computing (ICHEC)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Optimizing domain decomposition in an ocean model: the case of NEMO

  • Tintó, Oriol
  • Acosta Cobos, Mario César
  • Castrillo, Miguel
  • Cortés, Ana
  • Sanchez, Alicia
  • Serradell, Kim
  • Doblas-Reyes, Francisco
Earth System Models are critical tools for the study of our climate and its future trends. These models are in constant evolution and their growing complexity entails an incrementing demand of the resources they require. Since the cost of using these state-of-the-art models is huge, looking closely at the factors that are able to impact their computational performance is mandatory. In the case of the state-of-the-art ocean model NEMO (Nucleus for European Modelling of the Ocean), used in many projects around the world, not enough attention has been given to the domain decomposition. In this work we show the impact that the selection of a particular domain decomposition can have on computational performance and how the proposed methodology substantially improves it., The research leading to these results has received funding from the EU H2020 Framework Programme under grant agreement no 675191 and by the Ministerio de Economa y Competitividad (MINECO) under grant TIN201453234C21R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol
  • Castrillo Melguizo, Miguel
  • Acosta Cobos, Mario César
  • Mula Valls, Josep Oriol
  • Sánchez Lorente, Alícia
  • Serradell Maronda, Kim
  • Cortés Fité, Ana
  • Doblas Reyes, Francisco
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge., The research leading to these results has received funding from the EU ESiWACE H2020 Framework Programme under grant agreement no. 675191, from the Severo Ochoa (SEV-2011-00067) programme of the Spanish Government and from the Ministerio de Economia y Competitividad under contract TIN2014-53234-C2-1R., Peer Reviewed, Postprint (published version)
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Bitplane image coding with parallel coefficient processing

  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
  • Enfedaque Montes, Pablo
  • Moure, Juan C.|||0000-0001-6697-0331
  • Sánchez Silva, Víctor Francisco|||0000-0002-7089-7031
Image coding systems have been traditionally tailored for multiple instruction, multiple data (MIMD) computing. In general, they partition the (transformed) image in codeblocks that can be coded in the cores of MIMD-based processors. Each core executes a sequential flow of instructions to process the coefficients in the codeblock, independently and asynchronously from the others cores. Bitplane coding is a common strategy to code such data. Most of its mechanisms require sequential processing of the coefficients. The last years have seen the upraising of processing accelerators with enhanced computational performance and power efficiency whose architecture is mainly based on the single instruction, multiple data (SIMD) principle. SIMD computing refers to the execution of the same instruction to multiple data in a lockstep synchronous way. Unfortunately, current bitplane coding strategies cannot fully profit from such processors due to inherently sequential coding task. This paper presents bitplane image coding with parallel coefficient (BPC-PaCo) processing, a coding method that can process many coefficients within a codeblock in parallel and synchronously. To this end, the scanning order, the context formation, the probability model, and the arithmetic coder of the coding engine have been re-formulated. The experimental results suggest that the penalization in coding performance of BPC-PaCo with respect to the traditional strategies is almost negligible.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

GPU implementation of bitplane coding with parallel coefficient processing for high performance image compression

  • Enfedaque Montes, Pablo
  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
  • Moure, Juan C.|||0000-0001-6697-0331
The fast compression of images is a requisite in many applications like TV production, teleconferencing, or digital cinema. Many of the algorithms employed in current image compression standards are inherently sequential. High performance implementations of such algorithms often require specialized hardware like field integrated gate arrays. Graphics Processing Units (GPUs) do not commonly achieve high performance on these algorithms because they do not exhibit fine-grain parallelism. Our previous work introduced a new core algorithm for wavelet-based image coding systems. It is tailored for massive parallel architectures. It is called bitplane coding with parallel coefficient processing (BPC-PaCo). This paper introduces the first high performance, GPU-based implementation of BPC-PaCo. A detailed analysis of the algorithm aids its implementation in the GPU. The main insights behind the proposed codec are an efficient thread-to-data mapping, a smart memory management, and the use of efficient cooperation mechanisms to enable inter-thread communication. Experimental results indicate that the proposed implementation matches the requirements for high resolution (4 K) digital cinema in real time, yielding speedups of 30x with respect to the fastest implementations of current compression standards. Also, a power consumption evaluation shows that our implementation consumes 40 x less energy for equivalent performance than state-of-the-art methods.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Approaching long genomic regions and large recombination rates with msParSm as an alternative to MaCS

  • Montemuiño Sosa, Carlos
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Moure, Juan C.|||0000-0001-6697-0331
  • Vera Rodríguez, Gonzalo|||0000-0001-6990-0216
  • Hernández Budé, Porfidio|||0000-0002-8592-934X
  • Ramos Onsins, Sebastián Ernesto|||0000-0002-1776-140X
The msParSm application is an evolution of msPar, the parallel version of the coalescent simulation program ms, which removes the limitation for simulating long stretches of DNA sequences with large recombination rates, without compromising the accuracy of the standard coalescence. This work introduces msParSm, describes its significant performance improvements over msPar and its shared memory parallelization details, and shows how it can get better, if not similar, execution times than MaCS. Two case studies with different mutation rates were analyzed, one approximating the human average and the other approximating the Drosophila melanogaster average. Source code is available at https://github.com/cmontemuino/msparsm.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Optimized next-generation sequencing genotype-haplotype calling for genome variability analysis

  • Navarro Fernández, Javier
  • Nevado, Bruno
  • Hernández Budé, Porfidio|||0000-0002-8592-934X
  • Vera Rodríguez, Gonzalo|||0000-0001-6990-0216
  • Ramos Onsins, Sebastián Ernesto|||0000-0002-1776-140X
The accurate estimation of nucleotide variability using next-generation sequencing data is challenged by the high number of sequencing errors produced by new sequencing technologies, especially for nonmodel species, where reference sequences may not be available and the read depth may be low due to limited budgets. The most popular single-nucleotide polymorphism (SNP) callers are designed to obtain a high SNP recovery and low false discovery rate but are not designed to account appropriately the frequency of the variants. Instead, algorithms designed to account for the frequency of SNPs give precise results for estimating the levels and the patterns of variability. These algorithms are focused on the unbiased estimation of the variability and not on the high recovery of SNPs. Here, we implemented a fast and optimized parallel algorithm that includes the method developed by Roesti et al and Lynch, which estimates the genotype of each individual at each site, considering the possibility to call both bases from the genotype, a single one or none. This algorithm does not consider the reference and therefore is independent of biases related to the reference nucleotide specified. The pipeline starts from a BAM file converted to pileup or mpileup format and the software outputs a FASTA file. The new program not only reduces the running times but also, given the improved use of resources, it allows its usage with smaller computers and large parallel computers, expanding its benefits to a wider range of researchers. The output file can be analyzed using software for population genetics analysis, such as the R library PopGenome, the software VariScan, and the program mstatspop for analysis considering positions with missing data.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Coalition structure generation problems, optimization and parallelization of the IDP algorithm in multicore systems

  • Cruz Mencía, Francisco
  • Cerquides Bueno, Jesus|||0000-0002-3752-644X
  • Rodríguez-Aguilar, Juan A.
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Moure, Juan C.|||0000-0001-6697-0331
  • Ramchurn, Sarvapali D.
  • Svensson, Kim
The coalition structure generation problem is well known in the area of multi-agent systems. Its goal is to establish coalitions between agents while maximizing the global welfare. Among the existing different algorithms designed to solve the coalition structure generation problem, DP and IDP are the ones with smaller temporal complexity. After analyzing the operation of the dynamic programming and improved dynamic programming algorithms, we have identified which are the most frequent operations and propose an optimized method. In addition, we study and implement a method for dividing the work into different threads. To describe incremental improvements of the algorithm design, we first compare performance of an improved single central processing unit core version where we obtain speedups ranging from 7 × to 11 × . Then, we describe the best resource use in a multi-thread optimized version where we obtain an additional 7.5 × speedup running in a 12-core machine.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Scalability of a multi-physics system for forest fire spread prediction in multi-core platforms

  • Farguell, Àngel|||0000-0003-2395-220X
  • Cortés Fité, Ana|||0000-0003-1697-1293
  • Margalef, Tomàs|||0000-0001-6384-7389
  • Miró, Josep Ramon|||0000-0003-2838-6083
  • Mercader, J.
Advances in high-performance computing have led to an improvement in modeling multi-physics systems because of the capacity to solve complex numerical systems in a reasonable time. WRF-SFIRE is a multi-physics system that couples the atmospheric model WRF and the forest fire spread model called SFIRE with the objective of considering the atmosphere-fire interactions. In systems like WRF-SFIRE, the trade-off between result accuracy and time required to deliver that result is crucial. So, in this work, we analyze the influence of WRF-SFIRE settings (grid resolutions) into the forecasts accuracy and into the execution times on multi-core platforms using OpenMP and MPI parallel programming paradigms.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Finding, analysing and solving MPI communication bottlenecks in Earth System models

  • Tintó Prims, Oriol|||0000-0003-3182-3942
  • Castrillo, Miguel|||0000-0003-1826-623X
  • Acosta, Mario C.|||0000-0001-7054-8168
  • Mula-Valls, Oriol
  • Sanchez Lorente, Alicia
  • Serradell, Kim|||0000-0001-8230-4347
  • Cortés Fité, Ana|||0000-0003-1697-1293
  • Doblas-Reyes, Francisco J.|||0000-0002-6622-4280
It is a matter of consensus that the ability to efficiently use current and future high performance computing systems is crucial for science, however, the reality is that the performance currently achieved by most of the parallel scientific applications is far from desired. Despite inter-process communication has already been a matter of study in many different works, it is a fact that their recommendations are not taken into account in most of computational model development processes, at least in the case of Earth Science. This work presents a methodology that aims to help scientists working with computational models using inter-process communication, to deal with the difficulties they face when trying to understand their applications behaviour. Following a series of steps that are presented here, both users and developers will learn how to identify performance issues by characterizing applications scalability, identifying which parts present a bad performance and understand the role that inter-process communication plays. In this work, the Nucleus for European Modelling of the Ocean (NEMO), the state-of-the-art European global ocean circulation model, will be used as an example of success. It is a community code widely used in Europe, to the extent that more than a hundred million core hours are used every year in experiments involving NEMO. In the analysis exercise, it is shown how to answer the questions of where, why and what is degrading model's scalability, and how this information can help developers in finding solutions that will mitigate their eventual issues. This document also demonstrates how performance analysis carried out with small size experiments, using limited resources, can lead to optimizations that will impact bigger experiments running on thousands of cores, making it easier to deal with the exascale challenge.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Designing a benchmark for the performance evaluation of agent-based simulation applications on HPC

  • Moreno, Andreu
  • Rodríguez, Juan J.
  • Beltrán Mora, Daniel
  • Sikora, Anna|||0000-0003-0090-4109
  • Jorba i Esteve, Josep|||0000-0002-5810-4748
  • César Galobardes, Eduardo|||0000-0002-9729-8557
Agent-based modeling and simulation (ABMS) is a class of computational models for simulating the actions and interactions of autonomous agents with the goal of assessing their effects on a system as a whole. Several frameworks for generating parallel ABMS applications have been developed taking advantage of their common characteristics, but there is a lack of a general benchmark for comparing the performance of the generated applications. We propose and design a benchmark that takes into consideration the most common characteristics of this type of applications and includes parameters for influencing their relevant performance aspects. We provide an initial implementation of the benchmark for FLAME, FLAME GPU, Repast HPC and EcoLab, some of the most popular parallel ABMS platforms, and use it for comparing the applications generated by these platforms. The obtained results are mostly in agreement with previous studies, but the designed and implemented specification has allowed for testing a wider set of aspects, such as the number of interacting agents, the amount of interchanged data or the evolution of the workload and obtaining more reliable results.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

A History-based resource manager for genome analysis workflows applications on clusters with heterogeneous nodes

  • Badosa, Ferran|||0000-0002-9162-2918
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Acevedo Giménez, César Esteban
  • Vera Rodríguez, Gonzalo|||0000-0001-6990-0216
  • Ripoll Aracil, Ana|||0000-0001-6874-9278
Bioinformatics workflows require large amounts of resources and are commonly executed in clusters. Determining the adequate amount of resources for bioinformatics applications is a tricky matter, since the resource usage of a single application might vary substantially from one execution to the next. Resource management systems in clusters don't consider these variations and subsequent needs. As a result, the computing power offered by clusters is not harnessed properly, compromising both application performance and resource efficiency. To tackle these issues, we propose a History-Based Resource Manager for bioinformatics workflows applications running on clusters with heterogeneous nodes. The proposed resource manager features a prediction model that generates multiple performance predictions for each job under different combinations of cluster resources. Furthermore, the proposed resource manager includes a scheduling algorithm that considers the degree of multiprogramming of the nodes, scheduling combinations of applications for simultaneous same-node execution upon their compatibility. To test the proposed resource manager, we process two workloads formed by different amounts of workflows made up by common bioinformatics applications. Results prove that for the given cases, the proposed resource manager improves the performance obtained with SLURM, using First Come First Served policy. The proposal shows an average workflow makespan improvement range between 28 and 35%, averaging 32%, an average workflow efficiency improvement range between 75 and 83%, averaging 79%, and an average resource usage improvement range between 96 and 101%, averaging 99%. Furthermore, the proposed scheduling algorithm can improve the average workflow makespan by a range of values between 26 and 36%, averaging 31%, compared to Max-Min and Min-Min algorithms.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

A Critical Path File Location (CPFL) algorithm for data-aware multiworkflow scheduling on HPC clusters

  • Acevedo Giménez, César Esteban
  • Hernández Budé, Porfidio|||0000-0002-8592-934X
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Méndez Muñoz, Víctor|||0000-0002-9044-1189
A representative set of workflows found in bioinformatics pipelines must deal with large data sets. Most scientific workflows are defined as Direct Acyclic Graphs (DAGs). Despite DAGs are useful to understand dependence relationships, they do not provide any information about input, output and temporal data files. This information about the location of files of data intensive applications helps to avoid performance issues. This paper presents a multiworkflow store-aware scheduler in a cluster environment called Critical Path File Location (CPFL) policy where the access time to disk is more relevant than network, as an extension of the classical list scheduling policies. Our purpose is to find the best location of data files in a hierarchical storage system. The resulting algorithm is tested in an HPC cluster and in a simulated cluster scenario with bioinformatics synthetic workflows, and largely used benchmarks like Montage and Epigenomics. The resulting simulator is tuned and validated with the first test results from the real infrastructure. The evaluation of our proposal shows promising results up to 70% on benchmarks in real HPC clusters using 128 cores and up to 69% of makespan improvement on simulated 512 cores clusters with a deviation between 0.9% and 3% regarding the real HPC cluster.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Embedded real-time stereo estimation via semi-global matching on the GPU

  • Hernández Juárez, Daniel|||0000-0001-5878-1549
  • Chacón, Alejandro|||0000-0001-8851-7618
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Vázquez, David
  • Moure, Juan C.|||0000-0001-6697-0331
  • López Peña, Antonio M.|||0000-0002-6979-5783
Dense, robust and real-time computation of depth information from stereo-camera systems is a computationally demanding requirement for robotics, advanced driver assistance systems (ADAS) and autonomous vehicles. Semi-Global Matching (SGM) is a widely used algorithm that propagates consistency constraints along several paths across the image. This work presents a real-time system producing reliable disparity estimation results on the new embedded energy-efficient GPU devices. Our design runs on a Tegra X1 at 42 frames per second (fps) for an image size of 640×480, 128 disparity levels, and using 4 path directions for the SGM method.