Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 361
Encontrada(s) 4 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Accelerating Edit-Distance Sequence Alignment on GPU Using the Wavefront Algorithm

  • Aguado-Puig, Quim|||0000-0003-4871-3192
  • Marco-Sola, Santiago|||0000-0001-7951-3914
  • Moure, Juan C.|||0000-0001-6697-0331
  • Castells-Rufas, David|||0000-0002-7181-9705
  • Alvarez, Lluc
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Moreto, Miquel|||0000-0002-9848-8758
Sequence alignment remains a fundamental problem with practical applications ranging from pattern recognition to computational biology. Traditional algorithms based on dynamic programming are hard to parallelize, require significant amounts of memory, and fail to scale for large inputs. This work presents eWFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute the exact edit-distance sequence alignment based on the wavefront alignment algorithm (WFA). This approach exploits the similarities between the input sequences to accelerate the alignment process while requiring less memory than other algorithms. Our implementation takes full advantage of the massive parallel capabilities of modern GPUs to accelerate the alignment process. In addition, we propose a succinct representation of the alignment data that successfully reduces the overall amount of memory required, allowing the exploitation of the fast shared memory of a GPU. Our results show that our GPU implementation outperforms by 3- 9× the baseline edit-distance WFA implementation running on a 20 core machine. As a result, eWFA-GPU is up to 265 times faster than state-of-the-art CPU implementation, and up to 56 times faster than state-of-the-art GPU implementations.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

WFA-GPU: Gap-affine pairwise read-alignment using GPUs

  • Aguado-Puig, Quim|||0000-0003-4871-3192
  • Doblas, Max
  • Matzoros, Christos
  • Espinosa, Antonio|||0000-0002-6460-3789
  • Moure, Juan C.|||0000-0001-6697-0331
  • Marco-Sola, Santiago|||0000-0001-7951-3914
  • Moreto, Miquel|||0000-0002-9848-8758
Advances in genomics and sequencing technologies demand faster and more scalable analysis methods that can process longer sequences with higher accuracy. However, classical pairwise alignment methods, based on dynamic programming (DP), impose impractical computational requirements to align long and noisy sequences like those produced by PacBio and Nanopore technologies. The recently proposed wavefront alignment (WFA) algorithm paves the way for more efficient alignment tools, improving time and memory complexity over previous methods. However, high-performance computing (HPC) platforms require efficient parallel algorithms and tools to exploit the computing resources available on modern accelerator-based architectures. This paper presents WFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute exact gap-affine alignments based on the WFA algorithm. We present the algorithmic adaptations and performance optimizations that allow exploiting the massively parallel capabilities of modern GPU devices to accelerate the alignment computations. In particular, we propose a CPU-GPU co-design capable of performing inter-sequence and intra-sequence parallel sequence alignment, combining a succinct WFA-data representation with an efficient GPU implementation. As a result, we demonstrate that our implementation outperforms the original multi-threaded WFA implementation by up to 4.3× and up to 18.2× when using heuristic methods on long and noisy sequences. Compared to other state-of-the-art tools and libraries, the WFA-GPU is up to 29× faster than other GPU implementations and up to four orders of magnitude faster than other CPU implementations. Furthermore, WFA-GPU is the only GPU solution capable of correctly aligning long reads using a commodity GPU. WFA-GPU code and documentation are publicly available at https://github.com/quim0/WFA-GPU.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Locality-aware cache random replacement policies

  • Benedicte, Pedro
  • Abella, Jaume
  • Cazorla, Francisco J.
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] Measurement-Based Probabilistic Timing Analysis (MBPTA) facilitates the analysis of complex software running on hardware comprising high-performance features. MBPTA also aims at preventing additional analysis costs for timing analysis techniques and preserving the confidence on derived WCET estimates. Cache behavior has a deep influence on WCET estimates and hence on "the amount of software" that can be consolidated onto a single hardware platform. Deterministic replacement policies such as LRU (Least Recently Used) and NMRU (Non-Most Recently Used) have systematic pathological cases that may lead to high execution times and WCET estimates. Instead, random replacement (RR) decreases pathological cases probability, at the cost of temporal locality. We present two new MBPTA-amenable replacement policies that completely remove the presented pathological cases. The first policy, Random Permutations (RP) preserves higher temporal locality than RR; while the second, NMRU Random Permutations (NMRURP), also protects the Most Recently Used line from eviction. Both proposed policies build upon restricted random replacement choices. Our simulation evaluation (validated against a real prototype) using the Malardalen benchmarks and a case study shows that RP and NMRURP deliver both high average performance (within 1% of LRUs and NRMU performance) and tight WCET estimates 11% and 24% lower than those of RR., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under grant TIN2015-65316-P, the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No. 772773) and the HiPEACH Network of Excellence. Pedro Benedicte and Jaume Abella have been partially supported by the MINECO under FPU15/01394 grant and Ramon y Cajal postdoctoral fellowship number RYC-2013-14717 respectively.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Improving the management efficiency of GPU workloads in data centers through GPU virtualization

  • Iserte, Sergio
  • Reaño González, Carlos
  • Prades, Javier|||0000-0003-3349-2200
  • Silla, Federico|||0000-0002-6435-1200
[EN] Graphics processing units (GPUs) are currently used in data centers to reduce the execution time of compute-intensive applications. However, the use of GPUs presents several side effects, such as increased acquisition costs and larger space requirements. Furthermore, GPUs require a nonnegligible amount of energy even while idle. Additionally, GPU utilization is usually low for most applications. In a similar way to the use of virtual machines, using virtual GPUs may address the concerns associated with the use of these devices. In this regard, the remote GPU virtualization mechanism could be leveraged to share the GPUs present in the computing facility among the nodes of the cluster. This would increase overall GPU utilization, thus reducing the negative impact of the increased costs mentioned before. Reducing the amount of GPUs installed in the cluster could also be possible. However, in the same way as job schedulers map GPU resources to applications, virtual GPUs should also be scheduled before job execution. Nevertheless, current job schedulers are not able to deal with virtual GPUs. In this paper, we analyze the performance attained by a cluster using the remote Compute Unified Device Architecture middleware and a modified version of the Slurm scheduler, which is now able to assign remote GPUs to jobs. Results show that cluster throughput, measured as jobs completed per time unit, is doubled at the same time that the total energy consumption is reduced up to 40%. GPU utilization is also increased., Generalitat Valenciana, Grant/Award Number: PROMETEO/2017/077; MINECO and FEDER, Grant/Award Number: TIN2014-53495-R, TIN2015-65316-P and TIN2017-82972-R
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Worst-Case Energy Consumption: A New Challenge for Battery-Powered Critical Devices

  • Trilla, David
  • Abella, Jaume
  • Cazorla, Francisco J.
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] The number of (edge) devices connected to the IoT is on the rise, reaching hundreds of billions in the next years. Many devices will implement some type of critical functionality, for instance in the medical market this includes infusion pumps and implantable defibrillators. Energy awareness is mandatory in the design of IoT devices given their huge impact on worldwide energy consumption and the fact that many of them are battery powered. Critical IoT devices further require addressing new energy-related challenges. On the one hand, factoring in the impact of energy-solutions on device's performance, providing evidence of adherence to domain-specific safety standards. On the other hand, deriving safe worst-case energy consumption (WCEC) estimates is fundamental to ensure the system can continuously operate under a pre-established set of power/energy caps, safely delivering its critical functionality. In this line, we analyze for the first time the impact that different hardware physical parameters have on both model-based and measurement-based WCEC modeling, for which we also show the main challenges they face compared to chip manufacturers' current practice for energy modeling and validation. Under the set of constraints that emanate from how certain physical parameters can be actually modeled, we show that measurement-based WCEC is a promising way forward for WCEC estimation., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under grant TIN2015-65316-P and the HiPEAC Network of Excellence. Jaume Abella has been partially supported by the MINECO under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717. Carles Hernndez is jointly funded by the MINECO and FEDER funds through grant TIN2014-60404-JIN.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Probabilistic Worst-Case Timing Analysis: Taxonomy and Comprehensive Survey

  • Cazorla, Francisco J.
  • Kosmidis, L.
  • Mezzetti, E.
  • Abella, Jaume
  • Vardanega, Tullio
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] The unabated increase in the complexity of the hardware and software components of modern embedded real-time systems has given momentum to a host of research in the use of probabilistic and statistical techniques for timing analysis. In the last few years, that front of investigation has yielded a body of scientific literature vast enough to warrant some comprehensive taxonomy of motivations, strategies of application, and directions of research. This survey addresses this very need, singling out the principal techniques in the state of the art of timing analysis that employ probabilistic reasoning at some level, building a taxonomy of them, discussing their relative merit and limitations, and the relations among them. In addition to offering a comprehensive foundation to savvy probabilistic timing analysis, this article also identifies the key challenges to be addressed to consolidate the scientific soundness and industrial viability of this emerging field., This work has also been partially supported by the Spanish Ministry of Science and Innovation under grant TIN2015-65316-P, the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No. 772773), and the HiPEAC Network of Excellence. Jaume Abella was partially supported by the Ministry of Economy and Competitiveness under a Ramon y Cajal postdoctoral fellowship (RYC-2013-14717). Enrico Mezzetti has been partially supported by the Spanish Ministry of Economy and Competitiveness under Juan de la Cierva-Incorporación postdoctoral fellowship No. IJCI-2016-27396.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

DMRlib: Easy-coding and efficient resource management for job malleability

  • Iserte, Sergio
  • Mayo, Rafael
  • Peña Monferrer, Antonio José
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Process malleability has proved to have a highly positive impact on the resource utilization and global productivity in data centers compared with the conventional static resource allocation policy. However, the non-negligible additional development effort this solution imposes has constrained its adoption by the scientific programming community. In this work, we present DMRlib, a library designed to offer the global advantages of process malleability while providing a minimalist MPI-like syntax. The library includes a series of predefined communication patterns that greatly ease the development of malleable applications. In addition, we deploy several scenarios to demonstrate the positive impact of process malleability featuring different scalability patterns. Concretely, we study two job submission modes (rigid and moldable) in order to identify the best-case scenarios for malleability using metrics such as resource allocation rate, completed jobs per second, and energy consumption. The experiments prove that our elastic approach may improve global throughput by a factor higher than 3x compared to the traditional workloads of non-malleable jobs, This work was supported in part by Projects TIN2014-53495-R, TIN2015-65316-P, and TIN2017-82972-R from MINECO and FEDER. This project has received funding fromthe European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska Curie Grant 749516. Thework of Sergio Isertewas supported by a postdoctoral fellowship from Generalitat Valenciana and European Social Fund APOSTD/2020/026. Finally, the authors would like to thank the anonymous reviewers whose suggestions significantly improved the quality of this article
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

DMR API: Improving cluster productivity by turning applications into malleable

  • Iserte Agut, Sergio
  • Mayo Gual, Rafael
  • Beltrán, Vicenç
  • Peña Monferrer, Antonio José
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Adaptive workloads can change on-the-fly the configuration of their jobs, in terms of number of processes. To carry out these job reconfigurations, we have designed a methodology which enables a job to communicate with the resource manager and, through the runtime. to change its number of MPI ranks. The collaboration between both the workload manager-aware of the queue of jobs and the resources allocation-and the parallel runtime-able to transparently handle the processes and the program data-is crucial for our throughput-aware malleability methodology. Hence, when a job triggers a reconfiguration, the resource manager will check the cluster status and return the appropriate action: i) expand, if there are spare resources; ii) shrink, if queued jobs can be initiated; or iii) none, if no change can improve the global productivity. In this paper, we describe the internals of our framework and demonstrate how it reduces the global workload completion time along with providing a more efficient usage of the underlying resources. For this purpose, we present a thorough study of the adaptive workloads processing by showing the detailed behavior of our framework in representative experiments. (C) 2018 Elsevier B.V. All rights reserved.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Two-sided orthogonal reductions to condensed forms on asymmetric multicore processors

  • Alonso-Jordá, Pedro|||0000-0002-6882-6592
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Catalán, Sandra
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
[EN] We investigate how to leverage the heterogeneous resources of an Asymmetric Multicore Processor (AMP) in order to deliver high performance in the reduction to condensed forms for the solution of dense eigenvalue and singular-value problems. The routines that realize this type of two-sided orthogonal reductions (TSOR) in LAPACK are especially challenging, since a significant fraction of their floating-point operations are cast in terms of memory-bound kernels while the remaining part corresponds to efficient compute-bound kernels. To deal with this scenario: (1) we leverage implementations of memory-bound and compute-bound kernels specifically tuned for AMPs; (2) we select the algorithmic block size for the TSOR routines via a practical model; and (3) we adjust the type and number of cores to use at each step of the reduction. Our experiments validate the model and assess the performance of our asymmetry-aware TSOR routines, using an ARMv7 big.LITTLE AMP, for three key operations: the reduction to tridiagonal form for symmetric eigenvalue problems, the reduction to Hessenberg form for non-symmetric eigenvalue problems, and the reduction to bidiagonal form for singular-value problems., The researchers from Universidad Jaume I were supported by project TIN2014-53495-R of MINECO and FEDER, and the FPU program of MECD. The researcher from Universitat Politecnica de Valencia was supported by the Generalitat Valenciana PROMETEOII/2014/003. The researcher from Universitat Politecnica de Catalunya was supported by projects TIN2015-65316-P from the Spanish Ministry of Education and 2014 SGR 1051 from the Generalitat de Catalunya, Dep. d'Innovacio, Universitats i Empresa.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Static scheduling of the LU factorization with look-ahead on asymmetric multicore processors

  • Catalán, Sandra
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We analyze the benefits of look-ahead in the parallel execution of the LU factorization with partial pivoting (LUpp) in two distinct "asymmetric" multicore scenarios. The first one corresponds to an actual hardware-asymmetric architecture such as the Samsung Exynos 5422 system-on-chip (SoC), equipped with an ARM big.LITTLE processor consisting of a quad core Cortex-A15 cluster plus a quad-core Cortex-A7 cluster. For this scenario, we propose a careful mapping of the different types of tasks appearing in LUpp to the computational resources, in order to produce an efficient architecture-aware exploitation of the computational resources integrated in this SoC. The second asymmetric configuration appears in a hardware-symmetric multicore architecture where the cores can individually operate at a different frequency levels. In this scenario, we show how to employ the frequency slack to accelerate the tasks in the critical path of LUpp in order to produce a faster global execution as well as a lower energy consumption. (C) 2018 Elsevier B.V. All rights reserved., The researchers from Universidad Jaume I were supported by projects TIN2014-53495-R and TIN2017-82972-R of MINECO and FEDER, and the FPU program of MECD. The researcher from Universitat Politecnica de Catalunya was supported by projects TIN2015-65316-P of MINECO and FEDER and 2017-SGR-1414 from the Generalitat de Catalunya.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Efficient update of determinants of rmany-electron wave function overlaps

  • Alonso-Jordá, Pedro|||0000-0002-6882-6592
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Davidovic, Davor
  • Sapunar, Marin
  • Herrero, Jose R.
[EN] The calculation of overlaps between many-electron wave functions at different nuclear geometries during nonadiabatic dynamics simulations requires the evaluation of a large number of determinants of matrices that differ only in a few rows/columns. While this calculation is fast for small systems, its cost grows faster than the alternative electronic structure calculation used to obtain the wave functions. For wave functions that can be written as a CIS expansion, all determinants can be computed using the set of level-2 minors of the reference matrix. However, this is still a costly computation for large systems. In this paper, we provide an algorithm for efficiently calculating all level-2 minors of a matrix by re-utilizing and updating the LU factorization for the determinants of the minors. This approach results in a parallel version of the algorithm that is up to an order of magnitude faster then the current best parallel implementation. The algorithm thus allows the computation of exact wave function overlaps for relatively large systems, with a high density of states, at virtually no cost compared with the electronic structure calculations. Furthermore, the new algorithm opens the path to further investigations in efficient computing of the exact wave function overlaps for complex wave functions such as MR-CIS and MR-CISD. Program summary Program Title: CIS Overlap Licensing provisions: MIT Programming language: FORTRAN 2008, C Nature of problem: Calculation of overlaps between CIS type wave functions at different nuclear geometries during nonadiabatic dynamics simulation requires calculating a large number of connected determinants and scales with the seventh power of the size of the system being studied. Without additional approximations, for large systems this computation becomes more costly than the electronic structure calculation used to obtain the wave functions. Solution method: All of the determinants required for a CIS wave function overlap calculation can be derived from the set of all level-2 minors of the matrix of overlaps between the reference Slater determinants. We developed an algorithm for efficiently computing all level-2 minors of a matrix. This part of the computational process was the bottleneck in previous solutions to the problem and is now an order of magnitude faster resulting in significantly faster overall calculation of the wave function overlaps., Enrique S. Quintana-Orti was supported by project TIN2017-82972-R, Pedro Alonso-Jorda by project TIN2017-89314-P-AR, and Jose R. Herrero by project TIN2015-65316-P, all of the MINECO and FEDER. The latter was also funded by the Generalitat de Catalunya (contract 2017-SGR-1414). Marin Sapunar and Davor Davidovic were supported by Croatian Science Foundation under grant HRZZ IP-2016-06-1142.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Energy balance between voltage-frequency scaling and resilience for linear algebra routines on low-power multicore architectures

  • Catalán, Sandra
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Near Threshold Voltage (NTV) computing has been recently proposed as a technique to save energy, at the cost of incurring higher error rates including, among others, Silent Data Corruption (SDC). In this paper, we evaluate the energy efficiency of dense linear algebra routines using several low-power multicore processors and we analyze whether the potential energy reduction achieved when scaling the processor to operate at a low voltage compensates the cost of integrating a fault tolerance mechanism that tackles SDC. Our study targets algorithmic-based fault-tolerant versions of the dense matrix-vector and matrix(matrix) multiplication kernels (GEMV and GEMM, respectively), using the BLIS framework, as well as an implementation of the LU factorization with partial pivoting built on top of GEMM, Furthermore, we tailor the study for a number of representative 32-bit and 64-bit multicore processors from ARM that were specifically designed for energy efficiency. (C) 2017 Elsevier B.V. All rights reserved., The researchers from Universidad Jaume I were supported by project CICYT TIN2014-53495-R of MINECO and FEDER, and the FPU program of MECD. The researcher from Universitat Politecnica de Catalunya was supported by projects TIN2015-65316-P from the Spanish Ministry of Education and 2014 SGR 1051 from the Generalitat de Catalunya, Dep. d'Innovacio, Universitats i Empresa.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

A case for malleable thread-level linear algebra libraries: The LU factorization with partial pivoting

  • Catalán, Sandra
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • van de Geijn, Robert
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We propose two novel techniques for overcoming load-imbalance encountered when implementing so-called look-ahead mechanisms in relevant dense matrix factorizations for the solution of linear systems. Both techniques target the scenario where two thread teams are created/activated during the factorization, with each team in charge of performing an independent task/branch of execution. The first technique promotes worker sharing (WS) between the two tasks, allowing the threads of the task that completes first to be reallocated for use by the costlier task. The second technique allows a fast task to alert the slower task of completion, enforcing the early termination (ET) of the second task, and a smooth transition of the factorization procedure into the next iteration. The two mechanisms are instantiated via a new malleable thread-level implementation of the basic linear algebra subprograms, and their benefits are illustrated via an implementation of the LU factorization with partial pivoting enhanced with look-ahead. Concretely, our experimental results on an Intel-Xeon system with 12 cores show the benefits of combining WS+ET, reporting competitive performance in comparison with a task-parallel runtime-based solution., This work was supported in part by the Spanish Ministerio de Economia y Competitividad under Project TIN2014-53495-R, Project TIN2015-65316-P, and Project TIN2017-82972-R, in part by the H2020 EU FETHPC "INTERTWinE" under Project 671602, in part by the Generalitat de Catalunya under Project 2017-SGR-1414, and in part by the NSF under Grant ACI-1550493.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Multi-threaded dense linear algebra libraries for low-power asymmetric multicore processors

  • Catalán, Sandra
  • Herrero, José R.
  • Igual Peña, Francisco Daniel
  • Rodríguez-Sánchez, Rafael
  • Adeniyi-Jones, Chris
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Dense linear algebra libraries, such as BLAS and LAPACK, provide a relevant collection of numerical tools for many scientific and engineering applications. While there exist high performance implementations of the BLAS (and LAPACK) functionality for many current multi-threaded architectures, the adaption of these libraries for asymmetric multicore processors (AMPs) is still pending. In this paper we address this challenge by developing an asymmetry-aware implementation of the BLAS, based on the BLIS framework, and tailored for AMPs equipped with two types of cores: fast/power-hungry versus slow/energy-efficient. For this purpose, we integrate coarse-grain and fine-grain parallelization strategies into the library routines which, respectively, dynamically distribute the workload between the two core types and statically repartition this work among the cores of the same type. Our results on an ARM (R) big.LITTLE (TM) processor embedded in the Exynos 5422 SoC, using the asymmetry-aware version of the BLAS and a plain migration of the legacy version of LAPACK, experimentally assess the benefits, limitations, and potential of this approach from the perspectives of both throughput and energy efficiency. (C) 2016 Elsevier B.V. All rights reserved., The researchers from Universidad Jaume I were supported by projects CICYT TIN2011-23283 and TIN2014-53495-R of MINECO and FEDER, and the FPU program of MECD. The researcher from Universidad Complutense de Madrid was supported by project CICYT TIN2015-65277-R. The researcher from Universitat Politecnica de Catalunya was supported by projects TIN2015-65316-P from the Spanish Ministry of Education and 2014 SGR 1051 from the Generalitat de Catalunya, Dep. dinnovacio, Universitats i Empresa.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Predictive Reliability and Fault Management in Exascale Systems: State of the Art and Perspectives

  • Canal, Ramon
  • Tornero-Gavilá, Rafael
  • Cilardo, Alessandro
  • Massari, Giuseppe
  • Reghenzani, Federico
  • Fornaciari, William
  • Zapater, Marina
  • Atienza, David
  • Oleksiak, Ariel
  • Piatek, Wojciech
  • Abella, Jaume
  • Hernández Luz, Carles|||0000-0003-1192-6223
[EN] Performance and power constraints come together with Complementary Metal Oxide Semiconductor technology scaling in future Exascale systems. Technology scaling makes each individual transistor more prone to faults and, due to the exponential increase in the number of devices per chip, to higher system fault rates. Consequently, High-performance Computing (HPC) systems need to integrate prediction, detection, and recovery mechanisms to cope with faults efficiently. This article reviews fault detection, fault prediction, and recovery techniques in HPC systems, from electronics to system level. We analyze their strengths and limitations. Finally, we identify the promising paths to meet the reliability levels of Exascale systems., This work has received funding from the European Union's Horizon 2020 (H2020) research and innovation program under the FET-HPC Grant Agreement No. 801137 (RECIPE). Jaume Abella was also partially supported by the Ministry of Economy and Competitiveness of Spain under Contract No. TIN2015-65316-P and under Ramon y Cajal Postdoctoral Fellowship No. RYC-2013-14717, as well as by the HiPEAC Network of Excellence. Ramon Canal is partially supported by the Generalitat de Catalunya under Contract No. 2017SGR0962.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Computing Safe Contention Bounds for Multicore Resources with Round-Robin and FIFO Arbitration

  • Fernández, Gabriel
  • Jalle, Javier
  • Abella, Jaume
  • Quiñones, Eduardo
  • Vardanega, Tullio
  • Cazorla, Francisco J.
Numerous researchers have studied the contention that arises among tasks running in parallel on a multicore processor. Most of those studies seek to derive a tight and sound upper-bound for the worst-case delay with which a processor resource may serve an incoming request, when its access is arbitrated using time-predictable policies such as round-robin or FIFO. We call this value upper-bound delay (ubd). Deriving trustworthy ubd statically is possible when sufficient public information exists on the timing latency incurred on access to the resource of interest. Unfortunately however, that is rarely granted for commercial-of-the-shelf (COTS) processors. Therefore, the users resort to measurement observations on the target processor and thus compute a >measured> ubd. However, using ubd to compute worst-case execution time values for programs running on COTS multicore processors requires qualification on the soundness of the result. In this paper, we present a measurement-based methodology to derive a ubd under round-robin (RoRo) and first-in-first-out (FIFO) arbitration, which accurately approximates ubd from above, without needing latency information from the hardware provider. Experimental results, obtained on multiple processor configurations, demonstrate the robustness of the proposed methodology., The research leading to this work has received funding from: the European Union's Horizon 2020 research and innovation programme under grant agreement No 644080 (SAFURE); the European Space Agency under Contract 789.2013 and NPI Contract 40001102880; and COST Action IC1202, Timing Analysis On Code-Level (TACLe). This work has also been partially supported by the Spanish Ministry of Science and Innovation under grant TIN2015-65316-P. Jaume Abella has been partially supported by the MINECO under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717. The authors would like to thank Paul Caheny for his help with the proofreading of this document., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

COMPSs-Mobile: Parallel Programming for Mobile Cloud Computing

  • Lordan, Francesc
  • Badia, Rosa M.
The advent of the Cloud and the popularization of mobile devices have led us to a shift in computing access where users have an interactive display, and heavy computations run remotely, in the Cloud servers. COMPSs-Mobile is a framework that aims to ease the development of energy-efficient and high-performing applications for this kind of environment. The framework provides an infrastructure-unaware programming model that allows developers to code regular Android applications whose computation is transparently parallelized and partially offloaded to remote resources. This paper gives an overview of the programming model and describes the internal components of the toolkit which supports it focusing on the offloading and checkpointing mechanisms. It also presents the results of some tests conducted to evaluate the behavior of the solution and to measure the potential benefits in Android applications., This work has been supported by the Spanish Goverment (contracts TIN2012-34557, TIN2015-65316-P and grants BES-2013-067167 of the Research Training Program and SEV-2011-00067 of Severo Ochoa Program), by Generalitat de Catalunya (contract 2014-SGR-1051) and by the European Commission (ASCETiC project, FP7-ICT-2013.1.2 contract 610874)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

LightDock: a new multi-scale approach to protein–protein docking

  • Jiménez-García, Brian
  • Roel-Touris, Jorge
  • Romero-Durana, Miguel
  • Vidal, Miquel
  • Jiménez-González, Daniel
  • Fernández-Recio, Juan
[Motivation] Computational prediction of protein–protein complex structure by docking can provide structural and mechanistic insights for protein interactions of biomedical interest. However, current methods struggle with difficult cases, such as those involving flexible proteins, low-affinity complexes or transient interactions. A major challenge is how to efficiently sample the structural and energetic landscape of the association at different resolution levels, given that each scoring function is often highly coupled to a specific type of search method. Thus, new methodologies capable of accommodating multi-scale conformational flexibility and scoring are strongly needed., [Results] We describe here a new multi-scale protein–protein docking methodology, LightDock, capable of accommodating conformational flexibility and a variety of scoring functions at different resolution levels. Implicit use of normal modes during the search and atomic/coarse-grained combined scoring functions yielded improved predictive results with respect to state-of-the-art rigidbody docking, especially in flexible cases., B.J-G was supported by a FPI fellowship from the Spanish Ministry of Economy and Competitiveness. This work was supported by grants BIO2013-48213-R, SEV-2015-0493, TIN2015-65316-P, and BIO2016- 79930-R from the Spanish Ministry of Economy and Competitiveness, GA 687698 from the EU H2020 program, and 2014-SGR-1051 from Universitats i Empresa program of Generalitat de Catalunya., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Exploring the capabilities of support vector machines in detecting silent data corruptions

  • Subasi, Omer
  • Di, Sheng
  • Bautista-Gomez, Leonardo
  • Balaprakash, Prasanna
  • Unsal, Osman
  • Labarta, Jesús
  • Cristal, Adrian
  • Krishnamoorthy, Sriram
  • Cappello, Franck
As the exascale era approaches, the increasing capacity of high-performance computing (HPC) systems with targeted power and energy budget goals introduces significant challenges in reliability. Silent data corruptions (SDCs), or silent errors, are one of the major sources that corrupt the execution results of HPC applications without being detected. In this work, we explore a set of novel SDC detectors – by leveraging epsilon-insensitive support vector machine regression – to detect SDCs that occur in HPC applications. The key contributions are threefold. (1) Our exploration takes temporal, spatial, and spatiotemporal features into account and analyzes different detectors based on different features. (2) We provide an in-depth study on the detection ability and performance with different parameters, and we optimize the detection range carefully. (3) Experiments with eight real-world HPC applications show that support-vector-machine-based detectors can achieve detection sensitivity (i.e., recall) up to 99% yet suffer a less than 1% false positive rate for most cases. Our detectors incur low performance overhead, 5% on average, for all benchmarks studied in this work., This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under Award Number 66905, program manager Lucy Nowell. Pacific Northwest National Laboratory is operated by Battelle for DOE under Contract DE-AC05-76RL01830. In addition, this material is based upon work supported by the National Science Foundation under Grant No. 1619253, and also by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, program manager Lucy Nowell, under contract number DE-AC02-06CH11357 (DOE Catalog project) and in part by the European Union FEDER funds under contract TIN2015-65316-P., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Task-based programming in COMPSs to converge from HPC to big data

  • Conejero, Javier
  • Corella, Sandra
  • Badia, Rosa M.
  • Labarta, Jesús
Task-based programming has proven to be a suitable model for high-performance computing (HPC) applications. Different implementations have been good demonstrators of this fact and have promoted the acceptance of task-based programming in the OpenMP standard. Furthermore, in recent years, Apache Spark has gained wide popularity in business and research environments as a programming model for addressing emerging big data problems. COMP Superscalar (COMPSs) is a task-based environment that tackles distributed computing (including Clouds) and is a good alternative for a task-based programming model for big data applications. This article describes why we consider that task-based programming models are a good approach for big data applications. The article includes a comparison of Spark and COMPSs in terms of architecture, programming model, and performance. It focuses on the differences that both frameworks have in structural terms, on their programmability interface, and in terms of their efficiency by means of three widely known benchmarking kernels: Wordcount, Kmeans, and Terasort. These kernels enable the evaluation of the more important functionalities of both programming models and analyze different work flows and conditions. The main results achieved from this comparison are (1) COMPSs is able to extract the inherent parallelism from the user code with minimal coding effort as opposed to Spark, which requires the existing algorithms to be adapted and rewritten by explicitly using their predefined functions, (2) it is an improvement in terms of performance when compared with Spark, and (3) COMPSs has shown to scale better than Spark in most cases. Finally, we discuss the advantages and disadvantages of both frameworks, highlighting the differences that make them unique, thereby helping to choose the right framework for each particular objective., This work is supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), and by Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272). Javier Conejero's postdoctoral contract is cofinanced by the Ministry of Economy and Competitiveness under the Juan de la Cierva Formación postdoctoral fellowship number FJCI-2015-24651. This work is also supported by the Intel-BSC Exascale Lab. The Human Brain Project receives funding from the EU's Seventh Framework Programme (FP7/ 2007-2013) under grant agreement no. 604102., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Towards mobile cloud computing with single sign-on access

  • Lordan, Francesc
  • Jensen, Jens G.
  • Badia, Rosa M.
The low computing power of mobile devices impedes the development of mobile applications with a heavy computing load. Mobile Cloud Computing (MCC) has emerged as the solution to this by connecting mobile devices with the “infinite” computing power of the Cloud. As mobile devices typically communicate over untrusted networks, it becomes necessary to secure the communications to avoid privacy-sensitive data breaches. This paper presents work on implementing MCC applications with secure communications. For that purpose, we built on COMPSs-Mobile, a redesigned implementation of the COMP Superscalar (COMPSs) framework aiming to MCC platorms. COMPSs-Mobile automatically exploits the parallelism inherent in an application and orchestrates its execution on loosely-coupled distributed environment. To avoid a vendor lock-in, this extension leverages on the Generic Security Services Application Program Interface (GSSAPI) (RFC2743) as a generic way to access security services to provide communications with authentication, secrecy and integrity. Besides, GSSAPI allows applications to take profit of more advanced features, such as Federated Identity or Single Sign-On, which the underlying security framework could provide. To validate the practicality of the proposal, we use Kerberos as the security services provider to implement SSO; however, applications do not authenticate themselves and require users to obtain and place the credentials beforehand. To evaluate the performance, we conducted some tests running an application on a smartphone offloading tasks to a private cloud. Our results show that the overhead of securing the communications is acceptable., This work has been supported by the Spanish Government (contracts TIN2012-34557, TIN2015-65316-P and grants BES-2013-067167, EEBB-I-15-09808 of the Research Training Program and SEV-2011-00067 of Severo Ochoa Program), by Generalitat de Catalunya (contract 2014-SGR-1051) and by the European Commission (ASCETiC project, FP7-ICT-2013.1.2 contract 610874). The second author was partially supported by the European Commission’s Horizon2020 program under grant agreement 653965 (AARC)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Transparent orchestration of task-based parallel applications in containers platforms

  • Ramon-Cortes, Cristian
  • Serven, Albert
  • Ejarque, Jorge
  • Lezzi, Daniele
  • Badia, Rosa M.
This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a user-transparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way to develop task-based parallel applications from sequential codes, and containers management platforms that ease the deployment of applications in computing environments (as Docker, Mesos or Singularity). This framework provides scientists and developers with an easy way to implement parallel distributed applications and deploy them in a one-click fashion. We have built a prototype which integrates COMPSs with different containers engines in different scenarios: i) a Docker cluster, ii) a Mesos cluster, and iii) Singularity in an HPC cluster. We have evaluated the overhead in the building phase, deployment and execution of two benchmark applications compared to a Cloud testbed based on KVM and OpenStack and to the usage of bare metal nodes. We have observed an important gain in comparison to cloud environments during the building and deployment phases. This enables better adaptation of resources with respect to the computational load. In contrast, we detected an extra overhead during the execution, which is mainly due to the multi-host Docker networking., This work is partly supported by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316 project, by the Generalitat de Catalunya under contracts 2014-SGR-1051 and 2014-SGR-1272, and by the European Union through the Horizon 2020 research and innovation program under grant 690116 (EUBra-BIGSEA Project)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A hierarchic task-based programming model for distributed heterogeneous computing

  • Ejarque, Jorge
  • Domínguez, Marc
  • Badia, Rosa M.
Distributed computing platforms are evolving to heterogeneous ecosystems with Clusters, Grids and Clouds introducing in its computing nodes, processors with different core architectures, accelerators (i.e. GPUs, FPGAs), as well as different memories and storage devices in order to achieve better performance with lower energy consumption. As a consequence of this heterogeneity, programming applications for these distributed heterogeneous platforms becomes a complex task. Additionally to the complexity of developing an application for distributed platforms, developers must also deal now with the complexity of the different computing devices inside the node. In this article, we present a programming model that aims to facilitate the development and execution of applications in current and future distributed heterogeneous parallel architectures. This programming model is based on the hierarchical composition of the COMP Superscalar and Omp Superscalar programming models that allow developers to implement infrastructure-agnostic applications. The underlying runtime enables applications to adapt to the infrastructure without the need of maintaining different versions of the code. Our programming model proposal has been evaluated on real platforms, in terms of heterogeneous resource usage, performance and adaptation., The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the European Commission through the Horizon 2020 Research and Innovation program under contract 687584 (TANGO project) by the Spanish Government under contract TIN2015-65316 and grant SEV-2015-0493 (Severo Ochoa Program) and by Generalitat de Catalunya under contracts 2014-SGR-1051 and 2014-SGR-1272., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Locality-aware cache random replacement policies

  • Benedicte, Pedro
  • Hernandez, Carles
  • Abella, Jaume
  • Cazorla, Francisco J.
Measurement-Based Probabilistic Timing Analysis (MBPTA) facilitates the analysis of complex software running on hardware comprising high-performance features. MBPTA also aims at preventing additional analysis costs for timing analysis techniques and preserving the confidence on derived WCET estimates. Cache behavior has a deep influence on WCET estimates and hence on “the amount of software” that can be consolidated onto a single hardware platform. Deterministic replacement policies such as LRU (Least Recently Used) and NMRU (Non-Most Recently Used) have systematic pathological cases that may lead to high execution times and WCET estimates. Instead, random replacement (RR) decreases pathological cases probability, at the cost of temporal locality. We present two new MBPTA-amenable replacement policies that completely remove the presented pathological cases. The first policy, Random Permutations (RP) preserves higher temporal locality than RR; while the second, NMRU Random Permutations (NMRURP), also protects the Most Recently Used line from eviction. Both proposed policies build upon restricted random replacement choices. Our simulation evaluation (validated against a real prototype) using the Mälardalen benchmarks and a case study shows that RP and NMRURP deliver both high average performance (within 1% of LRUs and NRMU performance) and tight WCET estimates 11% and 24% lower than those of RR., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under grant TIN2015-65316-P, the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No. 772773) and the HiPEACH Network of Excellence. Pedro Benedicte and Jaume Abella have been partially supported by the MINECO under FPU15/01394 grant and Ramon y Cajal postdoctoral fellowship number RYC-2013-14717 respectively., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

On the maturity of parallel applications for asymmetric multi-core processors

  • Chronaki, Kallia
  • Moreto, Miquel
  • Casas, Marc
  • Rico, Alejandro
  • Badia, Rosa M.
  • Ayguade, Eduard
  • Valero, Mateo
Asymmetric multi-cores (AMCs) are a successful architectural solution for both mobile devices and supercomputers. By maintaining two types of cores (fast and slow) AMCs are able to provide high performance under the facility power budget. This paper performs the first extensive evaluation of how portable are the current HPC applications for such supercomputing systems. Specifically we evaluate several execution models on an ARM big.LITTLE AMC using the PARSEC benchmark suite that includes representative highly parallel applications. We compare schedulers at the user, OS and runtime levels, using both static and dynamic options and multiple configurations, and assess the impact of these options on the well-known problem of balancing the load across AMCs. Our results demonstrate that scheduling is more effective when it takes place in the runtime system level as it improves the baseline by 23%, while the heterogeneous-aware OS scheduling solution improves the baseline by 10%., This work has been supported by the RoMoL ERC Advanced Grant (GA 321253), by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation (contracts TIN2015-65316-P), by the Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), and by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 671697 and No. 779877. Kallia Chronaki has been partially supported by the Ministry of Economy and Competitiveness under Ramon y Cajal fellowship number RYC-2016-21104., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Hunting for open clusters in Gaia DR2: 582 new open clusters in the Galactic disc

  • Castro-Ginard, Alfred
  • Jordi, Carme
  • Luri, Xavier
  • Cid-Fuentes, Javier Álvarez
  • Casamiquela, Laia
  • Anders, Friedrich
  • Cantat-Gaudin, Tristan
  • Monguió-Tortajada, Marta
  • Balaguer-Nuñez, Lola
  • Solà, Salvi
  • Badia, Rosa M.
[Context] Open clusters are key targets for studies of Galaxy structure and evolution, and stellar physics. Since the Gaia data release 2 (DR2), the discovery of undetected clusters has shown that previous surveys were incomplete., [Aims] Our aim is to exploit the Big Data capabilities of machine learning to detect new open clusters in Gaia DR2, and to complete the open cluster sample to enable further studies of the Galactic disc., [Methods] We use a machine-learning based methodology to systematically search the Galactic disc for overdensities in the astrometric space and identify the open clusters using photometric information. First, we used an unsupervised clustering algorithm, DBSCAN, to blindly search for these overdensities in Gaia DR2 (l, b, ϖ, μα*, μδ), and then we used a deep learning artificial neural network trained on colour–magnitude diagrams to identify isochrone patterns in these overdensities, and to confirm them as open clusters., [Results] We find 582 new open clusters distributed along the Galactic disc in the region |b| < 20°. We detect substructure in complex regions, and identify the tidal tails of a disrupting cluster UBC 274 of ∼3 Gyr located at ∼2 kpc., [Conclusions] Adapting the mentioned methodology to a Big Data environment allows us to target the search using the physical properties of open clusters instead of being driven by computational limitations. This blind search for open clusters in the Galactic disc increases the number of known open clusters by 45%., Funding for the DPAC has been provided by national institutions, in particular the institutions participating in the Gaia Multilateral Agreement. The Gaia mission website is http://www.cosmos.esa.int/gaia. The authors are current or past members of the ESA Gaia mission team and of the Gaia DPAC. This work was partially supported by the MINECO (Spanish Ministry of Economy) through grant ESP2016-80079-C2-1-R and RTI2018-095076-B-C21 (MINECO/FEDER, UE), and MDM-2014-0369 of ICCUB (Unidad de Excelencia “María de Maeztu”). This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement H2020-MSCA-COFUND-2016-754433. This work has been partially supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), by Generalitat de Catalunya (contract 2014-SGR-1051). The research leading to these results has also received funding from the collaboration between Fujitsu and BSC (Script Language Platform). L.C. acknowledges support from “programme national de physique stellaire” (PNPS) and from the “programme national cosmologie et galaxies”.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Early Data Release 3, The celestial reference frame (Gaia-CRF3)

  • Klioner, S. A.
  • Bernet, Marcel
  • Castañeda, J.
  • Clotet, M.
  • Fabricius, Claus
  • González-Vidal, J. J.
  • Portell, Jordi
  • Torra, Ferran
  • Torra, Jordi
  • Jordi, Carme
  • Luri, Xavier
  • Masana, Eduard
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration: Klioner, S. A. et al., [Context] Gaia-CRF3 is the celestial reference frame for positions and proper motions in the third release of data from the Gaia mission, Gaia DR3 (and for the early third release, Gaia EDR3, which contains identical astrometric results). The reference frame is defined by the positions and proper motions at epoch 2016.0 for a specific set of extragalactic sources in the (E)DR3 catalogue., [Aims] We describe the construction of Gaia-CRF3 and its properties in terms of the distributions in magnitude, colour, and astrometric quality., [Methods] Compact extragalactic sources in Gaia DR3 were identified by positional cross-matching with 17 external catalogues of quasi-stellar objects (QSO) and active galactic nuclei (AGN), followed by astrometric filtering designed to remove stellar contaminants. Selecting a clean sample was favoured over including a higher number of extragalactic sources. For the final sample, the random and systematic errors in the proper motions are analysed, as well as the radio-optical offsets in position for sources in the third realisation of the International Celestial Reference Frame (ICRF3)., [Results] Gaia-CRF3 comprises about 1.6 million QSO-like sources, of which 1.2 million have five-parameter astrometric solutions in Gaia DR3 and 0.4 million have six-parameter solutions. The sources span the magnitude range G = 13–21 with a peak density at 20.6 mag, at which the typical positional uncertainty is about 1 mas. The proper motions show systematic errors on the level of 12 µas yr–1 on angular scales greater than 15 deg. For the 3142 optical counterparts of ICRF3 sources in the S/X frequency bands, the median offset from the radio positions is about 0.5 mas, but it exceeds 4 mas in either coordinate for 127 sources. We outline the future of Gaia-CRF in the next Gaia data releases. Appendices give further details on the external catalogues used, how to extract information about the Gaia-CRF3 sources, potential (Galactic) confusion sources, and the estimation of the spin and orientation of an astrometric solution., The Gaia mission and data processing have financially been supported by, in alphabetical order by country: the Algerian Centre de Recherche en Astronomie, Astrophysique et Géophysique of Bouzareah Observatory; the Austrian Fonds zur Förderung der wissenschaftlichen Forschung (FWF) Hertha Firnberg Programme through grants T359, P20046, and P23737; the BELgian federal Science Policy Office (BELSPO) through various PROgramme de Développement d’Expériences scientifiques (PRODEX) grants and the Polish Academy of Sciences - Fonds Wetenschappelijk Onderzoek through grant VS.091.16N, and the Fonds de la Recherche Scientifique (FNRS), and the Research Council of Katholieke Universiteit (KU) Leuven through grant C16/18/005 (Pushing AsteRoseismology to the next level with TESS, GaiA, and the Sloan DIgital Sky SurvEy – PARADISE); the Brazil-France exchange programmes Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) and Coordenação de Aperfeicoamento de Pessoal de Nível Superior (CAPES) – Comité Français d’Evaluation de la Coopération Universitaire et Scientifique avec le Brésil (COFECUB); the Chilean Agencia Nacional de Investigación y Desarrollo (ANID) through Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT) Regular Project 1210992 (L. Chemin); the National Natural Science Foundation of China (NSFC) through grants 11573054, 11703065, and 12173069, the China Scholarship Council through grant 201806040200, and the Natural Science Foundation of Shanghai through grant 21ZR1474100; the Tenure Track Pilot Programme of the Croatian Science Foundation and the École Polytechnique Fédérale de Lausanne and the project TTP-2018-07-1171 ‘Mining the Variable Sky’, with the funds of the Croatian-Swiss Research Programme; the Czech-Republic Ministry of Education, Youth, and Sports through grant LG 15010 and INTER-EXCELLENCE grant LTAUSA18093, and the Czech Space Office through ESA PECS contract 98058; the Danish Ministry of Science; the Estonian Ministry of Education and Research through grant IUT40-1; the European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTN-CT-2006-033481), through Marie Curie project PIOF-GA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transfer-of-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Sklodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Sub-percent calibration of the extra-galactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE); the European Science Foundation (ESF), in the framework of the Gaia Research for European Astronomy Training Research Network Programme (GREAT-ESF); the European Space Agency (ESA) in the framework of the Gaia project, through the Plan for European Cooperating States (PECS) programme through contracts C98090 and 4000106398/12/NL/KML for Hungary, through contract 4000115263/15/NL/IB for Germany, and through PROgramme de Développement d’Expériences scientifiques (PRODEX) grant 4000127986 for Slovenia; the Academy of Finland through grants 299543, 307157, 325805, 328654, 336546, and 345115 and the Magnus Ehrnrooth Foundation; the French Centre National d’Études Spatiales (CNES), the Agence Nationale de la Recherche (ANR) through grant ANR-10-IDEX-0001-02 for the ‘Investissements d’avenir’ programme, through grant ANR-15-CE31-0007 for project ‘Modelling the Milky Way in the Gaia era’ (MOD4Gaia), through grant ANR-14-CE33-0014-01 for project ‘The Milky Way disc formation in the Gaia era’ (ARCHEOGAL), through grant ANR-15-CE31-0012-01 for project ‘Unlocking the potential of Cepheids as primary distance calibrators’ (UnlockCepheids), through grant ANR-19-CE31-0017 for project ‘Secular evolution of galxies’ (SEGAL), and through grant ANR-18-CE31-0006 for project ‘Galactic Dark Matter’ (GaDaMa), the Centre National de la Recherche Scientifique (CNRS) and its SNO Gaia of the Institut des Sciences de l’Univers (INSU), its Programmes Nationaux: Cosmologie et Galaxies (PNCG), Gravitation Références Astronomie Métrologie (PNGRAM), Planétologie (PNP), Physique et Chimie du Milieu Interstellaire (PCMI), and Physique Stellaire (PNPS), the ‘Action Fédératrice Gaia’ of the Observatoire de Paris, the Région de Franche-Comté, the Institut National Polytechnique (INP) and the Institut National de Physique nucléaire et de Physique des Particules (IN2P3) co-funded by CNES; the German Aerospace Agency (Deutsches Zentrum für Luft- und Raumfahrt e.V., DLR) through grants 50QG0501, 50QG0601, 50QG0602, 50QG0701, 50QG0901, 50QG1001, 50QG1101, 50QG1401, 50QG1402, 50QG1403, 50QG1404, 50QG1904, 50QG2101, 50QG2102, and 50QG2202, and the Centre for Information Services and High Performance Computing (ZIH) at the Technische Universität Dresden for generous allocations of computer time; the Hungarian Academy of Sciences through the Lendület Programme grants LP2014-17 and LP2018-7 and the Hungarian National Research, Development, and Innovation Office (NKFIH) through grant KKP-137523 (‘SeismoLab’); the Science Foundation Ireland (SFI) through a Royal Society - SFI University Research Fellowship (M. Fraser); the Israel Ministry of Science and Technology through grant 3-18143 and the Tel Aviv University Center for Artificial Intelligence and Data Science (TAD) through a grant; the Agenzia Spaziale Italiana (ASI) through contracts I/037/08/0, I/058/10/0, 2014-025-R.0, 2014-025-R.1.2015, and 2018-24-HH.0 to the Italian Istituto Nazionale di Astrofisica (INAF), contract 2014-049-R.0/1/2 to ΓNAF for the Space Science Data Centre (SSDC, formerly known as the ASI Science Data Center, ASDC), contracts I/008/10/0, 2013/030/I.0, 2013-030-I.0.1-2015, and 2016-17-I.0 to the Aerospace Logistics Technology Engineering Company (ALTEC S.p.A.), INAF, and the Italian Ministry of Education, University, and Research (Ministero dell’Istruzione, dell’Università e della Ricerca) through the Premiale project ‘Mining The Cosmos Big Data and Innovative Italian Technology for Frontier Astrophysics and Cosmology’ (MITiC); the Netherlands Organisation for Scientific Research (NWO) through grant NWO-M-614.061.414, through a VICI grant (A. Helmi), and through a Spinoza prize (A. Helmi), and the Netherlands Research School for Astronomy (NOVA); the Polish National Science Centre through HARMONIA grant 2018/30/M/ST9/00311 and DAINA grant 2017/27/L/ST9/03221 and the Ministry of Science and Higher Education (MNiSW) through grant DIR/WK/2018/12; the Portuguese Fundação para a Ciência e a Tecnologia (FCT) through national funds, grants SFRH/BD/128840/2017 and PTDC/FIS-AST/30389/2017, and work contract DL 57/2016/CP1364/CT0006, the Fundo Europeu de Desenvolvimento Regional (FEDER) through grant POCI-01-0145-FEDER-030389 and its Programa Operacional Competitividade e Internacionalização (COMPETE2020) through grants UIDB/04434/2020 and UIDP/04434/2020, and the Strategic Programme UIDB/00099/2020 for the Centro de Astrof sica e Gravitação (CENTRA); the Slovenian Research Agency through grant P1-0188; the Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-B-C22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘Mar a de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnolog as de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre – Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’); the Swedish National Space Agency (SNSA/Rymdstyrelsen); the Swiss State Secretariat for Education, Research, and Innovation through the Swiss Activités Nationales Complémentaires and the Swiss National Science Foundation through an Eccellenza Professorial Fellowship (award PCEFP2_194638 for R. Anderson); the United Kingdom Particle Physics and Astronomy Research Council (PPARC), the United Kingdom Science and Technology Facilities Council (STFC), and the United Kingdom Space Agency (UKSA) through the following grants to the University of Bristol, the University of Cambridge, the University of Edinburgh, the University of Leicester, the Mullard Space Sciences Laboratory of University College London, and the United Kingdom Rutherford Appleton Laboratory (RAL): PP/D006511/1, PP/D006546/1, PP/D006570/1, ST/I000852/1, ST/J005045/1, ST/K00056X/1, ST/K000209/1, ST/K000756/1, ST/L006561/1, ST/N000595/1, ST/N000641/1, ST/N000978/1, ST/N001117/1, ST/S000089/1, ST/S000976/1, ST/S000984/1, ST/S001123/1, ST/S001948/1, ST/S001980/1, ST/S002103/1, ST/V000969/1, ST/W002469/1, ST/W002493/1, ST/W002671/1, ST/W002809/1, and EP/V520342/1. Further acknowledgments can be found at https://gea.esac.esa.int/archive/documentation/GEDR3/Miscellaneous/sec_acknowl/., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: A golden sample of astrophysical parameters

  • Creevey, O. L.
  • Jordi, Carme
  • Luri, Xavier
  • Masana, Eduard
  • Portell, Jordi
  • Fabricius, Claus
  • Figueras, Francesca
  • Anders, Friedrich
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Castañeda, J.
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Siddiqui, Hassan
  • Solano, Enrique
  • Torra, Ferran
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration, et al., [Context] Gaia Data Release 3 (DR3) provides a wealth of new data products for the astronomical community to exploit, including astrophysical parameters for half a billion stars. In this work, we demonstrate the high quality of these data products and illustrate their use in different astrophysical contexts., [Aims] We produce homogeneous samples of stars with high-quality astrophysical parameters by exploiting Gaia DR3, while focusing on many regimes across the Hertzsprung-Russell (HR) diagram; spectral types OBA, FGKM, and ultracool dwarfs (UCDs). We also focus on specific subsamples of particular interest to the community: solar analogues, carbon stars, and the Gaia spectrophotometric standard stars (SPSS)., [Methods] We query the astrophysical parameter tables along with other tables in Gaia DR3 to derive the samples of the stars of interest. We validate our results using the Gaia catalogue itself and by comparison with external data., [Results] We produced six homogeneous samples of stars with high-quality astrophysical parameters across the HR diagram for the community to exploit. We first focus on three samples that span a large parameter space: young massive disc stars (OBA; about 3 Million), FGKM spectral type stars (about 3 Million), and UCDs (about 20 000). We provide these sources along with additional information (either a flag or complementary parameters) as tables that are made available in the Gaia archive. We also identify 15 740 bone fide carbon stars and 5863 solar analogues, and provide the first homogeneous set of stellar parameters of the SPSS sample. We demonstrate some applications of these samples in different astrophysical contexts. We use a subset of the OBA sample to illustrate its usefulness in analysing the Milky Way rotation curve. We then use the properties of the FGKM stars to analyse known exoplanet systems. We also analyse the ages of some unseen UCD-companions to the FGKM stars. We additionally predict the colours of the Sun in various passbands (Gaia, 2MASS, WISE) using the solar-analogue sample., [Conclusions] Gaia DR3 contains a wealth of new high-quality astrophysical parameters for the community to exploit., This work presents results from the European Space Agency (ESA) space mission Gaia. Gaia data are processed by the Gaia Data Processing and Analysis Consortium (DPAC). Funding for the DPAC is provided by national institutions, in particular the institutions participating in the Gaia MultiLateral Agreement (MLA). The Gaia mission website is https://www. cosmos.esa.int/gaia. The Gaia archive website is https://archives.esac.esa.int/gaia., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES 2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-B-C22, and TIN2015-65316-P (‘Com putación de Altas Prestaciones VII’), the Juan de la Cierva Incor poración Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Devel opment Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departa ment d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Chemical cartography of the Milky Way

  • Recio-Blanco, A.
  • Antoja, Teresa
  • Cantat-Gaudin, Tristan
  • Jordi, Carme
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Solano, Enrique
  • Torra, Ferran
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration: et al., [Context] The motion of stars has been used to reveal details of the complex history of the Milky Way, in constant interaction with its environment. Nevertheless, to reconstruct the Galactic history puzzle in its entirety, the chemo-physical characterisation of stars is essential. Previous Gaia data releases were supported by a smaller, heterogeneous, and spatially biased mixture of chemical data from ground-based observations., [Aims] Gaia Data Release 3 opens a new era of all-sky spectral analysis of stellar populations thanks to the nearly 5.6 million stars observed by the Radial Velocity Spectrometer (RVS) and parametrised by the GSP-Spec module. In this work, we aim to demonstrate the scientific quality of Gaia’s Milky Way chemical cartography through a chemo-dynamical analysis of disc and halo populations., [Methods] Stellar atmospheric parameters and chemical abundances provided by Gaia DR3 spectroscopy are combined with DR3 radial velocities and EDR3 astrometry to analyse the relationships between chemistry and Milky Way structure, stellar kinematics, and orbital parameters., [Results] The all-sky Gaia chemical cartography allows a powerful and precise chemo-dynamical view of the Milky Way with unprecedented spatial coverage and statistical robustness. First, it reveals the strong vertical symmetry of the Galaxy and the flared structure of the disc. Second, the observed kinematic disturbances of the disc – seen as phase space correlations – and kinematic or orbital substructures are associated with chemical patterns that favour stars with enhanced metallicities and lower [α/Fe] abundance ratios compared to the median values in the radial distributions. This is detected both for young objects that trace the spiral arms and older populations. Several α, iron-peak elements and at least one heavy element trace the thin and thick disc properties in the solar cylinder. Third, young disc stars show a recent chemical impoverishment in several elements. Fourth, the largest chemo-dynamical sample of open clusters analysed so far shows a steepening of the radial metallicity gradient with age, which is also observed in the young field population. Finally, the Gaia chemical data have the required coverage and precision to unveil galaxy accretion debris and heated disc stars on halo orbits through their [α/Fe] ratio, and to allow the study of the chemo-dynamical properties of globular clusters., [Conclusions] Gaia DR3 chemo-dynamical diagnostics open new horizons before the era of ground-based wide-field spectroscopic surveys. They unveil a complex Milky Way that is the outcome of an eventful evolution, shaping it to the present day., This work presents results from the European Space Agency (ESA) space mission Gaia (https://www.cosmos.esa.int/gaia). Gaia data are processed by the Gaia Data Processing and Analysis Consortium (DPAC). Funding for the DPAC is provided by national institutions, in particular the institutions participating in the Gaia MultiLateral Agreement (MLA). The Gaia archive website is https://archives.esac.esa.int/gaia., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTN-CT-2006-033481), through Marie Curie project PIOF-GA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transfer-of-Knowledge (ToK) fellow ship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of selfgravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Sub-percent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-B-C22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Exploring and mapping the diffuse interstellar band at 862 nm

  • Schultheis, Mathias
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Ferran
  • Torra, Jordi
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration: et al., [Context] Diffuse interstellar bands (DIBs) are common interstellar absorption features in spectroscopic observations but their origins remain unclear. DIBs play an important role in the life cycle of the interstellar medium (ISM) and can also be used to trace Galactic structure., [Aims] Here, we demonstrate the capacity of the Gaia-Radial Velocity Spectrometer (RVS) in Gaia DR3 to reveal the spatial distribution of the unknown molecular species responsible for the most prominent DIB at 862 nm in the RVS passband, exploring the Galactic ISM within a few kiloparsecs from the Sun., [Methods] The DIBs are measured within the GSP-Spec module using a Gaussian profile fit for cool stars and a Gaussian process for hot stars. In addition to the equivalent widths and their uncertainties, Gaia DR3 provides their characteristic central wavelength, width, and quality flags., [Results] We present an extensive sample of 476 117 individual DIB measurements obtained in a homogeneous way covering the entire sky. We compare spatial distributions of the DIB carrier with interstellar reddening and find evidence that DIB carriers are present in a local bubble around the Sun which contains nearly no dust. We characterised the DIB equivalent width with a local density of 0.19 ± 0.04 Å kpc−1 and a scale height of 98.60−8.46+11.10 pc. The latter is smaller than the dust scale height, indicating that DIBs are more concentrated towards the Galactic plane. We determine the rest-frame wavelength with unprecedented precision (λ0 = 8620.86  ± 0.019 Å in air) and reveal a remarkable correspondence between the DIB velocities and the CO gas velocities, suggesting that the 862 nm DIB carrier is related to macro-molecules., [Conclusions] We demonstrate the unique capacity of Gaia to trace the spatial structure of the Galactic ISM using the 862 nm DIB., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE); – the European Science Foundation (ESF), in the framework of the Gaia Research for European Astronomy Training Research Network Programme (GREAT-ESF)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: External calibration of BP/RP low-resolution spectroscopic data

  • Montegriffo, P.
  • Carrasco, J. M.
  • Jordi, Carme
  • Fabricius, Claus
Montegriffo, P., et al., [Context] Gaia Data Release 3 contains astrometry and photometry results for about 1.8 billion sources based on observations collected by the European Space Agency (ESA) Gaia satellite during the first 34 months of its operational phase (the same period covered by Gaia early Data Release 3; Gaia EDR3). Low-resolution spectra for 220 million sources are one of the important new data products included in this release., [Aims] In this paper, we focus on the external calibration of low-resolution spectroscopic content, describing the input data, algorithms, data processing, and the validation of the results. Particular attention is given to the quality of the data and to a number of features that users may need to take into account to make the best use of the catalogue., [Methods] We calibrated an instrument model to relate mean Gaia spectra to the corresponding spectral energy distributions (SEDs) using an extended set of calibrators: this includes modelling of the instrument dispersion relation, transmission, and line spread functions. Optimisation of the model is achieved through total least-squares regression, accounting for errors in Gaia and external spectra., [Results] The resulting instrument model can be used for forward modelling of Gaia spectra or for inverse modelling of externally calibrated spectra in absolute flux units., [Conclusions] The absolute calibration derived in this paper provides an essential ingredient for users of BP/RP spectra. It allows users to connect BP/RP spectra to absolute fluxes and physical wavelengths., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excel lent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE); the European Science Foundation (ESF), in the framework of the Gaia Research for European Astronomy Training Research Network Programme (GREAT-ESF); the Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015- 0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/ 155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Gaia scan-angle-dependent signals and spurious periods

  • Holl, Berry
  • Fabricius, Claus
  • Portell, Jordi
  • Lindegren, Lennart
  • Panuzzo, Pasquale
  • Bernet, Marcel
  • Castañeda, J.
  • Jevardat de Fombelle, G.
  • Audard, Marc
  • Ducourant, Christine
  • Harrison, Diana L.
  • Evans, Dafydd W.
  • Busso, Giorgia
  • Sozzetti, Alessandro
  • Gosset, E.
  • Arenou, Frédéric
  • Angeli, Francesca de
  • Riello, Marco
  • Eyer, Laurent
  • Rimoldini, Lorenzo
  • Gavras, Panagiotis
  • Mowlavi, Nami
  • Nienartowicz, K.
  • Lecoeur-Taïbi, I.
  • García Lario, Pedro
  • Pourbaix, Dimitri
[Context] Gaia Data Release 3 (Gaia DR3) time series data may contain spurious signals related to the time-dependent scan angle., [Aims] We aim to explain the origin of scan-angle-dependent signals and how they can lead to spurious periods, provide statistics to identify them in the data, and suggest how to deal with them in Gaia DR3 data and in future releases., [Methods] Using real Gaia (DR3) data alongside numerical and analytical models, we visualise and explain the features observed in the data., [Results] We demonstrated with Gaia (DR3) data that source structure (multiplicity or extendedness) or pollution from close-by bright objects can cause biases in the image parameter determination from which photometric, astrometric, and (indirectly) radial velocity time series are derived. These biases are a function of the time-dependent scan direction of the instrument and thus can introduce scan-angle-dependent signals, which due to the scanning-law-induced sampling of Gaia can result in specific spurious periodic signals. Numerical simulations in which a period search is performed on Gaia time series with a scan-angle-dependent signal qualitatively reproduce the general structure observed in the spurious period distribution of photometry and astrometry, and the associated spatial distributions on the sky. A variety of statistics allows for the deeper understanding and identification of affected sources., [Conclusions] The origin of the scan-angle-dependent signals and subsequent spurious periods is well understood and is mostly caused by fixed-orientation optical pairs with a separation < 0.5″ (including binaries with P ≫ 5 y) and (cores of) distant galaxies. Although most of the sources with affected derived parameters have been filtered out from the Gaia archive nss_two_body_orbit and several vari-tables, Gaia DR3 data remain that should be treated with care (no sources were filtered from gaia_source). Finally, the various statistics discussed in the paper can be used to identify and filter affected sources and also reveal new information about them that is not available through other means, especially in terms of binarity on sub-arcsecond scale., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) and grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Sub-percent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER,UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral pro gramme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional DevelopmentFund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre – Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded byMICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: G RVSphotometry from the RVS spectra

  • Sartoretti, Paola
  • Jordi, Carme
Sartoretti, P., et al., [Context] Gaia Data Release 3 (DR3) contains the first release of magnitudes estimated from the integration of Radial Velocity Spectrometer (RVS) spectra for a sample of about 32.2 million stars brighter than GRVS ∼ 14 mag (or G ∼ 15 mag)., [Aims] In this paper, we describe the data used and the approach adopted to derive and validate the GRVS magnitudes published in DR3. We also provide estimates of the GRVS passband and associated GRVS zero-point., [Methods] We derived GRVS photometry from the integration of RVS spectra over the wavelength range from 846 to 870 nm. We processed these spectra following a procedure similar to that used for DR2, but incorporating several improvements that allow a better estimation of GRVS. These improvements pertain to the stray-light background estimation, the line spread function calibration, and the detection of spectra contaminated by nearby relatively bright sources. We calibrated the GRVS zero-point every 30 h based on the reference magnitudes of constant stars from the HIPPARCOS catalogue, and used them to transform the integrated flux of the cleaned and calibrated spectra into epoch magnitudes. The GRVS magnitude of a star published in DR3 is the median of the epoch magnitudes for that star. We estimated the GRVS passband by comparing the RVS spectra of 108 bright stars with their flux-calibrated spectra from external spectrophotometric libraries., [Results] The GRVS magnitude provides information that is complementary to that obtained from the G, GBP, and GRP magnitudes, which is useful for constraining stellar metallicity and interstellar extinction. The median precision of GRVS measurements ranges from about 0.006 mag for the brighter stars (i.e. with 3.5≲ GRVS ≲6.5 mag) to 0.125 mag at the faint end. The derived GRVS passband shows that the effective transmittance of the RVS is approximately 1.23 times better than the pre-launch estimate., This work presents results from the European Space Agency (ESA) space mission Gaia. The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 745617 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Mapping the asymmetric disc of the Milky Way

  • Drimmel, Ronald
  • Romero-Gómez, Mercè
  • Ramos, Pau
  • Cantat-Gaudin, Tristan
  • Figueras, Francesca
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Carrasco, J. M.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Ferran
  • Torra, Jordi
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration: et al., [Context] With the most recent Gaia data release, the number of sources with complete 6D phase space information (position and velocity) has increased to well over 33 million stars, while stellar astrophysical parameters are provided for more than 470 million sources, and more than 11 million variable stars are identified., [Aims] Using the astrophysical parameters and variability classifications provided in Gaia DR3, we selected various stellar populations to explore and identify non-axisymmetric features in the disc of the Milky Way in configuration and velocity space., [Methods] Using more about 580 000 sources identified as hot OB stars, together with 988 known open clusters younger than 100 Myr, we mapped the spiral structure associated with star formation 4−5 kpc from the Sun. We selected over 2800 Classical Cepheids younger than 200 Myr that show spiral features extending as far as 10 kpc from the Sun in the outer disc. We also identified more than 8.7 million sources on the red giant branch (RGB), of which 5.7 million have line-of-sight velocities. This later sample allows the velocity field of the Milky Way to be mapped as far as 8 kpc from the Sun, including the inner disc., [Results] The spiral structure revealed by the young populations is consistent with recent results using Gaia EDR3 astrometry and source lists based on near-infrared photometry, showing the Local (Orion) Arm to be at least 8 kpc long, and an outer arm consistent with what is seen in HI surveys, which seems to be a continuation of the Perseus arm into the third quadrant. The subset of RGB stars with velocities clearly reveals the large-scale kinematic signature of the bar in the inner disc, as well as evidence of streaming motions in the outer disc that might be associated with spiral arms or bar resonances. A local comparison of the velocity field of the OB stars reveals similarities and differences with the RGB sample., [Conclusions] This cursory study of Gaia DR3 data shows there is a rich bounty of kinematic information to be explored more deeply, which will undoubtedly lead us to a clearer understanding of the dynamical nature of the non-axisymmetric structures of the Milky Way., This work presents results from the European Space Agency (ESA) space mission Gaia., The European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre – Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Processing and validation of BP/RP low-resolution spectral data

  • Angeli, Francesca de
  • Weiler, Michael
  • Carrasco, J. M.
  • Jordi, Carme
  • Fabricius, Claus
De Angeli, F., et al., [Context] Blue (BP) and Red (RP) Photometer low-resolution spectral data are one of the exciting new products in Gaia Data Release 3 (Gaia DR3). These data have also been used to derive astrometry and integrated photometry in Gaia Early Data Release 3 and astrophysical parameters and Solar System object reflectance spectra in Gaia DR3., [Aims] In this paper, we give an overview of the processing techniques that allow raw satellite data of multiple transits per source to be converted into combined spectra calibrated to an internal reference system, resulting in low-resolution BP and RP mean spectra. We describe how we overcome challenges due to the complexity of the on-board instruments and to the various observation strategies. Furthermore, we show highlights from our scientific validation of the results. This work covers the internal calibration of BP/RP spectra to a self-consistent mean instrument, while the calibration of the BP/RP spectra to the absolute reference system of physical flux and wavelength is covered by one of the accompanying Gaia DR3 papers., [Methods] We calibrate about 65 billion individual transit spectra onto the same mean BP/RP instrument through a series of calibration steps, including background subtraction, calibration of the CCD geometry, and an iterative procedure for the calibration of CCD efficiency as well as variations of the line-spread function and dispersion across the focal plane and in time. The calibrated transit spectra are then combined for each source in terms of an expansion into continuous basis functions. We discuss the configuration of these basis functions., [Results] Time-averaged mean spectra covering the optical to near-infrared wavelength range [330, 1050] nm are published for approximately 220 million objects. Most of these are brighter than G = 17.65 but some BP/RP spectra are published for sources down to G = 21.43. Their signal-to-noise ratio (S/N) varies significantly over the wavelength range covered, and with magnitude and colour of the observed objects, with sources around G = 15 having a S/N above 100 in some wavelength ranges. The top-quality BP/RP spectra are achieved for sources with magnitudes 9 <  G <  12, with S/N reaching 1000 in the central part of the RP wavelength range. Scientific validation suggests that the internal calibration was generally successful. However, there is some evidence for imperfect calibrations at the bright end G <  11, where calibrated BP/RP spectra can exhibit systematic flux variations that exceed their estimated flux uncertainties. We also report that, due to long-range noise correlations, BP/RP spectra can exhibit wiggles when sampled in pseudo-wavelength., [Conclusions] The Gaia DR3 data products are the expansion coefficients and corresponding covariance matrices for BP and RP separately. Users are encouraged to work with the data in this format, with full covariance information showing that correlations between coefficients are typically very low. Documentation and instructions on how to access and use BP/RP spectral data from the archive are also provided., This work presents results from the European Space Agency (ESA) space mission Gaia., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Sub-percent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-B-C22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Properties and validation of the radial velocities

  • Katz, D.
  • Antoja, Teresa
  • Fabricius, Claus
Katz, D., et al., [Context] Gaia Data Release 3 (Gaia DR3) contains the second release of the combined radial velocities. It is based on the spectra collected during the first 34 months of the nominal mission. The longer time baseline and the improvements of the pipeline made it possible to push the processing limit from GRVS = 12 in Gaia DR2 to GRVS = 14 mag., [Aims] We describe the new functionalities implemented for Gaia DR3, the quality filters applied during processing and post-processing, and the properties and performance of the published velocities., [Methods] For Gaia DR3, several functionalities were upgraded or added to the spectroscopic pipeline. The calibrations were improved in order to better model the temporal evolution of the straylight and of the instrumental point spread function (PSF). The overlapped spectra, which were mostly discarded in Gaia DR2, are now handled by a dedicated module. The hot star template mismatch, which prevented publication of hot stars in Gaia DR2, is largely mitigated now, down to GRVS = 12 mag. The combined radial velocity of stars brighter than or equal to GRVS = 12 mag is calculated in the same way as in Gaia DR2, that is, as the median of the epoch radial velocity time series. The combined radial velocity of the fainter stars is measured from the average of the cross-correlation functions., [Results] Gaia DR3 contains the combined radial velocities of 33 812 183 stars. With respect to Gaia DR2, the temperature interval has been expanded from Teff ∈ [3600, 6750] K to Teff ∈ [3100, 14 500] K for the bright stars (GRVS ≤ 12 mag) and [3100, 6750] K for the fainter stars. The radial velocities sample a significant part of the Milky Way: they reach a few kiloparsecs beyond the Galactic centre in the disc and up to about 10−15 kpc vertically into the inner halo. The median formal precision of the velocities is 1.3 km s−1 at GRVS = 12 and 6.4 km s−1 at GRVS = 14 mag. The velocity zeropoint exhibits a small systematic trend with magnitude that starts around GRVS = 11 mag and reaches about 400 m s−1 at GRVS = 14 mag. A correction formula is provided that can be applied to the published data. The Gaia DR3 velocity scale agrees satisfactorily with APOGEE, GALAH, GES, and RAVE; the systematic differences mostly remain below a few hundred m s−1. The properties of the radial velocities are also illustrated with specific objects: open clusters, globular clusters, and the Large Magellanic Cloud. For example, the precision of the data allows mapping the line-of-sight rotational velocities of the globular cluster 47 Tuc and of the Large Magellanic Cloud., This work presents results from the European Space Agency (ESA) space mission Gaia. Gaia data are being processed by the Gaia Data Processing and Analysis Consortium (DPAC)., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Pulsations in main sequence OBAF-type stars

  • De Ridder, Joris
  • Jordi, Carme
  • Luri, Xavier
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Castañeda, J.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Ferran
  • Torra, Jordi
  • Weiler, Michael
  • Gaia Collaboration
De Ridder, Joris et al., [Context] The third Gaia data release provides photometric time series covering 34 months for about 10 million stars. For many of those stars, a characterisation in Fourier space and their variability classification are also provided. This paper focuses on intermediate- to high-mass (IHM) main sequence pulsators (M ≥  1.3 M⊙) of spectral types O, B, A, or F, known as β Cep, slowly pulsating B (SPB), δ Sct, and γ Dor stars. These stars are often multi-periodic and display low amplitudes, making them challenging targets to analyse with sparse time series., [Aims] We investigate the extent to which the sparse Gaia DR3 data can be used to detect OBAF-type pulsators and discriminate them from other types of variables. We aim to probe the empirical instability strips and compare them with theoretical predictions. The most populated variability class is that of the δ Sct variables. For these stars, we aim to confirm their empirical period-luminosity (PL) relation, and verify the relation between their oscillation amplitude and rotation., [Methods] All datasets used in this analysis are part of the Gaia DR3 data release. The photometric time series were used to perform a Fourier analysis, while the global astrophysical parameters necessary for the empirical instability strips were taken from the Gaia DR3 gspphot tables, and the v sin i data were taken from the Gaia DR3 esphs tables. The δ Sct PL relation was derived using the same photometric parallax method as the one recently used to establish the PL relation for classical Cepheids using Gaia data., [Results] We show that for nearby OBAF-type pulsators, the Gaia DR3 data are precise and accurate enough to pinpoint them in the Hertzsprung-Russell (HR) diagram. We find empirical instability strips covering broader regions than theoretically predicted. In particular, our study reveals the presence of fast rotating gravity-mode pulsators outside the strips, as well as the co-existence of rotationally modulated variables inside the strips as reported before in the literature. We derive an extensive period–luminosity relation for δ Sct stars and provide evidence that the relation features different regimes depending on the oscillation period. We demonstrate how stellar rotation attenuates the amplitude of the dominant oscillation mode of δ Sct stars., [Conclusions] The Gaia DR3 time-series photometry already allows for the detection of the dominant (non-)radial oscillation mode in about 100 000 intermediate- and high-mass dwarfs across the entire sky. This detection capability will increase as the time series becomes longer, allowing the additional delivery of frequencies and amplitudes of secondary pulsation modes., This work presents results from the European Space Agency (ESA) space mission Gaia. The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia(MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innova tion (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Gali cia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund Galicia 2014-2020 Programme), through grant ED431G2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT 2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Stellar multiplicity, a teaser for the hidden treasure

  • Arenou, Frédéric
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Portell, Jordi
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Torra, Jordi
  • Gaia Collaboration
Gaia Collaboration: et al., [Context] The Gaia DR3 catalogue contains, for the first time, about 800 000 solutions with either orbital elements or trend parameters for astrometric, spectroscopic, and eclipsing binaries, and combinations of these three., [Aims] With this paper, we aim to illustrate the huge potential of this large non-single-star catalogue., [Methods] Using the orbital solutions and models of the binaries, we have built a catalogue of tens of thousands of stellar masses or lower limits thereof, some with consistent flux ratios. Properties concerning the completeness of the binary catalogues are discussed, statistical features of the orbital elements are explained, and a comparison with other catalogues is performed., [Results] Illustrative applications are proposed for binaries across the Hertzsprung-Russell Diagram (HRD). Binarity is studied in the giant branch and a search for genuine spectroscopic binaries among long-period variables is performed. The discovery of new EL CVn systems illustrates the potential of combining variability and binarity catalogues. Potential compact object companions are presented, mainly white dwarf companions or double degenerates, but one candidate neutron star is also found. Towards the bottom of the main sequence, the orbits of previously suspected binary ultracool dwarfs are determined and new candidate binaries are discovered. The long awaited contribution of Gaia to the analysis of the substellar regime shows the brown dwarf desert around solar-type stars using true rather than minimum masses, and provides new important constraints on the occurrence rates of substellar companions to M dwarfs. Several dozen new exoplanets are proposed, including two with validated orbital solutions and one super-Jupiter orbiting a white dwarf, all being candidates requiring confirmation. Besides binarity, higher order multiple systems are also found., [Conclusions] By increasing the number of known binary orbits by more than one order of magnitude, Gaia DR3 will provide a rich reservoir of dynamical masses and an important contribution to the analysis of stellar multiplicity., This work presents results from the European Space Agency (ESA) space mission Gaia. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. [951549]). The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excel lent science programmes through Marie Skłodowska-Curie. grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918- M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre – Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Sci ence Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918- M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: Summary of the content and survey properties

  • Vallenari, A.
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Figueras, Francesca
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Ferran
  • Torra, Jordi
  • Weiler, Michael
  • Gaia Collaboration
Gaia Collaboration: et al., [Context] We present the third data release of the European Space Agency’s Gaia mission, Gaia DR3. This release includes a large variety of new data products, notably a much expanded radial velocity survey and a very extensive astrophysical characterisation of Gaia sources., [Aims] We outline the content and the properties of Gaia DR3, providing an overview of the main improvements in the data processing in comparison with previous data releases (where applicable) and a brief discussion of the limitations of the data in this release., [Methods] The Gaia DR3 catalogue is the outcome of the processing of raw data collected with the Gaia instruments during the first 34 months of the mission by the Gaia Data Processing and Analysis Consortium., [Results] The Gaia DR3 catalogue contains the same source list, celestial positions, proper motions, parallaxes, and broad band photometry in the G, GBP, and GRP pass-bands already present in the Early Third Data Release, Gaia EDR3. Gaia DR3 introduces an impressive wealth of new data products. More than 33 million objects in the ranges GRVS < 14 and 3100 < Teff < 14 500, have new determinations of their mean radial velocities based on data collected by Gaia. We provide GRVS magnitudes for most sources with radial velocities, and a line broadening parameter is listed for a subset of these. Mean Gaia spectra are made available to the community. The Gaia DR3 catalogue includes about 1 million mean spectra from the radial velocity spectrometer, and about 220 million low-resolution blue and red prism photometer BP/RP mean spectra. The results of the analysis of epoch photometry are provided for some 10 million sources across 24 variability types. Gaia DR3 includes astrophysical parameters and source class probabilities for about 470 million and 1500 million sources, respectively, including stars, galaxies, and quasars. Orbital elements and trend parameters are provided for some 800 000 astrometric, spectroscopic and eclipsing binaries. More than 150 000 Solar System objects, including new discoveries, with preliminary orbital solutions and individual epoch observations are part of this release. Reflectance spectra derived from the epoch BP/RP spectral data are published for about 60 000 asteroids. Finally, an additional data set is provided, namely the Gaia Andromeda Photometric Survey, consisting of the photometric time series for all sources located in a 5.5 degree radius field centred on the Andromeda galaxy., [Conclusions] This data release represents a major advance with respect to Gaia DR2 and Gaia EDR3 because of the unprecedented quantity, quality, and variety of source astrophysical data. To date this is the largest collection of all-sky spectrophotometry, radial velocities, variables, and astrophysical parameters derived from both low- and high-resolution spectra and includes a spectrophotometric and dynamical survey of SSOs of the highest accuracy. The non-single star content surpasses the existing data by orders of magnitude. The quasar host and galaxy light profile collection is the first such survey that is all sky and space based. The astrophysical information provided in Gaia DR3 will unleash the full potential of Gaia’s exquisite astrometric, photometric, and radial velocity surveys., The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral pro gramme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A 2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comuni caciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: The Solar System survey

  • Tanga, P.
  • Portell, Jordi
Tanga, P., et al., [Context] The third data release by the Gaia mission of the European Space Agency (DR3) is the first release to provide the community with a large sample of observations for more than 150 thousand Solar System objects, including asteroids and natural planetary satellites. The release contains astrometry (over 23 million epochs) and photometry, along with average reflectance spectra of 60518 asteroids and osculating elements., [Aims] We present an overview of the procedures that have been implemented over several years of development and tests to process Solar System data at the level of accuracy that Gaia can reach. We illustrate the data properties and potential with some practical examples., [Methods] In order to allow the users of Gaia DR3 to best exploit the data, we explain the assumptions and approaches followed in the implementation of the data processing pipeline for Solar System processing, and their effects in terms of data filtering, optimisation, and performances. We then test the data quality by analysing post-fit residuals to adjusted orbits, the capacity of detecting subtle dynamical effects (wobbling due to satellites or shape and Yarkovsky acceleration), and to reproduce known properties of asteroid photometry (phase curves and rotational light curves)., [Results] The DR3 astrometric accuracy is a clear improvement over the data published in DR2, which concerned a very limited sample of asteroids. The performance of the data reduction is met, and is illustrated by the capacity of detecting milliarcsecond-level wobbling of the asteroid photocentre that is due to satellite or shape effects and contributes to Yarkovsky effect measurements., [Conclusions] The third data release can in terms of data completeness and accuracy be considered the first full-scale realisation of the Solar System survey by Gaia., This work presents results from the European Space Agency (ESA) space mission Gaia. The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie TransferofKnowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: The extragalactic content

  • Bailer-Jones, Coryn
  • Jordi, Carme
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Antoja, Teresa
  • Balaguer-Nuñez, Lola
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Jordi
  • Weiler, Michael
  • Gaia Collaboration
The Gaia Galactic survey mission is designed and optimized to obtain astrometry, photometry, and spectroscopy of nearly two billion stars in our Galaxy. Yet as an all-sky multi-epoch survey, Gaia also observes several million extragalactic objects down to a magnitude of G ∼ 21 mag. Due to the nature of the Gaia onboard-selection algorithms, these are mostly point-source-like objects. Using data provided by the satellite, we have identified quasar and galaxy candidates via supervised machine learning methods, and estimate their redshifts using the low resolution BP/RP spectra. We further characterise the surface brightness profiles of host galaxies of quasars and of galaxies from pre-defined input lists. Here we give an overview of the processing of extragalactic objects, describe the data products in Gaia DR3, and analyse their properties. Two integrated tables contain the main results for a high completeness, but low purity (50−70%), set of 6.6 million candidate quasars and 4.8 million candidate galaxies. We provide queries that select purer sub-samples of these containing 1.9 million probable quasars and 2.9 million probable galaxies (both ∼95% purity). We also use high quality BP/RP spectra of 43 thousand high probability quasars over the redshift range 0.05−4.36 to construct a composite quasar spectrum spanning restframe wavelengths from 72−1000 nm., This work presents results from the European Space Agency (ESA) space mission Gaia. The European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Development Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A-2019/155, and ED481A-2021/296, the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre – Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Science Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia Data Release 3: The Galaxy in your preferred colours: Synthetic photometry from Gaia low-resolution spectra

  • Montegriffo, P.
  • Bellazzini, M.
  • Carrasco, J. M.
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Antoja, Teresa
  • Bartolomé, S.
  • Bernet, Marcel
  • Cantat-Gaudin, Tristan
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • González-Vidal, J. J.
  • Jiménez-Arranz, Ó.
  • Julbe, F.
  • Madrero Pardo, P.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Mor, Roger
  • Peñalosa Esteller, X.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Solano, Enrique
  • Torra, Ferran
  • Torra, Jordi
  • Gaia Collaboration
Gaia Collaboration: et al., Gaia Data Release 3 provides novel flux-calibrated low-resolution spectrophotometry for ≃220 million sources in the wavelength range 330 nm ≤ λ ≤ 1050 nm (XP spectra). Synthetic photometry directly tied to a flux in physical units can be obtained from these spectra for any passband fully enclosed in this wavelength range. We describe how synthetic photometry can be obtained from XP spectra, illustrating the performance that can be achieved under a range of different conditions – for example passband width and wavelength range – as well as the limits and the problems affecting it. Existing top-quality photometry can be reproduced within a few per cent over a wide range of magnitudes and colour, for wide and medium bands, and with up to millimag accuracy when synthetic photometry is standardised with respect to these external sources. Some examples of potential scientific application are presented, including the detection of multiple populations in globular clusters, the estimation of metallicity extended to the very metal-poor regime, and the classification of white dwarfs. A catalogue providing standardised photometry for ≃2.2 × 108 sources in several wide bands of widely used photometric systems is provided (Gaia Synthetic Photometry Catalogue; GSPC) as well as a catalogue of ≃105 white dwarfs with DA/non-DA classification obtained with a Random Forest algorithm (Gaia Synthetic Photometry Catalogue for White Dwarfs; GSPC-WD)., This work presents results from the European Space Agency (ESA) space mission Gaia. The European Commission's Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTNCT-2006-033481), through Marie Curie project PIOFGA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transferof-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union'?s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grant 745617 (Our Galaxy at full HD – Gal-HD) and 895174 (The build-up and fate of self-gravitating systems in the Universe) as well as grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids – CepBin), 716155 (Structured ACCREtion Disks – SACCRED), 951549 (Subpercent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), and 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE)., The Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MICIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, ESP2016-80079-C2-2-R, FPU16/03827, PDC2021-121059-C22, RTI2018-095076-BC22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493), and MICIN/AEI/10.13039/501100011033 (and the European Union through European Regional Develop ment Fund ‘A way of making Europe’) through grant RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu') through grant CEX2019-000918-M, the University of Barcelona’s official doctoral pro gramme for the development of an R+D+i project through an Ajuts de Personal Investigador en Formació (APIF) grant, the Spanish Virtual Observatory through project AyA2017-84089, the Galician Regional Government, Xunta de Galicia, through grants ED431B-2021/36, ED481A 2019/155, and ED481A-2021/296, the Centro de Investi gación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia and the European Union (European Regional Development Fund – Galicia 2014-2020 Programme), through grant ED431G-2019/01, the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercom putación (BSC-CNS) through activities AECT-2017-2-0002,AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT 2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Univer sitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Progra mació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowship RYC2018-025968-I funded by MICIN/AEI/10.13039/501100011033 and the European Sci ence Foundation (‘Investing in your future’)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia focused product release: Spatial distribution of two diffuse interstellar bands

  • Schultheis, Mathias
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Anders, Friedrich
  • Antoja, Teresa
  • Bartolomé, S.
  • Bernet, Marcel
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • Gregori Barrera, V.
  • Jiménez-Arranz, Ó.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió-Tortajada, Marta
  • Prša, A.
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Sarrate Riera, C.
  • Solano, Enrique
  • Torra, Ferran
  • Weiler, Michael
  • Balaguer-Nuñez, Lola
  • Gaia Collaboration
Gaia Collaboration: et al., Diffuse interstellar bands (DIBs) are absorption features seen in optical and infrared spectra of stars and extragalactic objects that are probably caused by large and complex molecules in the galactic interstellar medium (ISM). Here we investigate the Galactic distribution and properties of two DIBs identified in almost six million stellar spectra collected by the Gaia Radial Velocity Spectrometer. These measurements constitute a part of the Gaia Focused Product Release to be made public between the Gaia DR3 and DR4 data releases. In order to isolate the DIB signal from the stellar features in each individual spectrum, we identified a set of 160 000 spectra at high Galactic latitudes (|b| ⩾ 65°) covering a range of stellar parameters which we consider to be the DIB-free reference sample. Matching each target spectrum to its closest reference spectra in stellar parameter space allowed us to remove the stellar spectrum empirically, without reference to stellar models, leaving a set of six million ISM spectra. Using the star’s parallax and sky coordinates, we then allocated each ISM spectrum to a voxel (VOlume piXEL) on a contiguous three-dimensional grid with an angular size of 1.8° (level 5 HEALPix) and 29 unequally sized distance bins. Identifying the two DIBs at 862.1 nm (λ862.1) and 864.8 nm (λ864.8) in the stacked spectra, we modelled their shapes and report the depth, central wavelength, width, and equivalent width (EW) for each, along with confidence bounds on these measurements. We then explored the properties and distributions of these quantities and compared them with similar measurements from other surveys. Our main results are as follows: (1) the strength and spatial distribution of the DIB λ862.1 are very consistent with what was found in Gaia DR3, but for this work we attained a higher signal-to-noise ratio in the stacked spectra to larger distances, which allowed us to trace DIBs in the outer spiral arm and beyond the Scutum–Centaurus spiral arm; (2) we produced an all-sky map below ±65° of Galactic latitude to ~4000 pc of both DIB features and their correlations; (3) we detected the signals of DIB λ862.1 inside the Local Bubble (≲200 pc); and (4) there is a reasonable correlation with the dust reddening found from stellar absorption and EWs of both DIBs with a correlation coefficient of 0.90 for λ862.1 and 0.77 for λ864.8., This work presents results from the European Space Agency (ESA) space mission Gaia. Gaia data are being processed by the Gaia Data Processing and Analysis Consortium (DPAC). Funding for the DPAC is provided by national institutions, in particular the institutions participating in the Gaia MultiLat eral Agreement (MLA). The Gaia mission website is https: //www.cosmos.esa.int/gaia. The Gaia archive website is https://archives.esac.esa.int/gaia. The Gaia mission and data processing have financially been supported by, in alphabetical order by country: – the Algerian Centre de Recherche en Astronomie, Astro physique et Géophysique of Bouzareah Observatory; – the Austrian Fonds zur Förderung der wissenschaftlichen Forschung (FWF) Hertha Firnberg Programme through grants T359, P20046, and P23737; – the BELgian federal Science Policy Office (BEL SPO) through various PROgramme de Développement d’Expériences scientifiques (PRODEX) grants of the European Space Agency (ESA), the Research Foundation Flanders (Fonds Wetenschappelijk Onderzoek) through grant VS.091.16N, the Fonds de la Recherche Scientifique (FNRS), and the Research Council of Katholieke Univer siteit (KU) Leuven through grant C16/18/005 (Pushing AsteRoseismology to the next level with TESS, GaiA, and the Sloan DIgital Sky SurvEy – PARADISE); – the Brazil-France exchange programmes Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) and Coordenação de Aperfeicoamento de Pessoal de Nível Supe rior (CAPES) - Comité Français d’Evaluation de la Coopéra tion Universitaire et Scientifique avec le Brésil (COFECUB); – the Chilean Agencia Nacional de Investigación y Desar rollo (ANID) through Fondo Nacional de Desarrollo Cientí fico y Tecnológico (FONDECYT) Regular Project 1210992 (L. Chemin); – the National Natural Science Foundation of China (NSFC) through grants 11573054, 11703065, and 12173069, the China Scholarship Council through grant 201806040200, and the Natural Science Foundation of Shanghai through grant 21ZR1474100; – the Tenure Track Pilot Programme of the Croatian Sci ence Foundation and the École Polytechnique Fédérale de Lausanne and the project TTP-2018-07-1171 ‘Mining the Variable Sky’, with the funds of the Croatian-Swiss Research Programme; – the Czech-Republic Ministry of Education, Youth, and Sports through grant LG 15010 and INTER-EXCELLENCE grant LTAUSA18093, and the Czech Space Office through ESA PECS contract 98058; – the Danish Ministry of Science; – the Estonian Ministry of Education and Research through grant IUT40-1; – the European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTN CT-2006-033481), through Marie Curie project PIOF GA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transfer-of Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Ser vices (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; – the European Cooperation in Science and Technology (COST) through COST Action CA18104 ‘Revealing the Milky Way with Gaia (MW-Gaia)’; – the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excel lent science programmes through Marie Skłodowska-Curie grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from bina ries and Cepheids – CepBin), 716155 (Structured ACCRE tion Disks – SACCRED), 745617 (Our Galaxy at full HD – Gal-HD), 895174 (The build-up and fate of self-gravitating systems in the Universe), 951549 (Sub-percent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), 101004214 (Innovative Scientific Data Explo ration and Exploitation Applications for Space Sciences – EXPLORE), 101004719 (OPTICON-RadioNET Pilot), 101055318 (The 3D motion of the Interstellar Medium with ESO and ESA telescopes – ISM-FLOW), and 101063193 (Evolutionary Mechanisms in the Milky waY; the Gaia Data Release 3 revolution – EMMY); – the European Science Foundation (ESF), in the framework of the Gaia Research for European Astronomy Training Research Network Programme (GREAT-ESF); – the European Space Agency (ESA) in the framework of the Gaia project, through the Plan for European Cooper ating States (PECS) programme through contracts C98090 and 4000106398/12/NL/KML for Hungary, through contract 4000115263/15/NL/IB for Germany, through PROgramme de Développement d’Expériences scientifiques (PRODEX) grants 4000132054 for Hungary and through contract 4000132226/20/ES/CM; – the Academy of Finland through grants 299543, 307157, 325805, 328654, 336546, and 345115 and the Magnus Ehrn rooth Foundation; – the French Centre National d’Études Spatiales (CNES), the Agence Nationale de la Recherche (ANR) through grant ANR-10-IDEX-0001-02 for the ‘Investissements d’avenir’ programme, through grant ANR-15-CE31-0007 for project ‘Modelling the Milky Way in the Gaia era’ (MOD4Gaia), through grant ANR-14-CE33-0014-01 for project ‘The Milky Way disc formation in the Gaia era’ (ARCHEOGAL), through grant ANR-15-CE31-0012-01 for project ‘Unlock ing the potential of Cepheids as primary distance cali brators’ (UnlockCepheids), through grant ANR-19-CE31-0017 for project ‘Secular evolution of galaxies’ (SEGAL), and through grant ANR-18-CE31-0006 for project ‘Galac tic Dark Matter’ (GaDaMa), the Centre National de la Recherche Scientifique (CNRS) and its SNO Gaia of the Institut des Sciences de l’Univers (INSU), its Programmes Nationaux: Cosmologie et Galaxies (PNCG), Gravitation Références Astronomie Métrologie (PNGRAM), Planétolo gie (PNP), Physique et Chimie du Milieu Interstellaire (PCMI), and Physique Stellaire (PNPS), supported by INSU along with the Institut National de Physique (INP) and the Institut National de Physique nucléaire et de Physique des Particules (IN2P3), and co-funded by CNES; the ‘Action Fédératrice Gaia’ of the Observatoire de Paris, and the Région de Franche-comté; – the German Aerospace Agency (Deutsches Zentrum für Luft- und Raumfahrt e.V., DLR) through grants 50QG0501, 50QG0601, 50QG0602, 50QG0701, 50QG0901, 50QG1001, 50QG1101, 50QG1401, 50QG1402, 50QG1403, 50QG1404, 50QG1904, 50QG2101, 50QG2102, and 50QG2202, and the Centre for Information Services and High Performance Computing (ZIH) at the Technische Universität Dresden for generous allocations of computer time; – the Hungarian Academy of Sciences through the János Bolyai Research Scholarship (G. Marton and Z. Nagy), the Lendület Programme grants LP2014-17 and LP2018-7 and the Hungarian National Research, Development, and Innovation Office (NKFIH) through grant KKP-137523 (‘SeismoLab’); – the Science Foundation Ireland (SFI) through a Royal Soci ety - SFI University Research Fellowship (M. Fraser); – the Israel Ministry of Science and Technology through grant 3-18143 and the Israel Science Foundation (ISF) through grant 1404/22; – the Agenzia Spaziale Italiana (ASI) through contracts I/037/08/0, I/058/10/0, 2014-025-R.0, 2014-025-R.1.2015, and 2018-24-HH.0 and its addendum 2018-24-HH.1-2022 to the Italian Istituto Nazionale di Astrofisica (INAF), contract 2014-049-R.0/1/2, 2022-14-HH.0 to INAF for the Space Science Data Centre (SSDC, formerly known as the ASI Science Data Center, ASDC), contracts I/008/10/0, 2013/030/I.0, 2013-030-I.0.1-2015, and 2016-17-I.0 to the Aerospace Logistics Technology Engineering Company (ALTEC S.p.A.), INAF, and the Italian Ministry of Education, University, and Research (Ministero dell’Istruzione, dell’Università e della Ricerca) through the Premiale project ‘MIning The Cosmos Big Data and Innovative Italian Tech nology for Frontier Astrophysics and Cosmology’ (MITiC); – the Netherlands Organisation for Scientific Research (NWO) through grant NWO-M-614.061.414, through a VICI grant (A. Helmi), and through a Spinoza prize (A. Helmi), and the Netherlands Research School for Astronomy (NOVA); – the Polish National Science Centre through HAR MONIA grant 2018/30/M/ST9/00311 and DAINA grant 2017/27/L/ST9/03221 and the Ministry of Sci ence and Higher Education (MNiSW) through grant DIR/WK/2018/12; – the Portuguese Fundação para a Ciência e a Tecnologia (FCT) through national funds, grants 2022.06962.PTDC and 2022.03993.PTDC, and work contract DL 57/2016/CP1364/CT0006, grants UIDB/04434/2020 and UIDP/04434/2020 for the Instituto de Astrofísica e Ciências do Espaço (IA), grants UIDB/00408/2020 and UIDP/00408/2020 for LASIGE, and grants UIDB/00099/2020 and UIDP/00099/2020 for the Centro de Astrofísica e Gravitação (CENTRA); – the Slovenian Research Agency through grant P1-0188; – the Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MCIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, FPU16/03827, RTI2018-095076-B-C22, PID2021-122842OB-C22, PDC2021-121059-C22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493) and MCIN/AEI/10.13039/501100011033/EU FEDER and Next Generation EU/PRTR (PRTR C17.I1); the European Union through European Regional Development Fund ‘A way of making Europe’ through grants PID2021-122842OB-C21, PID2021-125451NA-I00, CNS2022-13523 and RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia ‘María de Maeztu’) through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en For mació (APIF) grant, the Spanish Virtual Observatory project funded by MCIN/AEI/10.13039/501100011033/through grant PID2020-112949GB-I00; the Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia through the collaboration agreement to reinforce CIGUS research centers, research consolidation grant ED431B 2021/36 and scholarships from Xunta de Galicia and the EU - ESF ED481A-2019/155 and ED481A 2021/296; the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT 2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowships RYC2018-025968-I, RYC2021-031683-I and RYC2021-033762-I, funded by MICIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR and the European Science Foundation (‘Investing in your future’); the Port d’Informació Científica (PIC), through a collaboration between the Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT) and the Institut de Física d’Altes Energies (IFAE), supported by the call for grants for Scientific and Technical Equipment 2021 of the State Program for Knowledge Generation and Scientific and Technological Strengthening of the R+D+i System, financed by MCIN/AEI/ 10.13039/501100011033 and the EU NextGeneration/PRTR (Hadoop Cluster for the comprehensive management of massive scientific data, reference EQC2021-007479-P); – the Swedish National Space Agency (SNSA/Rymdstyrelsen); – the Swiss State Secretariat for Education, Research, and Innovation through the Swiss Activités Nationales Com plémentaires and the Swiss National Science Founda tion through an Eccellenza Professorial Fellowship (award PCEFP2_194638 for R. Anderson); – the United Kingdom Particle Physics and Astronomy Research Council (PPARC), the United Kingdom Science and Technology Facilities Council (STFC), and the United Kingdom Space Agency (UKSA) through the following grants to the University of Bristol, Brunel University London, the Open University, the University of Cambridge, the University of Edinburgh, the University of Leicester, the Mullard Space Sci ences Laboratory of University College London, and the United Kingdom Rutherford Appleton Laboratory (RAL): PP/D006503/1, PP/D006511/1, PP/D006546/1, PP/D006570/1, PP/D006791/1, ST/I000852/1, ST/J005045/1, ST/K00056X/1, ST/K000209/1, ST/K000756/1, ST/K000578/1, ST/L002388/1, ST/L006553/1, ST/L006561/1, ST/N000595/1, ST/N000641/1, ST/N000978/1, ST/N001117/1, ST/S000089/1, ST/S000976/1, ST/S000984/1, ST/S001123/1, ST/S001948/1, ST/S001980/1, ST/S002103/1, ST/V000969/1, ST/W002469/1, ST/W002493/1, ST/W002671/1, ST/W002809/1, EP/V520342/1, ST/X00158X/1, ST/X001601/1, ST/X001636/1, ST/X001687/1, ST/X002667/1, ST/X002683/1 and ST/X002969/1. The Gaia project and data processing have made use of: – the Set of Identifications, Measurements, and Bibliog raphy for Astronomical Data (SIMBAD, Wenger et al. 2000), the ‘Aladin sky atlas’ (Bonnarel et al. 2000; Boch & Fernique 2014), and the VizieR catalogue access tool (Ochsenbein et al. 2000), all operated at the Centre de Données astronomiques de Strasbourg (CDS); – the National Aeronautics and Space Administration (NASA) Astrophysics Data System (ADS); – the SPace ENVironment Information System (SPENVIS), initiated by the Space Environment and Effects Section (TEC-EES) of ESA and developed by the Belgian Insti tute for Space Aeronomy (BIRA-IASB) under ESA contract through ESA’s General Support Technologies Programme (GSTP), administered by the BELgian federal Science Pol icy Office (BELSPO); – the software products TOPCAT, STIL, and STILTS (Taylor 2005, 2006); – Matplotlib (Hunter 2007); – IPython (Pérez & Granger 2007); – Astropy, a community-developed core Python package for Astronomy (Astropy Collaboration 2018); – R (R Core Team 2013); – the HEALPix package (Górski et al. 2005, http://healpix.sourceforge.net/); – Vaex (Breddels & Veljanoski 2018); – the HIPPARCOS-2 catalogue (van Leeuwen 2007). The HIPPARCOS and Tycho catalogues were constructed under the responsibility of large scientific teams collaborating with ESA. The Consortia Leaders were Lennart Lindegren (Lund, Sweden: NDAC) and Jean Kovalevsky (Grasse, France: FAST), together responsible for the HIPPARCOS Catalogue; Erik Høg (Copenhagen, Denmark: TDAC) responsible for the Tycho Catalogue; and Catherine Turon (Meudon, France: INCA) responsible for the HIPPARCOS Input Catalogue (HIC); – the Tycho-2 catalogue (Høg et al. 2000), the construction of which was supported by the Velux Foundation of 1981 and the Danish Space Board; – The Tycho double star catalogue (TDSC, Fabricius et al. 2002), based on observations made with the ESA HIPPAR COS astrometry satellite, as supported by the Danish Space Board and the United States Naval Observatory through their double-star programme; – data products from the Two Micron All Sky Survey (2MASS, Skrutskie et al. 2006), which is a joint project of the University of Massachusetts and the Infrared Pro cessing and Analysis Center (IPAC) / California Institute of Technology, funded by the National Aeronautics and Space Administration (NASA) and the National Science Foundation (NSF) of the USA; – the ninth data release of the AAVSO Photometric All-Sky Survey (APASS, Henden et al. 2016), funded by the Robert Martin Ayers Sciences Fund; – the first data release of the Pan-STARRS survey (Chambers et al. 2016; Magnier et al. 2020a; Waters et al. 2020; Magnier et al. 2020c,b; Flewelling et al. 2020). The Pan-STARRS1 Surveys (PS1) and the PS1 public sci ence archive have been made possible through contributions by the Institute for Astronomy, the University of Hawaii, the Pan-STARRS Project Office, the Max-Planck Society and its participating institutes, the Max Planck Institute for Astronomy, Heidelberg and the Max Planck Institute for Extraterrestrial Physics, Garching, The Johns Hopkins University, Durham University, the University of Edinburgh, the Queen’s University Belfast, the Harvard-Smithsonian Center for Astrophysics, the Las Cumbres Observatory Global Telescope Network Incorporated, the National Central University of Taiwan, the Space Telescope Science Institute, the National Aeronautics and Space Administration (NASA) through grant NNX08AR22G issued through the Planetary Science Division of the NASA Science Mission Directorate, the National Science Foundation through grant AST-1238877, the University of Maryland, Eotvos Lorand University (ELTE), the Los Alamos National Laboratory, and the Gordon and Betty Moore Foundation; – the second release of the Guide Star Catalogue (GSC2.3, Lasker et al. 2008). The Guide Star Catalogue II is a joint project of the Space Telescope Science Institute (STScI) and the Osservatorio Astrofisico di Torino (OATo). STScI is operated by the Association of Universities for Research in Astronomy (AURA), for the National Aeronautics and Space Administration (NASA) under contract NAS5-26555. OATo is operated by the Italian National Institute for Astrophysics (INAF). Additional support was provided by the European Southern Observatory (ESO), the Space Telescope European Coordinating Facility (STECF), the International GEMINI project, and the European Space Agency (ESA) Astrophysics Division (nowadays SCI-S); – the eXtended, Large (XL) version of the catalogue of Posi tions and Proper Motions (PPM-XL, Roeser et al. 2010); – data products from the Wide-field Infrared Survey Explorer (WISE), which is a joint project of the University of Cal ifornia, Los Angeles, and the Jet Propulsion Laboratory/-California Institute of Technology, and NEOWISE, which is a project of the Jet Propulsion Laboratory/California Insti tute of Technology. WISE and NEOWISE are funded by the National Aeronautics and Space Administration (NASA); – the first data release of the United States Naval Obser vatory (USNO) Robotic Astrometric Telescope (URAT-1, Zacharias et al. 2015); – the fourth data release of the United States Naval Obser vatory (USNO) CCD Astrograph Catalogue (UCAC-4, Zacharias et al. 2013); – the sixth and final data release of the Radial Velocity Exper iment (RAVE DR6, Steinmetz et al. 2020a,b). Funding for RAVE has been provided by the Leibniz Institute for Astro physics Potsdam (AIP), the Australian Astronomical Obser vatory, the Australian National University, the Australian Research Council, the French National Research Agency, the German Research Foundation (SPP 1177 and SFB 881), the European Research Council (ERC-StG 240271 Galactica), the Istituto Nazionale di Astrofisica at Padova, the Johns Hopkins University, the National Science Foundation of the USA (AST-0908326), the W.M. Keck foundation, the Macquarie University, the Netherlands Research School for Astronomy, the Natural Sciences and Engineering Research Council of Canada, the Slovenian Research Agency, the Swiss National Science Foundation, the Science & Tech nology Facilities Council of the UK, Opticon, Strasbourg Observatory, and the Universities of Basel, Groningen, Hei delberg, and Sydney. The RAVE website is at https:// www.rave-survey.org/; – the first data release of the Large sky Area Multi-Object Fibre Spectroscopic Telescope (LAMOST DR1, Luo et al. 2015); – the K2 Ecliptic Plane Input Catalogue (EPIC, Huber et al. 2016); – the ninth data release of the Sloan Digitial Sky Survey (SDSS DR9, Ahn et al. 2012). Funding for SDSS-III has been provided by the Alfred P. Sloan Foundation, the Par ticipating Institutions, the National Science Foundation, and the United States Department of Energy Office of Science. The SDSS-III website is http://www.sdss3.org/. SDSS III is managed by the Astrophysical Research Consortium for the Participating Institutions of the SDSS-III Collaboration including the University of Arizona, the Brazilian Participa tion Group, Brookhaven National Laboratory, Carnegie Mel lon University, University of Florida, the French Participa tion Group, the German Participation Group, Harvard Uni versity, the Instituto de Astrofísica de Canarias, the Michigan State/Notre Dame/JINA Participation Group, Johns Hop kins University, Lawrence Berkeley National Laboratory, Max Planck Institute for Astrophysics, Max Planck Institute for Extraterrestrial Physics, New Mexico State University, New York University, Ohio State University, Pennsylva nia State University, University of Portsmouth, Princeton University, the Spanish Participation Group, University of Tokyo, University of Utah, Vanderbilt University, University of Virginia, University of Washington, and Yale University; – the thirteenth release of the Sloan Digital Sky Survey (SDSS DR13, Albareti et al. 2017). Funding for SDSS-IV has been provided by the Alfred P. Sloan Foundation, the United States Department of Energy Office of Science, and the Par ticipating Institutions. SDSS-IV acknowledges support and resources from the Center for High-Performance Computing at the University of Utah. The SDSS web site is https: //www.sdss.org/. SDSS-IV is managed by the Astrophys ical Research Consortium for the Participating Institutions of the SDSS Collaboration including the Brazilian Partici pation Group, the Carnegie Institution for Science, Carnegie Mellon University, the Chilean Participation Group, the French Participation Group, Harvard-Smithsonian Center for Astrophysics, Instituto de Astrofísica de Canarias, The Johns Hopkins University, Kavli Institute for the Physics and Mathematics of the Universe (IPMU) / University of Tokyo, the Korean Participation Group, Lawrence Berke ley National Laboratory, Leibniz Institut für Astrophysik Potsdam (AIP), Max-Planck-Institut für Astronomie (MPIA Heidelberg), Max-Planck-Institut für Astrophysik (MPA Garching), Max-Planck-Institut für Extraterrestrische Physik (MPE), National Astronomical Observatories of China, New Mexico State University, New York University, University of Notre Dame, Observatário Nacional / MCTI, The Ohio State University, Pennsylvania State University, Shanghai Astronomical Observatory, United Kingdom Participation Group, Universidad Nacional Autónoma de México, University of Arizona, University of Colorado Boulder, University of Oxford, University of Portsmouth, University of Utah, University of Virginia, University of Washington, University of Wisconsin, Vanderbilt University, and Yale University; – the second release of the SkyMapper catalogue (SkyMap per DR2, Onken et al. 2019, Digital Object Identifier 10.25914/5ce60d31ce759). The national facility capability for SkyMapper has been funded through grant LE130100104 from the Australian Research Council (ARC) Linkage Infrastructure, Equipment, and Facilities (LIEF) programme, awarded to the University of Sydney, the Australian National University, Swinburne University of Technology, the Uni versity of Queensland, the University of Western Australia, the University of Melbourne, Curtin University of Technol ogy, Monash University, and the Australian Astronomical Observatory. SkyMapper is owned and operated by The Australian National University’s Research School of Astronomy and Astrophysics. The survey data were processed and provided by the SkyMapper Team at the Australian National University. The SkyMapper node of the All-Sky Virtual Observatory (ASVO) is hosted at the National Computational Infrastructure (NCI). Development and support the SkyMapper node of the ASVO has been funded in part by Astronomy Australia Limited (AAL) and the Aus tralian Government through the Commonwealth’s Education Investment Fund (EIF) and National Collaborative Research Infrastructure Strategy (NCRIS), particularly the National eResearch Collaboration Tools and Resources (NeCTAR) and the Australian National Data Service Projects (ANDS); – the Gaia-ESO Public Spectroscopic Survey (GES, Gilmore et al. 2023; Randich et al. 2023). The Gaia-ESO Survey is based on data products from observations made with ESO Telescopes at the La Silla Paranal Observatory under programme ID 188.B-3002. Public data releases are available through the ESO Science Portal. The project has received funding from the Leverhulme Trust (project RPG-2012-541), the European Research Council (project ERC-2012-AdG 320360-Gaia-ESO-MW), and the Istituto Nazionale di Astrofisica, INAF (2012: CRA 1.05.01.09.16; 2013: CRA 1.05.06.02.07)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Gaia focused product release: A catalogue of sources around quasars to search for strongly lensed quasars

  • Gaia Collaboration
  • Jordi, Carme
  • Luri, Xavier
  • Castañeda, J.
  • Fabricius, Claus
  • Masana, Eduard
  • Portell, Jordi
  • Antoja, Teresa
  • Bartolomé, S.
  • Bernet, Marcel
  • Carrasco, J. M.
  • Figueras, Francesca
  • García-Gutiérrez, A.
  • Gregori Barrera, V.
  • Jiménez-Arranz, Ó.
  • Marín Pina, Daniel
  • Masip, Adriá
  • Molina, Daniel
  • Monguió, Maria
  • Racero, Elena
  • Ramos, Pau
  • Romero-Gómez, Mercè
  • Sáez-Núñez, Albert
  • Sánchez-Giménez, Victor
  • Sarrate Riera, C.
  • Solano, Enrique
  • Torra, Ferran
  • Weiler, Michael
  • Balaguer-Nuñez, Lola
Gaia Collaboration: et al., [Context] Strongly lensed quasars are fundamental sources for cosmology. The Gaia space mission covers the entire sky with the unprecedented resolution of 0.18″ in the optical, making it an ideal instrument to search for gravitational lenses down to the limiting magnitude of 21. Nevertheless, the previous Gaia Data Releases are known to be incomplete for small angular separations such as those expected for most lenses., [Aims] We present the Data Processing and Analysis Consortium GravLens pipeline, which was built to analyse all Gaia detections around quasars and to cluster them into sources, thus producing a catalogue of secondary sources around each quasar. We analysed the resulting catalogue to produce scores that indicate source configurations that are compatible with strongly lensed quasars., [Methods] GravLens uses the DBSCAN unsupervised clustering algorithm to detect sources around quasars. The resulting catalogue of multiplets is then analysed with several methods to identify potential gravitational lenses. We developed and applied an outlier scoring method, a comparison between the average BP and RP spectra of the components, and we also used an extremely randomised tree algorithm. These methods produce scores to identify the most probable configurations and to establish a list of lens candidates., [Results] We analysed the environment of 3 760 032 quasars. A total of 4 760 920 sources, including the quasars, were found within 6″ of the quasar positions. This list is given in the Gaia archive. In 87% of cases, the quasar remains a single source, and in 501 385 cases neighbouring sources were detected. We propose a list of 381 lensed candidates, of which we identified 49 as the most promising ones. Beyond these candidates, the associate tables in this Focused Product Release allow the entire community to explore the unique Gaia data for strong lensing studies further., This work presents results from the European Space Agency (ESA) space mission Gaia; the European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTN-CT-2006-033481), through Marie Curie project PIOF-GA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transfer-of-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Programme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Services (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network; the European Research Council (ERC) through grants 320360, 647208, and 834148 and through the European Union’s Horizon 2020 research and innovation and excellent science programmes through Marie Skłodowska-Curie grants 687378 (Small Bodies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub-percent distance scale from binaries and Cepheids - CepBin), 716155 (Structured ACCREtion Disks - SACCRED), 745617 (Our Galaxy at full HD -Gal-HD), 895174 (The build-up and fate of self-gravitating systems in the Universe), 951549 (Sub-percent calibration of the extragalactic distance scale in the era of big surveys – UniverScale), 101004214 (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences – EXPLORE), 101004719 (OPTICON-RadioNET Pilot), 101055318 (The 3D motion of the Interstellar Medium with ESO and ESA telescopes – ISM-FLOW), and 101063193 (Evolutionary Mechanisms in the Milky waY; the Gaia Data Release 3 revolution – EMMY); the Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MCIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, FPU16/03827, RTI2018-095076-B-C22, PID2021-122842OB-C22, PDC2021-121059-C22, and TIN2015-65316-P ('Computation de Altas Prestaciones VII'), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493) and MCIN/AEI/10.13039/501100011033/ EU FEDER and Next Generation EU/PRTR (PRTR- C17.I1); the European Union through European Regional Development Fund 'A way of making Europe' through grants PID2021-122842OB-C21, PID2021-125451NA-I00, CNS2022-13523 and RTI2018-095076-B-C21, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia 'Mar a de Maeztu') through grant CEX2019-000918-M, the University of Barcelona’s official doctoral programme for the development of an R+D+i project through an Ajuts de Personal Investigador en For-mació (APIF) grant, the Spanish Virtual Observatory project funded by MCIN/AEI/10.13039/501100011033/ through grant PID2020-112949GB-I00; the Centro de Investigación en Tecnolog as de la Información y las Comunicaciones (CITIC), funded by the Xunta de Galicia through the collaboration agreement to reinforce CIGUS research centers, research consolidation grant ED431B 2021/36 and scholarships from Xunta de Galicia and the EU – ESF ED481A-2019/155 and ED481A 2021/296; the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Centre -Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3-0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT-2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d'Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project 'Models de Programació i Entorns d'Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowships RYC2018-025968-I, RYC2021-031683-I and RYC2021-033762-I, funded by MICIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR and the European Science Foundation ('Investing in your future'); the Port d'Informació Cientifica (PIC), through a collaboration between the Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT) and the Institut de F sica d'Altes Energies (IFAE), supported by the call for grants for Scientific and Technical Equipment 2021 of the State Program for Knowledge Generation and Scientific and Technological Strengthening of the R+D+i System, financed by MCIN/AEI/ 10.13039/501100011033 and the EU NextGeneration/PRTR (Hadoop Cluster for the comprehensive management of massive scientific data, reference EQC2021-007479-P)., With funding from the Spanish government through the "Severo Ochoa Centre of Excellence" accreditation (CEX2019-000918-M)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Discovery of a dormant 33 solar-mass black hole in pre-release Gaia astrometry

  • Gaia Collaboration
  • Panuzzo, Pasquale
Gaia Collaboration: P. Panuzzo et al., [Context]. Gravitational waves from black-hole (BH) merging events have revealed a population of extra-galactic BHs residing in short-period binaries with masses that are higher than expected based on most stellar evolution models – and also higher than known stellar-origin black holes in our Galaxy. It has been proposed that those high-mass BHs are the remnants of massive metal-poor stars., [Aims] Gaia astrometry is expected to uncover many Galactic wide-binary systems containing dormant BHs, which may not have been detected before. The study of this population will provide new information on the BH-mass distribution in binaries and shed light on their formation mechanisms and progenitors., [Methods] As part of the validation efforts in preparation for the fourth Gaia data release (DR4), we analysed the preliminary astrometric binary solutions, obtained by the Gaia Non-Single Star pipeline, to verify their significance and to minimise false-detection rates in high-mass-function orbital solutions., [Results] The astrometric binary solution of one source, Gaia BH3, implies the presence of a 32.70 ± 0.82 M BH in a binary system with a period of 11.6 yr. Gaia radial velocities independently validate the astrometric orbit. Broad-band photometric and spectroscopic data show that the visible component is an old, very metal-poor giant of the Galactic halo, at a distance of 590 pc., [Conclusions] The BH in the Gaia BH3 system is more massive than any other Galactic stellar-origin BH known thus far. The low metallicity of the star companion supports the scenario that metal-poor massive stars are progenitors of the high-mass BHs detected by gravitational-wave telescopes. The Galactic orbit of the system and its metallicity indicate that it might belong to the Sequoia halo substructure. Alternatively, and more plausibly, it could belong to the ED-2 stream, which likely originated from a globular cluster that had been disrupted by the Milky Way., This work presents results from the European Space Agency (ESA) space mission Gaia. Gaia data are being processed by the Gaia Data Processing and Analysis Consortium (DPAC). Funding for the DPAC is provided by national institutions, in particular the institutions participating in the Gaia MultiLat eral Agreement (MLA). The Gaia mission website is https: //www.cosmos.esa.int/gaia. The Gaia archive website is https://archives.esac.esa.int/gaia. The Gaia mission and data processing have financially been supported by, in alphabetical order by country:...... the European Commission’s Sixth Framework Programme through the European Leadership in Space Astrometry (ELSA) Marie Curie Research Training Network (MRTN CT-2006-033481), through Marie Curie project PIOF GA-2009-255267 (Space AsteroSeismology & RR Lyrae stars, SAS-RRL), and through a Marie Curie Transfer of-Knowledge (ToK) fellowship (MTKD-CT-2004-014188); the European Commission’s Seventh Framework Pro gramme through grant FP7-606740 (FP7-SPACE-2013-1) for the Gaia European Network for Improved data User Ser vices (GENIUS) and through grant 264895 for the Gaia Research for European Astronomy Training (GREAT-ITN) network;...the European Research Council (ERC) through grants 320360 (The Gaia-ESO Milky Way Survey), 647208 (Do intermediate-mass black holes exist?), 687378 (Small Bod ies: Near and Far), 682115 (Using the Magellanic Clouds to Understand the Interaction of Galaxies), 695099 (A sub percent distance scale from binaries and Cepheids – Cep Bin), 745617 (Our Galaxy at full HD – Gal-HD), 834148 (Accelerating Galactic Archeology), 895174 (The build-up and fate of self-gravitating systems in the Universe), 947660 (Measuring Hubble’s Constant to 1% with Pulsating Stars – H1PStars), 951549 (Sub-percent calibration of the extra galactic distance scale in the era of big surveys – Uni verScale), 101004214 (Innovative Scientific Data Explo ration and Exploitation Applications for Space Sciences – EXPLORE), 101004719 (OPTICON-RadioNET Pilot), 101055318 (The 3D motion of the Interstellar Medium with ESO and ESA telescopes – ISM-FLOW), 101063193 (Evo lutionary Mechanisms in the Milky waY: the Gaia Data Release 3 revolution – EMMY), 101093572 (StarDance: the non-canonical evolution of stars in clusters) and 101135205 (HORIZON-CL4-2023-SPACE-01-71 SPACIOUS project);....the Spanish Ministry of Economy (MINECO/FEDER, UE), the Spanish Ministry of Science and Innovation (MCIN), the Spanish Ministry of Education, Culture, and Sports, and the Spanish Government through grants BES-2016-078499, BES-2017-083126, BES-C-2017-0085, ESP2016-80079-C2-1-R, FPU16/03827, RTI2018-095076- B-C22, PID2021-122842OB-C22, PDC2021-121059-C22, and TIN2015-65316-P (‘Computación de Altas Prestaciones VII’), the Juan de la Cierva Incorporación Programme (FJCI-2015-2671 and IJC2019-04862-I for F. Anders), the Severo Ochoa Centre of Excellence Programme (SEV2015-0493) and MCIN/AEI/10.13039/501100011033/ EU FEDER and Next Generation EU/PRTR (PRTR C17.I1 and CNS2022-135232); the European Union through European Regional Development Fund ‘A way of mak ing Europe’ through grants PID2021-122842OB-C21 and PID2021-125451NA-I00, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelen cia ‘María de Maeztu’) through grant CEX2019-000918- M, the University of Barcelona’s official doctoral pro gramme for the development of an R+D+i projec through an Ajuts de Personal Investigador en Forma ció (APIF) grant, the Spanish Virtual Observatory project funded by MCIN/AEI/10.13039/501100011033/ through grant PID2020-112949GB-I00; the Centro de Investi gación en Tecnologías de la Información y las Comunica ciones (CITIC), funded by the Xunta de Galicia through the collaboration agreement to reinforce CIGUS research centers, research consolidation grant ED431B 2021/36 and scholarships from Xunta de Galicia and the EU - ESF ED481A-2019/155 and ED481A 2021/296; the Red Española de Supercomputación (RES) computer resources at MareNostrum, the Barcelona Supercomputing Cen tre - Centro Nacional de Supercomputación (BSC-CNS) through activities AECT-2017-2-0002, AECT-2017-3-0006, AECT-2018-1-0017, AECT-2018-2-0013, AECT-2018-3- 0011, AECT-2019-1-0010, AECT-2019-2-0014, AECT 2019-3-0003, AECT-2020-1-0004, and DATA-2020-1-0010, the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya through grant 2014-SGR-1051 for project ‘Models de Programació i Entorns d’Execució Parallels’ (MPEXPAR), and Ramon y Cajal Fellowships RYC2018-025968-I, RYC2021-031683-I and RYC2021- 033762-I, funded by MICIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR and the European Science Foundation (‘Investing in your future’); the Port d’Informació Científica (PIC), through a collaboration between the Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT) and the Institut de Física d’Altes Energies (IFAE), supported by the grant EQC2021-007479-P funded by MCIN/AEI/ 10.13039/501100011033 and by the "European Union NextGenerationEU/PRTR), and also by MICIIN with funding from European Union NextGenerationEU(PRTR C17.I1) and by Generalitat de Catalunya……, With funding from the Spanish government through the "María de Maeztu Unit of Excellence" accreditation (CEX2019-000918-M), Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Workflows for science: A challenge when facing the convergence of HPC and big data

  • Badia, Rosa M.
  • Ayguade, Eduard
  • Labarta, Jesus
Workflows have been used traditionally as a mean to describe and implement the computing usually parametric studies and explorations searching for the best solution that scientific researchers want to perform. A workflow is not only the computing application, but a way of documenting a process. Science workflows may be of very different nature depending on the area of research, matching the actual experiment that the scientist want to perform. Workflow Management Systems are environments that offer the researchers tools to define, publish, execute and document their workflows. In some cases, the science workflows are used to generate data; in other cases are used to analyse existing data; only in a few cases, workflows are used both to generate and analyse data. The design of experiments is in some cases generated blindly, without a clear idea of which points are relevant to be computed/simulated, ending up with huge amount of computation that is performed following a brute-force strategy. However, the evolution of systems and the large amount of data generated by the applications require an in-situ analysis of the data, thus requiring new solutions to develop workflows that includes both the simulation/computational part and the analytic part. What is more, the fact that both components, computation and analytics, can be run together will enable the possibility of defining more dynamic workflows, with new computations being decided by the analytics in a more efficient way. The first part of the paper will review current approaches that a set of scientific communities follows in the development of their workflows. Due to the election of several scientific communities and use cases using a specific Workflow Management System, this survey maybe incomplete with regard a complete revision of the literature about workflows, but we expect that the reader appreaciates the effort performed in trying to see the scientific communities needs and requirements. The second part of the paper will propose a new software architecture to develop a new family of end-to-end workflows that enables the management of dynamic workflows composed of simulations, analytics and visualization, including inputs/outputs from streams., This work has been supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), by Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272). This work is also supported by the Intel-BSC Exascale Lab. The Human Brain Project receives funding from the EU’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no 604102., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

On the maturity of parallel applications for asymmetric multi-core processors

  • Chronaki, Kallia
  • Moretó Planas, Miquel|||0000-0002-9848-8758
  • Casas, Marc|||0000-0003-4564-2093
  • Rico, Alejandro
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Valero Cortés, Mateo|||0000-0003-2917-2482
Asymmetric multi-cores (AMCs) are a successful architectural solution for both mobile devices and supercomputers. By maintaining two types of cores (fast and slow) AMCs are able to provide high performance under the facility power budget. This paper performs the first extensive evaluation of how portable are the current HPC applications for such supercomputing systems. Specifically we evaluate several execution models on an ARM big.LITTLE AMC using the PARSEC benchmark suite that includes representative highly parallel applications. We compare schedulers at the user, OS and runtime levels, using both static and dynamic options and multiple configurations, and assess the impact of these options on the well-known problem of balancing the load across AMCs. Our results demonstrate that scheduling is more effective when it takes place in the runtime system level as it improves the baseline by 23%, while the heterogeneous-aware OS scheduling solution improves the baseline by 10%., This work has been supported by the RoMoL ERC Advanced Grant (GA 321253), by the European HiPEAC Network of Excellence, by the Spanish Ministry of Science and Innovation (contracts TIN2015-65316-P), by the Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), and by the European Union's Horizon 2020 research and innovation programme under grant agreement No 671697 and No. 779877. M. Moretó has been partially supported by the Ministry of Economy and Competitiveness under Ramon y Cajal fellowship number RYC-2016-21104., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A fault-tolerant last level cache for CMPs operating at ultra-low voltage

  • Ferrerón, Alexandra
  • Alastruey, Jesús
  • Suárez Gracía, Dario
  • Monreal Arnal, Teresa|||0000-0002-0458-2234
  • Ibáñez Marín, Pablo Enrique
  • Viñals Yúfera, Víctor
Voltage scaling to values near the threshold voltage is a promising technique to hold off the many-core power wall. However, as voltage decreases, some SRAM cells are unable to operate reliably and show a behavior consistent with a hard fault. Block disabling is a micro-architectural technique that allows low-voltage operation by deactivating faulty cache entries, at the expense of reducing the effective cache capacity. In the case of the last-level cache, this capacity reduction leads to an increase in off-chip memory accesses, diminishing the overall energy benefit of reducing the voltage supply. In this work, we exploit the reuse locality and the intrinsic redundancy of multi-level inclusive hierarchies to enhance the performance of block disabling with negligible cost. The proposed fault-aware last-level cache management policy maps critical blocks, those not present in private caches and with a higher probability of being reused, to active cache entries. Our evaluation shows that this fault-aware management results in up to 37.3% and 54.2% fewer misses per kilo instruction (MPKI) than block disabling for multiprogrammed and parallel workloads, respectively. This translates to performance enhancements of up to 13% and 34.6% for multiprogrammed and parallel workloads, respectively., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

EOmesh: combined flow balancing and deterministic routing for reduced WCET estimates in embedded real-time systems

  • Cardona Nadal, Jordi
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Hernández Luz, Carles
  • Cazorla Almeida, Francisco Javier
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The increasing performance needs in critical real-time embedded systems (CRTES) can only be satisfied with the use of high-performance manycore processors. While NoC-based manycore systems are popular in the high-performance domain due to their high average performance, they challenge deriving tight Worst-Case Execution Time (WCET) estimates, as needed in CRTES. Weighted meshes have been proposed to alleviate NoCs pathological behavior – caused by large bandwidth imbalance – by making locally unbalanced arbitration decisions to reach globally balanced bandwidth. In this paper we show that existing weighted mesh solutions do not completely remove unwanted imbalance, in particular for nodes subject to high congestion. We propose EOmesh, an approach that combines heterogeneous predictable routing and weight allocations that delivers near-optimal bandwidth allocation across cores without increasing NoC complexity. EOmesh, which can be implemented either by hardware means or by software means on top of regular weighted meshes, improves the average performance and WCET results of the reference weighted mesh design., This work has been partially supported by the SpanishMinistry of Economy and Competitiveness (MINECO) un-der grant TIN2015-65316-P and the HiPEAC Network ofExcellence. Jaume Abella has been partially supported bythe MINECO under Ramon y Cajal postdoctoral fellowshipnumber RYC-2013-14717. Carles Hern ́andez is jointly fundedby the MINECO and FEDER funds through grant TIN2014-60404-JIN. Francisco J. Cazorla is partially funded by theEuropean Research Council (ERC) under EU’s H2020 research/innovation programme (grant No. 772773), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Freezing time emulating new and faster devices with virtual machines

  • Bona, Luis C.E.
  • Elias, Alessandro
  • Ziviani, Andre P.
  • Nou, Ramon
  • Cortés, Toni|||0000-0002-2537-8937
  • Alves, Marco A.Z.
Recent proposals of emerging data storage devices make it necessary to reevaluate all levels of the storage hierarchy to optimize the software stack performance. However, these new devices are not always widely available and therefore early experiments may be impossible. Emulators aim at mimicking as close as possible the behavior of a component, nonetheless, emulating new and fast storage devices is a challenging task due to time perception. In this work, we propose an approach to emulate storage devices using virtual machines (VMs) allowing the evaluation of a new device within a real system. We use a technique called freezing time, which pauses a VM to manipulate its clock and hide the real I/O completion time. Our approach is implemented at the hypervisor level and it is transparent to the guest operating system or application. We evaluate the technique under a real system using regular magnetic disks to emulate faster storage devices. Our method presented a latency error of 6.5% compared to a real device. Moreover, decoupled experiment between two laboratories, at the Barcelona Super Computing Center (BSC) in Spain, and the Center of Computer Science and Free Software (C3SL) in Brazil, demonstrated that our approach is reproducible and promising to allow the virtual evaluation of next-gen storage devices., This work was partially supported by the Spanish Ministry of Science and Innovation under the TIN2015-65316 Grant, the Generalitat de Catalunya under contract 2014-SGR-1051, the Serrapilheira Institute (Grant number Serra-1709-16621), as well as the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no. 671951 (NEXTGenIO) for the extensions added after the MASCOTS paper., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A fault-tolerant algorithm for distributed resource allocation

  • Pessolani, Pablo
  • Jara, Oscar
  • Gonnet, Silvio
  • Cortés, Toni|||0000-0002-2537-8937
  • Tinetti, Fernando
Resource allocation is a usual problem that must be faced during a distributed system design. Despite the large number of algorithms proposed in literature to solve this problem, most papers lack of detailed descriptions about how to turn these algorithms into real-world reliable protocols. This article presents a fault-tolerant algorithm for distributed resource allocation named SLOTS which is implemented as an executable protocol. It allocates resources among members in a fairly manner using simple heuristics and employing a donation approach. SLOTS supports the dynamic behavior of clusters and provides high availability services. It bases its fault-tolerance properties and membership changes in atomic sets of operations (like transactions) using services provided by an underlying Group Communication System., La participación de Toni Cortes en este trabajo ha sido financiada por el Gobierno de España (subvención SEV2015-0493 del programa Severo Ochoa) por el Ministerio Español de Ciencia e Innovación (contrato TIN2015-65316) y la Generalitat de Catalunya (contrato 2014-SGR-1051). La participación de Fernando G. Tinetti en este trabajo ha sido financiada por la UNLP (Facultad de Informática) y la CIC Provincia de Buenos Aires, Argentina., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Fitting processor architectures for measurement-based probabilistic timing analysis

  • Kosmidis, Leonidas
  • Quiñones, Eduardo
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Vardanega, Tullio
  • Hernández, Carles
  • Gianarro, Andrea
  • Broster, Ian
  • Cazorla Almeida, Francisco Javier
The pressing market demand for competitive performance/cost ratios compels Critical Real-Time Embedded Systems industry to employ feature-rich hardware. The ensuing rise in hardware complexity however makes worst-case execution time (WCET) analysis of software programs - which is often required, especially for programs at the highest levels of integrity - an even harder challenge. State-of-the-art WCET analysis techniques are hampered by the soaring cost and complexity of obtaining accurate knowledge of the internal operation of advanced processors and the difficulty of relating data obtained from measurement observations with reliable worst-case behaviour. This frustrating conundrum calls for novel solutions, with low intrusiveness on development practice. Measurement-Based Probabilistic Timing Analysis (MBPTA) techniques offer the opportunity to simultaneously reduce the cost of acquiring the knowledge needed for computing reliable WCET bounds and gain increased confidence in the representativeness of measurement observations. This paper describes the changes required in the design of several high-performance features - massively used in modern processors - to meet MBPTA requirements., This work has received funding from the European Community's Seventh 1025 Framework Programme [FP7/2007-2013] under grant agreement 611085 (PROXIMA, www.proxima-project.eu). Support was also provided by the Ministry of Science and Technology of Spain under contract TIN2015-65316-P and the HiPEAC Network of Excellence. Leonidas Kosmidis is funded by the Spanish Ministry of Education under FPU grant AP2010-4208. Jaume Abella has been 1030 partially supported by the MINECO under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717. The authors wish to acknowledge Michael Houston, Liliana Cucu-Grosjean and Luca Santinelli for contributing to the genesis of this work., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Performance and energy consumption of HPC workloads on a cluster based on Arm ThunderX2 CPU

  • Mantovani, Filippo|||0000-0003-3559-4825
  • Garcia Gasulla, Marta|||0000-0003-3682-9905
  • Gracia, Jose
  • Stafford, Esteban
  • Banchelli Gracia, Fabio
  • Josep Fabregó, Marc
  • Criado Ledesma, Joel
  • Nachtmann, Mathias
In this paper, we analyze the performance and energy consumption of an Arm-based high-performance computing (HPC) system developed within the European project Mont-Blanc 3. This system, called Dibona, has been integrated by ATOS/Bull, and it is powered by the latest Marvell’s CPU, ThunderX2. This CPU is the same one that powers the Astra supercomputer, the first Arm-based supercomputer entering the Top500 in November 2018. We study from micro-benchmarks up to large production codes. We include an interdisciplinary evaluation of three scientific applications (a finite-element fluid dynamics code, a smoothed particle hydrodynamics code, and a lattice Boltzmann code) and the Graph 500 benchmark, focusing on parallel and energy efficiency as well as studying their scalability up to thousands of Armv8 cores. For comparison, we run the same tests on state-of-the-art x86 nodes included in Dibona and the Tier-0 supercomputer MareNostrum4. Our experiments show that the ThunderX2 has a 25% lower performance on average, mainly due to its small vector unit yet somewhat compensated by its 30% wider links between the CPU and the main memory. We found that the software ecosystem of the Armv8 architecture is comparable to the one available for Intel. Our results also show that ThunderX2 delivers similar or better energy-to-solution and scalability, proving that Arm-based chips are legitimate contenders in the market of next-generation HPC systems., The authors thank the support team of Dibona operating at ATOS/Bull. This work is partially supported by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), the Spanish Ministry of Science and Technology project (TIN2015-65316-P), the Generalitat de Catalunya (2017-SGR-1414), the European Community’s Seventh Framework Programme [FP7/2007-2013] and Horizon 2020 under the Mont-Blanc projects (grant agreements n. 288777, 610402 and 671697), the European POP2 Center of Excellence (grant agreement n. 824080), and the Human Brain Project SGA2 (grant agreement n. 785907)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Locality-aware cache random replacement policies

  • Benedicte Illescas, Pedro|||0000-0003-1670-7783
  • Hernández Luz, Carles
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Cazorla Almeida, Francisco Javier
© <2018> Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Measurement-Based Probabilistic Timing Analysis (MBPTA) facilitates the analysis of complex software running on hardware comprising high-performance features. MBPTA also aims at preventing additional analysis costs for timing analysis techniques and preserving the confidence on derived WCET estimates. Cache behavior has a deep influence on WCET estimates and hence on “the amount of software” that can be consolidated onto a single hardware platform. Deterministic replacement policies such as LRU (Least Recently Used) and NMRU (Non-Most Recently Used) have systematic pathological cases that may lead to high execution times and WCET estimates. Instead, random replacement (RR) decreases pathological cases probability, at the cost of temporal locality. We present two new MBPTA-amenable replacement policies that completely remove the presented pathological cases. The first policy, Random Permutations (RP) preserves higher temporal locality than RR; while the second, NMRU Random Permutations (NMRURP), also protects the Most Recently Used line from eviction. Both proposed policies build upon restricted random replacement choices. Our simulation evaluation (validated against a real prototype) using the Mälardalen benchmarks and a case study shows that RP and NMRURP deliver both high average performance (within 1% of LRUs and NRMU performance) and tight WCET estimates 11% and 24% lower than those of RR., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Vector processing-aware advanced clock-gating techniques for low-power fused multiply-add

  • Ratkovic, Ivan
  • Palomar Pérez, Óscar|||0000-0001-6729-4187
  • Stanic, Milan
  • Unsal, Osman Sabri
  • Cristal Kestelman, Adrián|||0000-0003-1277-9296
  • Valero Cortés, Mateo|||0000-0003-2917-2482
The need for power efficiency is driving a rethink of design decisions in processor architectures. While vector processors succeeded in the high-performance market in the past, they need a retailoring for the mobile market that they are entering now. Floating-point (FP) fused multiply-add (FMA), being a functional unit with high power consumption, deserves special attention. Although clock gating is a well-known method to reduce switching power in synchronous designs, there are unexplored opportunities for its application to vector processors, especially when considering active operating mode. In this research, we comprehensively identify, propose, and evaluate the most suitable clock-gating techniques for vector FMA units (VFUs). These techniques ensure power savings without jeopardizing the timing. We evaluate the proposed techniques using both synthetic and “real-world” application-based benchmarking. Using vector masking and vector multilane-aware clock gating, we report power reductions of up to 52%, assuming active VFU operating at the peak performance. Among other findings, we observe that vector instruction-based clock-gating techniques achieve power savings for all vector FP instructions. Finally, when evaluating all techniques together, using “real-world” benchmarking, the power reductions are up to 80%. Additionally, in accordance with processor design trends, we perform this research in a fully parameterizable and automated fashion., The research leading to these results has received funding from the RoMoL ERC Advanced Grant GA 321253 and is supported in part by the European Union (FEDER funds) under contract TTIN2015-65316-P. The work of I. Ratkovic was supported by a FPU research grant from the Spanish MECD., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

DRMaestro: orchestrating disaggregated resources on virtualized data-centers

  • Carneiro do Amaral, Marcelo
  • Polo Bardés, Jorda
  • Carrera Pérez, David|||0000-0003-4898-3424
  • González, Nelson
  • Yang, Chih-Chieh
  • Morari, Alessandro
  • D'Amora, Bruce
  • Youssef, Alaa
  • Steinder, Malgorzata
Modern applications demand resources at an unprecedented level. In this sense, data-centers are required to scale efficiently to cope with such demand. Resource disaggregation has the potential to improve resource-efficiency by allowing the deployment of workloads in more flexible ways. Therefore, the industry is shifting towards disaggregated architectures, which enables new ways to structure hardware resources in data centers. However, determining the best performing resource provisioning is a complicated task. The optimality of resource allocation in a disaggregated data center depends on its topology and the workload collocation. This paper presents DRMaestro, a framework to orchestrate disaggregated resources transparently from the applications. DRMaestro uses a novel flow-network model to determine the optimal placement in multiple phases while employing best-efforts on preventing workload performance interference. We first evaluate the impact of disaggregation regarding the additional network requirements under higher network load. The results show that for some applications the impact is minimal, but other ones can suffer up to 80% slowdown in the data transfer part. After that, we evaluate DRMaestro via a real prototype on Kubernetes and a trace-driven simulation. The results show that DRMaestro can reduce the total job makespan with a speedup of up to ≈1.20x and decrease the QoS violation up to ≈2.64x comparing with another orchestrator that does not support resource disaggregation., This project is supported by the IBM/BSC Technology Center for Supercomputing collaboration agreement. It has also received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 639595). It is also partially supported by the Ministry of Economy of Spain under contract TIN2015-65316-P and Generalitat de Catalunya under contract 2014SGR1051, by the ICREA Academia program, and by the BSC-CNS Severo Ochoa program (SEV-2015-0493)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A visual embedding for the unsupervised extraction of abstract semantics

  • Garcia Gasulla, Dario|||0000-0001-6732-5641
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
  • Béjar Alonso, Javier|||0000-0001-5281-3888
  • Cortés García, Claudio Ulises|||0000-0003-0192-3096
  • Suzumura, Toyotaro
  • Chen, R
Vector-space word representations obtained from neural network models have been shown to enable semantic operations based on vector arithmetic. In this paper, we explore the existence of similar information on vector representations of images. For that purpose we define a methodology to obtain large, sparse vector representations of image classes, and generate vectors through the state-of-the-art deep learning architecture GoogLeNet for 20 K images obtained from ImageNet. We first evaluate the resultant vector-space semantics through its correlation with WordNet distances, and find vector distances to be strongly correlated with linguistic semantics. We then explore the location of images within the vector space, finding elements close in WordNet to be clustered together, regardless of significant visual variances (e.g., 118 dog types). More surprisingly, we find that the space unsupervisedly separates complex classes without prior knowledge (e.g., living things). Afterwards, we consider vector arithmetics. Although we are unable to obtain meaningful results on this regard, we discuss the various problem we encountered, and how we consider to solve them. Finally, we discuss the impact of our research for cognitive systems, focusing on the role of the architecture being used., This work is partially supported by the Joint Study Agreement no. W156463 under the IBM/BSC Deep Learning Center agreement, by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project and by the Generalitat de Catalunya (contracts 2014-SGR-1051), and by the Core Research for Evolutional Science and Technology (CREST) program of Japan Science and Technology Agency (JST)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

On the assessment of probabilistic WCET estimates reliability for arbitrary programs

  • Milutinovic, Suzana|||0000-0003-2429-0949
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Cazorla, Francisco
Measurement-Based Probabilistic Timing Analysis (MBPTA) has been shown to be an industrially viable method to estimate the Worst-Case Execution Time (WCET) of real-time programs running on processors including several high-performance features. MBPTA requires hardware/software support so that program’s execution time, and so its WCET, has a probabilistic behaviour and can be modelled with probabilistic and statistic methods. MBPTA also requires that those events with high impact on execution time are properly captured in the (R) runs made at analysis time. Thus, a representativeness argument is needed to provide evidence that those events have been captured. This paper addresses the MBPTA representativeness problems caused by set-associative caches and presents a novel representativeness validation method (ReVS) for cache placement. Building on cache simulation, ReVS explores the probability and impact (miss count) of those cache placements that can occur during operation. ReVS determines the number of runs R ', which can be higher than R, such that those cache placements with the highest impact are effectively observed in the analysis runs, and hence, MBPTA can be reliably applied to estimate the WCET., This work has received funding from the European Community’s FP7 program [FP7/2007–2013] under grant agreement 611085 (PROXIMA, www.proximaproject.eu). Support was also provided by the Ministry of Science and Technology of Spain under contract TIN2015-65316-P and the HiPEAC Network of Excellence. Jaume Abella has been partially supported by the MINECO under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Using Arm’s scalable vector extension on stencil codes

  • Armejach Sanosa, Adrià|||0000-0003-2869-668X
  • Caminal Pallarés, Helena
  • Cebrián González, Juan Manuel
  • Langarita, Rubén
  • González-Alberquilla, Rekai
  • Adeniyi-Jones, Chris
  • Valero Cortés, Mateo|||0000-0003-2917-2482
  • Casas, Marc|||0000-0003-4564-2093
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Data-level parallelism is frequently ignored or underutilized. Achieved through vector/SIMD capabilities, it can provide substantial performance improvements on top of widely used techniques such as thread-level parallelism. However, manual vectorization is a tedious and costly process that needs to be repeated for each specific instruction set or register size. In addition, automatic compiler vectorization is susceptible to code complexity, and usually limited due to data and control dependencies. To address some of these issues, Arm recently released a new vector ISA, the scalable vector extension (SVE), which is vector-length agnostic (VLA). VLA enables the generation of binary files that run regardless of the physical vector register length. In this paper, we leverage the main characteristics of SVE to implement and optimize stencil computations, ubiquitous in scientific computing. We show that SVE enables easy deployment of textbook optimizations like loop unrolling, loop fusion, load trading or data reuse. Our detailed simulations using vector lengths ranging from 128 to 2048 bits show that these optimizations can lead to performance improvements over straightforward vectorized code of up to 1.57×. In addition, we show that certain optimizations can hurt performance due to reduced arithmetic intensity and instruction overheads, and provide insight useful for compiler optimizers., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

ReD: A reuse detector for content selection in exclusive shared last-level caches

  • Díaz, Javier
  • Monreal Arnal, Teresa|||0000-0002-0458-2234
  • Ibáñez Marín, Pablo Enrique
  • Llaberia Griñó, José M.|||0000-0002-3753-4108
  • Viñals Yúfera, Víctor
The reference stream reaching a chip multiprocessor Shared Last-Level Cache (SLLC) shows poor temporal locality, making conventional cache management policies inefficient. Few proposals address this problem for exclusive caches. In this paper, we propose the Reuse Detector (ReD), a new content selection mechanism for exclusive hierarchies that leverages reuse locality at the SLLC, a property that states that blocks referenced more than once are more likely to be accessed in the near future. Being placed between each L2 private cache and the SLLC, ReD prevents the insertion of blocks without reuse into the SLLC. It is designed to overcome problems affecting similar recent mechanisms (low accuracy, reduced visibility window and detector thrashing). ReD improves performance over other state-of-the-art proposals (CHAR, Reuse Cache and EAF cache). Compared with the baseline system with no content selection, it reduces the SLLC miss rate (MPI) by 10.1% and increases harmonic IPC by 9.5%., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Increasing the Reliability of Software Timing Analysis for Cache-Based Processors

  • Milutinovic, Suzana|||0000-0003-2429-0949
  • Mezzetti, Enrico|||0000-0002-1886-2931
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Cazorla, Francisco J.
Real-time systems are witnessing a significant increase in critical software's size, complexity, and performance needs, which can only be satisfied with high-performance hardware features. Cache memories, pervasively used to improve average performance, complicate Worst-Case Execution Time analysis: cache placement (i.e., how software objects are mapped to cache) during the testing phase does not only critically affect the observed performance, but also proves to be arduous to control and preserve up to operation. The probabilistic variant of Measurement-Based Timing Analysis (MBPTA) responds to this challenge by deploying time-randomized caches that naturally explore a different random cache placement in each run, relieving the user from producing tests that intercept relevant Cache Conflict Placements (CCP). Yet, to meet an adequate probabilistic CCP coverage, the user is required to collect a minimum number of measurements. We present two mechanisms, CCP-RM and CCP-HRP, to identify CCP with relevant probability of occurrence and large impact on execution-time, for the random modulo (RM) and hash-based random placement (HRP) policies. CCP-RM and CCP-HRP enable a reliable application of MBPTA by computing the number of runs $R^{\prime }$R' necessary to meet the desired CCP coverage. We exhaustively evaluate CCP-RM and CCP-HRP, showing their effectiveness on well-known benchmarks and a railway case study, on top of an accurate simulator and a concrete RTL implementation., This work has received funding from the Spanish Ministry of Science and Innovation under grant TIN2015-65316-P and the HiPEAC Network of Excellence. The Ministry of Economy and Competitiveness partially supported Suzana Milutinovic under FPI grant (BES-2016-077561), Jaume Abella under Ramon y Cajal postdoctoral fellowship (RYC-2013-14717) and Enrico Mezzetti under Juan de la Cierva-Incorporación postdoctoral fellowship (IJCI-2016-27396)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A resilient and distributed near real-time traffic forecasting application for Fog computing environments

  • Pérez, Juan L.
  • Gutierrez-Torre, Alberto
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Carrera, David
In this paper we propose an architecture for a city-wide traffic modeling and prediction service based on the Fog Computing paradigm. The work assumes an scenario in which a number of distributed antennas receive data generated by vehicles across the city. In the Fog nodes data is collected, processed in local and intermediate nodes, and finally forwarded to a central Cloud location for further analysis. We propose a combination of a data distribution algorithm, resilient to back-haul connectivity issues, and a traffic modeling approach based on deep learning techniques to provide distributed traffic forecasting capabilities. In our experiments, we leverage real traffic logs from one week of Floating Car Data (FCD) generated in the city of Barcelona by a road-assistance service fleet comprising thousands of vehicles. FCD was processed across several simulated conditions, ranging from scenarios in which no connectivity failures occurred in the Fog nodes, to situations with long and frequent connectivity outage periods. For each scenario, the resilience and accuracy of both the data distribution algorithm, and the learning methods were analyzed. Results show that the data distribution process running in the Fog nodes is resilient to back-haul connectivity issues and is able to deliver data to the Cloud location even in presence of severe connectivity problems. Additionally, the proposed traffic modeling and forecasting method exhibits better behavior when run distributed in the Fog instead of centralized in the Cloud, especially when connectivity issues occur that force data to be delivered out of order to the Cloud., This project is partially supported by the European Research Council (ERC), Spain under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 639595). It is also partially supported by the Ministry of Economy of Spain under contract TIN2015-65316-P and Generalitat de Catalunya, Spain under contract 2014SGR1051, by the ICREA Academia program, and by the BSC-CNS Severo Ochoa program (SEV-2015-0493). The authors gratefully acknowledge the Reial Automvil Club de Catalunya (RACC) for the dataset of Floating Car Data provided., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reuse Detector: improving the management of STT-RAM SLLCs

  • Rodríguez Rodríguez, Roberto
  • Díaz Maag, Javier
  • Castro, Fernando
  • Ibáñez Marín, Pablo Enrique
  • Chaver Martínez, Daniel A.
  • Viñals Yúfera, Víctor
  • Sáez Alcaide, Juan Carlos
  • Prieto Matías, Manuel
  • Piñuel, Luis
  • Monreal Arnal, Teresa|||0000-0002-0458-2234
  • Llaberia Griñó, José M.|||0000-0002-3753-4108
Various constraints of Static Random Access Memory (SRAM) are leading to consider new memory technologies as candidates for building on-chip shared last-level caches (SLLCs). Spin-Transfer Torque RAM (STT-RAM) is currently postulated as the prime contender due to its better energy efficiency, smaller die footprint and higher scalability. However, STT-RAM also exhibits some drawbacks, like slow and energy-hungry write operations that need to be mitigated before it can be used in SLLCs for the next generation of computers. In this work, we address these shortcomings by leveraging a new management mechanism for STT-RAM SLLCs. This approach is based on the previous observation that although the stream of references arriving at the SLLC of a Chip MultiProcessor (CMP) exhibits limited temporal locality, it does exhibit reuse locality, i.e. those blocks referenced several times manifest high probability of forthcoming reuse. As such, conventional STT-RAM SLLC management mechanisms, mainly focused on exploiting temporal locality, result in low efficient behavior. In this paper, we employ a cache management mechanism that selects the contents of the SLLC aimed to exploit reuse locality instead of temporal locality. Specifically, our proposal consists in the inclusion of a Reuse Detector (RD) between private cache levels and the STT-RAM SLLC. Its mission is to detect blocks that do not exhibit reuse, in order to avoid their insertion in the SLLC, hence reducing the number of write operations and the energy consumption in the STT-RAM. Our evaluation, using multiprogrammed workloads in quad-core, eight-core and 16-core systems, reveals that our scheme reports on average, energy reductions in the SLLC in the range of 37–30%, additional energy savings in the main memory in the range of 6–8% and performance improvements of 3% (quad-core), 7% (eight-core) and 14% (16-core) compared with an STT-RAM SLLC baseline where no RD is employed. More importantly, our approach outperforms DASCA, the state-of-the-art STT-RAM SLLC management, reporting—depending on the specific scenario and the kind of applications used—SLLC energy savings in the range of 4–11% higher than those of DASCA, delivering higher performance in the range of 1.5–14% and additional improvements in DRAM energy consumption in the range of 2–9% higher than DASCA., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A new strategy for multitarget drug discovery/repositioning through the identification of similar 3D amino acid patterns among proteins structures: The case of Tafluprost and its efects on cardiac ion channels

  • Valdés Jiménez, Alejandro Mauricio
  • Jiménez González, Daniel|||0000-0001-6064-7883
  • Kiper, Aytug K.
  • Rinné, Susanne
  • Decher, Niels
  • González, Wendy
  • Reyes Parada, Miguel
  • Núñez Vivanco, Gabriel
The identification of similar three-dimensional (3D) amino acid patterns among different proteins might be helpful to explain the polypharmacological profile of many currently used drugs. Also, it would be a reasonable first step for the design of novel multitarget compounds. Most of the current computational tools employed for this aim are limited to the comparisons among known binding sites, and do not consider several additional important 3D patterns such as allosteric sites or other conserved motifs. In the present work, we introduce Geomfinder2.0, which is a new and improved version of our previously described algorithm for the deep exploration and discovery of similar and druggable 3D patterns. As compared with the original version, substantial improvements that have been incorporated to our software allow: (i) to compare quaternary structures, (ii) to deal with a list of pairs of structures, (iii) to know how druggable is the zone where similar 3D patterns are detected and (iv) to significantly reduce the execution time. Thus, the new algorithm achieves up to 353x speedup as compared to the previous sequential version, allowing the exploration of a significant number of quaternary structures in a reasonable time. In order to illustrate the potential of the updated Geomfinder version, we show a case of use in which similar 3D patterns were detected in the cardiac ions channels NaV1.5 and TASK-1. These channels are quite different in terms of structure, sequence and function and both have been regarded as important targets for drugs aimed at treating atrial fibrillation. Finally, we describe the in vitro effects of tafluprost (a drug currently used to treat glaucoma, which was identified as a novel putative ligand of NaV1.5 and TASK-1) upon both ion channels’ activity and discuss its possible repositioning as a novel antiarrhythmic drug., This research was funded by the Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT) grants numbers 1191133, 1170662 and from Spanish Ministry of Economy and Competitiveness (projects SEV-2015-0493 and TIN2015-65316-P, grant BES-2016-078046), and from Generalitat de Catalunya (contracts 2017-SGR-1414 and 2017-SGR-1328). The financial support by DICYT-USACH grant 5392102RP-ACDicyt is also acknowledged. The web-server is hosted in the cluster obtained with the grant CONICYT-FONDEQUIP-EQM160063., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

RVOS: end-to-end recurrent network for video object segmentation

  • Ventura Royo, Carles|||0000-0002-3055-339X
  • Bellver, Míriam
  • Girbau Xalabarder, Andreu
  • Salvador Aguilera, Amaia|||0000-0002-9908-1685
  • Marqués Acosta, Fernando|||0000-0001-8311-1168
  • Giró Nieto, Xavier|||0000-0002-9935-5332
Multiple object video object segmentation is a challenging task, specially for the zero-shot case, when no object mask is given at the initial frame and the model has to find the objects to be segmented along the sequence. In our work, we propose a Recurrent network for multiple object Video Object Segmentation (RVOS) that is fully end-to-end trainable. Our model incorporates recurrence on two different domains: (i) the spatial, which allows to discover the different object instances within a frame, and (ii) the temporal, which allows to keep the coherence of the segmented objects along time. We train RVOS for zero-shot video object segmentation and are the first ones to report quantitative results for DAVIS-2017 and YouTube-VOS benchmarks. Further, we adapt RVOS for one-shot video object segmentation by using the masks obtained in previous time steps as inputs to be processed by the recurrent module. Our model reaches comparable results to state-of-the-art techniques in YouTube-VOS benchmark and outperforms all previous video object segmentation methods not using online learning in the DAVIS-2017 benchmark. Moreover, our model achieves faster inference runtimes than previous methods, reaching 44ms/frame on a P100 GPU., This research was supported by the Spanish Ministry ofEconomy and Competitiveness and the European RegionalDevelopment Fund (TIN2015-66951-C2-2-R, TIN2015-65316-P & TEC2016-75976-R), the BSC-CNS SeveroOchoa SEV-2015-0493 and LaCaixa-Severo Ochoa Inter-national Doctoral Fellowship programs, the 2017 SGR 1414and the Industrial Doctorates 2017-DI-064 & 2017-DI-028from the Government of Catalonia, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Automatic generation of workload profiles using unsupervised learning pipelines

  • Buchaca Prats, David
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Carrera Pérez, David|||0000-0003-4898-3424
The complexity of resource usage and power consumption on cloud-based applications makes the understanding of application behavior through expert examination difficult. The difficulty increases when applications are seen as “black boxes”, where only external monitoring can be retrieved. Furthermore, given the different amount of scenarios and applications, automation is required. Here we examine and model application behavior by finding behavior phases. We use Conditional Restricted Boltzmann Machines (CRBM) to model time-series containing resources traces measurements like CPU, Memory and IO. CRBMs can be used to map a given given historic window of trace behaviour into a single vector. This low dimensional and time-aware vector can be passed through clustering methods, from simplistic ones like k-means to more complex ones like those based on Hidden Markov Models (HMM). We use these methods to find phases of similar behaviour in the workloads. Our experimental evaluation shows that the proposed method is able to identify different phases of resource consumption across different workloads. We show that the distinct phases contain specific resource patterns that distinguish them., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

“Dust in the wind...”, deep learning application to wind energy time series forecasting

  • Manero Font, Jaume
  • Béjar Alonso, Javier|||0000-0001-5281-3888
  • Cortés García, Claudio Ulises|||0000-0003-0192-3096
To balance electricity production and demand, it is required to use different prediction techniques extensively. Renewable energy, due to its intermittency, increases the complexity and uncertainty of forecasting, and the resulting accuracy impacts all the different players acting around the electricity systems around the world like generators, distributors, retailers, or consumers. Wind forecasting can be done under two major approaches, using meteorological numerical prediction models or based on pure time series input. Deep learning is appearing as a new method that can be used for wind energy prediction. This work develops several deep learning architectures and shows their performance when applied to wind time series. The models have been tested with the most extensive wind dataset available, the National Renewable Laboratory Wind Toolkit, a dataset with 126,692 wind points in North America. The architectures designed are based on different approaches, Multi-Layer Perceptron Networks (MLP), Convolutional Networks (CNN), and Recurrent Networks (RNN). These deep learning architectures have been tested to obtain predictions in a 12-h ahead horizon, and the accuracy is measured with the coefficient of determination, the R² method. The application of the models to wind sites evenly distributed in the North America geography allows us to infer several conclusions on the relationships between methods, terrain, and forecasting complexity. The results show differences between the models and confirm the superior capabilities on the use of deep learning techniques for wind speed forecasting from wind time series data., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Extending the OpenCHK Model with advanced checkpoint features

  • Maroñas Bravo, Marcos
  • Mateo Bellido, Sergi
  • Keller, Kai Rasmus
  • Bautista Gomez, Leonardo|||0000-0002-0814-5779
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Beltran Querol, Vicenç
One of the major challenges in using extreme scale systems efficiently is to mitigate the impact of faults. Application-level checkpoint/restart (CR) methods provide the best trade-off between productivity, robustness, and performance. There are many solutions implementing CR at the application level. They all provide advanced I/O capabilities to minimize the overhead introduced by CR. Nevertheless, there is still room for improvement in terms of programmability and flexibility, because end-users must manually serialize and deserialize application state using low-level APIs, modify the flow of the application to consider restarts, or rewrite CR code whenever the backend library changes. In this work, we propose a set of compiler directives and clauses that allow users to specify CR operations in a simple way. Our approach supports the common CR features provided by all the CR libraries. However, it can also be extended to support advanced features that are only available in some CR libraries, such as differential checkpointing, the use of HDF5 format, and the possibility of using fault-tolerance-dedicated threads. The result of our evaluation revealed a high increase in programmability. On average, we reduced the number of lines of code by 71%, 94%, and 64% for FTI, SCR, and VeloC, respectively, and no additional overhead was perceived using our solution compared to using the backend libraries directly. Finally, portability is enhanced because our programming model allows the use of any backend library without changing any code., This work is supported by the Spanish Ministerio de Ciencia, Innovación y Universidades (TIN2015-65316-P) and by the Generalitat de Catalunya (2014-SGR-1051). This project received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) and the Horizon 2020 (H2020) funding framework under grant agreement no. H2020-FETHPC-754304 (DEEPEST)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Considerations in using OpenCL on GPUs and FPGAs for throughput-oriented genomics workloads

  • Cadenelli, Nicola
  • Jaksic, Zoran
  • Polo Bardés, Jordà
  • Carrera Pérez, David|||0000-0003-4898-3424
The recent upsurge in the available amount of health data and the advances in next-generation sequencing are setting the ground for the long-awaited precision medicine. To process this deluge of data, bioinformatics workloads are becoming more complex and more computationally demanding. For this reasons they have been extended to support different computing architectures, such as GPUs and FPGAs, to leverage the form of parallelism typical of each of such architectures. The paper describes how a genomic workload such as k-mer frequency counting that takes advantage of a GPU can be offloaded to one or even more FPGAs. Moreover, it performs a comprehensive analysis of the FPGA acceleration comparing its performance to a non-accelerated configuration and when using a GPU. Lastly, the paper focuses on how, when using accelerators with a throughput-oriented workload, one should also take into consideration both kernel execution time and how well each accelerator board overlaps kernels and PCIe transferred. Results show that acceleration with two FPGAs can improve both time- and energy-to-solution for the entire accelerated part by a factor of 1.32x. Per contra, acceleration with one GPU delivers an improvement of 1.77x in time-to-solution but of a lower 1.49x in energy-to-solution due to persistently higher power consumption. The paper also evaluates how future FPGA boards with components (i.e., off-chip memory and PCIe) on par with those of the GPU board could provide an energy-efficient alternative to GPUs., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

BioExcel Building Blocks, a software library for interoperable biomolecular simulation workflows

  • Andrio, Pau
  • Hospital, Adam
  • Conejero Bañón, Javier
  • Jordá, Luis
  • del Pino, Marc
  • Codo, Laia
  • Soiland Reyes, Stian
  • Goble, Carole
  • Lezzi, Daniele|||0000-0001-5081-7244
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Orozco López, Modesto
  • Gelpi Buchaca, Josep Lluís
In the recent years, the improvement of software and hardware performance has made biomolecular simulations a mature tool for the study of biological processes. Simulation length and the size and complexity of the analyzed systems make simulations both complementary and compatible with other bioinformatics disciplines. However, the characteristics of the software packages used for simulation have prevented the adoption of the technologies accepted in other bioinformatics fields like automated deployment systems, workflow orchestration, or the use of software containers. We present here a comprehensive exercise to bring biomolecular simulations to the “bioinformatics way of working”. The exercise has led to the development of the BioExcel Building Blocks (BioBB) library. BioBB’s are built as Python wrappers to provide an interoperable architecture. BioBB’s have been integrated in a chain of usual software management tools to generate data ontologies, documentation, installation packages, software containers and ways of integration with workflow managers, that make them usable in most computational environments., We thank The Barcelona Supercomputing Center for providing all the HPC resources to launch the test executions presented in this manuscript. Te project was supported by the following grants: BioExcel Center of Excellence (Horizon 2020 Framework Programme. Grants 675728, and 823830). Elixir-Excelerate (Horizon 2020 Framework Programme. Grant 676559). Spanish National Institute for Bioinformatics (Institute of Health Carlos III. Grants PT13/0001/0019, PT13/0001/0028, PT17/0009/0001). Spanish Government, Severo Ochoa Grant SEV2015-0493, TIN2015-65316-P, Generalitat de Catalunya. 2014-SGR-1051, 2017-SGR-1110., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

AxleDB: A novel programmable query processing platform on FPGA

  • Salami, Behzad
  • Malazgirt, Gorker Alp
  • Arcas-Abella, Oriol
  • Yurdakul, Arda
  • Sonmez, Nehir
With the rise of Big Data, providing high-performance query processing capabilities through the acceleration of the database analytic has gained significant attention. Leveraging Field Programmable Gate Array (FPGA) technology, this approach can lead to clear benefits. In this work, we present the design and implementation of AxleDB: An FPGA-based platform that enables fast query processing for database systems by melding novel database-specific accelerators with commercial-off-the-shelf (COTS) storage using modern interfaces, in a novel, unified, and a programmable environment. AxleDB can perform a large subset of SQL queries through its set of instructions that can map compute-intensive database operations, such as filter, arithmetic, aggregate, group by, table join, or sort, on to the specialized high-throughput accelerators. To minimize the amount of SSD I/O operations required, AxleDB also supports hardware MinMax indexing for databases. We evaluated AxleDB with five decision support queries from the TPC-H benchmark suite and achieved a speedup from 1.8X to 34.2X and energy efficiency from 2.8X to 62.1X, in comparison to the state-of-the-art DBMS, i.e., PostgreSQL and MonetDB., The research leading to these results has received funding from the European Union Seventh Framework Program (FP7) (under the AXLE project GA number 318633), the Ministry of Economy and Competitiveness of Spain (under contract number TIN2015-65316-p), Turkish Ministry of Development TAM Project (number 2007K120610), and Bogazici University Scientific Projects (number 7060)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A case for malleable thread-level linear algebra libraries: The LU factorization with partial pivoting

  • Catalán Pallarés, Sandra
  • Herrero Zaragoza, José Ramón|||0000-0002-4060-367X
  • Quintana Ortí, Enrique Salvador
  • Rodríguez Sánchez, Rafael
  • Van De Geijn, Robert
We propose two novel techniques for overcoming load-imbalance encountered when implementing so-called look-ahead mechanisms in relevant dense matrix factorizations for the solution of linear systems. Both techniques target the scenario where two thread teams are created/activated during the factorization, with each team in charge of performing an independent task/branch of execution. The first technique promotes worker sharing (WS) between the two tasks, allowing the threads of the task that completes first to be reallocated for use by the costlier task. The second technique allows a fast task to alert the slower task of completion, enforcing the early termination (ET) of the second task, and a smooth transition of the factorization procedure into the next iteration. The two mechanisms are instantiated via a new malleable thread-level implementation of the basic linear algebra subprograms, and their benefits are illustrated via an implementation of the LU factorization with partial pivoting enhanced with look-ahead. Concretely, our experimental results on an Intel-Xeon system with 12 cores show the benefits of combining WS+ET, reporting competitive performance in comparison with a task-parallel runtime-based solution., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

WFA-GPU: Gap-affine pairwise read-alignment using GPUs

  • Aguado Puig, Quim|||0000-0003-4871-3192
  • Doblas Font, Max|||0000-0002-8909-3033
  • Matzoros, Christos
  • Espinosa Morales, Antonio
  • Moure López, Juan Carlos
  • Marco Sola, Santiago|||0000-0001-7951-3914
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Motivation: Advances in genomics and sequencing technologies demand faster and more scalable analysis methods that can process longer sequences with higher accuracy. However, classical pairwise alignment methods, based on dynamic programming (DP), impose impractical computational requirements to align long and noisy sequences like those produced by PacBio, and Nanopore technologies. The recently proposed WFA algorithm paves the way for more efficient alignment tools, improving time and memory complexity over previous methods. However, high-performance computing (HPC) platforms require efficient parallel algorithms and tools to exploit the computing resources available on modern accelerator-based architectures. Results: This paper presents WFA-GPU, a GPU (Graphics Processing Unit)-accelerated tool to compute exact gap-affine alignments based on the WFA algorithm. We present the algorithmic adaptations and performance optimizations that allow exploiting the massively parallel capabilities of modern GPU devices to accelerate the alignment computations. In particular, we propose a CPU-GPU co-design capable of performing inter-sequence and intra-sequence parallel sequence alignment, combining a succinct WFA-data representation with an efficient GPU implementation. As a result, we demonstrate that our implementation outperforms the original multi-threaded WFA implementation by up to 4.3 × and up to 18.2 × when using heuristic methods on long and noisy sequences. Compared to other state-of-the-art tools and libraries, the WFA-GPU is up to 29 × faster than other GPU implementations and up to four orders of magnitude faster than other CPU implementations. Furthermore, WFA-GPU is the only GPU solution capable of correctly aligning long reads using a commodity GPU., This research was supported by the European Union Regional Development Fund within the framework of the ERDF Operational Program of Catalonia 2014-2020 with a grant of 50% of total cost eligible under theDRAC project [001-P-001723] and Lenovo-BSC Contract-Framework Contract (2020). This work has also been granted by the Spanish Ministerio de Ciencia e Innovacion MCIN AEI/10.13039/501100011033 under contracts [PID2020-113614RB-C21] and [TIN2015-65316-P],NextGenerationEU/PRTR (projectTED2021132634A-I00), and by the Generalitat de Catalunya GenCat-DIUiE (GRR) (contracts [2021-SGR-00574], [2017-SGR-1328], [2017-SGR313], and [2017-SGR-1414]). M.M. was partially supported by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal fellowship number [RYC-2016-21104]. S.M. was supported by Juan de la Cierva fellowship grant [IJC2020-045916-I] funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR. Q.A.was supported by the Spanish Ministerio de Ciencia e Innovacion under grant [PRE2021-101059]., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reducing memory requirements for large size LBM simulations on GPUs

  • Valero-Lara, Pedro
The scientific community in its never-ending road of larger and more efficient computational resources is in need of more efficient implementations that can adapt efficiently on the current parallel platforms. Graphics processing units are an appropriate platform that cover some of these demands. This architecture presents a high performance with a reduced cost and an efficient power consumption. However, the memory capacity in these devices is reduced and so expensive memory transfers are necessary to deal with big problems. Today, the lattice-Boltzmann method (LBM) has positioned as an efficient approach for Computational Fluid Dynamics simulations. Despite this method is particularly amenable to be efficiently parallelized, it is in need of a considerable memory capacity, which is the consequence of a dramatic fall in performance when dealing with large simulations. In this work, we propose some initiatives to minimize such demand of memory, which allows us to execute bigger simulations on the same platform without additional memory transfers, keeping a high performance. In particular, we present 2 new implementations, LBM-Ghost and LBM-Swap, which are deeply analyzed, presenting the pros and cons of each of them., This project was funded by the Spanish Ministry of Economy and Competitiveness (MINECO): BCAM Severo Ochoa accreditation SEV-2013-0323, MTM2013-40824, Computación de Altas Prestaciones VII TIN2015-65316-P, by the Basque Excellence Research Center (BERC 2014-2017) pro- gram by the Basque Government, and by the Departament d' Innovació, Universitats i Empresa de la Generalitat de Catalunya, under project MPEXPAR: Models de Programació i Entorns d' Execució Paral·lels (2014-SGR-1051). We also thank the support of the computing facilities of Extremadura Research Centre for Advanced Technologies (CETA-CIEMAT) and NVIDIA GPU Research Center program for the provided resources, as well as the support of NVIDIA through the BSC/UPC NVIDIA GPU Center of Excellence., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Simulating the behavior of the human brain on GPUS

  • Valero-Lara, Pedro
  • Martinez-Perez, Ivan
  • Sirvent, Raul
  • Peña, Antonio J.
  • Martorell Bofill, Xavier|||0000-0002-0417-3430
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
The simulation of the behavior of the Human Brain is one of the most important challenges in computing today. The main problem consists of finding efficient ways to manipulate and compute the huge volume of data that this kind of simulations need, using the current technology. In this sense, this work is focused on one of the main steps of such simulation, which consists of computing the Voltage on neurons’ morphology. This is carried out using the Hines Algorithm and, although this algorithm is the optimum method in terms of number of operations, it is in need of non-trivial modifications to be efficiently parallelized on GPUs. We proposed several optimizations to accelerate this algorithm on GPU-based architectures, exploring the limitations of both, method and architecture, to be able to solve efficiently a high number of Hines systems (neurons). Each of the optimizations are deeply analyzed and described. Two different approaches are studied, one for mono-morphology simulations (batch of neurons with the same shape) and one for multi-morphology simulations (batch of neurons where every neuron has a different shape). In mono-morphology simulations we obtain a good performance using just a single kernel to compute all the neurons. However this turns out to be inefficient on multi-morphology simulations. Unlike the previous scenario, in multi-morphology simulations a much more complex implementation is necessary to obtain a good performance. In this case, we must execute more than one single GPU kernel. In every execution (kernel call) one specific part of the batch of the neurons is solved. These parts can be seen as multiple and independent tridiagonal systems. Although the present paper is focused on the simulation of the behavior of the Human Brain, some of these techniques, in particular those related to the solving of tridiagonal systems, can be also used for multiple oil and gas simulations. Our studies have proven that the optimizations proposed in the present work can achieve high performance on those computations with a high number of neurons, being our GPU implementations about 4× and 8× faster than the OpenMP multicore implementation (16 cores), using one and two NVIDIA K80 GPUs respectively. Also, it is important to highlight that these optimizations can continue scaling, even when dealing with a very high number of neurons., This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 720270 (HBP SGA1), from the Spanish Ministry of Economy and Competitiveness under the project Computación de Altas Prestaciones VII (TIN2015-65316-P), the Departament d’Innovació, Universitats i Empresa de la Generalitat de Catalunya, under project MPEXPAR: Models de Programació i Entorns d’Execució Parallels (2014-SGR-1051). We thank the support of NVIDIA through the BSC/UPC NVIDIA GPU Center of Excellence, and the European Union’s Horizon 2020 Research and Innovation Program under the Marie Sklodowska-Curie Grant Agreement No. 749516., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

The Hipster Approach for Improving Cloud System Efficiency, The Hipster Approach for Improving Cloud System Efficiency

  • Nishtala, Rajiv
  • Carpenter, Paul Matthew
  • Petrucci, Vinicius
  • Martorell, Xavier
In 2013, U.S. data centers accounted for 2.2% of the country’s total electricity consumption, a figure that is projected to increase rapidly over the next decade. Many important data center workloads in cloud computing are interactive, and they demand strict levels of quality-of-service (QoS) to meet user expectations, making it challenging to optimize power consumption along with increasing performance demands. This article introduces Hipster, a technique that combines heuristics and reinforcement learning to improve resource efficiency in cloud systems. Hipster explores heterogeneous multi-cores and dynamic voltage and frequency scaling for reducing energy consumption while managing the QoS of the latency-critical workloads. To improve data center utilization and make best usage of the available resources, Hipster can dynamically assign remaining cores to batch workloads without violating the QoS constraints for the latency-critical workloads. We perform experiments using a 64-bit ARM big.LITTLE platform and show that, compared to prior work, Hipster improves the QoS guarantee for Web-Search from 80% to 96%, and for Memcached from 92% to 99%, while reducing the energy consumption by up to 18%. Hipster is also effective in learning and adapting automatically to specific requirements of new incoming workloads just enough to meet the QoS and optimize resource consumption., This work has been partially supported by the European Union FP7 program through the Mont-Blanc-3 (FP7-ICT-671697) and EUROSERVER (FP7-ICT-610456) projects, by the Ministerio de Economia y Competitividad under contract Computación de Altas Prestaciones VII (TIN2015- 65316-P), and the Departament de Innovació, Universitats i Empresa de la Generalitat de Catalunya, under project MPEXPAR: Models de Programació i Entorns d Execució Paral lels (2014-SGR-1051). Prior Publication: Rajiv Nishtala, Paul Carpenter, Vinicius Petrucci and Xavier Martorell. Hipster: Hybrid Task Manager for Latency-Critical Cloud Workloads. In Proceedings of the 23rd High Performance and Computer Architecture (HPCA 2017). In this work, we extend our previous work in several ways. First, we present an analysis of the size of the reward lookup table and an optimization for the table to improve the scalability of our reinforcement learning mechanism. Second, we demonstrate Hipster’s capability to adapt to changes in the latency-critical application at runtime and still satisfy QoS guarantees of the new incoming applications. Lastly, we present a deployment methodology for setting up new applications managed by Hipster’s runtime system. Author’s addresses: Rajiv Nishtala and Xavier Martorell, Universitat Politècnica de Catalunya and Barcelona Supercomputing Center; Paul Carpenter, Barcelona Supercomputing Center; Vincius Petrucci, Federal University of Bahia, Salvador, Brazil. emails:{rajiv.nishtala, paul.carpenter, xavier.martorell}@bsc.es; email: vpetrucci@ufba.br . ACM acknowledges that this contribution was authored or co-authored by an employee, or contractor of the national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. Permission to make digital or hard copies for personal or classroom use is granted. Copies must bear this notice and the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored. To copy otherwise, distribute, republish, or post, requires prior speci c permission and/or a fee. Request permissions from permissions@acm.org., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Burst-aware predictive autoscaling for containerized microservices

  • Abdullah, Muhammad
  • Iqbal, Waheed
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Polo Bardés, Jorda
  • Carrera Pérez, David|||0000-0003-4898-3424
Autoscaling methods are used for cloud-hosted applications to dynamically scale the allocated resources for guaranteeing Quality-of-Service (QoS). The public-facing application serves dynamic workloads, which contain bursts and pose challenges for autoscaling methods to ensure application performance. Existing State-of-the-art autoscaling methods are burst-oblivious to determine and provision the appropriate resources. For dynamic workloads, it is hard to detect and handle bursts online for maintaining application performance. In this article, we propose a novel burst-aware autoscaling method which detects burst in dynamic workloads using workload forecasting, resource prediction, and scaling decision making while minimizing response time service-level objectives (SLO) violations. We evaluated our approach through a trace-driven simulation, using multiple synthetic and realistic bursty workloads for containerized microservices, improving performance when comparing against existing state-of-the-art autoscaling methods. Such experiments show an increase of × 1.09 in total processed requests, a reduction of × 5.17 for SLO violations, and an increase of × 0.767 cost as compared to the baseline method., This work was partially supported by the European Research Council (ERC) under the EU Horizon 2020 programme (GA 639595), the Spanish Ministry of Economy, Industry and Competitiveness (TIN2015-65316-P and IJCI2016-27485) and the Generalitat de Catalunya (2014-SGR-1051)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Efficient development of high performance data analytics in Python

  • Álvarez Cid-Fuentes, Javier
  • Alvarez, Pol
  • Amela Milian, Ramon
  • Ishii, Kuninori
  • Morizawa, Rafael K.
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
Our society is generating an increasing amount of data at an unprecedented scale, variety, and speed. This also applies to numerous research areas, such as genomics, high energy physics, and astronomy, for which large-scale data processing has become crucial. However, there is still a gap between the traditional scientific computing ecosystem and big data analytics tools and frameworks. On the one hand, high performance computing (HPC) programming models lack productivity, and do not provide means for processing large amounts of data in a simple manner. On the other hand, existing big data processing tools have performance issues in HPC environments, and are not general-purpose. In this paper, we propose and evaluate PyCOMPSs, a task-based programming model for Python, as an excellent solution for distributed big data processing in HPC infrastructures. Among other useful features, PyCOMPSs offers a highly productive general-purpose programming model, is infrastructure-agnostic, and provides transparent data management with support for distributed storage systems. We show how two machine learning algorithms (Cascade SVM and K-means) can be developed with PyCOMPSs, and evaluate PyCOMPSs’ productivity based on these algorithms. Additionally, we evaluate PyCOMPSs performance on an HPC cluster using up to 1,536 cores and 320 million input vectors. Our results show that PyCOMPSs achieves similar performance and scalability to MPI in HPC infrastructures, while providing a much more productive interface that allows the easy development of data analytics algorithms., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement H2020-MSCA-COFUND2016-754433. This work has been supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), by Generalitat de Catalunya, Spain (contract 2014-SGR-1051). The research leading to these results has also received funding from the collaboration between Fujitsu and BSC (Script Language Platform)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Adaptive sliding windows for improved estimation of data center resource utilization

  • Baig, Shuja-ur-Rehman
  • Iqbal, Waheed
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Carrera Pérez, David|||0000-0003-4898-3424
Accurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous infrastructures, and multi-tenant co-hosted applications. Existing prediction methods use fixed size observation windows which cannot produce accurate results because of not being adaptively adjusted to capture local trends in the most recent data. Therefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick changing trends. In this paper we propose a deep learning-based adaptive window size selection method, dynamically limiting the sliding window size to capture the trend for the latest resource utilization, then build an estimation model for each trend period. We evaluate the proposed method against multiple baseline and state-of-the-art methods, using real data-center workload data sets. The experimental evaluation shows that the proposed solution outperforms those state-of-the-art approaches and yields 16 to 54% improved prediction accuracy compared to the baseline methods., This work is partially supported by the European ResearchCouncil (ERC) under the EU Horizon 2020 programme(GA 639595), the Spanish Ministry of Economy, Industry andCompetitiveness (TIN2015-65316-P and IJCI2016-27485), theGeneralitat de Catalunya, Spain (2014-SGR-1051) and Universityof the Punjab, Pakistan. The statements made herein are solelythe responsibility of the authors., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Formalization of block pruning: reducing the number of cells computed in exact biological sequence comparison algorithms

  • De Sandes, Edans
  • Teodoro, George
  • Walter, Maria Emilia
  • Martorell Bofill, Xavier|||0000-0002-0417-3430
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Melo, Alba
This is a pre-copyedited, author-produced version of an article accepted for publication in The Computer Journal following peer review. The version of record Edans F O Sandes, George L M Teodoro, Maria Emilia M T Walter, Xavier Martorell, Eduard Ayguade, Alba C M A Melo; Formalization of Block Pruning: Reducing the Number of Cells Computed in Exact Biological Sequence Comparison Algorithms, The Computer Journal, Volume 61, Issue 5, 1 May 2018, Pages 687–713 is available online at: The Computer Journal https://academic.oup.com/comjnl/article-abstract/61/5/687/4539903 and https://doi.org/10.1093/comjnl/bxx090., Biological sequence comparison algorithms that compute the optimal local and global alignments calculate a dynamic programming (DP) matrix with quadratic time complexity. The DP matrix H is calculated with a recurrence relation in which the value of each cell Hi,j is the result of a maximum operation on the cells’ values Hi-1,j-1, Hi-1,j and Hi,j-1 added or subtracted by a constant value. Therefore, it can be noticed that the difference between the value of cell Hi,j being calculated and the values of direct neighbor cells previously computed respect well-defined upper and lower bounds. Using these bounds, we can show that it is possible to determine the maximum and the minimum value of every cell in H, for a given reference cell. We use this result to define a generic pruning method which determines the cells that can pruned (i.e. no need to be computed since they will not contribute to the final solution), accelerating the computation but keeping the guarantee that the optimal result will be produced. The goal of this paper is thus to investigate and formalize properties of the DP matrix in order to estimate and increase the pruning method efficiency. We also show that the pruning efficiency depends mainly on three characteristics: (a) the order in which the cells of H are calculated, (b) the values of the parameters used in the recurrence relation and (c) the contents of the sequences compared., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

HDOT — An approach towards productive programming of hybrid applications

  • Ciesko, Jan
  • Martínez Ferrer, Pedro José|||0000-0002-6097-6676
  • Peñacoba Veigas, Raúl
  • Teruel García, Xavier
  • Beltran Querol, Vicenç
bulk synchronous parallel (BSP) communication model can hinder performance increases. This is due to the complexity to handle load imbalances, to reduce serialisation imposed by blocking communication patterns, to overlap communication with computation and, finally, to deal with increasing memory overheads. The MPI specification provides advanced features such as non-blocking calls or shared memory to mitigate some of these factors. However, applying these features efficiently usually requires significant changes on the application structure. Task parallel programming models are being developed as a means of mitigating the abovementioned issues but without requiring extensive changes on the application code. In this work, we present a methodology to develop hybrid applications based on tasks called hierarchical domain over-decomposition with tasking (HDOT). This methodology overcomes most of the issues found on MPI-only and traditional hybrid MPI+OpenMP applications. However, by emphasising the reuse of data partition schemes from process-level and applying them to task-level, it enables a natural coexistence between MPI and shared-memory programming models. The proposed methodology shows promising results in terms of programmability and performance measured on a set of applications., This work has been developed with the support of the European Union H2020 program through the INTERTWinE project (agreement number 671602); the Severo Ochoa Program awarded by the Spanish Government (SEV-2015-0493); the Generalitat de Catalunya (contract 2017-SGR-1414); and the Spanish Ministry of Science and Innovation (TIN2015-65316-P, Computaci on de Altas Prestaciones VII). The authors gratefully acknowledge Dr. Arnaud Mura, CNRS researcher at Institut PPRIME in France, for the numerical tool CREAMS. Finally, the manuscript has greatly bene ted from the precise comments of the reviewers., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

An approach to task-based parallel programming for undergraduate students

  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Jiménez González, Daniel|||0000-0001-6064-7883
This paper presents the description of a compulsory parallel programming course in the bachelor degree in Informatics Engineering at the Barcelona School of Informatics, Universitat Politècnica de Catalunya UPC-BarcelonaTech. The main focus of the course is on the shared-memory programming paradigm, which facilitates the presentation of fundamental aspects and notions of parallel computing. Unlike the “traditional” loop-based approach, which is the focus of parallel programming courses in other universities, this course presents the parallel programming concepts using a task-based approach. Tasking allows students to explore a broader set of parallel decomposition strategies, including linear, iterative and recursive strategies, and their implementation using the current version of OpenMP (OpenMP 4.5), which offers mechanisms (pragmas and intrinsic functions) to easily map these strategies into parallel programs. Simple models to understand the benefits of a task decomposition and the trade-offs introduced by different kinds of overheads are included in the course, together with the use of tools that allow an easy exploration of different task decomposition strategies and their potential parallelism (Tareador) and instrumentation and analysis of task parallel executions on real machines (Extrae and Paraver)., This work has been supported by the grant SEV-2015-0493 of the Severo Ochoa Program, awarded by the Spanish Gov- ernment, by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P) and by Generalitat de Catalunya (contracts 2014-MOOC-00057 and 2014-SGR-1051). We also thank the anonymous reviewers and editor for their comments during the review process, other professors that have been in- volved in the implementation of the course and Paul Carpenter at BSC for his corrections and suggestions to improve the text.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Emergent behaviors in the Internet of things: The ultimate ultra-large-scale system

  • Roca, Damian
  • Nemirovsky, Daniel
  • Nemirovsky, Mario
  • Milito, Rodolfo
  • Valero Cortés, Mateo|||0000-0003-2917-2482
To reach its potential, the Internet of Things (IoT) must break down the silos that limit applications' interoperability and hinder their manageability. Doing so leads to the building of ultra-large-scale systems (ULSS) in several areas, including autonomous vehicles, smart cities, and smart grids. The scope of ULSS is both large and complex. Thus, the authors propose Hierarchical Emergent Behaviors (HEB), a paradigm that builds on the concepts of emergent behavior and hierarchical organization. Rather than explicitly programming all possible decisions in the vast space of ULSS scenarios, HEB relies on the emergent behaviors induced by local rules at each level of the hierarchy. The authors discuss the modifications to classical IoT architectures required by HEB, as well as the new challenges. They also illustrate the HEB concepts in reference to autonomous vehicles. This use case paves the way to the discussion of new lines of research., Damian Roca work was supported by a Doctoral Scholarship provided by Fundación La Caixa. This work has been supported by the Spanish Government (Severo Ochoa grants SEV2015-0493) and by the Spanish Ministry of Science and Innovation (contracts TIN2015-65316-P)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Performance characterization of video analytics workloads in heterogeneous edge infrastructures

  • Rivas Barragan, Daniel
  • Guim Bernat, Francesc
  • Polo Bardés, Jorda
  • Carrera Pérez, David|||0000-0003-4898-3424
Powered by deep learning, video analytic applications process millions of camera feeds in real-time to extract meaningful information from their surroundings. And this number grows by the minute. To avoid saturating the backhaul network and provide lower latencies, a distributed and heterogeneous edge cloud is postulated as a key enabler for widespread video analytics. This article provides a complete characterization of end-to-end video analytics across a set of hardware platforms and different neural network architectures. Each platform is selected to fill a different gap in a distributed, shared, and heterogeneous infrastructure. Moreover, we analyze how performance scales on each of these platforms with respect to the amount of resources dedicated to video analytics. Finally, we extract the key conclusions of the characterization to build an experimental model to estimate performance and cost of end-to-end video analytics in different edge scenarios. Our experiments show that managing video analytics workloads efficiently requires awareness of both, the platforms in which these are executed, and the full end-to-end pipeline. To the best of our knowledge, this is the first work that provides a complete characterization of end-to-end video analytics in heterogeneous edge platforms., This work was supported by the Ministry of Economy of Spain under contract TIN2015-65316-P and the Ministry of Science under contract PID2019-107255GB-C21/AEI/10.13039/501100011033; the Generalitat de Catalunya under contract 2014SGR1051; the ICREA Academia program; and Intel Corporation, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Randomization for safer, more reliable and secure, high-performance automotive processors

  • Trilla Rodríguez, David
  • Cazorla Almeida, Francisco Javier
  • Hernández Luz, Carles
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
The automotive domain is witnessing a relentless transition to autonomous cars demanding high-performance processors to timely execute complex, critical, decision-making software. The other side of the coin is that high-performance processors include hardware features like shared multilevel caches and multiple cores that expose the system to significant security threats, challenge time predictability, and jeopardize reliable operation due to the use of advanced process technology. In this paper, we discuss how introducing randomization in the non-functional behavior of certain hardware components helps to achieve a three-fold objective while preserving high-average performance capabilities of high-performance processors: improving the security of complex processors, favoring time predictability via probabilistic analysis, and enhancing reliability against aging and voltage noise., This work was supported in part by the Spanish Ministry of Science and Innovation under Grant TIN2015-65316-P. The work of J. Abella was supported in part by the Ministry of Economy and Competitiveness under the Ramon y Cajal fellowship number RYC-2013-14717., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Accelerating edit-distance sequence alignment on GPU using the wavefront algorithm

  • Aguado Puig, Quim|||0000-0003-4871-3192
  • Marco Sola, Santiago|||0000-0001-7951-3914
  • Moure López, Juan Carlos
  • Castells Rufas, David
  • Álvarez Martí, Lluc|||0000-0003-0506-8867
  • Espinosa Morales, Antonio
  • Moretó Planas, Miquel|||0000-0002-9848-8758
Sequence alignment remains a fundamental problem with practical applications ranging from pattern recognition to computational biology. Traditional algorithms based on dynamic programming are hard to parallelize, require significant amounts of memory, and fail to scale for large inputs. This work presents eWFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute the exact edit-distance sequence alignment based on the wavefront alignment algorithm (WFA). This approach exploits the similarities between the input sequences to accelerate the alignment process while requiring less memory than other algorithms. Our implementation takes full advantage of the massive parallel capabilities of modern GPUs to accelerate the alignment process. In addition, we propose a succinct representation of the alignment data that successfully reduces the overall amount of memory required, allowing the exploitation of the fast shared memory of a GPU. Our results show that our GPU implementation outperforms by 3- 9× the baseline edit-distance WFA implementation running on a 20 core machine. As a result, eWFA-GPU is up to 265 times faster than state-of-the-art CPU implementation, and up to 56 times faster than state-of-the-art GPU implementations., This work was supported in part by the European Unions’s Horizon 2020 Framework Program through the DeepHealth Project under Grant 825111; in part by the European Union Regional Development Fund within the Framework of the European Regional Development Fund (ERDF) Operational Program of Catalonia 2014–2020 with a Grant of 50% of Total Cost Eligible through the Designing RISC-V-based Accelerators for next-generation Computers Project under Grant 001-P-001723; in part by the Ministerio de Ciencia e Innovacion (MCIN) Agencia Estatal de Investigación (AEI)/10.13039/501100011033 under Contract PID2020-113614RB-C21 and Contract TIN2015-65316-P; and in part by the Generalitat de Catalunya (GenCat)-Departament de Recerca i Universitats (DIUiE) (GRR) under Contract 2017-SGR-313, Contract 2017-SGR-1328, and Contract 2017-SGR-1414. The work of Miquel Moreto was supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under Ramon y Cajal Fellowship under Grant RYC-2016-21104., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Design trade-offs for emerging HPC processors based on mobile market technology

  • Armejach Sanosa, Adrià|||0000-0003-2869-668X
  • Casas, Marc|||0000-0003-4564-2093
  • Moretó Planas, Miquel|||0000-0002-9848-8758
This is a post-peer-review, pre-copyedit version of an article published in The Journal of Supercomputing. The final authenticated version is available online at: http://dx.doi.org/10.1007/s11227-019-02819-4, High-performance computing (HPC) is at the crossroads of a potential transition toward mobile market processor technology. Unlike in prior transitions, numerous hardware vendors and integrators will have access to state-of-the-art processor designs due to Arm’s licensing business model. This fact gives them greater flexibility to implement custom HPC-specific designs. In this paper, we undertake a study to quantify the different energy-performance trade-offs when architecting a processor based on mobile market technology. Through detailed simulations over a representative set of benchmarks, our results show that: (i) a modest amount of last-level cache per core is sufficient, leading to significant power and area savings; (ii) in-order cores offer favorable trade-offs when compared to out-of-order cores for a wide range of benchmarks; and (iii) heterogeneous configurations help to improve processor performance and energy efficiency., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Integrating blocking and non-blocking MPI primitives with task-based programming models

  • Sala Penadés, Kevin|||0000-0001-8233-1185
  • Teruel García, Xavier
  • Pérez Cáncer, Josep Maria|||0000-0002-0558-7600
  • Peña, Antonio J.
  • Beltran, Vicenç
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
In this paper we present the Task-Aware MPI library (TAMPI) that integrates both blocking and non-blocking MPI primitives with task-based programming models. The TAMPI library leverages two new runtime APIs to improve both programmability and performance of hybrid applications. The first API allows to pause and resume the execution of a task depending on external events. This API is used to improve the interoperability between blocking MPI communication primitives and tasks. When an MPI operation executed inside a task blocks, the task running is paused so that the runtime system can schedule a new task on the core that became idle. Once the blocked MPI operation is completed, the paused task is put again on the runtime system’s ready queue, so eventually it will be scheduled again and its execution will be resumed. The second API defers the release of dependencies associated with a task completion until some external events are fulfilled. This API is composed only of two functions, one to bind external events to a running task and another function to notify about the completion of external events previously bound. TAMPI leverages this API to bind non-blocking MPI operations with tasks, deferring the release of their task dependencies until both task execution and all its bound MPI operations are completed. Our experiments reveal that the enhanced features of TAMPI not only simplify the development of hybrid MPI+OpenMP applications that use blocking or non-blocking MPI primitives but they also naturally overlap computation and communication phases, which improves application performance and scalability by removing artificial dependencies across communication tasks., This work has been developed with the support of the European Union H2020 Programme through both the INTERTWinE project (agreement no. 671602) and the Marie Skłodowska-Curie grant (agreement no. 749516); the Spanish Ministry of Economy and Competitiveness through the Severo Ochoa Program (SEV-2015-0493); the Spanish Ministry of Science and Innovation (TIN2015-65316-P) and the Generalitat de Catalunya (2017-SGR1414)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

The AXIOM project: IoT on heterogeneous embedded platforms

  • Filgueras Izquierdo, Antonio|||0000-0001-6071-3254
  • Vidal, Miquel|||0000-0002-1973-8289
  • Mateu, Marc
  • Jiménez González, Daniel|||0000-0001-6064-7883
  • Álvarez Martínez, Carlos|||0000-0003-0536-5183
  • Martorell Bofill, Xavier|||0000-0002-0417-3430
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Theodoropoulos, Dimitris
  • Pnevmatikatos, Dionisis
  • Gai, Paolo
  • Garzarella, Stefano
  • Oro de Herta, David
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
  • Bettin, Nicola
  • Pomella, Alberto
  • Giorgi, Roberto
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The AXIOM project aims at providing an environment for Cyber-Physical Systems. Smart Video Surveillance targets public environments, involving real-time face detection in crowds. Smart Home Living targets home environments and access control. These applications are used as experimental usecases for the AXIOM platform, currently based on the Xilinx Zynq-7000 SoCs. We have integrated the Xilinx Vivado HLS tool for the FPGA support within the OmpSs programming model, to enable OpenMP-like programming in the FPGA. This paper presents the programming environment, and the evaluation of the most computationally expensive parts of the target applications., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Reconciling Time Predictability and Performance in Future Computing Systems

  • Cazorla, Francisco J.
  • Abella Ferrer, Jaume|||0000-0001-7951-4028
  • Mezzetti, Enrico|||0000-0002-1886-2931
  • Hernandez, Carles
  • Vardanega, Tullio
  • Bernat, Guillem
MBTA studies the system’s timing in analysis scenarios, to determine upper bounds to the worst-case execution-time behavior that may occur at operation. MBTA’s challenge is to construct analysis-time scenarios that help compute WCET estimates that upper-bound operation-time behavior. This evidently requires ensuring that all factors with bearing on the execution conditions that the program may incur during operation are duly considered in the analysis. In fact, the factors that originate from low-level hardware resources are far more difficult to get at for the user than those that proceed from the software. This paper addresses the former challenge, with solutions that entail simple changes to hardware design and MBTA methods, which achieve quality results without sacrificing performance. On those grounds, we maintain that the adoption of the following design principles yields an effective application of MBTA to high-performance systems., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under grant TIN2015-65316-P and the HiPEAC Network of Excellence. Carles Hernandez is jointly funded by the Spanish Ministry of Economy and Competitiveness (MINECO) and FEDER funds through grant TIN2014-60404-JIN. Jaume Abella has been partially supported by the MINECO under Ramon y Cajal postdoctoral fellowship number RYC-2013-14717., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Ad hoc file systems for high-performance computing

  • Brinkmann, Andre
  • Mohror, Kathryn
  • Yu, Weikuan
  • Carns, Philip
  • Cortés, Toni|||0000-0002-2537-8937
  • Klasky, Scott A.
  • Miranda Bueno, Alberto
  • Pfreundt, Franz-Josef
  • Ross, Robert B.
  • Vef, Marc-André
Storage backends of parallel compute clusters are still based mostly on magnetic disks, while newer and faster storage technologies such as flash-based SSDs or non-volatile random access memory (NVRAM) are deployed within compute nodes. Including these new storage technologies into scientific workflows is unfortunately today a mostly manual task, and most scientists therefore do not take advantage of the faster storage media. One approach to systematically include nodelocal SSDs or NVRAMs into scientific workflows is to deploy ad hoc file systems over a set of compute nodes, which serve as temporary storage systems for single applications or longer-running campaigns. This paper presents results from the Dagstuhl Seminar 17202 “Challenges and Opportunities of User-Level File Systems for HPC” and discusses application scenarios as well as design strategies for ad hoc file systems using node-local storage media. The discussion includes open research questions, such as how to couple ad hoc file systems with the batch scheduling environment and how to schedule stage-in and stage-out processes of data between the storage backend and the ad hoc file systems. Also presented are strategies to build ad hoc file systems by using reusable components for networking and how to improve storage device compatibility. Various interfaces and semantics are presented, for example those used by the three ad hoc file systems BeeOND, GekkoFS, and BurstFS. Their presentation covers a range from file systems running in production to cutting-edge research focusing on reaching the performance limits of the underlying devices., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Real-time data center's telemetry reduction and reconstruction using Markov chain models

  • Baig, Shuja-ur-Rehman
  • Iqbal, Waheed
  • Berral García, Josep Lluís|||0000-0003-3037-3580
  • Erradi, Abdelkarim
  • Carrera Pérez, David|||0000-0003-4898-3424
Large-scale data centers are composed of thousands of servers organized in interconnected racks to offer services to users. These data centers continuously generate large amounts of telemetry data streams (e.g., hardware utilization metrics) used for multiple purposes, including resource management, workload characterization, resource utilization prediction, capacity planning, and real-time analytics. These telemetry streams require costly bandwidth utilization and storage space, particularly at medium-long term for large data centers. This paper addresses this problem by proposing and evaluating a system to efficiently reduce bandwidth and storage for telemetry data through real-time modeling using Markov chain based methods. Our proposed solution was evaluated using real telemetry datasets and compared with polynomial regression methods for reducing and reconstructing data. Experimental results show that data can be lossy compressed up to 75% for bandwidth utilization and 95.33% for storage space, with reconstruction accuracy close to 92%., This work was supported in part by the European Research Council (ERC) under the EU Horizon 2020 programme under Grant GA 639595, in part by the Spanish Ministry of Economy, Industry and Competitiveness under Grant TIN2015-65316-P and Grant IJCI2016-27485, in part by the Generalitat de Catalunya under Grant 2014-SGR-1051, in part by the University of the Punjab, Pakistan, and in part by the Qatar National Research Fund (a member of Qatar Foundation) under NPRP Grant # NPRP9-224-1-049., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Task scheduling techniques for asymmetric multi-core systems

  • Chronaki, Kallia
  • Rico, Alejandro
  • Casas, Marc|||0000-0003-4564-2093
  • Moretó Planas, Miquel|||0000-0002-9848-8758
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Ayguadé Parra, Eduard|||0000-0002-5146-103X
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
  • Valero Cortés, Mateo|||0000-0003-2917-2482
As performance and energy efficiency have become the main challenges for next-generation high-performance computing, asymmetric multi-core architectures can provide solutions to tackle these issues. Parallel programming models need to be able to suit the needs of such systems and keep on increasing the application’s portability and efficiency. This paper proposes two task scheduling approaches that target asymmetric systems. These dynamic scheduling policies reduce total execution time either by detecting the longest or the critical path of the dynamic task dependency graph of the application, or by finding the earliest executor of a task. They use dynamic scheduling and information discoverable during execution, fact that makes them implementable and functional without the need of off-line profiling. In our evaluation we compare these scheduling approaches with two existing state-of the art heterogeneous schedulers and we track their improvement over a FIFO baseline scheduler. We show that the heterogeneous schedulers improve the baseline by up to 1.45 in a real 8-core asymmetric system and up to 2.1 in a simulated 32-core asymmetric chip., This work has been supported by the Spanish Government (SEV2015-0493), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P), by Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), by the RoMoL ERC Advanced Grant (GA 321253) and the European HiPEAC Network of Excellence. The Mont-Blanc project receives funding from the EU’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no 610402 and from the EU’s H2020 Framework Programme (H2020/2014-2020) under grant agreement no 671697. M. Moretó has been partially supported by the Ministry of Economy and Competitiveness under Juan de la Cierva postdoctoral fellowship number JCI-2012-15047. M. Casas is supported by the Secretary for Universities and Research of the Ministry of Economy and Knowledge of the Government of Catalonia and the Cofund programme of the Marie Curie Actions of the 7th R&D Framework Programme of the European Union (Contract 2013 BP B 00243)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A generic performance analysis technique applied to different CFD methods for HPC

  • Garcia Gasulla, Marta|||0000-0003-3682-9905
  • Banchelli Gracia, Fabio
  • Peiró Conde, Kilian
  • Ramírez Gargallo, Guillem
  • Houzeaux, Guillaume|||0000-0002-2592-1426
  • Hassan Saïdi, Ismaïl Ben
  • Tenaud, Christian
  • Spisso, Ivan
  • Mantovani, Filippo|||0000-0003-3559-4825
For complex engineering and scientific applications, Computational Fluid Dynamics (CFD) simulations require a huge amount of computational power. As such, it is of paramount importance to carefully assess the performance of CFD codes and to study them in depth for enabling optimisation and portability. In this paper, we study three complex CFD codes, OpenFOAM, Alya and CHORUS representing two numerical methods, namely the finite volume and finite-element methods, on both structured and unstructured meshes. To all codes, we apply a generic performance analysis method based on a set of metrics helping the code developer in spotting the critical points that can potentially limit the scalability of a parallel application. We show the root cause of the performance bottlenecks studying the three applications on the MareNostrum4 supercomputer. We conclude providing hints for improving the performance and the scalability of each application., This work is partially supported by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European POP CoE (GA n. 824080)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction

  • Campos Camunez, Victor
  • Jou, Brendan
  • Giró Nieto, Xavier|||0000-0002-9935-5332
Visual multimedia have become an inseparable part of our digital social lives, and they often capture moments tied with deep affections. Automated visual sentiment analysis tools can provide a means of extracting the rich feelings and latent dispositions embedded in these media. In this work, we explore how Convolutional Neural Networks (CNNs), a now de facto computational machine learning tool particularly in the area of Computer Vision, can be specifically applied to the task of visual sentiment prediction. We accomplish this through fine-tuning experiments using a state-of-the-art CNN and via rigorous architecture analysis, we present several modifications that lead to accuracy improvements over prior art on a dataset of images from a popular social media platform. We additionally present visualizations of local patterns that the network learned to associate with image sentiment for insight into how visual positivity (or negativity) is perceived by the model., This work has been developed in the framework of the BigGraph TEC2013-43935-R project, funded by the Spanish Ministerio de Economía y Competitividad and the European Regional Development Fund (ERDF). It has been supported by the Severo Ochoa Program’s SEV2015-0493 grant awarded by the Spanish Government, the TIN2015-65316 project by the Spanish Ministerio de Economía y Competitividad and contracts 2014-SGR-1051 by Generalitat de Catalunya. The Image Processing Group at the UPC is a SGR14 Consolidated Research Group recognized and sponsored by the Catalan Government (Generalitat de Catalunya) through its AGAUR office. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the GeForce GTX Titan Z and X used in this work and the support of BSC/UPC NVIDIA GPU Center of Excellence., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

GMAI: Understanding and exploiting the internals of GPU resource allocation in critical systems

  • Calderón Torres, Alejandro Josué
  • Kosmidis, Leonidas
  • Nicolás Ramírez, Carlos Fernando
  • Cazorla Almeida, Francisco Javier
  • Onaindia, Peio
Critical real-time systems require strict resource provisioning in terms of memory and timing. The constant need for higher performance in these systems has led industry to recently include GPUs. However, GPU software ecosystems are by their nature closed source, forcing system engineers to consider them as black boxes, complicating resource provisioning. In this work, we reverse engineer the internal operations of the GPU system software to increase the understanding of their observed behaviour and how resources are internally managed. We present our methodology that is incorporated in GMAI (GPU Memory Allocation Inspector), a tool that allows system engineers to accurately determine the exact amount of resources required by their critical systems, avoiding underprovisioning. We first apply our methodology on a wide range of GPU hardware from different vendors showing its generality in obtaining the properties of the GPU memory allocators. Next, we demonstrate the benefits of such knowledge in resource provisioning of two case studies from the automotive domain, where the actual memory consumption is up to 5.6× more than the memory requested by the application., This work has been partially supported by the Spanish Ministry of Science and Innovation under grant TIN2015-65316-P, the HiPEAC Network of Excellence and the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation programme (grant agreement No. 772773). Leonidas Kosmidis is also funded by the Spanish Ministry of Economy and Competitiveness (MINECO) under a Juan de la Cierva Formación postdoctoral fellowship (FJCI-2017-34095)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Runtime Mechanisms to Survive New HPC Architectures: A Use-Case in Human Respiratory Simulations

  • Garcia-Gasulla, Marta
  • Mantovani, Filippo|||0000-0003-3559-4825
  • Josep-Fabrego, Marc
  • Eguzkitza, Beatriz|||0000-0002-3302-6667
  • Houzeaux, Guillaume|||0000-0002-2592-1426
Computational Fluid and Particle Dynamics (CFPD) simulations are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes demand high-performance computing (HPC) resources. For these reasons we introduce and evaluate in this paper system software techniques for improving performance and tolerate load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on Intel-, IBM-, and Arm-based HPC technologies ranked in the Top500 supercomputers, showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2x, across different architectures, while applying runtime techniques and keeping constant the computational resources., This work is partially supported by the Spanish Government (SEV-2015-0493), by the Spanish Ministry of Science and Technology project (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European Mont-Blanc projects (288777, 610402 and 671697)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Analyzing the impact of communication imbalance in high-speed networks

  • Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
  • Gil, Marisa|||0000-0001-9755-3311
  • Martorell Bofill, Xavier|||0000-0002-0417-3430
In this work we analyze the communication load imbalance generated by irregular-data applications running in a multi-node cluster. Experimental approaches to diminish communication load imbalance are evaluated using a hybrid programming model MPI+OpenMP including certain optimizations like computation-communication overlap, issuing communications in parallel and a new proposal based on message fragmentation in order to take advantage of the eager-protocol. Performance results show that overlapped versions can obtain a great benefit of this optimization because it avoids switching to rendez-vous protocols. However, non-overlapped versions showed better performance than overlapped ones. To evaluate also the impact due to network latency, the work has been tested on two high-speed interconnection networks: Infiniband and 10Gigabit Ethernet. In this case, the optimizations in the non-overlapped miniFE benchmark reached and improvement up to 7% on Infiniband and 11% on 10Gigabit., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

MPI+X: task-based parallelization and dynamic load balance of finite element assembly

  • Garcia-Gasulla, Marta
  • Houzeaux, Guillaume|||0000-0002-2592-1426
  • Ferrer, Roger
  • Artigues, Antoni
  • López, Victor
  • Labarta Mancho, Jesús José|||0000-0002-7489-4727
  • Vázquez, Mariano|||0000-0002-2526-6708
The main computing tasks of a finite element code(FE) for solving partial differential equations (PDE's) are the algebraic system assembly and the iterative solver. This work focuses on the first task, in the context of a hybrid MPI+X paradigm. Although we will describe algorithms in the FE context, a similar strategy can be straightforwardly applied to other discretization methods, like the finite volume method. The matrix assembly consists of a loop over the elements of the MPI partition to compute element matrices and right-hand sides and their assemblies in the local system to each MPI partition. In a MPI+X hybrid parallelism context, X has consisted traditionally of loop parallelism using OpenMP. Several strate- gies have been proposed in the literature to implement this loop parallelism, like coloring or substructuring techniques to circumvent the race condition that appears when assembling the element system into the local system. The main drawback of the first technique is the decrease of the IPC due to bad spatial locality. The second technique avoids this issue but requires extensive changes in the implementation, which can be cumbersome when several element loops should be treated. We propose an alternative, based on the task parallelism of the element loop using some extensions to the OpenMP programming model. The task- ification of the assembly solves both aforementioned problems. In addition, dynamic load balance will be applied using the DLB library, especially efficient in the presence of hybrid meshes, where the relative costs of the different elements is impossible to estimate a priori. This paper presents the proposed methodology, its implementation and its validation through the solution of large computational mechanics problems up to 16k cores., The use of large part of a supercomputer, even more in normal conditions of use, is never an innocuous exercise. The research leading to these results has received funding from: the European Union's Horizon 2020 Programme (2014–2020) and from Brazilian Ministry of Science, Technology and Innovation through Rede Nacional de Pesquisa (RNP), HPC4E Project, grant agreement 689772; the Energy oriented Centre of Excellence (EoCoE), grant agreement number 676629, funded within the Horizon2020 framework of the European Union; The Spanish Government (grant SEV2015-0493 of the Severo Ochoa Program); the Spanish Ministry of Science and Innovation (contract TIN2015-65316-P); the Generalitat de Catalunya (contract 2014-SGR-1051); the Intel-BSC Exascale Lab collaboration project. Comissió Interdepartamental de Recerca i Innovació Tecnológica(Interdepartmental Commission for Research and Technological Innovation), Sí
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Distributed training of deep neural networks with spark: The MareNostrum experience

  • Cruz, Leonel
  • Tous Liesa, Rubén|||0000-0002-1409-5843
  • Otero Calviño, Beatriz|||0000-0002-9194-559X
Deployment of a distributed deep learning technology stack on a large parallel system is a very complex process, involving the integration and configuration of several layers of both, general-purpose and custom software. The details of such kind of deployments are rarely described in the literature. This paper presents the experiences observed during the deployment of a technology stack to enable deep learning workloads on MareNostrum, a petascale supercomputer. The components of a layered architecture, based on the usage of Apache Spark, are described and the performance and scalability of the resulting system is evaluated. This is followed by a discussion about the impact of different configurations including parallelism, storage and networking alternatives, and other aspects related to the execution of deep learning workloads on a traditional HPC setup. The derived conclusions should be useful to guide similarly complex deployments in the future., Peer Reviewed