Póster presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 24 al 28 de junio de 2018., The DEEP Hybrid DataCloud project aims to provide a bridge towards a more flexible exploitation of intensive computing resources by the research communities, enabling access to the latest technologies and specialised HPC hardware, like GPUs or low-latency interconnects, to be able to explore large datasets. We will integrate the intensive computing services under a Hybrid Cloud approach, assuring interoperability with unmodified HPC resources as well as the upcoming EOSC platform. This will simplify access to resources that may not be easily reachable by researchers at scale. A number of pilot applications exploiting very large datasets in e.g. Biology, Earth Observation, Network Security, and Physics are proposed. They will be integrated in a testbed with significant HPC resources, including latest generation GPUs, to evaluate the performance and scalability of the solutions. A DevOps approach will be implemented to provide the chain to ensure the quality of the software and services released, that will also be offered to the developers of research applications. We aim to release a “DEEP as a Service” solution to offer an easy integration path for new communities and users to develop applications requiring deep learning techniques, parallel post-processing of very large data, and analysis of massive online data streams., The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777435, and started on November 1, 2017, with the runtime of 30 months., Peer Reviewed

Trabajo presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 16 al 20 de junio de 2019., The DEEP-HybridDataCloud project researches on intensive computing techniques such as deep learning, that require specialized GPU hardware to explore very large datasets, through a hybrid-cloud approach that enables the access to such resources. DEEP is built on User-centric policy, i.e. we understand the needs of our user communities and help them to combine their services in a way that encapsulates technical details the end user does not have to deal with. DEEP takes care to support users of different levels of experience by providing different integration paths. We show our current solutions to the problem, which among others include the Open Catalog for deep learning applications, DEEP-as-a-Service API for providing web access to machine learning models, CI/CD pipeline for user applications, Testbed resources. We also present our use-cases tackling various problems by means of deep learning and serving to demonstrate usefulness and scalability of our approach., DEEP HybridDataCloud receives funding from the European Union's Horizon 2020 research and innovation programme under agreement RIA 777435.

Trabajo presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 16 al 20 de junio de 2019., This work was developed with the support of the H2020 projects: INDIGO-DataCloud (RIA 653549), DEEP-Hybrid-DataCloud, (RIA 777435), EOSC-hub (RIA 777536). This work was developed with the support of the Portuguese Foundation for Science and Technology under the project: Infraestrutura Nacional de Computação Distribuída 01/SAICT/2016 - nº 02215

Trabajo presentado a IBERGRID: Delivering Innovative Computing and Data services to Researchers, celebrado en Santiago de Compostela (España) del 23 al 26 de septiembre de 2019., DEEP Hybrid DataCloud project was proposed with the necessity to support different amount of intensive computing techniques over specialized hardware, like HPC or GPUs. The project focus on the integration of this specialized, and expensive, hardware under a Cloud Platform as OpenStack that can be used on-demand by researchers of different areas. Within this project, a set of building blocks whose solution is called >DEEP as a Service> was implemented to make the application deployment more easier for the user. For this development, it is necessary to provide the researchers with access to these technologies as friendly but powerful services able to exploit very large datasets. On the one hand we have the gpu resources and on the other hand the users and their applications to run over those resources. DEEP needs to provide a service that controls how users can use those resources in an efficiently way. Although there are multiple technologies that address this problem as a queuing system, Apache Mesos has been developed to do it in an effective and controlled manner. Apache Mesos is a technology that abstracts the resources from different machines, like cpu, gpu, ram and storage to provide a scheduling and distributed system across the whole cloud environment. Mesos is easily to deploy over cpu-based systems but gpu needs a more tricky configuration as it is shown in this solution. As an added value, this solution provides an apache2 configuration for authenticate users from different communities by the current AAI service in DEEP. The proposed presentation will show the design and deployment of Mesos to work over gpus resources inside the Deep Hybrid DataCloud Project scope., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.

Resumen del trabajo presentado a la Conferencia Bio Diversity Next, celebrada en Leiden (Países Bajos) del 22 al 25 de octubre de 2019., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.