Update on November 2022., The purpose of this document is to define a set of quality standards, procedures and best practices to conform a Software Quality Assurance plan to serve as a reference within the European research ecosystem related projects for the adequate development and timely delivery of software products., The INDIGO-DataCloud, DEEP-Hybrid-DataCloud and eXtreme-DataCloud projects have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 653549, 777435 and 777367 respectively., No

Available soon., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This report summarises the work of WP1 on the Data Management considerations and initial plan
for the DEEP-HybridDataCloud project. This document describes the types of data that will be
generated or collected during the project, the standards that will be used and the ways in which the
data may be exploited and shared including the data security and ethical aspects., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This report summarises the work of WP2 on the initial plan on the selection of Use Cases,
providing a key input to the design of the DEEP-HybridDataCloud testbed and laying out the
DEEP-HybridDataCloud solutions will be used. This document includes a description of the
Research Communities involved and of the use-cases proposed, including Data Management and
Computational Intensive issues. Based on the inputs provided by the WP2 partners, an initial list of
requirements for each application has been collected, that has been used in turn to assemble a list of
common requirements that has been provided as input to the discussion with the technical work
packages of the DEEP-HybridDataCloud project., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document describes the design of the PaaS Layer and the detailed work plan for the WP5 of
the DEEP-HybridDataCloud project, which aims at adopting and extending the PaaS orchestration solution for cloud infrastructures realised within the INDIGO-DataCloud project. In particular, the main focus is on the architecture and features of the hybrid cloud solution., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document describes the state of the art of technologies for supporting bare-metal, accelerators
and HPC in cloud and proposes an initial implementation plan. Available technologies will be
analyzed from different points of views: stand-alone use, integration with cloud middleware,
support for accelerators and HPC platforms. Based on results of these analyses, an initial
implementation plan will be proposed containing information on what features should be
developed and what components should be improved in the next period of the project., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document provides a description of the design, architecture and work plan the DEEPHybridDataCloud
Work Package 6 (WP6) in order to provide the DEEP as a Service solution. As
such it provides an overview of the state of the art of the relevant components and technologies, as
well as a technology readiness level assessment with regards to the required functionality, the
required interactions with other work packages in the project, as well as the detailed work plan and
risk assessment for each of the activities., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

The deliverable contains the plan, design, architecture and deployment of the Pilot Preview testbed based on technical requirements and descriptions from the WP2 use cases. The services and components developed during the DEEP-HybridDataCloud by the teams part of the WP4, 5 and 6
are deployed, tested and validated by end-users in this testbed. It also describes how the Pilot Preview testbed services and components will be integrated with EOSC production infrastructure and other external resource providers., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This deliverable describes the first implementation of the software platform for accessing
accelerators and HPC. The list of components included in the software platform is based on the
analysis provided by Deliverable D4.1. This document provides detailed descriptions of software
components used in the platforms, the work done on each component and its current status.
Evaluation of achieved results and implementation plan for the next periods are also included., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document complements deliverable D5.1 Definition of the Architecture of the Hybrid Cloud
(D5.1) with the specific prototype developments carried out to support the deployment of hybrid
infrastructures across multiple IaaS Cloud sites. The document describes the technical challenges,
the evolution of the components to support this prototype and a roadmap of implementation
towards the final release of the High Level Hybrid Cloud solutions., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document provides an updated description of the prototype implementation of the DEEP as a Service solution that is being developed within the DEEP-Hybrid-DataCloud project Work Package 6 (WP6). As such it provides an overview of the state of the art of the relevant components and technologies, as well as a technology readiness level assessment with regards to the required functionality, the required interactions with other work packages in the project, as well as the detailed work plan and risk assessment for each of the activities., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

Póster presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 24 al 28 de junio de 2018., The DEEP Hybrid DataCloud project aims to provide a bridge towards a more flexible exploitation of intensive computing resources by the research communities, enabling access to the latest technologies and specialised HPC hardware, like GPUs or low-latency interconnects, to be able to explore large datasets. We will integrate the intensive computing services under a Hybrid Cloud approach, assuring interoperability with unmodified HPC resources as well as the upcoming EOSC platform. This will simplify access to resources that may not be easily reachable by researchers at scale. A number of pilot applications exploiting very large datasets in e.g. Biology, Earth Observation, Network Security, and Physics are proposed. They will be integrated in a testbed with significant HPC resources, including latest generation GPUs, to evaluate the performance and scalability of the solutions. A DevOps approach will be implemented to provide the chain to ensure the quality of the software and services released, that will also be offered to the developers of research applications. We aim to release a “DEEP as a Service” solution to offer an easy integration path for new communities and users to develop applications requiring deep learning techniques, parallel post-processing of very large data, and analysis of massive online data streams., The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777435, and started on November 1, 2017, with the runtime of 30 months., Peer Reviewed

DEEP as a Service (DEEPaaS) is a REST API that is focused on providing access to machine learning models. By using DEEPaaS users can easily run a REST API in fron of their model, thus accesing its functionality via HTTP calls. Quickstart: The best way to quickly try the DEEPaaS API is trough: make run. This command will install a virtualenv (in the `virtualenv` directory) with DEEPaaS and all its dependencies and will run the DEEPaaS REST API, listening on `127.0.0.1:5000`. If you browse to `http://127.0.0.1:5000` you will get the swagger documentation page. Develop mode: If you want to run the code in develop mode (i.e. `pip install -e`), you can issue the following command before: make develop., DEEP as a Service (DEEPaaS) is a REST API that is focused on providing access to machine learning models. By using DEEPaaS users can easily run a REST API in fron of their model, thus accesing its functionality via HTTP calls., Proyecto: Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (DEEP-HybridDataCloud)., Peer reviewed

Apache License, Version 2.0, This is a plug-and-play tool to train and evaluate an image classifier on a custom dataset using deep neural networks., DEEP Hybrid-Datacloud, GA 777435., Peer reviewed

The combined impact of new computing resources and techniques with an increasing avalanche of large datasets, is transforming many research areas and may lead to technological breakthroughs that can be used by billions of people. In the recent years, machine Learning and especially its subfield Deep Learning have seen impressive advances. Techniques developed within these two fields are now able to analyze and learn from huge amounts of real world examples in a disparate formats. While the number of Machine Learning algorithms is extensive and growing, their implementations through frameworks and libraries is also extensive and growing too. The software devel- opment in this field is fast paced with a large number of open-source software coming from the academy, industry, start-ups or wider open-source commu- nities. This survey presents a recent time-slide comprehensive overview with comparisons as well as trends in development and usage of cutting-edge Arti- ficial Intelligence software. It also provides an overview of massive parallelism support that is capable of scaling computation effectively and efficiently in the era of Big Data., Álvaro López García, Ignacio Heredia, Giang Nguyen, Viet Tran, Stefan Dlugolinsky,
Martin Bobák, and Ladislav Hluchý are supported by the project DEEP-HybridDataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 777435.
Giang Nguyen, Viet Tran, Stefan Dlugolinsky, Martin Bobák, and Ladislav Hluchý are also supported by the Project VEGA 2/0167/16 “Methods and algorithms for the semantic processing of Big Data in distributed computing environment”. The authors would like to thanks to all colleagues, especially for Ján Astaloš for
knowledge sharing and teamwork., Peer reviewed

Nowadays, in the era of Internet of Things when everything is connected via the Internet, the number of mobile devices has risen exponentially up to billions around the world. In line with this increase, the volume of data generated is enormous and has attracted malefactors who do ill deeds to others. For hackers, one of the popular threads to mobile devices is to spread malware. These actions are very difficult to prevent because the application installation and configuration rights are set by owners, who usually have very low knowledge or do not care about the security. In this study, our aim is to improve security in the environment of mobile devices by proposing a novel system to detect malware intrusions automatically. Our solution is based on modelling user behaviours and applying the heuristic analysis approach to mobile logs generated during the device operation process. Although behaviours of individual users have a significant impact on the social cyber-security, to achieve the user awareness has still remained one of the major challenges today. For this task, there is proposed a light-weight semantic formalization in the form of physical and logical taxonomy for classifying the collected raw log data. Then a set of techniques is used, like sliding windows, lemmatization, feature selection, term weighting, and so on, to process data. Meanwhile, malware detection tasks are performed based on incremental machine learning mechanisms, because of the potential complexity of this tasks. The solution is developed in the manner to allow the scalability with several blocks that cover pre-processing raw collected logs from mobile devices, automatically creating datasets for machine learning methods, using the best selected model for detecting suspicious activity surrounding malware intrusions, and supporting decision making using a predictive risk factor. We experimented cautiously with the proposal and achieved test results confirm the effectiveness and feasibility of the proposed system in applying to the large-scale mobile environment., This work is supported by the projects Slovak VEGA 2/0167/16 “Methods and algorithms for the semantic processing of Big Data in distributed computing environment”, Vietnamese MOET’s No. B2017-BKA-32 “Research on developing software framework to integrate IoT gateways for fog computing deployed on multi-cloud environment”, and DEEP-HybridDataCloud EU H2020-777435
“Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud”., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

This dataset contains various network monitoring channels of computing infrastructure collected during the period of almost 15 months in 2018/2019. It is aggregated over 3 different sliding windows., Agencia Financiadora: EC; acuerdo de subvención: H2020 777435., Peer reviewed

Apache License 2.0, Agencia Financiadora: EC. Acuerdo de subvención: H2020 777435, No

Apache License 2.0, EC: H2020 777435., Peer reviewed

Apache License 2.0, EC: H2020 777435, Peer reviewed

Apache License 2.0, EC: H2020 777435, Peer reviewed

Apache 2.0, EC: H2020 777435, Peer reviewed

Apache 2.0, EC: H2020 777435, Peer reviewed

Arguably, Python is one of the most widely used programming languages in data analytics, data mining, artificial intelligence, machine learning and deep learning (Nguyen et al., 2019). DEEPaaS API is a Python package that provides a REST API (Fielding, 2000) that can be used to easily expose the underlying model functionality over HTTP. Developers can plug their applications into DEEPaaS API with minimal requirements and modifications, allowing for easier interaction with the underlying functionality, both for training, validating and testing., This software has been developed within the DEEP-Hybrid-DataCloud (Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud) project that has received funding from the European Union’s Horizon 2020 research and innovation programme
under grant agreement No 777435., Peer reviewed

Continuous high frequency water quality monitoring is becoming a critical task to support water management. Despite the advancements in sensor technologies, certain variables cannot be easily and/or economically monitored in-situ and in real time. In these cases, surrogate measures can be used to make estimations by means of data-driven models. In this work, variables that are commonly measured in-situ are used as surrogates to estimate the concentrations of nutrients in a rural catchment and in an urban one, making use of machine learning models, specifically Random Forests. The results are compared with those of linear modelling using the same number of surrogates, obtaining a reduction in the Root Mean Squared Error (RMSE) of up to 60.1%. The profit from including up to seven surrogate sensors was computed, concluding that adding more than 4 and 5 sensors in each of the catchments respectively was not worthy in terms of error improvement., This work has been supported by the projects XDC “eXtreme-DataCloud” and DEEP-HybridDataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that have received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 777367 and No. 777435 respectively., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

ISC High Performance: International Conference on High Performance Computing., Ever growing interest and usage of deep learning rises a question on the performance of various infrastructures suitable for training of neural networks. We present here our approach and first results of tests performed with TensorFlow Benchmarks which use best practices for multi-GPU and distributed training. We pack the Benchmarks in Docker containers and execute them by means of uDocker and Singularity container tools on a single machine and in the HPC environment. The Benchmarks comprise a number of convolutional neural network models run across synthetic data and e.g. the ImageNet dataset. For the same Nvidia K80 GPU card we achieve the same performance in terms of processed images per second and similar scalability between 1-2-4 GPUs as presented by the TensorFlow developers. We therefore do not obtain statistically significant overhead due to the usage of containers in the multi-GPU case, and the approach of using TF Benchmarks in a Docker container can be applied across various systems., uDocker is being developed within the DEEP HybridDataCloud project, which receives funding from the European Union’s Horizon 2020 research and innovation program under agreement RIA 777435. A part of this work was performed on the computational resource ForHLR-II funded by the Ministry of Science, Research and the Arts Baden-Wuerttemberg and DFG (“Deutsche Forschungsgemeinschaft”).

Trabajo presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 16 al 20 de junio de 2019., The DEEP-HybridDataCloud project researches on intensive computing techniques such as deep learning, that require specialized GPU hardware to explore very large datasets, through a hybrid-cloud approach that enables the access to such resources. DEEP is built on User-centric policy, i.e. we understand the needs of our user communities and help them to combine their services in a way that encapsulates technical details the end user does not have to deal with. DEEP takes care to support users of different levels of experience by providing different integration paths. We show our current solutions to the problem, which among others include the Open Catalog for deep learning applications, DEEP-as-a-Service API for providing web access to machine learning models, CI/CD pipeline for user applications, Testbed resources. We also present our use-cases tackling various problems by means of deep learning and serving to demonstrate usefulness and scalability of our approach., DEEP HybridDataCloud receives funding from the European Union's Horizon 2020 research and innovation programme under agreement RIA 777435.

Trabajo presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 16 al 20 de junio de 2019., This work was developed with the support of the H2020 projects: INDIGO-DataCloud (RIA 653549), DEEP-Hybrid-DataCloud, (RIA 777435), EOSC-hub (RIA 777536). This work was developed with the support of the Portuguese Foundation for Science and Technology under the project: Infraestrutura Nacional de Computação Distribuída 01/SAICT/2016 - nº 02215

Trabajo presentado a IBERGRID: Delivering Innovative Computing and Data services to Researchers, celebrado en Santiago de Compostela (España) del 23 al 26 de septiembre de 2019., DEEP Hybrid DataCloud project was proposed with the necessity to support different amount of intensive computing techniques over specialized hardware, like HPC or GPUs. The project focus on the integration of this specialized, and expensive, hardware under a Cloud Platform as OpenStack that can be used on-demand by researchers of different areas. Within this project, a set of building blocks whose solution is called >DEEP as a Service> was implemented to make the application deployment more easier for the user. For this development, it is necessary to provide the researchers with access to these technologies as friendly but powerful services able to exploit very large datasets. On the one hand we have the gpu resources and on the other hand the users and their applications to run over those resources. DEEP needs to provide a service that controls how users can use those resources in an efficiently way. Although there are multiple technologies that address this problem as a queuing system, Apache Mesos has been developed to do it in an effective and controlled manner. Apache Mesos is a technology that abstracts the resources from different machines, like cpu, gpu, ram and storage to provide a scheduling and distributed system across the whole cloud environment. Mesos is easily to deploy over cpu-based systems but gpu needs a more tricky configuration as it is shown in this solution. As an added value, this solution provides an apache2 configuration for authenticate users from different communities by the current AAI service in DEEP. The proposed presentation will show the design and deployment of Mesos to work over gpus resources inside the Deep Hybrid DataCloud Project scope., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.

Resumen del trabajo presentado a la Conferencia Bio Diversity Next, celebrada en Leiden (Países Bajos) del 22 al 25 de octubre de 2019., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.

In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models., This work was supported by the project DEEP-Hybrid-DataCloud ‘‘Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud’’ that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under
Grant 777435., Peer reviewed

The work presented in this paper deals with a proactive network monitoring for security and protection of computing infrastructures. We provide an exploitation of an intelligent module, in the form of a as a machine learning application using deep learning modeling, in order to enhance functionality of intrusion detection system supervising network traffic flows. Currently, intrusion detection systems work well for network monitoring in near real-time and they effectively deal with threats in a reactive way. Deep learning is the emerging generation of artificial intelligence techniques and one of the most promising candidates for intelligence integration into traditional solutions leading to quality improvement of the original solutions. The work presented in this paper faces the challenge of cooperation between deep learning techniques and large-scale data processing. The outcomes obtained from extensive and careful experiments show the applicability and feasibility of simultaneously modelled multiple monitoring channels using deep learning techniques. The proper joining of deep learning modelling with scalable data preprocessing ensures high quality and stability of model performance in dynamic and fast-changing environments such as network traffic flow monitoring., This work was supported by the Designing and Enabling E-Infrastructures for Intensive Processing in a Hybrid DataCloud
(DEEP-Hybrid-DataCloud) Project through the European Union’s Horizon 2020 Research and Innovation Programme under Grant 777435 and VEGA 2/0125/20 New Methods and Approaches for Distributed Scalable Computing, Peer reviewed

This work provides a tool to train and evaluate an image regression model. The code is prepared for liquid volume estimation of glass containers as part of the BrainGut_WineUp project. It was adapted from the original image classification module developed in the DEEP Hybrid-DataCloud project., Miriam Cobo acknowledges support from Consejo Superior de Investigaciones Científicas (CSIC) and Instituto de Física de Cantabria (IFCA) via the “Proyecto Branyas” project., Peer reviewed

Deep learning is nowadays at the forefront of artificial intelligence. More precisely, the use of convolutional neural networks has drastically improved the learning capabilities of computer vision applications, being able to directly consider raw data without any prior feature extraction. Advanced methods in the machine learning field, such as adaptive momentum algorithms or dropout regularization, have dramatically improved the convolutional neural networks predicting ability, outperforming that of conventional fully connected neural networks. This work summarizes, in an intended didactic way, the main aspects of these cutting-edge techniques from a medical imaging perspective., We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI)., Consejo Superior de Investigaciones Cientificas (JS-CSIC-BMCSO-0920) Deep-Hybrid DataCloud (H2020—Grant agreement No 777435) Servicio Cantabro de Salud., Peer reviewed

Deep learning has been postulated as a solution for numerous problems in different branches of science. Given the resource-intensive nature of these models, they often need to be executed on specialized hardware such graphical processing units (GPUs) in a distributed manner. In the academic field, researchers get access to this kind of resources through High Performance Computing (HPC) clusters. This kind of infrastructures make the training of these models difficult due to their multi-user nature and limited user permission. In addition, different HPC clusters may possess different peculiarities that can entangle the research cycle (e.g., libraries dependencies). In this paper we develop a workflow and methodology for the distributed training of deep learning models in HPC clusters which provides researchers with a series of novel advantages. It relies on udocker as containerization tool and on Horovod as library for the distribution of the models across multiple GPUs. udocker does not need any special permission, allowing researchers to run the entire workflow without relying on any administrator. Horovod ensures the efficient distribution of the training independently of the deep learning framework used. Additionally, due to containerization and specific features of the workflow, it provides researchers with a cluster-agnostic way of running their models. The experiments carried out show that the workflow offers good scalability in the distributed training of the models and that it easily adapts to different clusters., This work has been supported by the project DEEP-Hybrid-DataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement Number 777435. J. González-Abad and A. López García would also like to acknowledge the support of the funding from the Spanish Agencia Estatal de Investigación through the Unidad de Excelencia María de Maeztu with reference MDM-2017-0765., Peer reviewed

In this work the applicability of an ensemble of population and machine learning models to predict the evolution of the COVID-19 pandemic in Spain is evaluated, relying solely on public datasets. Firstly, using only incidence data, we trained machine learning models and adjusted classical ODE-based population models, especially suited to capture long term trends. As a novel approach, we then made an ensemble of these two families of models in order to obtain a more robust and accurate prediction. We then proceed to improve machine learning models by adding more input features: vaccination, human mobility and weather conditions. However, these improvements did not translate to the overall ensemble, as the different model families had also different prediction patterns. Additionally, machine learning models degraded when new COVID variants appeared after training. We finally used Shapley Additive Explanation values to discern the relative importance of the different input features for the machine learning models’ predictions. The conclusion of this work is that the ensemble of machine learning models and population models can be a promising alternative to SEIR-like compartmental models, especially given that the former do not need data from recovered patients, which are hard to collect and generally unavailable., The authors acknowledge the funding and support from the project Distancia-COVID (CSICCOV19-039) of the CSIC funded by a contribution of AENA; from the Universidad de Cantabria and the Consejería de Universidades, Igualdad, Cultura y Deporte of the Gobierno de Cantabria via the “Instrumentación y ciencia de datos para sondear la naturaleza del universo” project; from the Spanish Ministry of Science, Innovation and Universities through the María de Maeztu programme for Units of Excellence in R&D (MDM-2017-0765); and the support from the project DEEP-Hybrid-DataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 777435. This research work was also funded by the European Commission - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI Salud Global). The authors would also like to thank the Spanish Ministry of Transport, Mobility and Urban Agenda (MITMA) and the Instituto Nacional de Estadística (INE) for releasing as open data the Big Data mobility study and the DataCOVID mobility data. Also, the authors would like to acknowledge the volunteers compiling the per-province dataset of COVID-19 incidence in Spain in the early phases of the pandemic outbreak., Peer reviewed

Coronary artery tortuosity is usually an undetected condition in patients undergoing coronary angiography. This condition requires a longer examination by the specialist to be detected. Yet, detailed knowledge of the morphology of coronary arteries is essential for planning any interventional treatment, such as stenting. We aimed to analyze coronary artery tortuosity in coronary angiography with artificial intelligence techniques to develop an algorithm capable of automatically detecting this condition in patients. This work uses deep learning techniques, in particular, convolutional neural networks, to classify patients into tortuous or non-tortuous based on their coronary angiography. The developed model was trained both on left (Spider) and right (45°/0°) coronary angiographies following a fivefold cross-validation procedure. A total of 658 coronary angiographies were included. Experimental results demonstrated satisfactory performance of our image-based tortuosity detection system, with a test accuracy of (87 ± 6)%. The deep learning model had a mean area under the curve of 0.96 ± 0.03 over the test sets. The sensitivity, specificity, positive predictive values, and negative predictive values of the model for detecting coronary artery tortuosity were (87 ± 10)%, (88 ± 10)%, (89 ± 8)%, and (88 ± 9)%, respectively. Deep learning convolutional neural networks were found to have comparable sensitivity and specificity with independent experts’ radiological visual examination for detecting coronary artery tortuosity for a conservative threshold of 0.5. These findings have promising applications in the field of cardiology and medical imaging., M. C. acknowledges support from Consejo Superior de Investigaciones Científicas (CSIC) and Institute of Physics of Cantabria (IFCA) given by the European Commission – NextGenerationEU, through CSIC’s Global Health Platform (“PTI Salud Global”). The authors acknowledge support from Universidad Autónoma de Chile. We also acknowledge the support from the Advanced Computing and e-Science group at the Institute of Physics of Cantabria (IFCA-CSIC-UC) and from the DEEP-HybridDataCloud H2020 project (Grant Agreement 777435). I. H. acknowledges support from Universidad de Cantabria and Consejería de Universidades, Igualdad, Cultura y Deporte del Gobierno de Cantabria via the “Instrumentación y ciencia de datos para sondear la naturaleza del universo” project., Peer reviewed

[EN] Serverless computing and, in particular, the functions as a service model has become a convincing paradigm for the development and implementation of highly scalable applications in the cloud. This is due to the transparent management of three key functionalities: triggering of functions due to events, automatic provisioning and scalability of resources, and fine-grained pay-per-use. This article presents a serverless web-based scientific gateway to execute the inference phase of previously trained machine learning and artificial intelligence models. The execution of the models is performed both in Amazon Web Services and in on-premises clouds with the OSCAR framework for serverless scientific computing. In both cases, the computing infrastructure grows elastically according to the demand adopting scale-to-zero approaches to minimize costs. The web interface provides an improved user experience by simplifying the use of the models. The usage of machine learning in a computing platform that can use both on-premises clouds and public clouds constitutes a step forward in the adoption of serverless computing for scientific applications., AI-SPRINT "Artificial Intelligence in Secure PRIvacy-preserving computing coN-Tinuum", Grant/Award Number: 101016577; DEEP-Hybrid-DataCloud "Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud", Grant/Award Number: 777435; SERCLOCO, Grant/Award Numbers: MCIN/AEI/10.13039/501100011033, PID2020-113126RB-I00

[EN] In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models., This work was supported by the project DEEP-Hybrid-DataCloud ``Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud'' that has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant 777435

[EN] Virtual clusters are widely used computing platforms than can be deployed in multiple cloud platforms. The ability to dynamically grow and shrink the number of nodes has paved the way for customised elastic computing both for High Performance Computing and High Throughput Computing workloads. However, elasticity is typically restricted to a single cloud site, thus hindering the ability to provision computational resources from multiple geographically distributed cloud sites. To this aim, this paper introduces an architecture of open-source components that coherently deploy a virtual elastic cluster across multiple cloud sites to perform large-scale computing. These hybrid virtual elastic clusters are automatically deployed and configured using an Infrastructure as Code (IaC) approach on a distributed hybrid testbed that spans different organizations, including on-premises and public clouds, supporting automated tunneling of communications across the cluster nodes with advanced VPN topologies. The results indicate that cluster-based computing of embarrassingly parallel jobs can benefit from hybrid virtual clusters that aggregate computing resources from multiple cloud back-ends and bring them together into a dedicated, albeit virtual network., The work presented in this article has been partially funded by project DEEP Hybrid-DataCloud (grant agreement No 777435). GM and MC would also like to thank the Spanish "Ministerio de Economia, Industria y Competitividad" for the project "BigCLOE" with reference number TIN2016-79951-R. Computational resources at CESNET, used in the real-world use case, were supplied by the project "e-Infrastruktura CZ" (e-INFRA LM2018140) provided within the program Projects of Large Research, Development and Innovations Infrastructures