Update on November 2022., The purpose of this document is to define a set of quality standards, procedures and best practices to conform a Software Quality Assurance plan to serve as a reference within the European research ecosystem related projects for the adequate development and timely delivery of software products., The INDIGO-DataCloud, DEEP-Hybrid-DataCloud and eXtreme-DataCloud projects have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 653549, 777435 and 777367 respectively., No

Available soon., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This report summarises the work of WP1 on the Data Management considerations and initial plan
for the DEEP-HybridDataCloud project. This document describes the types of data that will be
generated or collected during the project, the standards that will be used and the ways in which the
data may be exploited and shared including the data security and ethical aspects., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This report summarises the work of WP2 on the initial plan on the selection of Use Cases,
providing a key input to the design of the DEEP-HybridDataCloud testbed and laying out the
DEEP-HybridDataCloud solutions will be used. This document includes a description of the
Research Communities involved and of the use-cases proposed, including Data Management and
Computational Intensive issues. Based on the inputs provided by the WP2 partners, an initial list of
requirements for each application has been collected, that has been used in turn to assemble a list of
common requirements that has been provided as input to the discussion with the technical work
packages of the DEEP-HybridDataCloud project., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document describes the design of the PaaS Layer and the detailed work plan for the WP5 of
the DEEP-HybridDataCloud project, which aims at adopting and extending the PaaS orchestration solution for cloud infrastructures realised within the INDIGO-DataCloud project. In particular, the main focus is on the architecture and features of the hybrid cloud solution., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document describes the state of the art of technologies for supporting bare-metal, accelerators
and HPC in cloud and proposes an initial implementation plan. Available technologies will be
analyzed from different points of views: stand-alone use, integration with cloud middleware,
support for accelerators and HPC platforms. Based on results of these analyses, an initial
implementation plan will be proposed containing information on what features should be
developed and what components should be improved in the next period of the project., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document provides a description of the design, architecture and work plan the DEEPHybridDataCloud
Work Package 6 (WP6) in order to provide the DEEP as a Service solution. As
such it provides an overview of the state of the art of the relevant components and technologies, as
well as a technology readiness level assessment with regards to the required functionality, the
required interactions with other work packages in the project, as well as the detailed work plan and
risk assessment for each of the activities., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

The deliverable contains the plan, design, architecture and deployment of the Pilot Preview testbed based on technical requirements and descriptions from the WP2 use cases. The services and components developed during the DEEP-HybridDataCloud by the teams part of the WP4, 5 and 6
are deployed, tested and validated by end-users in this testbed. It also describes how the Pilot Preview testbed services and components will be integrated with EOSC production infrastructure and other external resource providers., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This deliverable describes the first implementation of the software platform for accessing
accelerators and HPC. The list of components included in the software platform is based on the
analysis provided by Deliverable D4.1. This document provides detailed descriptions of software
components used in the platforms, the work done on each component and its current status.
Evaluation of achieved results and implementation plan for the next periods are also included., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document complements deliverable D5.1 Definition of the Architecture of the Hybrid Cloud
(D5.1) with the specific prototype developments carried out to support the deployment of hybrid
infrastructures across multiple IaaS Cloud sites. The document describes the technical challenges,
the evolution of the components to support this prototype and a roadmap of implementation
towards the final release of the High Level Hybrid Cloud solutions., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

This document provides an updated description of the prototype implementation of the DEEP as a Service solution that is being developed within the DEEP-Hybrid-DataCloud project Work Package 6 (WP6). As such it provides an overview of the state of the art of the relevant components and technologies, as well as a technology readiness level assessment with regards to the required functionality, the required interactions with other work packages in the project, as well as the detailed work plan and risk assessment for each of the activities., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., No

DEEP as a Service (DEEPaaS) is a REST API that is focused on providing access to machine learning models. By using DEEPaaS users can easily run a REST API in fron of their model, thus accesing its functionality via HTTP calls. Quickstart: The best way to quickly try the DEEPaaS API is trough: make run. This command will install a virtualenv (in the `virtualenv` directory) with DEEPaaS and all its dependencies and will run the DEEPaaS REST API, listening on `127.0.0.1:5000`. If you browse to `http://127.0.0.1:5000` you will get the swagger documentation page. Develop mode: If you want to run the code in develop mode (i.e. `pip install -e`), you can issue the following command before: make develop., DEEP as a Service (DEEPaaS) is a REST API that is focused on providing access to machine learning models. By using DEEPaaS users can easily run a REST API in fron of their model, thus accesing its functionality via HTTP calls., Proyecto: Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (DEEP-HybridDataCloud)., Peer reviewed

Apache License, Version 2.0, This is a plug-and-play tool to train and evaluate an image classifier on a custom dataset using deep neural networks., DEEP Hybrid-Datacloud, GA 777435., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

Apache License 2.0, Agencia Financiadora: EC. Acuerdo de subvención: H2020 777435, No

Apache License 2.0, EC: H2020 777435., Peer reviewed

Apache License 2.0, EC: H2020 777435, Peer reviewed

Apache License 2.0, EC: H2020 777435, Peer reviewed

Apache 2.0, EC: H2020 777435, Peer reviewed

Apache 2.0, EC: H2020 777435, Peer reviewed

Arguably, Python is one of the most widely used programming languages in data analytics, data mining, artificial intelligence, machine learning and deep learning (Nguyen et al., 2019). DEEPaaS API is a Python package that provides a REST API (Fielding, 2000) that can be used to easily expose the underlying model functionality over HTTP. Developers can plug their applications into DEEPaaS API with minimal requirements and modifications, allowing for easier interaction with the underlying functionality, both for training, validating and testing., This software has been developed within the DEEP-Hybrid-DataCloud (Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud) project that has received funding from the European Union’s Horizon 2020 research and innovation programme
under grant agreement No 777435., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

AVAILABLE SOON., Project: DEEP-HybridDataCloud - Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud (777435)., Peer reviewed

ISC High Performance: International Conference on High Performance Computing., Ever growing interest and usage of deep learning rises a question on the performance of various infrastructures suitable for training of neural networks. We present here our approach and first results of tests performed with TensorFlow Benchmarks which use best practices for multi-GPU and distributed training. We pack the Benchmarks in Docker containers and execute them by means of uDocker and Singularity container tools on a single machine and in the HPC environment. The Benchmarks comprise a number of convolutional neural network models run across synthetic data and e.g. the ImageNet dataset. For the same Nvidia K80 GPU card we achieve the same performance in terms of processed images per second and similar scalability between 1-2-4 GPUs as presented by the TensorFlow developers. We therefore do not obtain statistically significant overhead due to the usage of containers in the multi-GPU case, and the approach of using TF Benchmarks in a Docker container can be applied across various systems., uDocker is being developed within the DEEP HybridDataCloud project, which receives funding from the European Union’s Horizon 2020 research and innovation program under agreement RIA 777435. A part of this work was performed on the computational resource ForHLR-II funded by the Ministry of Science, Research and the Arts Baden-Wuerttemberg and DFG (“Deutsche Forschungsgemeinschaft”).

This work provides a tool to train and evaluate an image regression model. The code is prepared for liquid volume estimation of glass containers as part of the BrainGut_WineUp project. It was adapted from the original image classification module developed in the DEEP Hybrid-DataCloud project., Miriam Cobo acknowledges support from Consejo Superior de Investigaciones Científicas (CSIC) and Instituto de Física de Cantabria (IFCA) via the “Proyecto Branyas” project., Peer reviewed