The combined impact of new computing resources and techniques with an increasing avalanche of large datasets, is transforming many research areas and may lead to technological breakthroughs that can be used by billions of people. In the recent years, machine Learning and especially its subfield Deep Learning have seen impressive advances. Techniques developed within these two fields are now able to analyze and learn from huge amounts of real world examples in a disparate formats. While the number of Machine Learning algorithms is extensive and growing, their implementations through frameworks and libraries is also extensive and growing too. The software devel- opment in this field is fast paced with a large number of open-source software coming from the academy, industry, start-ups or wider open-source commu- nities. This survey presents a recent time-slide comprehensive overview with comparisons as well as trends in development and usage of cutting-edge Arti- ficial Intelligence software. It also provides an overview of massive parallelism support that is capable of scaling computation effectively and efficiently in the era of Big Data., Álvaro López García, Ignacio Heredia, Giang Nguyen, Viet Tran, Stefan Dlugolinsky,
Martin Bobák, and Ladislav Hluchý are supported by the project DEEP-HybridDataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 777435.
Giang Nguyen, Viet Tran, Stefan Dlugolinsky, Martin Bobák, and Ladislav Hluchý are also supported by the Project VEGA 2/0167/16 “Methods and algorithms for the semantic processing of Big Data in distributed computing environment”. The authors would like to thanks to all colleagues, especially for Ján Astaloš for
knowledge sharing and teamwork., Peer reviewed

Nowadays, in the era of Internet of Things when everything is connected via the Internet, the number of mobile devices has risen exponentially up to billions around the world. In line with this increase, the volume of data generated is enormous and has attracted malefactors who do ill deeds to others. For hackers, one of the popular threads to mobile devices is to spread malware. These actions are very difficult to prevent because the application installation and configuration rights are set by owners, who usually have very low knowledge or do not care about the security. In this study, our aim is to improve security in the environment of mobile devices by proposing a novel system to detect malware intrusions automatically. Our solution is based on modelling user behaviours and applying the heuristic analysis approach to mobile logs generated during the device operation process. Although behaviours of individual users have a significant impact on the social cyber-security, to achieve the user awareness has still remained one of the major challenges today. For this task, there is proposed a light-weight semantic formalization in the form of physical and logical taxonomy for classifying the collected raw log data. Then a set of techniques is used, like sliding windows, lemmatization, feature selection, term weighting, and so on, to process data. Meanwhile, malware detection tasks are performed based on incremental machine learning mechanisms, because of the potential complexity of this tasks. The solution is developed in the manner to allow the scalability with several blocks that cover pre-processing raw collected logs from mobile devices, automatically creating datasets for machine learning methods, using the best selected model for detecting suspicious activity surrounding malware intrusions, and supporting decision making using a predictive risk factor. We experimented cautiously with the proposal and achieved test results confirm the effectiveness and feasibility of the proposed system in applying to the large-scale mobile environment., This work is supported by the projects Slovak VEGA 2/0167/16 “Methods and algorithms for the semantic processing of Big Data in distributed computing environment”, Vietnamese MOET’s No. B2017-BKA-32 “Research on developing software framework to integrate IoT gateways for fog computing deployed on multi-cloud environment”, and DEEP-HybridDataCloud EU H2020-777435
“Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud”., Peer reviewed

Continuous high frequency water quality monitoring is becoming a critical task to support water management. Despite the advancements in sensor technologies, certain variables cannot be easily and/or economically monitored in-situ and in real time. In these cases, surrogate measures can be used to make estimations by means of data-driven models. In this work, variables that are commonly measured in-situ are used as surrogates to estimate the concentrations of nutrients in a rural catchment and in an urban one, making use of machine learning models, specifically Random Forests. The results are compared with those of linear modelling using the same number of surrogates, obtaining a reduction in the Root Mean Squared Error (RMSE) of up to 60.1%. The profit from including up to seven surrogate sensors was computed, concluding that adding more than 4 and 5 sensors in each of the catchments respectively was not worthy in terms of error improvement., This work has been supported by the projects XDC “eXtreme-DataCloud” and DEEP-HybridDataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that have received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 777367 and No. 777435 respectively., Peer reviewed

In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models., This work was supported by the project DEEP-Hybrid-DataCloud ‘‘Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud’’ that has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under
Grant 777435., Peer reviewed

The work presented in this paper deals with a proactive network monitoring for security and protection of computing infrastructures. We provide an exploitation of an intelligent module, in the form of a as a machine learning application using deep learning modeling, in order to enhance functionality of intrusion detection system supervising network traffic flows. Currently, intrusion detection systems work well for network monitoring in near real-time and they effectively deal with threats in a reactive way. Deep learning is the emerging generation of artificial intelligence techniques and one of the most promising candidates for intelligence integration into traditional solutions leading to quality improvement of the original solutions. The work presented in this paper faces the challenge of cooperation between deep learning techniques and large-scale data processing. The outcomes obtained from extensive and careful experiments show the applicability and feasibility of simultaneously modelled multiple monitoring channels using deep learning techniques. The proper joining of deep learning modelling with scalable data preprocessing ensures high quality and stability of model performance in dynamic and fast-changing environments such as network traffic flow monitoring., This work was supported by the Designing and Enabling E-Infrastructures for Intensive Processing in a Hybrid DataCloud
(DEEP-Hybrid-DataCloud) Project through the European Union’s Horizon 2020 Research and Innovation Programme under Grant 777435 and VEGA 2/0125/20 New Methods and Approaches for Distributed Scalable Computing, Peer reviewed

Deep learning is nowadays at the forefront of artificial intelligence. More precisely, the use of convolutional neural networks has drastically improved the learning capabilities of computer vision applications, being able to directly consider raw data without any prior feature extraction. Advanced methods in the machine learning field, such as adaptive momentum algorithms or dropout regularization, have dramatically improved the convolutional neural networks predicting ability, outperforming that of conventional fully connected neural networks. This work summarizes, in an intended didactic way, the main aspects of these cutting-edge techniques from a medical imaging perspective., We acknowledge support of the publication fee by the CSIC Open Access Publication Support Initiative through its Unit of Information Resources for Research (URICI)., Consejo Superior de Investigaciones Cientificas (JS-CSIC-BMCSO-0920) Deep-Hybrid DataCloud (H2020—Grant agreement No 777435) Servicio Cantabro de Salud., Peer reviewed

Deep learning has been postulated as a solution for numerous problems in different branches of science. Given the resource-intensive nature of these models, they often need to be executed on specialized hardware such graphical processing units (GPUs) in a distributed manner. In the academic field, researchers get access to this kind of resources through High Performance Computing (HPC) clusters. This kind of infrastructures make the training of these models difficult due to their multi-user nature and limited user permission. In addition, different HPC clusters may possess different peculiarities that can entangle the research cycle (e.g., libraries dependencies). In this paper we develop a workflow and methodology for the distributed training of deep learning models in HPC clusters which provides researchers with a series of novel advantages. It relies on udocker as containerization tool and on Horovod as library for the distribution of the models across multiple GPUs. udocker does not need any special permission, allowing researchers to run the entire workflow without relying on any administrator. Horovod ensures the efficient distribution of the training independently of the deep learning framework used. Additionally, due to containerization and specific features of the workflow, it provides researchers with a cluster-agnostic way of running their models. The experiments carried out show that the workflow offers good scalability in the distributed training of the models and that it easily adapts to different clusters., This work has been supported by the project DEEP-Hybrid-DataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement Number 777435. J. González-Abad and A. López García would also like to acknowledge the support of the funding from the Spanish Agencia Estatal de Investigación through the Unidad de Excelencia María de Maeztu with reference MDM-2017-0765., Peer reviewed

In this work the applicability of an ensemble of population and machine learning models to predict the evolution of the COVID-19 pandemic in Spain is evaluated, relying solely on public datasets. Firstly, using only incidence data, we trained machine learning models and adjusted classical ODE-based population models, especially suited to capture long term trends. As a novel approach, we then made an ensemble of these two families of models in order to obtain a more robust and accurate prediction. We then proceed to improve machine learning models by adding more input features: vaccination, human mobility and weather conditions. However, these improvements did not translate to the overall ensemble, as the different model families had also different prediction patterns. Additionally, machine learning models degraded when new COVID variants appeared after training. We finally used Shapley Additive Explanation values to discern the relative importance of the different input features for the machine learning models’ predictions. The conclusion of this work is that the ensemble of machine learning models and population models can be a promising alternative to SEIR-like compartmental models, especially given that the former do not need data from recovered patients, which are hard to collect and generally unavailable., The authors acknowledge the funding and support from the project Distancia-COVID (CSICCOV19-039) of the CSIC funded by a contribution of AENA; from the Universidad de Cantabria and the Consejería de Universidades, Igualdad, Cultura y Deporte of the Gobierno de Cantabria via the “Instrumentación y ciencia de datos para sondear la naturaleza del universo” project; from the Spanish Ministry of Science, Innovation and Universities through the María de Maeztu programme for Units of Excellence in R&D (MDM-2017-0765); and the support from the project DEEP-Hybrid-DataCloud “Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud” that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 777435. This research work was also funded by the European Commission - NextGenerationEU (Regulation EU 2020/2094), through CSIC’s Global Health Platform (PTI Salud Global). The authors would also like to thank the Spanish Ministry of Transport, Mobility and Urban Agenda (MITMA) and the Instituto Nacional de Estadística (INE) for releasing as open data the Big Data mobility study and the DataCOVID mobility data. Also, the authors would like to acknowledge the volunteers compiling the per-province dataset of COVID-19 incidence in Spain in the early phases of the pandemic outbreak., Peer reviewed

Coronary artery tortuosity is usually an undetected condition in patients undergoing coronary angiography. This condition requires a longer examination by the specialist to be detected. Yet, detailed knowledge of the morphology of coronary arteries is essential for planning any interventional treatment, such as stenting. We aimed to analyze coronary artery tortuosity in coronary angiography with artificial intelligence techniques to develop an algorithm capable of automatically detecting this condition in patients. This work uses deep learning techniques, in particular, convolutional neural networks, to classify patients into tortuous or non-tortuous based on their coronary angiography. The developed model was trained both on left (Spider) and right (45°/0°) coronary angiographies following a fivefold cross-validation procedure. A total of 658 coronary angiographies were included. Experimental results demonstrated satisfactory performance of our image-based tortuosity detection system, with a test accuracy of (87 ± 6)%. The deep learning model had a mean area under the curve of 0.96 ± 0.03 over the test sets. The sensitivity, specificity, positive predictive values, and negative predictive values of the model for detecting coronary artery tortuosity were (87 ± 10)%, (88 ± 10)%, (89 ± 8)%, and (88 ± 9)%, respectively. Deep learning convolutional neural networks were found to have comparable sensitivity and specificity with independent experts’ radiological visual examination for detecting coronary artery tortuosity for a conservative threshold of 0.5. These findings have promising applications in the field of cardiology and medical imaging., M. C. acknowledges support from Consejo Superior de Investigaciones Científicas (CSIC) and Institute of Physics of Cantabria (IFCA) given by the European Commission – NextGenerationEU, through CSIC’s Global Health Platform (“PTI Salud Global”). The authors acknowledge support from Universidad Autónoma de Chile. We also acknowledge the support from the Advanced Computing and e-Science group at the Institute of Physics of Cantabria (IFCA-CSIC-UC) and from the DEEP-HybridDataCloud H2020 project (Grant Agreement 777435). I. H. acknowledges support from Universidad de Cantabria and Consejería de Universidades, Igualdad, Cultura y Deporte del Gobierno de Cantabria via the “Instrumentación y ciencia de datos para sondear la naturaleza del universo” project., Peer reviewed

[EN] Serverless computing and, in particular, the functions as a service model has become a convincing paradigm for the development and implementation of highly scalable applications in the cloud. This is due to the transparent management of three key functionalities: triggering of functions due to events, automatic provisioning and scalability of resources, and fine-grained pay-per-use. This article presents a serverless web-based scientific gateway to execute the inference phase of previously trained machine learning and artificial intelligence models. The execution of the models is performed both in Amazon Web Services and in on-premises clouds with the OSCAR framework for serverless scientific computing. In both cases, the computing infrastructure grows elastically according to the demand adopting scale-to-zero approaches to minimize costs. The web interface provides an improved user experience by simplifying the use of the models. The usage of machine learning in a computing platform that can use both on-premises clouds and public clouds constitutes a step forward in the adoption of serverless computing for scientific applications., AI-SPRINT "Artificial Intelligence in Secure PRIvacy-preserving computing coN-Tinuum", Grant/Award Number: 101016577; DEEP-Hybrid-DataCloud "Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud", Grant/Award Number: 777435; SERCLOCO, Grant/Award Numbers: MCIN/AEI/10.13039/501100011033, PID2020-113126RB-I00

[EN] In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models., This work was supported by the project DEEP-Hybrid-DataCloud ``Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud'' that has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant 777435

[EN] Virtual clusters are widely used computing platforms than can be deployed in multiple cloud platforms. The ability to dynamically grow and shrink the number of nodes has paved the way for customised elastic computing both for High Performance Computing and High Throughput Computing workloads. However, elasticity is typically restricted to a single cloud site, thus hindering the ability to provision computational resources from multiple geographically distributed cloud sites. To this aim, this paper introduces an architecture of open-source components that coherently deploy a virtual elastic cluster across multiple cloud sites to perform large-scale computing. These hybrid virtual elastic clusters are automatically deployed and configured using an Infrastructure as Code (IaC) approach on a distributed hybrid testbed that spans different organizations, including on-premises and public clouds, supporting automated tunneling of communications across the cluster nodes with advanced VPN topologies. The results indicate that cluster-based computing of embarrassingly parallel jobs can benefit from hybrid virtual clusters that aggregate computing resources from multiple cloud back-ends and bring them together into a dedicated, albeit virtual network., The work presented in this article has been partially funded by project DEEP Hybrid-DataCloud (grant agreement No 777435). GM and MC would also like to thank the Spanish "Ministerio de Economia, Industria y Competitividad" for the project "BigCLOE" with reference number TIN2016-79951-R. Computational resources at CESNET, used in the real-world use case, were supplied by the project "e-Infrastruktura CZ" (e-INFRA LM2018140) provided within the program Projects of Large Research, Development and Innovations Infrastructures