Pasar al contenido principal

CLUSTERS VIRTUALES ELASTICOS Y MIGRABLES SOBRE INFRAESTRUCTURAS CLOUD HIBRIDAS

TIN2013-44390-R

•
Nombre agencia financiadora Ministerio de Economía y Competitividad
Acrónimo agencia financiadora MINECO
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2013
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Unidad de gestión Secretaría de Estado de Educación, Formación Profesional y Universidades
Centro beneficiario UNIVERSITAT POLITÈCNICA DE VALÈNCIA
Identificador persistente http://dx.doi.org/10.13039/501100003329

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

OMWS: A Web Service Interface for Ecological Niche Modelling

  • Giovanni, Renato de
  • Torres Serrano, Erik
  • Amaral, Rafael B.
  • Rebello, Vinod
  • Canhos, Vanderlei P.
  • Blanquer Espert, Ignacio|||0000-0003-1692-8922
[EN] Ecological niche modelling (ENM) experiments often involve a high number of tasks to be performed. Such tasks may consume a significant amount of computing resources and take a long time to complete, especially when using personal computers. OMWS is a Web service interface that allows more powerful computing back-ends to be remotely exploited by other applications to carry out ENM tasks. Its latest version includes a new operation that can be used to specify complex workflows in a single request, adding the possibility of using workflow management systems on parallel computing back-end. In this paper we describe the OMWS protocol and compare its most recent version with the previous one by running the same ENM experiment using two functionally equivalent clients, each designed for one of the OMWS interface versions. Different back-end configurations were used to investigate how the performance scales for each protocol version when more processing power is made available. Results show that the new version outperforms (in a factor of 2) the previous one when more computing resources are used., The latest version of OMWS contains improvements coming from different sets of requirements originated from two projects that funded their corresponding implementation: EUBrazilOpenBio14, with grants from the European Commission and the National Council for Scientific and Technological Development of Brazil (CNPq) of the Brazilian Ministry of Science and Technology (MCT), and BioVeL, with grants from the European Commission. Server infrastructure was operated through a provisioning system developed in the frame of the Spanish project CLUVIEM (TIN2013-44390-R) funded by the "Ministerio de Economía y Competitividad".
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Container-based Virtual Elastic Clusters

  • Alfonso Laguna, Carlos De|||0000-0002-2378-021X
  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Moltó, Germán|||0000-0002-8049-253X
[EN] eScience demands large-scale computing clusters to support the efficient execution of resource-intensive scientific applications. Virtual Machines (VMs) have introduced the ability to provide customizable execution environments, at the expense of performance loss for applications. However, in recent years, containers have emerged as a light-weight virtualization technology compared to VMs. Indeed, the usage of containers for virtual clusters allows better performance for the applications and fast deployment of additional working nodes, for enhanced elasticity. This paper focuses on the deployment, configuration and management of Virtual Elastic computer Clusters (VEC) dedicated to process scientific workloads. The nodes of the scientific cluster are hosted in containers running on bare-metal machines. The opensource tool Elastic Cluster for Docker (EC4Docker) is introduced, integrated with Docker Swarm to create auto-scaled virtual computer clusters of containers across distributed deployments. We also discuss the benefits and limitations of this solution and analyse the performance of the developed tools under a real scenario by means of a scientific use case that demonstrates the feasibility of the proposed approach., This work has been developed under the support of the program "Ayudas para la contratacion de personal investigador en formacion de catheter predoctoral, programa VALi+d", grant number ACIF/2013/003, from the Conselleria d'Educacio of the Generalitat Valenciana. The authors wish to thank the financial support received form The Spanish Ministry of Economy and Competitiveness to develop the project "CLUVIEM", with reference TIN2013-44390-R.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Automatic memory-based vertical elasticity and oversubscription on cloud platforms

  • Moltó, Germán|||0000-0002-8049-253X
  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Alfonso Laguna, Carlos De|||0000-0002-2378-021X
Hypervisors and Operating Systems support vertical elasticity techniques such as memory ballooning to dynamically assign the memory of Virtual Machines (VMs). However, current Cloud Management Platforms (CMPs), such as OpenNebula or OpenStack, do not currently support dynamic vertical elasticity. This paper describes a system that integrates with the CMP to provide automatic vertical elasticity to adapt the memory size of the VMs to their current memory consumption, featuring live migration to prevent overload scenarios, without downtime for the VMs. This enables an enhanced VM-per-host consolidation ratio while maintaining the Quality of Service for VMs, since their memory is dynamically increased as necessary. The feasibility of the development is assessed via two case studies based on OpenNebula featuring (i) horizontal and vertical elastic virtual clusters on a production Grid infrastructure and (ii) elastic multi-tenant VMs that run Docker containers coupled with live migration techniques. The results show that memory oversubscription can be integrated on CMPs to deliver automatic memory management without severely impacting the performance of the VMs. This results in a memory management framework for on-premises Clouds that features live migration to safely enable transient oversubscription of physical resources in a CMP. © 2015 Elsevier B.V. All rights reserved., The authors would like to thank the Spanish "Ministerio de Economia y Competitividad" for the project CLUVIEM (TIN2013-44390-R) and the European Commission for the project INDIGO-DataCloud with grant number 653549.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Managing Workflows on top of a Cloud Computing Orchestrator for using heterogeneous environments on e-Science

  • Carrión Collado, Abel Antonio
  • Kotowski, Nelson
  • Jardim, Rodrigo
  • Dávila, Alberto M. R.
  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Blanquer Espert, Ignacio|||0000-0003-1692-8922
[EN] Scientific workflows (SWFs) are widely used to model processes in e-Science. SWFs are executed by means of workflow management systems (WMSs), which orchestrate the workload on top of computing infrastructures. The advent of cloud computing infrastructures has opened the door of using on-demand infrastructures to complement or even replace local infrastructures. However, new issues have arisen, such as the integration of hybrid resources or the compromise between infrastructure reutilisation and elasticity. In this article, we present an ad hoc solution for managing workflows exploiting the capabilities of cloud orchestrators to deploy resources on demand according to the workload and to combine heterogeneous cloud providers (such as on-premise clouds and public clouds) and traditional infrastructures (clusters) to minimise costs and response time. The work does not propose yet another WMS but demonstrates the benefits of the integration of cloud orchestration when running complex workflows. The article shows several configuration experiments from a realistic comparative genomics workflow called Orthosearch, to migrate memory-intensive workload to public infrastructures while keeping other blocks of the experiment running locally. The article computes running time and cost suggesting best practices., This paper wants to acknowledge the support of the EUBrazilCC project, funded by the European Commission (STREP 614048) and the Brazilian MCT/CNPq N. 13/2012, for the use of its infrastructure. The authors would like also to thank the Spanish 'Ministerio de Economia y Competitividad' for the project 'Clusters Virtuales Elasticos y Migrables sobre Infraestructuras Cloud Hibridas' with reference TIN2013-44390-R.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Self-managed Cost-efficient Virtual Elastic Clusters on Hybrid Cloud Infrastructures

  • Calatrava Arroyo, Amanda|||0000-0002-9018-9171
  • Moltó, Germán|||0000-0002-8049-253X
  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Alonso Abalos, José Miguel|||0000-0001-6812-7364
  • Romero Alcalde, Eloy
In this study, we describe the further development of Elastic Cloud Computing Cluster (EC3), a tool for creating self-managed cost-efficient virtual hybrid elastic clusters on top of Infrastructure as a Service (IaaS) clouds. By using spot instances and checkpointing techniques, EC3 can significantly reduce the total execution cost as well as facilitating automatic fault tolerance. Moreover, EC3 can deploy and manage hybrid clusters across on-premises and public cloud resources, thereby introducing cloud bursting capabilities. We present the results of a case study that we conducted to assess the effectiveness of the tool based on the structural dynamic analysis of buildings. In addition, we evaluated the checkpointing algorithms in a real cloud environment with existing workloads to study their effectiveness. The results demonstrate the feasibility and benefits of this type of cluster for computationally intensive applications. © 2016 Elsevier B.V. All rights reserved., This study was supported by the program "Ayudas para la contratacion de personal investigador en formacion de caracter pre doctoral, programa VALi+d" under grant number ACIF/2013/003 from the Conselleria d'Educacio of the Generalitat Valenciana. We are also grateful for financial support received from The Spanish Ministry of Economy and Competitiveness to develop the project "CLUVIEM" under grant reference TIN2013-44390-R. Finally, we express our gratitude to D. David Ruzafa for support with the arduous task of analyzing the executions data.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

A platform to deploy customized scientific virtual infrastructures on the cloud

  • Caballer Fernández, Miguel|||0000-0001-9393-3077
  • Segrelles Quilis, José Damián|||0000-0001-5698-7965
  • Moltó, Germán|||0000-0002-8049-253X
  • Blanquer Espert, Ignacio|||0000-0003-1692-8922
This paper presents a software platform to dynamically deploy complex scientific virtual computing infrastructures, on top of Infrastructure as a Service (IaaS) Clouds. The platform orchestrates different services to provision the virtual computing resources. It dynamically installs the appropriate software to satisfy the requirements of a researcher, both on public and on-premise Clouds. The platform provides a web interface to enable the users to easily management of the lifecycle of virtual infrastructures. It enables users to define infrastructures, share them with other users, deploy and relinquish them, add or remove resources dynamically, create and share application recipes, etc. The paper also describes three case studies to deploy complex infrastructures, namely a Hadoop cluster, a single-node to perform NGS sequencing and a gateway for users to access the European Grid Infrastructure (EGI). This platform promotes a better use of on-premise hardware resources of a research center by allocating the computing resources just-in-time to the specific life time of the virtual infrastructures as well as the deployment of the very same infrastructures on a public Cloud., The authors would to thank the Spanish "Ministerio de Economia y Competitividad" for the project "Clusters Virtuales Elasticos y Migrables sobre Infraestructuras Cloud Hibridas" with reference TIN2013-44390-R.