Skip to main content

COLABORACION ROBOT-HUMANO PARA EL TRANSPORTE Y ENTREGA DE MERCANCIAS

PID2019-106702RB-C21

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2019
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 53
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Collaborative-AI: Social robots accompanying and approaching people

  • Sanfeliu, Alberto
  • Repiso, Ely
  • Garrell, Anaís
Trabajo presentado en el International First International Workshop on New Foundations for Human-Centered AI (NeHuAI), celebrado en Santiago de Compostela (España), el 4 de septiembre de 2020, Collaborative AI to approach or to accompany people using social robots will be a fundamental robotics field in the near future. If we desire to share and to collaborate with social robots during the development of our daily task, social robots should be able to develop Collaborative AI task, such us accompanying or approaching people. In this article, we will present the robot-people accompaniment and approaching missions through the four levels of abstractions of Collaborative AI systems and describe the main Collaborative AI functionalities that are needed for these missions. We will also show the system that we have developed for accompany one or two pedestrians and approaching on person by a robot in an urban space, Work supported by the Spanish Ministry of Science project ROCOTRANSP (PID2019-106702RB-C21-RAEI/FEDER EU) by Ministerio de Ciencia e Innovacion , the EU AI4EU project (H2020-ICT-2018-2-825619), by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Ely Repiso is suported by the FPI-grant, BES-2014-067713.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Social robot navigation tasks: Combining machine learning techniques and social force model

  • Gil Viyuela, Óscar
  • Garrell, Anaís
  • Sanfeliu, Alberto
Social robot navigation in public spaces, buildings or private houses is a difficult problem that is not well solved due to environmental constraints (buildings, static objects etc.), pedestrians and other mobile vehicles. Moreover, robots have to move in a human-aware manner—that is, robots have to navigate in such a way that people feel safe and comfortable. In this work, we present two navigation tasks, social robot navigation and robot accompaniment, which combine machine learning techniques with the Social Force Model (SFM) allowing human-aware social navigation. The robots in both approaches use data from different sensors to capture the environment knowledge as well as information from pedestrian motion. The two navigation tasks make use of the SFM, which is a general framework in which human motion behaviors can be expressed through a set of functions depending on the pedestrians’ relative and absolute positions and velocities. Additionally, in both social navigation tasks, the robot’s motion behavior is learned using machine learning techniques: in the first case using supervised deep learning techniques and, in the second case, using Reinforcement Learning (RL). The machine learning techniques are combined with the SFM to create navigation models that behave in a social manner when the robot is navigating in an environment with pedestrians or accompanying a person. The validation of the systems was performed with a large set of simulations and real-life experiments with a new humanoid robot denominated IVO and with an aerial robot. The experiments show that the combination of SFM and machine learning can solve human-aware robot navigation in complex dynamic environments., This research was supported by the grant MDM-2016-0656 funded by MCIN/AEI/ 10.13039/501100011033, the grant ROCOTRANSP PID2019-106702RB-C21 funded by MCIN/AEI/ 10.13039/501100011033 and the grant CANOPIES H2020-ICT-2020-2-101016906 funded by the European Union.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Attention deep learning based model for predicting the 3D human body pose using the robot human handover phases

  • Laplaza Galindo, Javier
  • Pumarola, Albert
  • Moreno-Noguer, Francesc
  • Sanfeliu, Alberto
Trabajo presentado en el International Symposium on Robot and Human Interactive Communication (RO-MAN), celebrado en Vancouver (Canadá), del 8 al 12 de agosto de 2021, This work proposes a human motion prediction model for handover operations. We use in this work, the different phases of the handover operation to improve the human motion predictions. Our attention deep learning based model takes into account the position of the robot’s End Effector (REE) and the phase in the handover operation to predict future human poses. Our model outputs a distribution of possible positions rather than one deterministic position, a key feature in order to allow robots to collaborate with humans. We provide results of the human upper body and the human right hand, also referred as Human End Effector (HEE).The attention deep learning based model has been trained and evaluated with a dataset created using human volunteers and an anthropomorphic robot, simulating handover operations where the robot is the giver and the human the receiver. For each operation, the human skeleton is obtained with an Intel RealSense D435i camera attached inside the robot’s head. The results shown a great improvement of the human’s right hand prediction and 3D body compared with other methods., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT2020-2-101016906)
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A New Methodological Approach to Analyze Human Roles in Human-Robot Interaction Scenarios

  • Puig-Pey, Ana
  • Sanfeliu, Alberto
  • Leroux, Christoph
  • Dario, Paolo
  • Rasso, Rossela
  • Arrue, Begoña C.
  • Soueres, Philip
  • Dailami, Farid
  • Vasco, Valentina
  • Munih, Marko
  • Ijspeert, Auke
  • Roozing, Wesley
Trabajo presentado en el IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO), celebrado en Long Beach, CA (Estados Unidos), del 28 al 30 de mayo de 2022, The influence of robotics in the job market has socio-economic consequences and the inevitable loss of job positions, but in the Human-Robot Interaction (HRI) scenarios, where humans and robots work together, new jobs and business models will be created. These HRI scenarios can be seen in manufacturing, medicine, maintenance, education, etc. and in the future, in most of the daily tasks. The purpose of this article is to analyze from the perspective of Human-Robot Interaction scenarios, which new HRI human roles and HRI tasks categories has to be taken into account. The new HRI human roles will serve to define and homogenize the transverse skills needed for workers working in HRI scenarios and for analyzing the human roles in HRI experiments. We introduce a methodological approach using the TERRINet HRI Template, which facilitates the analysis of tasks that in the near future will be done by humans working with robots. The Template has been used and tested in 20 study cases of HRI scenarios, validating the methodology and looking for homogenization in human roles., Work supported under the European project TERRINet (H2020- INFRAIA-2017-1-two-stage-730994) and the Spanish State Research Agency through the ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033))
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Classification of humans social relations within urban areas

  • Castro Arcusa, Oscar
  • Repiso, Ely
  • Garrell, Anaís
  • Sanfeliu, Alberto
Trabajo presentado en el ROBOT2022: Fifth Iberian Robotics Conference, celebrada en Zaragoza (España), del 20 al 22 de noviembre de 2022, This paper presents the design of deep learning architectures which allow to classify the social relationship existing between two people who are walking in a side-by-side formation into four possible categories --colleagues, couple, family or friendship. The models are developed using Neural Networks or Recurrent Neural Networks to achieve the classification and are trained and evaluated using a database obtained from humans walking together in an urban environment. The best achieved model accomplishes a good accuracy in the classification problem and its results enhance the outcomes from a previous study [1]. In addition, we have developed several models to classify the social interactions in two categories --¿intimate" and "acquaintances", where the best model achieves a very good performance, and for a real robot this classification is enough to be able to customize its behavior to its users. Furthermore, the proposed models show their future potential to improve its efficiency and to be implemented in a real robot., This work has been supported by the Artificial Intelligence for Human–Robot Interaction (AI4HRI) project ANR-20-IADJ-0006. Also, this Work has been supported under the Spanish State Research Agency through the ROCOTRANSP project (PID2019-106702RB-C21/AEI/10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT-2020-2-101016906).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Context and Intention for 3D Human Motion Prediction: Experimentation and User study in Handover Tasks

  • Laplaza Galindo, Javier
  • Garrell, Anaís
  • Moreno-Noguer, Francesc
  • Sanfeliu, Alberto
Trabajo presentado en el 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), celebrado en Nápoles (Italia), del 29 de agosto al 2 de septiembre de 2022, In this work we present a novel attention deep learning model that uses context and human intention for 3D human body motion prediction in handover human-robot tasks. This model uses a multi-head attention architecture which incorporates as inputs the human motion, the robot end effector and the position of the obstacles. The outputs of the model are the predicted motion of the human body and the predicted human intention. We use this model to analyze a handover collaborative task with a robot where the robot is able to predict the future motion of the human and use this information in it¿s planner. Several experiments are performed where human volunteers fill a standard poll to rate different features, taking into account when the robot uses the prediction versus when the robot doesn¿t use the prediction., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT- 2020-2-101016906)
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

The LogiSmile Project - Piloting Autonomous Vehicles for Last-Mile Logistics in European cities

  • Lemardelé, Clément
  • Baldó, Albert
  • Aniculaesei, Adina
  • Rausch, Andreas
  • Conill, Mariona
  • Everding, Lars
  • Vietor, Thomas
  • Hegerhorst, Torben
  • Henze, Roman
  • Mátyus, László
  • Pagès, Laia
  • Rocag, Vidal
  • Sanfeliu, Alberto
  • Santamaria-Navarro, Àngel
  • Tóháti, István
Trabajo presentado en la 15th Conference on Transport Engineering, celebrada en San Cristobal de La Laguna, Tenerife (España), del 14 al 16 de junio de 2023, The use of autonomous technologies for last-mile logistics has the potential to reduce operation costs, cut emissions from the delivery sector, improve safety levels in communities, and provide efficient delivery solutions in areas which experience access regulations. Unfortunately, the implementation of real-life, economically-sustainable, and safe use cases in open urban environments remains scarce, especially in Europe. As a consequence, there is a real need for pilot tests of innovative delivery schemes based on autonomous technologies, not only in numerical simulations, but also in representative urban settings throughout Europe. In this paper, we will present the LogiSmile project, in which a consortium of European partners is piloting a transport management system for autonomous last-mile logistics. Beyond describing the delivery characteristics and use cases of this system, we also highlight the lessons learnt during the course of the project about managing innovation in the urban mobility field, gathering insight form the innovation triple helix (academia, industry and public sector)., The authors would like to acknowledge the participation of all LogiSmile partners in the project and the EIT Urban Mobility for its support. The LogiSmile project was co-funded by the EIT UM under the KAVA 22140. This work was also partially supported by the Spanish Ministry of Science and Innovation under the project AUDEL (TED2021-131759A-I00, funded by MCIN/AEI/10.13039/501100011033 and by the "European Union NextGenerationEU/ PRTR"), the project ROCOTRANSP (PID2019-106702RB-C2 funded by MCIN/ AEI /10.13039/501100011033) and by the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Human acceptance in the Human-Robot Interaction scenario for last-mile goods delivery

  • Puig-Pey, Ana
  • Zamora, J. Ll
  • Amante, B.
  • Moreno, J.
  • Garrell, Anaís
  • Grau Saldes, Antoni
  • Bolea, Yolanda
  • Santamaria, A.
  • Sanfeliu, Alberto
Trabajo presentado en el IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO), celebrado en Berlín (Alemania), del 5 al 7 de junio de 2023, The introduction of robotic technology in an existing scenario must be analyzed from the point of view of all the human roles involved in that scenario. In the case of dealing with urban public space, the analysis must consider a large group of citizens who carry out different activities on it. The purpose of this article is to analyze the human roles and the human acceptance when the robotic technology is introduced in the last mile distribution of goods in urban areas. In this work, we start with the description of the Human-Robot Interaction (HRI) scenario for last mile goods delivery, we describe the human roles and we propose a set of relevant indicators to evaluate the human acceptance for this task. Finally, we evaluate the human acceptance through qualitative interviews and quantitative surveys. The study has been done for the peer end-users and bystanders human roles and around 100 people participated., Work supported under the European project LogiSmile with the grant (EIT-UM-2020-22140) and by the Spanish State Research Agency through the ROCOTRANSP project with the grant (PID2019-106702RB-C21 / AEI / 10.13039/501100011033) and AUDEL project (TED2021-131759A-I00)), Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Improving Human-Robot Interaction Effectiveness in Human-Robot Collaborative Object Transportation Using Force Prediction

  • Domínguez Vidal, José Enrique
  • Sanfeliu, Alberto
Trabajo presentado en la International Conference on Intelligent Robots and Systems (IROS), celebrada en Detroit, MI (Estados Unidos), del 1 al 5 de octubre de 2023, In this work, we analyse the use of a prediction of the human's force in a Human-Robot collaborative object transportation task at a middle distance. We check that this force prediction can improve multiple parameters associated with effective Human-Robot Interaction (HRI) such as perception of the robot's contribution to the task, comfort or trust in the robot in a physical Human Robot Interaction (pHRI). We present a Deep Learning model that allows to predict the force that a human will exert in the next 1 s using as inputs the force previously exerted by the human, the robot's velocity and environment information obtained from the robot's LiDAR. Its success rate is up to 92.3% in testset and up to 89.1 % in real experiments. We demonstrate that this force prediction, in addition to being able to be used directly to detect changes in the human's intention, can be processed to obtain an estimate of the human's desired trajectory. We have validated this approach with a user study involving 18 volunteers., Work supported under the European project CANOPIES (H2020- ICT-2020-2-101016906) and the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 / AEI /10.13039/501100011033) and by JST Moonshot R & D Grant Number JPMJMS2011. The first author acknowledges Spanish FPU grant with ref. FPU19/06582., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Shared Task Representation for Human–Robot Collaborative Navigation: The Collaborative Search Case

  • Dalmasso, Marc
  • Domínguez Vidal, José Enrique
  • Torres-Rodríguez, Iván J.
  • Jiménez, Pablo
  • Garrell, Anaís
  • Sanfeliu, Alberto
Recent research in Human Robot Collaboration (HRC) has spread and specialised in many sub-fields. Many show considerable advances, but the human–robot collaborative navigation (HRCN) field seems to be stuck focusing on implicit collaboration settings, on hypothetical or simulated task allocation problems, on shared autonomy or on having the human as a manager. This work takes a step forward by presenting an end-to-end system capable of handling real-world human–robot collaborative navigation tasks. This system makes use of the Social Reward Sources model (SRS), a knowledge representation to simultaneously tackle task allocation and path planning, proposes a multi-agent Monte Carlo Tree Search (MCTS) planner for human–robot teams, presents the collaborative search as a testbed for HRCN and studies the usage of smartphones for communication in this setting. The detailed experiments prove the viability of the approach, explore collaboration roles adopted by the human–robot team and test the acceptability and utility of different communication interface designs., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and ROCOTRANSP project (PID2019- 106702RB-C21 / AEI / 10.13039/501100011033), the European research grant TERRINet (H2020-INFRAIA-2017-1-730994) and by JST Moonshot R & D Grant Number JPMJMS2011-85., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Adaptive Social Planner to Accompany People in Real-Life Dynamic Environments

  • Repiso, Ely
  • Garrell, Anaís
  • Sanfeliu, Alberto
Robots must develop the ability to socially navigate in uncontrolled urban environments to be able to be included in our daily lives. This paper presents a new robot navigation framework called the adaptive social planner (ASP) and a robotic system, which includes the ASP. Our results and previous work show that the ASP can adapt to different collaborative tasks involving humans and robots, such as independent robot navigation, human-robot accompaniment, a robot approaching people, robot navigation tasks that combine learning techniques, and human-drone interactions. Our approach in this paper focuses on demonstrating how the ASP can be customized to implement two new methods for group accompaniment: the adaptive social planner using a V-formation model to accompany groups of people (ASP-VG) and the adaptive social planner using a side-by-side model to accompany groups of people (ASP-SG). These two methods result in a robot accompanying groups of people by anticipating human and uncontrolled urban environment behaviors. Also, we develop four new robot skills to deal with unexpected human behaviors, such as rearrangement of the position of the companions inside the group, unforeseen changes in the velocity of the robot companions, occlusions among group members, and changes in the direction toward destinations in the environment. Moreover, we develop different performance metrics, based on social distances, to evaluate the tasks of the robot. In addition, we present the guidelines followed in performing the real-life experiments with volunteers, including a human-robot speech interaction to help humans create a relationship with the robot to be genuinely involved in the mutual accompaniment. Finally, we include an exhaustive validation of the methods by evaluating the behavior of the robot through synthetic and real-life experiments. We incorporate five user studies to evaluate aspects related to social acceptability and preferences of people regarding both types of robot group accompaniment., ROCOTRANSP: National Project. Funder: Ministerio de Ciencia e Innovacion (MCIN) y Agencia Española de Investigacion (AEI). Award Number: PID2019-106702RB-C21 MCIN / AEI/10.13039/501100011033. Grant Recipient: Alberto Sanfeliu Cortés (UPC). TERRINET: European Project. Funder: European Commission. Award Number: H2020-INFRAIA-2017-1-two-stage-730994. Grant Recipient: Alberto Sanfeliu Cortés (UPC)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Perception–Intention–Action Cycle in Human–Robot Collaborative Tasks: The Collaborative Lightweight Object Transportation Use-Case

  • Domínguez Vidal, José Enrique
  • Rodríguez Linares, Nicolás Adrián
  • Sanfeliu, Alberto
This study proposes to improve the reliability, robustness and human-like nature of Human–Robot Collaboration (HRC). For that, the classical Perception–Action cycle is extended to a Perception–Intention–Action (PIA) cycle, which includes an Intention stage at the same level as the Perception one, being in charge of obtaining both the implicit and the explicit intention of the human, opposing to classical approaches based on inferring everything from perception. This complete cycle is exposed theoretically including its use of the concept of Situation Awareness, which is shown as a key element for the correct understanding of the current situation and future action prediction. This enables the assignment of roles to the agents involved in a collaborative task and the building of collaborative plans. To visualize the cycle, a collaborative transportation task is used as a use-case. A force-based model is designed to combine the robot’s perception of its environment with the force exerted by the human and other factors in an illustrative way. Finally, a total of 58 volunteers participate in two rounds of experiments. In these, it is shown that the human agrees to explicitly state their intention without undue extra effort and that the human understands that this helps to minimize robot errors or misunderstandings. It is also shown that a system that correctly combines inference with explicit elicitation of the human’s intention is the best rated by the human on multiple parameters related to effective Human–Robot Interaction (HRI), such as perceived safety or trust in the robot., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Work supported under the European project CANOPIES (H2020- ICT-2020-2-101016906), the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 MCIN/ AEI /10.13039/501100011033) and the JST Moonshot R & D Grant No. JPMJMS2011-85. The first author acknowledges Spanish FPU Grant with ref. FPU19/06582., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A Methodological Proposal for the Integration of Robotic Services in the Urban Public Space of Cities, Propuesta metodológica para la integración de servicios robóticos en el espacio público urbano de las ciudades

  • Puig-Pey, Ana
  • Amante, Beatriz
  • Bolea, Yolanda
  • Sanfeliu, Alberto
The integration of urban services with robotic technology in the public space of cities requires an indepth analysis of both the characteristics of the technological system and the existing urban structure. The proposal of this work, based on the methodology of the case study, proposes a process to develop the research, and selects the case study of last mile distribution services with the autonomous roboticplatform ONA. Throughout the process, a theoretical analysis is proposed, to which data, empirical evidence and rounds of discussion between professionals in urban planning, robotics, architecture and engineering, as well as end users and citizens, are added, seeking to reach a consensus on the characteristics and requirements of the technology that must be integrated and the changes that must be made in the urban public space, both in its planning and design. The RUS (Robotic Urban Services) Template, incorporating the characteristics of a specific urban robotic service and the list of urban aspects to consider, is used as a guide of the opportunities and the constraints detected for the integration., Work supported by the research projects: ROCOTRANSP: Robot-human collaboration for goods transportation and delivery, with the Grant PID2019-106702RB-C21- funded by MCIN/ AEI /10.13039/501100011033 and LOGISMILE 2023: Last mile logistics for autonomous goods delivery with the grant EIT-UM-2023-23374. Also, our most sincere thanks to the reviewers of this article, who have contributed to structure and give clarity to the document., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Perception-intention-action cycle as a human acceptable way for improving human-robot collaborative tasks

  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Rodríguez Linares, Nicolás Adrián|||0000-0003-4834-223X
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© ACM 2023. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, https://doi.org/10.1145/3568294.3580149., In Human-Robot Collaboration (HRC) tasks, the classical Perception-Action cycle can not fully explain the collaborative behaviour of the human-robot pair until it is extended to Perception-Intention-Action (PIA) cycle, giving to the human's intention a key role at the same level of the robot's perception and not as a subblock of this. Although part of the human's intention can be perceived or inferred by the other agent, this is prone to misunderstandings so the true intention has to be explicitly informed in some cases to fulfill the task. Here, we explore both types of intention and we combine them with the robot's perception through the concept of Situation Awareness (SA). We validate the PIA cycle and its acceptance by the user with a preliminary experiment in an object transportation task showing that its usage can increase trust in the robot., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Perception–intention–action cycle in human–robot collaborative tasks: the collaborative lightweight object transportation use-case

  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Rodríguez Linares, Nicolás Adrián|||0000-0003-4834-223X
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
This study proposes to improve the reliability, robustness and human-like nature of Human–Robot Collaboration (HRC). For that, the classical Perception–Action cycle is extended to a Perception–Intention–Action (PIA) cycle, which includes an Intention stage at the same level as the Perception one, being in charge of obtaining both the implicit and the explicit intention of the human, opposing to classical approaches based on inferring everything from perception. This complete cycle is exposed theoretically including its use of the concept of Situation Awareness, which is shown as a key element for the correct understanding of the current situation and future action prediction. This enables the assignment of roles to the agents involved in a collaborative task and the building of collaborative plans. To visualize the cycle, a collaborative transportation task is used as a use-case. A force-based model is designed to combine the robot’s perception of its environment with the force exerted by the human and other factors in an illustrative way. Finally, a total of 58 volunteers participate in two rounds of experiments. In these, it is shown that the human agrees to explicitly state their intention without undue extra effort and that the human understands that this helps to minimize robot errors or misunderstandings. It is also shown that a system that correctly combines inference with explicit elicitation of the human’s intention is the best rated by the human on multiple parameters related to effective Human–Robot Interaction (HRI), such as perceived safety or trust in the robot., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Work supported under the European project CANOPIES (H2020- ICT-2020-2-101016906), the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 MCIN/ AEI /10.13039/501100011033) and the JST Moonshot R & D Grant No. JPMJMS2011-85. The first author acknowledges Spanish FPU Grant with ref. FPU19/06582., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Social robot navigation tasks: combining machine learning techniques and social force model

  • Gil Viyuela, Óscar|||0000-0001-8812-3168
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), Social robot navigation in public spaces, buildings or private houses is a difficult problem that is not well solved due to environmental constraints (buildings, static objects etc.), pedestrians and other mobile vehicles. Moreover, robots have to move in a human-aware manner—that is, robots have to navigate in such a way that people feel safe and comfortable. In this work, we present two navigation tasks, social robot navigation and robot accompaniment, which combine machine learning techniques with the Social Force Model (SFM) allowing human-aware social navigation. The robots in both approaches use data from different sensors to capture the environment knowledge as well as information from pedestrian motion. The two navigation tasks make use of the SFM, which is a general framework in which human motion behaviors can be expressed through a set of functions depending on the pedestrians’ relative and absolute positions and velocities. Additionally, in both social navigation tasks, the robot’s motion behavior is learned using machine learning techniques: in the first case using supervised deep learning techniques and, in the second case, using Reinforcement Learning (RL). The machine learning techniques are combined with the SFM to create navigation models that behave in a social manner when the robot is navigating in an environment with pedestrians or accompanying a person. The validation of the systems was performed with a large set of simulations and real-life experiments with a new humanoid robot denominated IVO and with an aerial robot. The experiments show that the combination of SFM and machine learning can solve human-aware robot navigation in complex dynamic environments., This research was supported by the grant MDM-2016-0656 funded by MCIN/AEI / 10.13039/501100011033, the grant ROCOTRANSP PID2019-106702RB-C21 funded by MCIN/AEI/ 10.13039/501100011033 and the grant CANOPIES H2020-ICT-2020-2-101016906 funded by the European Union., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Efficient hand gesture recognition for human-robot interaction

  • Peral Sánchez, Marc|||0000-0001-6521-6476
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
In this paper, we present an efficient and reliable deep-learning approach that allows users to communicate with robots via hand gesture recognition. Contrary to other works which use external devices such as gloves [1] or joysticks [2] to tele-operate robots, the proposed approach uses only visual information to recognize user's instructions that are encoded in a set of pre-defined hand gestures. Particularly, the method consists of two modules which work sequentially to extract 2D landmarks of hands –ie. joints positions– and to predict the hand gesture based on a temporal representation of them. The approach has been validated in a recent state-of-the-art dataset where it outperformed other methods that use multiple pre-processing steps such as optical flow and semantic segmentation. Our method achieves an accuracy of 87.5% and runs at 10 frames per second. Finally, we conducted real-life experiments with our IVO robot to validate the framework during the interaction process., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

The LogiSmile project: piloting autonomous vehicles for lastmile logistics in European cities

  • Lemardelé, Clément|||0000-0003-1673-6509
  • Baldó Canut, Albert
  • Pagès Giralt, Laia|||0000-0002-1608-6326
  • Aniculaesei, Adina
  • Rausch, Andreas
  • Conill de Azpiazu, Maria
  • Everding, Lars
  • Hegerhorst, Torben
  • Henze, Roman
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
The use of autonomous technologies for last-mile logistics has the potential to reduce operation costs, cut emissions from the delivery sector, improve safety levels in communities, and provide efficient delivery solutions in areas which experience access regulations. Unfortunately, the implementation of real-life, economically-sustainable, and safe use cases in open urban environments remains scarce, especially in Europe. As a consequence, there is a real need for pilot tests of innovative delivery schemes based on autonomous technologies, not only in numerical simulations, but also in representative urban settings throughout Europe. In this paper, we will present the LogiSmile project, in which a consortium of European partners is piloting a transport management system for autonomous last-mile logistics. Beyond describing the delivery characteristics and use cases of this system, we also highlight the lessons learnt during the course of the project about managing innovation in the urban mobility field, gathering insight form the innovation triple helix (academia, industry and public sector)., The authors would like to acknowledge the participation of all LogiSmile partners in the project and the EIT Urban Mobility for its support. The LogiSmile project was co-funded by the EIT UM under the KAVA 22140. This work was also partially supported by the Spanish Ministry of Science and Innovation under the project AUDEL (TED2021-131759A-I00, funded by MCIN/AEI/10.13039/501100011033 and by the "European Union NextGenerationEU/ PRTR"), the project ROCOTRANSP (PID2019-106702RB-C2 funded by MCIN/ AEI /10.13039/501100011033) and by the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Improving human-robot interaction effectiveness in human-robot collaborative object transportation using force prediction

  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this work, we analyse the use of a prediction of the human’s force in a Human-Robot collaborative object transportation task at a middle distance. We check that this force prediction can improve multiple parameters associated with effective Human-Robot Interaction (HRI) such as percep- tion of the robot’s contribution to the task, comfort or trust in the robot in a physical Human Robot Interaction (pHRI). We present a Deep Learning model that allows to predict the force that a human will exert in the next 1 s using as inputs the force previously exerted by the human, the robot’s velocity and environment information obtained from the robot’s LiDAR. Its success rate is up to 92.3% in testset and up to 89.1% in real experiments. We demonstrate that this force prediction, in addition to being able to be used directly to detect changes in the human’s intention, can be processed to obtain an estimate of the human’s desired trajectory. We have validated this approach with a user study involving 18 volunteers., Work supported under the European project CANOPIES (H2020 ICT-2020-2-101016906) and the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 / AEI /10.13039/501100011033) and by JST Moonshot R&D Grant Number JPMJMS2011.The first author acknowledges Spanish FPU grant with ref.FPU19/06582., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Human acceptance in the Human-Robot Interaction scenario for last-mile goods delivery

  • Puig-Pey Clavería, Ana María|||0000-0002-6073-7448
  • Zamora i Mestre, Joan-Lluís|||0000-0002-7705-6171
  • Amante García, Beatriz|||0000-0001-5231-1706
  • Moreno Sanz, Joan|||0000-0003-2342-3308
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Grau Saldes, Antoni|||0000-0003-4112-3325
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The introduction of robotic technology in an existing scenario must be analyzed from the point of view of all the human roles involved in that scenario. In the case of dealing with urban public space, the analysis must consider a large group of citizens who carry out different activities on it. The purpose of this article is to analyze the human roles and the human acceptance when the robotic technology is introduced in the last mile distribution of goods in urban areas. In this work, we start with the description of the Human-Robot Interaction (HRI) scenario for last mile goods delivery, we describe the human roles and we propose a set of relevant indicators to evaluate the human acceptance for this task. Finally, we evaluate the human acceptance through qualitative interviews and quantitative surveys. The study has been done for the peer end-users and bystanders human roles and around 100 people participated., Work supported under the European project LogiSmile with the grant (EIT-UM-2020-22140) and by the Spanish State Research Agency through the ROCOTRANSP project with the grant (PID2019-106702RB-C21 / AEI / 10.13039/501100011033) and AUDEL project (TED2021-131759A-I00)), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Attention deep learning based model for predicting the 3D Human Body Pose using the Robot Human Handover Phases

  • Laplaza Galindo, Javier|||0000-0003-4864-5759
  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This work proposes a human motion prediction model for handover operations. We use in this work, the different phases of the handover operation to improve the human motion predictions. Our attention deep learning based model takes into account the position of the robot’s End Effector and the phase in the handover operation to predict future human poses. Our model outputs a distribution of possible positions rather than one deterministic position, a key feature in order to allow robots to collaborate with humans. The attention deep learning based model has been trained and evaluated with a dataset created using human volunteers and an anthropomorphic robot, simulating handover operations where the robot is the giver and the human the receiver. For each operation, the human skeleton is obtained with an Intel RealSense D435i camera attached inside the robot’s head. The results shown a great improvement of the human’s right hand prediction and 3D body compared with other methods., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI /10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT-2020-2-101016906), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

The LogiSmile Project - Piloting autonomous vehicles for last-mile Logistics in european cities

  • Conill de Azpiazu, Maria
  • Vietor, Tim
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Lemardelé, Clément|||0000-0003-1673-6509
  • Baldó Oliva, Albert
  • Aniculaesei, Adina
  • Rausch, Andreas
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
The use of autonomous technologies for last-mile logistics has the potential to reduce operation costs, cut emissions from the delivery sector, improve safety levels in communities, and provide efficient delivery solutions in areas which experience access regulations. Unfortunately, the implementation of real-life, economically-sustainable, and safe use cases in open urban environments remains scarce, especially in Europe. As a consequence, there is a real need for pilot tests of innovative delivery schemes based on autonomous technologies, not only in numerical simulations, but also in representative urban settings throughout Europe. In this paper, we will present the LogiSmile project, in which a consortium of European partners is piloting a transport management system for autonomous last-mile logistics. Beyond describing the delivery characteristics and use cases of this system, we also highlight the lessons learnt during the course of the project about managing innovation in the urban mobility field, gathering insight form the innovation triple helix (academia, industry and public sector)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Classification of humans social relations within urban areas

  • Castro Arcusa, Oscar
  • Repiso Polo, Ely|||0000-0001-6540-8347
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
This paper presents the design of deep learning architectures which allow to classify the social relationship existing between two people who are walking in a side-by-side formation into four possible categories --colleagues, couple, family or friendship. The models are developed using Neural Networks or Recurrent Neural Networks to achieve the classification and are trained and evaluated using a database obtained from humans walking together in an urban environment. The best achieved model accomplishes a good accuracy in the classification problem and its results enhance the outcomes from a previous study [1]. In addition, we have developed several models to classify the social interactions in two categories --“intimate" and "acquaintances", where the best model achieves a very good performance, and for a real robot this classification is enough to be able to customize its behavior to its users. Furthermore, the proposed models show their future potential to improve its efficiency and to be implemented in a real robot., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Collaborative-AI: Social robots accompanying and approaching people

  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Repiso Polo, Ely|||0000-0001-6540-8347
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
Collaborative AI to approach or to accompany people using social robots will be a fundamental robotics field in the near future. If we desire to share and to collaborate with social robots during the development of our daily task, social robots should be able to develop Collaborative AI task, such us accompanying or approaching people. In this article, we will present the robot-people accompaniment and approaching missions through the four levels of abstractions of Collaborative AI systems and describe the main Collaborative AI functionalities that are needed for these missions. We will also show the system that we have developed for accompany one or two pedestrians and approaching on person by a robot in an urban space., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Shared task representation for human–robot collaborative navigation: the collaborative search case

  • Dalmasso Blanch, Marc|||0000-0003-1734-7863
  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Torres Rodriguez, Ivan Jesús
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© The Author(s) 2023, Recent research in Human Robot Collaboration (HRC) has spread and specialised in many sub-fields. Many show considerable advances, but the human–robot collaborative navigation (HRCN) field seems to be stuck focusing on implicit collaboration settings, on hypothetical or simulated task allocation problems, on shared autonomy or on having the human as a manager. This work takes a step forward by presenting an end-to-end system capable of handling real-world human–robot collaborative navigation tasks. This system makes use of the Social Reward Sources model (SRS), a knowledge representation to simultaneously tackle task allocation and path planning, proposes a multi-agent Monte Carlo Tree Search (MCTS) planner for human–robot teams, presents the collaborative search as a testbed for HRCN and studies the usage of smartphones for communication in this setting. The detailed experiments prove the viability of the approach, explore collaboration roles adopted by the human–robot team and test the acceptability and utility of different communication interface designs., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and ROCOTRANSP project (PID2019- 106702RB-C21 / AEI / 10.13039/501100011033), the European research grant TERRINet (H2020-INFRAIA-2017-1-730994) and by JST Moonshot R & D Grant Number JPMJMS2011-85., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Human-robot collaborative multi-agent path planning using Monte Carlo tree search and social reward sources

  • Dalmasso Blanch, Marc|||0000-0003-1734-7863
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The collaboration between humans and robots in an object search task requires the achievement of shared plans obtained from communicating and negotiating. In this work, we assume that the robot computes, as a first step, a multiagent plan for both itself and the human. Then, both plans are submitted to human scrutiny, who either agrees or modifies it forcing the robot to adapt its own restrictions or preferences. This process is repeated along the search task as many times as required by the human. Our planner is based on a decentralized variant of Monte Carlo Tree Search (MCTS), with one robot and one human as agents. Moreover, our algorithm allows the robot and the human to optimize their own actions by maintaining a probability distribution over the plans in a joint action space. The method allows an objective function definition over action sequences, it assumes intermittent communication, it is anytime and suitable for on-line replanning. To test it, we have developed a human-robot communication mobile phone interface. Validation is provided by real-life search experiments of a Parcheesi token in an urban space, including also an acceptability study., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033), TERRINet (H2020-INFRAIA-2017-1-two-stage730994) and AI4EU (H2020-ICT-2018-2-825619), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Robots in waste management

  • Amante García, Beatriz|||0000-0001-5231-1706
  • Puig-Pey Clavería, Ana María|||0000-0002-6073-7448
  • Zamora i Mestre, Joan-Lluís|||0000-0002-7705-6171
  • Moreno Sanz, Joan|||0000-0003-2342-3308
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
There are different strategies for urban waste management. At present, solutions are being sought for example to the noise that the collections cause to the inhabitants in the cities, as well as the environmental impact. The introduction of technology and particularly robots in the last time, open an opportunity to develop different applications for the robots last mile. This work presents a comparative analysis based on existing literature of the environmental and economic impacts of different waste collection and management systems. It explores how a robotic system could be introduced into public spaces. For this purpose, a new deployment protocol for public spaces has been developed, addressing these interactions. Surveys and personalized interviews with pedestrians in the presence of the robot in an urban center were conducted. Based on the collected data, a protocol has been designed, along with a proposed design applicable to urban centers, small towns, or isolated residential areas. Many challenges remain, including a comprehensive environmental analysis of the robotic waste collection system, and calculating the actual costs of implementation, operation, and maintenance over time., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Identification of potential technical conflicts and opportunities arising from the insertion of collaborative transport robots (CTRs) in the near urban environment: a preliminary field study, Identificación de posibles conflictos y oportunidades técnicas derivadas de la inserción de robots de transporte colaborativo (RTCs) en el entorno urbano próximo: un estudio de campo preliminar, Identificació de potencials conflictes tècnics i oportunitats derivades de la inserció de robots de transport col·laboratius (CTR) en l'entorn urbà proper: un estudi de camp preliminar

  • Zamora i Mestre, Joan-Lluís|||0000-0002-7705-6171
  • Grau Saldes, Antoni|||0000-0003-4112-3325
  • Garcia Carrera, David|||0000-0002-5907-2469
The Collaborative Transport Robot (CTR) is the result of the maturation of electric transport technologies combined with artificial intelligence, applied to a vital task in the near urban environment: the retail transport of goods in collaboration with its citizens. These devices are already appearing in some cities in the service of the new economy of digital commerce. It is necessary to consider the possible technical implications, suspecting both conflicts and opportunities, that their insertion in the near urban environment may trigger. This contribution is part of a project carried out in collaboration with a robotics engineering research institute concerned with the functional challenges posed by the introduction of CTRs in the daily mobility of this urban environment. To this end, observations have been made in various towns in the metropolitan area of Barcelona, taking advantage of the pilot field tests with CTRs; the considerations issued have been based on a qualitative methodology and have been ordered according to the twelve main vectors of this technology. The images collected during the observations show that many of the challenges detected will manifest themselves in the future according to determining aspects such as the service management profile of the CTRs, the speed of their implementation, the diligence of the authorities in their regulation or the capacity of these devices to acquire the necessary information in real time for their circulation in the near urban environment.  , El Robot de Transport Col·laboratiu (CTR) és el resultat de la maduració de les tecnologies de transport elèctric combinades amb la intel·ligència artificial, aplicades a una tasca essencial en l’entorn urbà proper: el transport minorista de mercaderies en col·laboració amb la ciutadania. Aquests dispositius ja estan apareixent en algunes ciutats al servei de la nova economia del comerç digital. Cal considerar les possibles implicacions tècniques, tant conflictes com oportunitats, que podria desencadenar la seva inserció en l’entorn urbà proper. Aquesta contribució s’emmarca dins un projecte realitzat en col·laboració amb un institut d’investigació en enginyeria robòtica preocupat pels reptes funcionals que planteja la introducció dels CTRs en la mobilitat quotidiana d’aquest entorn urbà. Per a això, s’han realitzat observacions en diversos municipis de l’àrea metropolitana de Barcelona, aprofitant les proves pilot amb CTRs; les consideracions exposades s’han basat en una metodologia qualitativa i s’han ordenat segons els dotze vectors principals d’aquesta tecnologia. Les imatges recollides durant les observacions mostren que molts dels reptes detectats es manifestaran en el futur en funció d’aspectes determinants com el perfil de gestió del servei dels CTRs, la velocitat de la seva implantació, la diligència de les autoritats en la seva regulació o la capacitat d’aquests dispositius per adquirir en temps real la informació necessària per a la seva circulació en l’entorn urbà proper., El Robot de Transporte Colaborativo (RTC) es el resultado de la maduración de las tecnologías de transporte eléctrico combinadas con la inteligencia artificial, aplicadas a una tarea vital en el entorno urbano de proximidad: el transporte de mercancías al detalle en colaboración con sus ciudadanos. Estos dispositivos ya están apareciendo en algunas ciudades al servicio de la nueva economía del comercio digital. Es necesario considerar las posibles implicaciones técnicas, sospesando tanto conflictos y oportunidades, que su inserción en el espacio urbano de proximidad pueda desencadenar. Esta contribución forma parte de un proyecto realizado en colaboración con un instituto de investigación en ingeniería robótica preocupado por los retos funcionales que plantearía la introducción de los RTC en la movilidad diaria de este entorno urbano. Para ello, se han realizado observaciones en diversas poblaciones del área metropolitana de Barcelona, aprovechando la realización de unas pruebas piloto de campo con RTCs; las consideraciones emitidas se han basado en una metodología cualitativa y se han ordenado de acuerdo con los doce vectores principales de esta tecnología. Las imágenes recogidas durante las observaciones muestran que muchos de los retos detectados se manifestaran en el futuro según aspectos determinantes como el perfil de gestión del servicio de los RTCs, la rapidez de su implantación, la diligencia de las autoridades en su regulación o la capacidad de estos dispositivos para adquirir la información necesaria en tiempo real para su circulación en el entorno urbano de proximidad., This article was written within the framework of the research project: ROCOTRANSP Robothuman collaboration for the transport and delivery of goods, conducted in the period 1 June 2020 to 31 May 2023 at the Institut de Robòtica i Informàtica Industrial, and funded by the Agencia Estatal de Investigación (PID2019-106702RB-C21)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Context and intention for 3D human motion prediction: experimentation and user study in handover tasks

  • Laplaza Galindo, Javier|||0000-0003-4864-5759
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this work we present a novel attention deep learning model that uses context and human intention for 3D human body motion prediction in handover human-robot tasks. This model uses a multi-head attention architecture which incorporates as inputs the human motion, the robot end effector and the position of the obstacles. The outputs of the model are the predicted motion of the human body and the predicted human intention. We use this model to analyze a handover collaborative task with a robot where the robot is able to predict the future motion of the human and use this information in it’s planner. Several experiments are performed where human volunteers fill a standard poll to rate different features, taking into account when the robot uses the prediction versus when the robot doesn’t use the prediction., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT2020-2-101016906)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

User-friendly smartphone interface to share knowledge in human-robot collaborative search tasks

  • Domínguez Vidal, José Enrique|||0000-0002-0397-9248
  • Torres Rodriguez, Ivan Jesús
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Long-distance human-robot collaborative tasks require robust forms of knowledge-sharing among agents in order to optimize the performance of the task. In this paper, we propose to take advantage of the proliferation of mobile phones to use them as a reliable low-cost communication interface, as opposed to the use of specific gadgets or speech and gesture recognition techniques that are highly prone to failure in the presence of noise or occlusions. Our interface is focused on search tasks, and it allows the user to share with the other agents real-time information such as their position, their intention or even what they would like the other agents to do. To test its acceptability, a user study was conducted with 20 volunteers in a human-human scenario. A second round of experiments with other 30 volunteers was conducted to test different ways to encourage user interaction with our interface. Finally, real-life experiments were also conducted with a robot to apply skills learned to the desired scenario. We found a statistically significant improvement in the amount of information exchanged between agents., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656) and ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Context attention: human motion prediction using context information and deep learning attention models

  • Laplaza Galindo, Javier|||0000-0003-4864-5759
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
This work proposes a human motion prediction model for handover operations. The model uses a multi-headed attention architecture to process the human skeleton data together with contextual data from the operation. This contextual data consists on the position of the robot’s End Effector (REE). The model input is a sequence of 5 seconds skeleton position and it outputs the predicted 2.5 future seconds position. We provide results of the human upper body and the human right hand or Human End Effector (HEE). The attention deep learning based model has been trained and evaluated with a dataset created using human volunteers and an anthropomorphic robot, simulating handover operations where the robot is the giver and the human the receiver. For each operation, the human skeleton is obtained using OpenPose with an Intel RealSense D435i camera set inside the robot’s head. The results show a great improvement of the human’s right hand prediction and 3D body compared with other methods., Work supported under the Spanish State Research Agency through the ROCOTRANSP project (PID2019-106702RB-C21/AEI/10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT-2020-2-101016906)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A methodological proposal for the integration of robotic services in the urban public space of cities, Propuesta metodológica para la integración de servicios robóticos en el espacio público urbano de las ciudades, Proposta metodològica per a la integració de serveis robòtics a l'espai públic urbà de les ciutats

  • Puig-Pey Clavería, Ana María|||0000-0002-6073-7448
  • Amante García, Beatriz|||0000-0001-5231-1706
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
The integration of urban services with robotic technology in the public space of cities requires an in-depth analysis of both the characteristics of the technological system and the existing urban structure. The proposal of this work, based on the methodology of the case study, proposes a process to develop the research, and selects the case study of last mile distribution services with the autonomous robotic platform ONA. Throughout the process, a theoretical analysis is proposed, to which data, empirical evidence and rounds of discussion between professionals in urban planning, robotics, architecture and engineering, as well as end users and citizens, are added, seeking to reach a consensus on the characteristics and requirements of the technology that must be integrated and the changes that must be made in the urban public space, both in its planning and design. The RUS (Robotic Urban Services) Template, incorporating the characteristics of a specific urban robotic service and the list of urban aspects to consider, is used as a guide of the opportunities and the constraints detected for the integration.  , La integració dels serveis urbans amb tecnologia robòtica a l’espai públic de les ciutats requereix una anàlisi en profunditat tant de les característiques del sistema tecnològic com de l’estructura urbana existent. La proposta d’aquest treball, basada en la metodologia de l’estudi de cas, planteja un procés per desenvolupar la recerca i selecciona com a cas d’estudi els serveis de distribució d’última milla amb la plataforma robòtica autònoma ONA. Al llarg del procés, es proposa una anàlisi teòrica a la qual s’afegeixen dades, evidència empírica i rondes de discussió entre professionals de la planificació urbana, la robòtica, l’arquitectura i l’enginyeria, així com usuaris finals i ciutadania, amb l’objectiu d’arribar a un consens sobre les característiques i requisits de la tecnologia que s’ha d’integrar i els canvis que caldrà introduir a l’espai públic urbà, tant en la seva planificació com en el seu disseny. La plantilla RUS (Robotic Urban Services), que incorpora les característiques d’un servei robòtic urbà específic i el llistat d’aspectes urbans a tenir en compte, s’utilitza com a guia per identificar les oportunitats i les limitacions detectades per a la seva integració., La integración de servicios urbanos con tecnología robótica en el espacio público de las ciudades requiere de un análisis en profundidad tanto de las características del sistema tecnológico como de la estructura urbana existente. La propuesta de este trabajo, basada en la metodología del caso de estudio, propone un proceso para desarrollar la investigación, y selecciona el caso de estudio de servicios de distribución de última milla con la plataforma robótica autónoma ONA.  A lo largo del proceso, se propone un análisis teórico al que se suman datos, evidencias empíricas y rondas de discusión entre profesionales del urbanismo, robótica, arquitectura e ingeniería, así como usuarios finales y ciudadanos, buscando llegar a un consenso sobre las características y requerimientos de la tecnología que se debe integrar y los cambios que deben realizarse en el espacio público urbano, tanto en el planeamiento como en el diseño del mismo. La Plantilla RUS (Servicios Robóticos Urbanos), que incorpora las características de un servicio robótico urbano específico y la lista de aspectos urbanos a considerar, se utiliza como guía de las oportunidades y las restricciones detectadas para la integración., Work supported by the research projects: ROCOTRANSP: Robot-human collaboration for goods transportation and delivery, with the Grant PID2019-106702RB-C21- funded by MCIN/ AEI /10.13039/501100011033 and LOGISMILE 2023: Last mile logistics for autonomous goods delivery with the grant EIT-UM-2023-23374. Also, our most sincere thanks to the reviewers of this article, who have contributed to structure and give clarity to the document., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Integration of collaborative transport robotics applied to micro-mobility in the local urban environment: proposed recommendations, Integración de la robótica de transporte colaborativa aplicada a la micromovilidad en el entornourbano local: propuesta de recomendaciones, Integració de la robòtica col·laborativa de transport aplicada a la micromobilitat en l’entorn urbà local: recomanacions proposades

  • Moreno Sanz, Joan|||0000-0003-2342-3308
  • Zamora i Mestre, Joan-Lluís|||0000-0002-7705-6171
  • Puig-Pey Clavería, Ana María|||0000-0002-6073-7448
The development of collaborative transport robotics (CTR) is the result of the development of electric transport technologies and artificial intelligence applied to the transport of goods and services in the near urban environment (NUE). In the future, the appropriate integration of these devices in this environment will depend on the implementation models of the services provided by the CTRs, on the promoters associated with their deployment, whether distribution operators or public administrations, and finally, on the spatial integration and social acceptance derived from them. This contribution is part of a project carried out in collaboration with a robotics engineering research institute concerned with the challenges posed by the introduction of CTRs in this urban environment. The objectives of this contribution are, firstly, to establish a contextual framework for the management of collaborative transport robotics in the NUE; and secondly, to develop recommendations for improving the management of collaborative transport robotics in the NUE. The methodology employed is based on a documentary review of the conceptualisation of the system and a subsequent qualitative analysis considering three dimensions: technological, spatial and managerial. The results presented support the recommendation of the need for an integrated management of CTRs that incorporates public and private actors, and highlight both the potential spatial impact and the influence on citizens' behaviour that can be derived from the implementation of this system in the urban environment of proximity., El desenvolupament de la robòtica col·laborativa de transport (RCT) és fruit de l’evolució de les tecnologies de transport elèctric i de la intel·ligència artificial aplicades al transport de mercaderies i serveis en l’entorn urbà proper (EUP). En el futur, la correcta integració d’aquests dispositius en aquest entorn dependrà dels models d’implementació dels serveis proporcionats per les RCT, dels promotors associats al seu desplegament, ja siguin operadors de distribució o administracions públiques, i finalment, de la integració espacial i l’acceptació social que en se’n derivi. Aquesta contribució forma part d’un projecte realitzat en col·laboració amb un institut de recerca en enginyeria robòtica preocupat pels reptes que suposa la introducció de les RCT en aquest entorn urbà. Els objectius d’aquesta aportació són, en primer lloc, establir un marc contextual per a la gestió de la robòtica col·laborativa de transport en l’EUP; i en segon lloc, desenvolupar recomanacions per a la millora de la gestió de la robòtica col·laborativa de transport en l’EUP. La metodologia emprada es basa en una revisió documental de la conceptualització del sistema i en una anàlisi qualitativa posterior que considera tres dimensions: tecnològica, espacial i gestora. Els resultats presentats avalen la recomanació de la necessitat d’una gestió integrada de les RCT que incorpori actors públics i privats, i destaquen tant l’impacte espacial potencial com la influència en el comportament ciutadà que pot derivar-se de la implementació d’aquest sistema en l’entorn urbà de proximitat., El desarrollo de la robótica de transporte colaborativo (CTR) es el resultado de la puesta a punto de las tecnologías del transporte eléctrico y de la inteligencia artificial, aplicadas al transporte de bienes y servicios en el entorno urbano de proximidad (NUE). En un futuro, la integración adecuada de estos dispositivos en este entorno dependerá de los modelos de implantación de los servicios prestado por los CTR, de los promotores asociados a su despliegue, ya sean operadores de distribución o administraciones públicas y, finalmente, de la integración espacial y la aceptación social derivadas. Esta contribución forma parte de un proyecto realizado en colaboración con un instituto de investigación en ingeniería robótica preocupado por los retos que plantearía la introducción de los CTR en este entorno urbano. Los objetivos de esta contribución son, en primer lugar, establecer un marco contextual para la gestión de la robótica colaborativa del transporte en el NUE; y en segundo lugar, elaborar unas recomendaciones para mejorar dicha gestión. La metodología empleada se basa en una revisión documental realizada sobre la conceptualización del sistema y un análisis cualitativo posterior considerando tres dimensiones: tecnológica, espacial y de gestión. Los resultados presentados sostienen la recomendación de la necesidad de una gestión integral de los CTR que incorpore a agentes públicos y privados, y destacan tanto el impacto espacial potencial como su influencia en el comportamiento de los ciudadanos que se pueden derivar de la implantación de este sistema en el entorno urbano de proximidad., This article has been written within the framework of the research project: ROCOTRANSP Robot-human collaboration for the transport and delivery of goods developed in the period 1 June 2020 to 31 May 2023, at the Institut de Robòtica i Informàtica Industrial, and funded by the Agencia Estatal de Investigación (PID2019-106702RB-C21)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Adaptive social planner to accompany people in real-life dynamic environments

  • Repiso Polo, Ely|||0000-0001-6540-8347
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
Robots must develop the ability to socially navigate in uncontrolled urban environments to be able to be included in our daily lives. This paper presents a new robot navigation framework called the adaptive social planner (ASP) and a robotic system, which includes the ASP. Our results and previous work show that the ASP can adapt to different collaborative tasks involving humans and robots, such as independent robot navigation, human-robot accompaniment, a robot approaching people, robot navigation tasks that combine learning techniques, and human-drone interactions. Our approach in this paper focuses on demonstrating how the ASP can be customized to implement two new methods for group accompaniment: the adaptive social planner using a V-formation model to accompany groups of people (ASP-VG) and the adaptive social planner using a side-by-side model to accompany groups of people (ASP-SG). These two methods result in a robot accompanying groups of people by anticipating human and uncontrolled urban environment behaviors. Also, we develop four new robot skills to deal with unexpected human behaviors, such as rearrangement of the position of the companions inside the group, unforeseen changes in the velocity of the robot companions, occlusions among group members, and changes in the direction toward destinations in the environment. Moreover, we develop different performance metrics, based on social distances, to evaluate the tasks of the robot. In addition, we present the guidelines followed in performing the real-life experiments with volunteers, including a human-robot speech interaction to help humans create a relationship with the robot to be genuinely involved in the mutual accompaniment. Finally, we include an exhaustive validation of the methods by evaluating the behavior of the robot through synthetic and real-life experiments. We incorporate five user studies to evaluate aspects related to social acceptability and preferences of people regarding both types of robot group accompaniment., ROCOTRANSP: National Project. Funder: Ministerio de Ciencia e Innovacion (MCIN) y Agencia Española de Investigacion (AEI). Award Number: PID2019-106702RB-C21 MCIN / AEI/10.13039/501100011033. Grant Recipient: Alberto Sanfeliu Cortés (UPC). TERRINET: European Project. Funder: European Commission. Award Number: H2020-INFRAIA-2017-1-two-stage-730994. Grant Recipient: Alberto Sanfeliu Cortés (UPC)., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Collaborative-AI: Social robots accompanying and approaching people

  • Sanfeliu Cortés, Alberto
  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
Collaborative AI to approach or to accompany people using social robots will be a fundamental robotics field in the near future. If we desire to share and to collaborate with social robots during the development of our daily task, social robots should be able to develop Collaborative AI task, such us accompanying or approaching people. In this article, we will present the robot-people accompaniment and approaching missions through the four levels of abstractions of Collaborative AI systems and describe the main Collaborative AI functionalities that are needed for these missions. We will also show the system that we have developed for accompany one or two pedestrians and approaching on person by a robot in an urban space., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Adaptive social planner to accompany people in real-life dynamic environments

  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
Robots must develop the ability to socially navigate in uncontrolled urban environments to be able to be included in our daily lives. This paper presents a new robot navigation framework called the adaptive social planner (ASP) and a robotic system, which includes the ASP. Our results and previous work show that the ASP can adapt to different collaborative tasks involving humans and robots, such as independent robot navigation, human-robot accompaniment, a robot approaching people, robot navigation tasks that combine learning techniques, and human-drone interactions. Our approach in this paper focuses on demonstrating how the ASP can be customized to implement two new methods for group accompaniment: the adaptive social planner using a V-formation model to accompany groups of people (ASP-VG) and the adaptive social planner using a side-by-side model to accompany groups of people (ASP-SG). These two methods result in a robot accompanying groups of people by anticipating human and uncontrolled urban environment behaviors. Also, we develop four new robot skills to deal with unexpected human behaviors, such as rearrangement of the position of the companions inside the group, unforeseen changes in the velocity of the robot companions, occlusions among group members, and changes in the direction toward destinations in the environment. Moreover, we develop different performance metrics, based on social distances, to evaluate the tasks of the robot. In addition, we present the guidelines followed in performing the real-life experiments with volunteers, including a human-robot speech interaction to help humans create a relationship with the robot to be genuinely involved in the mutual accompaniment. Finally, we include an exhaustive validation of the methods by evaluating the behavior of the robot through synthetic and real-life experiments. We incorporate five user studies to evaluate aspects related to social acceptability and preferences of people regarding both types of robot group accompaniment., ROCOTRANSP: National Project. Funder: Ministerio de Ciencia e Innovacion (MCIN) y Agencia Española de Investigacion (AEI). Award Number: PID2019-106702RB-C21 MCIN / AEI/10.13039/501100011033. Grant Recipient: Alberto Sanfeliu Cortés (UPC). TERRINET: European Project. Funder: European Commission. Award Number: H2020-INFRAIA-2017-1-two-stage-730994. Grant Recipient: Alberto Sanfeliu Cortés (UPC)., Peer Reviewed, Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Efficient hand gesture recognition for human-robot interaction

  • Peral Sánchez, Marc
  • Sanfeliu Cortés, Alberto
  • Garrell Zulueta, Anais
In this paper, we present an efficient and reliable deep-learning approach that allows users to communicate with robots via hand gesture recognition. Contrary to other works which use external devices such as gloves [1] or joysticks [2] to tele-operate robots, the proposed approach uses only visual information to recognize user's instructions that are encoded in a set of pre-defined hand gestures. Particularly, the method consists of two modules which work sequentially to extract 2D landmarks of hands –ie. joints positions– and to predict the hand gesture based on a temporal representation of them. The approach has been validated in a recent state-of-the-art dataset where it outperformed other methods that use multiple pre-processing steps such as optical flow and semantic segmentation. Our method achieves an accuracy of 87.5% and runs at 10 frames per second. Finally, we conducted real-life experiments with our IVO robot to validate the framework during the interaction process., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Human-robot collaborative multi-agent path planning using Monte Carlo tree search and social reward sources

  • Dalmasso Blanch, Marc
  • Garrell Zulueta, Anais
  • Domínguez Vidal, José Enrique
  • Jiménez Schlegl, Pablo
  • Sanfeliu Cortés, Alberto
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The collaboration between humans and robots in an object search task requires the achievement of shared plans obtained from communicating and negotiating. In this work, we assume that the robot computes, as a first step, a multiagent plan for both itself and the human. Then, both plans are submitted to human scrutiny, who either agrees or modifies it forcing the robot to adapt its own restrictions or preferences. This process is repeated along the search task as many times as required by the human. Our planner is based on a decentralized variant of Monte Carlo Tree Search (MCTS), with one robot and one human as agents. Moreover, our algorithm allows the robot and the human to optimize their own actions by maintaining a probability distribution over the plans in a joint action space. The method allows an objective function definition over action sequences, it assumes intermittent communication, it is anytime and suitable for on-line replanning. To test it, we have developed a human-robot communication mobile phone interface. Validation is provided by real-life search experiments of a Parcheesi token in an urban space, including also an acceptability study., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033), TERRINet (H2020-INFRAIA-2017-1-two-stage730994) and AI4EU (H2020-ICT-2018-2-825619), Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Context attention: human motion prediction using context information and deep learning attention models

  • Laplaza Galindo, Javier
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
This work proposes a human motion prediction model for handover operations. The model uses a multi-headed attention architecture to process the human skeleton data together with contextual data from the operation. This contextual data consists on the position of the robot’s End Effector (REE). The model input is a sequence of 5 seconds skeleton position and it outputs the predicted 2.5 future seconds position. We provide results of the human upper body and the human right hand or Human End Effector (HEE). The attention deep learning based model has been trained and evaluated with a dataset created using human volunteers and an anthropomorphic robot, simulating handover operations where the robot is the giver and the human the receiver. For each operation, the human skeleton is obtained using OpenPose with an Intel RealSense D435i camera set inside the robot’s head. The results show a great improvement of the human’s right hand prediction and 3D body compared with other methods., Work supported under the Spanish State Research Agency through the ROCOTRANSP project (PID2019-106702RB-C21/AEI/10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT-2020-2-101016906)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Context and intention for 3D human motion prediction: experimentation and user study in handover tasks

  • Laplaza Galindo, Javier
  • Garrell Zulueta, Anais
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this work we present a novel attention deep learning model that uses context and human intention for 3D human body motion prediction in handover human-robot tasks. This model uses a multi-head attention architecture which incorporates as inputs the human motion, the robot end effector and the position of the obstacles. The outputs of the model are the predicted motion of the human body and the predicted human intention. We use this model to analyze a handover collaborative task with a robot where the robot is able to predict the future motion of the human and use this information in it’s planner. Several experiments are performed where human volunteers fill a standard poll to rate different features, taking into account when the robot uses the prediction versus when the robot doesn’t use the prediction., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT2020-2-101016906)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Improving human-robot interaction effectiveness in human-robot collaborative object transportation using force prediction

  • Domínguez Vidal, José Enrique
  • Sanfeliu Cortés, Alberto
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this work, we analyse the use of a prediction of the human’s force in a Human-Robot collaborative object transportation task at a middle distance. We check that this force prediction can improve multiple parameters associated with effective Human-Robot Interaction (HRI) such as percep- tion of the robot’s contribution to the task, comfort or trust in the robot in a physical Human Robot Interaction (pHRI). We present a Deep Learning model that allows to predict the force that a human will exert in the next 1 s using as inputs the force previously exerted by the human, the robot’s velocity and environment information obtained from the robot’s LiDAR. Its success rate is up to 92.3% in testset and up to 89.1% in real experiments. We demonstrate that this force prediction, in addition to being able to be used directly to detect changes in the human’s intention, can be processed to obtain an estimate of the human’s desired trajectory. We have validated this approach with a user study involving 18 volunteers., Work supported under the European project CANOPIES (H2020 ICT-2020-2-101016906) and the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 / AEI /10.13039/501100011033) and by JST Moonshot R&D Grant Number JPMJMS2011.The first author acknowledges Spanish FPU grant with ref.FPU19/06582., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Shared task representation for human–robot collaborative navigation: the collaborative search case

  • Dalmasso Blanch, Marc
  • Domínguez Vidal, José Enrique
  • Torres Rodriguez, Ivan Jesús
  • Jiménez Schlegl, Pablo
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© The Author(s) 2023, Recent research in Human Robot Collaboration (HRC) has spread and specialised in many sub-fields. Many show considerable advances, but the human–robot collaborative navigation (HRCN) field seems to be stuck focusing on implicit collaboration settings, on hypothetical or simulated task allocation problems, on shared autonomy or on having the human as a manager. This work takes a step forward by presenting an end-to-end system capable of handling real-world human–robot collaborative navigation tasks. This system makes use of the Social Reward Sources model (SRS), a knowledge representation to simultaneously tackle task allocation and path planning, proposes a multi-agent Monte Carlo Tree Search (MCTS) planner for human–robot teams, presents the collaborative search as a testbed for HRCN and studies the usage of smartphones for communication in this setting. The detailed experiments prove the viability of the approach, explore collaboration roles adopted by the human–robot team and test the acceptability and utility of different communication interface designs., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and ROCOTRANSP project (PID2019- 106702RB-C21 / AEI / 10.13039/501100011033), the European research grant TERRINet (H2020-INFRAIA-2017-1-730994) and by JST Moonshot R & D Grant Number JPMJMS2011-85., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Classification of humans social relations within urban areas

  • Castro Arcusa, Oscar
  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
This paper presents the design of deep learning architectures which allow to classify the social relationship existing between two people who are walking in a side-by-side formation into four possible categories --colleagues, couple, family or friendship. The models are developed using Neural Networks or Recurrent Neural Networks to achieve the classification and are trained and evaluated using a database obtained from humans walking together in an urban environment. The best achieved model accomplishes a good accuracy in the classification problem and its results enhance the outcomes from a previous study [1]. In addition, we have developed several models to classify the social interactions in two categories --“intimate" and "acquaintances", where the best model achieves a very good performance, and for a real robot this classification is enough to be able to customize its behavior to its users. Furthermore, the proposed models show their future potential to improve its efficiency and to be implemented in a real robot., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Human acceptance in the Human-Robot Interaction scenario for last-mile goods delivery

  • Puig-Pey Clavería, Ana María
  • Zamora i Mestre, Joan-Lluís
  • Amante García, Beatriz
  • Moreno Sanz, Joan
  • Garrell Zulueta, Anais
  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Santamaria Navarro, Àngel
  • Sanfeliu Cortés, Alberto
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The introduction of robotic technology in an existing scenario must be analyzed from the point of view of all the human roles involved in that scenario. In the case of dealing with urban public space, the analysis must consider a large group of citizens who carry out different activities on it. The purpose of this article is to analyze the human roles and the human acceptance when the robotic technology is introduced in the last mile distribution of goods in urban areas. In this work, we start with the description of the Human-Robot Interaction (HRI) scenario for last mile goods delivery, we describe the human roles and we propose a set of relevant indicators to evaluate the human acceptance for this task. Finally, we evaluate the human acceptance through qualitative interviews and quantitative surveys. The study has been done for the peer end-users and bystanders human roles and around 100 people participated., Work supported under the European project LogiSmile with the grant (EIT-UM-2020-22140) and by the Spanish State Research Agency through the ROCOTRANSP project with the grant (PID2019-106702RB-C21 / AEI / 10.13039/501100011033) and AUDEL project (TED2021-131759A-I00)), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A methodological proposal for the integration of robotic services in the urban public space of cities

  • Puig-Pey Clavería, Ana María
  • Amante García, Beatriz
  • Bolea Monte, Yolanda
  • Sanfeliu Cortés, Alberto
The integration of urban services with robotic technology in the public space of cities requires an indepth analysis of both the characteristics of the technological system and the existing urban structure. The proposal of this work, based on the methodology of the case study, proposes a process to develop the research, and selects the case study of last mile distribution services with the autonomous robotic platform ONA. Throughout the process, a theoretical analysis is proposed, to which data, empirical evidence and rounds of discussion between professionals in urban planning, robotics, architecture and engineering, as well as end users and citizens, are added, seeking to reach a consensus on the characteristics and requirements of the technology that must be integrated and the changes that must be made in the urban public space, both in its planning and design. The RUS (Robotic Urban Services) Template, incorporating the characteristics of a specific urban robotic service and the list of urban aspects to consider, is used as a guide of the opportunities and the constraints detected for the integration., Work supported by the research projects: ROCOTRANSP: Robot-human collaboration for goods transportation and delivery, with the Grant PID2019-106702RB-C21- funded by MCIN/ AEI /10.13039/501100011033 and LOGISMILE 2023: Last mile logistics for autonomous goods delivery with the grant EIT-UM-2023-23374. Also, our most sincere thanks to the reviewers of this article, who have contributed to structure and give clarity to the document., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The LogiSmile project: piloting autonomous vehicles for lastmile logistics in European cities

  • Lemardelé, Clément
  • Baldó Canut, Albert
  • Pagès Giralt, Laia
  • Aniculaesei, Adina
  • Rausch, Andreas
  • Conill de Azpiazu, Maria
  • Everding, Lars
  • Hegerhorst, Torben
  • Henze, Roman
  • Sanfeliu Cortés, Alberto
  • Santamaria Navarro, Àngel
The use of autonomous technologies for last-mile logistics has the potential to reduce operation costs, cut emissions from the delivery sector, improve safety levels in communities, and provide efficient delivery solutions in areas which experience access regulations. Unfortunately, the implementation of real-life, economically-sustainable, and safe use cases in open urban environments remains scarce, especially in Europe. As a consequence, there is a real need for pilot tests of innovative delivery schemes based on autonomous technologies, not only in numerical simulations, but also in representative urban settings throughout Europe. In this paper, we will present the LogiSmile project, in which a consortium of European partners is piloting a transport management system for autonomous last-mile logistics. Beyond describing the delivery characteristics and use cases of this system, we also highlight the lessons learnt during the course of the project about managing innovation in the urban mobility field, gathering insight form the innovation triple helix (academia, industry and public sector)., The authors would like to acknowledge the participation of all LogiSmile partners in the project and the EIT Urban Mobility for its support. The LogiSmile project was co-funded by the EIT UM under the KAVA 22140. This work was also partially supported by the Spanish Ministry of Science and Innovation under the project AUDEL (TED2021-131759A-I00, funded by MCIN/AEI/10.13039/501100011033 and by the "European Union NextGenerationEU/ PRTR"), the project ROCOTRANSP (PID2019-106702RB-C2 funded by MCIN/ AEI /10.13039/501100011033) and by the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The LogiSmile Project - Piloting autonomous vehicles for last-mile Logistics in european cities

  • Conill de Azpiazu, Maria
  • Vietor, Tim
  • Sanfeliu Cortés, Alberto
  • Lemardelé, Clément
  • Baldó Oliva, Albert
  • Aniculaesei, Adina
  • Rausch, Andreas
  • Santamaria Navarro, Àngel
The use of autonomous technologies for last-mile logistics has the potential to reduce operation costs, cut emissions from the delivery sector, improve safety levels in communities, and provide efficient delivery solutions in areas which experience access regulations. Unfortunately, the implementation of real-life, economically-sustainable, and safe use cases in open urban environments remains scarce, especially in Europe. As a consequence, there is a real need for pilot tests of innovative delivery schemes based on autonomous technologies, not only in numerical simulations, but also in representative urban settings throughout Europe. In this paper, we will present the LogiSmile project, in which a consortium of European partners is piloting a transport management system for autonomous last-mile logistics. Beyond describing the delivery characteristics and use cases of this system, we also highlight the lessons learnt during the course of the project about managing innovation in the urban mobility field, gathering insight form the innovation triple helix (academia, industry and public sector)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

User-friendly smartphone interface to share knowledge in human-robot collaborative search tasks

  • Domínguez Vidal, José Enrique
  • Torres Rodriguez, Ivan Jesús
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Long-distance human-robot collaborative tasks require robust forms of knowledge-sharing among agents in order to optimize the performance of the task. In this paper, we propose to take advantage of the proliferation of mobile phones to use them as a reliable low-cost communication interface, as opposed to the use of specific gadgets or speech and gesture recognition techniques that are highly prone to failure in the presence of noise or occlusions. Our interface is focused on search tasks, and it allows the user to share with the other agents real-time information such as their position, their intention or even what they would like the other agents to do. To test its acceptability, a user study was conducted with 20 volunteers in a human-human scenario. A second round of experiments with other 30 volunteers was conducted to test different ways to encourage user interaction with our interface. Finally, real-life experiments were also conducted with a robot to apply skills learned to the desired scenario. We found a statistically significant improvement in the amount of information exchanged between agents., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016- 0656) and ROCOTRANSP project (PID2019-106702RB-C21 / AEI / 10.13039/501100011033)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Social robot navigation tasks: combining machine learning techniques and social force model

  • Gil Viyuela, Óscar
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), Social robot navigation in public spaces, buildings or private houses is a difficult problem that is not well solved due to environmental constraints (buildings, static objects etc.), pedestrians and other mobile vehicles. Moreover, robots have to move in a human-aware manner—that is, robots have to navigate in such a way that people feel safe and comfortable. In this work, we present two navigation tasks, social robot navigation and robot accompaniment, which combine machine learning techniques with the Social Force Model (SFM) allowing human-aware social navigation. The robots in both approaches use data from different sensors to capture the environment knowledge as well as information from pedestrian motion. The two navigation tasks make use of the SFM, which is a general framework in which human motion behaviors can be expressed through a set of functions depending on the pedestrians’ relative and absolute positions and velocities. Additionally, in both social navigation tasks, the robot’s motion behavior is learned using machine learning techniques: in the first case using supervised deep learning techniques and, in the second case, using Reinforcement Learning (RL). The machine learning techniques are combined with the SFM to create navigation models that behave in a social manner when the robot is navigating in an environment with pedestrians or accompanying a person. The validation of the systems was performed with a large set of simulations and real-life experiments with a new humanoid robot denominated IVO and with an aerial robot. The experiments show that the combination of SFM and machine learning can solve human-aware robot navigation in complex dynamic environments., This research was supported by the grant MDM-2016-0656 funded by MCIN/AEI / 10.13039/501100011033, the grant ROCOTRANSP PID2019-106702RB-C21 funded by MCIN/AEI/ 10.13039/501100011033 and the grant CANOPIES H2020-ICT-2020-2-101016906 funded by the European Union., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Robots in waste management

  • Amante García, Beatriz
  • Puig-Pey Clavería, Ana María
  • Zamora i Mestre, Joan-Lluís
  • Moreno Sanz, Joan
  • Sanfeliu Cortés, Alberto
There are different strategies for urban waste management. At present, solutions are being sought for example to the noise that the collections cause to the inhabitants in the cities, as well as the environmental impact. The introduction of technology and particularly robots in the last time, open an opportunity to develop different applications for the robots last mile. This work presents a comparative analysis based on existing literature of the environmental and economic impacts of different waste collection and management systems. It explores how a robotic system could be introduced into public spaces. For this purpose, a new deployment protocol for public spaces has been developed, addressing these interactions. Surveys and personalized interviews with pedestrians in the presence of the robot in an urban center were conducted. Based on the collected data, a protocol has been designed, along with a proposed design applicable to urban centers, small towns, or isolated residential areas. Many challenges remain, including a comprehensive environmental analysis of the robotic waste collection system, and calculating the actual costs of implementation, operation, and maintenance over time., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Perception-intention-action cycle as a human acceptable way for improving human-robot collaborative tasks

  • Domínguez Vidal, José Enrique
  • Rodríguez Linares, Nicolás Adrián
  • Sanfeliu Cortés, Alberto
© ACM 2023. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, https://doi.org/10.1145/3568294.3580149., In Human-Robot Collaboration (HRC) tasks, the classical Perception-Action cycle can not fully explain the collaborative behaviour of the human-robot pair until it is extended to Perception-Intention-Action (PIA) cycle, giving to the human's intention a key role at the same level of the robot's perception and not as a subblock of this. Although part of the human's intention can be perceived or inferred by the other agent, this is prone to misunderstandings so the true intention has to be explicitly informed in some cases to fulfill the task. Here, we explore both types of intention and we combine them with the robot's perception through the concept of Situation Awareness (SA). We validate the PIA cycle and its acceptance by the user with a preliminary experiment in an object transportation task showing that its usage can increase trust in the robot., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Attention deep learning based model for predicting the 3D Human Body Pose using the Robot Human Handover Phases

  • Laplaza Galindo, Javier
  • Pumarola Peris, Albert
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This work proposes a human motion prediction model for handover operations. We use in this work, the different phases of the handover operation to improve the human motion predictions. Our attention deep learning based model takes into account the position of the robot’s End Effector and the phase in the handover operation to predict future human poses. Our model outputs a distribution of possible positions rather than one deterministic position, a key feature in order to allow robots to collaborate with humans. The attention deep learning based model has been trained and evaluated with a dataset created using human volunteers and an anthropomorphic robot, simulating handover operations where the robot is the giver and the human the receiver. For each operation, the human skeleton is obtained with an Intel RealSense D435i camera attached inside the robot’s head. The results shown a great improvement of the human’s right hand prediction and 3D body compared with other methods., Work supported under the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656), the ROCOTRANSP project (PID2019-106702RB-C21 / AEI /10.13039/501100011033)) and the EU project CANOPIES (H2020- ICT-2020-2-101016906), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Perception–intention–action cycle in human–robot collaborative tasks: the collaborative lightweight object transportation use-case

  • Domínguez Vidal, José Enrique
  • Rodríguez Linares, Nicolás Adrián
  • Sanfeliu Cortés, Alberto
This study proposes to improve the reliability, robustness and human-like nature of Human–Robot Collaboration (HRC). For that, the classical Perception–Action cycle is extended to a Perception–Intention–Action (PIA) cycle, which includes an Intention stage at the same level as the Perception one, being in charge of obtaining both the implicit and the explicit intention of the human, opposing to classical approaches based on inferring everything from perception. This complete cycle is exposed theoretically including its use of the concept of Situation Awareness, which is shown as a key element for the correct understanding of the current situation and future action prediction. This enables the assignment of roles to the agents involved in a collaborative task and the building of collaborative plans. To visualize the cycle, a collaborative transportation task is used as a use-case. A force-based model is designed to combine the robot’s perception of its environment with the force exerted by the human and other factors in an illustrative way. Finally, a total of 58 volunteers participate in two rounds of experiments. In these, it is shown that the human agrees to explicitly state their intention without undue extra effort and that the human understands that this helps to minimize robot errors or misunderstandings. It is also shown that a system that correctly combines inference with explicit elicitation of the human’s intention is the best rated by the human on multiple parameters related to effective Human–Robot Interaction (HRI), such as perceived safety or trust in the robot., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Work supported under the European project CANOPIES (H2020- ICT-2020-2-101016906), the MINECO/AEI ROCOTRANSP project (PID2019-106702RB-C21 MCIN/ AEI /10.13039/501100011033) and the JST Moonshot R & D Grant No. JPMJMS2011-85. The first author acknowledges Spanish FPU Grant with ref. FPU19/06582., Peer Reviewed, Postprint (published version)