Pasar al contenido principal

COLABORACION ROBOTS-HUMANOS PARA EL TRANSPORTE DE PRODUCTOS EN ZONAS URBANAS

DPI2016-78957-R

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2016
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 28
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Joint coarse-and-fine reasoning for deep optical flow

  • Vaquero, Victor
  • Ros, German
  • Moreno-Noguer, Francesc
  • López, Antonio M.
  • Sanfeliu, Alberto
Trabajo presentado a la IEEE International Conference on Image Processing (ICIP), celebrada en Beijing (China) del 17 al 20 de septiembre de 2017., We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets., This work was partially supported by European AEROARMS project (H2020-ICT-2014-1-644271) and CICYT projects ColRobTransp (DPI2016-78957-R), ROBINSTRUCT (TIN2014-58178-R)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Low resolution lidar-based multi object tracking for driving applications

  • Pino, Iván del
  • Vaquero, Victor
  • Masini, Beatrice
  • Solà, Joan
  • Moreno-Noguer, Francesc
  • Sanfeliu, Alberto
  • Andrade-Cetto, Juan
Trabajo presentado a la Third Iberian Robotics Conference, celebrada en Sevilla del 22 al 24 de noviembre de 2017., Vehicle detection and tracking in real scenarios are key com- ponents to develop assisted and autonomous driving systems. Lidar sen- sors are specially suitable for this task, as they bring robustness to harsh weather conditions while providing accurate spatial information. How- ever, the resolution provided by point cloud data is very scarce in com- parison to camera images. In this work we explore the possibilities of Deep Learning (DL) methodologies applied to low resolution 3D lidar sensors such as the Velodyne VLP-16 (PUCK), in the context of vehicle detection and tracking. For this purpose we developed a lidar-based sys- tem that uses a Convolutional Neural Network (CNN), to perform point- wise vehicle detection using PUCK data, and Multi-Hypothesis Extended Kalman Filters (MH-EKF), to estimate the actual position and veloci- ties of the detected vehicles. Comparative studies between the proposed lower resolution (VLP-16) tracking system and a high-end system, using Velodyne HDL-64, were carried out on the Kitti Tracking Benchmark dataset. Moreover, to analyze the influence of the CNN-based vehicle detection approach, comparisons were also performed with respect to the geometric-only detector. The results demonstrate that the proposed low resolution Deep Learning architecture is able to successfully accom- plish the vehicle detection task, outperforming the geometric baseline approach. Moreover, it has been observed that our system achieves a similar tracking performance to the high-end HDL-64 sensor at close range. On the other hand, at long range, detection is limited to half the distance of the higher-end sensor., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects ROBINSTRUCT (TIN2014-58178-R) and COLROBTRANSP (DPI2016-78957-R), by the Spanish Ministry of Education FPU grant (FPU15/04446), the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and by the EU H2020 project LOGIMATIC (H2020-Galileo-2015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Deconvolutional networks for point-cloud vehicle detection and tracking in driving scenarios

  • Vaquero, Victor
  • Pino, Iván del
  • Moreno-Noguer, Francesc
  • Solà, Joan
  • Sanfeliu, Alberto
  • Andrade-Cetto, Juan
Trabajo presentado a la European Conference on Mobile Robots (ECMR), celebrada en Paris (Francia) del 6 al 8 de septiembre de 2017., Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects ROBINSTRUCT (TIN2014-58178-R) and COLROBTRANSP (DPI2016-78957-R), by the Spanish Ministry of Education FPU grant (FPU15/04446), the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI MDM-2016-0656) and by the EU H2020 project LOGIMATIC (H2020-Galileo-015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Deep lidar CNN to understand the dynamics of moving vehicles

  • Vaquero, Victor
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la IEEE International Conference on Robotics and Automation (ICRA), celebrado en Brisbane (Australia), del 21 al 25 de mayo de 2018, Perception technologies in Autonomous Driving are experiencing their golden age due to the advances in Deep Learning. Yet, most of these systems rely on the semantically rich information of RGB images. Deep Learning solutions applied to the data of other sensors typically mounted on autonomous cars (e.g. lidars or radars) are not explored much. In this paper we propose a novel solution to understand the dynamics of moving vehicles of the scene from only lidar information. The main challenge of this problem stems from the fact that we need to disambiguate the proprio-motion of the “observer” vehicle from that of the external “observed” vehicles. For this purpose, we devise a CNN architecture which at testing time is fed with pairs of consecutive lidar scans. However, in order to properly learn the parameters of this network, during training we introduce a series of so-called pretext tasks which also leverage on image data. These tasks include semantic information about vehicleness and a novel lidar-flow feature which combines standard image-based optical flow with lidar scans. We obtain very promising results and show that including distilled image information only during training, allows improving the inference results of the network at test time, even when image data is no longer used., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects HuMoUR (TIN2017-90086-R) and COLROBTRANSP (DPI2016-78957-R) and the Spanish State Research Agency through the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656). The authors also thank Nvidia for hardware donation under the GPU grant program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Hallucinating dense optical flow from sparse lidar for autonomous vehicles

  • Vaquero, Victor
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la 24th International Conference on Pattern Recognition (ICPR), celebrada en Beijing (China), del 20 al 24 de agosto de 2018, In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle. This is intended to be used as a drop-in replacement of any image-based optical flow system when images are not reliable due to e.g. adverse weather conditions or at night. In order to infer high resolution 2D flows from discrete range data we devise a three-block architecture of multiscale filters that combines multiple intermediate objectives, both in the lidar and image domain. To train this network we introduce a dataset with approximately 20K lidar samples of the Kitti dataset which we have augmented with a pseudo ground-truth image-based optical flow computed using FlowNet2. We demonstrate the effectiveness of our approach on Kitti, and show that despite using the low-resolution and sparse measurements of the lidar, we can regress dense optical flow maps which are at par with those estimated with image-based methods., This work has been supported by the Spanish Ministry of Economy, Industry and Competitiveness projects COLROBTRANSP (DPI2016-78957-R), HuMoUR (TIN2017-90086-R), the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence (MDM-2016- 0656), and the EU project LOGIMATIC (H2020-Galileo2015-1-687534). We also thank Nvidia for hardware donation under the GPU Grant Program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Effects of a social force model reward in robot navigation based on deep reinforcement learning

  • Gil Viyuela, Óscar
  • Sanfeliu, Alberto
Trabajo presentado en el 4th Iberian Robotics Conference, celebrado en Oporto (Portugal) en noviembre de 2019, In this paper is proposed an inclusion of the Social Force Model (SFM) into a concrete Deep Reinforcement Learning (RL) framework for robot navigation. These types of techniques have demonstrated to be useful to deal with different types of environments to achieve a goal. In Deep RL, a description of the world to describe the states and a reward adapted to the environment are crucial elements to get the desire behaviour and achieve a high performance. For this reason, this work adds a dense reward function based on SFM and uses the forces in the states like an additional description. Furthermore, obstacles are added to improve the behaviour of works that only consider moving agents. This SFM inclusion can offer a better description of the obstacles for the navigation. Several simulations have been done to check the effects of these modifications in the average performance., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Human-robot collaborative navigation search using social reward sources

  • Dalmasso, Marc
  • Garrell, Anaís
  • Jiménez, Pablo
  • Sanfeliu, Alberto
Trabajo presentado en el 4th Iberian Robotics Conference, celebrado en Oporto (Portugal) en noviembre de 2019, This paper proposes a Social Reward Sources (SRS) design for a Human-Robot Collaborative Navigation (HRCN) task: human-robot collaborative search. It is a flexible approach capable of handling the collaborative task, human-robot interaction and environment restrictions, all integrated on a common environment. We modelled task rewards based on unexplored area observability and isolation and evaluated the model through different levels of human-robot communication.The models are validated through quantitative evaluation against both agents¿ individual performance and qualitative surveying of participants¿ perception. After that, the three proposed communication levels are com-pared against each other using the previous metrics., Work supported under projects ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet (H2020-INFRAIA-2017-1-two-stage-730994) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Teaching a Drone to Accompany a Person from Demonstrations using Non-Linear ASFM

  • Garrell, Anaís
  • Coll, Carles
  • Alquézar Mancho, Renato
  • Sanfeliu, Alberto
Trabajo presentado en la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Macao (China), del 3 al 8 de noviembre de 2019, In this paper, we present a new method based on the Aerial Social Force Model (ASFM) to allow human-drone side-by-side social navigation in real environments. To tackle this problem, the present work proposes a new nonlinear-based approach using Neural Networks. To learn and test the rightness of the new approach, we built a new dataset with simulated environments and we recorded motion controls provided by a human expert tele-operating the drone. The recorded data is then used to train a neural network which maps interaction forces to acceleration commands. The system is also reinforced with a human path prediction module to improve the drone¿s navigation, as well as, a collision detection module to completely avoid possible impacts. Moreover, a performance metric is defined which allows us to numerically evaluate and compare the fulfillment of the different learned policies. The method was validated by a large set of simulations; we also conducted real-life experiments with an autonomous drone to verify the framework described for the navigation process. In addition, a user study has been realized to reveal the social acceptability of the method., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU), and European Project GAUSS (H2020-GALILEO-GSA-2017-1-776293) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Robot approaching and engaging people in a human-robot companion framework

  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a new model to make robots capable of approaching and engaging people with a human-like behavior, while they are walking in a side-by-side formation with a person. This method extends our previous work [1], which allows the robot to adapt its navigation behaviour according to the person being accompanied and the dynamic environment. In the current work, the robot is able to predict the best encounter point between the human-robot group and the approached person. Then, in the encounter point the robot modifies its position to achieve an engagement with both people. The encounter point is computed using a gradient descent method that takes into account all people predictions. Moreover, we make use of the Extended Social Force Model (ESFM), and it is modified to include the dynamic goal. The method has been validated over several situations and in real-life experiments, in addition, a user study has been realized to reveal the social acceptability of the robot in this task., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Aerial social force model: a new framework to accompany people using autonomous flying robots

  • Garrell Zulueta, Anais
  • Garza Elizondo, Luís Alberto
  • Villamizar Vergel, Michael Alejandro
  • Herrero Cotarelo, Fernando
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We proposed a novel Aerial Social Force Model (ASFM) that allows autonomous flying robots to accompany humans in urban environments in a safe and comfortable manner. To date, we are not aware of other state-of-the-art method that accomplish this task. The proposed approach is a 3D version of the Social Force Model (SFM) for the field of aerial robots which includes an interactive human-robot navigation scheme capable of predicting human motions and intentions so as to safely accompany them to their final destination. ASFM also introduces a new metric to fine-tune the parameters of the force model, and to evaluate the performance of the aerial robot companion based on comfort and distance between the robot and humans. The presented approach is extensively validated in diverse simulations and real experiments, and compared against other similar works in the literature. ASFM attains remarkable results and shows that it is a valuable framework for social robotics applications, such as guiding people or human-robot interaction., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Sampling operation with robotic UAV

  • Guerra Paradas, Edmundo
  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Munguía Alcalá, Rodrigo Francisco
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This work presents a solution to automatize sampling tasks in a wastewater treatment plant with open air basins. At the behest of human operators, a set of UAVs managed as a network of autonomous agents will perform sample missions by taking direct measurements (through a multiparametric probe) or capturing samples and carrying them to the laboratory with specific developed hardware. These capabilities allow the proposed solution to act as a virtual sensor network with sampling points deployed and connected at any point reachable by UAVs. The hardware prototypes are fully described, with focus on the integration of systems, and the software architecture used is analysed and fully justified. Special focus was put on the localization problem, and several solutions were evaluated. Experimental results of the prototype UAV and sampling probe built are provided to validate the hardware designs, with focus on the localizations tasks., This research has been funded by Spanish Ministery of Innovation and Science CICYT DPI2016-78957-R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Effects of a social force model reward in robot navigation based on deep reinforcement learning

  • Gil Viyuela, Óscar
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, In this paper is proposed an inclusion of the Social Force Model (SFM) into a concrete Deep Reinforcement Learning (RL) framework for robot navigation. These types of techniques have demonstrated to be useful to deal with different types of environments to achieve a goal. In Deep RL, a description of the world to describe the states and a reward adapted to the environment are crucial elements to get the desire behaviour and achieve a high performance. For this reason, this work adds a dense reward function based on SFM and uses the forces in the states like an additional description. Furthermore, obstacles are added to improve the behaviour of works that only consider moving agents. This SFM inclusion can offer a better description of the obstacles for the navigation. Several simulations have been done to check the effects of these modifications in the average performance., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Unsupervised person image synthesis in arbitrary poses

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Pose identification and updating in autonomous vehicles

  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Munguía Alcalá, Rodrigo Francisco
In this paper, a novel algorithm to know the pose of any autonomous vehicle is described. Such a system (Attitude and Heading Reference System, AHRS) is essential for real time vehicle navigation, guidance and control applications. For low funded projects, with simple sensors, efficient and robust algorithms become necessary for an acceptable performance, and the well-known extended Kalman filter (EKF) fulfills those requirements. In this kind of applications, the use of the EKF in direct configuration has been much less explored than its counterpart, the EKF in indirect configuration. Specifically, in this paper a novel method based on an Extended Kalman Filter in direct configuration is proposed, where the filter is explicitly derived from both kinematic and errors models. Experiments with real data show that the proposed method is able to maintain an accurate and drift-free attitude and heading estimation., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

People's V-formation and side-by-side model adapted to accompany groups of people by social robots

  • Repiso Polo, Ely
  • Zanlungo, Francesco
  • Kanda, Takayuki
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a new method to allow robots to accompany a person or a group of people imitating pedestrians behavior. Two-people groups usually walk in a side-by-side formation and three-people groups walk in a V-formation so that they can see each other. For this reason, the proposed method combines a Side-by-side and V-formation pedestrian model with the Anticipative Kinodynamic Planner (AKP). Combining these methods, the robot is able to do an anticipatory accompaniment of groups of humans, as well as to avoid static and dynamic obstacles in advance, while keeping the prescribed formations. The proposed framework allows also a dynamical re-positioning of the robot, if the physical position of the partners change in the group formation. Furthermore, people have a randomness factor that the robot has to manage, for that reason, the system was adapted to deal with changes in people's velocity, orientation and occlusions. Finally, the method has been validated using synthetic experiments and real-life experiments with our Tibi robot. In addition, a user study has been realized to reveal the social acceptability of the method., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

3DPeople: modeling the geometry of dressed humans

  • Pumarola Peris, Albert
  • Sánchez Riera, Jordi
  • Choi, Gary
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Recent advances in 3D human shape estimation build upon parametric representations that model very well the shape of the naked body, but are not appropriate to represent the clothing geometry. In this paper, we present an approach to model dressed humans and predict their geometry from single images. We contribute in three fundamental aspects of the problem, namely, a new dataset, a novel shape parameterization algorithm and an end-to-end deep generative network for predicting shape. First, we present 3DPeople, a large-scale synthetic dataset with 2.5 Million photo-realistic images of 80 subjects performing 70 activities and wearing diverse outfits. Besides providing textured 3D meshes for clothes and body, we annotate the dataset with segmentation masks, skeletons, depth, normal maps and optical flow. All this together makes 3DPeople suitable for a plethora of tasks., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Teaching a drone to accompany a person from demonstrations using non-linear ASFM

  • Garrell Zulueta, Anais
  • Coll Gomilla, Carles
  • Alquézar Mancho, René
  • Sanfeliu Cortés, Alberto
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper, we present a new method based on the Aerial Social Force Model (ASFM) to allow human-drone side-by-side social navigation in real environments. To tackle this problem, the present work proposes a new nonlinear-based approach using Neural Networks. To learn and test the rightness of the new approach, we built a new dataset with simulated environments and we recorded motion controls provided by a human expert tele-operating the drone. The recorded data is then used to train a neural network which maps interaction forces to acceleration commands. The system is also reinforced with a human path prediction module to improve the drone’s navigation, as well as, a collision detection module to completely avoid possible impacts. Moreover, a performance metric is defined which allows us to numerically evaluate and compare the fulfillment of the different learned policies. The method was validated by a large set of simulations; we also conducted real-life experiments with an autonomous drone to verify the framework described for the navigation process. In addition, a user study has been realized to reveal the social acceptability of the method., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Low resolution lidar-based multi object tracking for driving applications

  • del Pino Bastida, Iván
  • Vaquero Gómez, Víctor
  • Massini, Beatrice
  • Solà Ortega, Joan
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
  • Andrade-Cetto, Juan
The final publication is available at link.springer.com, Vehicle detection and tracking in real scenarios are key com- ponents to develop assisted and autonomous driving systems. Lidar sen- sors are specially suitable for this task, as they bring robustness to harsh weather conditions while providing accurate spatial information. How- ever, the resolution provided by point cloud data is very scarce in com- parison to camera images. In this work we explore the possibilities of Deep Learning (DL) methodologies applied to low resolution 3D lidar sensors such as the Velodyne VLP-16 (PUCK), in the context of vehicle detection and tracking. For this purpose we developed a lidar-based sys- tem that uses a Convolutional Neural Network (CNN), to perform point- wise vehicle detection using PUCK data, and Multi-Hypothesis Extended Kalman Filters (MH-EKF), to estimate the actual position and veloci- ties of the detected vehicles. Comparative studies between the proposed lower resolution (VLP-16) tracking system and a high-end system, using Velodyne HDL-64, were carried out on the Kitti Tracking Benchmark dataset. Moreover, to analyze the influence of the CNN-based vehicle detection approach, comparisons were also performed with respect to the geometric-only detector. The results demonstrate that the proposed low resolution Deep Learning architecture is able to successfully accom- plish the vehicle detection task, outperforming the geometric baseline approach. Moreover, it has been observed that our system achieves a similar tracking performance to the high-end HDL-64 sensor at close range. On the other hand, at long range, detection is limited to half the distance of the higher-end sensor., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Temporal segmentation of human actions in video sequences

  • Carmona Leyva, José María
  • Climent Vilaró, Joan
Most of the published works concerning action recognition, usually assume that the action sequences have been previously segmented in time, that is, the action to be recognized starts with the first sequence frame and ends with the last one. However, temporal segmentation of actions in sequences is not an easy task, and is always prone to errors. In this paper, we present a new technique to automatically extract human actions from a video sequence. Our approach presents several contributions. First of all, we use a projection template scheme and find spatio-temporal features and descriptors within the projected surface, rather than extracting them in the whole sequence. For projecting the sequence we use a variant of the R transform, which has never been used before for temporal action segmentation. Instead of projecting the original video sequence, we project its optical flow components, preserving important information about action motion. We test our method on a publicly available action dataset, and the results show that it performs very well segmenting human actions compared with the state-of-the-art methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Towards robust 6-DoF detection in uncontrolled lightning enviroments

  • Mora, Alejandro
  • Guerra Paradas, Edmundo
  • Manzanares Brotons, Manuel
  • Grau Saldes, Antoni
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., One of the most critical issues that arises when controlling a robot is the necessity to locate itself in the environment. Ranging from industrial processes applications to outdoor mobile robots, landmarks are used to such purpose: by recognizing them, the robots are able to know where they are with the aid of computer vision. Fiducial markers are the cheapest and one of the most common solutions to this issue: 2D planar patterns that embed some information that can be identified using artificial vision techniques. Many different typologies of markers as well as image processing algorithms are being implemented nowadays, using C++ / Python / MATLAB® libraries and ROS as middleware. In this work we have evaluated the robustness of various fiducial marker typologies, documenting the main technical aspects of the used implementations and commenting how each algorithm operates. Aggregated results are presented and discussed, evaluating the different implementations tested., This research was funded by the Ministry of Economy, Industry and Competitiveness through project DPI2016-78957-R., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Deconvolutional networks for point-cloud vehicle detection and tracking in driving scenarios

  • Vaquero Gómez, Víctor
  • del Pino Bastida, Iván
  • Moreno-Noguer, Francesc
  • Solà Ortega, Joan
  • Sanfeliu Cortés, Alberto
  • Andrade-Cetto, Juan
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Geometry-aware network for non-rigid shape prediction from a single view

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Porzi, Lorenzo
  • Sanfeliu Cortés, Alberto
  • Lepetit, Vincent
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a {it geometry-aware} deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Human-robot collaborative navigation search using social reward sources

  • Dalmasso Blanch, Marc
  • Garrell Zulueta, Anais
  • Jiménez Schlegl, Pablo
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, This paper proposes a Social Reward Sources (SRS) design for a Human-Robot Collaborative Navigation (HRCN) task: human-robot collaborative search. It is a flexible approach capable of handling the collaborative task, human-robot interaction and environment restrictions, all integrated on a common environment. We modelled task rewards based on unexplored area observability and isolation and evaluated the model through different levels of human-robot communication.The models are validated through quantitative evaluation against both agents’ individual performance and qualitative surveying of participants’ perception. After that, the three proposed communication levels are com-pared against each other using the previous metrics., Work supported under projects ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet (H2020-INFRAIA-2017-1-two-stage-730994) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Considering the anchoring problem in robotic intelligent bin picking

  • Escudero Rodrigo, Diego
  • Alquézar Mancho, René
Random Bin Picking means the selection by a robot of a particular item from a container (or bin) in which there are many items randomly distributed. Generalist robots and the Anchoring Problem should be considered if we want to provide a more general solution, since users want that it works with different type of items that are not known 'a priori'. Therefore, we are working on an approach in which robot learning and human-robot interaction are used to anchor control primitives and robot skills to objects and action symbols while the robot system is running, but we are limiting the scope to the packaging domain. In this paper we explain how to use our system to do anchoring in Robotic Bin Picking., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Hallucinating dense optical flow from sparse lidar for autonomous vehicles

  • Vaquero Gómez, Víctor
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle. This is intended to be used as a drop-in replacement of any image-based optical flow system when images are not reliable due to e.g. adverse weather conditions or at night. In order to infer high resolution 2D flows from discrete range data we devise a three-block architecture of multiscale filters that combines multiple intermediate objectives, both in the lidar and image domain. To train this network we introduce a dataset with approximately 20K lidar samples of the Kitti dataset which we have augmented with a pseudo ground-truth image-based optical flow computed using FlowNet2. We demonstrate the effectiveness of our approach on Kitti, and show that despite using the low-resolution and sparse measurements of the lidar, we can regress dense optical flow maps which are at par with those estimated with image-based methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

GANimation: anatomically-aware facial animation from a single image

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Martinez, Aleix M.
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild., Peer Reviewed, Award-winning, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Image processing techniques in vision-based robot navigation

  • Munguía Alcalá, Rodrigo Francisco
  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
Motivated by the problems of vision-based mobile robot map building and localization, we present a comparative study of statistical methods for matching image features in a wide base line between learning and recognition phases. A general methodology called feature-class method for the problem of fast matching image features in a wide base line is described in the context of mobile robots. The objective of this work is to discuss and to show the performance of such methods in an example of visual SLAM, with experiments done with real data., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic regulation of wastewater discharge

  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
Wastewater plants, mainly with secondary treatments, discharge polluted water to environment that cannot be used in any human activity. When those dumps are in the sea it is expected that most of the biological pollutants die or almost disappear before water reaches human range. This natural withdrawal of bacteria, viruses and other pathogens is due to some conditions such as the salt water of the sea and the sun effect, and the dumps areas are calculated taking into account these conditions. However, under certain meteorological phenomena water arrives to the coast without the full disappearance of pollutant elements. In Mediterranean Sea there are some periods of adverse climatic conditions that pollute the coast near the wastewater dumping. In this paper, authors present an automatic control that prevents such pollution episodes using two mathematical models, one for the pollutant transportation and the other for the pollutant removal in wastewater spills, Peer Reviewed, Postprint (author's final draft)