Pasar al contenido principal

COLABORACION ROBOTS-HUMANOS PARA EL TRANSPORTE DE PRODUCTOS EN ZONAS URBANAS

DPI2016-78957-R

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2016
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 48
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Joint coarse-and-fine reasoning for deep optical flow

  • Vaquero, Victor
  • Ros, German
  • Moreno-Noguer, Francesc
  • López, Antonio M.
  • Sanfeliu, Alberto
Trabajo presentado a la IEEE International Conference on Image Processing (ICIP), celebrada en Beijing (China) del 17 al 20 de septiembre de 2017., We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets., This work was partially supported by European AEROARMS project (H2020-ICT-2014-1-644271) and CICYT projects ColRobTransp (DPI2016-78957-R), ROBINSTRUCT (TIN2014-58178-R)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Boosted Random Ferns for object detection

  • Villamizar, Michael
  • Andrade-Cetto, Juan
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation., This work was partially supported by the Catalan Agency for Management of University and Research Grants for the Consolidated Group VIS (2014 SGR 897). A.S. acknowledges support also from project ColRobTransp (DPI2016-78957-R), and J.A. and F.M. from project RobInstruct (TIN2014-58178-R), both funded by the Spanish Ministry of Economy, Industry and Competitiveness. J.A., A.S., and F.M. were also partially supported by the European Union projects AEROARMS (H2020-ICT-2014-1-644271) and LOGIMATIC (H2020-Galileo-2015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Low resolution lidar-based multi object tracking for driving applications

  • Pino, Iván del
  • Vaquero, Victor
  • Masini, Beatrice
  • Solà, Joan
  • Moreno-Noguer, Francesc
  • Sanfeliu, Alberto
  • Andrade-Cetto, Juan
Trabajo presentado a la Third Iberian Robotics Conference, celebrada en Sevilla del 22 al 24 de noviembre de 2017., Vehicle detection and tracking in real scenarios are key com- ponents to develop assisted and autonomous driving systems. Lidar sen- sors are specially suitable for this task, as they bring robustness to harsh weather conditions while providing accurate spatial information. How- ever, the resolution provided by point cloud data is very scarce in com- parison to camera images. In this work we explore the possibilities of Deep Learning (DL) methodologies applied to low resolution 3D lidar sensors such as the Velodyne VLP-16 (PUCK), in the context of vehicle detection and tracking. For this purpose we developed a lidar-based sys- tem that uses a Convolutional Neural Network (CNN), to perform point- wise vehicle detection using PUCK data, and Multi-Hypothesis Extended Kalman Filters (MH-EKF), to estimate the actual position and veloci- ties of the detected vehicles. Comparative studies between the proposed lower resolution (VLP-16) tracking system and a high-end system, using Velodyne HDL-64, were carried out on the Kitti Tracking Benchmark dataset. Moreover, to analyze the influence of the CNN-based vehicle detection approach, comparisons were also performed with respect to the geometric-only detector. The results demonstrate that the proposed low resolution Deep Learning architecture is able to successfully accom- plish the vehicle detection task, outperforming the geometric baseline approach. Moreover, it has been observed that our system achieves a similar tracking performance to the high-end HDL-64 sensor at close range. On the other hand, at long range, detection is limited to half the distance of the higher-end sensor., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects ROBINSTRUCT (TIN2014-58178-R) and COLROBTRANSP (DPI2016-78957-R), by the Spanish Ministry of Education FPU grant (FPU15/04446), the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and by the EU H2020 project LOGIMATIC (H2020-Galileo-2015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Deconvolutional networks for point-cloud vehicle detection and tracking in driving scenarios

  • Vaquero, Victor
  • Pino, Iván del
  • Moreno-Noguer, Francesc
  • Solà, Joan
  • Sanfeliu, Alberto
  • Andrade-Cetto, Juan
Trabajo presentado a la European Conference on Mobile Robots (ECMR), celebrada en Paris (Francia) del 6 al 8 de septiembre de 2017., Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects ROBINSTRUCT (TIN2014-58178-R) and COLROBTRANSP (DPI2016-78957-R), by the Spanish Ministry of Education FPU grant (FPU15/04446), the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI MDM-2016-0656) and by the EU H2020 project LOGIMATIC (H2020-Galileo-015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

Considering the anchoring problem in robotic intelligent bin picking

  • Escudero-Rodrigo, Diego
  • Alquézar Mancho, Renato
Random Bin Picking means the selection by a robot of a particular item from a container (or bin) in which there are many items randomly distributed. Generalist robots and the Anchoring Problem should be considered if we want to provide a more general solution, since users want that it works with different type of items that are not known ‘a priori’. Therefore, we are working on an approach in which robot learning and human-robot interaction are used to anchor control primitives and robot skills to objects and action symbols while the robot system is running, but we are limiting the scope to the packaging domain. In this paper we explain how to use our system to do anchoring in Robotic Bin Picking., This work was partially funded by Spanish research project DPI2016-78957-R., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Deep lidar CNN to understand the dynamics of moving vehicles

  • Vaquero, Victor
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la IEEE International Conference on Robotics and Automation (ICRA), celebrado en Brisbane (Australia), del 21 al 25 de mayo de 2018, Perception technologies in Autonomous Driving are experiencing their golden age due to the advances in Deep Learning. Yet, most of these systems rely on the semantically rich information of RGB images. Deep Learning solutions applied to the data of other sensors typically mounted on autonomous cars (e.g. lidars or radars) are not explored much. In this paper we propose a novel solution to understand the dynamics of moving vehicles of the scene from only lidar information. The main challenge of this problem stems from the fact that we need to disambiguate the proprio-motion of the “observer” vehicle from that of the external “observed” vehicles. For this purpose, we devise a CNN architecture which at testing time is fed with pairs of consecutive lidar scans. However, in order to properly learn the parameters of this network, during training we introduce a series of so-called pretext tasks which also leverage on image data. These tasks include semantic information about vehicleness and a novel lidar-flow feature which combines standard image-based optical flow with lidar scans. We obtain very promising results and show that including distilled image information only during training, allows improving the inference results of the network at test time, even when image data is no longer used., This work has been supported by the Spanish Ministry of Economy and Competitiveness projects HuMoUR (TIN2017-90086-R) and COLROBTRANSP (DPI2016-78957-R) and the Spanish State Research Agency through the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656). The authors also thank Nvidia for hardware donation under the GPU grant program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Hallucinating dense optical flow from sparse lidar for autonomous vehicles

  • Vaquero, Victor
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la 24th International Conference on Pattern Recognition (ICPR), celebrada en Beijing (China), del 20 al 24 de agosto de 2018, In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle. This is intended to be used as a drop-in replacement of any image-based optical flow system when images are not reliable due to e.g. adverse weather conditions or at night. In order to infer high resolution 2D flows from discrete range data we devise a three-block architecture of multiscale filters that combines multiple intermediate objectives, both in the lidar and image domain. To train this network we introduce a dataset with approximately 20K lidar samples of the Kitti dataset which we have augmented with a pseudo ground-truth image-based optical flow computed using FlowNet2. We demonstrate the effectiveness of our approach on Kitti, and show that despite using the low-resolution and sparse measurements of the lidar, we can regress dense optical flow maps which are at par with those estimated with image-based methods., This work has been supported by the Spanish Ministry of Economy, Industry and Competitiveness projects COLROBTRANSP (DPI2016-78957-R), HuMoUR (TIN2017-90086-R), the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence (MDM-2016- 0656), and the EU project LOGIMATIC (H2020-Galileo2015-1-687534). We also thank Nvidia for hardware donation under the GPU Grant Program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Robust and Real-Time Detection and Tracking of Moving Objects with Minimum 2D LiDAR Information to Advance Autonomous Cargo Handling in Ports

  • Vaquero, Victor
  • Repiso, Ely
  • Sanfeliu, Alberto
Detecting and tracking moving objects (DATMO) is an essential component for autonomous driving and transportation. In this paper, we present a computationally low-cost and robust DATMO system which uses as input only 2D laser rangefinder (LRF) information. Due to its low requirements both in sensor needs and computation, our DATMO algorithm is meant to be used in current Autonomous Guided Vehicles (AGVs) to improve their reliability for the cargo transportation tasks at port terminals, advancing towards the next generation of fully autonomous transportation vehicles. Our method follows a Detection plus Tracking paradigm. In the detection step we exploit the minimum information of 2D-LRFs by segmenting the elements of the scene in a model-free way and performing a fast object matching to pair segmented elements from two different scans. In this way, we easily recognize dynamic objects and thus reduce consistently by between two and five times the computational burden of the adjacent tracking method. We track the final dynamic objects with an improved Multiple-Hypothesis Tracking (MHT), to which special functions for filtering, confirming, holding, and deleting targets have been included. The full system is evaluated in simulated and real scenarios producing solid results. Specifically, a simulated port environment has been developed to gather realistic data of common autonomous transportation situations such as observing an intersection, joining vehicle platoons, and perceiving overtaking maneuvers. We use different sensor configurations to demonstrate the robustness and adaptability of our approach. We additionally evaluate our system with real data collected in a port terminal the Netherlands. We show that it is able to accomplish the vehicle following task successfully, obtaining a total system recall of more than 98% while running faster than 30 Hz, This work has been partly funded by the EU project CargoANTs FP7-SST-2013-605598, the Spanish MINECO projects DPI2016-78957-R (AEI/FEDER EU) and Unidad de Excelencia Maria de Maeztu 2016 (MDM-2016-0656)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Adaptive Side-by-Side Social Robot Navigation to Approach and Interact with People

  • Repiso, Ely
  • Garrell, Anaís
  • Sanfeliu, Alberto
This paper presents a new framework for how autonomous social robots approach and accompany people in urban environments. The method discussed allows the robot to accompany a person and approach to other one, by adapting its own navigation in anticipation of future interactions with other people or contact with static obstacles. The contributions of the paper are manifold: firstly, we extended the Social Force model and the Anticipative Kinodynamic Planner (Ferrer and Sanfeliu, in: IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2014) to the case of an adaptive side-by-side navigation; secondly, we enhance side-by-side navigation with an approaching task and a final positioning that allows the robot to interact with both people; and finally, we use findings from experiments of real-life observations of people walking in pairs to define the parameters of the human–robot interaction in our case of adaptive side-by-side. The method was validated by a large set of simulations; we also conducted real-life experiments with our robot, Tibi, to validate the framework described for the interaction process. In addition, we carried out various surveys and user studies to indicate the social acceptability of the robots performance of the accompanying, approaching and positioning tasks., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-R. AEI/FEDER EU), by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and EU TERRINET project (H2020-INFRAIA-2017-1-two-stage-730994). Ely Repiso is also supported by Spanish Ministry of Science and Innovation under a FPI-grant (BES-2014-067713)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Anticipatory kinodynamic motion planner for computing the best path and velocity trajectory in autonomous driving

  • Pérez Talamino, Jordi
  • Sanfeliu, Alberto
This paper presents an approach, using an anticipatory kinodynamic motion planner, for obtaining the best trajectory and velocity profile for autonomous driving in dynamic complex environments, such as driving in urban scenarios. The planner discretizes the road search space and looks for the best vehicle path and velocity profile at each control period of time, assuming that the static and dynamic objects have been detected. The main contributions of the work are in the anticipatory kinodynamic motion planner, in a fast method for obtaining the G-splines for path generation, and in a method to compute and select the best velocity profile at each candidate path that fulfills the vehicle kinodynamic constraints, taking into account the passenger comfort. The method has been developed and tested in MATLAB through a set of simulations in different representative scenarios, involving fixed obstacles and moving vehicles. The outcome of the simulations shows that the anticipatory kinodynamic planner performs correctly in diverse dynamic scenarios, maintaining smooth accelerations for passenger comfort., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

GANimation: One-Shot Anatomically Consistent Facial Animation

  • Pumarola, Albert
  • Agudo Martínez, Antonio
  • Martínez, Aleix M.
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Recent advances in generative adversarial networks (GANs) have shown impressive results for the task of facial expression synthesis. The most successful architecture is StarGAN (Choi et al. in CVPR, 2018), that conditions GANs’ generation process with images of a specific domain, namely a set of images of people sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content and granularity of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on action units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combining several of them. Additionally, we propose a weakly supervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit a novel self-learned attention mechanism that makes our network robust to changing backgrounds, lighting conditions and occlusions. Extensive evaluation shows that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild. The code of this work is publicly available at https://github.com/albertpumarola/GANimation., This work is partially supported by an Amazon Research Award, by the Spanish Ministry of Economy and Competitiveness under Projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957 and María de Maeztu Seal of Excellence MDM-2016-0656; by the EU Project AEROARMS ICT-2014-1-644271; and by the Grant R01-DC- 014498 of the National Institute of Health. We also thank Nvidia for hardware donation under the GPU Grant Program.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

3DPeople: Modeling the geometry of dressed humans

  • Pumarola, Albert
  • Sánchez-Riera, Jordi
  • Choi, Gary P. T.
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en el 17th International Conference on Computer Vision, celebrado Seúl (Corea del Sur), del 27 de octubre al 2 de noviembre de 2019, Recent advances in 3D human shape estimation build upon parametric representations that model very well the shape of the naked body, but are not appropriate to represent the clothing geometry. In this paper, we present an approach to model dressed humans and predict their geometry from single images. We contribute in three fundamental aspects of the problem, namely, a new dataset, a novel shape parameterization algorithm and an end-to-end deep generative network for predicting shape. First, we present 3DPeople, a large-scale synthetic dataset with 2.5 Million photo-realistic images of 80 subjects performing 70 activities and wearing diverse outfits. Besides providing textured 3D meshes for clothes and body, we annotate the dataset with segmentation masks, skeletons, depth, normal maps and optical flow. All this together makes 3DPeople suitable for a plethora of tasks., This work is supported in part by an Amazon Research Award, the Croucher Foundation and the Spanish MiNeCo under projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957-R and María de Maeztu Seal of Excellence MDM-2016-0656.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Effects of a social force model reward in robot navigation based on deep reinforcement learning

  • Gil Viyuela, Óscar
  • Sanfeliu, Alberto
Trabajo presentado en el 4th Iberian Robotics Conference, celebrado en Oporto (Portugal) en noviembre de 2019, In this paper is proposed an inclusion of the Social Force Model (SFM) into a concrete Deep Reinforcement Learning (RL) framework for robot navigation. These types of techniques have demonstrated to be useful to deal with different types of environments to achieve a goal. In Deep RL, a description of the world to describe the states and a reward adapted to the environment are crucial elements to get the desire behaviour and achieve a high performance. For this reason, this work adds a dense reward function based on SFM and uses the forces in the states like an additional description. Furthermore, obstacles are added to improve the behaviour of works that only consider moving agents. This SFM inclusion can offer a better description of the obstacles for the navigation. Several simulations have been done to check the effects of these modifications in the average performance., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Human-robot collaborative navigation search using social reward sources

  • Dalmasso, Marc
  • Garrell, Anaís
  • Jiménez, Pablo
  • Sanfeliu, Alberto
Trabajo presentado en el 4th Iberian Robotics Conference, celebrado en Oporto (Portugal) en noviembre de 2019, This paper proposes a Social Reward Sources (SRS) design for a Human-Robot Collaborative Navigation (HRCN) task: human-robot collaborative search. It is a flexible approach capable of handling the collaborative task, human-robot interaction and environment restrictions, all integrated on a common environment. We modelled task rewards based on unexplored area observability and isolation and evaluated the model through different levels of human-robot communication.The models are validated through quantitative evaluation against both agents¿ individual performance and qualitative surveying of participants¿ perception. After that, the three proposed communication levels are com-pared against each other using the previous metrics., Work supported under projects ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet (H2020-INFRAIA-2017-1-two-stage-730994) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Online learning and detection of faces with low human supervision

  • Villamizar, Michael
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
We present an efficient,online,and interactive approach for computing a classifier, called Wild Lady Ferns (WiLFs), for face learning and detection using small human supervision. More precisely, on the one hand, WiLFs combine online boosting and extremely randomized trees (Random Ferns) to compute progressively an efficient and discriminative classifier. On the other hand, WiLFs use an interactive human-machine approach that combines two complementary learning strategies to reduce considerably the degree of human supervision during learning. While the first strategy corresponds to query-by-boosting active learning, that requests human assistance over difficult samples in function of the classifier confidence, the second strategy refers to a memory-based learning which uses ¿ Exemplar-based Nearest Neighbors (¿ENN) to assist automatically the classifier. A pre-trained Convolutional Neural Network (CNN) is used to perform ¿ENN with high-level feature descriptors. The proposed approach is therefore fast (WilFs run in 1 FPS using a code not fully optimized), accurate (we obtain detection rates over 82% in complex datasets), and labor-saving (human assistance percentages of less than 20%). As a byproduct, we demonstrate that WiLFs also perform semi-automatic annotation during learning, as while the classifier is being computed, WiLFs are discovering faces instances in input images which are used subsequently for training online the classifier. The advantages of our approach are demonstrated in synthetic and publicly available databases, showing comparable detection rates as offline approaches that require larger amounts of handmade training data., This work is partially supported by the Spanish Ministry of Economy and Competitiveness under projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957 and María de Maeztu Seal of Excellence MDM- 2016-0656.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Teaching a Drone to Accompany a Person from Demonstrations using Non-Linear ASFM

  • Garrell, Anaís
  • Coll, Carles
  • Alquézar Mancho, Renato
  • Sanfeliu, Alberto
Trabajo presentado en la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Macao (China), del 3 al 8 de noviembre de 2019, In this paper, we present a new method based on the Aerial Social Force Model (ASFM) to allow human-drone side-by-side social navigation in real environments. To tackle this problem, the present work proposes a new nonlinear-based approach using Neural Networks. To learn and test the rightness of the new approach, we built a new dataset with simulated environments and we recorded motion controls provided by a human expert tele-operating the drone. The recorded data is then used to train a neural network which maps interaction forces to acceleration commands. The system is also reinforced with a human path prediction module to improve the drone¿s navigation, as well as, a collision detection module to completely avoid possible impacts. Moreover, a performance metric is defined which allows us to numerically evaluate and compare the fulfillment of the different learned policies. The method was validated by a large set of simulations; we also conducted real-life experiments with an autonomous drone to verify the framework described for the navigation process. In addition, a user study has been realized to reveal the social acceptability of the method., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU), and European Project GAUSS (H2020-GALILEO-GSA-2017-1-776293) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

People's adaptive side-by-side model evolved to accompany groups of people by social robots

  • Repiso, Ely
  • Garrell, Anaís
  • Sanfeliu, Alberto
The presented method implements a robot accompaniment in an adaptive side-by-side formation of a single person or a group of people. The method enhances our previous robot adaptive side-by-side behavior allowing the robot to accompany a group of people, not only one person. Adaptive means that the robot is capable to adjust its position and velocity to the behavior of the group being accompanied, without bothering other pedestrians in the environment, as well as facilitating the group navigation to avoid static and dynamic obstacles. Furthermore, the robot can deal with the random factor of human behavior in several situations. Firstly, if other people interfere in the path of the companions, the robot leaves space for the person of the group that has to avoid those other people, by approaching the other companion. Also, without invading any personal space. Secondly, if the people of the group changes their physical position inside the group formation, then the robot adapts to them dynamically by changing from the lateral position to the central position of the formation or otherwise. Thirdly, the robot adapts to changes in the velocity of people in the group and other people that interfere in the path of the group, in magnitude and orientation. Fourthly, the robot can deal with occlusions of one accompanied person by the other. The method has been validated using synthetic experiments and real-life experiments with our robot. Finally, we developed an user study comparing the method with a Wizard of Oz., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet (H2020-INFRAIA-2017-1-two-stage-730994), by the Spanish State Research Agency through the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Ely Repiso is also suported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2014-067713
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Tesis doctoral (DoctoralThesis)

Lidar-Based Scene Understanding for Autonomous Driving Using Deep Learning

  • Vaquero, Victor
Tesis llevada a cabo para conseguir el grado de Doctor por la Universidad Politécnica de Cataluña.--2020-02-25, [EN] With over 1.35 million fatalities related to traffic accidents worldwide, autonomous driving was foreseen at the beginning of this century as a feasible solution to improve security in our roads. Nevertheless, it is meant to disrupt our transportation paradigm, allowing to reduce congestion, pollution, and costs, while increasing the accessibility, efficiency, and reliability of the transportation for both people and goods. Although some advances have gradually been transferred into commercial vehicles in the way of Advanced Driving Assistance Systems (ADAS) such as adaptive cruise control, blind spot detection or automatic parking, however, the technology is far from mature. A full understanding of the scene is actually needed so that allowing the vehicles to be aware of the surroundings, knowing the existing elements of the scene, as well as their motion, intentions and interactions. In this PhD dissertation, we explore new approaches for understanding driving scenes from 3D LiDAR point clouds by using Deep Learning methods. To this end, in Part I we analyze the scene from a static perspective using independent frames to detect the neighboring vehicles. Next, in Part II we develop new ways for understanding the dynamics of the scene. Finally, in Part III we apply all the developed methods to accomplish higher level challenges such as segmenting moving obstacles while obtaining their rigid motion vector over the ground. More specifically, in Chapter 2 we develop a 3D vehicle detection pipeline based on a multi-branch deep-learning architecture and propose a Front (FR-V) and a Bird’s Eye view (BE-V) as 2D representations of the 3D point cloud to serve as input for training our models. Later on, in Chapter 3 we apply and further test this method on two real uses-cases, for pre-filtering moving obstacles while creating maps to better localize ourselves on subsequent days, as well as for vehicle tracking. From the dynamic perspective, in Chapter 4 we learn from the 3D point cloud a novel dynamic feature that resembles optical flow from RGB images. For that, we develop a new approach to leverage RGB optical flow as pseudo ground truth for training purposes but allowing the use of only 3D LiDAR data at inference time. Additionally, in Chapter 5 we explore the benefits of combining classification and regression learning problems to face the optical flow estimation task in a joint coarse-and-fine manner. Lastly, in Chapter 6 we gather the previous methods and demonstrate that with these independent tasks we can guide the learning of higher challenging problems such as segmentation and motion estimation of moving vehicles from our own moving perspective., [ES] Con más de 1,35 millones de muertes por accidentes de tráfico en el mundo, a principios de siglo se predijo que la conducción autónoma sería una solución viable para mejorar la seguridad en nuestras carreteras. Además la conducción autónoma está destinada a cambiar nuestros paradigmas de transporte, permitiendo reducir la congestión del tráfico, la contaminación y el coste, a la vez que aumentando la accesibilidad, la eficiencia y confiabilidad del transporte tanto de personas como de mercancías. Aunque algunos avances, como el control de crucero adaptativo, la detección de puntos ciegos o el estacionamiento automático, se han transferido gradualmente a vehículos comerciales en la forma de los Sistemas Avanzados de Asistencia a la Conducción (ADAS), la tecnología aún no ha alcanzado el suficiente grado de madurez. Se necesita una comprensión completa de la escena para que los vehículos puedan entender el entorno, detectando los elementos presentes, así como su movimiento, intenciones e interacciones. En la presente tesis doctoral, exploramos nuevos enfoques para comprender escenarios de conducción utilizando nubes de puntos en 3D capturadas con sensores LiDAR, para lo cual empleamos métodos de aprendizaje profundo. Con este fin, en la Parte I analizamos la escena desde una perspectiva estática para detectar vehículos. A continuación, en la Parte II, desarrollamos nuevas formas de entender las dinámicas del entorno. Finalmente, en la Parte III aplicamos los métodos previamente desarrollados para lograr desafíos de nivel superior, como segmentar obstáculos dinámicos a la vez que estimamos su vector de movimiento sobre el suelo. Específicamente, en el Capítulo 2 detectamos vehículos en 3D creando una arquitectura de aprendizaje profundo de dos ramas y proponemos una vista frontal (FR-V) y una vista de pájaro (BE-V) como representaciones 2D de la nube de puntos 3D que sirven como entrada para entrenar nuestros modelos. Más adelante, en el Capítulo 3 aplicamos y probamos aún más este método en dos casos de uso reales, tanto para filtrar obstáculos en movimiento previamente a la creación de mapas sobre los que poder localizarnos mejor en los días posteriores, como para el seguimiento de vehículos. Desde la perspectiva dinámica, en el Capítulo 4 aprendemos de la nube de puntos en 3D una característica dinámica novedosa que se asemeja al flujo óptico sobre imágenes RGB. Para ello, desarrollamos un nuevo enfoque que aprovecha el flujo óptico RGB como pseudo muestras reales para entrenamiento, usando solo information 3D durante la inferencia. Además, en el Capítulo 5 exploramos los beneficios de combinar los aprendizajes de problemas de clasificación y regresión para la tarea de estimación de flujo óptico de manera conjunta. Por último, en el Capítulo 6 reunimos los métodos anteriores y demostramos que con estas tareas independientes podemos guiar el aprendizaje de problemas de más alto nivel, como la segmentación y estimación del movimiento de vehículos desde nuestra propia perspectiva, This work has been partially supported by the European projects: FP7-SST-2013-RTD-1-605598 Cargo-ANTS: Cargo handling by Automated Next generation Transportation Systems for ports and terminals. FLAG-ERA-JTC-2016 ROBOCOM++: Rethinking robotics for the robot companion of the future. As well as by the Spanish Ministry of Education and Science under the research project: DPI2016-78957-R ColRobTransp: Colaboración robots-humanos para el transporte de productos en zonas urbanas
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Dual-Branch CNNs for Vehicle Detection and Tracking on LiDAR Data

  • Vaquero, Victor
  • Pino, Iván del
  • Moreno-Noguer, Francesc
  • Solà, Joan
  • Sanfeliu, Alberto
  • Andrade-Cetto, Juan
We present a novel vehicle detection and tracking system that works solely on 3D LiDAR information. Our approach segments vehicles using a dual-view representation of the 3D LiDAR point cloud on two independently trained convolutional neural networks, one for each view. A bounding box growing algorithm is applied to the fused output of the networks to properly enclose the segmented vehicles. Bounding boxes are grown using a probabilistic method that takes into account also occluded areas. The final vehicle bounding boxes act as observations for a multi-hypothesis tracking system which allows to estimate the position and velocity of the observed vehicles. We thoroughly evaluate our system on the KITTI benchmarks both for detection and tracking separately and show that our dual-branch classifier consistently outperforms previous single-branch approaches, improving or directly competing to other state of the art LiDAR-based methods., This work is supported by the EU project LOGIMATIC (H2020-Galileo2015-1-687534), the Spanish State Research Agency through the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656), the ColRobTransp project (DPI2016-78957-R AEI/FEDER EU), the EB-SLAM project (DPI2017-89564-P), and the FPU grant FPU15/04446.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Tesis doctoral (DoctoralThesis)

Bridging the gap between reconstruction and synthesis

  • Pumarola, Albert
Tesis doctoral presentada para lograr el título de Doctor por la Universidad Politécnica de Cataluña, programa de doctorado: Automatic Control, Robotics and Computer Vision.--2021-10-13, [EN] 3D reconstruction and image synthesis are two of the main pillars in computer vision. Early works focused on simple tasks such as multi-view reconstruction and texture synthesis. With the spur of Deep Learning, the field has rapidly progressed, making it possible to achieve more complex and high level tasks. For example, the 3D reconstruction results of traditional multi-view approaches are currently obtained with single view methods. Similarly, early pattern based texture synthesis works have resulted in techniques that allow generating novel high-resolution images. In this thesis we have developed a hierarchy of tools that cover all these range of problems, lying at the intersection of computer vision, graphics and machine learning. We tackle the problem of 3D reconstruction and synthesis in the wild. Importantly, we advocate for a paradigm in which not everything should be learned. Instead of applying Deep Learning naively we propose novel representations, layers and architectures that directly embed prior 3D geometric knowledge for the task of 3D reconstruction and synthesis. We apply these techniques to problems including scene/person reconstruction and photo-realistic rendering. We first address methods to reconstruct a scene and the clothed people in it while estimating the camera position. Then, we tackle image and video synthesis for clothed people in the wild. Finally, we bridge the gap between reconstruction and synthesis under the umbrella of a unique novel formulation. Extensive experiments conducted along this thesis show that the proposed techniques improve the performance of Deep Learning models in terms of the quality of the reconstructed 3D shapes / synthesised images, while reducing the amount of supervision and training data required to train them. In summary, we provide a variety of low, mid and high level algorithms that can be used to incorporate prior knowledge into different stages of the Deep Learning pipeline and improve performance in tasks of 3D reconstruction and image synthesis., [CAT] La reconstrucció 3D i la síntesi d'imatges són dos dels pilars fonamentals en visió per computador. Els estudis previs es centren en tasques senzilles com la reconstrucció amb informació multi-càmera i la síntesi de textures. Amb l'aparició del "Deep Learning", aquest camp ha progressat ràpidament, fent possible assolir tasques molt més complexes. Per exemple, per obtenir una reconstrucció 3D, tradicionalment s'utilitzaven mètodes multi-càmera, en canvi ara, es poden obtenir a partir d'una sola imatge. De la mateixa manera, els primers treballs de síntesi de textures basats en patrons han donat lloc a tècniques que permeten generar noves imatges completes en alta resolució. En aquesta tesi, hem desenvolupat una sèrie d'eines que cobreixen tot aquest ventall de problemes, situats en la intersecció entre la visió per computador, els gràfics i l'aprenentatge automàtic. Abordem el problema de la reconstrucció i la síntesi 3D en el món real. És important destacar que defensem un paradigma on no tot s'ha d'aprendre. Enlloc d'aplicar el "Deep Learning" de forma naïve, proposem representacions novedoses i arquitectures que incorporen directament els coneixements geomètrics ja existents per a aconseguir la reconstrucció 3D i la síntesi d'imatges. Nosaltres apliquem aquestes tècniques a problemes com ara la reconstrucció d'escenes/persones i a la renderització d'imatges fotorealistes. Primer abordem els mètodes per reconstruir una escena, les persones vestides que hi ha i la posició de la càmera. A continuació, abordem la síntesi d'imatges i vídeos de persones vestides en situacions quotidianes. I finalment, aconseguim, a través d'una nova formulació única, connectar la reconstrucció amb la síntesi. Els experiments realitzats al llarg d'aquesta tesi demostren que les tècniques proposades milloren el rendiment dels models de "Deepp Learning" pel que fa a la qualitat de les reconstruccions i les imatges sintetitzades alhora que redueixen la quantitat de dades necessàries per entrenar-los. En resum, proporcionem una varietat d'algoritmes de baix, mitjà i alt nivell que es poden utilitzar per incorporar els coneixements previs a les diferents etapes del "Deep Learning" i millorar el rendiment en tasques de reconstrucció 3D i síntesi d'imatges., This work has been partially supported by: - EU project AEROARMS - Aerial Robotics Inspection System, 2015-2019. Code: H2020-ICT-2014- 1-644271 - National Project ColRobTransp - Robotics Last Mile Transportation in Urban Areas, 2016-2019. Code: DPI2016-78957-R - Google Research Award - Geometry-aware CNNs for Non-Rigid Reconstruction, 2017-2018 - Amazon Research Award - Geometry-aware 3D Human Body Animation from Still Photos, 2018- 2019 - Google Research Award - GANimation3D: Unsupervised 3D Face Animation from Monocular Images, 2019-2020
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Robot approaching and engaging people in a human-robot companion framework

  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a new model to make robots capable of approaching and engaging people with a human-like behavior, while they are walking in a side-by-side formation with a person. This method extends our previous work [1], which allows the robot to adapt its navigation behaviour according to the person being accompanied and the dynamic environment. In the current work, the robot is able to predict the best encounter point between the human-robot group and the approached person. Then, in the encounter point the robot modifies its position to achieve an engagement with both people. The encounter point is computed using a gradient descent method that takes into account all people predictions. Moreover, we make use of the Extended Social Force Model (ESFM), and it is modified to include the dynamic goal. The method has been validated over several situations and in real-life experiments, in addition, a user study has been realized to reveal the social acceptability of the robot in this task., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Aerial social force model: a new framework to accompany people using autonomous flying robots

  • Garrell Zulueta, Anais
  • Garza Elizondo, Luís Alberto
  • Villamizar Vergel, Michael Alejandro
  • Herrero Cotarelo, Fernando
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We proposed a novel Aerial Social Force Model (ASFM) that allows autonomous flying robots to accompany humans in urban environments in a safe and comfortable manner. To date, we are not aware of other state-of-the-art method that accomplish this task. The proposed approach is a 3D version of the Social Force Model (SFM) for the field of aerial robots which includes an interactive human-robot navigation scheme capable of predicting human motions and intentions so as to safely accompany them to their final destination. ASFM also introduces a new metric to fine-tune the parameters of the force model, and to evaluate the performance of the aerial robot companion based on comfort and distance between the robot and humans. The presented approach is extensively validated in diverse simulations and real experiments, and compared against other similar works in the literature. ASFM attains remarkable results and shows that it is a valuable framework for social robotics applications, such as guiding people or human-robot interaction., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Sampling operation with robotic UAV

  • Guerra Paradas, Edmundo
  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Munguía Alcalá, Rodrigo Francisco
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This work presents a solution to automatize sampling tasks in a wastewater treatment plant with open air basins. At the behest of human operators, a set of UAVs managed as a network of autonomous agents will perform sample missions by taking direct measurements (through a multiparametric probe) or capturing samples and carrying them to the laboratory with specific developed hardware. These capabilities allow the proposed solution to act as a virtual sensor network with sampling points deployed and connected at any point reachable by UAVs. The hardware prototypes are fully described, with focus on the integration of systems, and the software architecture used is analysed and fully justified. Special focus was put on the localization problem, and several solutions were evaluated. Experimental results of the prototype UAV and sampling probe built are provided to validate the hardware designs, with focus on the localizations tasks., This research has been funded by Spanish Ministery of Innovation and Science CICYT DPI2016-78957-R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Effects of a social force model reward in robot navigation based on deep reinforcement learning

  • Gil Viyuela, Óscar
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, In this paper is proposed an inclusion of the Social Force Model (SFM) into a concrete Deep Reinforcement Learning (RL) framework for robot navigation. These types of techniques have demonstrated to be useful to deal with different types of environments to achieve a goal. In Deep RL, a description of the world to describe the states and a reward adapted to the environment are crucial elements to get the desire behaviour and achieve a high performance. For this reason, this work adds a dense reward function based on SFM and uses the forces in the states like an additional description. Furthermore, obstacles are added to improve the behaviour of works that only consider moving agents. This SFM inclusion can offer a better description of the obstacles for the navigation. Several simulations have been done to check the effects of these modifications in the average performance., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126., Work supported by the Spanish Ministry of Science and Innovation under project ColRobTransp (DPI2016-78957-RAEI/FEDER EU) and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Óscar Gil is also supported by Spanish Ministry of Science and Innovation under a FPI-grant, BES-2017-082126., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Unsupervised person image synthesis in arbitrary poses

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Pose identification and updating in autonomous vehicles

  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Munguía Alcalá, Rodrigo Francisco
In this paper, a novel algorithm to know the pose of any autonomous vehicle is described. Such a system (Attitude and Heading Reference System, AHRS) is essential for real time vehicle navigation, guidance and control applications. For low funded projects, with simple sensors, efficient and robust algorithms become necessary for an acceptable performance, and the well-known extended Kalman filter (EKF) fulfills those requirements. In this kind of applications, the use of the EKF in direct configuration has been much less explored than its counterpart, the EKF in indirect configuration. Specifically, in this paper a novel method based on an Extended Kalman Filter in direct configuration is proposed, where the filter is explicitly derived from both kinematic and errors models. Experiments with real data show that the proposed method is able to maintain an accurate and drift-free attitude and heading estimation., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Boosted Random ferns for object detection

  • Villamizar Vergel, Michael Alejandro
  • Andrade-Cetto, Juan
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Adaptive side-by-side social robot navigation to approach and interact with people

  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, This paper presents a new framework for how autonomous social robots approach and accompany people in urban environments. The method discussed allows the robot to accompany a person and approach to other one, by adapting its own navigation in anticipation of future interactions with other people or contact with static obstacles. The contributions of the paper are manifold: firstly, we extended the Social Force model and the Anticipative Kinodynamic Planner to the case of an adaptive side-by-side navigation; secondly, we enhance side-by-side navigation with an approaching task and a final positioning that allows the robot to interact with both people; and finally, we use findings from experiments of real-life observations of people walking in pairs to define the parameters of the human-robot interaction in our case of adaptive side-by-side. The method was validated by a large set of simulations; we also conducted real-life experiments with our robot, Tibi, to validate the framework described for the interaction process. In addition, we carried out various surveys and user studies to indicate the social acceptability of the robot’s performance of the accompanying, approaching and positioning tasks., Work supported by the Spanish Ministry of Science and Innovationunder project ColRobTransp (DPI2016-78957-R. AEI/FEDER EU),by the Spanish State Research Agency through the Maria de MaeztuSeal of Excellence to IRI (MDM-2016-0656) and EU TERRINETproject (H2020-INFRAIA-2017-1-two-stage-730994). Ely Repiso isalso supported by Spanish Ministry of Science and Innovation under aFPI-grant (BES-2014-067713), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

EKF-based parameter identification of multi-rotor unmanned aerial vehiclesmodels

  • Munguía Alcalá, Rodrigo Francisco
  • Urzúa Sarquis, Isidro
  • Grau Saldes, Antoni
This work presents a method for estimating the model parameters of multi-rotor unmanned aerial vehicles by means of an extended Kalman filter. Different from test-bed based identification methods, the proposed approach estimates all the model parameters of a multi-rotor aerial vehicle, using a single online estimation process that integrates measurements that can be obtained directly from onboard sensors commonly available in this kind of UAV. In order to develop the proposed method, the observability property of the system is investigated by means of a nonlinear observability analysis. First, the dynamic models of three classes of multi-rotor aerial vehicles are presented. Then, in order to carry out the observability analysis, the state vector is augmented by considering the parameters to be identified as state variables with zero dynamics. From the analysis, the sets of measurements from which the model parameters can be estimated are derived. Furthermore, the necessary conditions that must be satisfied in order to obtain the observability results are given. An extensive set of computer simulations is carried out in order to validate the proposed method. According to the simulation results, it is feasible to estimate all the model parameters of a multi-rotor aerial vehicle in a single estimation process by means of an extended Kalman filter that is updated with measurements obtained directly from the onboard sensors. Furthermore, in order to better validate the proposed method, the model parameters of a custom-built quadrotor were estimated from actual flight log data. The experimental results show that the proposed method is suitable to be practically applied, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

People's V-formation and side-by-side model adapted to accompany groups of people by social robots

  • Repiso Polo, Ely
  • Zanlungo, Francesco
  • Kanda, Takayuki
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a new method to allow robots to accompany a person or a group of people imitating pedestrians behavior. Two-people groups usually walk in a side-by-side formation and three-people groups walk in a V-formation so that they can see each other. For this reason, the proposed method combines a Side-by-side and V-formation pedestrian model with the Anticipative Kinodynamic Planner (AKP). Combining these methods, the robot is able to do an anticipatory accompaniment of groups of humans, as well as to avoid static and dynamic obstacles in advance, while keeping the prescribed formations. The proposed framework allows also a dynamical re-positioning of the robot, if the physical position of the partners change in the group formation. Furthermore, people have a randomness factor that the robot has to manage, for that reason, the system was adapted to deal with changes in people's velocity, orientation and occlusions. Finally, the method has been validated using synthetic experiments and real-life experiments with our Tibi robot. In addition, a user study has been realized to reveal the social acceptability of the method., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

3DPeople: modeling the geometry of dressed humans

  • Pumarola Peris, Albert
  • Sánchez Riera, Jordi
  • Choi, Gary
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Recent advances in 3D human shape estimation build upon parametric representations that model very well the shape of the naked body, but are not appropriate to represent the clothing geometry. In this paper, we present an approach to model dressed humans and predict their geometry from single images. We contribute in three fundamental aspects of the problem, namely, a new dataset, a novel shape parameterization algorithm and an end-to-end deep generative network for predicting shape. First, we present 3DPeople, a large-scale synthetic dataset with 2.5 Million photo-realistic images of 80 subjects performing 70 activities and wearing diverse outfits. Besides providing textured 3D meshes for clothes and body, we annotate the dataset with segmentation masks, skeletons, depth, normal maps and optical flow. All this together makes 3DPeople suitable for a plethora of tasks., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Design and implementation of a virtual sensor network for smart waste water monitoring

  • Guerra Paradas, Edmundo
  • Bolea Monte, Yolanda
  • Gámiz Caro, Javier Francisco
  • Grau Saldes, Antoni
A high density virtual sensor network has been designed to robotize sampling tasks associated with monitoring of water treatment plant with open air basins. The designed virtual network is to be actually implemented by a network of autonomous UAV sensor platform, which was designed using a multi-agent system approach. Each UAV sensor platform can deploy a multiparametric sensor able to provide analyzed metrics in real-time, or a custom designed sample capture probe to be delivered to a laboratory through a special landing pad. This way, as the sensing hardware is deployed on-board UAVs, the system can analyze any point of the open air basins, operating as if an arbitrarily large amount of sensors or sampling points had been deployed through the installation. A general view of the implementation of the system is provided: in terms of hardware, the prototype platform built is described, with emphasis in the technical integration aspects; the software is mainly described in terms of the design principles used and how the chosen technologies support them to solve specific challenges. Evaluation of the components of the system built is performed independently: autonomous flight performance has been tested on-site; the accuracy of the localization technologies deemed as deployable options has been evaluated in controlled flights; and the viability of the sample capture device designed and built has been experimentally tested., This research has been funded by the Spanish Ministry of Science, with the project DPI2016-78957-R, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

People's adaptive side-by-side model evolved to accompany groups of people by social robots

  • Repiso Polo, Ely
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The presented method implements a robot accompaniment in an adaptive side-by-side formation of a single person or a group of people. The method enhances our previous robot adaptive side-by-side behavior allowing the robot to accompany a group of people, not only one person. Adaptive means that the robot is capable to adjust its position and velocity to the behavior of the group being accompanied, without bothering other pedestrians in the environment, as well as facilitating the group navigation to avoid static and dynamic obstacles. Furthermore, the robot can deal with the random factor of human behavior in several situations. Firstly, if other people interfere in the path of the companions, the robot leaves space for the person of the group that has to avoid those other people, by approaching the other companion. Also, without invading any personal space. Secondly, if the people of the group changes their physical position inside the group formation, then the robot adapts to them dynamically by changing from the lateral position to the central position of the formation or otherwise. Thirdly, the robot adapts to changes in the velocity of people in the group and other people that interfere in the path of the group, in magnitude and orientation. Fourthly, the robot can deal with occlusions of one accompanied person by the other. The method has been validated using synthetic experiments and real-life experiments with our robot. Finally, we developed an user study comparing the method with a Wizard of Oz., This work was supported in part by the Spanish Ministry of Science and Innovationunder Project ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet(H2020-INFRAIA-2017-1-two-stage-730994) and in part by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI(MDM-2016-0656). Ely Repiso is also supported by the Spanish Ministry ofScience and Innovation under a FPI-grant, BES-2014-067713., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Monocular visual SLAM based on a cooperative UAV–target system

  • Trujillo, Juan Carlos
  • Munguía Alcalá, Rodrigo Francisco
  • Sarquís Urzúa Cevallos, Isidro
  • Guerra Paradas, Edmundo
  • Grau Saldes, Antoni
To obtain autonomy in applications that involve Unmanned Aerial Vehicles (UAVs), the capacity of self-location and perception of the operational environment is a fundamental requirement. To this effect, GPS represents the typical solution for determining the position of a UAV operating in outdoor and open environments. On the other hand, GPS cannot be a reliable solution for a different kind of environments like cluttered and indoor ones. In this scenario, a good alternative is represented by the monocular SLAM (Simultaneous Localization and Mapping) methods. A monocular SLAM system allows a UAV to operate in a priori unknown environment using an onboard camera to simultaneously build a map of its surroundings while at the same time locates itself respect to this map. So, given the problem of an aerial robot that must follow a free-moving cooperative target in a GPS denied environment, this work presents a monocular-based SLAM approach for cooperative UAV–Target systems that addresses the state estimation problem of (i) the UAV position and velocity, (ii) the target position and velocity, (iii) the landmarks positions (map). The proposed monocular SLAM system incorporates altitude measurements obtained from an altimeter. In this case, an observability analysis is carried out to show that the observability properties of the system are improved by incorporating altitude measurements. Furthermore, a novel technique to estimate the approximate depth of the new visual landmarks is proposed, which takes advantage of the cooperative target. Additionally, a control system is proposed for maintaining a stable flight formation of the UAV with respect to the target. In this case, the stability of control laws is proved using the Lyapunov theory. The experimental results obtained from real data as well as the results obtained from computer simulations show that the proposed scheme can provide good performance., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Teaching a drone to accompany a person from demonstrations using non-linear ASFM

  • Garrell Zulueta, Anais
  • Coll Gomilla, Carles
  • Alquézar Mancho, René
  • Sanfeliu Cortés, Alberto
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper, we present a new method based on the Aerial Social Force Model (ASFM) to allow human-drone side-by-side social navigation in real environments. To tackle this problem, the present work proposes a new nonlinear-based approach using Neural Networks. To learn and test the rightness of the new approach, we built a new dataset with simulated environments and we recorded motion controls provided by a human expert tele-operating the drone. The recorded data is then used to train a neural network which maps interaction forces to acceleration commands. The system is also reinforced with a human path prediction module to improve the drone’s navigation, as well as, a collision detection module to completely avoid possible impacts. Moreover, a performance metric is defined which allows us to numerically evaluate and compare the fulfillment of the different learned policies. The method was validated by a large set of simulations; we also conducted real-life experiments with an autonomous drone to verify the framework described for the navigation process. In addition, a user study has been realized to reveal the social acceptability of the method., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Low resolution lidar-based multi object tracking for driving applications

  • del Pino Bastida, Iván
  • Vaquero Gómez, Víctor
  • Massini, Beatrice
  • Solà Ortega, Joan
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
  • Andrade-Cetto, Juan
The final publication is available at link.springer.com, Vehicle detection and tracking in real scenarios are key com- ponents to develop assisted and autonomous driving systems. Lidar sen- sors are specially suitable for this task, as they bring robustness to harsh weather conditions while providing accurate spatial information. How- ever, the resolution provided by point cloud data is very scarce in com- parison to camera images. In this work we explore the possibilities of Deep Learning (DL) methodologies applied to low resolution 3D lidar sensors such as the Velodyne VLP-16 (PUCK), in the context of vehicle detection and tracking. For this purpose we developed a lidar-based sys- tem that uses a Convolutional Neural Network (CNN), to perform point- wise vehicle detection using PUCK data, and Multi-Hypothesis Extended Kalman Filters (MH-EKF), to estimate the actual position and veloci- ties of the detected vehicles. Comparative studies between the proposed lower resolution (VLP-16) tracking system and a high-end system, using Velodyne HDL-64, were carried out on the Kitti Tracking Benchmark dataset. Moreover, to analyze the influence of the CNN-based vehicle detection approach, comparisons were also performed with respect to the geometric-only detector. The results demonstrate that the proposed low resolution Deep Learning architecture is able to successfully accom- plish the vehicle detection task, outperforming the geometric baseline approach. Moreover, it has been observed that our system achieves a similar tracking performance to the high-end HDL-64 sensor at close range. On the other hand, at long range, detection is limited to half the distance of the higher-end sensor., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Temporal segmentation of human actions in video sequences

  • Carmona Leyva, José María
  • Climent Vilaró, Joan
Most of the published works concerning action recognition, usually assume that the action sequences have been previously segmented in time, that is, the action to be recognized starts with the first sequence frame and ends with the last one. However, temporal segmentation of actions in sequences is not an easy task, and is always prone to errors. In this paper, we present a new technique to automatically extract human actions from a video sequence. Our approach presents several contributions. First of all, we use a projection template scheme and find spatio-temporal features and descriptors within the projected surface, rather than extracting them in the whole sequence. For projecting the sequence we use a variant of the R transform, which has never been used before for temporal action segmentation. Instead of projecting the original video sequence, we project its optical flow components, preserving important information about action motion. We test our method on a publicly available action dataset, and the results show that it performs very well segmenting human actions compared with the state-of-the-art methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Towards robust 6-DoF detection in uncontrolled lightning enviroments

  • Mora, Alejandro
  • Guerra Paradas, Edmundo
  • Manzanares Brotons, Manuel
  • Grau Saldes, Antoni
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., One of the most critical issues that arises when controlling a robot is the necessity to locate itself in the environment. Ranging from industrial processes applications to outdoor mobile robots, landmarks are used to such purpose: by recognizing them, the robots are able to know where they are with the aid of computer vision. Fiducial markers are the cheapest and one of the most common solutions to this issue: 2D planar patterns that embed some information that can be identified using artificial vision techniques. Many different typologies of markers as well as image processing algorithms are being implemented nowadays, using C++ / Python / MATLAB® libraries and ROS as middleware. In this work we have evaluated the robustness of various fiducial marker typologies, documenting the main technical aspects of the used implementations and commenting how each algorithm operates. Aggregated results are presented and discussed, evaluating the different implementations tested., This research was funded by the Ministry of Economy, Industry and Competitiveness through project DPI2016-78957-R., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Deconvolutional networks for point-cloud vehicle detection and tracking in driving scenarios

  • Vaquero Gómez, Víctor
  • del Pino Bastida, Iván
  • Moreno-Noguer, Francesc
  • Solà Ortega, Joan
  • Sanfeliu Cortés, Alberto
  • Andrade-Cetto, Juan
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Vehicle detection and tracking is a core ingredient for developing autonomous driving applications in urban scenarios. Recent image-based Deep Learning (DL) techniques are obtaining breakthrough results in these perceptive tasks. However, DL research has not yet advanced much towards processing 3D point clouds from lidar range-finders. These sensors are very common in autonomous vehicles since, despite not providing as semantically rich information as images, their performance is more robust under harsh weather conditions than vision sensors. In this paper we present a full vehicle detection and tracking system that works with 3D lidar information only. Our detection step uses a Convolutional Neural Network (CNN) that receives as input a featured representation of the 3D information provided by a Velodyne HDL-64 sensor and returns a per-point classification of whether it belongs to a vehicle or not. The classified point cloud is then geometrically processed to generate observations for a multi-object tracking system implemented via a number of Multi-Hypothesis Extended Kalman Filters (MH-EKF) that estimate the position and velocity of the surrounding vehicles. The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach. Our lidar-based approach uses about a 4% of the data needed for an image-based detector with similarly competitive results., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Geometry-aware network for non-rigid shape prediction from a single view

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Porzi, Lorenzo
  • Sanfeliu Cortés, Alberto
  • Lepetit, Vincent
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a {it geometry-aware} deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Online learning and detection of faces with low human supervision

  • Villamizar Vergel, Michael Alejandro
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, We present an efficient,online,and interactive approach for computing a classifier, called Wild Lady Ferns (WiLFs), for face learning and detection using small human supervision. More precisely, on the one hand, WiLFs combine online boosting and extremely randomized trees (Random Ferns) to compute progressively an efficient and discriminative classifier. On the other hand, WiLFs use an interactive human-machine approach that combines two complementary learning strategies to reduce considerably the degree of human supervision during learning. While the first strategy corresponds to query-by-boosting active learning, that requests human assistance over difficult samples in function of the classifier confidence, the second strategy refers to a memory-based learning which uses ¿ Exemplar-based Nearest Neighbors (¿ENN) to assist automatically the classifier. A pre-trained Convolutional Neural Network (CNN) is used to perform ¿ENN with high-level feature descriptors. The proposed approach is therefore fast (WilFs run in 1 FPS using a code not fully optimized), accurate (we obtain detection rates over 82% in complex datasets), and labor-saving (human assistance percentages of less than 20%). As a byproduct, we demonstrate that WiLFs also perform semi-automatic annotation during learning, as while the classifier is being computed, WiLFs are discovering faces instances in input images which are used subsequently for training online the classifier. The advantages of our approach are demonstrated in synthetic and publicly available databases, showing comparable detection rates as offline approaches that require larger amounts of handmade training data., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Vision-based SLAM system for MAVs in GPS-denied environments

  • Urzua, Sarquis
  • Munguía Alcalá, Rodrigo Francisco
  • Grau Saldes, Antoni
Using a camera, a micro aerial vehicle (MAV) can perform visual-based navigation in periods or circumstances when GPS is not available, or when it is partially available. In this context, the monocular simultaneous localization and mapping (SLAM) methods represent an excellent alternative, due to several limitations regarding to the design of the platform, mobility and payload capacity that impose considerable restrictions on the available computational and sensing resources of the MAV. However, the use of monocular vision introduces some technical difficulties as the impossibility of directly recovering the metric scale of the world. In this work, a novel monocular SLAM system with application to MAVs is proposed. The sensory input is taken from a monocular downward facing camera, an ultrasonic range finder and a barometer. The proposed method is based on the theoretical findings obtained from an observability analysis. Experimental results with real data confirm those theoretical findings and show that the proposed method is capable of providing good results with low-cost hardware., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Human-robot collaborative navigation search using social reward sources

  • Dalmasso Blanch, Marc
  • Garrell Zulueta, Anais
  • Jiménez Schlegl, Pablo
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, This paper proposes a Social Reward Sources (SRS) design for a Human-Robot Collaborative Navigation (HRCN) task: human-robot collaborative search. It is a flexible approach capable of handling the collaborative task, human-robot interaction and environment restrictions, all integrated on a common environment. We modelled task rewards based on unexplored area observability and isolation and evaluated the model through different levels of human-robot communication.The models are validated through quantitative evaluation against both agents’ individual performance and qualitative surveying of participants’ perception. After that, the three proposed communication levels are com-pared against each other using the previous metrics., Work supported under projects ColRobTransp (DPI2016-78957-RAEI/FEDER EU), TERRINet (H2020-INFRAIA-2017-1-two-stage-730994) and by the Spanish State Research Agency through the Maria de Maeztu Seal of Excellence to IRI (MDM-2016-0656)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Considering the anchoring problem in robotic intelligent bin picking

  • Escudero Rodrigo, Diego
  • Alquézar Mancho, René
Random Bin Picking means the selection by a robot of a particular item from a container (or bin) in which there are many items randomly distributed. Generalist robots and the Anchoring Problem should be considered if we want to provide a more general solution, since users want that it works with different type of items that are not known 'a priori'. Therefore, we are working on an approach in which robot learning and human-robot interaction are used to anchor control primitives and robot skills to objects and action symbols while the robot system is running, but we are limiting the scope to the packaging domain. In this paper we explain how to use our system to do anchoring in Robotic Bin Picking., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Hallucinating dense optical flow from sparse lidar for autonomous vehicles

  • Vaquero Gómez, Víctor
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle. This is intended to be used as a drop-in replacement of any image-based optical flow system when images are not reliable due to e.g. adverse weather conditions or at night. In order to infer high resolution 2D flows from discrete range data we devise a three-block architecture of multiscale filters that combines multiple intermediate objectives, both in the lidar and image domain. To train this network we introduce a dataset with approximately 20K lidar samples of the Kitti dataset which we have augmented with a pseudo ground-truth image-based optical flow computed using FlowNet2. We demonstrate the effectiveness of our approach on Kitti, and show that despite using the low-resolution and sparse measurements of the lidar, we can regress dense optical flow maps which are at par with those estimated with image-based methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

GANimation: anatomically-aware facial animation from a single image

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Martinez, Aleix M.
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild., Peer Reviewed, Award-winning, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Image processing techniques in vision-based robot navigation

  • Munguía Alcalá, Rodrigo Francisco
  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
Motivated by the problems of vision-based mobile robot map building and localization, we present a comparative study of statistical methods for matching image features in a wide base line between learning and recognition phases. A general methodology called feature-class method for the problem of fast matching image features in a wide base line is described in the context of mobile robots. The objective of this work is to discuss and to show the performance of such methods in an example of visual SLAM, with experiments done with real data., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic regulation of wastewater discharge

  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
Wastewater plants, mainly with secondary treatments, discharge polluted water to environment that cannot be used in any human activity. When those dumps are in the sea it is expected that most of the biological pollutants die or almost disappear before water reaches human range. This natural withdrawal of bacteria, viruses and other pathogens is due to some conditions such as the salt water of the sea and the sun effect, and the dumps areas are calculated taking into account these conditions. However, under certain meteorological phenomena water arrives to the coast without the full disappearance of pollutant elements. In Mediterranean Sea there are some periods of adverse climatic conditions that pollute the coast near the wastewater dumping. In this paper, authors present an automatic control that prevents such pollution episodes using two mathematical models, one for the pollutant transportation and the other for the pollutant removal in wastewater spills, Peer Reviewed, Postprint (author's final draft)