Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 50
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Los primeros robots manipuladores aéreos

  • Ollero Baturone, Aníbal
  • Viguria, Antidio
  • Maza, Ivan
  • Trujillo Soto, Miguel Ángel
  • Heredia, Guillermo
  • Cano, Raúl
  • Caballero, Fernando
  • Lasagni, Fernando
  • Acosta, J. Á.
  • Santos, Silvia de los
  • Cobano-Suárez, José-Antonio
[Resumen] En este artículo se describen los primeros robots manipuladores aéreos dotados con brazos robóticos de seis grados de libertad y capacidades de percepción y planificación que se han desarrollado en el mundo. Estos robots son el resultado del proyecto ARCAS (Aerial Robotics Cooperative Assembly System) coordinado por el primero de los autores y financiado por el Séptimo Programa Marco de la Unión Europea. Los resultados de este proyecto han comenzado a emplearse, entre otros, en el proyecto H2020 AEROARMS y en el español AEROMAIN. El artículo introduce las diferentes plataformas utilizadas en el proyecto y a continuación resume las principales características de los sistemas de control, percepción y planificación desarrollados. Se resume también la integración y validación del sistema., Este trabajo ha sido financiado parcialmente por el Proyecto Europeo del FP7 ARCAS (ICT-2011-287617) y por el H2020 AEROARMS (ICT-23-2014-644271), así como por el proyecto nacional AEROMAIN DPI2014-59383-C2-1-R del Ministerio de Economía y Competitividad, https://doi.org/10.17979/spudc.9788497498081
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Discriminative learning of deep convolutional feature point descriptors

  • Simo-Serra, Edgar
  • Trulls, Eduard
  • Ferraz, Luis
  • Kokkinos, Iasonas
  • Fua, Pascal
  • Moreno-Noguer, Francesc
Trabajo presentado a la International Conference on Computer Vision celebrada en Santiago (España) del 7 al 13 de diciembre de 2015., Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available., This work was partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project ViSen PCIN-2013-047, by EU projects AEROARMS H2020-ICT-2014-1-644271, ISUPPORT H2020-ICT-2014-1-643666 and MOBOT FP7-ICT-2011-600796, and by the ERC project MicroNano., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Matchability prediction for full-search template matching algorithms

  • Peñate-Sánchez, Adrián
  • Porzi, Lorenzo
  • Moreno-Noguer, Francesc
Trabajo presentado a la International Conference on 3D Vision, celebrada en Lyon (Francia) del 19 al 22 de octubre de 2015., While recent approaches have shown that it is possible to do template matching by exhaustively scanning the parameter space, the resulting algorithms are still quite demanding. In this paper we alleviate the computational load of these algorithms by proposing an efficient approach for predicting the match ability of a template, before it is actually performed. This avoids large amounts of unnecessary computations. We learn the match ability of templates by using dense convolutional neural network descriptors that do not require ad-hoc criteria to characterize a template. By using deep learning descriptions of patches we are able to predict match ability over the whole image quite reliably. We will also show how no specific training data is required to solve problems like panorama stitching in which you usually require data from the scene in question. Due to the highly parallelizable nature of this tasks we offer an efficient technique with a negligible computational cost at test time., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under project RobInstruct TIN2014-58178-R and ERA-Net Chistera project ViSen PCIN-2013-047; and by the EU project AEROARMS H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

BASS: Boundary-aware superpixel segmentation

  • Rubio Romano, Antonio
  • Yu, LongLong
  • Simo-Serra, Edgar
  • Moreno-Noguer, Francesc
Trabajo presentado a la 23rd International Conference on Pattern Recognition, celebrada en Cancún (México) del 5 al 8 de diciembre de 2016., We propose a new superpixel algorithm based on exploiting the boundary information of an image, as objects in images can generally be described by their boundaries. Our proposed approach initially estimates the boundaries and uses them to place superpixel seeds in the areas in which they are more dense. Afterwards, we minimize an energy function in order to expand the seeds into full superpixels. In addition to standard terms such as color consistency and compactness, we propose using the geodesic distance which concentrates small superpixels in regions of the image with more information, while letting larger superpixels cover more homogeneous regions. By both improving the initialization using the boundaries and coherency of the superpixels with geodesic distances, we are able to maintain the coherency of the image structure with fewer superpixels than other approaches. We show the resulting algorithm to yield smaller Variation of Information metrics in seven different datasets while maintaining Undersegmentation Error values similar to the state-of-the-art methods., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147 and by the EU project AEROARMS H2020-ICT-2014-1-644271. A. Rubio is supported by the industrial doctorate grant 2015-DI-010 of the AGAUR., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Learning the hidden human knowledge of UAV pilots when navigating in a cluttered environament for improving Path Plannig

  • Alzugaray, Ignacio
  • Sanfeliu, Alberto
  • Sanfeliu, Alberto
Trabajo presentado a la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Daejeon (Korea) del 9 al 14 de octubre de 2016., We propose in this work a new model of how the hidden human knowledge (HHK) of UAV pilots can be incorporated in the UAVs path planning generation. We intuitively know that human's pilots barely manage or even attempt to drive the UAV through a path that is optimal attending to some criteria as an optimal planner would suggest. Although human pilots might get close but not reach the optimal path proposed by some planner that optimizes over time or distance, the final effect of this differentiation could be not only surprisingly better, but also desirable. In the best scenario for optimality, the path that human pilots generate would deviate from the optimal path as much as the hidden knowledge that its perceives is injected into the path. The aim of our work is to use real human pilot paths to learn the hidden knowledge using repulsion fields and to incorporate this knowledge afterwards in the environment obstacles as cause of the deviation from optimality. We present a strategy of learning this knowledge based on attractor and repulsors, the learning method and a modified RRT* that can use this knowledge for path planning. Finally we do real-life tests and we compare the resulting paths with and without this knowledge., This work was supported by the Spanish Ministry of Science and Innovation, project Rob-Int-Coop DPI2013-42458-P and AEROARMS European project H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Nonlinear model predictive control for aerial manipulation

  • Lunni, Dario
  • Santamaria-Navarro, Àngel
  • Rossi, Roberto
  • Rocco, Paolo
  • Bascetta, Luca
  • Andrade-Cetto, Juan
Trabajo presentado a la International Conference on Unmanned Aircraft Systems (ICUAS), celebrada en Miami (US) del 13 al 16 de junio de 2017., This paper presents a nonlinear model predictive controller to follow desired 3D trajectories with the end effector of an unmanned aerial manipulator (i.e., a multirotor with a serial arm attached). To the knowledge of the authors, this is the first time that such controller runs online and on board a limited computational unit to drive a kinematically augmented aerial vehicle. Besides the trajectory following target, we explore the possibility of accomplishing other tasks during flight by taking advantage of the system redundancy. We define several tasks designed for aerial manipulators and show in simulation case studies how they can be achieved by either a weighting strategy, within a main optimization process, or a hierarchical approach consisting on nested optimizations. Moreover, experiments are presented to demonstrate the performance of such controller in a real robot., Their work was partially funded by the EU project AEROARMS H2020-ICT-2014-1-644271, and by the Spanish Ministry of Economy and Competitiveness project ROBINSTRUCT TIN2014-58178-R., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

A flexible hardware-in-the-loop architecture for UAVs

  • Lepej, Peter
  • Santamaria-Navarro, Àngel
  • Solà, Joan
Trabajo presentado a la International Conference on Unmanned Aircraft Systems (ICUAS), celebrada en Miami (US) del 13 al 16 de junio de 2017., As robotic technology matures, fully autonomous robots become a realistic possibility, but demand very complex solutions to be rapidly engineered. In order to be able to quickly set up a working autonomous system, and to reduce the gap between simulated and real experiments, we propose a modular, upgradeable and flexible hardware-in-the-loop (HIL) architecture, which hybridizes the simulated and real settings. We take as use case the autonomous exploration of dense forests with UAVs, with the aim of creating useful maps for forest inspection, cataloging, or to compute other metrics such as total wood volume. As the first step in the development of the full system, in this paper we implement a fraction of this architecture, comprising assisted localization, and automatic methods for mapping, planning and motion execution. Specifically we are able to simulate the use of a 3D LIDAR endowed below an actual UAV autonomously navigating among simulated obstacles, thus the platform safety is not compromised. The full system is modular and takes profit of pieces either publicly available or easily programmed. We highlight the flexibility of the proposed HIL architecture to rapidly configure different experimental setups with a UAV in challenging terrain. Moreover, it can be extended to other robotic fields without further design. The HIL system uses the multi-platform ROS capabilities and only needs a motion capture system as external extra hardware, which is becoming standard equipment in all research labs dealing with mobile robots., This work has been funded by the EU project AEROARMS H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

PL-SLAM: Real-time monocular visual SLAM with points and lines

  • Pumarola, Albert
  • Vakhitov, Alexander
  • Agudo Martínez, Antonio
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE International Conference on Robotics and Automation (ICRA), celebrada en Singapur del 29 de mayo al 3 de junio de 2017., Low textured scenes are well known to be one of the main Achilles heels of geometric computer vision algorithms relying on point correspondences, and in particular for visual SLAM. Yet, there are many environments in which, despite being low textured, one can still reliably estimate line-based geometric primitives, for instance in city and indoor scenes, or in the so-called ``Manhattan worlds'', where structured edges are predominant. In this paper we propose a solution to handle these situations. Specifically, we build upon ORB-SLAM, presumably the current state-of-the-art solution both in terms of accuracy as efficiency, and extend its formulation to simultaneously handle both point and line correspondences. We propose a solution that can even work when most of the points are vanished out from the input images, and, interestingly it can be initialized from solely the detection of line correspondences in three consecutive frames. We thoroughly evaluate our approach and the new initialization strategy on the TUM RGB-D benchmark and demonstrate that the use of lines does not only improve the performance of the original ORB-SLAM solution in poorly textured frames, but also systematically improves it in sequence frames combining points and lines, without compromising the efficiency., This work has been partially supported by the EU project AEROARMS H2020-ICT-2014-1-644271, by the MINECO projects RobInstruct TIN2014-58178-R and Rob-Int-Coop DPI2013-42458-P, by the ERA-Net Chistera project IDRESS PCIN-2015-147 and by the Russian MES grant RFMEFI61516X0003., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Depth-aware convolutional neural networks for accurate 3D pose estimation in RGB-D images

  • Porzi, Lorenzo
  • Peñate-Sánchez, Adrián
  • Ricci, Elisa
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Vancouver (Canada) del 24 al 28 de septiembre de 2017., Most recent approaches to 3D pose estimation from RGB-D images address the problem in a two-stage pipeline. First, they learn a classifier-typically a random forest-to predict the position of each input pixel on the object surface. These estimates are then used to define an energy function that is minimized w.r.t. the object pose. In this paper, we focus on the first stage of the problem and propose a novel classifier based on a depth-aware Convolutional Neural Network. This classifier is able to learn a scale-adaptive regression model that yields very accurate pixel-level predictions, allowing to finally estimate the pose using a simple RANSAC-based scheme, with no need to optimize complex ad hoc energy functions. Our experiments on publicly available datasets show that our approach achieves remarkable improvements over state-of-the-art methods., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147, by the EU project AEROARMS H2020-ICT-2014-1-644271, by the EU project SECOND HANDS H2020-ICT-2014-1-643950 and by the Spanish State Research Agency through the Mara de Maeztu Seal of Excellence to IRI MDM-2016-0656., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Aerial social force model: A new framework to accompany people using autonomous flying robots

  • Garrell, Anaís
  • Garza Elizondo, Luis Alberto
  • Villamizar, Michael
  • Herrero, Fernando
  • Sanfeliu, Alberto
Trabajo presentado a la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Vancouver (Canada) del 24 al 28 de septiembre de 2017., In this paper, we proposed a novel Aerial Social Force Model (ASFM) that allows autonomous flying robots to accompany humans in urban environments in a safe and comfortable manner. To date, we are not aware of other state-of-the-art method that accomplish this task. The proposed approach is a 3D version of the Social Force Model (SFM) for the field of aerial robots which includes an interactive human-robot navigation scheme capable of predicting human motions and intentions so as to safely accompany them to their final destination. ASFM also introduces a new metric to fine-tune the parameters of the force model, and to evaluate the performance of the aerial robot companion based on comfort and distance between the robot and humans. The presented approach is extensively validated in diverse simulations and real experiments, and compared against other similar works in the literature. ASFM attains remarkable results and shows that it is a valuable framework for social robotics applications, such as guiding people or human-robot interaction., This work is supported by the Spanish State Research Agency through the Mara de Maeztu Seal of Excellence to IRI MDM-2016-0656. This work was supported by the Spanish Ministry of Science and Innovation, project ColRobTransp DPI2016-78957-R (AEI/FEDER, UE) and AEROARMS European project H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Multi-modal joint embedding for fashion product retrieval

  • Rubio Romano, Antonio
  • Yu, LongLong
  • Simo-Serra, Edgar
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE International Conference on Image Processing (ICIP), celebrada en Beijing (China) del 17 al 20 de septiembre de 2017., Finding a product in the fashion world can be a daunting task. Everyday, e-commerce sites are updating with thousands of images and their associated metadata (textual information), deepening the problem, akin to finding a needle in a haystack. In this paper, we leverage both the images and textual meta-data and propose a joint multi-modal embedding that maps both the text and images into a common latent space. Distances in the latent space correspond to similarity between products, allowing us to effectively perform retrieval in this latent space, which is both efficient and accurate. We train this embedding using large-scale real world e-commerce data by both minimizing the similarity between related products and using auxiliary classification networks to that encourage the embedding to have semantic meaning. We compare against existing approaches and show significant improvements in retrieval tasks on a large-scale e-commerce dataset. We also provide an analysis of the different metadata., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147 and by the EU project iAEROARMS H2020-ICT-2014-1-644271. A.Rubio is supported by the industrial doctorate grant 2015-DI-010 of the AGAUR., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Joint coarse-and-fine reasoning for deep optical flow

  • Vaquero, Victor
  • Ros, German
  • Moreno-Noguer, Francesc
  • López, Antonio M.
  • Sanfeliu, Alberto
Trabajo presentado a la IEEE International Conference on Image Processing (ICIP), celebrada en Beijing (China) del 17 al 20 de septiembre de 2017., We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets., This work was partially supported by European AEROARMS project (H2020-ICT-2014-1-644271) and CICYT projects ColRobTransp (DPI2016-78957-R), ROBINSTRUCT (TIN2014-58178-R)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Planar PØP: Feature-less pose estimation with applications in UAV localization

  • Amor-Martinez, Adrian
  • Santamaria-Navarro, Àngel
  • Herrero, Fernando
  • Ruiz Moreno, Alberto
  • Sanfeliu, Alberto
Trabajo presentado al IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), celebrado en Lausanne (Suiza) del 23 al 27 de octubre de 2016., We present a featureless pose estimation method that, in contrast to current Perspective-n-Point (PnP) approaches, it does not require n point correspondences to obtain the camera pose, allowing for pose estimation from natural shapes that do not necessarily have distinguished features like corners or intersecting edges. Instead of using n correspondences (e.g. extracted with a feature detector) we will use the raw polygonal representation of the observed shape and directly estimate the pose in the pose-space of the camera. This method compared with a general PnP method, does not require n point correspondences neither a priori knowledge of the object model (except the scale), which is registered with a picture taken from a known robot pose. Moreover, we achieve higher precision because all the information of the shape contour is used to minimize the area between the projected and the observed shape contours. To emphasize the non-use of n point correspondences between the projected template and observed contour shape, we call the method Planar PØP. The method is shown both in simulation and in a real application consisting on a UAV localization where comparisons with a precise ground-truth are provided., This work has been partially funded by the AEROARMS EU project H2020-ICT-2014-1-644271, the CICYT project DPI2013-42458-P, and by Spanish MINECO and EU FEDER grant TIN2015-66972-C5-3-R., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Geometry-aware network for non-rigid shape prediction from a single view

  • Pumarola, Albert
  • Agudo Martínez, Antonio
  • Porzi, Lorenzo
  • Sanfeliu, Alberto
  • Lepetit, Vincent
  • Moreno-Noguer, Francesc
Trabajo presentado en la IEEE/CVF Conference on Computer Vision and Pattern Recognition, celebrada en Salt Lake City (UT, USA), del 18 al 23 de junio de 2018, We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a geometry-aware deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time., This work is supported in part by a Google Faculty Research Award, by the Spanish Ministry of Science and Innovation under projects HuMoUR TIN2017- 90086-R, ColRobTransp DPI2016-78957 and Mar´ıa de Maeztu Seal of Excellence MDM-2016-0656; and by the EU project AEROARMS ICT-2014-1-644271. We also thank Nvidia for hardware donation., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Unsupervised person image synthesis in arbitrary poses

  • Pumarola, Albert
  • Agudo Martínez, Antonio
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la IEEE/CVF Conference on Computer Vision and Pattern Recognition, celebrada en Salt Lake City (UT, USA), del 18 al 23 de junio de 2018, We present a novel approach for synthesizing photorealistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner [16, 35], i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches., This work is supported in part by a Google Faculty Research Award, by the Spanish Ministry of Science and Innovation under projects HuMoUR TIN2017- 90086-R, ColRobTransp DPI2016-78957 and Mar´ıa de Maeztu Seal of Excellence MDM-2016-0656; and by the EU project AEROARMS ICT-2014-1-644271. We also thank Nvidia for hardware donation under the GPU Grant Program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

GANimation: Anatomically-aware facial animation from a single image

  • Pumarola, Albert
  • Agudo Martínez, Antonio
  • Martínez, Aleix M.
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado en la 15th European Conference, celebrada en Munich, (Germany) del 8 al 14 de septiembre de 2018, Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs’ generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild., This work is partially supported by the Spanish Ministry of Economy and Competitiveness under projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957 and María de Maeztu Seal of Excellence MDM-2016-0656; by the EU project AEROARMS ICT-2014-1-644271; and by the Grant R01-DC- 014498 of the National Institute of Health. We also thank Nvidia for hardware donation under the GPU Grant Program., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Unsupervised person image synthesis in arbitrary poses

  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • Agudo Martínez, Antonio|||0000-0001-6845-4998
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

BASS: boundary-aware superpixel segmentation

  • Rubio Romano, Antonio
  • Yu, Longlong
  • Simó Serra, Edgar
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We propose a new superpixel algorithm based on exploiting the boundary information of an image, as objects in images can generally be described by their boundaries. Our proposed approach initially estimates the boundaries and uses them to place superpixel seeds in the areas in which they are more dense. Afterwards, we minimize an energy function in order to expand the seeds into full superpixels. In addition to standard terms such as color consistency and compactness, we propose using the geodesic distance which concentrates small superpixels in regions of the image with more information, while letting larger superpixels cover more homogeneous regions. By both improving the initialization using the boundaries and coherency of the superpixels with geodesic distances, we are able to maintain the coherency of the image structure with fewer superpixels than other approaches. We show the resulting algorithm to yield smaller Variation of Information metrics in seven different datasets while maintaining Undersegmentation Error values similar to the state-of-the-art methods., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

A flexible hardware-in-the-loop architecture for UAVs

  • Lepej, Peter
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Solà, Joan
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., As robotic technology matures, fully autonomous robots become a realistic possibility, but demand very complex solutions to be rapidly engineered. In order to be able to quickly set up a working autonomous system, and to reduce the gap between simulated and real experiments, we propose a modular, upgradeable and flexible hardware-in-the-loop (HIL) architecture, which hybridizes the simulated and real settings. We take as use case the autonomous exploration of dense forests with UAVs, with the aim of creating useful maps for forest inspection, cataloging, or to compute other metrics such as total wood volume. As the first step in the development of the full system, in this paper we implement a fraction of this architecture, comprising assisted localization, and automatic methods for mapping, planning and motion execution. Specifically we are able to simulate the use of a 3D LIDAR endowed below an actual UAV autonomously navigating among simulated obstacles, thus the platform safety is not compromised. The full system is modular and takes profit of pieces either publicly available or easily programmed. We highlight the flexibility of the proposed HIL architecture to rapidly configure different experimental setups with a UAV in challenging terrain. Moreover, it can be extended to other robotic fields without further design. The HIL system uses the multi-platform ROS capabilities and only needs a motion capture system as external extra hardware, which is becoming standard equipment in all research labs dealing with mobile robots., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Aerial robotics in building inspection and maintenance

  • Grau Saldes, Antoni|||0000-0003-4112-3325
  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Puig-Pey Clavería, Ana María|||0000-0002-6073-7448
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
Buildings need periodic revision about their state, materials degrade with time and repairs or renewals have to be made driven by maintenance needs or safety requirements. That happens with any kind of buildings and constructions: housing, architecture masterpieces, old and ancient buildings and industrial buildings. Currently, nearly all of these tasks are carried out by human intervention. In order to carry out the inspection or maintenance, humans need to access to roofs, façades or other areas hard to reach and otherwise potentially hazardous location to perform the task. In some cases, it might not be feasible to access for inspection. For instance, in industry buildings operation must be often interrupted to allow for safe execution of such tasks; these shutdowns not only lead to substantial production loss, but the shutdown and start-up operation itself causes risks to human and environment. In touristic buildings, access has to be restricted with the consequent losses and inconveniences to visitors. The use of aerial robots can help to perform this kind of hazardous operations in an autonomous way, not only teleoperated. Robots are able to carry sensors to detect failures of many types and to locate them in a previously generated map, which the robot uses to navigate. Some of those sensors are cameras in different spectra (visual, near-infrared, UV), laser, LIDAR, ultrasounds and inertial sensory system. If the sensory part is crucial to inspect hazardous areas in buildings, the actuation is also important: the aerial robot can carry small robots (mainly crawler) to be deployed to perform more in-depth operation where the contact between the sensors and the material is basic (any kind of metallic part: pipes, roofs, panels…). The aerial robot has the ability to recover the deployed small crawler to be reused again. In this paper, authors will explain the research that they are conducting in this area and propose future research areas and applications with aerial, ground, submarine and other autonomous robots within the construction field., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Multi-modal joint embedding for fashion product retrieval

  • Rubio Romano, Antonio
  • Yu, Longlong
  • Simó Serra, Edgar
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Finding a product in the fashion world can be a daunting task. Everyday, e-commerce sites are updating with thousands of images and their associated metadata (textual information), deepening the problem, akin to finding a needle in a haystack. In this paper, we leverage both the images and textual meta-data and propose a joint multi-modal embedding that maps both the text and images into a common latent space. Distances in the latent space correspond to similarity between products, allowing us to effectively perform retrieval in this latent space, which is both efficient and accurate. We train this embedding using large-scale real world e-commerce data by both minimizing the similarity between related products and using auxiliary classification networks to that encourage the embedding to have semantic meaning. We compare against existing approaches and show significant improvements in retrieval tasks on a large-scale e-commerce dataset. We also provide an analysis of the different metadata., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Nonlinear model predictive control for aerial manipulation

  • Lunni, Dario
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Rossi, Roberto
  • Rocco, Paolo
  • Bascetta, Luca
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a nonlinear model predictive controller to follow desired 3D trajectories with the end effector of an unmanned aerial manipulator (i.e., a multirotor with a serial arm attached). To the knowledge of the authors, this is the first time that such controller runs online and on board a limited computational unit to drive a kinematically augmented aerial vehicle. Besides the trajectory following target, we explore the possibility of accomplishing other tasks during flight by taking advantage of the system redundancy. We define several tasks designed for aerial manipulators and show in simulation case studies how they can be achieved by either a weighting strategy, within a main optimization process, or a hierarchical approach consisting on nested optimizations. Moreover, experiments are presented to demonstrate the performance of such controller in a real robot., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Robotic monocular SLAM in wastewater treatment plants with a sampling device

  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Munguía Alcalá, Rodrigo Francisco
  • Grau Saldes, Antoni|||0000-0003-4112-3325
The final publication is available at link.springer.com, A novel application is presented in this paper. Wastewater treatment plants need to sample water to follow the cleaned process continuously. This task is very tedious for laboratory workers and, mainly, can be done in reduced areas of the large basins that form the whole plant, specifically only in the edges of those basins. The new proposal is to enlarge the sampling area and reduce the load for workers. A new unmanned aerial vehicle has been designed together with a novel tool to take water samples. Moreover, a new architecture for the mission plan is presented in form of multi-agents. Experiments have been done in order to test the feasibility of the proposal., This research has been funded by AEROARMS EU Project H2020-ICT-2014-1-644271, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

GANimation: anatomically-aware facial animation from a single image

  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • Agudo Martínez, Antonio|||0000-0001-6845-4998
  • Martinez, Aleix M.
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild., Peer Reviewed, Award-winning
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Geometry-aware network for non-rigid shape prediction from a single view

  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • Agudo Martínez, Antonio|||0000-0001-6845-4998
  • Porzi, Lorenzo
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Lepetit, Vincent
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a {it geometry-aware} deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Matchability prediction for full-search template matching algorithms

  • Peñate Sánchez, Adrián
  • Porzi, Lorenzo
  • Moreno-Noguer, Francesc
While recent approaches have shown that it is possible to do template matching by exhaustively scanning the parameter space, the resulting algorithms are still quite demanding. In this paper we alleviate the computational load of these algorithms by proposing an efficient approach for predicting the match ability of a template, before it is actually performed. This avoids large amounts of unnecessary computations. We learn the match ability of templates by using dense convolutional neural network descriptors that do not require ad-hoc criteria to characterize a template. By using deep learning descriptions of patches we are able to predict match ability over the whole image quite reliably. We will also show how no specific training data is required to solve problems like panorama stitching in which you usually require data from the scene in question. Due to the highly parallelizable nature of this tasks we offer an efficient technique with a negligible computational cost at test time., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Aerial social force model: a new framework to accompany people using autonomous flying robots

  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Garza Elizondo, Luís Alberto
  • Villamizar Vergel, Michael Alejandro
  • Herrero Cotarelo, Fernando|||0000-0003-4116-0265
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We proposed a novel Aerial Social Force Model (ASFM) that allows autonomous flying robots to accompany humans in urban environments in a safe and comfortable manner. To date, we are not aware of other state-of-the-art method that accomplish this task. The proposed approach is a 3D version of the Social Force Model (SFM) for the field of aerial robots which includes an interactive human-robot navigation scheme capable of predicting human motions and intentions so as to safely accompany them to their final destination. ASFM also introduces a new metric to fine-tune the parameters of the force model, and to evaluate the performance of the aerial robot companion based on comfort and distance between the robot and humans. The presented approach is extensively validated in diverse simulations and real experiments, and compared against other similar works in the literature. ASFM attains remarkable results and shows that it is a valuable framework for social robotics applications, such as guiding people or human-robot interaction., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

PL-SLAM: real-time monocular visual SLAM with points and lines

  • Pumarola Peris, Albert|||0000-0003-4185-6991
  • Vakhitov, A.
  • Agudo Martínez, Antonio|||0000-0001-6845-4998
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Low textured scenes are well known to be one of the main Achilles heels of geometric computer vision algorithms relying on point correspondences, and in particular for visual SLAM. Yet, there are many environments in which, despite being low textured, one can still reliably estimate line-based geometric primitives, for instance in city and indoor scenes, or in the so-called, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Temporal segmentation of human actions in video sequences

  • Carmona Leyva, José María
  • Climent Vilaró, Joan|||0000-0002-8593-6603
Most of the published works concerning action recognition, usually assume that the action sequences have been previously segmented in time, that is, the action to be recognized starts with the first sequence frame and ends with the last one. However, temporal segmentation of actions in sequences is not an easy task, and is always prone to errors. In this paper, we present a new technique to automatically extract human actions from a video sequence. Our approach presents several contributions. First of all, we use a projection template scheme and find spatio-temporal features and descriptors within the projected surface, rather than extracting them in the whole sequence. For projecting the sequence we use a variant of the R transform, which has never been used before for temporal action segmentation. Instead of projecting the original video sequence, we project its optical flow components, preserving important information about action motion. We test our method on a publicly available action dataset, and the results show that it performs very well segmenting human actions compared with the state-of-the-art methods., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Discriminative learning of deep convolutional feature point descriptors

  • Simó Serra, Edgar
  • Trulls Fortuny, Eduard
  • Ferraz, Luis
  • Kokkinos, Iasonas
  • Fua, Pascal
  • Moreno-Noguer, Francesc
Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and test- ing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Joint coarse-and-fine reasoning for deep optical flow

  • Vaquero Gómez, Víctor|||0000-0003-2516-2551
  • Ros, German
  • Moreno-Noguer, Francesc
  • López, Antonio Manuel
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Planar PØP: feature-less pose estimation with applications in UAV localization

  • Amor Martínez, Adrián
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Herrero Cotarelo, Fernando|||0000-0003-4116-0265
  • Ruiz, Alberto
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We present a featureless pose estimation method that, in contrast to current Perspective-n-Point (PnP) approaches, it does not require n point correspondences to obtain the camera pose, allowing for pose estimation from natural shapes that do not necessarily have distinguished features like corners or intersecting edges. Instead of using n correspondences (e.g. extracted with a feature detector) we will use the raw polygonal representation of the observed shape and directly estimate the pose in the pose-space of the camera. This method compared with a general PnP method, does not require n point correspondences neither a priori knowledge of the object model (except the scale), which is registered with a picture taken from a known robot pose. Moreover, we achieve higher precision because all the information of the shape contour is used to minimize the area between the projected and the observed shape contours. To emphasize the non-use of n point correspondences between the projected template and observed contour shape, we call the method Planar PØP. The method is shown both in simulation and in a real application consisting on a UAV localization where comparisons with a precise ground-truth are provided., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Learning the hidden human knowledge of UAV pilots when navigating in a cluttered environment for improving path planning

  • Alzugaray López, Ignacio
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose in this work a new model of how the hidden human knowledge (HHK) of UAV pilots can be incorporated in the UAVs path planning generation. We intuitively know that human’s pilots barely manage or even attempt to drive the UAV through a path that is optimal attending to some criteria as an optimal planner would suggest. Although human pilots might get close but not reach the optimal path proposed by some planner that optimizes over time or distance, the final effect of this differentiation could be not only surprisingly better, but also desirable. In the best scenario for optimality, the path that human pilots generate would deviate from the optimal path as much as the hidden knowledge that its perceives is injected into the path. The aim of our work is to use real human pilot paths to learn the hidden knowledge using repulsion fields and to incorporate this knowledge afterwards in the environment obstacles as cause of the deviation from optimality. We present a strategy of learning this knowledge based on attractor and repulsors, the learning method and a modified RRT* that can use this knowledge for path planning. Finally we do real-life tests and we compare the resulting paths with and without this knowledge.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Multi-modal joint embedding for fashion product retrieval

  • Rubio Romano, Antonio
  • Yu, Longlong
  • Simó Serra, Edgar
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Finding a product in the fashion world can be a daunting task. Everyday, e-commerce sites are updating with thousands of images and their associated metadata (textual information), deepening the problem, akin to finding a needle in a haystack. In this paper, we leverage both the images and textual meta-data and propose a joint multi-modal embedding that maps both the text and images into a common latent space. Distances in the latent space correspond to similarity between products, allowing us to effectively perform retrieval in this latent space, which is both efficient and accurate. We train this embedding using large-scale real world e-commerce data by both minimizing the similarity between related products and using auxiliary classification networks to that encourage the embedding to have semantic meaning. We compare against existing approaches and show significant improvements in retrieval tasks on a large-scale e-commerce dataset. We also provide an analysis of the different metadata., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Aerial social force model: a new framework to accompany people using autonomous flying robots

  • Garrell Zulueta, Anais
  • Garza Elizondo, Luís Alberto
  • Villamizar Vergel, Michael Alejandro
  • Herrero Cotarelo, Fernando
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We proposed a novel Aerial Social Force Model (ASFM) that allows autonomous flying robots to accompany humans in urban environments in a safe and comfortable manner. To date, we are not aware of other state-of-the-art method that accomplish this task. The proposed approach is a 3D version of the Social Force Model (SFM) for the field of aerial robots which includes an interactive human-robot navigation scheme capable of predicting human motions and intentions so as to safely accompany them to their final destination. ASFM also introduces a new metric to fine-tune the parameters of the force model, and to evaluate the performance of the aerial robot companion based on comfort and distance between the robot and humans. The presented approach is extensively validated in diverse simulations and real experiments, and compared against other similar works in the literature. ASFM attains remarkable results and shows that it is a valuable framework for social robotics applications, such as guiding people or human-robot interaction., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Unsupervised person image synthesis in arbitrary poses

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Learning the hidden human knowledge of UAV pilots when navigating in a cluttered environment for improving path planning

  • Alzugaray López, Ignacio
  • Sanfeliu Cortés, Alberto
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose in this work a new model of how the hidden human knowledge (HHK) of UAV pilots can be incorporated in the UAVs path planning generation. We intuitively know that human’s pilots barely manage or even attempt to drive the UAV through a path that is optimal attending to some criteria as an optimal planner would suggest. Although human pilots might get close but not reach the optimal path proposed by some planner that optimizes over time or distance, the final effect of this differentiation could be not only surprisingly better, but also desirable. In the best scenario for optimality, the path that human pilots generate would deviate from the optimal path as much as the hidden knowledge that its perceives is injected into the path. The aim of our work is to use real human pilot paths to learn the hidden knowledge using repulsion fields and to incorporate this knowledge afterwards in the environment obstacles as cause of the deviation from optimality. We present a strategy of learning this knowledge based on attractor and repulsors, the learning method and a modified RRT* that can use this knowledge for path planning. Finally we do real-life tests and we compare the resulting paths with and without this knowledge., Accepted version
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A flexible hardware-in-the-loop architecture for UAVs

  • Lepej, Peter
  • Santamaria Navarro, Àngel
  • Solà, Joan
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., As robotic technology matures, fully autonomous robots become a realistic possibility, but demand very complex solutions to be rapidly engineered. In order to be able to quickly set up a working autonomous system, and to reduce the gap between simulated and real experiments, we propose a modular, upgradeable and flexible hardware-in-the-loop (HIL) architecture, which hybridizes the simulated and real settings. We take as use case the autonomous exploration of dense forests with UAVs, with the aim of creating useful maps for forest inspection, cataloging, or to compute other metrics such as total wood volume. As the first step in the development of the full system, in this paper we implement a fraction of this architecture, comprising assisted localization, and automatic methods for mapping, planning and motion execution. Specifically we are able to simulate the use of a 3D LIDAR endowed below an actual UAV autonomously navigating among simulated obstacles, thus the platform safety is not compromised. The full system is modular and takes profit of pieces either publicly available or easily programmed. We highlight the flexibility of the proposed HIL architecture to rapidly configure different experimental setups with a UAV in challenging terrain. Moreover, it can be extended to other robotic fields without further design. The HIL system uses the multi-platform ROS capabilities and only needs a motion capture system as external extra hardware, which is becoming standard equipment in all research labs dealing with mobile robots., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Nonlinear model predictive control for aerial manipulation

  • Lunni, Dario
  • Santamaria Navarro, Àngel
  • Rossi, Roberto
  • Rocco, Paolo
  • Bascetta, Luca
  • Andrade-Cetto, Juan
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This paper presents a nonlinear model predictive controller to follow desired 3D trajectories with the end effector of an unmanned aerial manipulator (i.e., a multirotor with a serial arm attached). To the knowledge of the authors, this is the first time that such controller runs online and on board a limited computational unit to drive a kinematically augmented aerial vehicle. Besides the trajectory following target, we explore the possibility of accomplishing other tasks during flight by taking advantage of the system redundancy. We define several tasks designed for aerial manipulators and show in simulation case studies how they can be achieved by either a weighting strategy, within a main optimization process, or a hierarchical approach consisting on nested optimizations. Moreover, experiments are presented to demonstrate the performance of such controller in a real robot., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Joint coarse-and-fine reasoning for deep optical flow

  • Vaquero Gómez, Víctor
  • Ros, German
  • Moreno-Noguer, Francesc
  • López, Antonio Manuel
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning. The coarse reasoning is performed over a discrete classification space to obtain a general rough solution, while the fine details of the solution are obtained over a continuous regression space. In our approach both components are jointly estimated, which proved to be beneficial for improving estimation accuracy. Additionally, we propose a new network architecture, which combines coarse and fine components by treating the fine estimation as a refinement built on top of the coarse solution, and therefore adding details to the general prediction. We apply our approach to the challenging problem of optical flow estimation and empirically validate it against state-of-the-art CNN-based solutions trained from scratch and tested on large optical flow datasets., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Temporal segmentation of human actions in video sequences

  • Carmona Leyva, José María
  • Climent Vilaró, Joan
Most of the published works concerning action recognition, usually assume that the action sequences have been previously segmented in time, that is, the action to be recognized starts with the first sequence frame and ends with the last one. However, temporal segmentation of actions in sequences is not an easy task, and is always prone to errors. In this paper, we present a new technique to automatically extract human actions from a video sequence. Our approach presents several contributions. First of all, we use a projection template scheme and find spatio-temporal features and descriptors within the projected surface, rather than extracting them in the whole sequence. For projecting the sequence we use a variant of the R transform, which has never been used before for temporal action segmentation. Instead of projecting the original video sequence, we project its optical flow components, preserving important information about action motion. We test our method on a publicly available action dataset, and the results show that it performs very well segmenting human actions compared with the state-of-the-art methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

BASS: boundary-aware superpixel segmentation

  • Rubio Romano, Antonio
  • Yu, Longlong
  • Simó Serra, Edgar
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We propose a new superpixel algorithm based on exploiting the boundary information of an image, as objects in images can generally be described by their boundaries. Our proposed approach initially estimates the boundaries and uses them to place superpixel seeds in the areas in which they are more dense. Afterwards, we minimize an energy function in order to expand the seeds into full superpixels. In addition to standard terms such as color consistency and compactness, we propose using the geodesic distance which concentrates small superpixels in regions of the image with more information, while letting larger superpixels cover more homogeneous regions. By both improving the initialization using the boundaries and coherency of the superpixels with geodesic distances, we are able to maintain the coherency of the image structure with fewer superpixels than other approaches. We show the resulting algorithm to yield smaller Variation of Information metrics in seven different datasets while maintaining Undersegmentation Error values similar to the state-of-the-art methods., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Matchability prediction for full-search template matching algorithms

  • Peñate Sánchez, Adrián
  • Porzi, Lorenzo
  • Moreno-Noguer, Francesc
While recent approaches have shown that it is possible to do template matching by exhaustively scanning the parameter space, the resulting algorithms are still quite demanding. In this paper we alleviate the computational load of these algorithms by proposing an efficient approach for predicting the match ability of a template, before it is actually performed. This avoids large amounts of unnecessary computations. We learn the match ability of templates by using dense convolutional neural network descriptors that do not require ad-hoc criteria to characterize a template. By using deep learning descriptions of patches we are able to predict match ability over the whole image quite reliably. We will also show how no specific training data is required to solve problems like panorama stitching in which you usually require data from the scene in question. Due to the highly parallelizable nature of this tasks we offer an efficient technique with a negligible computational cost at test time., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Aerial robotics in building inspection and maintenance

  • Grau Saldes, Antoni
  • Guerra Paradas, Edmundo
  • Bolea Monte, Yolanda
  • Puig-Pey Clavería, Ana María
  • Sanfeliu Cortés, Alberto
Buildings need periodic revision about their state, materials degrade with time and repairs or renewals have to be made driven by maintenance needs or safety requirements. That happens with any kind of buildings and constructions: housing, architecture masterpieces, old and ancient buildings and industrial buildings. Currently, nearly all of these tasks are carried out by human intervention. In order to carry out the inspection or maintenance, humans need to access to roofs, façades or other areas hard to reach and otherwise potentially hazardous location to perform the task. In some cases, it might not be feasible to access for inspection. For instance, in industry buildings operation must be often interrupted to allow for safe execution of such tasks; these shutdowns not only lead to substantial production loss, but the shutdown and start-up operation itself causes risks to human and environment. In touristic buildings, access has to be restricted with the consequent losses and inconveniences to visitors. The use of aerial robots can help to perform this kind of hazardous operations in an autonomous way, not only teleoperated. Robots are able to carry sensors to detect failures of many types and to locate them in a previously generated map, which the robot uses to navigate. Some of those sensors are cameras in different spectra (visual, near-infrared, UV), laser, LIDAR, ultrasounds and inertial sensory system. If the sensory part is crucial to inspect hazardous areas in buildings, the actuation is also important: the aerial robot can carry small robots (mainly crawler) to be deployed to perform more in-depth operation where the contact between the sensors and the material is basic (any kind of metallic part: pipes, roofs, panels…). The aerial robot has the ability to recover the deployed small crawler to be reused again. In this paper, authors will explain the research that they are conducting in this area and propose future research areas and applications with aerial, ground, submarine and other autonomous robots within the construction field., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Robotic monocular SLAM in wastewater treatment plants with a sampling device

  • Guerra Paradas, Edmundo
  • Bolea Monte, Yolanda
  • Munguía Alcalá, Rodrigo Francisco
  • Grau Saldes, Antoni
The final publication is available at link.springer.com, A novel application is presented in this paper. Wastewater treatment plants need to sample water to follow the cleaned process continuously. This task is very tedious for laboratory workers and, mainly, can be done in reduced areas of the large basins that form the whole plant, specifically only in the edges of those basins. The new proposal is to enlarge the sampling area and reduce the load for workers. A new unmanned aerial vehicle has been designed together with a novel tool to take water samples. Moreover, a new architecture for the mission plan is presented in form of multi-agents. Experiments have been done in order to test the feasibility of the proposal., This research has been funded by AEROARMS EU Project H2020-ICT-2014-1-644271, Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

PL-SLAM: real-time monocular visual SLAM with points and lines

  • Pumarola Peris, Albert
  • Vakhitov, A.
  • Agudo Martínez, Antonio
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Low textured scenes are well known to be one of the main Achilles heels of geometric computer vision algorithms relying on point correspondences, and in particular for visual SLAM. Yet, there are many environments in which, despite being low textured, one can still reliably estimate line-based geometric primitives, for instance in city and indoor scenes, or in the so-called, Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Discriminative learning of deep convolutional feature point descriptors

  • Simó Serra, Edgar
  • Trulls Fortuny, Eduard
  • Ferraz, Luis
  • Kokkinos, Iasonas
  • Fua, Pascal
  • Moreno-Noguer, Francesc
Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify. By using the L2 distance during both training and test- ing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Geometry-aware network for non-rigid shape prediction from a single view

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Porzi, Lorenzo
  • Sanfeliu Cortés, Alberto
  • Lepetit, Vincent
  • Moreno-Noguer, Francesc
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a {it geometry-aware} deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Planar PØP: feature-less pose estimation with applications in UAV localization

  • Amor Martínez, Adrián
  • Santamaria Navarro, Àngel
  • Herrero Cotarelo, Fernando
  • Ruiz, Alberto
  • Sanfeliu Cortés, Alberto
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We present a featureless pose estimation method that, in contrast to current Perspective-n-Point (PnP) approaches, it does not require n point correspondences to obtain the camera pose, allowing for pose estimation from natural shapes that do not necessarily have distinguished features like corners or intersecting edges. Instead of using n correspondences (e.g. extracted with a feature detector) we will use the raw polygonal representation of the observed shape and directly estimate the pose in the pose-space of the camera. This method compared with a general PnP method, does not require n point correspondences neither a priori knowledge of the object model (except the scale), which is registered with a picture taken from a known robot pose. Moreover, we achieve higher precision because all the information of the shape contour is used to minimize the area between the projected and the observed shape contours. To emphasize the non-use of n point correspondences between the projected template and observed contour shape, we call the method Planar PØP. The method is shown both in simulation and in a real application consisting on a UAV localization where comparisons with a precise ground-truth are provided., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

GANimation: anatomically-aware facial animation from a single image

  • Pumarola Peris, Albert
  • Agudo Martínez, Antonio
  • Martinez, Aleix M.
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN, that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild., Peer Reviewed, Award-winning, Postprint (author's final draft)