Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 51
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

MSClique: multiple structure discovery through the maximum weighted clique problem

  • Sanromà, Gerard
  • Penate-Sanchez, Adrian
  • Alquézar, René
  • Serratosa, Francesc
  • Moreno-Noguer, Francesc
  • Andrade-Cetto, Juan
  • González Ballester, Miguel Ángel, 1973-
We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under Project DPI-2011-27510 and the ERA-Net Chistera project ViSen PCIN-2013-047. This work has also been partially supported by EU H2020, Call H2020-ICT-23-2014-1 (RIA) under Project 644271. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

MSClique: Multiple structure discovery through the maximum weighted clique problem

  • Sanroma, Gerard
  • Peñate-Sánchez, Adrián
  • Alquézar Mancho, Renato
  • Serratosa, Francesc
  • Moreno-Noguer, Francesc
  • Andrade-Cetto, Juan
  • González Ballester, Miguel Angel
This is an open access article distributed under the terms of the Creative Commons Attribution License., We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under Project DPI-2011-27510 and the ERA-Net Chistera project ViSen PCIN-2013-047. This work has also been partially supported by EU H2020, Call H2020-ICT-23-2014-1 (RIA) under Project 644271., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Hybrid visual servoing with hierarchical task composition for aerial manipulation

  • Lippiello, Vincenzo
  • Cacace, Jonathan
  • Santamaria-Navarro, Àngel
  • Andrade-Cetto, Juan
  • Trujillo, Miguel Ángel
  • Rodríguez Esteves, Yamnia
  • Viguria, Antidio
In this letter, a hybrid visual servoing with a hierarchical task-composition control framework is described for aerial manipulation, i.e., for the control of an aerial vehicle endowed with a robot arm. The proposed approach suitably combines into a unique hybrid-control framework the main benefits of both image-based and position-based control schemes. Moreover, the underactuation of the aerial vehicle has been explicitly taken into account in a general formulation, together with a dynamic smooth activation mechanism. Both simulation case studies and experiments are presented to demonstrate the performance of the proposed technique., This work was supported by the ARCAS and AEROARMS large-scale integrating projects, funded by the European Community’s under Grants FP7-ICT-287617 and H2020-ICT-644271, respectively., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Interactive multiple object learning with scanty human supervision

  • Villamizar, Michael
  • Garrell, Anaís
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
We present a fast and online human-robot interaction approach that progressively learns multiple object classifiers using scanty human supervision. Given an input video stream recorded during the human-robot interaction, the user just needs to annotate a small fraction of frames to compute object specific classifiers based on random ferns which share the same features. The resulting methodology is fast (in a few seconds, complex object appearances can be learned), versatile (it can be applied to unconstrained scenarios), scalable (real experiments show we can model up to 30 different object classes), and minimizes the amount of human intervention by leveraging the uncertainty measures associated to each classifier. We thoroughly validate the approach on synthetic data and on real sequences acquired with a mobile platform in indoor and outdoor scenarios containing a multitude of different objects. We show that with little human assistance, we are able to build object classifiers robust to viewpoint changes, partial occlusions, varying lighting and cluttered backgrounds., This work has been partially funded by the Spanish Ministry of Economy and Competitiveness under projects ERA-Net Chistera project ViSen PCIN-2013-047, RobInstruct TIN2014-58178-R, ROBOT-INT-COOP DPI2013-42458-P, and by the EU project AEROARMS H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Trajectory generation for unmanned aerial manipulators through quadratic programming

  • Rossi, Roberto
  • Santamaria-Navarro, Àngel
  • Andrade-Cetto, Juan
  • Rocco, Paolo
In this paper a trajectory generation approach using quadratic programming is described for aerial manipulation, i.e. for the control of an aerial vehicle equipped with a robot arm. The proposed approach applies the online active set strategy to generate a feasible trajectory of the joints, in order to accomplish a set of tasks with defined bounds and constraint inequalities. The definition of the problem in the acceleration domain allows to integrate and perform a large set of tasks and, as a result, to obtain smooth motion of the joints. A weighting strategy, associated with a normalization procedure, allows to easily define the relative importance of the tasks. This approach is useful to accomplish different phases of a mission with different redundancy resolution strategies. The performance of the proposed technique is demonstrated through real experiments with all the algorithms running onboard in real time. In particular, the aerial manipulator can successfully perform navigation and interaction phases, while keeping motion within prescribed bounds and avoiding collisions with external obstacles., Their work has been partially funded by the EU project AEROARMS H2020-ICT2014-1-644271 and by the Spanish Ministry of Economy and Competitiveness project ROBINSTRUCT TIN2014-58178-R., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Uncalibrated visual servo for unmanned aerial manipulation

  • Santamaria-Navarro, Àngel
  • Grosch, Patrick
  • Lippiello, Vincenzo
  • Solà, Joan
  • Andrade-Cetto, Juan
This paper addresses the problem of autonomous servoing an unmanned redundant aerial manipulator using computer vision. The overactuation of the system is exploited by means of a hierarchical control law, which allows to prioritize several tasks during flight. We propose a safety-related primary task to avoid possible collisions. As a secondary task, we present an uncalibrated image-based visual servo strategy to drive the arm end-effector to a desired position and orientation by using a camera attached to it. In contrast to the previous visual servo approaches, a known value of camera focal length is not strictly required. To further improve flight behavior, we hierarchically add one task to reduce dynamic effects by vertically aligning the arm center of gravity to the multirotor gravitational vector, and another one that keeps the arm close to a desired configuration of high manipulability and avoiding arm joint limits. The performance of the hierarchical control law, with and without activation of each of the tasks, is shown in simulations and in real experiments confirming the viability of such prioritized control scheme for aerial manipulation., This work has been funded by the EU project AEROARMS H2020-ICT2014-1-644271 and by the Spanish Ministry of Economy and Competitiveness project ROBINSTRUCT TIN2014-58178-R., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Searching and tracking people in urban environments with static and dynamic obstacles

  • Goldhoorn, Alex
  • Garrell, Anaís
  • Alquézar Mancho, Renato
  • Sanfeliu, Alberto
Searching and tracking people in crowded urban areas where they can be occluded by static or dynamic obstacles is an important behavior for social robots which assist humans in urban outdoor environments. In this work, we propose a method that can handle in real-time searching and tracking people using a Highest Belief Particle Filter Searcher and Tracker. It makes use of a modified Particle Filter (PF), which, in contrast to other methods, can do both searching and tracking of a person under uncertainty, with false negative detections, lack of a person detection, in continuous space and real-time. Moreover, this method uses dynamic obstacles to improve the predicted possible location of the person. Comparisons have been made with our previous method, the Adaptive Highest Belief Continuous Real-time POMCP Follower, in different conditions and with dynamic obstacles. Real-life experiments have been done during two weeks with a mobile service robot in two urban environments of Barcelona with other people walking around., This work has been partially funded by the EU project AEROARMS European projectH2020-ICT-2014-1-644271 and the CICYT project DPI2013-42458-P., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Teaching robot’s proactive behavior using human assistance

  • Garrell, Anaís
  • Villamizar, Michael
  • Moreno-Noguer, Francesc
  • Sanfeliu, Alberto
In recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status. We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks., This work was supported by the Spanish Ministry of Science and Innovation, project Rob-Int-Coop DPI2013-42458-P and AEROARMS European project H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Learning depth-aware deep representations for robotic perception

  • Porzi, Lorenzo
  • Rota Buló, Samuel
  • Peñate-Sánchez, Adrián
  • Ricci, Elisa
  • Moreno-Noguer, Francesc
Exploiting RGB-D data by means of Convolutional Neural Networks (CNNs) is at the core of a number of robotics applications, including object detection, scene semantic segmentation and grasping. Most existing approaches, however, exploit RGB-D data by simply considering depth as an additional input channel for the network. In this paper we show that the performance of deep architectures can be boosted by introducing DaConv, a novel, general-purpose CNN block which exploits depth to learn scale-aware feature representations. We demonstrate the benefits of DaConv on a variety of robotics oriented tasks, involving affordance detection, object coordinate regression and contour detection in RGB-D images. In each of these experiments we show the potential of the proposed block and how it can be readily integrated into existing CNN architectures., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147, by the EU project AEROARMS H2020-ICT-2014-1-644271 and by the EU project SECOND HANDS H2020-ICT-2014-1-643950., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Random clustering ferns for multimodal object recognition

  • Villamizar, Michael
  • Garrell, Anaís
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
We propose an efficient and robust method for the recognition of objects exhibiting multiple intra-class modes, where each one is associated with a particular object appearance. The proposed method, called random clustering ferns, combines synergically a single and real-time classifier, based on the boosted assembling of extremely randomized trees (ferns), with an unsupervised and probabilistic approach in order to recognize efficiently object instances in images and discover simultaneously the most prominent appearance modes of the object through tree-structured visual words. In particular, we use boosted random ferns and probabilistic latent semantic analysis to obtain a discriminative and multimodal classifier that automatically clusters the response of its randomized trees in function of the visual object appearance. The proposed method is validated extensively in synthetic and real experiments, showing that the method is capable of detecting objects with diverse and complex appearance distributions in real-time performance., This work has been partially funded by the Spanish Ministry of Economy and Competitiveness under projects ERA-Net Chistera project ViSen PCIN-2013-047, RobInstruct TIN2014-58178-R, ROBOT-INT-COOP DPI2013-42458-P, and by the EU project AEROARMS H2020-ICT-2014-1-644271., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Boosted Random Ferns for object detection

  • Villamizar, Michael
  • Andrade-Cetto, Juan
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation., This work was partially supported by the Catalan Agency for Management of University and Research Grants for the Consolidated Group VIS (2014 SGR 897). A.S. acknowledges support also from project ColRobTransp (DPI2016-78957-R), and J.A. and F.M. from project RobInstruct (TIN2014-58178-R), both funded by the Spanish Ministry of Economy, Industry and Competitiveness. J.A., A.S., and F.M. were also partially supported by the European Union projects AEROARMS (H2020-ICT-2014-1-644271) and LOGIMATIC (H2020-Galileo-2015-1-687534)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

  • Santamaria-Navarro, Àngel
  • Loianno, Giuseppe
  • Solà, Joan
  • Kumar, Vijay
  • Andrade-Cetto, Juan
The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public., This work was funded by the project ROBINSTRUCT (TIN2014-58178-R) and the Ramón y Cajal posdoctoral fellowship (RYC-2012-11604) from the Spanish Ministry of Economy and Competitiveness, by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656) and by the EU H2020 project AEROARMS (H2020-ICT-2014-1-644271)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

GANimation: One-Shot Anatomically Consistent Facial Animation

  • Pumarola, Albert
  • Agudo Martínez, Antonio
  • Martínez, Aleix M.
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Recent advances in generative adversarial networks (GANs) have shown impressive results for the task of facial expression synthesis. The most successful architecture is StarGAN (Choi et al. in CVPR, 2018), that conditions GANs’ generation process with images of a specific domain, namely a set of images of people sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content and granularity of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on action units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combining several of them. Additionally, we propose a weakly supervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit a novel self-learned attention mechanism that makes our network robust to changing backgrounds, lighting conditions and occlusions. Extensive evaluation shows that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild. The code of this work is publicly available at https://github.com/albertpumarola/GANimation., This work is partially supported by an Amazon Research Award, by the Spanish Ministry of Economy and Competitiveness under Projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957 and María de Maeztu Seal of Excellence MDM-2016-0656; by the EU Project AEROARMS ICT-2014-1-644271; and by the Grant R01-DC- 014498 of the National Institute of Health. We also thank Nvidia for hardware donation under the GPU Grant Program.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

MSClique: Multiple structure discovery through the maximum weighted clique problem

  • Sanromà Güell, Gerard
  • Peñate Sánchez, Adrián
  • Alquézar Mancho, René|||0000-0002-6420-0517
  • Serratosa Casanelles, Francesc|||0000-0001-6112-5913
  • Moreno-Noguer, Francesc
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
  • González Ballester, Miguel Ángel
We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Detection and positioning of pipes and columns with autonomous multicopter drones

  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Munguía Alcalá, Rodrigo Francisco
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Grau Saldes, Antoni|||0000-0003-4112-3325
A multimodal sensory array to accurately position aerial multicopter drones with respect to pipes has been studied, and a solution exploiting both LiDAR and vision sensors has been proposed. Several challenges, including detection of pipes and other cylindrical elements in sensor space and validation of the elements detected, have been studied. A probabilistic parametric method has been applied to segment and position cylinders with LIDAR, while several vision-based techniques have been tested to find the contours of the pipe, combined with conic estimation cylinder pose recovery. Multiple solutions have been studied and analyzed, evaluating their results. This allowed proposing an approach that combines both LiDAR and vision to produce robust and accurate pipe detection. This combined solution is validated with real experimental data., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

UAV visual and laser sensors fusion for detection and positioning in industrial applications

  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Munguía Alcalá, Rodrigo Francisco
  • Grau Saldes, Antoni|||0000-0003-4112-3325
This work presents a solution to localize Unmanned Autonomous Vehicles with respect to pipes and other cylindrical elements found in inspection and maintenance tasks both in industrial and civilian infrastructures. The proposed system exploits the different features of vision and laser based sensors, combining them to obtain accurate positioning of the robot with respect to the cylindrical structures. A probabilistic (RANSAC-based) procedure is used to segment possible cylinders found in the laser scans, and this is used as a seed to accurately determine the robot position through a computer vision system. The priors obtained from the laser scan registration help to solve the problem of determining the apparent contour of the cylinders. In turn this apparent contour is used in a degenerate quadratic conic estimation, enabling to visually estimate the pose of the cylinder.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Delayed monocular SLAM approach applied to unmanned aerial vehicles

  • Munguía Alcalá, Rodrigo Francisco
  • Urzua, Sarquis
  • Grau Saldes, Antoni|||0000-0003-4112-3325
In recent years, many researchers have addressed the issue of making Unmanned Aerial Vehicles (UAVs) more and more autonomous. In this context, the state estimation of the vehicle position is a fundamental necessity for any application involving autonomy. However, the problem of position estimation could not be solved in some scenarios, even when a GPS signal is available, for instance, an application requiring performing precision manoeuvres in a complex environment. Therefore, some additional sensory information should be integrated into the system in order to improve accuracy and robustness. In this work, a novel vision-based simultaneous localization and mapping (SLAM) method with application to unmanned aerial vehicles is proposed. One of the contributions of this work is to design and develop a novel technique for estimating features depth which is based on a stochastic technique of triangulation. In the proposed method the camera is mounted over a servo-controlled gimbal that counteracts the changes in attitude of the quadcopter. Due to the above assumption, the overall problem is simplified and it is focused on the position estimation of the aerial vehicle. Also, the tracking process of visual features is made easier due to the stabilized video. Another contribution of this work is to demonstrate that the integration of very noisy GPS measurements into the system for an initial short period of time is enough to initialize the metric scale. The performance of this proposed method is validated by means of experiments with real data carried out in unstructured outdoor environments. A comparative study shows that, when compared with related methods, the proposed approach performs better in terms of accuracy and computational time., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Loianno, Giuseppe
  • Solà Ortega, Joan|||0000-0002-2933-3381
  • Kumar, Vijay
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
The final publication is available at link.springer.com, The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Interactive multiple object learning with scanty human supervision

  • Villamizar Vergel, Michael Alejandro
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, We present a fast and online human-robot interaction approach that progressively learns multiple object classifiers using scanty human supervision. Given an input video stream recorded during the human robot interaction, the user just needs to annotate a small fraction of frames to compute object specific classifiers based on random ferns which share the same features. The resulting methodology is fast (in a few seconds, complex object appearances can be learned), versatile (it can be applied to unconstrained scenarios), scalable (real experiments show we can model up to 30 different object classes), and minimizes the amount of human intervention by leveraging the uncertainty measures associated to each classifier.; We thoroughly validate the approach on synthetic data and on real sequences acquired with a mobile platform in indoor and outdoor scenarios containing a multitude of different objects. We show that with little human assistance, we are able to build object classifiers robust to viewpoint changes, partial occlusions, varying lighting and cluttered backgrounds. (C) 2016 Elsevier Inc. All rights reserved., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Teaching robot's proactive behavior using human assistance

  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Villamizar Vergel, Michael Alejandro
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
The final publication is available at link.springer.com, In recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status. We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Cooperative monocular-based SLAM for multi-UAV systems in GPS-denied environments

  • Trujillo, Juan Carlos
  • Munguía Alcalá, Rodrigo Francisco
  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Grau Saldes, Antoni|||0000-0003-4112-3325
This work presents a cooperative monocular-based SLAM approach for multi-UAV systems that can operate in GPS-denied environments. The main contribution of the work is to show that, using visual information obtained from monocular cameras mounted onboard aerial vehicles flying in formation, the observability properties of the whole system are improved. This fact is especially notorious when compared with other related visual SLAM configurations. In order to improve the observability properties, some measurements of the relative distance between the UAVs are included in the system. These relative distances are also obtained from visual information. The proposed approach is theoretically validated by means of a nonlinear observability analysis. Furthermore, an extensive set of computer simulations is presented in order to validate the proposed approach. The numerical simulation results show that the proposed system is able to provide a good position and orientation estimation of the aerial vehicles flying in formation., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Vision-based SLAM system for unmanned aerial vehicles

  • Munguía Alcalá, Rodrigo Francisco
  • Urzua, Sarquis
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Grau Saldes, Antoni|||0000-0003-4112-3325
The present paper describes a vision-based simultaneous localization and mapping system to be applied to Unmanned Aerial Vehicles (UAVs). The main contribution of this work is to propose a novel estimator relying on an Extended Kalman Filter. The estimator is designed in order to fuse the measurements obtained from: (i) an orientation sensor (AHRS); (ii) a position sensor (GPS); and (iii) a monocular camera. The estimated state consists of the full state of the vehicle: position and orientation and their first derivatives, as well as the location of the landmarks observed by the camera. The position sensor will be used only during the initialization period in order to recover the metric scale of the world. Afterwards, the estimated map of landmarks will be used to perform a fully vision-based navigation when the position sensor is not available. Experimental results obtained with simulations and real data show the benefits of the inclusion of camera measurements into the system. In this sense the estimation of the trajectory of the vehicle is considerably improved, compared with the estimates obtained using only the measurements from the position sensor, which are commonly low-rated and highly noisy.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Random clustering ferns for multimodal object recognition

  • Villamizar Vergel, Michael Alejandro
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, We propose an efficient and robust method for the recognition of objects exhibiting multiple intra-class modes, where each one is associated with a particular object appearance. The proposed method, called random clustering ferns, combines synergically a single and real-time classifier, based on the boosted assembling of extremely randomized trees (ferns), with an unsupervised and probabilistic approach in order to recognize efficiently object instances in images and discover simultaneously the most prominent appearance modes of the object through tree-structured visual words. In particular, we use boosted random ferns and probabilistic latent semantic analysis to obtain a discriminative and multimodal classifier that automatically clusters the response of its randomized trees in function of the visual object appearance. The proposed method is validated extensively in synthetic and real experiments, showing that the method is capable of detecting objects with diverse and complex appearance distributions in real-time performance., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Visual-based SLAM configurations for cooperative multi-UAV systems with a lead agent: an observability-based approach

  • Trujillo, Juan Carlos
  • Munguía Alcalá, Rodrigo Francisco
  • Guerra Paradas, Edmundo|||0000-0002-6696-0982
  • Grau Saldes, Antoni|||0000-0003-4112-3325
In this work, the problem of the cooperative visual-based SLAM for the class of multi-UA systems that integrates a lead agent has been addressed. In these kinds of systems, a team of aerial robots flying in formation must follow a dynamic lead agent, which can be another aerial robot, vehicle or even a human. A fundamental problem that must be addressed for these kinds of systems has to do with the estimation of the states of the aerial robots as well as the state of the lead agent. In this work, the use of a cooperative visual-based SLAM approach is studied in order to solve the above problem. In this case, three different system configurations are proposed and investigated by means of an intensive nonlinear observability analysis. In addition, a high-level control scheme is proposed that allows to control the formation of the UAVs with respect to the lead agent. In this work, several theoretical results are obtained, together with an extensive set of computer simulations which are presented in order to numerically validate the proposal and to show that it can perform well under different circumstances (e.g., GPS-challenging environments). That is, the proposed method is able to operate robustly under many conditions providing a good position estimation of the aerial vehicles and the lead agent as well., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Boosted Random ferns for object detection

  • Villamizar Vergel, Michael Alejandro
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Teaching robot’s proactive behavior using human assistance

  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Villamizar Vergel, Michael Alejandro
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
The final publication is available at link.springer.com, In recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status.We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Searching and tracking people in urban environments with static and dynamic obstacles

  • Goldhoorn, Alex|||0000-0001-9257-2564
  • Garrell Zulueta, Anais|||0000-0002-4629-0723
  • Alquézar Mancho, René|||0000-0002-6420-0517
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
© <year>. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Searching and tracking people in crowded urban areas where they can be occluded by static or dynamic obstacles is an important behavior for social robots which assist humans in urban outdoor environments. In this work, we propose a method that can handle in real-time searching and tracking people using a Highest Belief Particle Filter Searcher and Tracker. It makes use of a modified Particle Filter (PF), which, in contrast to other methods, can do both searching and tracking of a person under uncertainty, with false negative detections, lack of a person detection, in continuous space and real-time. Moreover, this method uses dynamic obstacles to improve the predicted possible location of the person. Comparisons have been made with our previous method, the Adaptive Highest Belief Continuous Real-time POMCP Follower, in different conditions and with dynamic obstacles. Real-life experiments have been done during two weeks with a mobile service robot in two urban environments of Barcelona with other people walking around., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Trajectory generation for unmanned aerial manipulators through quadratic programming

  • Rossi, Roberto
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
  • Rocco, Paolo
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper a trajectory generation approach using quadratic programming is described for aerial manipulation, i.e. for the control of an aerial vehicle equipped with a robot arm. The proposed approach applies the online active set strategy to generate a feasible trajectory of the joints, in order to accomplish a set of tasks with defined bounds and constraint inequalities. The definition of the problem in the acceleration domain allows to integrate and perform a large set of tasks and, as a result, to obtain smooth motion of the joints. A weighting strategy, associated with a normalization procedure, allows to easily define the relative importance of the tasks. This approach is useful to accomplish different phases of a mission with different redundancy resolution strategies. The performance of the proposed technique is demonstrated through real experiments with all the algorithms running onboard in real time. In particular, the aerial manipulator can successfully perform navigation and interaction phases, while keeping motion within prescribed bounds and avoiding collisions with external obstacles., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Hybrid visual servoing with hierarchical task composition for aerial manipulation

  • Lippiello, Vincenzo
  • Cacace, Jonathan
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
  • Trujillo, Miguel Angel
  • Rodrigez Esteves, Yamnia
  • Viguria, Antidio
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper a hybrid visual servoing with a hierarchical task-composition control framework is described for aerial manipulation, i.e. for the control of an aerial vehicle endowed with a robot arm. The proposed approach suitably combines into a unique hybrid-control framework the main benefits of both image-based and position-based control schemes. Moreover, the underactuation of the aerial vehicle has been explicitly taken into account in a general formulation, together with a dynamic smooth activation mechanism. Both simulation case studies and experiments are presented to demonstrate the performance of the proposed technique., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Learning depth-aware deep representations for robotic perception

  • Porzi, Lorenzo
  • Rota Bulò, Samuel
  • Peñate Sánchez, Adrián
  • Ricci, Elisa
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Exploiting RGB-D data by means of Convolutional Neural Networks (CNNs) is at the core of a number of robotics applications, including object detection, scene semantic segmentation and grasping. Most existing approaches, however, exploit RGB-D data by simply considering depth as an additional input channel for the network. In this paper we show that the performance of deep architectures can be boosted by introducing DaConv, a novel, general-purpose CNN block which exploits depth to learn scale-aware feature representations. We demonstrate the benefits of DaConv on a variety of robotics oriented tasks, involving affordance detection, object coordinate regression and contour detection in RGB-D images. In each of these experiments we show the potential of the proposed block and how it can be readily integrated into existing CNN architectures., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Quadrotor multi-model for control purposes

  • Grau Saldes, Antoni|||0000-0003-4112-3325
  • Bolea Monte, Yolanda|||0000-0002-7075-9878
  • Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
In this work, a multi-model of a quadrotor is developed in order to control this system. The kinematic model of each part of the quadrotor will be derived using the Euler angles, and also the dynamics model of the quadrotor will be calculated based on the first principles of a rigid body using the Newton-Euler formulation. Furthermore, the following assumptions are used :1) The structure is completely rigid and perfectly symmetric. 2) The center of mass is in the origin of the quadrotor fixed frame. 3) The thrusts are proportional to the square of the motors rotational speed. A state-space model (kinematics and dynamics) is developed by physical laws. But, this deduced model presents several no linearities that are produced by three factors: the orientation (Pitch, Roll and Yaw), the control action and the angular velocities. To be able to control the quadrotor system in simple, linear and manageable way, it is necessary to linearize the system. Two method are possible: a classical linearization around several set-points and a multi-model linearization. In this case, a multi-model linearization is proposed due to the obtained control model will be used to compute a multi-model controller using fuzzy techniques. Fuzzy control techniques are suitable for linear parameter varying systems with no linearities, as our quadrotor., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Vision-based SLAM system for unmanned aerial vehicles

  • Munguía Alcalá, Rodrigo Francisco
  • Urzua, Sarquis
  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
The present paper describes a vision-based simultaneous localization and mapping system to be applied to Unmanned Aerial Vehicles (UAVs). The main contribution of this work is to propose a novel estimator relying on an Extended Kalman Filter. The estimator is designed in order to fuse the measurements obtained from: (i) an orientation sensor (AHRS); (ii) a position sensor (GPS); and (iii) a monocular camera. The estimated state consists of the full state of the vehicle: position and orientation and their first derivatives, as well as the location of the landmarks observed by the camera. The position sensor will be used only during the initialization period in order to recover the metric scale of the world. Afterwards, the estimated map of landmarks will be used to perform a fully vision-based navigation when the position sensor is not available. Experimental results obtained with simulations and real data show the benefits of the inclusion of camera measurements into the system. In this sense the estimation of the trajectory of the vehicle is considerably improved, compared with the estimates obtained using only the measurements from the position sensor, which are commonly low-rated and highly noisy., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Searching and tracking people in urban environments with static and dynamic obstacles

  • Goldhoorn, Alex
  • Garrell Zulueta, Anais
  • Alquézar Mancho, René
  • Sanfeliu Cortés, Alberto
© <year>. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, Searching and tracking people in crowded urban areas where they can be occluded by static or dynamic obstacles is an important behavior for social robots which assist humans in urban outdoor environments. In this work, we propose a method that can handle in real-time searching and tracking people using a Highest Belief Particle Filter Searcher and Tracker. It makes use of a modified Particle Filter (PF), which, in contrast to other methods, can do both searching and tracking of a person under uncertainty, with false negative detections, lack of a person detection, in continuous space and real-time. Moreover, this method uses dynamic obstacles to improve the predicted possible location of the person. Comparisons have been made with our previous method, the Adaptive Highest Belief Continuous Real-time POMCP Follower, in different conditions and with dynamic obstacles. Real-life experiments have been done during two weeks with a mobile service robot in two urban environments of Barcelona with other people walking around., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Boosted Random ferns for object detection

  • Villamizar Vergel, Michael Alejandro
  • Andrade-Cetto, Juan
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper we introduce the Boosted Random Ferns (BRFs) to rapidly build discriminative classifiers for learning and detecting object categories. At the core of our approach we use standard random ferns, but we introduce four main innovations that let us bring ferns from an instance to a category level, and still retain efficiency. First, we define binary features on the histogram of oriented gradients-domain (as opposed to intensity-), allowing for a better representation of intra-class variability. Second, both the positions where ferns are evaluated within the sliding window, and the location of the binary features for each fern are not chosen completely at random, but instead we use a boosting strategy to pick the most discriminative combination of them. This is further enhanced by our third contribution, that is to adapt the boosting strategy to enable sharing of binary features among different ferns, yielding high recognition rates at a low computational cost. And finally, we show that training can be performed online, for sequentially arriving images. Overall, the resulting classifier can be very efficiently trained, densely evaluated for all image locations in about 0.1 seconds, and provides detection rates similar to competing approaches that require expensive and significantly slower processing times. We demonstrate the effectiveness of our approach by thorough experimentation in publicly available datasets in which we compare against state-of-the-art, and for tasks of both 2D detection and 3D multi-view estimation., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

MSClique: multiple structure discovery through the maximum weighted clique problem

  • Sanromà, Gerard
  • Penate-Sanchez, Adrian
  • Alquézar, René
  • Serratosa, Francesc
  • Moreno-Noguer, Francesc
  • Andrade-Cetto, Juan
  • González Ballester, Miguel Ángel, 1973-
We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under Project DPI-2011-27510 and the ERA-Net Chistera project ViSen PCIN-2013-047. This work has also been partially supported by EU H2020, Call H2020-ICT-23-2014-1 (RIA) under Project 644271. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Hybrid visual servoing with hierarchical task composition for aerial manipulation

  • Lippiello, Vincenzo
  • Cacace, Jonathan
  • Santamaria Navarro, Àngel
  • Andrade-Cetto, Juan
  • Trujillo, Miguel Angel
  • Rodrigez Esteves, Yamnia
  • Viguria, Antidio
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper a hybrid visual servoing with a hierarchical task-composition control framework is described for aerial manipulation, i.e. for the control of an aerial vehicle endowed with a robot arm. The proposed approach suitably combines into a unique hybrid-control framework the main benefits of both image-based and position-based control schemes. Moreover, the underactuation of the aerial vehicle has been explicitly taken into account in a general formulation, together with a dynamic smooth activation mechanism. Both simulation case studies and experiments are presented to demonstrate the performance of the proposed technique., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Interactive multiple object learning with scanty human supervision

  • Villamizar Vergel, Michael Alejandro
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/, We present a fast and online human-robot interaction approach that progressively learns multiple object classifiers using scanty human supervision. Given an input video stream recorded during the human robot interaction, the user just needs to annotate a small fraction of frames to compute object specific classifiers based on random ferns which share the same features. The resulting methodology is fast (in a few seconds, complex object appearances can be learned), versatile (it can be applied to unconstrained scenarios), scalable (real experiments show we can model up to 30 different object classes), and minimizes the amount of human intervention by leveraging the uncertainty measures associated to each classifier.; We thoroughly validate the approach on synthetic data and on real sequences acquired with a mobile platform in indoor and outdoor scenarios containing a multitude of different objects. We show that with little human assistance, we are able to build object classifiers robust to viewpoint changes, partial occlusions, varying lighting and cluttered backgrounds. (C) 2016 Elsevier Inc. All rights reserved., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Learning depth-aware deep representations for robotic perception

  • Porzi, Lorenzo
  • Rota Bulò, Samuel
  • Peñate Sánchez, Adrián
  • Ricci, Elisa
  • Moreno-Noguer, Francesc
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Exploiting RGB-D data by means of Convolutional Neural Networks (CNNs) is at the core of a number of robotics applications, including object detection, scene semantic segmentation and grasping. Most existing approaches, however, exploit RGB-D data by simply considering depth as an additional input channel for the network. In this paper we show that the performance of deep architectures can be boosted by introducing DaConv, a novel, general-purpose CNN block which exploits depth to learn scale-aware feature representations. We demonstrate the benefits of DaConv on a variety of robotics oriented tasks, involving affordance detection, object coordinate regression and contour detection in RGB-D images. In each of these experiments we show the potential of the proposed block and how it can be readily integrated into existing CNN architectures., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Trajectory generation for unmanned aerial manipulators through quadratic programming

  • Rossi, Roberto
  • Santamaria Navarro, Àngel
  • Andrade-Cetto, Juan
  • Rocco, Paolo
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., In this paper a trajectory generation approach using quadratic programming is described for aerial manipulation, i.e. for the control of an aerial vehicle equipped with a robot arm. The proposed approach applies the online active set strategy to generate a feasible trajectory of the joints, in order to accomplish a set of tasks with defined bounds and constraint inequalities. The definition of the problem in the acceleration domain allows to integrate and perform a large set of tasks and, as a result, to obtain smooth motion of the joints. A weighting strategy, associated with a normalization procedure, allows to easily define the relative importance of the tasks. This approach is useful to accomplish different phases of a mission with different redundancy resolution strategies. The performance of the proposed technique is demonstrated through real experiments with all the algorithms running onboard in real time. In particular, the aerial manipulator can successfully perform navigation and interaction phases, while keeping motion within prescribed bounds and avoiding collisions with external obstacles., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

UAV visual and laser sensors fusion for detection and positioning in industrial applications

  • Guerra Paradas, Edmundo
  • Munguía Alcalá, Rodrigo Francisco
  • Grau Saldes, Antoni
This work presents a solution to localize Unmanned Autonomous Vehicles with respect to pipes and other cylindrical elements found in inspection and maintenance tasks both in industrial and civilian infrastructures. The proposed system exploits the different features of vision and laser based sensors, combining them to obtain accurate positioning of the robot with respect to the cylindrical structures. A probabilistic (RANSAC-based) procedure is used to segment possible cylinders found in the laser scans, and this is used as a seed to accurately determine the robot position through a computer vision system. The priors obtained from the laser scan registration help to solve the problem of determining the apparent contour of the cylinders. In turn this apparent contour is used in a degenerate quadratic conic estimation, enabling to visually estimate the pose of the cylinder., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Quadrotor multi-model for control purposes

  • Grau Saldes, Antoni
  • Bolea Monte, Yolanda
  • Sanfeliu Cortés, Alberto
In this work, a multi-model of a quadrotor is developed in order to control this system. The kinematic model of each part of the quadrotor will be derived using the Euler angles, and also the dynamics model of the quadrotor will be calculated based on the first principles of a rigid body using the Newton-Euler formulation. Furthermore, the following assumptions are used :1) The structure is completely rigid and perfectly symmetric. 2) The center of mass is in the origin of the quadrotor fixed frame. 3) The thrusts are proportional to the square of the motors rotational speed. A state-space model (kinematics and dynamics) is developed by physical laws. But, this deduced model presents several no linearities that are produced by three factors: the orientation (Pitch, Roll and Yaw), the control action and the angular velocities. To be able to control the quadrotor system in simple, linear and manageable way, it is necessary to linearize the system. Two method are possible: a classical linearization around several set-points and a multi-model linearization. In this case, a multi-model linearization is proposed due to the obtained control model will be used to compute a multi-model controller using fuzzy techniques. Fuzzy control techniques are suitable for linear parameter varying systems with no linearities, as our quadrotor., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

  • Santamaria Navarro, Àngel
  • Loianno, Giuseppe
  • Solà Ortega, Joan
  • Kumar, Vijay
  • Andrade-Cetto, Juan
The final publication is available at link.springer.com, The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Random clustering ferns for multimodal object recognition

  • Villamizar Vergel, Michael Alejandro
  • Garrell Zulueta, Anais
  • Sanfeliu Cortés, Alberto
  • Moreno-Noguer, Francesc
The final publication is available at link.springer.com, We propose an efficient and robust method for the recognition of objects exhibiting multiple intra-class modes, where each one is associated with a particular object appearance. The proposed method, called random clustering ferns, combines synergically a single and real-time classifier, based on the boosted assembling of extremely randomized trees (ferns), with an unsupervised and probabilistic approach in order to recognize efficiently object instances in images and discover simultaneously the most prominent appearance modes of the object through tree-structured visual words. In particular, we use boosted random ferns and probabilistic latent semantic analysis to obtain a discriminative and multimodal classifier that automatically clusters the response of its randomized trees in function of the visual object appearance. The proposed method is validated extensively in synthetic and real experiments, showing that the method is capable of detecting objects with diverse and complex appearance distributions in real-time performance., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Visual-based SLAM configurations for cooperative multi-UAV systems with a lead agent: an observability-based approach

  • Trujillo, Juan Carlos
  • Munguía Alcalá, Rodrigo Francisco
  • Guerra Paradas, Edmundo
  • Grau Saldes, Antoni
In this work, the problem of the cooperative visual-based SLAM for the class of multi-UA systems that integrates a lead agent has been addressed. In these kinds of systems, a team of aerial robots flying in formation must follow a dynamic lead agent, which can be another aerial robot, vehicle or even a human. A fundamental problem that must be addressed for these kinds of systems has to do with the estimation of the states of the aerial robots as well as the state of the lead agent. In this work, the use of a cooperative visual-based SLAM approach is studied in order to solve the above problem. In this case, three different system configurations are proposed and investigated by means of an intensive nonlinear observability analysis. In addition, a high-level control scheme is proposed that allows to control the formation of the UAVs with respect to the lead agent. In this work, several theoretical results are obtained, together with an extensive set of computer simulations which are presented in order to numerically validate the proposal and to show that it can perform well under different circumstances (e.g., GPS-challenging environments). That is, the proposed method is able to operate robustly under many conditions providing a good position estimation of the aerial vehicles and the lead agent as well., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Teaching robot's proactive behavior using human assistance

  • Garrell Zulueta, Anais
  • Villamizar Vergel, Michael Alejandro
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, In recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status. We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Teaching robot’s proactive behavior using human assistance

  • Garrell Zulueta, Anais
  • Villamizar Vergel, Michael Alejandro
  • Moreno-Noguer, Francesc
  • Sanfeliu Cortés, Alberto
The final publication is available at link.springer.com, In recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status.We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Delayed monocular SLAM approach applied to unmanned aerial vehicles

  • Munguía Alcalá, Rodrigo Francisco
  • Urzua, Sarquis
  • Grau Saldes, Antoni
In recent years, many researchers have addressed the issue of making Unmanned Aerial Vehicles (UAVs) more and more autonomous. In this context, the state estimation of the vehicle position is a fundamental necessity for any application involving autonomy. However, the problem of position estimation could not be solved in some scenarios, even when a GPS signal is available, for instance, an application requiring performing precision manoeuvres in a complex environment. Therefore, some additional sensory information should be integrated into the system in order to improve accuracy and robustness. In this work, a novel vision-based simultaneous localization and mapping (SLAM) method with application to unmanned aerial vehicles is proposed. One of the contributions of this work is to design and develop a novel technique for estimating features depth which is based on a stochastic technique of triangulation. In the proposed method the camera is mounted over a servo-controlled gimbal that counteracts the changes in attitude of the quadcopter. Due to the above assumption, the overall problem is simplified and it is focused on the position estimation of the aerial vehicle. Also, the tracking process of visual features is made easier due to the stabilized video. Another contribution of this work is to demonstrate that the integration of very noisy GPS measurements into the system for an initial short period of time is enough to initialize the metric scale. The performance of this proposed method is validated by means of experiments with real data carried out in unstructured outdoor environments. A comparative study shows that, when compared with related methods, the proposed approach performs better in terms of accuracy and computational time., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Detection and positioning of pipes and columns with autonomous multicopter drones

  • Guerra Paradas, Edmundo
  • Munguía Alcalá, Rodrigo Francisco
  • Bolea Monte, Yolanda
  • Grau Saldes, Antoni
A multimodal sensory array to accurately position aerial multicopter drones with respect to pipes has been studied, and a solution exploiting both LiDAR and vision sensors has been proposed. Several challenges, including detection of pipes and other cylindrical elements in sensor space and validation of the elements detected, have been studied. A probabilistic parametric method has been applied to segment and position cylinders with LIDAR, while several vision-based techniques have been tested to find the contours of the pipe, combined with conic estimation cylinder pose recovery. Multiple solutions have been studied and analyzed, evaluating their results. This allowed proposing an approach that combines both LiDAR and vision to produce robust and accurate pipe detection. This combined solution is validated with real experimental data., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Cooperative monocular-based SLAM for multi-UAV systems in GPS-denied environments

  • Trujillo, Juan Carlos
  • Munguía Alcalá, Rodrigo Francisco
  • Guerra Paradas, Edmundo
  • Grau Saldes, Antoni
This work presents a cooperative monocular-based SLAM approach for multi-UAV systems that can operate in GPS-denied environments. The main contribution of the work is to show that, using visual information obtained from monocular cameras mounted onboard aerial vehicles flying in formation, the observability properties of the whole system are improved. This fact is especially notorious when compared with other related visual SLAM configurations. In order to improve the observability properties, some measurements of the relative distance between the UAVs are included in the system. These relative distances are also obtained from visual information. The proposed approach is theoretically validated by means of a nonlinear observability analysis. Furthermore, an extensive set of computer simulations is presented in order to validate the proposed approach. The numerical simulation results show that the proposed system is able to provide a good position and orientation estimation of the aerial vehicles flying in formation., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

MSClique: Multiple structure discovery through the maximum weighted clique problem

  • Sanromà Güell, Gerard
  • Peñate Sánchez, Adrián
  • Alquézar Mancho, René
  • Serratosa Casanelles, Francesc
  • Moreno-Noguer, Francesc
  • Andrade-Cetto, Juan
  • González Ballester, Miguel Ángel
We present a novel approach for feature correspondence and multiple structure discovery in computer vision. In contrast to existing methods, we exploit the fact that point-sets on the same structure usually lie close to each other, thus forming clusters in the image. Given a pair of input images, we initially extract points of interest and extract hierarchical representations by agglomerative clustering. We use the maximum weighted clique problem to find the set of corresponding clusters with maximum number of inliers representing the multiple structures at the correct scales. Our method is parameter-free and only needs two sets of points along with their tentative correspondences, thus being extremely easy to use. We demonstrate the effectiveness of our method in multiple-structure fitting experiments in both publicly available and in-house datasets. As shown in the experiments, our approach finds a higher number of structures containing fewer outliers compared to state-of-the-art methods., Peer Reviewed, Postprint (published version)
idUS. Depósito de Investigación de la Universidad de Sevilla
Publicación Artículo científico (JournalArticle)

On-Line RSSI-Range Model Learning for Target Localization and Tracking

  • Martínez de Dios, José Ramiro
  • Ollero Baturone, Aníbal
  • Fernández Jiménez, Francisco José
  • Regoli, Carolina
This article belongs to the Special Issue QoS in Wireless Sensor/Actuator Networks and Systems: http://www.mdpi.com/journal/jsan/special_issues/QoS_netw_syst, The interactions of Received Signal Strength Indicator (RSSI) with the environment are very difficult to be modeled, inducing significant errors in RSSI-range models and highly disturbing target localization and tracking methods. Some techniques adopt a training-based approach in which they off-line learn the RSSI-range characteristics of the environment in a prior training phase. However, the training phase is a time-consuming process and must be repeated in case of changes in the environment, constraining flexibility and adaptability. This paper presents schemes in which each anchor node on-line learns its RSSI-range models adapted to the particularities of its environment and then uses its trained model for target localization and tracking. Two methods are presented. The first uses the information of the location of anchor nodes to dynamically adapt the RSSI-range model. In the second one, each anchor node uses estimates of the target location –anchor nodes are assumed equipped with cameras—to on-line adapt its RSSI-range model. The paper presents both methods, describes their operation integrated in localization and tracking schemes and experimentally evaluates their performance in the UBILOC testbed, Unión Europea EU Project MULTIDRONE H2020-ICT-2016-2017/H2020-ICT-2016-1, Unión Europea EU Project AEROARMS H2020-ICT-2014-1-644271, AEROMAIN Spanish R&D plan DPI2014-59383-C2-1-R, Unión Europea EU Project AEROBI H2020-ICT-2015-1-687384