Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 158
Encontrada(s) 2 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Real-time segmentation of stereo videos on a portable system with a mobile GPU

  • Abramov, Alexey
  • Pauwels, Karl
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
In mobile robotic applications, visual information needs to be processed fast despite resource limitations of the mobile system. Here a novel real-time framework for model-free spatio-temporal segmentation of stereo videos is presented. It combines real-time optical flow and stereo with image segmentation and runs on a portable system with an integrated mobile GPU. The system performs on-line, automatic and dense segmentation of stereo videos and serves as a visual front-end for preprocessing in mobile robots, providing a condensed representation of the scene which can potentially be utilized in various applications, e.g., object manipulation, manipulation recognition, visual servoing. The method was tested on real-world sequences with arbitrary motions including videos acquired with a moving camera., The work has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Cooperation, Theme 3, Information and Communication Technologies) under grant agreement no. 269959, IntellAct. B. Dellen was supported by the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship. K. Pauwels acknowledges support from the Spanish Ministry for Science and Innovation via a Juan de la Cierva fellowship (JCI-2011-11019)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Robot learning from demonstration in the force domain

  • Rozo, Leonel
  • Jiménez Schlegl, Pablo
  • Torras, Carme
6 páginas, 6 figuras.-- Presentado al Workshop on Agents Learning Interactively from Human Teachers(ALIHT) celebrado en Barcelona del 16 al 22 de julio de 2011., Researchers are becoming aware of the importance of other information sources besides visual data in robot learning by demonstration (LbD). Force-based perceptions are shown to convey very relevant information – missed by visual and position sensors – for learning specific tasks. In this paper, we review some recent works using forces as input data in LbD and Human-Robot interaction (HRI) scenarios, and propose a complete learning framework for teaching force-based manipulation skills to a robot through a haptic device. We suggest to use haptic interfaces not only as a demonstration tool but also as a communication channel between the human and the robot, getting the teacher more involved in the teaching process by experiencing the force signals sensed by the robot. Within the proposed framework, we provide solutions for treating force signals, extracting relevant information about the task, encoding the training data and generalizing to perform successfully under unknown conditions., This research is partially sponsored by the European projects GARNICS (FP7-247947) and IntellAct (FP7-269959), the Spanish projects MIPRCV (Consolider Ingenio 2010 project CSD2007-00018) and PAU (DPI2008-06022). L. Rozo was supported by the CSIC under a JAE-PREDOC scholarship., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Robot learning from demonstration of force-based tasks with multiple solution trajectories

  • Rozo, Leonel
  • Jiménez Schlegl, Pablo
  • Torras, Carme
Presentado al ICAR 2011 celebrado en Tallin (Estonia) del 20 al 23 de junio de 2011., A learning framework with a bidirectional communication channel is proposed, where a human performs several demonstrations of a task using a haptic device (providing him/her with force-torque feedback) while a robot captures these executions using only its force-based perceptive system. Our work departs from the usual approaches to learning by demonstration in that the robot has to execute the task blindly, relying only on force-torque perceptions, and, more essential, we address goal-driven manipulation tasks with multiple solution trajectories, whereas most works tackle tasks that can be learned by just finding a generalization at the trajectory level. To cope with these multiple-solution tasks, in our framework demonstrations are represented by means of a Hidden Markov Model (HMM) and the robot reproduction of the task is performed using a modified version of Gaussian Mixture Regression that incorporates temporal information (GMRa) through the forward variable of the HMM. Also, we exploit the haptic device as a teaching and communication tool in a human-robot interaction context, as an alternative to kinesthetic-based teaching systems. Results show that the robot is able to learn a container-emptying task relying only on force-based perceptions and to achieve the goal from several non-trained initial conditions., This research is partially sponsored by the European projects GARNICS (FP7-247947) and IntellAct (FP7-269959), the Spanish projects Multimodal Interaction in Pattern Recognition and Computer Vision (MIPRCV) (Consolider Ingenio 2010 project CSD2007-00018) and PAU (DPI2008-06022). L. Rozo was supported by the CSIC under a JAE-PREDOC scholarship, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Execution of a dual-object (pushing) action with semantic event chains

  • Aksoy, Eren Erdal
  • Dellen, Babette
  • Tamosiunaite, Minija
  • Wörgötter, Florentin
Presentado en 11th IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS), Bled, Slovenia, 26-28 October, 2011., Execution of a manipulation after learning from demonstration many times requires intricate planning and control systems or some form of manual guidance for a robot. Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example., The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Cooperation, Theme 3, Information and Communication Technologies) under grant agreement no. 270273, Xperience and grant agreement no. 269959, IntellAct. B.D. acknowledges support from the Spanish Ministry for Science and Innovation via a Ramon y Cajal Fellowship., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Kinematic Bézier maps

  • Ulbrich, Stefan
  • Ruiz de Angulo, Vicente
  • Torras, Carme
  • Asfour, Tamim
  • Dillman, Rudiger
16 páginas, 18 figuras., The kinematics of a robot with many degrees of freedom is a very complex function. Learning this function for a large workspace with a good precision requires a huge number of training samples, i.e., robot movements. In this work, we introduce the Kinematic Bézier Map (KB-Map), a parametrizable model without the generality of other systems, but whose structure readily incorporates some of the geometric constraints of a kinematic function. In this way, the number of training samples required is drastically reduced. Moreover, the simplicity of the model reduces learning to solving a linear least squares problem. Systematic experiments have been carried out showing the excellent interpolation and extrapolation capabilities of KB-Maps and their relatively low sensitivity to noise., The work described in this paper was partially conducted within the EU Cognitive Systems projects IntellAct (FP-7-269959) and GRASP (FP-7-215821) funded by the European Commission. V. Ruiz de Angulo and C. Torras acknowledge support from the Generalitat de Catalunya under the consolidated Robotics group, and from the Spanish Ministry of Science and Education, under the projects DPI2010-18449 and CSD2007-00018, respecively., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Segmentation of dynamic objects from laser data

  • Ortega, Agustín
  • Andrade-Cetto, Juan
Presentado al ECMR 2011 celebrado en Örebro (Sweden) del 7 al 9 de septiembre., We present a method to segment dynamic objects from high-resolution low-rate laser scans. Data points are tagged as static or dynamic based on the classification of pixel data from registered imagery. Per-pixel background classes are adapted online as Gaussian mixtures, and their matching 3D points are classified accordingly. Special attention is paid to the correct calibration and synchronization of the scanner with the the accessory camera. Results of the method are shown for a small indoor sequence with several people following arbitrarily different trajectories., This work has been partially supported by the Mexican Council of Science and Technology with a PhD Scholarship to A. Ortega, by the Spanish Ministry of Science and Innovation under projects PAU (DPI2008-06022) and MIPRCV Consolider Ingenio (CSD2007-018), and by the CEEDS (FP7-ICT- 2009-5-95682) and INTELLACT (FP7-ICT2009-6-269959) projects of the EU. The authors thank M. Morta for the development of the 3D laser used for this research., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

FaMSA: Fast multi-scan alignment with partially known correspondences

  • Teniente, Ernesto H.
  • Andrade-Cetto, Juan
Presentado al ECMR 2011 celebrado en Örebro (Sweden) del 7 al 9 de septiembre., This paper presents FaMSA, an efficient method to boost 3D scan registration from partially known correspondence sets. This situation is typical at loop closure in large laser-based mapping sessions. In such cases, scan registration for consecutive point clouds has already been made during open loop traverse, and the point match history can be used to speed up the computation of new scan matches. FaMSA allows to quickly match a new scan with multiple consecutive scans at a time, with the consequent benefits in computational speed. Registration error is shown to be comparable to that of independent scan alignment. Results are shown for dense 3D outdoor scan matching., This work has been partially supported by the Mexican Council of Science and Technology with a PhD Scholarship to E.H. Teniente, by the Spanish Ministry of Science and Innovation under projects PAU (DPI2008-06022) and MIPRCV Consolider Ingenio (CSD2007-018), and by the CEEDS (FP7-ICT- 2009-5-95682) and INTELLACT (FP7-ICT2009-6-269959) projects of the EU., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Depth-supported real-time video segmentation with the Kinect

  • Alexey, Abramov
  • Pauwels, Karl
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
Trabajo presentado al WACV celebrado en Breckenridge (USA) del 9 al 11 de enero de 2012., We present a real-time technique for the spatiotemporal segmentation of color/depth movies. Images are segmented using a parallel Metropolis algorithm implemented on a GPU utilizing both color and depth information, acquired with the Microsoft Kinect. Segments represent the equilibrium states of a Potts model, where tracking of segments is achieved by warping obtained segment labels to the next frame using real-time optical flow, which reduces the number of iterations required for the Metropolis method to encounter the new equilibrium state. By including depth information into the framework, true objects boundaries can be found more easily, improving also the temporal coherency of the method. The algorithm has been tested for videos of medium resolutions showing human manipulations of objects. The framework provides an inexpensive visual front end for visual preprocessing of videos in industrial settings and robot labs which can potentially be used in various applications., This research has received funding by the EU GARNICS project FP7-247947 and the EU IntellAct project FP7-269959. B.Dellen acknowledges support from the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship. K. Pauwels acknowledges support from CEI BioTIC GENIL (CEB09-0010) of the MICINN CEI program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Dense outdoor 3D mapping and navigation with Pose SLAM

  • Teniente, Ernesto H.
  • Valencia, Rafael
  • Andrade-Cetto, Juan
Presentado al Workshop de Robótica Experimental celebrado en Sevilla del 28 al 29 de noviembre de 2011., Pose SLAM is the variant of SLAM where only the robot trajectory is estimated and in which landmarks are solely used to compute relative constraints between robot poses. In previous work, we have developed efficient methods to build Pose SLAM maps that ponder the information content on odometry and measurement links to keep the graph of poses sparse. In this paper we show results of Pose SLAM mapping with our custom built 3D laser and an outdoor all-terrain mobile robot. Finally, we argue that Pose SLAM graphs can be directly used as belief roadmaps and, thus, used for path planning under uncertainty. We show how to plan trajectories with the lowest accumulated robot pose uncertainty, i.e., the most reliable path to the goal, taking into account the encoded uncertainty in the map. Results of this navigation strategy are demonstrated with our outdoor robot., This work has been partially supported by the Mexican Council of Science and Technology with PhD Scholarships to E.H. Teniente and R. Valencia, by the Spanish Ministry of Science and Innovation under projects PAU (DPI2008-06022) and MIPRCV Consolider Ingenio (CSD2007-018), and by the CEEDS (FP7-ICT-2009-5-95682) and INTELLACT (FP7-ICT2009-6-269959) projects of the EU., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Using depth and appearance features for informed robot grasping of highly wrinkled clothes

  • Ramisa, Arnau
  • Alenyà, Guillem
  • Moreno-Noguer, Francesc
  • Torras, Carme
Trabajo presentado al ICRA celebrado en Minnesota del 14 al 18 de mayo 2012., Detecting grasping points is a key problem in cloth manipulation. Most current approaches follow a multiple re-grasp strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields to a desired configuration. In this paper, by contrast, we circumvent the need for multiple re-graspings by building a robust detector that identifies the grasping points, generally in one single step, even when clothes are highly wrinkled. In order to handle the large variability a deformed cloth may have, we build a Bag of Features based detector that combines appearance and 3D geometry features. An image is scanned using a sliding window with a linear classifier, and the candidate windows are refined using a non-linear SVM and a “grasp goodness” criterion to select the best grasping point. We demonstrate our approach detecting collars in deformed polo shirts, using a Kinect camera. Experimental results show a good performance of the proposed method not only in identifying the same trained textile object part under severe deformations and occlusions, but also the corresponding part in other clothes, exhibiting a degree of generalization., This work was supported by the Spanish Ministry of Science and Innovation under projects PAU+ DPI2011-27510, Consolider projects MIPRCV CSD2007-00018 and AT CSD2007-0022, the EU Project IntellAct FP7-ICT2009-6-269959 and by AGAUR through SGR-00155 and SGR-1434., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

A joint model for 2D and 3D pose estimation from a single image

  • Simo-Serra, Edgar
  • Quattoni, Ariadna
  • Torras, Carme
  • Moreno-Noguer, Francesc
Trabajo presentado a la CVPR celebrada en Portland del 23 al 28 de junio de 2013., We introduce a novel approach to automatically recover 3D human pose from a single image. Most previous work follows a pipelined approach: initially, a set of 2D features such as edges, joints or silhouettes are detected in the image, and then these observations are used to infer the 3D pose. Solving these two problems separately may lead to erroneous 3D poses when the feature detector has performed poorly. In this paper, we address this issue by jointly solving both the 2D detection and the 3D inference problems. For this purpose, we propose a Bayesian framework that integrates a generative model based on latent variables and discriminative 2D part detectors based on HOGs, and perform inference using evolutionary algorithms. Real experimentation demonstrates competitive results, and the ability of our methodology to provide accurate 2D and 3D pose estimations even when the 2D detectors are inaccurate., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under projects PAU+ DPI2011-27510, CINNOVA 201150E088 and 2009SGR155; by the EU project IntellAct FP7-ICT2009-6-269959 and by the ERA-Net CHISTERA project VISEN., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Depth-supported real-time video segmentation with the Kinect

  • Alexey, Abramov
  • Pauwels, Karl
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
This research has received funding by the EU GARNICS project FP7-247947 and the EU IntellAct project FP7-269959. B.Dellen acknowledges support from the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship. K. Pauwels acknowledges support from CEI BioTIC GENIL (CEB09-0010) of the MICINN CEI program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

External force estimation during compliant robot manipulation

  • Colomé, Adrià
  • Pardo, Diego
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado a la ICRA celebrada en Karlsruhe del 6 al 10 de mayo de 2013., This paper presents a method to estimate external forces exerted on a manipulator, avoiding the use of a sensor. The method is based on task-oriented dynamics model learning and a robust disturbance state observer. The combination of both leads to an efficient torque observer that can be incorporated to any control scheme. The use of a learned based approach avoids the need of analytical models of friction or Coriolis dynamics effects., This work is partially supported by EU Projects IntellAct (FP7-269959) and RALP (FP7-288233), by the Spanish Ministry of Science and Innovation under project PAU + DPI2011-27510, and by the Catalan Research Commission through SGR-00155, Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

FINDDD: A fast 3D descriptor to characterize textiles for robot manipulation

  • Ramisa, Arnau
  • Alenyà, Guillem
  • Moreno-Noguer, Francesc
  • Torras, Carme
Trabajo presentado al IROS celebrado en Tokyo del 3 al 7 de noviembre de 2013., Most current depth sensors provide 2.5D range images in which depth values are assigned to a rectangular 2D array. In this paper we take advantage of this structured information to build an efficient shape descriptor which is about two orders of magnitude faster than competing approaches, while showing similar performance in several tasks involving deformable object recognition. Given a 2D patch surrounding a point and its associated depth values, we build the descriptor for that point, based on the cumulative distances between their normals and a discrete set of normal directions. This processing is made very efficient using integral images, even allowing to compute descriptors for every range image pixel in a few seconds. The discriminative power of our descriptor, dubbed FINDDD, is evaluated in three different scenarios: recognition of specific cloth wrinkles, instance recognition from geometry alone, and detection of reliable and informed grasping points., This research is partially funded by the Spanish Ministry of Science and Innovation under project DPI2011-27510, by the EU Project IntellAct FP7-ICT2009-6-269959 and by the ERA-Net CHISTERA project ViSen PCIN-2013-047. A. Ramisa worked under CSIC/FSE JAE-Doc grant., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Large-scale image classification using ensembles of nested dichotomies

  • Ramisa, Arnau
  • Torras, Carme
Trabajo presentado al 16th International Conference of the Catalan Association for Artificial Intelligence en Vic del 23 al 25 de octubre de 2013., Many techniques to reduce the cost at test time in large-scale problems involve a hierarchical organization of classifiers, but are either too expensive to learn or degrade the classification performance. Conversely, in this work we show that using ensembles of randomized hierarchical decompositions of the original problem can both improve the accuracy and reduce the computational complexity at test time. The proposed method is evaluated in the ImageNet Large Scale Visual Recognition Challenge'10, with promising results., This work was supported by the EU project IntellAct FP7-269959, and by the Catalan Research Commission through SGR-00155. A. Ramisa worked under a JAE-Doc grant from CSIC and FSE., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Learning collaborative impedance-based robot behaviors

  • Rozo, Leonel
  • Calinon, Sylvain
  • Caldwell, Darwin G.
  • Jiménez Schlegl, Pablo
  • Torras, Carme
Trabajo presentado a la 27th AAAI Conference celebrada del 14 al 18 de julio de 2013 en Washington (USA)., Research in learning from demonstration has focused on transferring movements from humans to robots. However, a need is arising for robots that do not just replicate the task on their own, but that also interact with humans in a safe and natural way to accomplish tasks cooperatively. Robots with variable impedance capabilities opens the door to new challenging applications, where the learning algorithms must be extended by encapsulating force and vision information. In this paper we propose a framework to transfer impedance-based behaviors to a torque-controlled robot by kinesthetic teaching. The proposed model encodes the exam- ples as a task-parameterized statistical dynamical system, where the robot impedance is shaped by estimating virtual stiffness matrices from the set of demonstrations. A collaborative assembly task is used as testbed. The results show that the model can be used to modify the robot impedance along task execution to facilitate the collaboration, by triggering stiff and compliant behaviors in an on-line manner to adapt to the user's actions., This work was partially supported by the STIFF-FLOP European project (FP7-ICT-287728), IntellAct European project (FP7-269959) and the Spanish project PAU+ (DPI2011-27510). L. Rozo was supported by the CSIC under a JAE-PREDOC scholarship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Survey on assembly sequencing: a combinatorial and geometrical perspective

  • Jiménez Schlegl, Pablo
A systematic overview on the subject of assembly sequencing is presented. Sequencing lies at the core of assembly planning, and variants include finding a feasible sequence - respecting the precedence constraints between the assembly operations -, or determining an optimal one according to one or several operational criteria. The different ways of representing the space of feasible assembly sequences are described, as well as the search and optimization algorithms that can be used. Geometry plays a fundamental role in devising the precedence constraints between assembly operations, and this is the subject of the second part of the survey, which treats also motion in contact in the context of the actual performance of assembly operations. © 2011 Springer Science+Business Media, LLC., This work has been partially supported by the European project IntellAct (FP7-269959), by the Spanish Ministry of Science and Innovation under project PAU DPI2008-06022, and by the Catalan Research Commission through the Consolidated Robotics Group., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Information-gain view planning for free-form object reconstruction with a 3D ToF camera

  • Foix, Sergi
  • Kriegel, Simon
  • Fuchs, Stefan
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado al 14th International Conference ACIVS celebrada en la República Checa del 4 al 7 de septiembre de 2012., Active view planning for gathering data from an unexplored 3D complex scenario is a hard and still open problem in the computer vision community. In this paper, we present a general task-oriented approach based on an information-gain maximization that easily deals with such a problem. Our approach consists of ranking a given set of possible actions, based on their task-related gains, and then executing the best-ranked action to move the required sensor. An example of how our approach behaves is demonstrated by applying it over 3D raw data for real-time volume modelling of complex-shaped objects. Our setting includes a calibrated 3D time-of-flight (ToF) camera mounted on a 7 degrees of freedom (DoF) robotic arm. Noise in the sensor data acquisition, which is too often ignored, is here explicitly taken into account by computing an uncertainty matrix for each point, and refining this matrix each time the point is seen again. Results show that, by always choosing the most informative view, a complete model of a 3D free-form object is acquired and also that our method achieves a good compromise between speed and precision., This work has been partially supported by the EU IntellAct project FP7-ICT2009-6- 269959, the Spanish Ministry of Science and Innovation under the project DPI2011- 27510, and by KUKA Roboter GmbH. S. Foix is supported by PhD fellowship from CSIC’s JAE program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Object detection methods for robot grasping: Experimental assessment and tuning

  • Rigual, Ferran
  • Ramisa, Arnau
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado a la 15th Catalan Conference on Artificial Intelligence (CCIA) celebrada en Alicante del 24 al 26 de octubre de 2012., In this work we address the problem of object detection for the purpose of object manipulation in a service robotics scenario. Several implementations of state-of-the-art object detection methods were tested, and the one with the best performance was selected. During the evaluation, three main practical limitations of current methods were identified in relation with long-range object detection, grasping point detection and automatic learning of new objects; and practical solutions are proposed for the last two. Finally, the complete pipeline is evaluated in a real grasping experiment., This research is partially funded by the Spanish Ministry of Science and Innovation under projects DPI2008-06022 and MIPRCV Consolider Ingenio CSD2007-00018, and the Catalan Research Commission, and the EU Project IntellAct FP7-ICT2009-6-269959. A. Ramisa worked under the JAE-DOC grant from the CSIC and the FSE., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

POMDP approach to robotized clothes separation

  • Monsó, Pol
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado al IROS celebrado en Vilamoura (Portugal) del 7 al 12 de octubre de 2012., Rigid object manipulation with robots has mainly relied on precise, expensive models and deterministic sequences. Given the great complexity of accurately modeling deformable objects, their manipulation seems to call for a rather different approach. This paper proposes a probabilistic planner, based on a Partially Observable Markov Decision Process (POMDP), targeted at reducing the inherent uncertainty of deformable object sorting. It is shown that a small set of unreliable actions and inaccurate perceptions suffices to accomplish the task, provided faithful statistics on both of them are collected beforehand. The planner has been applied to a clothes sorting task in a real case context with a depth and color sensor and a robotic arm. Experimental results show the promise of the approach since more than 95% certainty of having isolated a piece of clothing is reached in an average of four steps for quite entangled initial clothing configurations., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under projects PAU+ DPI2011-27510 and the Catalan Research Commission 2009SGR155; and by the EU project IntellAct FP7-269959., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Redundant inverse kinematics: Experimental comparative review and two enhancements

  • Colomé, Adrià
  • Torras, Carme
Trabajo presentado al IROS celebrado en Vilamoura (Portugal) del 7 al 12 de octubre de 2012., Motivated by the need of a robust and practical Inverse Kinematics (IK) algorithm for the WAM robot arm, we reviewed the most used closed-loop methods for redundant robots, analysing their main points of concern: convergence, numerical error, singularity handling, joint limit avoidance, and the capability of reaching secondary goals. As a result of the experimental comparison, we propose two enhancements. The first is to filter the singular values of the Jacobian matrix before calculating its pseudoinverse in order to obtain a more numerically robust result. The second is to combine a continuous task priority strategy with selective damping to generate smoother trajectories. Experimentation on the WAM robot arm shows that these two enhancements yield an IK algorithm that improves on the reviewed state-of-the-art ones, in terms of the good compromise it achieves between time step length, Jacobian conditioning, multiple task performance, and computational time, thus constituting a very solid option in practice. This proposal is general and applicable to other redundant robots., This work is partially funded by the CogSys STREP IntellAct project (FP7-ICT-269959), the CogSys STREP Project GARNICS (FP7-ICT-247947) and the Catalan Research Commission through grant 2009SGR155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Single image 3D human pose estimation from noisy observations

  • Simo-Serra, Edgar
  • Ramisa, Arnau
  • Alenyà, Guillem
  • Torras, Carme
  • Moreno-Noguer, Francesc
Trabajo presentado al CVPR celebrado en Providence (USA) del 16 al 21 de junio de 2012., Markerless 3D human pose detection from a single image is a severely underconstrained problem because different 3D poses can have similar image projections. In order to handle this ambiguity, current approaches rely on prior shape models that can only be correctly adjusted if 2D image features are accurately detected. Unfortunately, although current 2D part detector algorithms have shown promising results, they are not yet accurate enough to guarantee a complete disambiguation of the 3D inferred shape. In this paper, we introduce a novel approach for estimating 3D human pose even when observations are noisy. We propose a stochastic sampling strategy to propagate the noise from the image plane to the shape space. This provides a set of ambiguous 3D shapes, which are virtually undistinguishable from their image projections. Disambiguation is then achieved by imposing kinematic constraints that guarantee the resulting pose resembles a 3D human shape. We validate the method on a variety of situations in which state-of-the-art 2D detectors yield either inaccurate estimations or partly miss some of the body parts., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under projects PAU+ DPI2011-27510 and Consolider Ingenio CSD2007-00018; and by the EU project IntellAct FP7-269959. A. Ramisa is supported by a CSIC JAE-DOC grant co-financed by FSE., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Planning surface cleaning tasks by learning uncertain drag actions outcomes

  • Martínez, David
  • Alenyà, Guillem
  • Torras, Carme
Presentado al 23rd ICAPS (Workshop on PlanRob) 2013 celebrado en Roma del 10 al 14 de junio., A method to perform cleaning tasks is presented where a robot manipulator autonomously grasps a textile and uses different dragging actions to clean a surface. Actions are imprecise, and probabilistic planning is used to select the best sequence of actions. The characterization of such actions is complex because the initial autonomous grasp of the textile introduces differences in the initial conditions that change the efficacy of the robot cleaning actions. We demonstrate that the action outcome probabilities can be learned very fast while the task is being executed, so as to progressively improve robot performance. The learner adds only a little overhead to the system compared to the improvements obtained. Experiments with a real robot show that the most effective plan varies depending on the initial grasp, and that plans become better after only a few learning iterations., This work was supported by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510, by EU Project IntellAct FP7-ICT2009-6-269959 and by the Catalan Research Commission through SGR-00155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Handling high parameter dimensionality in reinforcement learning with dynamic motor primitives

  • Colomé, Adrià
  • Alenyà, Guillem
  • Torras, Carme
Presentado al IEEE International Conference on Robotics and Automation celebrado en Karlsruhe (Alemania) del 6 al 10 de mayo de 2013., Dynamic Motor Primitives (DMP) are nowadays widely used as movement parametrization for learning trajectories, because of their linearity in the parameters, rescalation robustness and continuity. However, when learning a movement with DMP, where a set of gaussians distributed along the trajectory is used to approximate an acceleration excitation function, a very large number of gaussian approximations need to be performed. Adding them up for all joints yields too many parameters to be explored, thus requiring a prohibitive number of experiments/simulations to converge to a solution with an optimal (locally or globally) reward. We propose here two strategies to reduce this dimensionality: the first is to explore only the most significant directions in the parameter space, and the second is to add a reduced second set of gaussians that should only optimize the trajectory after fixing the gaussians that approximate the demonstrated movement., This work is partially funded by EU Project IntellAct (FP7-269959) and by the Spanish Ministry of Science and Innovation under project PAU+DPI2011-27510., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Positioning two redundant arms for cooperative manipulation of objects

  • Colomé, Adrià
  • Torras, Carme
Presentado al 6th International Workshop on Computational Kinematics (CK2013) celebrado en Barcelona del 12 al 15 de mayo., Bimanual manipulation of objects is receiving a lot of attention nowadays, but there is few literature addressing the design of the arms configuration. In this paper, we propose a way to analyze the relative positioning of two redundant arms, both equipped with spherical wrists, in order to obtain the best common workspace for grasping purposes. Considering the geometry of a robot with a spherical wrist, the Cartesian workspace can be discretized, with an easy representation of the feasible end-effector orientations at each point using bounding cones. After having characterized the workspace for one robot arm, we can evaluate how good each of the discretized poses relate with an identical arm in another position with a quality function that considers orientations. In the end, we obtain a quality value for each relative position of two arms, and we perform an optimization using genetic algorithms to obtain the best workspace for a cooperative task., This work is partially supported by EU Project IntellAct (FP7-269959), by the CSIC Project CIN-NOVA (201150E088), and by the Catalan Research Commission through SGR-00155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

A novel real-time edge-preserving smoothing filter

  • Reich, Simon
  • Abramov, Alexey
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
Presentado a la 8th International Conference on Computer Vision Theory and Applications celebrada en Barcelona del 21 al 24 de febrero de 2013., The segmentation of textured and noisy areas in images is a very challenging task due to the large variety of objects and materials in natural environments, which cannot be solved by a single similarity measure. In this paper, we address this problem by proposing a novel edge-preserving texture filter, which smudges the color values inside uniformly textured areas, thus making the processed image more workable for color-based image segmentation. Due to the highly parallel structure of the method, the implementation on a GPU runs in real-time, allowing us to process standard images within tens of milliseconds. By preprocessing images with this novel filter before applying a recent real-time color-based image segmentation method, we obtain significant improvements in performance for images from the Berkeley dataset, outperforming an alternative version using a standard bilateral filter for preprocessing. We further show that our combined approach leads to better segmentations in terms of a standard performance measure than graph-based and mean-shift segmentation for the Berkeley image dataset., The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Cooperation, Theme 3, Information and Communication Technologies) under grant agreement no. 269959, Intellact. B. Dellen acknowledges support from the Spanish Ministry of Science and Innovation through a Ramon y Cajal program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations

  • Dellen, Babette
  • Husain, Farzad
  • Torras, Carme
Presentado a la 8th International Conference on Computer Vision Theory and Applications celebrada en Barcelona del 21 al 24 de febrero de 2013., A novel framework for joint segmentation and tracking in depth videos of object surfaces is presented. Initially, the 3D colored point cloud obtained using the Kinect camera is used to segment the scene into surface patches, defined by quadratic functions. The computed segments together with their functional descriptions are then used to partition the depth image of the subsequent frame in a consistent manner with respect to the precedent frame. This way, solutions established in previous frames can be reused which improves the efficiency of the algorithm and the coherency of the segmentations along the movie. The algorithm is tested for scenes showing human and robot manipulations of objects. We demonstrate that the method can successfully segment and track the human/robot arm and object surfaces along the manipulations. The performance is evaluated quantitatively by measuring the temporal coherency of the segmentations and the segmentation covering using ground truth. The method provides a visual front-end designed for robotic applications, and can potentially be used in the context of manipulation recognition, visual servoing, and robot-grasping tasks., This research is partially funded by the EU projects GARNICS (FP7-247947) and IntellAct (FP7-269959), and the Grup consolidat 2009 SGR155. B. Dellen acknowledges support from the Spanish Ministry of Science and Innovation through a Ramon y Cajal program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Force-based robot learning of pouring skills using parametric hidden Markov models

  • Rozo, Leonel
  • Jiménez Schlegl, Pablo
  • Torras, Carme
Presentado al 9th RoMoCo celebrado en Kuslin (Polonia) del 3 al 5 de julio de 2013., Robot learning from demonstration faces new challenges when applied to tasks in which forces play a key role. Pouring liquid from a bottle into a glass is one such task, where not just a motion with a certain force profile needs to be learned, but the motion is subtly conditioned by the amount of liquid in the bottle. In this paper, the pouring skill is taught to a robot as follows. In a training phase, the human teleoperates the robot using a haptic device, and data from the demonstrations are statistically encoded by a parametric hidden Markov model, which compactly encapsulates the relation between the task parameter (dependent on the bottle weight) and the force-torque traces. Gaussian mixture regression is then used at the reproduction stage for retrieving the suitable robot actions based on the force perceptions. Computational and experimental results show that the robot is able to learn to pour drinks using the proposed framework, outperforming other approaches such as the classical hidden Markov models in that it requires less training, yields more compact encodings and shows better generalization capabilities., This research is partially sponsored by the European projects STIFF-FLOP (FP7-ICT-287728), IntellAct (FP7-269959), and the Spanish project PAU+ (DPI2011-27510). L. Rozo was supported by the CSIC under a JAE-PREDOC scholarship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Characterization of textile grasping experiments

  • Alenyà, Guillem
  • Ramisa, Arnau
  • Moreno-Noguer, Francesc
  • Torras, Carme
Presentado al International Conference on Robotics and Automation celebrado en USA del 14 al 18 de mayo de 2012., Grasping highly deformable objects, like textiles, is an emerging area of research that involves both perception and manipulation abilities. As new techniques appear, it becomes essential to design strategies to compare them. However, this is not an easy task, since the large state-space of textile objects explodes when coupled with the variability of grippers, robotic hands and robot arms performing the manipulation task. This high variability makes it very difficult to design experiments to evaluate the performance of a system in a repeatable way and compare it to others. We propose a framework to allow the comparison of different grasping methods for textile objects. Instead of measuring each component separately, we therefore propose a methodology to explicitly measure the vision-manipulation correlation by taking into account the throughput of the actions. Perceptions of deformable objects should be grouped into different clusters, and the different grasping actions available should be tested for each perception type to obtain the action-perception success ratio. This characterization potentially allows to compare very different systems in terms of specialized actions, perceptions or widely useful actions, along with the cost of performing each action. We will also show that this categorization is useful in manipulation planning of deformable objects., This work was supported by the Spanish Ministry of Science and Innovation under projects PAU+ DPI2011-27510, by the EU project INTELLACT 247947 FP7-269959 and by the Catalan Research Commission through SGR-00155. A. Ramisa worked under the JAE-DOC grant from the CSIC and the FSE., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

External force estimation for textile grasp detection

  • Colomé, Adrià
  • Pardo, Diego
  • Alenyà, Guillem
  • Torras, Carme
Presentado al International Conference on Intelligent Robots and Systems, Workshop: Beyond Robot Grasping-Modern Approaches for Learning Dynamic Manipulation celebrado en Portugal del 7 al 12 de octubre de 2012., Our current work on external force estimation without end-effector force sensor is presented. To verify if a grasp of a textile has been successful, the xternal wrench applied on the robot is computed online, with a state observer based on a LWPR model of a task., This work is partially funded by CogSys STREP Project IntellAct (FP7-ICT-269959), by the Spanish Ministry of Science and Innovation under project DPI2011-27510, and by the Catalan Research Commission through the Robotics consolidated group., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Dynamically consistent probabilistic model for robot motion learning

  • Pardo, Diego
  • Rozo, Leonel
  • Alenyà, Guillem
  • Torras, Carme
Presentado al International Conference on Intelligent Robots and Systems Workshop on Learning and Interaction in Haptic Robots (IROS LIHR) celebrado en Portugal del 7 al 12 de octubre de 2012., This work presents a probabilistic model for learning robot tasks from human demonstrations using kinesthetic teaching. The difference with respect to previous works is that a complete state of the robot is used to obtain a consistent representation of the dynamics of the task. The learning framework is based on hidden Markov models and Gaussian mixture regression, used for coding and reproducing the skills. Benefits of the proposed approach are shown in the execution of a simple self-crossing trajectory by a 7-DoF manipulator., This research is partially supported by μRALP (FP7-288233) and IntellAct (FP7-269959) projects., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Active learning of manipulation sequences

  • Martínez, David
  • Alenyà, Guillem
  • Jiménez Schlegl, Pablo
  • Torras, Carme
  • Rossmann, Jürgen
  • Wantia, Nils
  • Aksoy, Eren Erdal
  • Haller, Simon
  • Piater, Justus
Trabajo presentado al ICRA 2014 celebrado en Hong Kong del 31 de mayo al 7 de junio., We describe a system allowing a robot to learn goal-directed manipulation sequences such as steps of an assembly task. Learning is based on a free mix of exploration and instruction by an external teacher, and may be active in the sense that the system tests actions to maximize learning progress and asks the teacher if needed. The main component is a symbolic planning engine that operates on learned rules, defined by actions and their pre- and postconditions. Learned by model-based reinforcement learning, rules are immediately available for planning. Thus, there are no distinct learning and application phases. We show how dynamic plans, replanned after every action if necessary, can be used for automatic execution of manipulation sequences, for monitoring of observed manipulation sequences, or a mix of the two, all while extending and refining the rule base on the fly. Quantitative results indicate fast convergence using few training examples, and highly effective teacher intervention at early stages of learning., The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Cooperation, Theme 3, Information and Communication Technologies) under grant agreement no. 269959, IntellAct. D. Martínez is also supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant (FPU12-04173)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Dimensionality reduction and motion coordination in learning trajectories with dynamic movement primitives

  • Colomé, Adrià
  • Torras, Carme
Trabajo presentado al IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), celebrado en Chicago, Illinois (US) del 14 al 18 de septiembre., Dynamic Movement Primitives (DMP) are nowadays widely used as movement parametrization for learning trajectories, because of their linearity in the parameters, rescaling robustness and continuity. However, when learning a movement with a robot using DMP, many parameters may need to be tuned, requiring a prohibitive number of experiments/simulations to converge to a solution with a locally or globally optimal reward. We propose here strategies to palliate this dimensionality problem: the first is to explore only along the most significant directions in the parameter space, and the second is to add a reduced second set of Gaussians that would optimize the trajectory after fixing the Gaussians approximating the demonstrated movement. Both strategies result in less Gaussian computations and better performance on learning algorithms. To further speed up the learning and allow for a better biased exploration, we also propose to coordinate the motion of different joints, by computing a coordination matrix initialized with the demonstrated movement and then automatically updating it by eliminating the degrees of freedom least affecting task performance. Our three proposals have been experimentally tested and the obtained results show that similar (or even better) performance can be obtained at a significantly lower computational cost by reducing the dimensionality of the exploration space., This work is partially funded by EU Project IntellAct (FP7-269959) and by the Spanish Ministry of Science and Innovation under project PAU+DPI2011-27510 and by the CSIC Project CINNOVA (201150E088). A. Colomé is also supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant (AP2010-1989)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Learning RGB-D descriptors of garment parts for informed robot grasping

  • Ramisa, Arnau
  • Alenyà, Guillem
  • Alenyà, Guillem
  • Moreno-Noguer, Francesc
  • Torras, Carme
Robotic handling of textile objects in household environments is an emerging application that has recently received considerable attention thanks to the development of domestic robots. Most current approaches follow a multiple re-grasp strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields a desired configuration. In this work we propose a vision-based method, built on the Bag of Visual Words approach, that combines appearance and 3D information to detect parts suitable for grasping in clothes, even when they are highly wrinkled. We also contribute a new, annotated, garment part dataset that can be used for benchmarking classification, part detection, and segmentation algorithms. The dataset is used to evaluate our approach and several state-of-the-art 3D descriptors for the task of garment part detection. Results indicate that appearance is a reliable source of information, but that augmenting it with 3D information can help the method perform better with new clothing items., This research is partially funded by the Spanish Ministry of Science and Innovation under Project PAU+ DPI2011-2751, the EU Project IntellAct FP7-ICT2009-6-269959 and the ERA-Net Chistera Project ViSen PCIN-2013-047. A. Ramisa worked under the JAE-Doc grant from CSIC and FSE., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Realtime tracking and grasping of a moving object from range video

  • Husain, Farzad
  • Colomé, Adrià
  • Dellen, Babette
  • Alenyà, Guillem
  • Torras, Carme
Presentado al ICRA 2014 celebrado en Hong Kong del 31 de mayo al 7 de junio., In this paper we present an automated system that is able to track and grasp a moving object within the workspace of a manipulator using range images acquired with a Microsoft Kinect sensor. Realtime tracking is achieved by a geometric particle filter on the affine group. Based on the tracked output, the pose of a 7-DoF WAM robotic arm is continuously updated using dynamic motor primitives until a distance measure between the tracked object and the gripper mounted on the arm is below a threshold. Then, it closes its three fingers and grasps the object. The tracker works in real-time and is robust to noise and partial occlusions. Using only the depth data makes our tracker independent of texture which is one of the key design goals in our approach. An experimental evaluation is provided along with a comparison of the proposed tracker with state-of-the-art approaches, including the OpenNI-tracker. The developed system is integrated with ROS and made available as part of IRI's ROS stack., This work was supported by the EU project IntellAct FP7-269959, the project PAU+ DPI2011-27510 and the project CINNOVA 201150E088. B. Dellen was supported by the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Recognizing point clouds using conditional random fields

  • Husain, Farzad
  • Dellen, Babette
  • Torras, Carme
Trabajo presentado a la 22nd International Conference on Pattern Recognition (ICPR-2014), celebrada en Estocolmo (Suecia) del 24 al 28 de agosto., Detecting objects in cluttered scenes is a necessary step for many robotic tasks and facilitates the interaction of the robot with its environment. Because of the availability of efficient 3D sensing devices as the Kinect, methods for the recognition of objects in 3D point clouds have gained importance during the last years. In this paper, we propose a new supervised learning approach for the recognition of objects from 3D point clouds using Conditional Random Fields, a type of discriminative, undirected probabilistic graphical model. The various features and contextual relations of the objects are described by the potential functions in the graph. Our method allows for learning and inference from unorganized point clouds of arbitrary sizes and shows significant benefit in terms of computational speed during prediction when compared to a state-of-the-art approach based on constrained optimization., This work was supported by the EU project (IntellAct FP7-269959), the project PAU+ (DPI2011-27510), and the CSIC project CINNOVA (201150E088). B. Dellen was supported by the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Robust surface tracking in range image sequences

  • Husain, Farzad
  • Dellen, Babette
  • Torras, Carme
A novel robust method for surface tracking in range-image sequences is presented which combines a clustering method based on surface models with a particle-filter-based 2-D affine-motion estimator. Segmented regions obtained at previous time steps are used to create seed areas by comparing measured depth values with those obtained from surface-model fitting. The seed areas are further refined using a motion-probability region estimated by the particle-filter-based tracker through prediction of future states. This helps resolving ambiguities that arise when surfaces belonging to different objects are in physical contact with each other, for example during hand-object manipulations. Region growing allows recovering the complete segment area. The obtained segmented regions are then used to improve the predictions of the tracker for the next frame. The algorithm runs in quasi real-time and uses on-line learning, eliminating the need to have a prioriknowledge about the surface being tracked. We apply the method to in-house depth videos acquired with both time-of-flight and structured-light sensors, demonstrating object tracking in real-world scenarios, and we compare the results with those of an ICP-based tracker., This work received support from the CSIC project MVOD no. 201250E028, the EU project IntellAct FP7-269959, the project PAU+ DPI2011-27510 and the project CINNOVA 201150E088. B. Dellen was supported by the Spanish Ministry for Science and Innovation via a Ramon y Cajal fellowship RYC-2009-05324., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Planning robot manipulation to clean planar surfaces

  • Martínez, David
  • Alenyà, Guillem
  • Torras, Carme
This paper presents a new approach to plan high-level manipulation actions for cleaning surfaces in household environments, like removing dirt from a table using a rag. Dragging actions can change the distribution of dirt in an unpredictable manner, and thus the planning becomes challenging. We propose to define the problem using explicitly uncertain actions, and then plan the most effective sequence of actions in terms of time. However, some issues have to be tackled to plan efficiently with stochastic actions. States become hard to predict after executing a few actions, so replanning every few actions with newer perceptions gives the best results, and the trade-off between planning time and plan quality is also important. Finally a learner is integrated to provide adaptation to changes, such as different rag grasps, robots, or cleaning surfaces. We demonstrate experimentally, using two different robot platforms, that planning is more advantageous than simple reactive strategies for accomplishing complex tasks, while still providing a similar performance for easy tasks. We also performed experiments where the rag grasp was changed, and thus the behaviour of the dragging actions, showing that the learning capabilities allow the robot to double its performance with a new rag grasp after a few cleaning iterations., This work was supported by EU Project IntellActFP7-ICT2009-6-269959 and by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510. D. Martínez is also supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant (FPU12-04173)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Consistent depth video segmentation using adaptive surface models

  • Husain, Farzad
  • Dellen, Babette
  • Torras, Carme
We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using adaptive surface models. Starting from an initial configuration, the algorithm converges to a stable segmentation through a new iterative split-And-merge procedure, which includes an adaptive mechanism for the creation and removal of segments. This allows the segmentation to adjust to changing input data along the movie, leading to stable, temporally coherent, and traceable segments. We tested the method on a large variety of data acquired with different range imaging devices, including a structured-light sensor and a time-of-flight camera, and successfully segmented the videos into surface segments. We further demonstrated the feasibility of the approach using quantitative evaluations based on ground-truth data., This research is partially funded by the EU project IntellAct (FP7-269959), the Grup consolidat 2009 SGR155, the project PAU+ (DPI2011-27510), and the CSIC project CINNOVA (201150E088). B. Dellen acknowledges support from the Spanish Ministry of Science and Innovation through a Ramon y Cajal program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

DaLI: Deformation and light invariant descriptor

  • Simo-Serra, Edgar
  • Torras, Carme
  • Moreno-Noguer, Francesc
Recent advances in 3D shape analysis and recognition have shown that heat diffusion theory can be effectively used to describe local features of deforming and scaling surfaces. In this paper, we show how this description can be used to characterize 2D image patches, and introduce DaLI, a novel feature point descriptor with high resilience to non-rigid image transformations and illumination changes. In order to build the descriptor, 2D image patches are initially treated as 3D surfaces. Patches are then described in terms of a heat kernel signature, which captures both local and global information, and shows a high degree of invariance to non-linear image warps. In addition, by further applying a logarithmic sampling and a Fourier transform, invariance to photometric changes is achieved. Finally, the descriptor is compacted by mapping it onto a low dimensional subspace computed using Principal Component Analysis, allowing for an efficient matching. A thorough experimental validation demonstrates that DaLI is significantly more discriminative and robust to illuminations changes and image transformations than state of the art descriptors, even those specifically designed to describe non-rigid deformations., This work has been partially funded by the Spanish Ministry of Economy and Competitiveness under projects ERA-Net Chistera project ViSen PCIN-2013-047 and PAU+ DPI2011-27510, and by the EU project IntellAct FP7-ICT2009-6-269959., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Finding safe policies in model-based active learning

  • Martínez, David
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado al IROS: "Machine Learning in Planning and Control of Robot Motion Workshop" (IROS MLPC), celebrado en Chicago, Illinois (US) del 14 al 18 de septiembre., Este ítem (excepto textos e imágenes no creados por el autor) está sujeto a una licencia de Creative Commons: Attribution-NonCommercial-NoDerivs 3.0 Spain., Task learning in robotics is a time-consuming process, and model-based reinforcement learning algorithms have been proposed to learn with just a small amount of experiences. However, reducing the number of experiences used to learn implies that the algorithm may overlook crucial actions required to get an optimal behavior. For example, a robot may learn simple policies that have a high risk of not reaching the goal because they often fall into dead-ends. We propose a new method that allows the robot to reason about dead-ends and their causes. Analyzing its current model and experiences, the robot will hypothesize the possible causes for the dead-end, and identify the actions that may cause it, marking them as dangerous. Afterwards, whenever a dangerous action is included into a plan which has a high risk of leading to a dead-end, the special action request teacher confirmation will be triggered by the robot to actively confirm with a teacher that the planned risky action should be executed. This method permits learning safer policies with the addition of just a few teacher demonstration requests. Experimental validation of the approach is provided in two different scenarios: a robotic assembly task and a domain from the international planning competition. Our approach gets success ratios very close to 1 in problems where previous approaches had high probabilities of reaching dead-ends., This work was supported by EU Project IntellAct FP7-ICT2009-6-269959, by CSIC project MANIPlus 201350E102 and by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510. D. Martínez is also supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant (FPU12-04173)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Learning algorithms for robot manipulation of clothing and plant leaves

  • Torras, Carme
Trabajo presentado al ICRA Workshop on Advances in Robot Manipulation of Clothes and Flexible Objects, celebrado el 1 de junio de 2014 en Hong Kong., Research supported by the European projects PACO-PLUS, GARNICS and IntellAct, and the Spanish project PAU+., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Manipulation monitoring and robot intervention in complex manipulation sequences

  • Savarimuthu, Thiusius R.
  • Buch, Anders G.
  • Yang, Yang
  • Mustafa, Wail
  • Haller, Simon
  • Papon, Jeremie
  • Martínez, David
  • Aksoy, Eren Erdal
Trabajo presentado al IX Robotics Science and Systems: "Workshop on Robotic Monitoring" (RSS-WRM), celebrado en Berkeley (US) del 12 al 16 de julio de 2014.-- et al., Compared to machines, humans are intelligent and dexterous; they are indispensable for many complex tasks in areas such as flexible manufacturing or scientific experimentation. However, they are also subject to fatigue and inattention, which may cause errors. This motivates automated monitoring systems that verify the correct execution of manipulation sequences. To be practical, such a monitoring system should not require laborious programming., The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Cooperation, Theme 3, Information and Communication Technologies) under grant agreement no. 269959, IntellAct., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Robot manipulation in human environments: Challenges for learning algorithms

  • Torras, Carme
Resumen del trabajo presentado al Dagstuhl Seminar 2014 celebrado en Dagstuhl (Alemania) del 17 al 21 de febrero de 2014., The European projects PACO-PLUS, GARNICS and IntellAct, the Spanish projects PAU and PAU+, and the Catalan grant SGR-155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Relational reinforcement learning with guided demonstrations

  • Martínez, David
  • Alenyà, Guillem
  • Torras, Carme
Model-based reinforcement learning is a powerful paradigm for learning tasks in robotics. However, in-depth exploration is usually required and the actions have to be known in advance. Thus, we propose a novel algorithm that integrates the option of requesting teacher demonstrations to learn new domains with fewer action executions and no previous knowledge. Demonstrations allow new actions to be learned and they greatly reduce the amount of exploration required, but they are only requested when they are expected to yield a significant improvement because the teacher's time is considered to be more valuable than the robot's time. Moreover, selecting the appropriate action to demonstrate is not an easy task, and thus some guidance is provided to the teacher. The rule-based model is analyzed to determine the parts of the state that may be incomplete, and to provide the teacher with a set of possible problems for which a demonstration is needed. Rule analysis is also used to find better alternative models and to complete subgoals before requesting help, thereby minimizing the number of requested demonstrations. These improvements were demonstrated in a set of experiments, which included domains from the international planning competition and a robotic task. Adding teacher demonstrations and rule analysis reduced the amount of exploration required by up to 60% in some domains, and improved the success ratio by 35% in other domains., This research was supported by EU Project IntellAct FP7-ICT2009-6-269959 and by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510. D. Martínez was also supported by the Spanish Ministry of Education, Culture, and Sport via a FPU doctoral grant (FPU12-04173)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Service robots for citizens of the future

  • Torras, Carme
This paper summarizes the author’s presentation at the 26th Annual Conference of the Academia Europaea, Barcelona, July 2014., Erasmus Lecture 2014, Robots are no longer confined to factories; they are progressively spreading to urban, social and assistive domains. In order to become handy co-workers and helpful assistants, they must be endowed with quite different abilities from their industrial ancestors. Research on service robots aims to make them intrinsically safe to people, easy to teach by non-experts, able to manipulate not only rigid but also deformable objects, and highly adaptable to non-predefined and dynamic environments. Robots worldwide will share object and environmental models, their acquired knowledge and experiences through global databases and, together with the internet of things, will strongly change the citizens' way of life in so-called smart cities. This raises a number of social and ethical issues that are now being debated not only within the Robotics community but by society at large., The described research has been partially supported by the European projects PACO-PLUS (IST-FP6-IP-027657), GARNICS (FP7-ICT-247947) and IntellAct (FP7-ICT-269959), and the Spanish project PAU + (DPI2011-27510)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

ToF cameras for eye-in-hand robotics

  • Alenyà, Guillem
  • Foix, Sergi
  • Torras, Carme
This work was supported by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510, by the EU Project IntellAct FP7-ICT2009-6-269959 and by the Catalan Research Commission through SGR-00155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Perceiving dynamic environments: from surface geometry to semantic representation

  • Husain, Farzad
Universitat Politècnica de Catalunya, Programa de Doctorat: Automàtica, Robòtica i Visió., [EN]: Perceiving human environments is becoming increasingly fundamental with the gradual adaptation of robots for domestic use. High-level tasks such as the recognition of objects and actions need to be performed far the active engagement of the robot with its surroundings. Nowadays, the environment is primarily captured using visual information in the form of color and depth images. Visual cues obtained from these images serve as a base upan which perception-related applications are developed. Far example, using appearance models far detecting objects and extracting motion infarmation far recognizing actions. However, given the complex variations of naturally occurring scenes, extracting a set of robust visual cues becomes harder here than in other ontexts. In this thesis, we develop a hierarchy of tools to improve, the different aspects of robot perception in human-centered, possibly dynamic, environments. We start with the segmentation of single images and extend it to videos. Afterwards, we develop a surface tracking approach along with the incorporation of our video segmentation method. We then investigate the higher-level tasks of semantic segmentation and recognition. Finally, we focus on recognizing actions in videos. The introduction of Kinectstyle depth sensors is relatively new and its usage in the field of robotics cannot be found befare half a decade ago. Such sensors enable the acquisition of high-resolution color and depth images at a low cost. Given this opportunity, we dedícate a bulk of our work to the exploitation of the depth infarmation obtained using such sensors, thereby pushing forward the state-of-the-art in perception problems. The thesis is conceptually grouped into two parts. In the first part, we address the low-level tasks of segmentation and tracking with depth images. In many cases, depth data gives a better disambiguation of surface boundaries of different objects in a scene when compared to their color counterpart. We exploit this information in a novel depth segmentation scheme that fits quadratic surface models on different surfaces in a competing fashion . We further extend the method to the video domain by initializing the segmentation results and surface model parameters from the previous trame for the next trame. In this way, we successfully create a video segmentation algorithm, in which the segment label belonging to each surface becomes coherent over time. We also devise a particle-filter-based tracker that uses depth data to track a surface. The tracker is made more robust by combining it with our video segmentation approach. The segmentation results serve as a useful prior for high-level tasks. In the second part we deal with such tasks which include (i) object recognition, (ii) pixelwise object class segmentation, and (iii) action recognition . We propase (i) to address object recognition by creating context-aware conditional random field models. We show the importance of the context in object recognition by modeling geometrical relations between different objects in a scene. We perform (ii) object class segmentation using a convolutional neural network. We introduce a novel distance-from-wall feature and demonstrate its effectiveness in generating better class proposals for objects that are clase to the walls. The final part of the thesis deals with (iii) action recognition. We propase a 2D convolutional neural network extended to a concatenated 3D network that learns to extrae! features from the spatio-temporal domain of raw video data. The network is trained to predict an action label for each video. In summary, several perception aspects are addressed with the utilization of depth infarmation where available. Our main contributions are (a) the introduction of a depth video segmentation scheme, (b) a graphical model far object recognition, and our proposals of the deep learning models for (e) object class segmentation and (d) action recognition., [ES]: Los sistemas de percepción en entornos humanos son cada vez más importantes para la adaptación gradual de los robots a tareas domésticas. Tareas de alto nivel, tales como el reconocimiento de objetos y acciones, son necesarias para conseguir la participación activa del robot en dichas tareas. Hoy en día el entorno del robot es capturado principalmente usando información visual en forma de imágenes de color y profundidad. Las características visuales obtenidas a partir de estas imágenes sirven como base para el desarrollo de aplicaciones relacionadas con la percepción del robot. Por ejemplo, el uso de modelos de apariencia para la detección de objetos y la extracción de información del movimiento para el reconocimiento de acciones. Sin embargo, dado que las escenas pueden contener variaciones complejas, la extracción de un conjunto de características visuales puede convertirse en una tarea muy difícil. En la presente tesis hemos desarrollado una jerarquía de herramientas para mejorar diferentes aspectos de la percepción del robot en entornos humanos, posiblemente dinámicos. Esta tesis comienza con la segmentación de imágenes individuales, que luego se extiende a vídeos. Posteriormente, diseñamos un enfoque de seguimiento de superficies que incorpora nuestro método de segmentación de vídeos. A continuación, investigamos tareas de alto nivel para la segmentación semántica y el reconocimiento. Finalmente, nos centramos en el reconocimiento de acciones en vídeos. La introducción de sensores de profundidad tipo Kinect es relativamente nueva y su uso en el campo de la robótica empezó hace tan solo media década. Tales sensores permiten la adquisición de color y profundidad de imágenes de alta resolución a bajo coste. Dada esta oportunidad, dedicamos una buena parte de nuestro trabajo a la explotación de la información de profundidad obtenida a través de dichos sensores, mejorando el estado del arte en problemas de percepción. La tesis está conceptualmente dividida en dos partes. En primer lugar, abordamos las tareas de bajo nivel de segmentación y seguimiento con imágenes de profundidad. En muchos casos, los datos, de profundidad permiteuna mejor desambiguación de los límites de las superficies de diferentes objetos de una escena en comparación con los datos de color. Explotamos esta información en un nuevo esquema de segmentación de profundidad que ajusta modelos cuadráticos de superficies de forma competitiva. Extendemos el método a vídeos de modo que la etiquetación de superficies resulte coherente en el tiempo. También proponemos un rastreador basado en un filtro de partículas que utiliza los datos de profundidad para realizar el seguimiento de una superficie. El seguimiento se hace más robusto al combinarlo con nuestro enfoque de segmentación en vídeo. Los resultados de la segmentación son usados como información a priori para tareas de alto nivel. En la segunda parte nos ocupamos de este tipo de tareas que incluyen el (i) reconocimiento de objetos, (ii) la segmentación de clases de objetos a nivel de píxeles, y (iii) el reconocimiento de acciones. Proponemos (i) abordar el reconocimiento de objetos mediante la creación de modelos de campos aleatorios condicionales sensibles al contexto. Realizamos (ii) la segmentación de la clase del objeto utilizando una red neuronal de convolución. Se introduce una nueva característica de distancia-a-paredes y demostramos su eficacia en la mejora de la clasificación de objetos que están cerca de las paredes. La parte final de la tesis se ocupa del (iii) reconocimiento de acciones. Proponemos una red neuronal de convolución 2D extendida a una red 3D concatenada, que aprende a extraer las características del dominio espacio-temporal de los datos de vídeo. La red está capacitada para predecir la etiqueta de acción para cada vídeo., This work has been sponsored by the following: A PhD Scholarship and Projects GARNICS (FP7-ICT-2009-4-247947), CINNOVA (201150E088), PAU+(DPI2011-27510), IntellAct (FP7-ICT2009-6-269959) and Textil Rob (201550E028)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Task-oriented viewpoint planning for free-form objects

  • Foix, Sergi
A thesis submitted to the Universitat Politècnica de Catalunya to obtain the degree of Doctor of Philosophy. Doctoral programme: Automatic Control, Robotics and Computer Vision. This thesis was completed at: Institut de Robòtica i Informàtica Industrial, CSIC-UPC., [EN]: This thesis deals with active sensing and its use in real exploration tasks under both scene ambiguities and measurement uncertainties. While object modeling is the implicit objective of most of active sensing algorithms, in this work we have explored new strategies to deal with more generic and more complex tasks. Active sensing requires the ability of moving the perceptual system to gather new information. Our approach uses a robot manipulator with a 3D Time-of-Flight (ToF) camera attached to the end-effector. For a complex task, we have focused our attention on plant phenotyping. Plants are complex objects, with leaves that change their position and size along time. Valid viewpoints for a certain plant are hardly valid for a different one, even belonging to the same species. Some instruments, such as chlorophyll meters or disk sampling tools, require being precisely positioned over a particular location of the leaf. Therefore, their use requires the modeling of specific regions of interest of the plant, including also the free space needed for avoiding obstacles and approaching the leaf with tool. It is easy to observe that predefined camera trajectories are not valid here, and that usually with one single view it is very difficult to acquire all the required information. The overall objective of this thesis is to solve complex active sensing tasks by embedding their exploratory goal into a pre-estimated geometrical model, using information-gain as the fundamental guideline for the reward function. The main contributions can be divided in two groups: first, the evaluation of ToF cameras and their calibration to assess the uncertainty of the measurements (presented in Part I); and second, the proposal of a framework capable of embedding the task, modeled as free and occupied space, and that takes into account the modeled sensor's uncertainty to improve the action selection algorithm (presented in Part II). This thesis, has given rise to 14 publications, including 5 indexed journals, and its results have been used in the GARNICS European project. The complete framework is based on the Next-Best-View methodology and it can be summarized in the following main steps. First, an initial view of the object (e.g., a plant) is acquired. From this initial view and given a set of candidate viewpoints, the expected gain obtained by moving the robot and acquiring the next image is computed. This computation takes into account the uncertainty from all the different pixels of the sensor, the expected information based on a predefined task model, and the possible occlusions. Once the most promising view is selected, the robot moves, takes a new image, integrates this information intothe model, and evaluates again the set of remaining views. Finally, the task terminates when enough information is gathered. In our examples, this process enables the robot to perform a measurement on top of a leaf. The key ingredient is to model the complexity of the task in a layered representation of free-occupied occupancy grid maps. This allows to naturally encode the requirements of the task, to maintain and update the belief state with the measurements performed, to simulate and compute the expected gains of all potential viewpoints, and to encode the termination condition. During this work the technology of ToF cameras has incredibly evolved. Nowadays it is very popular and ToF cameras are already embedded in some consumer devices. Although the quality of the measurements has been considerably improved, it is still not uniform in the sensor. We believe, as it has been demonstrated in various experiments in this work, that a careful modeling of the sensor's uncertainty is highly beneficial and helps to design better decision systems. In our case, it enables a more realistic computation of the information gain measure, and consequently, a better selection criterion., [CA]: Aquesta tesi aborda el tema de la percepció activa i el seu ús en tasques d'exploració en entorns reals tot considerant la ambigüitat en l'escena i la incertesa del sistema de percepció. Al contrari de la majoria d'algoritmes de percepció activa, on el modelatge d'objectes sol ser l'objectiu implícit, en aquesta tesi hem explorat noves estratègies per poder tractar tasques genèriques i de major complexitat. Tot sistema de percepció activa requereix un aparell sensorial amb la capacitat de variar els seus paràmetres de forma controlada, per poder, d'aquesta manera, recopilar nova informació per resoldre una tasca determinada. En tasques d'exploració, la posició i orientació del sensor són paràmetres claus per resoldre la tasca. En el nostre estudi hem fet ús d'un robot manipulador com a sistema de posicionament i d'una càmera de profunditat de temps de vol (ToF), adherida al seu efector final, com a sistema de percepció. Com a tasca final, ens hem concentrat en l'adquisició de mesures sobre fulles dins de l'àmbit del fenotipatge de les plantes. Les plantes son objectes molt complexos, amb fulles que canvien de textura, posició i mida al llarg del temps. Això comporta diverses dificultats. Per una banda, abans de dur a terme una mesura sobre un fulla s'ha d'explorar l'entorn i trobar una regió que ho permeti. A més a més, aquells punts de vista que han estat adequats per una determinada planta difícilment ho seran per una altra, tot i sent les dues de la mateixa espècie. Per un altra banda, en el moment de la mesura, certs instruments, tals com els mesuradors de clorofil·la o les eines d'extracció de mostres, requereixen ser posicionats amb molta precisió. És necessari, doncs, disposar d'un model detallat d'aquestes regions d'interès, i que inclogui no només l'espai ocupat sinó també el lliure. Gràcies a la modelització de l'espai lliure es pot dur a terme una bona evitació d'obstacles i un bon càlcul de la trajectòria d'aproximació de l'eina a la fulla. En aquest context, és fàcil veure que, en general, amb un sol punt de vista, no n'hi haprou per adquirir tota la informació necessària per prendre una mesura, i que l'ús de trajectòries predeterminades no garanteixen l'èxit. L'objectiu general d'aquesta tesi és resoldre tasques complexes de percepció activa mitjançant la codificació del seu objectiu d'exploració en un model geomètric prèviament estimat, fent servir el guany d'informació com a guia fonamental dins de la funció de cost. Les principals contribucions d'aquesta tesi es poden dividir en dos grups: primer, l'avaluació de les càmeres ToF i el seu calibratge per poder avaluar la incertesa de les seves mesures (presentat en la Part I); i en segon lloc, la proposta d'un sistema capaç de codificar la tasca mitjançant el modelatge de l'espai lliure i ocupat, i que té en compte la incertesa del sensor per millorar la selecció de les accions (presentat en la Part II). Aquesta tesi ha donat lloc a 14 publicacions, incloent 5 en revistes indexades, i els resultats obtinguts s'han fet servir en el projecte Europeu GARNICS. La funcionalitat del sistema complet està basada en els mètodes Next-Best-View (següent-millor-vista) i es pot desglossar en els següents passos principals. En primer lloc, s'obté una vista inicial de l'objecte (p. ex., una planta). A partir d'aquesta vista inicial i d'un conjunt de vistes candidates, s'estima, per cada una d'elles, el guany d'informació resultant, tant de moure la càmera com d'obtenir una nova mesura. És rellevant dir que aquest càlcul té en compte la incertesa de cada un dels píxels del sensor, l'estimació de la informació basada en el model de la tasca preestablerta i les possibles oclusions. Un cop seleccionada la vista més prometedora, el robot es mou a la nova posició, pren una nova imatge, integra aquesta informació en el model i torna a avaluar, un altre cop, el conjunt de punts de vista restants. Per últim, la tasca acaba en el moment que es recopila suficient informació., This work has been partially supported by a JAE fellowship of the Spanish Scientific Research Council (CSIC), the Spanish Ministry of Science and Innovation, the Catalan Research Commission and the European Commission under the research projects: DPI2008-06022: PAU: Percepción y acción ante incertidumbre. DPI2011-27510: PAU+: Perception and Action in Robotics Problems with Large State Spaces. 201350E102: MANIPlus: Manipulación robotizada de objetos deformables. 2009-SGR-155: SGR ROBÒTICA: Grup de recerca consolidat - Grup de Robòtica. FP6-2004-IST-4-27657: EU PACO PLUS project. FP7-ICT-2009-4-247947: GARNICS: Gardening with a cognitive system. FP7-ICT-2009-6-269959: IntellAct: Intelligent observation and execution of Actions and manipulations., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Understanding human-centric images: from geometry to fashion

  • Simo-Serra, Edgar
Universitat Politècnica de Catalunya. Programa de Doctorat: Automàtica, Robòtica I Visió., [EN]: Understanding humans from photographs has always been a fundamental goal of computer vision. Early works focused on simple tasks such as detecting the location of individuals by means of bounding boxes. As the field progressed, harder and more higher level tasks have been undertaken. For example, from human detection came the 2D and 3D human pose estimation in which the task consisted of identifying the location in the image or space of all different body parts, e.g., head, torso, knees, arms, etc. Human attributes also became a great source of interest as they allow recognizing individuals and other properties such as gender or age. Later, the attention turned to the recognition of the action being performed. This, in general, relies on the previous works on pose estimation and attribute classification. Currently, even higher level tasks are being conducted such as predicting the motivations of human behavior or identifying the fashionability of an individual from a photograph. In this thesis we have developed a hierarchy of tools that cover all these range of problems, from low level feature point descriptors to high level fashion-aware conditional random fields models, all with the objective of understanding humans from monocular, RGB images. In order to build these high level models it is paramount to have a battery of robust and reliable low and mid level cues. Along these lines, we have proposed two low-level keypoint descriptors: one based on the theory of the heat diffusion on images, and the other that uses a convolutional neural network to learn discriminative image patch representations. We also introduce distinct low-level generative models for representing human pose: in particular we present a discrete model based on a directed acyclic graph and a continuous model that consists of poses clustered on a Riemannian manifold. As mid level cues we propose two 3D human pose estimation algorithms: one that estimates the 3D pose given a noisy 2D estimation, and an approach that simultaneously estimates both the 2D and 3D pose. Finally, we formulate higher level models built upon low and mid level cues for human understanding. Concretely, we focus on two different tasks in the context of fashion: semantic segmentation of clothing, and predicting the fashionability from images with metadata to ultimately provide fashion advice to the user. In summary, to robustly extract knowledge from images with the presence of humans it is necessary to build high level models that integrate low and mid level cues. In general, using and understanding strong features is critical for obtaining reliable performance. The main contribution of this thesis is in proposing a variety of low, mid and high level algorithms for human-centric images that can be integrated into higher level models for comprehending humans from photographs, as well as tackling novel fashion-oriented problems., [ES]: Siempre ha sido una meta fundamental de la visión por computador la comprensión de los seres humanos. Los primeros trabajos se fijaron en objetivos sencillos tales como la detección en imágenes de la posición de los individuos. A medida que la investigación progresó se emprendieron tareas mucho más complejas. Por ejemplo, a partir de la detección de los humanos se pasó a la estimación en dos y tres dimensiones de su postura por lo que la tarea consistía en identificar la localización en la imagen o el espacio de las diferentes partes del cuerpo, por ejemplo cabeza, torso, rodillas, brazos, etc...También los atributos humanos se convirtieron en una gran fuente de interés ya que permiten el reconocimiento de los individuos y de sus propiedades como el género o la edad. Más tarde, la atención se centró en el reconocimiento de la acción realizada. Todos estos trabajos reposan en las investigaciones previas sobre la estimación de las posturas y la clasificación de los atributos. En la actualidad, se llevan a cabo investigaciones de un nivel aún superior sobre cuestiones tales como la predicción de las motivaciones del comportamiento humano o la identificación del tallaje de un individuo a partir de una fotografía. En esta tesis desarrollamos una jerarquía de herramientas que cubre toda esta gama de problemas, desde descriptores de rasgos de bajo nivel a modelos probabilísticos de campos condicionales de alto nivel reconocedores de la moda, todos ellos con el objetivo de mejorar la comprensión de los humanos a partir de imágenes RGB monoculares. Para construir estos modelos de alto nivel es decisivo disponer de una batería de datos robustos y fiables de nivel bajo y medio. En este sentido, proponemos dos descriptores novedosos de bajo nivel: uno se basa en la teoría de la difusión de calor en las imágenes y otro utiliza una red neural convolucional para aprender representaciones discriminativas de trozos de imagen. También introducimos diferentes modelos de bajo nivel generativos para representar la postura humana: en particular presentamos un modelo discreto basado en un gráfico acíclico dirigido y un modelo continuo que consiste en agrupaciones de posturas en una variedad de Riemann. Como señales de nivel medio proponemos dos algoritmos estimadores de la postura humana: uno que estima la postura en tres dimensiones a partir de una estimación imprecisa en el plano de la imagen y otro que estima simultáneamente la postura en dos y tres dimensiones. Finalmente construimos modelos de alto nivel a partir de señales de nivel bajo y medio para la comprensión de la persona a partir de imágenes. En concreto, nos centramos en dos diferentes tareas en el ámbito de la moda: la segmentación semántica del vestido y la predicción del buen ajuste de la prenda a partir de imágenes con meta-datos con la finalidad de aconsejar al usuario sobre moda. En resumen, para extraer conocimiento a partir de imágenes con presencia de seres humanos es preciso construir modelos de alto nivel que integren señales de nivel medio y bajo. En general, el punto crítico para obtener resultados fiables es el empleo y la comprensión de rasgos fuertes. La aportación fundamental de esta tesis es la propuesta de una variedad de algoritmos de nivel bajo, medio y alto para el tratamiento de imágenes centradas en seres humanos que pueden integrarse en modelos de alto nivel, para mejor comprensión de los seres humanos a partir de fotografías, así como abordar problemas planteados por el buen ajuste de las prendas., This work has been partly sponsored by the following: - A four-year PhD scholarship (BES-2012-054905). - A four-month scholarship to visit the Toyota Technological Institute at Chicago (EEBB-I-14-08177). - A four-month scholarship to visit the Nakamura Laboratory in Japan (EEBB-I-15-09283). - Projects CINNOVA (201150E088), PAU+ (DPI2011-27510), ViSEN (ERA-Net Chistera PCIN-2013-047), and Intellact (FP7-ICT2009-6-269959)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Teaching a robot the semantics of assembly tasks

  • Savarimuthu, Thiusius R.
  • Buch, Anders G.
  • Schlette, Christian
  • Wantia, Nils
  • Roßmann, Jürgen
  • Martínez, David
  • Alenyà, Guillem
  • Torras, Carme
  • Ude, Ales
  • Nemec, Bojan
  • Kramberger, Aljaž
  • Wörgötter, Florentin
  • Aksoy, Eren Erdal
  • Papon, Jeremie
  • Haller, Simon
  • Piater, Justus
  • Krüger, Norbert
We present a three-level cognitive system in a learning by demonstration context. The system allows for learning and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level representation based on so-called 'semantic event chains.' We describe details of the representations and quantify the effect of the associated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the overall system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6., This work was supported in part by the EU Project IntellAct under Grant FP7-ICT-269959, and in part by the H2020 Project ReconCell under Grant H2020-FoF-680431., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Learning relational models with human interaction for planning in robotics

  • Martínez, David
A thesis submitted to the Universitat Politècnica de Catalunya to obtain the degree of Doctor of Philosophy. Doctoral Programme: Automatic Control, Robotics and Computer Vision. This thesis was completed at: Institut de Robòtica i Informàtica Industrial, CSIC-UPC., [EN]: Automated planning has proven to be useful to solve problems where an agent has to maximize a reward function by executing actions. As planners have been improved to salve more expressive and difficult problems, there is an increasing interest in using planning to improve efficiency in robotic tasks. However, planners rely on a domain model, which has to be either handcrafted or learned. Although learning domain models can be very costly, recent approaches provide generalization capabilities and integrate human feedback to reduce the amount of experiences required to learn. In this thesis we propase new methods that allow an agent with no previous knowledge to solve certain problems more efficiently by using task planning. First, we show how to apply probabilistic planning to improve robot performance in manipulation tasks (such as cleaning the dirt or clearing the tableware on a table). Planners obtain sequences of actions that get the best result in the long term, beating reactive strategies. Second, we introduce new reinforcement learning algorithms where the agent can actively request demonstrations from a teacher to learn new actions and speed up the learning process. In particular, we propase an algorithm that allows the user to set the mínimum quality to be achieved, where a better quality also implies that a larger number of demonstrations will be requested . Moreover, the learned model is analyzed to extract the unlearned or problematic parts of the model. This information allow the agent to provide guidance to the teacher when a demonstration is requested, and to avoid irrecoverable errors. Finally, a new domain model learner is introduced that, in addition to relational probabilistic action models, can also learn exogenous effects. This learner can be integrated with existing planners and reinforcement learning algorithms to salve a wide range of problems. In summary, we improve the use of learning and task planning to salve unknown tasks. The improvements allow an agent to obtain a larger benefit from planners, learn faster, balance the number of action executions and teacher demonstrations, avoid irrecoverable errors, interact with a teacher to solve difficult problems, and adapt to the behavior of other agents by learning their dynamics. All the proposed methods were compared with state-of-the-art approaches, and were also demonstrated in different scenarios, including challenging robotic tasks., [ES]: La planificación automática ha probado ser de gran utilidad para resolver problemas en los que un agente tiene que ejecutar acciones para maximizar una función de recompensa. A medida que los planificadores han sido capaces de resolver problemas cada vez más complejos, ha habido un creciente interés por utilizar dichos planificadores para mejorar la eficiencia de tareas robóticas. Sin embargo, los planificadores requieren un modelo del dominio, el cual puede ser creado a mano o aprendido. Aunque aprender modelos automáticamente puede ser costoso, recientemente han aparecido métodos que permiten la interacción persona-máquina y generalizan el conocimiento para reducir la cantidad de experiencias requeridas para aprender. En esta tesis proponemos nuevos métodos que permiten a un agente sin conocimiento previo de la tarea resolver problemas de forma más eficiente mediante el uso de planificación automática. Comenzaremos mostrando cómo aplicar planificación probabilística para mejorar la eficiencia de robots en tareas de manipulación (como limpiar suciedad o recoger una mesa). Los planificadores son capaces de obtener las secuencias de acciones que producen los mejores resultados a largo plazo, superando a las estrategias reactivas. Por otro lado, presentamos nuevos algoritmos de aprendizaje por refuerzo en los que el agente puede solicitar demostraciones a un profesor. Dichas demostraciones permiten al agente acelerar el aprendizaje o aprender nuevas acciones. En particular, proponemos un algoritmo que permite al usuario establecer la mínima suma de recompensas que es aceptable obtener, donde una recompensa más alta implica que se requerirán más demostraciones. Además, el modelo aprendido será analizado para identificar qué partes están incompletas o son problemáticas. Esta información permitirá al agente evitar errores irrecuperables y también guiar al profesor cuando se solicite una demostración. Finalmente, se ha introducido un nuevo método de aprendizaje para modelos de dominios que, además de obtener modelos relacionales de acciones probabilísticas, también puede aprender efectos exógenos. Mostraremos cómo integrar este método en algoritmos de aprendizaje por refuerzo para poder abordar una mayor cantidad de problemas. En resumen, hemos mejorado el uso de técnicas de aprendizaje y planificación para resolver tareas desconocidas a priori. Estas mejoras permiten a un agente aprovechar mejor los planificadores, aprender más rápido, elegir entre reducir el número de acciones ejecutadas o el número de demostraciones solicitadas, evitar errores irrecuperables, interactuar con un profesor para resolver problemas complejos, y adaptarse al comportamiento de otros agentes aprendiendo sus dinámicas. Todos los métodos propuestos han sido comparados con trabajos del estado del arte, y han sido evaluados en distintos escenarios, incluyendo tareas robóticas., This work has been partially supported by the following: FPU12-04173 The Spanish Ministry of Education, Culture, and Sport via a FPU doctoral grant. FP7-ICT2009-6-269959 The EU Project IntellAct. 201350E102 The CSIC project MANIPlus. TIN2014-58178-R The MINECO project RobInstruct., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Tesis doctoral (DoctoralThesis)

Bimanual robot skills: MP encoding, dimensionality reduction and reinforcement learning

  • Colomé, Adrià
A thesis submitted to the Universitat Politècnica de Catalunya to obtain the degree of Doctor of Philosophy. Doctoral programme: Automatic Control, Robotics and Computer Vision. This thesis was completed at: Institut de Robòtica i Informàtica Industrial, CSIC-UPC., [EN]: In our culture, robots have been in novels and cinema for a long time, but it has been specially in the last two decades when the improvements in hardware - better computational power and components - and advances in Artificial Intelligence (AI), have allowed robots to start sharing spaces with humans. Such situations require, aside from ethical considerations, robots to be able to move with both compliance and precision, and learn at different levels, such as perception, planning, and motion, being the latter the focus of this work. The first issue addressed in this thesis is inverse kinematics for redundant robot manipulators, i.e: positioning the robot joints so as to reach a certain end-effector pose. We opt for iterative solutions based on the inversion of the kinematic Jacobian of a robot, and propose to filter and limit the gains in the spectral domain, while also unifying such approach with a continuous, multipriority scheme. Such inverse kinematics method is then used to derive manipulability in the whole workspace of an antropomorphic arm, and the coordination of two arms is subsequently optimized by finding their best relative positioning. Having solved the kinematic issues, a robot learning within a human environment needs to move compliantly, with limited amount of force, in order not to harm any humans or cause any damage, while being as precise as possible. Therefore, we developed two dynamic models for the same redundant arm we had analysed kinematically: The first based on local models with Gaussian projections, and the second characterizing the most problematic term of the dynamics, namely friction. Such models allowed us to implement feed-forward controllers, where we can actively change the weights in the compliance-precision tradeoff. Moreover, we used such models to predict external forces acting on the robot, without the use of force sensors. Afterwards, we noticed that bimanual robots must coordinate their components (or limbs) and be able to adapt to new situations with ease. Over the last decade, a number of successful applications for learning robot motion tasks have been published. However, due to the complexity of a complete system including all the required elements, most of these applications involve only simple robots with a large number of high-end technology sensors, or consist of very simple and controlled tasks. Using our previous framework for kinematics and control, we relied on two types of movement primitives to encapsulate robot motion. Such movement primitives are very suitable for using reinforcement learning. In particular, we used direct policy search, which uses the motion parametrization as the policy itself. In order to improve the learning speed in real robot applications, we generalized a policy search algorithm to give some importance to samples yielding a bad result, and we paid special attention to the dimensionality of the motion parametrization. We reduced such dimensionality with linear methods, using the rewards obtained through motion repetition and execution. We tested such framework in a bimanual task performed by two antropomorphic arms, such as the folding of garments, showing how a reduced dimensionality can provide qualitative information about robot couplings and help to speed up the learning of tasks when robot motion executions are costly., [CA]: A la nostra cultura, els robots han estat presents en novel·les i cinema des de fa dècades, però ha sigut especialment en les últimes dues quan les millores en hardware (millors capacitats de còmput) i els avenços en intel·ligència artificial han permès que els robots comencin a compartir espais amb els humans. Aquestes situacions requereixen, a banda de consideracions ètiques, que els robots siguin capaços de moure's tant amb suavitat com amb precisió, i d'aprendre a diferents nivells, com són la percepció, planificació i moviment, essent l'última el centre d'atenció d'aquest treball. El primer problema adreçat en aquesta tesi és la cinemàtica inversa, i.e.: posicionar les articulacions del robot de manera que l'efector final estigui en una certa posició i orientació. Hem estudiat el camp de les solucions iteratives, basades en la inversió del Jacobià cinemàtic d'un robot, i proposem un filtre que limita els guanys en el seu domini espectral, mentre també unifiquem tal mètode dins un esquema multi-prioritat i continu. Aquest mètode per a la cinemàtica inversa és usat a l'hora d'encapsular tota la informació sobre l'espai de treball d'un braç antropomòrfic, i les capacitats de coordinació entre dos braços són optimitzades, tot trobant la seva millor posició relativa en l'espai. Havent resolt les dificultats cinemàtiques, un robot que aprèn en un entorn humà necessita moure's amb suavitat exercint unes forces limitades per tal de no causar danys, mentre es mou amb la màxima precisió possible. Per tant, hem desenvolupat dos models dinàmics per al mateix braç robòtic redundant que havíem analitzat des del punt de vista cinemàtic: El primer basat en models locals amb projeccions de Gaussianes i el segon, caracteritzant el terme més problemàtic i difícil de representar de la dinàmica, la fricció. Aquests models ens van permetre utilitzar controladors coneguts com "feed-forward", on podem canviar activament els guanys buscant l'equilibri precisió-suavitat que més convingui. A més, hem usat aquests models per a inferir les forces externes actuant en el robot, sense la necessitat de sensors de força. Més endavant, ens hem adonat que els robots bimanuals han de coordinar els seus components (braços) i ser capaços d'adaptar-se a noves situacions amb facilitat. Al llarg de l'última dècada, diverses aplicacions per aprendre tasques motores robòtiques amb èxit han estat publicades. No obstant, degut a la complexitat d'un sistema complet que inclogui tots els elements necessaris, la majoria d'aquestes aplicacions consisteixen en robots més aviat simples amb costosos sensors d'última generació, o a resoldre tasques senzilles en un entorn molt controlat. Utilitzant el nostre treball en cinemàtica i control, ens hem basat en dos tipus de primitives de moviment per caracteritzar la motricitat robòtica. Aquestes primitives de moviment són molt adequades per usar aprenentatge per reforç. En particular, hem usat la búsqueda directa de la política, un camp de l'aprenentatge per reforç que usa la parametrització del moviment com la pròpia política. Per tal de millorar la velocitat d'aprenentatge en aplicacions amb robots reals, hem generalitzat un algoritme de búsqueda directa de política per a donar importància a les mostres amb mal resultat, i hem donat especial atenció a la reducció de dimensionalitat en la parametrització dels moviments. Hem reduït la dimensionalitat amb mètodes lineals, utilitzant les recompenses obtingudes EN executar els moviments. Aquests mètodes han estat provats en tasques bimanuals com són plegar roba, usant dos braços antropomòrfics. Els resultats mostren com la reducció de dimensionalitat pot aportar informació qualitativa d'una tasca, i al mateix temps ajuda a aprendre-la més ràpid quan les execucions amb robots reals són costoses., This work has been supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant AP2010-1989. This thesis has also been partially supported by the research projects: FP7-ICT-2009-6-269959 IntellAct: Intelligent observation and execution of Actions and manipulations (European Project). 201350E102 MANIPlus: Manipulación robotizada de objetos deformables (CSIC Project). 2009 SGR 155 : SGR ROBÒTICA: Grup de recerca consolidat - Grup de Robòtica (Catalan government). TIN2014-58178-R RobInstruct: Instructing robots using natural communication skills (Spanish national project). PCIN-2015-147 I-DRESS: Assistive interactive robotic system for support in dressing (European Project)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Characterization of textile grasping experiments

  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Ramisa Ayats, Arnau
  • Moreno-Noguer, Francesc
  • Torras, Carme|||0000-0002-2933-398X
Grasping highly deformable objects, like textiles, is an emerging area of research that involves both percep- tion and manipulation abilities. As new techniques appear, it becomes essential to design strategies to compare them. However, this is not an easy task, since the large state-space of textile objects explodes when coupled with the variability of grippers, robotic hands and robot arms performing the manipulation task. This high variability makes it very difficult to design experiments to evaluate the performance of a system in a repeatable way and compare it to others. We propose a framework to allow the comparison of different grasping methods for textile objects. Instead of measuring each component separately, we there- fore propose a methodology to explicitly measure the vision- manipulation correlation by taking into account the throughput of the actions. Perceptions of deformable objects should be grouped into different clusters, and the different grasping actions available should be tested for each perception type to obtain the action-perception success ratio. This characterization potentially allows to compare very different systems in terms of specialized actions , perceptions or widely useful actions , along with the cost of performing each action. We will also show that this categorization is useful in manipulation planning of deformable objects., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Real-time segmentation of stereo videos on a portable system with a mobile GPU

  • Abramov, Alexey
  • Pauwels, Karl
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
In mobile robotic applications, visual information needs to be processed fast despite resource limitations of the mobile system. Here a novel real-time framework for model- free spatio-temporal segmentation of stereo videos is presented. It combines real-time optical flow and stereo with image seg- mentation and runs on a portable system with an integrated mobile GPU. The system performs on-line, automatic and dense segmentation of stereo videos and serves as a visual front- end for preprocessing in mobile robots, providing a condensed representation of the scene which can potentially be utilized in various applications, e.g., object manipulation, manipulation recognition, visual servoing. The method was tested on real-world sequences with arbitrary motions including videos acquired with a moving camera., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Robust surface tracking in range image sequences

  • Husain, Syed Farzad
  • Dellen, Babette
  • Torras, Carme|||0000-0002-2933-398X
A novel robust method for surface tracking in range-image sequences is presented which combines a clustering method based on surface models with a particle-filter-based 2-D affine-motion estimator. Segmented regions obtained at previous time steps are used to create seed areas by comparing measured depth values with those obtained from surface-model fitting. The seed areas are further refined using a motion-probability region estimated by the particle-filter-based tracker through prediction of future states. This helps resolving ambiguities that arise when surfaces belonging to different objects are in physical contact with each other, for example during hand-object manipulations. Region growing allows recovering the complete segment area. The obtained segmented regions are then used to improve the predictions of the tracker for the next frame. The algorithm runs in quasi real-time and uses on-line learning, eliminating the need to have a priori knowledge about the surface being tracked. We apply the method to in-house depth videos acquired with both time-of-flight and structured-light sensors, demonstrating object tracking in real-world scenarios, and we compare the results with those of an ICP-based tracker. (C) 2014 Elsevier Inc. All rights reserved., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Redundant inverse kinematics: experimental comparative review and two enhancements

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Torras, Carme|||0000-0002-2933-398X
Motivated by the need of a robust and practical Inverse Kinematics (IK) algorithm for the WAM robot arm, we reviewed the most used closed-loop methods for redundant robots, analysing their main points of concern: convergence, numerical error, singularity handling, joint limit avoidance, and the capability of reaching secondary goals. As a result of the experimental comparison, we propose two enhancements. The first is to filter the singular values of the Jacobian matrix before calculating its pseudoinverse in order to obtain a more numerically robust result. The second is to combine a continuous task priority strategy with selective damping to generate smoother trajectories. Experimentation on the WAM robot arm shows that these two enhancements yield an IK algorithm that improves on the reviewed state-of-the-art ones, in terms of the good compromise it achieves between time step length, Jacobian conditioning, multiple task performance, and computational time, thus constituting a very solid option in practice. This proposal is general and applicable to other redundant robots., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

POMDP approach to robotized clothes separation

  • Monsó Purtí, Pol
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
Rigid object manipulation with robots has mainly relied on precise, expensive models and deterministic sequences. Given the great complexity of accurately modeling deformable objects, their manipulation seems to call for a rather different approach. This paper proposes a probabilistic planner, based on a Partially Observable Markov Decision Process (POMDP), targeted at reducing the inherent uncertainty of deformable object sorting. It is shown that a small set of unreliable actions and inaccurate perceptions suffices to accomplish the task, provided faithful statistics on both of them are collected beforehand. The planner has been applied to a clothes sorting task in a real case context with a depth and color sensor and a robotic arm. Experimental results show the promise of the approach since more than 95% certainty of having isolated a piece of clothing is reached in an average of four steps for quite entangled initial clothing configurations., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Survey on assembly sequencing : a combinatorial and geometrical perspective

  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
A systematic overview on the subject of assembly sequencing is presented. Sequencing lies at the core of assembly planning, and variants include finding a feasible sequence—respecting the precedence constraints between the assembly operations—, or determining an optimal one according to one or several operational criteria. The different ways of representing the space of feasible assembly sequences are described, as well as the search and optimization algorithms that can be used. Geometry plays a fundamental role in devising the precedence constraints between assembly operations, and this is the subject of the second part of the survey, which treats also motion in contact in the context of the actual performance of assembly operations., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Using depth and appearance features for informed robot grasping of highly wrinkled clothes

  • Ramisa Ayats, Arnau
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Moreno-Noguer, Francesc
  • Torras, Carme|||0000-0002-2933-398X
Detecting grasping points is a key problem in cloth manipulation. Most current approaches follow a multiple regrasp strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields to a desired configuration. In this paper, by contrast, we circumvent the need for multiple re-graspings by building a robust detector that identifies the grasping points, generally in one single step, even when clothes are highly wrinkled. In order to handle the large variability a deformed cloth may have, we build a Bag of Features based detector that combines appearance and 3D geometry features. An image is scanned using a sliding window with a linear classifier, and the candidate windows are refined using a non-linear SVM and a “grasp goodness” criterion to select the best grasping point. We demonstrate our approach detecting collars in deformed polo shirts, using a Kinect camera. Experimental results show a good performance of the proposed method not only in identifying the same trained textile object part under severe deformations and occlusions, but also the corresponding part in other clothes, exhibiting a degree of generalization.
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Planning robot manipulation to clean planar surfaces

  • Martínez Martínez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
This paper presents a new approach to plan high-level manipulation actions for cleaning surfaces in household environments, like removing dirt from a table using a rag. Dragging actions can change the distribution of dirt in an unpredictable manner, and thus the planning becomes challenging. We propose to define the problem using explicitly uncertain actions, and then plan the most effective sequence of actions in terms of time. However, some issues have to be tackled to plan efficiently with stochastic actions. States become hard to predict after executing a few actions, so replanning every few actions with newer perceptions gives the best results, and the trade-off between planning time and plan quality is also important. Finally a learner is integrated to provide adaptation to changes, such as different rag grasps, robots, or cleaning surfaces. We demonstrate experimentally, using two different robot platforms, that planning is more advantageous than simple reactive strategies for accomplishing complex tasks, while still providing a similar performance for easy tasks. We also performed experiments where the rag grasp was changed, and thus the behaviour of the dragging actions, showing that the learning capabilities allow the robot to double its performance with a new rag grasp after a few cleaning iterations., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Relational reinforcement learning with guided demonstrations

  • Martínez Martínez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
© <year>. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0, Model-based reinforcement learning is a powerful paradigm for learning tasks in robotics. However, in-depth exploration is usually required and the actions have to be known in advance. Thus, we propose a novel algorithm that integrates the option of requesting teacher demonstrations to learn new domains with fewer action executions and no previous knowledge. Demonstrations allow new actions to be learned and they greatly reduce the amount of exploration required, but they are only requested when they are expected to yield a significant improvement because the teacher's time is considered to be more valuable than the robot's time. Moreover, selecting the appropriate action to demonstrate is not an easy task, and thus some guidance is provided to the teacher. The rule-based model is analyzed to determine the parts of the state that may be incomplete, and to provide the teacher with a set of possible problems for which a demonstration is needed. Rule analysis is also used to find better alternative models and to complete subgoals before requesting help, thereby minimizing the number of requested demonstrations. These improvements were demonstrated in a set of experiments, which included domains from the international planning competition and a robotic task. Adding teacher demonstrations and rule analysis reduced the amount of exploration required by up to 60% in some domains, and improved the success ratio by 35% in other domains, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Tesis doctoral (DoctoralThesis)

Perceiving dynamic environments : from surface geometry to semantic representation

  • Husain, Syed Farzad
Perceiving human environments is becoming increasingly fundamental with the gradual adaptation of robots for domestic use. High-level tasks such as the recognition of objects and actions need to be performed far the active engagement of the robot with its surroundings. Nowadays, the environment is primarily captured using visual information in the form of color and depth images. Visual cues obtained from these images serve as a base upan which perception-related applications are developed. Far example, using appearance models far detecting objects and extracting motion infarmation far recognizing actions. However, given the complex variations of naturally occurring scenes, extracting a set of robust visual cues becomes harder here than in other contexts. In this thesis, we develop a hierarchy of tools to improve the different aspects of robot perception in human-centered, possibly dynamic, environments. We start with the segmentation of single images and extend it to videos. Afterwards, we develop a surface tracking approach along with the incorporation of our video segmentation method. We then investigate the higher-level tasks of semantic segmentation and recognition. Finally, we focus on recognizing actions in videos. The introduction of Kinectstyle depth sensors is relatively new and its usage in the field of robotics cannot be found befare half a decade ago. Such sensors enable the acquisition of high-resolution color and depth images at a low cost. Given this opportunity, we dedícate a bulk of our work to the exploitation of the depth infarmation obtained using such sensors, thereby pushing forward the state-of-the-art in perception problems. The thesis is conceptually grouped into two parts. In the first part, we address the low-level tasks of segmentation and tracking with depth images. In many cases, depth data gives a better disambiguation of surface boundaries of different objects in a scene when compared to their color counterpart. We exploit this information in a novel depth segmentation scheme that fits quadratic surface models on different surfaces in a competing fashion . We further extend the method to the video domain by initializing the segmentation results and surface model parameters from the previous trame for the next trame. In this way, we successfully create a video segmentation algorithm, in which the segment label belonging to each surface becomes coherent over time. We also devise a particle-filter-based tracker that uses depth data to track a surface. The tracker is made more robust by combining it with our video segmentation approach. The segmentation results serve as a useful prior for high-level tasks. In the second part we deal with such tasks which include (i) object recognition, (ii) pixelwise object class segmentation, and (iii) action recognition . We propase (i) to address object recognition by creating context-aware conditional random field models. We show the importance of the context in object recognition by modeling geometrical relations between different objects in a scene. We perform (ii) object class segmentation using a convolutional neural network. We introduce a novel distance-from-wall feature and demonstrate its effectiveness in generating better class proposals for objects that are clase to the walls. The final part of the thesis deals with (iii) action recognition. We propase a 2D convolutional neural network extended to a concatenated 3D network that learns to extrae! features from the spatio-temporal domain of raw video data. The network is trained to predict an action label for each video. In summary, several perception aspects are addressed with the utilization of depth infarmation where available. Our main contributions are (a) the introduction of a depth video segmentation scheme, (b) a graphical model far object recognition, and our proposals of the deep learning models for (e) object class segmentation and (d) action recognition., Los sistemas de percepción en entornos humanos son cada vez más importantes para la adaptación gradual de los robots a tareas domésticas. Tareas de alto nivel, tales como el reconocimiento de objetos y acciones, son necesarias para conseguir la participación activa del robot en dichas tareas. Hoy en día el entorno del robot es capturado principalmente usando información visual en forma de imágenes de color y profundidad. Las características visuales obtenidas a partir de estas imágenes sirven como base para el desarrollo de aplicaciones relacionadas con la percepción del robot. Por ejemplo, el uso de modelos de apariencia para la detección de objetos y la extracción de información del movimiento para el reconocimiento de acciones. Sin embargo, dado que las escenas pueden contener variaciones complejas, la extracción de un conjunto de características visuales puede convertirse en una tarea muy difícil. En la presente tesis hemos desarrollado una jerarquía de herramientas para mejorar diferentes aspectos de la percepción del robot en entornos humanos, posiblemente dinámicos. Esta tesis comienza con la segmentación de imágenes individuales, que luego se extiende a vídeos. Posteriormente, diseñamos un enfoque de seguimiento de superficies que incorpora nuestro método de segmentación de vídeos. A continuación, investigamos tareas de alto nivel para la segmentación semántica y el reconocimiento. Finalmente, nos centramos en el reconocimiento de acciones en vídeos. La introducción de sensores de profundidad tipo Kinect es relativamente nueva y su uso en el campo de la robótica empezó hace tan solo media década. Tales sensores permiten la adquisición de color y profundidad de imágenes de alta resolución a bajo coste. Dada esta oportunidad, dedicamos una buena parte de nuestro trabajo a la explotación de la información de profundidad obtenida a través de dichos sensores, mejorando el estado del arte en problemas de percepción. La tesis está conceptualmente dividida en dos partes. En primer lugar, abordamos las tareas de bajo nivel de segmentación y seguimiento con imágenes de profundidad. En muchos casos, los datos de profundidad permite una mejor desambiguación de los límites de las superficies de diferentes objetos de una escena en comparación con los datos de color. Explotamos esta información en un nuevo esquema de segmentación de profundidad que ajusta modelos cuadráticos de superficies de forma competitiva. Extendemos el método a vídeos de modo que la etiquetación de superficies resulte coherente en el tiempo. También proponemos un rastreador basado en un filtro de partículas que utiliza los datos de profundidad para realizar el seguimiento de una superficie. El seguimiento se hace más robusto al combinarlo con nuestro enfoque de segmentación en vídeo. Los resultados de la segmentación son usados como información a priori para tareas de alto nivel. En la segunda parte nos ocupamos de este tipo de tareas que incluyen el (i) reconocimiento de objetos, (ii) la segmentación de clases de objetos a nivel de píxeles, y (iii) el reconocimiento de acciones. Proponemos (i) abordar el reconocimiento de objetos mediante la creación de modelos de campos aleatorios condicionales sensibles al contexto. Realizamos (ii) la segmentación de la clase del objeto utilizando una red neuronal de convolución. Se introduce una nueva característica de distancia-a-paredes y demostramos su eficacia en la mejora de la clasificación de objetos que están cerca de las paredes. La parte final de la tesis se ocupa del (iii) reconocimiento de acciones. Proponemos una red neuronal de convolución 2D extendida a una red 3D concatenada, que aprende a extraer las características del dominio espacio-temporal de los datos de vídeo. La red está capacitada para predecir la etiqueta de acción para cada vídeo.
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A robot learning from demonstration framework to perform force-based manipulation tasks

  • Rozo Castañeda, Leonel
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Torras, Carme|||0000-0002-2933-398X
This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human–robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher’s demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher’s one., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Tesis doctoral (DoctoralThesis)

Understanding human-centric images : from geometry to fashion

  • Simó Serra, Edgar
Premi "Marc Esteva Vivanco", a la millor tesi en intel·ligència artifical de l’Associació Catalana en Intel·ligència Artificial, Understanding humans from photographs has always been a fundamental goal of computer vision. Early works focused on simple tasks such as detecting the location of individuals by means of bounding boxes. As the field progressed, harder and more higher level tasks have been undertaken. For example, from human detection came the 2D and 3D human pose estimation in which the task consisted of identifying the location in the image or space of all different body parts, e.g., head, torso, knees, arms, etc. Human attributes also became a great source of interest as they allow recognizing individuals and other properties such as gender or age. Later, the attention turned to the recognition of the action being performed. This, in general, relies on the previous works on pose estimation and attribute classification. Currently, even higher level tasks are being conducted such as predicting the motivations of human behavior or identifying the fashionability of an individual from a photograph. In this thesis we have developed a hierarchy of tools that cover all these range of problems, from low level feature point descriptors to high level fashion-aware conditional random fields models, all with the objective of understanding humans from monocular, RGB images. In order to build these high level models it is paramount to have a battery of robust and reliable low and mid level cues. Along these lines, we have proposed two low-level keypoint descriptors: one based on the theory of the heat diffusion on images, and the other that uses a convolutional neural network to learn discriminative image patch representations. We also introduce distinct low-level generative models for representing human pose: in particular we present a discrete model based on a directed acyclic graph and a continuous model that consists of poses clustered on a Riemannian manifold. As mid level cues we propose two 3D human pose estimation algorithms: one that estimates the 3D pose given a noisy 2D estimation, and an approach that simultaneously estimates both the 2D and 3D pose. Finally, we formulate higher level models built upon low and mid level cues for human understanding. Concretely, we focus on two different tasks in the context of fashion: semantic segmentation of clothing, and predicting the fashionability from images with metadata to ultimately provide fashion advice to the user. In summary, to robustly extract knowledge from images with the presence of humans it is necessary to build high level models that integrate low and mid level cues. In general, using and understanding strong features is critical for obtaining reliable performance. The main contribution of this thesis is in proposing a variety of low, mid and high level algorithms for human-centric images that can be integrated into higher level models for comprehending humans from photographs, as well as tackling novel fashion-oriented problems., Siempre ha sido una meta fundamental de la visión por computador la comprensión de los seres humanos. Los primeros trabajos se fijaron en objetivos sencillos tales como la detección en imágenes de la posición de los individuos. A medida que la investigación progresó se emprendieron tareas mucho más complejas. Por ejemplo, a partir de la detección de los humanos se pasó a la estimación en dos y tres dimensiones de su postura por lo que la tarea consistía en identificar la localización en la imagen o el espacio de las diferentes partes del cuerpo, por ejemplo cabeza, torso, rodillas, brazos, etc...También los atributos humanos se convirtieron en una gran fuente de interés ya que permiten el reconocimiento de los individuos y de sus propiedades como el género o la edad. Más tarde, la atención se centró en el reconocimiento de la acción realizada. Todos estos trabajos reposan en las investigaciones previas sobre la estimación de las posturas y la clasificación de los atributos. En la actualidad, se llevan a cabo investigaciones de un nivel aún superior sobre cuestiones tales como la predicción de las motivaciones del comportamiento humano o la identificación del tallaje de un individuo a partir de una fotografía. En esta tesis desarrollamos una jerarquía de herramientas que cubre toda esta gama de problemas, desde descriptores de rasgos de bajo nivel a modelos probabilísticos de campos condicionales de alto nivel reconocedores de la moda, todos ellos con el objetivo de mejorar la comprensión de los humanos a partir de imágenes RGB monoculares. Para construir estos modelos de alto nivel es decisivo disponer de una batería de datos robustos y fiables de nivel bajo y medio. En este sentido, proponemos dos descriptores novedosos de bajo nivel: uno se basa en la teoría de la difusión de calor en las imágenes y otro utiliza una red neural convolucional para aprender representaciones discriminativas de trozos de imagen. También introducimos diferentes modelos de bajo nivel generativos para representar la postura humana: en particular presentamos un modelo discreto basado en un gráfico acíclico dirigido y un modelo continuo que consiste en agrupaciones de posturas en una variedad de Riemann. Como señales de nivel medio proponemos dos algoritmos estimadores de la postura humana: uno que estima la postura en tres dimensiones a partir de una estimación imprecisa en el plano de la imagen y otro que estima simultáneamente la postura en dos y tres dimensiones. Finalmente construimos modelos de alto nivel a partir de señales de nivel bajo y medio para la comprensión de la persona a partir de imágenes. En concreto, nos centramos en dos diferentes tareas en el ámbito de la moda: la segmentación semántica del vestido y la predicción del buen ajuste de la prenda a partir de imágenes con meta-datos con la finalidad de aconsejar al usuario sobre moda. En resumen, para extraer conocimiento a partir de imágenes con presencia de seres humanos es preciso construir modelos de alto nivel que integren señales de nivel medio y bajo. En general, el punto crítico para obtener resultados fiables es el empleo y la comprensión de rasgos fuertes. La aportación fundamental de esta tesis es la propuesta de una variedad de algoritmos de nivel bajo, medio y alto para el tratamiento de imágenes centradas en seres humanos que pueden integrarse en modelos de alto nivel, para mejor comprensión de los seres humanos a partir de fotografías, así como abordar problemas planteados por el buen ajuste de las prendas., Award-winning
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Active learning of manipulation sequences

  • Martínez Martínez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Torras, Carme|||0000-0002-2933-398X
  • Rossmann, Jürgen
  • Wantia, Nils
  • Eren Erdal, Aksoy
  • Haller, Simon
  • Piater, Justus
We describe a system allowing a robot to learn goal-directed manipulation sequences such as steps of an assembly task. Learning is based on a free mix of exploration and instruction by an external teacher, and may be active in the sense that the system tests actions to maximize learning progress and asks the teacher if needed. The main component is a symbolic planning engine that operates on learned rules, defined by actions and their pre- and postconditions. Learned by model-based reinforcement learning, rules are immediately available for planning. Thus, there are no distinct learning and application phases. We show how dynamic plans, replanned after every action if necessary, can be used for automatic execution of manipulation sequences, for monitoring of observed manipulation sequences, or a mix of the two, all while extending and refining the rule base on the fly. Quantitative results indicate fast convergence using few training examples, and highly effective teacher intervention at early stages of learning., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Dynamically consistent probabilistic model for robot motion learning

  • Pardo Ayala, Diego Esteban
  • Rozo Castañeda, Leonel
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
This work presents a probabilistic model for learning robot tasks from human demonstrations using kinesthetic teaching. The difference with respect to previous works is that a complete state of the robot is used to obtain a consistent representation of the dynamics of the task. The learning framework is based on hidden Markov models and Gaussian mixture regression, used for coding and reproducing the skills. Benefits of the proposed approach are shown in the execution of a simple self-crossing trajectory by a 7-DoF manipulator.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Execution of a dual-object (pushing) action with semantic event chains

  • Aksoy, Eren Erdal
  • Dellen, Babette
  • Tamosiunaite, Minija
  • Wörgötter, Florentin
Execution of a manipulation after learning from demonstration many times requires intricate planning and control systems or some form of manual guidance for a robot. Here we present a framework for manipulation execution based on the so called “Semantic Event Chain” which is an abstract description of relations between the objects in the scene. It captures the change of those relations during a manipulation and thereby provides the decisive temporal anchor points by which a manipulation is critically defined. Using semantic event chains a model of a manipulation can be learned. We will show that it is possible to add the required control parameters (the spatial anchor points) to this model, which can then be executed by a robot in a fully autonomous way. The process of learning and execution of semantic event chains is explained using a box pushing example.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Manipulation monitoring and robot intervention in complex manipulation sequences

  • Savarimuthu, Thiusius Rajeeth
  • Buch, Anders G.
  • Yang, Yang|||0009-0003-8266-3835
  • Mustafar, Wail
  • Haller, Simon
  • Papon, Jeremie
  • Martínez Martínez, David
  • Eren Erdal, Aksoy
Compared to machines, humans are intelligent and dexterous; they are indispensable for many complex tasks in areas such as flexible manufacturing or scientific experimentation. However, they are also subject to fatigue and inattention, which may cause errors. This motivates automated monitoring systems that verify the correct execution of manipulation sequences. To be practical, such a monitoring system should not require laborious programming., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Survey on assembly sequencing: a combinatorial and geometrical perspective

  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
A systematic overview on the subject of assembly sequencing is presented. Sequencing lies at the core of assembly planning, and variants include finding a feasible sequence—respecting the precedence constraints between the assembly operations—, or determining an optimal one according to one or several operational criteria. The different ways of representing the space of feasible assembly sequences are described, as well as the search and optimization algorithms that can be used. Geometry plays a fundamental role in devising the precedence constraints between assembly operations, and this is the subject of the second part of the survey, which treats also motion in contact in the context of the actual performance of assembly operations., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Depth-supported real-time video segmentation with the Kinect

  • Abramov, Alexey
  • Pauwels, Karl
  • Papon, Jeremie
  • Wörgötter, Florentin
  • Dellen, Babette
We present a real-time technique for the spatiotemporal segmentation of color/depth movies. Images are segmented using a parallel Metropolis algorithm implemented on a GPU utilizing both color and depth information, acquired with the Microsoft Kinect. Segments represent the equilibrium states of a Potts model, where tracking of segments is achieved by warping obtained segment labels to the next frame using real-time optical flow, which reduces the number of iterations required for the Metropolis method to encounter the new equilibrium state. By including depth information into the framework, true objects boundaries can be found more easily, improving also the temporal coherency of the method. The algorithm has been tested for videos of medium resolutions showing human manipulations of objects. The framework provides an inexpensive visual front end for visual preprocessing of videos in industrial settings and robot labs which can potentially be used in various applications., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Object detection methods for robot grasping: Experimental assessment and tuning

  • Rigual Aparici, Ferran
  • Ramisa Ayats, Arnau
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
In this work we address the problem of object detection for the purpose of object manipulation in a service robotics scenario. Several implementations of state-of-the-art object detection methods were tested, and the one with the best performance was selected. During the evaluation, three main practical limitations of current methods were identified in relation with long-range object detection, grasping point detection and automatic learning of new objects; and practical solutions are proposed for the last two. Finally, the complete pipeline is evaluated in a real grasping experiment.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Positioning two redundant arms for cooperative manipulation of objects

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Torras, Carme|||0000-0002-2933-398X
Bimanual manipulation of objects is receiving a lot of attention nowadays, but there is few literature addressing the design of the arms configuration. In this paper, we propose a way to analyze the relative positioning of two redundant arms, both equipped with spherical wrists, in order to obtain the best common workspace for grasping purposes. Considering the geometry of a robot with a spherical wrist, the Cartesian workspace can be discretized, with an easy representation of the feasible end-effector orientations at each point using bounding cones. After having characterized the workspace for one robot arm, we can evaluate how good each of the discretized poses relate with an identical arm in another position with a quality function that considers orientations. In the end, we obtain a quality value for each relative position of two arms, and we perform an optimization using genetic algorithms to obtain the best workspace for a cooperative task., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Handling high parameter dimensionality in reinforcement learning with dynamic motor primitives

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
Dynamic Motor Primitives (DMP) are nowadays widely used as movement parametrization for learning trajectories, because of their linearity in the parameters, rescalation robustness and continuity. However, when learning a movement with DMP, where a set of gaussians distributed along the trajectory is used to approximate an acceleration excitation function, a very large number of gaussian approximations need to be performed. Adding them up for all joints yields too many parameters to be explored, thus requiring a prohibitive number of experiments/simulations to converge to a solution with an optimal (locally or globally) reward. We propose here two strategies to reduce this dimensionality: the first is to explore only the most significant directions in the parameter space, and the second is to add a reduced second set of gaussians that should only optimize the trajectory after fixing the gaussians that approximate the demonstrated movement., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Dense outdoor 3D mapping and navigation with Pose SLAM

  • Teniente Avilés, Ernesto
  • Valencia Carreño, Rafael
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
Pose SLAM is the variant of SLAM where only the robot trajectory is estimated and in which landmarks are solely used to compute relative constraints between robot poses. In previous work, we have developed efficient methods to build Pose SLAM maps that ponder the information content on odometry and measurement links to keep the graph of poses sparse. In this paper we show results of Pose SLAM mapping with our custom built 3D laser and an outdoor all-terrain mobile robot. Finally, we argue that Pose SLAM graphs can be directly used as belief roadmaps and, thus, used for path planning under uncertainty. We show how to plan trajectories with the lowest accumulated robot pose uncertainty, i.e., the most reliable path to the goal, taking into account the encoded uncertainty in the map. Results of this navigation strategy are demonstrated with our outdoor robot., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Planning surface cleaning tasks by learning uncertain drag actions outcomes

  • Martínez Martínez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
A method to perform cleaning tasks is presented where a robot manipulator autonomously grasps a textile and uses different dragging actions to clean a surface. Ac- tions are imprecise, and probabilistic planning is used to select the best sequence of actions. The character- ization of such actions is complex because the initial autonomous grasp of the textile introduces differences in the initial conditions that change the efficacy of the robot cleaning actions. We demonstrate that the action outcome probabilities can be learned very fast while the task is being executed, so as to progressively improve robot performance. The learner adds only a little over- head to the system compared to the improvements ob- tained. Experiments with a real robot show that the most effective plan varies depending on the initial grasp, and that plans become better after only a few learning itera- tions, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A novel real-time edge-preserving smoothing filter

  • Reich, Simon
  • Abramov, Alexey
  • Papon, Jeremie
  • Worgotter, Florentin
  • Dellen, Babette
The segmentation of textured and noisy areas in images is a very challenging task due to the large variety of objects and materials in natural environments, which cannot be solved by a single similarity measure. In this paper, we address this problem by proposing a novel edge-preserving texture filter, which smudges the color values inside uniformly textured areas, thus making the processed image more workable for color-based image segmentation. Due to the highly parallel structure of the method, the implementation on a GPU runs in realtime, allowing us to process standard images within tens of milliseconds. By preprocessing images with this novel filter before applying a recent real-time color-based image segmentation method, we obtain significant improvements in performance for images from the Berkeley dataset, outperforming an alternative version using a standard bilateral filter for preprocessing. We further show that our combined approach leads to better segmentations in terms of a standard performance measure than graph-based and mean-shift segmentation for the Berkeley image dataset., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations

  • Dellen, Babette
  • Husain, Syed Farzad
  • Torras, Carme|||0000-0002-2933-398X
A novel framework for joint segmentation and tracking in depth videos of object surfaces is presented. Initially, the 3D colored point cloud obtained using the Kinect camera is used to segment the scene into surface patches, defined by quadratic functions. The computed segments together with their functional descriptions are then used to partition the depth image of the subsequent frame in a consistent manner with respect to the precedent frame. This way, solutions established in previous frames can be reused which improves the efficiency of the algorithm and the coherency of the segmentations along the movie. The algorithm is tested for scenes showing human and robot manipulations of objects. We demonstrate that the method can successfully segment and track the human/robot arm and object surfaces along the manipulations. The performance is evaluated quantitatively by measuring the temporal coherency of the segmentations and the segmentation covering using ground truth. The method provides a visual front-end designed for robotic applications, and can potentially be used in the context of manipulation recognition, visual servoing, and robot-grasping tasks, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Single image 3D human pose estimation from noisy observations

  • Simó Serra, Edgar
  • Ramisa Ayats, Arnau
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
  • Moreno-Noguer, Francesc
Markerless 3D human pose detection from a single image is a severely underconstrained problem because different 3D poses can have similar image projections. In order to handle this ambiguity, current approaches rely on prior shape models that can only be correctly adjusted if 2D image features are accurately detected. Unfortunately, although current 2D part detector algorithms have shown promising results, they are not yet accurate enough to guarantee a complete disambiguation of the 3D inferred shape. In this paper, we introduce a novel approach for estimating 3D human pose even when observations are noisy. We propose a stochastic sampling strategy to propagate the noise from the image plane to the shape space. This provides a set of ambiguous 3D shapes, which are virtually undistinguishable from their image projections. Disambiguation is then achieved by imposing kinematic constraints that guarantee the resulting pose resembles a 3D human shape. We validate the method on a variety of situations in which state-of-the-art 2D detectors yield either inaccurate estimations or partly miss some of the body parts.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Finding safe policies in model-based active learning

  • Martínez Martínez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
Task learning in robotics is a time-consuming process, and model-based reinforcement learning algorithms have been proposed to learn with just a small amount of experiences. However, reducing the number of experiences used to learn implies that the algorithm may overlook crucial actions required to get an optimal behavior. For example, a robot may learn simple policies that have a high risk of not reaching the goal because they often fall into dead-ends. We propose a new method that allows the robot to reason about dead-ends and their causes. Analyzing its current model and experiences, the robot will hypothesize the possible causes for the dead-end, and identify the actions that may cause it, marking them as dangerous. Afterwards, whenever a dangerous action is included into a plan which has a high risk of leading to a dead-end, the special action request teacher confirmation will be triggered by the robot to actively confirm with a teacher that the planned risky action should be executed. This method permits learning safer policies with the addition of just a few teacher demonstration requests. Experimental validation of the approach is provided in two different scenarios: a robotic assembly task and a domain from the international planning competition. Our approach gets success ratios very close to 1 in problems where previous approaches had high probabilities of reaching dead-ends, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Exhaustive linearization for robust camera pose and focal length estimation

  • Peñate Sanchez, Adrián
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
  • Moreno-Noguer, Francesc
We propose a novel approach for the estimation of the pose and focal length of a camera from a set of 3D-to-2D point correspondences. Our method compares favorably to competing approaches in that it is both more accurate than existing closed form solutions, as well as faster and also more accurate than iterative ones. Our approach is inspired on the EPnP algorithm, a recent O(n) solution for the calibrated case. Yet we show that considering the focal length as an additional unknown renders the linearization and relinearization techniques of the original approach no longer valid, especially with large amounts of noise. We present new methodologies to circumvent this limitation termed exhaustive linearization and exhaustive relinearization which perform a systematic exploration of the solution space in closed form. The method is evaluated on both real and synthetic data, and our results show that besides producing precise focal length estimation, the retrieved camera pose is almost as accurate as the one computed using the EPnP, which assumes a calibrated camera., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Tesis doctoral (DoctoralThesis)

Bimanual robot skills: MP encoding, dimensionality reduction and reinforcement learning

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
Aplicat embargament des de la data de defensa fins 1/7/2018, Premio a la mejor Tesis Doctoral sobre Robótica, Edición 2017, atorgat pel Comité Español de Automática., Finalista del 2018 George Girault PhD Award, from EuRobotics, In our culture, robots have been in novels and cinema for a long time, but it has been specially in the last two decades when the improvements in hardware - better computational power and components - and advances in Artificial Intelligence (AI), have allowed robots to start sharing spaces with humans. Such situations require, aside from ethical considerations, robots to be able to move with both compliance and precision, and learn at different levels, such as perception, planning, and motion, being the latter the focus of this work. The first issue addressed in this thesis is inverse kinematics for redundant robot manipulators, i.e: positioning the robot joints so as to reach a certain end-effector pose. We opt for iterative solutions based on the inversion of the kinematic Jacobian of a robot, and propose to filter and limit the gains in the spectral domain, while also unifying such approach with a continuous, multipriority scheme. Such inverse kinematics method is then used to derive manipulability in the whole workspace of an antropomorphic arm, and the coordination of two arms is subsequently optimized by finding their best relative positioning. Having solved the kinematic issues, a robot learning within a human environment needs to move compliantly, with limited amount of force, in order not to harm any humans or cause any damage, while being as precise as possible. Therefore, we developed two dynamic models for the same redundant arm we had analysed kinematically: The first based on local models with Gaussian projections, and the second characterizing the most problematic term of the dynamics, namely friction. Such models allowed us to implement feed-forward controllers, where we can actively change the weights in the compliance-precision tradeoff. Moreover, we used such models to predict external forces acting on the robot, without the use of force sensors. Afterwards, we noticed that bimanual robots must coordinate their components (or limbs) and be able to adapt to new situations with ease. Over the last decade, a number of successful applications for learning robot motion tasks have been published. However, due to the complexity of a complete system including all the required elements, most of these applications involve only simple robots with a large number of high-end technology sensors, or consist of very simple and controlled tasks. Using our previous framework for kinematics and control, we relied on two types of movement primitives to encapsulate robot motion. Such movement primitives are very suitable for using reinforcement learning. In particular, we used direct policy search, which uses the motion parametrization as the policy itself. In order to improve the learning speed in real robot applications, we generalized a policy search algorithm to give some importance to samples yielding a bad result, and we paid special attention to the dimensionality of the motion parametrization. We reduced such dimensionality with linear methods, using the rewards obtained through motion repetition and execution. We tested such framework in a bimanual task performed by two antropomorphic arms, such as the folding of garments, showing how a reduced dimensionality can provide qualitative information about robot couplings and help to speed up the learning of tasks when robot motion executions are costly., A la nostra cultura, els robots han estat presents en novel·les i cinema des de fa dècades, però ha sigut especialment en les últimes dues quan les millores en hardware (millors capacitats de còmput) i els avenços en intel·ligència artificial han permès que els robots comencin a compartir espais amb els humans. Aquestes situacions requereixen, a banda de consideracions ètiques, que els robots siguin capaços de moure's tant amb suavitat com amb precisió, i d'aprendre a diferents nivells, com són la percepció, planificació i moviment, essent l'última el centre d'atenció d'aquest treball. El primer problema adreçat en aquesta tesi és la cinemàtica inversa, i.e.: posicionar les articulacions del robot de manera que l'efector final estigui en una certa posició i orientació. Hem estudiat el camp de les solucions iteratives, basades en la inversió del Jacobià cinemàtic d'un robot, i proposem un filtre que limita els guanys en el seu domini espectral, mentre també unifiquem tal mètode dins un esquema multi-prioritat i continu. Aquest mètode per a la cinemàtica inversa és usat a l'hora d'encapsular tota la informació sobre l'espai de treball d'un braç antropomòrfic, i les capacitats de coordinació entre dos braços són optimitzades, tot trobant la seva millor posició relativa en l'espai. Havent resolt les dificultats cinemàtiques, un robot que aprèn en un entorn humà necessita moure's amb suavitat exercint unes forces limitades per tal de no causar danys, mentre es mou amb la màxima precisió possible. Per tant, hem desenvolupat dos models dinàmics per al mateix braç robòtic redundant que havíem analitzat des del punt de vista cinemàtic: El primer basat en models locals amb projeccions de Gaussianes i el segon, caracteritzant el terme més problemàtic i difícil de representar de la dinàmica, la fricció. Aquests models ens van permetre utilitzar controladors coneguts com "feed-forward", on podem canviar activament els guanys buscant l'equilibri precisió-suavitat que més convingui. A més, hem usat aquests models per a inferir les forces externes actuant en el robot, sense la necessitat de sensors de força. Més endavant, ens hem adonat que els robots bimanuals han de coordinar els seus components (braços) i ser capaços d'adaptar-se a noves situacions amb facilitat. Al llarg de l'última dècada, diverses aplicacions per aprendre tasques motores robòtiques amb èxit han estat publicades. No obstant, degut a la complexitat d'un sistema complet que inclogui tots els elements necessaris, la majoria d'aquestes aplicacions consisteixen en robots més aviat simples amb costosos sensors d'última generació, o a resoldre tasques senzilles en un entorn molt controlat. Utilitzant el nostre treball en cinemàtica i control, ens hem basat en dos tipus de primitives de moviment per caracteritzar la motricitat robòtica. Aquestes primitives de moviment són molt adequades per usar aprenentatge per reforç. En particular, hem usat la búsqueda directa de la política, un camp de l'aprenentatge per reforç que usa la parametrització del moviment com la pròpia política. Per tal de millorar la velocitat d'aprenentatge en aplicacions amb robots reals, hem generalitzat un algoritme de búsqueda directa de política per a donar importància a les mostres amb mal resultat, i hem donat especial atenció a la reducció de dimensionalitat en la parametrització dels moviments. Hem reduït la dimensionalitat amb mètodes lineals, utilitzant les recompenses obtingudes EN executar els moviments. Aquests mètodes han estat provats en tasques bimanuals com són plegar roba, usant dos braços antropomòrfics. Els resultats mostren com la reducció de dimensionalitat pot aportar informació qualitativa d'una tasca, i al mateix temps ajuda a aprendre-la més ràpid quan les execucions amb robots reals són costoses., Award-winning
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A joint model for 2D and 3D pose estimation from a single image

  • Simó Serra, Edgar
  • Quattoni, Ariadna Julieta|||0000-0002-7879-7174
  • Torras, Carme|||0000-0002-2933-398X
  • Moreno-Noguer, Francesc
We introduce a novel approach to automatically recover 3D human pose from a single image. Most previous work follows a pipelined approach: initially, a set of 2D features such as edges, joints or silhouettes are detected in the image, and then these observations are used to infer the 3D pose. Solving these two problems separately may lead to erroneous 3D poses when the feature detector has performed poorly. In this paper, we address this issue by jointly solving both the 2D detection and the 3D inference problems. For this purpose, we propose a Bayesian framework that integrates a generative model based on latent variables and discriminative 2D part detectors based on HOGs, and perform inference using evolutionary algorithms. Real experimentation demonstrates competitive results, and the ability of our methodology to provide accurate 2D and 3D pose estimations even when the 2D detectors are inaccurate., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Segmentation of dynamic objects from laser data

  • Ortega Jiménez, Agustín Alberto
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
We present a method to segment dynamic objects from high-resolution low-rate laser scans. Data points are tagged as static or dynamic based on the classification of pixel data from registered imagery. Per-pixel background classes are adapted online as Gaussian mixtures, and their matching 3D points are classified accordingly. Special attention is paid to the correct calibration and synchronization of the scanner with the the accessory camera. Results of the method are shown for a small indoor sequence with several people following arbitrarily different trajectories., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Information-gain view planning for free-form object reconstruction with a 3D ToF camera

  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Kriegel, Simon
  • Fuchs, Stefan
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
Active view planning for gathering data from an unexplored 3D complex scenario is a hard and still open problem in the computer vision community. In this paper, we present a general task-oriented approach based on an information-gain maximization that easily deals with such a problem. Our approach consists of ranking a given set of possible actions, based on their task-related gains, and then executing the best-ranked action to move the required sensor. An example of how our approach behaves is demonstrated by applying it over 3D raw data for real-time volume modelling of complex-shaped objects. Our setting includes a calibrated 3D time-of-flight (ToF) camera mounted on a 7 degrees of freedom (DoF) robotic arm. Noise in the sensor data acquisition, which is too often ignored, is here explicitly taken into account by computing an uncertainty matrix for each point, and refining this matrix each time the point is seen again. Results show that, by always choosing the most informative view, a complete model of a 3D free-form object is acquired and also that our method achieves a good compromise between speed and precision.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

External force estimation for textile grasp detection

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Pardo Ayala, Diego Esteban
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
Our current work on external force estimation without end-effector force sensor is resented.To verify if a grasp of a textile has been successful, the external wrench applied on the robot is computed online, with a state observer based on a LWPR [3] model of a task., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Consistent depth video segmentation using adaptive surface models

  • Husain, Syed Farzad
  • Dellen, Babette
  • Torras, Carme|||0000-0002-2933-398X
We propose a new approach for the segmentation of 3-D point clouds into geometric surfaces using adaptive surface models. Starting from an initial configuration, the algorithm converges to a stable segmentation through a new iterative split-and-merge procedure, which includes an adaptive mechanism for the creation and removal of segments. This allows the segmentation to adjust to changing input data along the movie, leading to stable, temporally coherent, and traceable segments. We tested the method on a large variety of data acquired with different range imaging devices, including a structured-light sensor and a time-of-flight camera, and successfully segmented the videos into surface segments. We further demonstrated the feasibility of the approach using quantitative evaluations based on ground-truth data., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

DaLI: deformation and light invariant descriptor

  • Simó Serra, Edgar
  • Torras, Carme|||0000-0002-2933-398X
  • Moreno-Noguer, Francesc
Recent advances in 3D shape analysis and recognition have shown that heat diffusion theory can be effectively used to describe local features of deforming and scaling surfaces. In this paper, we show how this description can be used to characterize 2D image patches, and introduce DaLI, a novel feature point descriptor with high resilience to non-rigid image transformations and illumination changes. In order to build the descriptor, 2D image patches are initially treated as 3D surfaces. Patches are then described in terms of a heat kernel signature, which captures both local and global information, and shows a high degree of invariance to non-linear image warps. In addition, by further applying a logarithmic sampling and a Fourier transform, invariance to photometric changes is achieved. Finally, the descriptor is compacted by mapping it onto a low dimensional subspace computed using Principal Component Analysis, allowing for an efficient matching. A thorough experimental validation demonstrates that DaLI is significantly more discriminative and robust to illuminations changes and image transformations than state of the art descriptors, even those specifically designed to describe non-rigid deformations., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Learning RGB-D descriptors of garment parts for informed robot grasping

  • Ramisa Ayats, Arnau
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Moreno-Noguer, Francesc
  • Torras, Carme|||0000-0002-2933-398X
Robotic handling of textile objects in household environments is an emerging application that has recently received considerable attention thanks to the development of domestic robots. Most current approaches follow a multiple re-grasp strategy for this purpose, in which clothes are sequentially grasped from different points until one of them yields a desired configuration. In this work we propose a vision-based method, built on the Bag of Visual Words approach, that combines appearance and 3D information to detect parts suitable for grasping in clothes, even when they are highly wrinkled. We also contribute a new, annotated, garment part dataset that can be used for benchmarking classification, part detection, and segmentation algorithms. The dataset is used to evaluate our approach and several state-of-the-art 3D descriptors for the task of garment part detection. Results indicate that appearance is a reliable source of information, but that augmenting it with 3D information can help the method perform better with new clothing items., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Teaching a robot the semantics of assembly tasks

  • Savarimuthu, Thiusius Rajeeth
  • Anders, Buch
  • Schlette, Christian
  • Wantia, Nils
  • Rossmann, Jürgen
  • Martinez Martinez, David
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
  • Ales, Ude
  • Bojan, Nemec
  • Aljaž, Kramberger
  • Worgotter, Florentin
  • Eren Erdal, Aksoy
  • Papon, Jeremie
  • Haller, Simon
  • Piater, Justus
  • Krüger, Norbert
© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We present a three-level cognitive system in a learning by demonstration context. The system allows for learn- ing and transfer on the sensorimotor level as well as the planning level. The fundamentally different data structures associated with these two levels are connected by an efficient mid-level represen- tation based on so-called “semantic event chains.” We describe details of the representations and quantify the effect of the asso- ciated learning procedures for each level under different amounts of noise. Moreover, we demonstrate the performance of the over- all system by three demonstrations that have been performed at a project review. The described system has a technical readiness level (TRL) of 4, which in an ongoing follow-up project will be raised to TRL 6., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Learning collaborative impedance-based robot behaviors

  • Rozo Castañeda, Leonel
  • Calinon, Sylvain
  • Caldwell, Darwin
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Torras, Carme|||0000-0002-2933-398X
Research in learning from demonstration has focused on transferring movements from humans to robots. However, a need is arising for robots that do not just replicate the task on their own, but that also interact with humans in a safe and natural way to accomplish tasks cooperatively. Robots with variable impedance capabilities opens the door to new challenging applications, where the learning algorithms must be extended by encapsulating force and vision information. In this paper we propose a framework to transfer impedance-based behaviors to a torque-controlled robot by kinesthetic teaching. The proposed model encodes the exam- ples as a task-parameterized statistical dynamical system, where the robot impedance is shaped by estimating virtual stiffness matrices from the set of demonstrations. A collaborative assembly task is used as testbed. The results show that the model can be used to modify the robot impedance along task execution to facilitate the collaboration, by triggering stiff and compliant behaviors in an on-line manner to adapt to the user's actions., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Tesis doctoral (DoctoralThesis)

Learning relational models with human interaction for planning in robotics

  • Martínez Martínez, David
Automated planning has proven to be useful to solve problems where an agent has to maximize a reward function by executing actions. As planners have been improved to salve more expressive and difficult problems, there is an increasing interest in using planning to improve efficiency in robotic tasks. However, planners rely on a domain model, which has to be either handcrafted or learned. Although learning domain models can be very costly, recent approaches provide generalization capabilities and integrate human feedback to reduce the amount of experiences required to learn. In this thesis we propase new methods that allow an agent with no previous knowledge to solve certain problems more efficiently by using task planning. First, we show how to apply probabilistic planning to improve robot performance in manipulation tasks (such as cleaning the dirt or clearing the tableware on a table). Planners obtain sequences of actions that get the best result in the long term, beating reactive strategies. Second, we introduce new reinforcement learning algorithms where the agent can actively request demonstrations from a teacher to learn new actions and speed up the learning process. In particular, we propase an algorithm that allows the user to set the mínimum quality to be achieved, where a better quality also implies that a larger number of demonstrations will be requested . Moreover, the learned model is analyzed to extract the unlearned or problematic parts of the model. This information allow the agent to provide guidance to the teacher when a demonstration is requested, and to avoid irrecoverable errors. Finally, a new domain model learner is introduced that, in addition to relational probabilistic action models, can also learn exogenous effects. This learner can be integrated with existing planners and reinforcement learning algorithms to salve a wide range of problems. In summary, we improve the use of learning and task planning to salve unknown tasks. The improvements allow an agent to obtain a larger benefit from planners, learn faster, balance the number of action executions and teacher demonstrations, avoid irrecoverable errors, interact with a teacher to solve difficult problems, and adapt to the behavior of other agents by learning their dynamics. All the proposed methods were compared with state-of-the-art approaches, and were also demonstrated in different scenarios, including challenging robotic tasks., La planificación automática ha probado ser de gran utilidad para resolver problemas en los que un agente tiene que ejecutar acciones para maximizar una función de recompensa. A medida que los planificadores han sido capaces de resolver problemas cada vez más complejos, ha habido un creciente interés por utilizar dichos planificadores para mejorar la eficiencia de tareas robóticas. Sin embargo, los planificadores requieren un modelo del dominio, el cual puede ser creado a mano o aprendido. Aunque aprender modelos automáticamente puede ser costoso, recientemente han aparecido métodos que permiten la interacción persona-máquina y generalizan el conocimiento para reducir la cantidad de experiencias requeridas para aprender. En esta tesis proponemos nuevos métodos que permiten a un agente sin conocimiento previo de la tarea resolver problemas de forma más eficiente mediante el uso de planificación automática. Comenzaremos mostrando cómo aplicar planificación probabilística para mejorar la eficiencia de robots en tareas de manipulación (como limpiar suciedad o recoger una mesa). Los planificadores son capaces de obtener las secuencias de acciones que producen los mejores resultados a largo plazo, superando a las estrategias reactivas. Por otro lado, presentamos nuevos algoritmos de aprendizaje por refuerzo en los que el agente puede solicitar demostraciones a un profesor. Dichas demostraciones permiten al agente acelerar el aprendizaje o aprender nuevas acciones. En particular, proponemos un algoritmo que permite al usuario establecer la mínima suma de recompensas que es aceptable obtener, donde una recompensa más alta implica que se requerirán más demostraciones. Además, el modelo aprendido será analizado para identificar qué partes están incompletas o son problemáticas. Esta información permitirá al agente evitar errores irrecuperables y también guiar al profesor cuando se solicite una demostración. Finalmente, se ha introducido un nuevo método de aprendizaje para modelos de dominios que, además de obtener modelos relacionales de acciones probabilísticas, también puede aprender efectos exógenos. Mostraremos cómo integrar este método en algoritmos de aprendizaje por refuerzo para poder abordar una mayor cantidad de problemas. En resumen, hemos mejorado el uso de técnicas de aprendizaje y planificación para resolver tareas desconocidas a priori. Estas mejoras permiten a un agente aprovechar mejor los planificadores, aprender más rápido, elegir entre reducir el número de acciones ejecutadas o el número de demostraciones solicitadas, evitar errores irrecuperables, interactuar con un profesor para resolver problemas complejos, y adaptarse al comportamiento de otros agentes aprendiendo sus dinámicas. Todos los métodos propuestos han sido comparados con trabajos del estado del arte, y han sido evaluados en distintos escenarios, incluyendo tareas robóticas.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Robot learning from demonstration of force-based tasks with multiple solution trajectories

  • Rozo Castañeda, Leonel
  • Jiménez Schlegl, Pablo|||0000-0003-3627-4938
  • Torras, Carme|||0000-0002-2933-398X
A learning framework with a bidirectional communication channel is proposed, where a human performs several demonstrations of a task using a haptic device (providing him/her with force-torque feedback) while a robot captures these executions using only its force-based perceptive system. Our work departs from the usual approaches to learning by demonstration in that the robot has to execute the task blindly, relying only on force-torque perceptions, and, more essential, we address goal-driven manipulation tasks with multiple solution trajectories, whereas most works tackle tasks that can be learned by just finding a generalization at the trajectory level. To cope with these multiple-solution tasks, in our framework demonstrations are represented by means of a Hidden Markov Model (HMM) and the robot reproduction of the task is performed using a modified version of Gaussian Mixture Regression that incorporates temporal information (GMRa) through the forward variable of the HMM. Also, we exploit the haptic device as a teaching and communication tool in a human-robot interaction context, as an alternative to kinesthetic-based teaching systems. Results show that the robot is able to learn a container-emptying task relying only on force-based perceptions and to achieve the goal from several non-trained initial conditions.
UPCommons. Portal del coneixement obert de la UPC
Publication Capítulo o parte de un libro (BookPart)

ToF cameras for eye-in-hand robotics

  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Torras, Carme|||0000-0002-2933-398X
A Time-of-Flight (ToF) camera is a relatively new type of sensor that delivers 3-dimensional images at high frame rate, simultaneously providing intensity data and range information for every pixel. It has been used in a wide range of applications and here we will describe the lessons learned by using such cameras to perform robotic tasks, specifically in eye-in-hand configurations. In such configurations, the camera is attached to the end-effector of a robot manipulator, so that new images can be obtained by actively changing the point of view of the camera., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

FINDDD: A fast 3D descriptor to characterize textiles for robot manipulation

  • Ramisa Ayats, Arnau
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Moreno-Noguer, Francesc
  • Torras, Carme|||0000-0002-2933-398X
Most current depth sensors provide 2.5D range images in which depth values are assigned to a rectangular 2D array. In this paper we take advantage of this structured information to build an efficient shape descriptor which is about two orders of magnitude faster than competing approaches, while showing similar performance in several tasks involving deformable object recognition. Given a 2D patch surrounding a point and its associated depth values, we build the descriptor for that point, based on the cumulative distances between their normals and a discrete set of normal directions. This processing is made very efficient using integral images, even allowing to compute descriptors for every range image pixel in a few seconds. The discriminative power of our descriptor, dubbed FINDDD, is evaluated in three different scenarios: recognition of specific cloth wrinkles, instance recognition from geometry alone, and detection of reliable and informed grasping points., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Dimensionality reduction and motion coordination in learning trajectories with dynamic movement primitives

  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Torras, Carme|||0000-0002-2933-398X
Dynamic Movement Primitives (DMP) are nowadays widely used as movement parametrization for learning trajectories, because of their linearity in the parameters, rescaling robustness and continuity. However, when learning a movement with a robot using DMP, many parameters may need to be tuned, requiring a prohibitive number of experiments/simulations to converge to a solution with a locally or globally optimal reward. We propose here strategies to palliate this dimensionality problem: the first is to explore only along the most significant directions in the parameter space, and the second is to add a reduced second set of Gaussians that would optimize the trajectory after fixing the Gaussians approximating the demonstrated movement. Both strategies result in less Gaussian computations and better performance on learning algorithms. To further speed up the learning and allow for a better biased exploration, we also propose to coordinate the motion of different joints, by computing a coordination matrix initialized with the demonstrated movement and then automatically updating it by eliminating the degrees of freedom least affecting task performance. Our three proposals have been experimentally tested and the obtained results show that similar (or even better) performance can be obtained at a significantly lower computational cost by reducing the dimensionality of the exploration space., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Realtime tracking and grasping of a moving object from range video

  • Husain, Syed Farzad
  • Colomé Figueras, Adrià|||0000-0001-9715-4062
  • Dellen, Babette
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
In this paper we present an automated system that is able to track and grasp a moving object within the workspace of a manipulator using range images acquired with a Microsoft Kinect sensor. Realtime tracking is achieved by a geometric particle filter on the affine group. Based on the tracked output, the pose of a 7-DoF WAM robotic arm is continuously updated using dynamic motor primitives until a distance measure between the tracked object and the gripper mounted on the arm is below a threshold. Then, it closes its three fingers and grasps the object. The tracker works in real-time and is robust to noise and partial occlusions. Using only the depth data makes our tracker independent of texture which is one of the key design goals in our approach. An experimental evaluation is provided along with a comparison of the proposed tracker with state-of-the-art approaches, including the OpenNI-tracker. The developed system is integrated with ROS and made available as part of IRI's ROS stack., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Service robots for citizens of the future

  • Torras, Carme|||0000-0002-2933-398X
Robots are no longer confined to factories; they are progressively spreading to urban, social and assistive domains. In order to become handy co-workers and helpful assistants, they must be endowed with quite different abilities from their industrial ancestors. Research on service robots aims to make them intrinsically safe to people, easy to teach by non-experts, able to manipulate not only rigid but also deformable objects, and highly adaptable to non-predefined and dynamic environments. Robots worldwide will share object and environmental models, their acquired knowledge and experiences through global databases and, together with the internet of things, will strongly change the citizens’ way of life in so-called smart cities. This raises a number of social and ethical issues that are now being debated not only within the Robotics community but by society at large., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

FaMSA: fast multi-scan alignment with partially known correspondences

  • Teniente Avilés, Ernesto
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
This paper presents FaMSA, an efficient method to boost 3D scan registration from partially known correspondence sets. This situation is typical at loop closure in large laser-based mapping sessions. In such cases, scan registration for consecutive point clouds has already been made during open loop traverse, and the point match history can be used to speed up the computation of new scan matches. FaMSA allows to quickly match a new scan with multiple consecutive scans at a time, with the consequent benefits in computational speed. Registration error is shown to be comparable to that of independent scan alignment. Results are shown for dense 3D outdoor scan matching.
UPCommons. Portal del coneixement obert de la UPC
Publication Tesis doctoral (DoctoralThesis)

Task-oriented viewpoint planning for free-form objects

  • Foix Salmerón, Sergi|||0000-0001-9249-6696
This thesis deals with active sensing and its use in real exploration tasks under both scene ambiguities and measurement uncertainties. While object modeling is the implicit objective of most of active sensing algorithms, in this work we have explored new strategies to deal with more generic and more complex tasks. Active sensing requires the ability of moving the perceptual system to gather new information. Our approach uses a robot manipulator with a 3D Time-of-Flight (ToF) camera attached to the end-effector. For a complex task, we have focused our attention on plant phenotyping. Plants are complex objects, with leaves that change their position and size along time. Valid viewpoints for a certain plant are hardly valid for a different one, even belonging to the same species. Some instruments, such as chlorophyll meters or disk sampling tools, require being precisely positioned over a particular location of the leaf. Therefore, their use requires the modeling of specific regions of interest of the plant, including also the free space needed for avoiding obstacles and approaching the leaf with tool. It is easy to observe that predefined camera trajectories are not valid here, and that usually with one single view it is very difficult to acquire all the required information. The overall objective of this thesis is to solve complex active sensing tasks by embedding their exploratory goal into a pre-estimated geometrical model, using information-gain as the fundamental guideline for the reward function. The main contributions can be divided in two groups: first, the evaluation of ToF cameras and their calibration to assess the uncertainty of the measurements (presented in Part I); and second, the proposal of a framework capable of embedding the task, modeled as free and occupied space, and that takes into account the modeled sensor's uncertainty to improve the action selection algorithm (presented in Part II). This thesis has given rise to 14 publications, including 5 indexed journals, and its results have been used in the GARNICS European project. The complete framework is based on the Next-Best-View methodology and it can be summarized in the following main steps. First, an initial view of the object (e.g., a plant) is acquired. From this initial view and given a set of candidate viewpoints, the expected gain obtained by moving the robot and acquiring the next image is computed. This computation takes into account the uncertainty from all the different pixels of the sensor, the expected information based on a predefined task model, and the possible occlusions. Once the most promising view is selected, the robot moves, takes a new image, integrates this information intothe model, and evaluates again the set of remaining views. Finally, the task terminates when enough information is gathered. In our examples, this process enables the robot to perform a measurement on top of a leaf. The key ingredient is to model the complexity of the task in a layered representation of free-occupied occupancy grid maps. This allows to naturally encode the requirements of the task, to maintain and update the belief state with the measurements performed, to simulate and compute the expected gains of all potential viewpoints, and to encode the termination condition. During this work the technology of ToF cameras has incredibly evolved. Nowadays it is very popular and ToF cameras are already embedded in some consumer devices. Although the quality of the measurements has been considerably improved, it is still not uniform in the sensor. We believe, as it has been demonstrated in various experiments in this work, that a careful modeling of the sensor's uncertainty is highly beneficial and helps to design better decision systems. In our case, it enables a more realistic computation of the information gain measure, and consequently, a better selection criterion., Aquesta tesi aborda el tema de la percepció activa i el seu ús en tasques d'exploració en entorns reals tot considerant la ambigüitat en l'escena i la incertesa del sistema de percepció. Al contrari de la majoria d'algoritmes de percepció activa, on el modelatge d'objectes sol ser l'objectiu implícit, en aquesta tesi hem explorat noves estratègies per poder tractar tasques genèriques i de major complexitat. Tot sistema de percepció activa requereix un aparell sensorial amb la capacitat de variar els seus paràmetres de forma controlada, per poder, d'aquesta manera, recopilar nova informació per resoldre una tasca determinada. En tasques d'exploració, la posició i orientació del sensor són paràmetres claus per resoldre la tasca. En el nostre estudi hem fet ús d'un robot manipulador com a sistema de posicionament i d'una càmera de profunditat de temps de vol (ToF), adherida al seu efector final, com a sistema de percepció. Com a tasca final, ens hem concentrat en l'adquisició de mesures sobre fulles dins de l'àmbit del fenotipatge de les plantes. Les plantes son objectes molt complexos, amb fulles que canvien de textura, posició i mida al llarg del temps. Això comporta diverses dificultats. Per una banda, abans de dur a terme una mesura sobre un fulla s'ha d'explorar l'entorn i trobar una regió que ho permeti. A més a més, aquells punts de vista que han estat adequats per una determinada planta difícilment ho seran per una altra, tot i sent les dues de la mateixa espècie. Per un altra banda, en el moment de la mesura, certs instruments, tals com els mesuradors de clorofil·la o les eines d'extracció de mostres, requereixen ser posicionats amb molta precisió. És necessari, doncs, disposar d'un model detallat d'aquestes regions d'interès, i que inclogui no només l'espai ocupat sinó també el lliure. Gràcies a la modelització de l'espai lliure es pot dur a terme una bona evitació d'obstacles i un bon càlcul de la trajectòria d'aproximació de l'eina a la fulla. En aquest context, és fàcil veure que, en general, amb un sol punt de vista no n'hi ha prou per adquirir tota la informació necessària per prendre una mesura, i que l'ús de trajectòries predeterminades no garanteixen l'èxit. L'objectiu general d'aquesta tesi és resoldre tasques complexes de percepció activa mitjançant la codificació del seu objectiu d'exploració en un model geomètric prèviament estimat, fent servir el guany d'informació com a guia fonamental dins de la funció de cost. Les principals contribucions d'aquesta tesi es poden dividir en dos grups: primer, l'avaluació de les càmeres ToF i el seu calibratge per poder avaluar la incertesa de les seves mesures (presentat en la Part I); i en segon lloc, la proposta d'un sistema capaç de codificar la tasca mitjançant el modelatge de l'espai lliure i ocupat, i que té en compte la incertesa del sensor per millorar la selecció de les accions (presentat en la Part II). Aquesta tesi ha donat lloc a 14 publicacions, incloent 5 en revistes indexades, i els resultats obtinguts s'han fet servir en el projecte Europeu GARNICS. La funcionalitat del sistema complet està basada en els mètodes Next-Best-View (següent-millor-vista) i es pot desglossar en els següents passos principals. En primer lloc, s'obté una vista inicial de l'objecte (p. ex., una planta). A partir d'aquesta vista inicial i d'un conjunt de vistes candidates, s'estima, per cada una d'elles, el guany d'informació resultant, tant de moure la càmera com d'obtenir una nova mesura. És rellevant dir que aquest càlcul té en compte la incertesa de cada un dels píxels del sensor, l'estimació de la informació basada en el model de la tasca preestablerta i les possibles oclusions. Un cop seleccionada la vista més prometedora, el robot es mou a la nova posició, pren una nova imatge, integra aquesta informació en el model i torna a avaluar, un altre cop, el conjunt de punts de vista restants. Per últim, la tasca acaba en el moment que es recopila suficient informació.