Pasar al contenido principal

SISTEMA ROBOTICO INTERACTIVO PARA LA ASISTENCIA EN VESTIRSE

PCIN-2015-147

•
Nombre agencia financiadora Ministerio de Economía y Competitividad
Acrónimo agencia financiadora MINECO
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos Programación Conjunta Internacional
Año convocatoria 2015
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Unidad de gestión Subdirección General de Proyectos Internacionales
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)
Identificador persistente http://dx.doi.org/10.13039/501100003329

Resultados relacionados

Resultados totales (Incluyendo duplicados): 32
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Execution fault recovery in robot programming by demonstration using multiple models

  • Hoyos, Jose
  • Prieto, Flavio
  • Alenyà, Guillem
  • Torras, Carme
Deformable object (e.g., clothes) manipulation by a robot in interaction with a human being presents several interesting challenges. Due to texture and deformability, the object can get hooked in the human limbs. Moreover, the human can change their limbs position and curvature, which require changes in the paths to be followed by the robot. To help solve these problems, in this paper we propose a technique of learning by demonstration able to adapt to changes in position and curvature of the object (human limb) and recover from execution faults (hooks). The technique is tested using simulations, but with data obtained from a real robot., Este trabajo fue financiado por la Universidad Nacional de Colombia y la Universidad del Quindío, y parcialmente por los proyectos Robinstruct (MINECO TIN2014-58178-R) y I-DRESS (ERA-net CHISTERA PCIN-2015-147)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Real-time 3D reconstruction of non-rigid shapes with a single moving camera

  • Agudo Martínez, Antonio
  • Moreno-Noguer, Francesc
  • Calvo, Begoña
  • Montiel, J. M. M.
This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small maps., This work has been partially supported by the Spanish Ministry of Science and Innovation under projects RT-SLAM DPI2015-67275-P, RobInstruct TIN2014-58178-R and Keratocono DPI2014-54981-R; by the FP7-SME-2013 606634 PopCorn project from the European Union FP7; by the ERA-net CHISTERA project VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147; and by a scholarship FPU12/04886 awarded by the Spanish MECD., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Mode-shape interpretation: Re-thinking modal space for recovering deformable shapes

  • Agudo Martínez, Antonio
  • Montiel, J. M. M.
  • Calvo, Begoña
  • Moreno-Noguer, Francesc
Trabajo presentado al WACV 2016: IEEE Winter Conference on Applications of Computer Vision, celebrado en Lake Placid, NY(US) del 7 al 9 de marzo de 2016., This paper describes an on-line approach for estimating non-rigid shape and camera pose from monocular video sequences. We assume an initial estimate of the shape at rest to be given and represented by a triangulated mesh, which is encoded by a matrix of the distances between every pair of vertexes. By applying spectral analysis on this matrix, we are then able to compute a low-dimensional shape basis, that in contrast to standard approaches, has a very direct physical interpretation and requires a much smaller number of modes to span a large variety of deformations, either for inextensible or extensible configurations. Based on this low-rank model, we then sequentially retrieve both camera motion and non-rigid shape in each image, optimizing the model parameters with bundle adjustment over a sliding window of image frames. Since the number of these parameters is small, specially when considering physical priors, our approach may potentially achieve real-time performance. Experimental results on real videos for different scenarios demonstrate remarkable robustness to artifacts such as missing and noisy observations., This work has been partially supported by the Spanish MCI under projects RobInstruct TIN2014-58178-R, SVMap DIP2012-32168 and Keratocono DPI2014-54981-R; by the ERA-net CHISTERA projects VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147; and by a scholarship FPU12/04886 of the Spanish MECD., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A quantitative analysis of dressing dynamics for robotic dressing assistance

  • Chance, Greg
  • Jevtić, Aleksandar
  • Caleb-Solly, Praminda
  • Dogramadzi, Sanja
Assistive robots have a great potential to address issues related to an aging population and an increased demand for caregiving. Successful deployment of robots working in close proximity with people requires consideration of both safety and human–robot interaction (HRI). One of the established activities of daily living where robots could play an assistive role is dressing. Using the correct force profile for robot control will be essential in this application of HRI requiring careful exploration of factors related to the user’s pose and the type of garments involved. In this paper, a Baxter robot was used to dress a jacket onto a mannequin and human participants considering several combinations of user pose and clothing type (base layers), while recording dynamic data from the robot, a load cell, and an IMU. We also report on suitability of these sensors for identifying dressing errors, e.g., fabric snagging. Data were analyzed by comparing the overlap of confidence intervals to determine sensitivity to dressing. We expand the analysis to include classification techniques such as decision tree and support vector machines using k-fold cross-validation. The 6-axis load cell successfully discriminated between clothing types with predictive model accuracies between 72 and 97%. Used independently, the IMU and Baxter sensors were insufficient to discriminate garment types with the IMU showing 40–72% accuracy, but when used in combination this pair of sensors achieved an accuracy similar to the more expensive load cell (98%). When observing dressing errors (snagging), Baxter’s sensors and the IMU data demonstrated poor sensitivity but applying machine learning methods resulted in model with high predicative accuracy and low false negative rates (≤5%). The results show that the load cell could be used independently for this application with good accuracy but a combination of the lower cost sensors could also be used without a significant loss in precision, which will be a key element in the robot control architecture for safe HRI., This research was supported by EPSRC and EU CHIST-ERA, UK EPSRC funding Project Reference: EP/N021703/1. This work was partially funded by the programme “Acciones de Programación Conjunta Internacional 2015” by the Spanish Ministry of Economy, Industry and Competitiveness (project reference num. PCIN-2015-147)., Peer reviewed, Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

BASS: Boundary-aware superpixel segmentation

  • Rubio Romano, Antonio
  • Yu, LongLong
  • Simo-Serra, Edgar
  • Moreno-Noguer, Francesc
Trabajo presentado a la 23rd International Conference on Pattern Recognition, celebrada en Cancún (México) del 5 al 8 de diciembre de 2016., We propose a new superpixel algorithm based on exploiting the boundary information of an image, as objects in images can generally be described by their boundaries. Our proposed approach initially estimates the boundaries and uses them to place superpixel seeds in the areas in which they are more dense. Afterwards, we minimize an energy function in order to expand the seeds into full superpixels. In addition to standard terms such as color consistency and compactness, we propose using the geodesic distance which concentrates small superpixels in regions of the image with more information, while letting larger superpixels cover more homogeneous regions. By both improving the initialization using the boundaries and coherency of the superpixels with geodesic distances, we are able to maintain the coherency of the image structure with fewer superpixels than other approaches. We show the resulting algorithm to yield smaller Variation of Information metrics in seven different datasets while maintaining Undersegmentation Error values similar to the state-of-the-art methods., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147 and by the EU project AEROARMS H2020-ICT-2014-1-644271. A. Rubio is supported by the industrial doctorate grant 2015-DI-010 of the AGAUR., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A 3D descriptor to detect task-oriented grasping points in clothing

  • Ramisa, Arnau
  • Alenyà, Guillem
  • Moreno-Noguer, Francesc
  • Torras, Carme
Manipulating textile objects with a robot is a challenging task, especially because the garment perception is difficult due to the endless configurations it can adopt, coupled with a large variety of colors and designs. Most current approaches follow a multiple re-grasp strategy, in which clothes are sequentially grasped from different points until one of them yields a recognizable configuration. In this work we propose a method that combines 3D and appearance information to directly select a suitable grasping point for the task at hand, which in our case consists of hanging a shirt or a polo shirt from a hook. Our method follows a coarse-to-fine approach in which, first, the collar of the garment is detected and, next, a grasping point on the lapel is chosen using a novel 3D descriptor. In contrast to current 3D descriptors, ours can run in real time, even when it needs to be densely computed over the input image. Our central idea is to take advantage of the structured nature of range images that most depth sensors provide and, by exploiting integral imaging, achieve speed-ups of two orders of magnitude with respect to competing approaches, while maintaining performance. This makes it especially adequate for robotic applications as we thoroughly demonstrate in the experimental section., This research is partially funded by the Spanish Ministry of Economy and Competitiveness under project TIN2014-58178-R, by the CSIC project MANIPlus (201350E102), and by the ERA-Net CHISTERA projects ViSen PCIN-2013-047 and I-DRESS PCIN-2015-147. A. Ramisa worked under the JAE-DOC grant from CSIC and FSE. The authors are grateful to the Nvidia donation program for its support with GPU cards., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A Bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images

  • Moreno-Noguer, Francesc
  • Porta, Josep M.
In this paper we bring the tools of the Simultaneous Localization and Map Building (SLAM) problem from a rigid to a deformable domain and use them to simultaneously recover the 3D shape of non-rigid surfaces and the sequence of poses of a moving camera. Under the assumption that the surface shape may be represented as a weighted sum of deformation modes, we show that the problem of estimating the modal weights along with the camera poses, can be probabilistically formulated as a maximum a posteriori estimate and solved using an iterative least squares optimization. In addition, the probabilistic formulation we propose is very general and allows introducing different constraints without requiring any extra complexity. As a proof of concept, we show that local inextensibility constraints that prevent the surface from stretching can be easily integrated. An extensive evaluation on synthetic and real data, demonstrates that our method has several advantages over current non-rigid shape from motion approaches. In particular, we show that our solution is robust to large amounts of noise and outliers and that it does not need to track points over the whole sequence nor to use an initialization close from the ground truth., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under projects RobInstruct TIN2014-58178-R and RobCab DPI2014-57220-C2-2-P; and by the ERA-net CHISTERA project I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Visual grasp point localization, classification and state recognition in robotic manipulation of cloth: An overview

  • Jiménez Schlegl, Pablo
Cloth manipulation by robots is gaining popularity among researchers because of its relevance, mainly (but not only) in domestic and assistive robotics. The required science and technologies begin to be ripe for the challenges posed by the manipulation of soft materials, and many contributions have appeared in the last years. This survey provides a systematic review of existing techniques for the basic perceptual tasks of grasp point localization, state estimation and classification of cloth items, from the perspective of their manipulation by robots. This choice is grounded on the fact that any manipulative action requires to instruct the robot where to grasp, and most garment handling activities depend on the correct recognition of the type to which the particular cloth item belongs and its state. The high inter- and intraclass variability of garments, the continuous nature of the possible deformations of cloth and the evident difficulties in predicting their localization and extension on the garment piece are challenges that have encouraged the researchers to provide a plethora of methods to confront such problems, with some promising results. The present review constitutes for the first time an effort in furnishing a structured framework of these works, with the aim of helping future contributors to gain both insight and perspective on the subject., This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under projects RobInstruct (TIN2014-58178-R) and I-DRESS: Assistive interactive robotic system for support in dressing (PCIN-2015-147)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Recovering pose and 3D deformable shape from multi-instance image ensembles

  • Agudo Martínez, Antonio
  • Moreno-Noguer, Francesc
Trabajo presentado a la 13th Asian Conference on Computer Vision (ACCV), celebrada en Taipei (Taiwan) del 20 al 24 de noviembre de 2016., In recent years, there has been a growing interest on tackling the Non-Rigid Structure from Motion problem (NRSfM), where the shape of a deformable object and the pose of a moving camera are simultaneously estimated from a monocular video sequence. Existing solutions are limited to single objects and continuous, smoothly changing sequences. In this paper we extend NRSfM to a multi-instance domain, in which the images do not need to have temporal consistency, allowing for instance, to jointly reconstruct the face of multiple persons from an unordered list of images. For this purpose, we present a new formulation of the problem based on a dual low-rank shape representation, that simultaneously captures the between- and within-individual deformations. The parameters of this model are learned using a variant of the probabilistic linear discriminant analysis that requires consecutive batches of expectation and maximization steps. The resulting approach estimates 3D deformable shape and pose of multiple instances from only 2D point observations on a collection images, without requiring pre-trained 3D data, and is shown to be robust to noisy measurements and missing points. We provide quantitative and qualitative evaluation on both synthetic and real data, and show consistent benefits compared to current state of the art., This work has been partially supported by the Spanish Ministry of Science and Innovation under project RobInstruct TIN2014-58178-R; by the ERA-net CHISTERA projects VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

3D human pose tracking priors using geodesic mixture models

  • Simo-Serra, Edgar
  • Torras, Carme
  • Moreno-Noguer, Francesc
We present a novel approach for learning a finite mixture model on a Riemannian manifold in which Euclidean metrics are not applicable and one needs to resort to geodesic distances consistent with the manifold geometry. For this purpose, we draw inspiration on a variant of the expectation-maximization algorithm, that uses a minimum message length criterion to automatically estimate the optimal number of components from multivariate data lying on an Euclidean space. In order to use this approach on Riemannian manifolds, we propose a formulation in which each component is defined on a different tangent space, thus avoiding the problems associated with the loss of accuracy produced when linearizing the manifold with a single tangent space. Our approach can be applied to any type of manifold for which it is possible to estimate its tangent space. Additionally, we consider using shrinkage covariance estimation to improve the robustness of the method, especially when dealing with very sparsely distributed samples. We evaluate the approach on a number of situations, going from data clustering on manifolds to combining pose and kinematics of articulated bodies for 3D human pose tracking. In all cases, we demonstrate remarkable improvement compared to several chosen baselines., This work was partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R and by the ERA-net CHISTERA project I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Combining local-physical and global-statistical models for sequential deformable shape from motion

  • Agudo Martínez, Antonio
  • Moreno-Noguer, Francesc
In this paper, we simultaneously estimate camera pose and non-rigid 3D shape from a monocular video, using a sequential solution that combines local and global representations. We model the object as an ensemble of particles, each ruled by the linear equation of the Newton’s second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency. The resulting approach allows to sequentially estimate shape and camera poses, while progressively learning a global low-rank model of the shape that is fed back into the optimization scheme, introducing thus, global constraints. The overall combination of local (physical) and global (statistical) constraints yields a solution that is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, without requiring any training data at all. Validation is done in a variety of real application domains, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our on-line methodology yields significantly more accurate reconstructions than competing sequential approaches, being even comparable to the more computationally demanding batch methods., This work has been partially supported by the Spanish Ministry of Science and Innovation under project RobInstruct TIN2014-58178-R; by a scholarship FPU12/04886 from the Spanish MECD; and by the ERA-net CHISTERA projects VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

PL-SLAM: Real-time monocular visual SLAM with points and lines

  • Pumarola, Albert
  • Vakhitov, Alexander
  • Agudo Martínez, Antonio
  • Sanfeliu, Alberto
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE International Conference on Robotics and Automation (ICRA), celebrada en Singapur del 29 de mayo al 3 de junio de 2017., Low textured scenes are well known to be one of the main Achilles heels of geometric computer vision algorithms relying on point correspondences, and in particular for visual SLAM. Yet, there are many environments in which, despite being low textured, one can still reliably estimate line-based geometric primitives, for instance in city and indoor scenes, or in the so-called ``Manhattan worlds'', where structured edges are predominant. In this paper we propose a solution to handle these situations. Specifically, we build upon ORB-SLAM, presumably the current state-of-the-art solution both in terms of accuracy as efficiency, and extend its formulation to simultaneously handle both point and line correspondences. We propose a solution that can even work when most of the points are vanished out from the input images, and, interestingly it can be initialized from solely the detection of line correspondences in three consecutive frames. We thoroughly evaluate our approach and the new initialization strategy on the TUM RGB-D benchmark and demonstrate that the use of lines does not only improve the performance of the original ORB-SLAM solution in poorly textured frames, but also systematically improves it in sequence frames combining points and lines, without compromising the efficiency., This work has been partially supported by the EU project AEROARMS H2020-ICT-2014-1-644271, by the MINECO projects RobInstruct TIN2014-58178-R and Rob-Int-Coop DPI2013-42458-P, by the ERA-Net Chistera project IDRESS PCIN-2015-147 and by the Russian MES grant RFMEFI61516X0003., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Global model with local interpretation for dynamic shape reconstruction

  • Agudo Martínez, Antonio
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE Winter Conference on Applications of Computer Vision (WACV), celebrada en Santa Rosa, California (US) del 24 al 31 de marzo de 2017., The most standard approach to resolve the inherent ambiguities of the non-rigid structure from motion problem is using low-rank models that approximate deforming shapes by a linear combination of rigid basis. These models are typically global, i.e., each shape basis contributes equally to all points of the surface. While this approach has been shown effective to represent smooth deformations, its performance degrades for surfaces composed of various regions, each following a different deformation rule. Piecewise methods attempt to capture this type of behavior by locally modeling surface patches, although they subsequently require enforcing global constraints to assemble back the patches. In this paper we propose an approach that combines the best of global and local models: it locally considers low-rank models but, by construction, does not need to impose global constraints to guarantee local patch continuity. We achieve this by a simple expectation maximization strategy that besides learning global shape bases, it locally adapts their contribution to each specific surface region. Furthermore, as a side contribution, in order to split the surface into different local patches, we propose a novel physically-based mesh segmentation approach that obeys an energy criterion. The complete framework is evaluated in both synthetic and real datasets, and shows an improved performance to competing methods., This work has been partially supported by the Spanish Ministry of Science and Innovation under project RobInstruct TIN2014-58178-R, by the ERA-net CHISTERA project I-DRESS PCIN-2015-147 and by the EU project LOGIMATIC H2020-Galileo-2015-1-687534., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Demonstration-free contextualized probabilistic movement primitives, further enhanced with obstacle avoidance

  • Colomé, Adrià
  • Torras, Carme
Trabajo presentado a la IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), celebrada en Vancouver (Canada) del 24 al 28 de septiembre de 2017., Movement Primitives (MPs) have been widely used over the last years for learning robot motion tasks with direct Policy Search (PS) reinforcement learning. Among them, Probabilistic Movement Primitives (ProMPs) are a kind of MP based on a stochastic representation over sets of trajectories, which benefits from the properties of probability operations. However, the generation of such ProMPs requires a set of demonstrations to capture motion variability. Additionally, using context variables to modify trajectories coded as MPs is a popular approach nowadays in order to adapt motion to environmental variables. This paper proposes a contextual representation of ProMPs that allows for an easy adaptation to changing situations through context variables, by reparametrizing motion with them. Moreover, we propose a way of initializing contextual trajectories without the need of real robot demonstrations, by setting an initial position, a final position, and a number of trajectory interest points, where the contextual variables are evaluated. The parametrizations obtained show to be accurate while relieving the user from the need of performing costly computations such as conditioning. Additionally, using this contextual representation, we propose a simple yet effective quadratic optimization-based obstacle avoidance method for ProMPs. Experiments in simulation and on a real robot show the promise of the approach., This work is partially funded by CSIC projects MANIPlus (201350E102) and TextilRob (201550E028), and by the European project I-DRESS (PCIN2015-147)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

3D human pose estimation from a single image via distance matrix regression

  • Moreno-Noguer, Francesc
Trabajo presentado a la 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), celebrada en Honolulu, Hawaii (US) del 21 al 26 de julio de 2016., This paper addresses the problem of 3D human pose estimation from a single image. We follow a standard two-step pipeline by first detecting the 2D position of the N body joints, and then using these observations to infer 3D pose. For the first step, we use a recent CNN-based detector. For the second step, most existing approaches perform 2N-to-3N regression of the Cartesian joint coordinates. We show that more precise pose estimates can be obtained by representing both the 2D and 3D human poses using $Ntimes N$ distance matrices, and formulating the problem as a 2D-to-3D distance matrix regression. For learning such a regressor we leverage on simple Neural Network architectures, which by construction, enforce positivity and symmetry of the predicted matrices. The approach has also the advantage to naturally handle missing observations and allowing to hypothesize the position of non-observed joints. Quantitative results on Humaneva and Human3.6M datasets demonstrate consistent performance gains over state-of-the-art. Qualitative evaluation on the images in-the-wild of the LSP dataset, using the regressor learned on Human3.6M, reveals very promising generalization results., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R and by the ERANet Chistera project I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

DUST: Dual union of spatio-temporal subspaces for monocular multiple object 3D reconstruction

  • Agudo Martínez, Antonio
  • Moreno-Noguer, Francesc
Trabajo presentado a la 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), celebrada en Honolulu, Hawaii (US) del 21 al 26 de julio de 2016., We present an approach to reconstruct the 3D shape of multiple deforming objects from incomplete 2D trajectories acquired by a single camera. Additionally, we simultaneously provide spatial segmentation (i.e., we identify each of the objects in every frame) and temporal clustering (i.e., we split the sequence into primitive actions). This advances existing work, which only tackled the problem for one single object and non-occluded tracks. In order to handle several objects at a time from partial observations, we model point trajectories as a union of spatial and temporal subspaces, and optimize the parameters of both modalities, the non-observed point tracks and the 3D shape via augmented Lagrange multipliers. The algorithm is fully unsupervised and results in a formulation which does not need initialization. We thoroughly validate the method on challenging scenarios with several human subjects performing different activities which involve complex motions and close interaction. We show our approach achieves state-of-the-art 3D reconstruction results, while it also provides spatial and temporal segmentation., This work has been partially supported by the Spanish Ministry of Science and Innovation under project RobInstruct TIN2014-58178-R, by the ERA-net CHISTERA project I-DRESS PCIN-2015-147 and by the EU project LOGIMATIC H2020-Galileo-2015-1-687534., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Multi-modal joint embedding for fashion product retrieval

  • Rubio Romano, Antonio
  • Yu, LongLong
  • Simo-Serra, Edgar
  • Moreno-Noguer, Francesc
Trabajo presentado a la IEEE International Conference on Image Processing (ICIP), celebrada en Beijing (China) del 17 al 20 de septiembre de 2017., Finding a product in the fashion world can be a daunting task. Everyday, e-commerce sites are updating with thousands of images and their associated metadata (textual information), deepening the problem, akin to finding a needle in a haystack. In this paper, we leverage both the images and textual meta-data and propose a joint multi-modal embedding that maps both the text and images into a common latent space. Distances in the latent space correspond to similarity between products, allowing us to effectively perform retrieval in this latent space, which is both efficient and accurate. We train this embedding using large-scale real world e-commerce data by both minimizing the similarity between related products and using auxiliary classification networks to that encourage the embedding to have semantic meaning. We compare against existing approaches and show significant improvements in retrieval tasks on a large-scale e-commerce dataset. We also provide an analysis of the different metadata., This work is partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R, by the ERA-Net Chistera project I-DRESS PCIN-2015-147 and by the EU project iAEROARMS H2020-ICT-2014-1-644271. A.Rubio is supported by the industrial doctorate grant 2015-DI-010 of the AGAUR., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Modal space: A physics-based model for sequential estimation of time-varying shape from monocular video

  • Agudo Martínez, Antonio
  • Montiel, J. M. M.
  • Agapito, Lourdes
  • Calvo, Begoña
This paper describes two sequential methods for recovering the camera pose together with the 3D shape of highly deformable surfaces from a monocular video. The nonrigid 3D shape is modeled as a linear combination of mode shapes with time-varying weights that define the shape at each frame and are estimated on-the-fly. The low-rank constraint is combined with standard smoothness priors to optimize the model parameters over a sliding window of image frames. We propose to obtain a physics-based shape basis using the initial frames on the video to code the time-varying shape along the sequence, reducing the problem from trilinear to bilinear. To this end, the 3D shape is discretized by means of a soup of elastic triangular finite elements where we apply a force balance equation. This equation is solved using modal analysis via a simple eigenvalue problem to obtain a shape basis that encodes the modes of deformation. Even though this strategy can be applied in a wide variety of scenarios, when the observations are denser, the solution can become prohibitive in terms of computational load. We avoid this limitation by proposing two efficient coarse-to-fine approaches that allow us to easily deal with dense 3D surfaces. This results in a scalable solution that estimates a small number of parameters per frame and could potentially run in real time. We show results on both synthetic and real videos with ground truth 3D data, while robustly dealing with artifacts such as noise and missing data., This work was partly supported by the MINECO projects RT-SLAM DPI2015-67275-P, RobInstruct TIN2014-58178-R and Keratocono DPI2014-54981-R; by the SecondHands project funded by the EU Horizon 2020 Research and Innovation programme under grant agreement No 643950; by the ERA-net CHISTERA projects VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147; and by a scholarship FPU12/04886 from the Spanish MECD., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Active garment recognition and target grasping point detection using deep learning

  • Corona Puyane, Enric
  • Alenyà, Guillem
  • Gabás Nova, Antoni
  • Torras, Carme
Identification and bi-manual handling of deformable objects, like textiles, is one of the most challenging tasks in the field of industrial and service robotics. Their unpredictable shape and pose makes it very difficult to identify the type of garment and locate the most relevant parts that can be used for grasping. In this paper, we propose an algorithm that first, identifies the type of garment and second, performs a search of the two grasping points that allow a robot to bring the garment to a known pose. We show that using an active search strategy it is possible to grasp a garment directly from predefined grasping points, as opposed to the usual approach based on multiple re-graspings of the lowest hanging parts. Our approach uses a hierarchy of three Convolutional Neural Networks (CNNs) with different levels of specialization, trained both with synthetic and real images. The results obtained in the three steps (recognition, first grasping point, second grasping point) are promising. Experiments with real robots show that most of the errors are due to unsuccessful grasps and not to the localization of the grasping points, thus a more robust grasping strategy is required., This work was partially supported by the EU CHIST-ERA I-DRESS project PCIN-2015-147, by the Spanish Ministry of Economy and Competitiveness under project Robinstruct TIN2014-58178-R, and by the CSIC project TextilRob 201550E028., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Robot motion adaptation through user intervention and reinforcement learning

  • Jevtić, Aleksandar
  • Colomé, Adrià
  • Alenyà, Guillem
  • Torras, Carme
Assistant robots are designed to perform specific tasks for the user, but their performance is rarely optimal, hence they are required to adapt to user preferences or new task requirements. In the previous work, the potential of an interactive learning framework based on user intervention and reinforcement learning (RL) was assessed. The framework allowed the user to correct an unfitted segment of the robot trajectory by using hand movements to guide the robot along a corrective path. So far, only the usability of the framework was evaluated through experiments with users. In the current work, the framework is described in detail and its ability to learn from a set of sample trajectories using an RL algorithm is analyzed. To evaluate the learning performance, three versions of the framework are proposed that differ in the method used to obtain the sample trajectories, which are: human-guided learning, autonomous learning, and combined human-guided with autonomous learning. The results show that the combination of the human-guided and autonomous learning achieved the best performance, and although it needed a higher number of sample trajectories than the human-guided learning, it required less user involvement. Autonomous learning alone obtained the lowest reward value and needed the highest number of sample trajectories., This work was performed as a part of the I-DRESS project (EU CHIST-ERA 2014) and funded by the programme “Acciones de Programación Conjunta Internacional 2015” of the Spanish Ministry of Economy, Industry and Competitiveness (ref. num. PCIN-2015-147), and partially supported by the Spanish national project RobInstruct (ref. num. TIN2014-58178-R)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

BreakingNews: Article annotation by image and text processing

  • Ramisa, Arnau
  • Yan, Fei
  • Moreno-Noguer, Francesc
  • Mikolajczyk, Krystian
Building upon recent Deep Neural Network architectures, current approaches lying in the intersection of Computer Vision and Natural Language Processing have achieved unprecedented breakthroughs in tasks like automatic captioning or image retrieval. Most of these learning methods, though, rely on large training sets of images associated with human annotations that specifically describe the visual content. In this paper we propose to go a step further and explore the more complex cases where textual descriptions are loosely related to the images. We focus on the particular domain of news articles in which the textual content often expresses connotative and ambiguous relations that are only suggested but not directly inferred from images. We introduce an adaptive CNN architecture that shares most of the structure for multiple tasks including source detection, article illustration and geolocation of articles. Deep Canonical Correlation Analysis is deployed for article illustration, and a new loss function based on Great Circle Distance is proposed for geolocation. Furthermore, we present BreakingNews, a novel dataset with approximately 100K news articles including images, text and captions, and enriched with heterogeneous meta-data (such as GPS coordinates and user comments). We show this dataset to be appropriate to explore all aforementioned problems, for which we provide a baseline performance using various Deep Learning architectures, and different representations of the textual and visual features. We report very promising results and bring to light several limitations of current state-of-the-art in this kind of domain, which we hope will help spur progress in the field., This work has been partially funded by Spanish Ministry of Economy and Competitiveness under project RobInstruct TIN2014-58178-R, by the ERA-net CHISTERA projects VISEN PCIN2013-047 and I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Robot-aided cloth classification using depth information and CNNs

  • Gabás Nova, Antoni
  • Corona Puyane, Enric
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado a la 9th International Conference on Articulated Motion and Deformable Objects (AMDO), celebrada en Palma de Mallorca (España) del 13 al 15 de julio de 2016., We present a system to deal with the problem of classifying garments from a pile of clothes. This system uses a robot arm to extract a garment and show it to a depth camera. Using only depth images of a partial view of the garment as input, a deep convolutional neural network has been trained to classify different types of garments. The robot can rotate the garment along the vertical axis in order to provide different views of the garment to enlarge the prediction confidence and avoid confusions. In addition to obtaining very high classification scores, compared to previous approaches to cloth classification that match the sensed data against a database, our system provides a fast and occlusion-robust solution to the problem., This work was partially supported by the EU CHISTERA I-DRESS project PCIN-2015-147, by the Spanish Ministry of Economy and Competitiveness under project Robinstruct TIN2014-58178-R, and by the CSIC project TextilRob 201550E028., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Structured prediction with output embeddings for semantic image annotation

  • Quattoni, Ariadna
  • Ramisa, Arnau
  • Swaroop Madhyastha, Pranava
  • Simo-Serra, Edgar
  • Moreno-Noguer, Francesc
Trabajo presentado a la conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL-HLT), celebrada en San Diego, California (US) del 12 al 17 de junio de 2016., We address the task of annotating images with semantic tuples. Solving this problem requires an algorithm able to deal with hundreds of classes for each argument of the tuple. In such contexts, data sparsity becomes a key challenge. We propose handling this sparsity by incorporating feature representations of both the inputs (images) and outputs (argument classes) into a factorized log-linear model., This work was partly funded by the Spanish MINECO project RobInstruct TIN2014-58178-R and by the ERA-net CHISTERA projects VISEN PCIN- 2013-047 and I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Accurate and linear time pose estimation from points and lines

  • Vakhitov, Alexander
  • Funke, Jan
  • Moreno-Noguer, Francesc
Póster presentado a la 14th European Conference on Computer Vision (ECCV), celebrada en Amsterdam (Netherlands) del 11 al 14 de octubre de 2016., The Perspective-n-Point (PnP) problem seeks to estimate the pose of a calibrated camera from n 3D-to-2D point correspondences. There are situations, though, where PnP solutions are prone to fail because feature point correspondences cannot be reliably estimated (e.g. scenes with repetitive patterns or with low texture). In such scenarios, one can still exploit alternative geometric entities, such as lines, yielding the so-called Perspective-n-Line (PnL) algorithms. Unfortunately, existing PnL solutions are not as accurate and efficient as their point-based counterparts. In this paper we propose a novel approach to introduce 3D-to-2D line correspondences into a PnP formulation, allowing to simultaneously process points and lines. For this purpose we introduce an algebraic line error that can be formulated as linear constraints on the line endpoints, even when these are not directly observable. These constraints can then be naturally integrated within the linear formulations of two state-of-the-art point-based algorithms, the OPnP and the EPnP, allowing them to indistinctly handle points, lines, or a combination of them. Exhaustive experiments show that the proposed formulation brings remarkable boost in performance compared to only point or only line based solutions, with a negligible computational overhead compared to the original OPnP and EPnP., This work is partly funded by the Russian MES grant RFMEFI61516X0003; by the Spanish MINECO project RobInstruct TIN2014-58178-R and by the ERA-Net Chistera project I-DRESS PCIN-2015-147., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

User evaluation of an interactive learning framework for single-arm and dual-arm robots

  • Jevtić, Aleksandar
  • Colomé, Adrià
  • Torras, Carme
Trabajo presentado a la International Conference on Social Robotics (ICSR), celebrada en Kansas (USA) del 1 al 3 de noviembre de 2016., Social robots are expected to adapt to their users and, like their human counterparts, learn from the interaction. In our previous work, we proposed an interactive learning framework that enables a user to intervene and modify a segment of the robot arm trajectory. The framework uses gesture teleoperation and reinforcement learning to learn new motions. In the current work, we compared the user experience with the proposed framework implemented on the single-arm and dual-arm Barrett¿s 7-DOF WAM robots equipped with a Microsoft Kinect camera for user tracking and gesture recognition. User performance and workload were measured in a series of trials with two groups of 6 participants using two robot settings in different order for counterbalancing. The experimental results showed that, for the same task, users required less time and produced shorter robot trajectories with the single-arm robot than with the dual-arm robot. The results also showed that the users who performed the task with the single-arm robot first experienced considerably less workload in performing the task with the dual-arm robot while achieving a higher task success rate in a shorter time., This work was supported by the Beatriu de Pinós fellowship, reference num.: 2013 BP-B 00239, jointly funded by the Government of Catalunya, Spain and the European Commission FP7 COFUND programme. The work was partially supported by the EU CHIST-ERA I-DRESS project, reference num. PCIN-2015-147, and the national Spanish project RobInstruct, reference num. TIN2014-58178-R., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Personalization framework for adaptive robotic feeding assistance

  • Canal, Gerard
  • Alenyà, Guillem
  • Alenyà, Guillem
  • Torras, Carme
Trabajo presentado a la International Conference on Social Robotics (ICSR), celebrada en Kansas (USA) del 1 al 3 de noviembre de 2016., The deployment of robots at home must involve robots with pre-defined skills and the capability of personalizing their behavior by non-expert users. A framework to tackle this personalization is presented and applied to an automatic feeding task. The personalization involves the caregiver providing several examples of feeding using Learning-by-Demostration, and a ProMP formalism to compute an overall trajectory and the variance along the path. Experiments show the validity of the approach in generating different feeding motions to adapt to user¿s preferences, automatically extracting the relevant task parameters. The importance of the nature of the demonstrations is also assessed, and two training strategies are compared., This work has been supported by the MINECO project RobInstruct TIN2014-58178-R and the ERA-Net CHIST-ERA project I-DRESS PCIN-2015-147. Gerard Canal is also supported by the Ministry of Economy and Knowledge of the Government of Catalonia via a FI-DGR 2016 fellowship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Tesis doctoral (DoctoralThesis)

Bimanual robot skills: MP encoding, dimensionality reduction and reinforcement learning

  • Colomé, Adrià
A thesis submitted to the Universitat Politècnica de Catalunya to obtain the degree of Doctor of Philosophy. Doctoral programme: Automatic Control, Robotics and Computer Vision. This thesis was completed at: Institut de Robòtica i Informàtica Industrial, CSIC-UPC., [EN]: In our culture, robots have been in novels and cinema for a long time, but it has been specially in the last two decades when the improvements in hardware - better computational power and components - and advances in Artificial Intelligence (AI), have allowed robots to start sharing spaces with humans. Such situations require, aside from ethical considerations, robots to be able to move with both compliance and precision, and learn at different levels, such as perception, planning, and motion, being the latter the focus of this work. The first issue addressed in this thesis is inverse kinematics for redundant robot manipulators, i.e: positioning the robot joints so as to reach a certain end-effector pose. We opt for iterative solutions based on the inversion of the kinematic Jacobian of a robot, and propose to filter and limit the gains in the spectral domain, while also unifying such approach with a continuous, multipriority scheme. Such inverse kinematics method is then used to derive manipulability in the whole workspace of an antropomorphic arm, and the coordination of two arms is subsequently optimized by finding their best relative positioning. Having solved the kinematic issues, a robot learning within a human environment needs to move compliantly, with limited amount of force, in order not to harm any humans or cause any damage, while being as precise as possible. Therefore, we developed two dynamic models for the same redundant arm we had analysed kinematically: The first based on local models with Gaussian projections, and the second characterizing the most problematic term of the dynamics, namely friction. Such models allowed us to implement feed-forward controllers, where we can actively change the weights in the compliance-precision tradeoff. Moreover, we used such models to predict external forces acting on the robot, without the use of force sensors. Afterwards, we noticed that bimanual robots must coordinate their components (or limbs) and be able to adapt to new situations with ease. Over the last decade, a number of successful applications for learning robot motion tasks have been published. However, due to the complexity of a complete system including all the required elements, most of these applications involve only simple robots with a large number of high-end technology sensors, or consist of very simple and controlled tasks. Using our previous framework for kinematics and control, we relied on two types of movement primitives to encapsulate robot motion. Such movement primitives are very suitable for using reinforcement learning. In particular, we used direct policy search, which uses the motion parametrization as the policy itself. In order to improve the learning speed in real robot applications, we generalized a policy search algorithm to give some importance to samples yielding a bad result, and we paid special attention to the dimensionality of the motion parametrization. We reduced such dimensionality with linear methods, using the rewards obtained through motion repetition and execution. We tested such framework in a bimanual task performed by two antropomorphic arms, such as the folding of garments, showing how a reduced dimensionality can provide qualitative information about robot couplings and help to speed up the learning of tasks when robot motion executions are costly., [CA]: A la nostra cultura, els robots han estat presents en novel·les i cinema des de fa dècades, però ha sigut especialment en les últimes dues quan les millores en hardware (millors capacitats de còmput) i els avenços en intel·ligència artificial han permès que els robots comencin a compartir espais amb els humans. Aquestes situacions requereixen, a banda de consideracions ètiques, que els robots siguin capaços de moure's tant amb suavitat com amb precisió, i d'aprendre a diferents nivells, com són la percepció, planificació i moviment, essent l'última el centre d'atenció d'aquest treball. El primer problema adreçat en aquesta tesi és la cinemàtica inversa, i.e.: posicionar les articulacions del robot de manera que l'efector final estigui en una certa posició i orientació. Hem estudiat el camp de les solucions iteratives, basades en la inversió del Jacobià cinemàtic d'un robot, i proposem un filtre que limita els guanys en el seu domini espectral, mentre també unifiquem tal mètode dins un esquema multi-prioritat i continu. Aquest mètode per a la cinemàtica inversa és usat a l'hora d'encapsular tota la informació sobre l'espai de treball d'un braç antropomòrfic, i les capacitats de coordinació entre dos braços són optimitzades, tot trobant la seva millor posició relativa en l'espai. Havent resolt les dificultats cinemàtiques, un robot que aprèn en un entorn humà necessita moure's amb suavitat exercint unes forces limitades per tal de no causar danys, mentre es mou amb la màxima precisió possible. Per tant, hem desenvolupat dos models dinàmics per al mateix braç robòtic redundant que havíem analitzat des del punt de vista cinemàtic: El primer basat en models locals amb projeccions de Gaussianes i el segon, caracteritzant el terme més problemàtic i difícil de representar de la dinàmica, la fricció. Aquests models ens van permetre utilitzar controladors coneguts com "feed-forward", on podem canviar activament els guanys buscant l'equilibri precisió-suavitat que més convingui. A més, hem usat aquests models per a inferir les forces externes actuant en el robot, sense la necessitat de sensors de força. Més endavant, ens hem adonat que els robots bimanuals han de coordinar els seus components (braços) i ser capaços d'adaptar-se a noves situacions amb facilitat. Al llarg de l'última dècada, diverses aplicacions per aprendre tasques motores robòtiques amb èxit han estat publicades. No obstant, degut a la complexitat d'un sistema complet que inclogui tots els elements necessaris, la majoria d'aquestes aplicacions consisteixen en robots més aviat simples amb costosos sensors d'última generació, o a resoldre tasques senzilles en un entorn molt controlat. Utilitzant el nostre treball en cinemàtica i control, ens hem basat en dos tipus de primitives de moviment per caracteritzar la motricitat robòtica. Aquestes primitives de moviment són molt adequades per usar aprenentatge per reforç. En particular, hem usat la búsqueda directa de la política, un camp de l'aprenentatge per reforç que usa la parametrització del moviment com la pròpia política. Per tal de millorar la velocitat d'aprenentatge en aplicacions amb robots reals, hem generalitzat un algoritme de búsqueda directa de política per a donar importància a les mostres amb mal resultat, i hem donat especial atenció a la reducció de dimensionalitat en la parametrització dels moviments. Hem reduït la dimensionalitat amb mètodes lineals, utilitzant les recompenses obtingudes EN executar els moviments. Aquests mètodes han estat provats en tasques bimanuals com són plegar roba, usant dos braços antropomòrfics. Els resultats mostren com la reducció de dimensionalitat pot aportar informació qualitativa d'una tasca, i al mateix temps ajuda a aprendre-la més ràpid quan les execucions amb robots reals són costoses., This work has been supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant AP2010-1989. This thesis has also been partially supported by the research projects: FP7-ICT-2009-6-269959 IntellAct: Intelligent observation and execution of Actions and manipulations (European Project). 201350E102 MANIPlus: Manipulación robotizada de objetos deformables (CSIC Project). 2009 SGR 155 : SGR ROBÒTICA: Grup de recerca consolidat - Grup de Robòtica (Catalan government). TIN2014-58178-R RobInstruct: Instructing robots using natural communication skills (Spanish national project). PCIN-2015-147 I-DRESS: Assistive interactive robotic system for support in dressing (European Project)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Tesis de maestría (MasterThesis)

Speech and motion recognition for a robot assistant in dressing

  • Flores Valle, Andrés
Master in Artificial Intelligence., Robot assistants are expected to help users with their everyday tasks, for which they must have the ability of recognizing users’ attention and intentions. This project will focus on attention and intention recognition from motion and speech, as well as development of a unified interaction framework that can efficiently combine these two modalities. The proposed framework will be tested in the assistive dressing scenario, i.e. helping the user to put on and take off a piece of clothing. [Methodology]: The project will start with an evaluation of existing tools for user motion tracking (Kinect, etc.) and speech recognition (e.g. ROS-based tools), and also with the study of relevant literature about multi-modal interaction. The expected outcomes of the project are: - Development of an algorithm for user following associated with the assistive dressing task. (User following requires both user motion tracking and robot motion planning.) - Definition of a speech vocabulary relevant to the assistive dressing task, which will allow the user to interrupt the robot and suggest a corrected action. - Development of an algorithm that combines two modalities and suggests robots actions based on the recognized user intention. [Platform]: The algorithms will be implemented on two Barret's WAM robotic arms and external depth cameras (Kinect 1 and Kinect 2)., The work is under the scope of the following projects: I-DRESS: Assistive interactive robotic system for support in dressing., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Joining high-level symbolic planning with low-level motion primitives in adaptive HRI: application to dressing assistance

  • Canal, Gerard
  • Pignat, Emmanuel
  • Alenyà, Guillem
  • Calinon, Sylvain
  • Torras, Carme
Trabajo presentado en la IEEE International Conference on Robotics and Automation (ICRA), celebrado en Brisbane (Australia), del 21 al 25 de mayo de 2018, For a safe and successful daily living assistance, far from the highly controlled environment of a factory, robots should be able to adapt to ever-changing situations. Programming such a robot is a tedious process that requires expert knowledge. An alternative is to rely on a high-level planner, but the generic symbolic representations used are not well suited to particular robot executions. Contrarily, motion primitives encode robot motions in a way that can be easily adapted to different situations. This paper presents a combined framework that exploits the advantages of both approaches. The number of required symbolic states is reduced, as motion primitives provide “smart actions” that take the current state and cope online with variations. Symbolic actions can include interactions (e.g., ask and inform) that are difficult to demonstrate. We show that the proposed framework can adapt to the user preferences (in terms of robot speed and robot verbosity), can readjust the trajectories based on the user movements, and can handle unforeseen situations. Experiments are performed in a shoe-dressing scenario. This scenario is particularly interesting because it involves a sufficient number of actions, and the human-robot interaction requires the handling of user preferences and unexpected reactions., This work has been supported by the ERA-Net CHIST-ERA project IDRESS (Spain’s Ministry of Economy and Competitiveness PCIN-2015- 147, Swiss National Science Foundation 20CH21-16085). It is also supported by the Spanish State Research Agency through the Mara de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Gerard Canal is also supported by the Spanish Ministry of Education, Culture and Sport via a FPU doctoral grant FPU15/00504., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Dimensionality reduction for dynamic movement primitives and application to bimanual manipulation of clothes

  • Colomé, Adrià
  • Torras, Carme
Dynamic movement primitives (DMPs) are widely used as movement parametrization for learning robot trajectories, because of their linearity in the parameters, rescaling robustness, and continuity. However, when learning a movement with DMPs, a very large number of Gaussian approximations needs to be performed. Adding them up for all joints yields too many parameters to be explored when using reinforcement learning (RL), thus requiring a prohibitive number of experiments/simulations to converge to a solution with a (locally or globally) optimal reward. In this paper, we address the process of simultaneously learning a DMP-characterized robot motion and its underlying joint couplings through linear dimensionality reduction (DR), which will provide valuable qualitative information leading to a reduced and intuitive algebraic description of such motion. The results in the experimental section not only show that we can effectively perform DR on DMPs while learning, but we can also obtain better learning curves, as well as additional information about each motion: linear mappings relating joint values and some latent variables., This work was partially developed in the context of the Advanced Grant CLOTHILDE (”CLOTH manIpulation Learning from DEmonstrations”), which has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 741930). This work is also partially funded by CSIC projects MANIPlus (201350E102) and TextilRob (201550E028), and Chist-Era Project I-DRESS (PCIN-2015-147)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Adapting robot task planning to user preferences: an assistive shoe dressing example

  • Canal, Gerard
  • Alenyà, Guillem
  • Torras, Carme
Healthcare robots will be the next big advance in humans’ domestic welfare, with robots able to assist elderly people and users with disabilities. However, each user has his/her own preferences, needs and abilities. Therefore, robotic assistants will need to adapt to them, behaving accordingly. Towards this goal, we propose a method to perform behavior adaptation to the user preferences, using symbolic task planning. A user model is built from the user’s answers to simple questions with a fuzzy inference system, and it is then integrated into the planning domain. We describe an adaptation method based on both the user satisfaction and the execution outcome, depending on which penalizations are applied to the planner’s rules. We demonstrate the application of the adaptation method in a simple shoe-fitting scenario, with experiments performed in a simulated user environment. The results show quick behavior adaptation, even when the user behavior changes, as well as robustness to wrong inference of the initial user model. Finally, some insights in a non-simulated world shoe-fitting setup are also provided., This work has been partially supported by the CHIST-ERA Project I-DRESS PCIN-2015-147, by the Spanish Ministry of Science and Innovation HuMoUR TIN2017-90086-R and the AEI through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Gerard Canal is also supported by the Spanish Ministry of Education, Culture and Sport via a doctoral grant FPU15/00504.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Tesis doctoral (DoctoralThesis)

Adapting robot behavior to user preferences in assistive scenarios

  • Canal, Gerard
Tesis llevada a cabo para conseguir el grado de Doctor por la Universidad Politécnica de Cataluña.--2020-03-09, Robotic assistants have inspired numerous books and science fiction movies. In the real world, these kinds of devices are a growing need amongst the elderly, who will continue requiring more assistance. While life expectancy is increasing, life quality is not necessarily doing so. Thus, we may find ourselves and our loved ones being dependent and needing another person to perform the most basic tasks, which has a strong psychological impact. Accordingly, assistive robots may be the definitive tool to give more quality of life by empowering dependent people and extending their independent living. Assisting users to perform daily activities requires adapting to them and their needs, as they might not be able to adapt to the robot. This thesis tackles adaptation and personalization issues through user preferences. We focus on physical tasks that involve close contact, as these present interesting challenges, and are of great importance for the user. Therefore, three tasks are mainly used throughout the thesis: assistive feeding, shoe fitting, and jacket dressing. We first describe a framework for robot behavior adaptation that illustrates how robots should be personalized for and by end-users or their assistants. Using this framework, non-technical users determine how the robot should behave. Then, we define the concept of preference for assistive robotics scenarios and establish a taxonomy, which includes hierarchies and groups of preferences, grounding definitions and concepts. We then show how the preferences in the taxonomy are used with AI planning systems to adapt the robot behavior to the preferences of the user obtained from simple questions. Our algorithms allow for long-term adaptations as well as to cope with misinferred user models. We further integrate the methods with low-level motion primitives that provide a more robust adaptation and behavior while lowering the number of needed actions and demonstrations. Moreover, we perform a deeper analysis of planning and preferences with the introduction of new algorithms to provide preference suggestions in planning domains. The thesis then concludes with a user study that evaluates the use of the preferences in the three real assistive robotics scenarios. The experiments show a clear understanding of the preferences of users, who were able to assess the impact of their preferences on the behavior of the robot. In summary, we provide tools and algorithms to design the robotic assistants of the future. Assistants that should be able to adapt to the assisted user needs and preferences, just as human assistants do nowadays., This work has been partially supported by the following doctoral grants: 2016FI_B 00660 FI-DGR (Ministry of Economy and Knowledge of the Government of Catalonia). FPU15/00504 FPU: Formación de Profesorado Universitario (Spanish Ministry of Education, Culture, and Sport). EST17/00371 Estancias Breves (FPU): Mobility grant (Spanish Ministry of Education, Culture, and Sport). This thesis has also been partially supported by the research projects: PCIN-2015-147 I-DRESS: Assistive interactive robotic system for support in dressing (ERA-Net CHIST-ERA project). 201350E102 MANIPlus: Manipulación robotizada de objetos deformables (Consejo Superior de Investigaciones Científicas - CSIC). H2020-ICT-2016-1-731761 IMAGINE: Robots understanding their actions by imagining their effects (European Commission). ERC-2016-ADG-741930 CLOTHILDE: Cloth manipulation learning from demonstration (European Research Council).