Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 36
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Outdoor view recognition based on landmark grouping and logistic regression

  • Todt, Eduardo
  • Torras, Carme
Vision-based robot localization outdoors has remained more elusive than its indoors counterpart. Drastic illumination changes and the scarceness of suitable landmarks are the main difficulties. This paper attempts to surmount them by deviating from the main trend of using local features. Instead, a global descriptor called landmark-view is defined, which aggregates the most visually-salient landmarks present in each scene. Thus, landmark co-occurrence and spatial and saliency relationships between them are added to the single landmark characterization, based on saliency and color distribution. A suitable framework to compare landmark-views is developed, and it is shown how this remarkably enhances the recognition performance, compared against single landmark recognition. A view-matching model is constructed using logistic regression. Experimentation using 45 views, acquired outdoors, containing 273 landmarks, yielded good recognition results. The overall percentage of correct view classification obtained was 80.6%, indicating the adequacy of the approach. © 2013 World Scientific Publishing Company., This work was partially funded by the GARNICS (Gardening with a Cognitive System) project FP7-ICT-247947, Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Robotized plant probing: Leaf segmentation utilizing time-of-flight data

  • Alenyà, Guillem
  • Dellen, Babette
  • Foix, Sergi
  • Torras, Carme
Supervision of long-lasting extensive botanic experiments is a promising robotic application that some recent technological advances have made feasible. Plant modeling for this application has strong demands, particularly in what concerns three-dimensional (3-D) information gathering and speed. © 2013 IEEE., This research is partially funded by the EU GARNICS project FP7-247947, the project PAU+ (DPI2011-27510), and the Grup consolidat SGR155. B.Dellen acknowledges support from the Spanish Ministry of Science and Innovation through a Ramon y Cajal program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

General robot kinematics decomposition without intermediate markers

  • Ulbrich, Stefan
  • Ruiz de Angulo, Vicente
  • Asfour, Tamim
  • Torras, Carme
  • Dillman, Rudiger
The calibration of serial manipulators with high numbers of degrees of freedom by means of machine learning is a complex and time-consuming task. With the help of a simple strategy, this complexity can be drastically reduced and the speed of the learning procedure can be increased: When the robot is virtually divided into shorter kinematic chains, these subchains can be learned separately and, hence, much more efficiently than the complete kinematics. Such decompositions, however, require either the possibility to capture the poses of all end- effectors of all subchains at the same time, or they are limited to robots that fulfill special constraints. In this work, an alternative decomposition is presented that does not suffer from these limitations. An offline training algorithm is provided in which the composite subchains are learned sequentially with dedicated movements. A second training scheme is provided to train composite chains simultaneously and online. Both schemes can be used together with many machine learning algorithms. In the simulations, an algorithm using Parameterized Self-Organizing Maps (PSOM) modified for online learning and Gaussian Mixture Models (GMM) were chosen to show the correctness of the approach. The experimental results show that, using a two-fold decomposition, the number of samples required to reach a given precision is reduced to twice the square root of the original number., The work described in this paper was partially conducted within the EU Cognitive Systems projects Xperience (FP-7-270273) and GARNICS (FP-7-247947) funded by the European Commission, and the Generalitat de Catalunya through the Robotics group (SGR2009-00155). V. Ruiz de Angulo acknowledges support from Spanish Ministry of Science and Education, under the project DPI2010-18449. C. Torras acknowledges support from the Consolider project MIPRCV (CSD2007-00018)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Learning the semantics of object-action relations by observation

  • Aksoy, Eren Erdal
  • Abramov, Alexey
  • Dörr, Johannes
  • Ning, Kejun
  • Dellen, Babette
  • Wörgötter, Florentin
Recognizing manipulations performed by a human and the transfer and execution of this by a robot is a difficult problem. We address this in the current study by introducing a novel representation of the relations between objects at decisive time points during a manipulation. Thereby, we encode the essential changes in a visual scenery in a condensed way such that a robot can recognize and learn a manipulation without prior object knowledge. To achieve this we continuously track image segments in the video and construct a dynamic graph sequence. Topological transitions of those graphs occur whenever a spatial relation between some segments has changed in a discontinuous way and these moments are stored in a transition matrix called the semantic event chain (SEC). We demonstrate that these time points are highly descriptive for distinguishing between different manipulations. Employing simple sub-string search algorithms, SECs can be compared and type-similar manipulations can be recognized with high confidence. As the approach is generic, statistical learning can be used to find the archetypal SEC of a given manipulation class. The performance of the algorithm is demonstrated on a set of real videos showing hands manipulating various objects and performing different actions. In experiments with a robotic arm, we show that the SEC can be learned by observing human manipulations, transferred to a new scenario, and then reproduced by the machine. © SAGE Publications 2011., The research leading to these results has received funding from the European Community's Seventh Framework Programme FP7=2007-2013 - Challenge 2 - Cognitive Systems, Interaction, Robotics - under grant agreement No 247947 - GARNICS. B.D. acknowledges support from the Spanish Ministry for Science and Innovation via a Ramon y Cajal Fellowship., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison

  • Kazmi, Wajahat
  • Foix, Sergi
  • Alenyà, Guillem
  • Andersen, Hans Jørgen
In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger). © 2013 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS)., This work is supported by the Danish Council for Strategic Research under project ASETA (www.aseta.dk) grant no. 09-067027, the Spanish Ministry of Science and Innovation under projects PAU + DPI2011-27510, the EU project GARNICS (FP7-247947) and the Catalan Research Commission (SGR-00155). Wajahat Kazmi is funded by ASETA and Sergi Foix is supported by PhD fellowship from CSIC's JAE program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Learning weakly correlated cause-effects for gardening with a cognitive system

  • Agostini, Alejandro
  • Torras, Carme
  • Wörgötter, Florentin
We propose a cognitive system that combines artificial intelligence techniques for planning and learning to execute tasks involving delayed and variable correlations between the actions executed and their expected effects. The system is applied to the task of controlling the growth of plants, where the evolution of the plant attributes strongly depends on different events taking place in the temporally distant past history of the plant. The main problem to tackle is how to efficiently detect these past events. This is very challenging since the inclusion of time could make the dimensionality of the search space extremely large and the collected training instances may only provide very limited information about the relevant combinations of events. To address this problem we propose a learning method that progressively identifies those events that are more likely to produce a sequence of changes under a plant treatment. Since the number of experiences is very limited compared to the size of the event space, we use a probabilistic estimate that takes into account the lack of experience to prevent biased estimations. Planning operators are generated from most accurately predicted sequences of changes. Planning and learning are integrated in a decision-making framework that operates without task interruptions by allowing a human gardener to instruct the treatments when the knowledge acquired so far is not enough to make a decision., This research was supported by the European Community Seventh Framework ProgrammeFP7/2007–2013 – Challenge 2 – Cognitive Systems, Interaction, Robotics – under Grant agreement No 247947 – GARNICS., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

  • Alenyà, Guillem
  • Foix, Sergi
  • Torras, Carme
Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details., This research is partially funded by the EU GARNICS project FP7-247947, by CSIC project MANIPlus 201350E102, by the Spanish Ministry of Science and Innovation under project PAU+DPI2011-27510, and the Catalan Research Commission under Grant SGR-155., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

ToF cameras for active vision in robotics

  • Alenyà, Guillem
  • Foix, Sergi
  • Torras, Carme
ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited. After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/ foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages., This work was supported by the EU project GARNICS FP7-247947, by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510, and by the Catalan Research Commission through SGR-00155, Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Modeling leaf growth of rosette plants using infrared stereo image sequences

  • Aksoy, Eren Erdal
  • Abramov, Alexey
  • Wörgötter, Florentin
  • Scharr, Hanno
  • Fischbach, Andreas
  • Dellen, Babette
In this paper, we present a novel multi-level procedure for finding and tracking leaves of a rosette plant, in our case up to 3 weeks old tobacco plants, during early growth from infrared-image sequences. This allows measuring important plant parameters, e.g. leaf growth rates, in an automatic and non-invasive manner. The procedure consists of three main stages: preprocessing, leaf segmentation, and leaf tracking. Leaf-shape models are applied to improve leaf segmentation, and further used for measuring leaf sizes and handling occlusions. Leaves typically grow radially away from the stem, a property that is exploited in our method, reducing the dimensionality of the tracking task. We successfully tested the method on infrared image sequences showing the growth of tobacco-plant seedlings up to an age of about 30. days, which allows measuring relevant plant growth parameters such as leaf growth rate. By robustly fitting a suitably modified autocatalytic growth model to all growth curves from plants under the same treatment, average plant growth models could be derived. Future applications of the method include plant-growth monitoring for optimizing plant production in green houses or plant phenotyping for plant research., The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007-2013 – Challenge 2 – Cognitive Systems, Interaction, Robotics – under grant agreement No 247947 – GARNICS. Babette Dellen acknowledges support from the Spanish Ministry for Science and Innovation through a Ramon y Cajal program., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Growth signatures of rosette plants from time-lapse video

  • Dellen, Babette
  • Scharr, Hanno
  • Torras, Carme
Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found., The research leading to these results has received funding from the European Commission Seventh Framework Programme FP7/2007-2013 Challenge 2 Cognitive Systems, Interaction, Robotics under grant agreement No 247947 GARNICS. B. Dellen acknowledges support from the Spanish Ministry for Science and Innovation through a Ramon y Cajal program., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Service robots for citizens of the future

  • Torras, Carme
This paper summarizes the author’s presentation at the 26th Annual Conference of the Academia Europaea, Barcelona, July 2014., Erasmus Lecture 2014, Robots are no longer confined to factories; they are progressively spreading to urban, social and assistive domains. In order to become handy co-workers and helpful assistants, they must be endowed with quite different abilities from their industrial ancestors. Research on service robots aims to make them intrinsically safe to people, easy to teach by non-experts, able to manipulate not only rigid but also deformable objects, and highly adaptable to non-predefined and dynamic environments. Robots worldwide will share object and environmental models, their acquired knowledge and experiences through global databases and, together with the internet of things, will strongly change the citizens' way of life in so-called smart cities. This raises a number of social and ethical issues that are now being debated not only within the Robotics community but by society at large., The described research has been partially supported by the European projects PACO-PLUS (IST-FP6-IP-027657), GARNICS (FP7-ICT-247947) and IntellAct (FP7-ICT-269959), and the Spanish project PAU + (DPI2011-27510)., Peer Reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A cognitive architecture for automatic gardening

  • Agostini, Alejandro
  • Alenyà, Guillem
  • Fischbach, Andreas
  • Scharr, Hanno
  • Wörgötter, Florentin
  • Torras, Carme
In large industrial greenhouses, plants are usually treated following well established protocols for watering, nutrients, and shading/light. While this is practical for the automation of the process, it does not tap the full potential for optimal plant treatment. To more efficiently grow plants, specific treatments according to the plant individual needs should be applied. Experienced human gardeners are very good at treating plants individually. Unfortunately, hiring a crew of gardeners to carry out this task in large greenhouses is not cost effective. In this work we present a cognitive system that integrates artificial intelligence (AI) techniques for decision-making with robotics techniques for sensing and acting to autonomously treat plants using a real-robot platform. Artificial intelligence techniques are used to decide the amount of water and nutrients each plant needs according to the history of the plant. Robotic techniques for sensing measure plant attributes (e.g. leaves) from visual information using 3D model representations. These attributes are used by the AI system to make decisions about the treatment to apply. Acting techniques execute robot movements to supply the plants with the specified amount of water and nutrients., This research was supported by the commission of the European Communities Seventh Framework Programme FP7/2007-2013 - Challenge 2 - Cognitive Systems, Interaction, Robotics - under grant agreement No. 247947 - GARNICS., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Torras, Carme|||0000-0002-2933-398X
Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Lock-in time-of-flight (ToF) cameras: a survey

  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Torras, Carme|||0000-0002-2933-398X
This paper reviews the state-of-the art in the field of lock-in ToF cameras, their advantages, their limitations, the existing calibration methods, and the way they are being used, sometimes in combination with other sensors. Even though lockin ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight and reduced power consumption have motivated their increasing usage in several research areas, such as computer graphics, machine vision and robotics., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

ToF cameras for active vision in robotics

  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Torras, Carme|||0000-0002-2933-398X
ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited. After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: analysis and comparison

  • Kazmi, Wajahat
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Andersen, Hans Jørgen
In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger). (C) 2013 International Society for Photogrammetly and Remote Sensing, Inc. (ISPRS) Published by Elsevier B.V. All rights reserved., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Learning weakly correlated cause-effects for gardening with a cognitive system

  • Agostini, Alejandro Gabriel
  • Torras, Carme|||0000-0002-2933-398X
  • Worgotter, Florentin
We propose a cognitive system that combines artificial intelligence techniques for planning and learning to execute tasks involving delayed and variable correlations between the actions executed and their expected effects. The system is applied to the task of controlling the growth of plants, where the evolution of the plant attributes strongly depends on different events taking place in the temporally distant past history of the plant. The main problem to tackle is how to efficiently detect these past events. This is very challenging since the inclusion of time could make the dimensionality of the search space extremely large and the collected training instances may only provide very limited information about the relevant combinations of events. To address this problem we propose a learning method that progressively identifies those events that are more likely to produce a sequence of changes under a plant treatment. Since the number of experiences is very limited compared to the size of the event space, we use a probabilistic estimate that takes into account the lack of experience to prevent biased estimations. Planning operators are generated from most accurately predicted sequences of changes. Planning and learning are integrated in a decision-making framework that operates without task interruptions by allowing a human gardener to instruct the treatments when the knowledge acquired so far is not enough to make a decision., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

General robot kinematics decomposition without intermediate markers

  • Ulbrich, Stefan
  • Ruiz de Angulo García, Vicente|||0000-0002-2067-7399
  • Asfour, Tamim
  • Torras, Carme|||0000-0002-2933-398X
  • Dillmann, Rüdiger
The calibration of serial manipulators with high numbers of degrees of freedom by means of machine learning is a complex and time-consuming task. With the help of a simple strategy, this complexity can be drastically reduced and the speed of the learning procedure can be increased: When the robot is virtually divided into shorter kinematic chains, these subchains can be learned separately and, hence, much more efficiently than the complete kinematics. Such decompositions, however, require either the possibility to capture the poses of all endeffectors of all subchains at the same time, or they are limited to robots that fulfill special constraints. In this work, an alternative decomposition is presented that does not suffer from these limitations. An offline training algorithm is provided in which the composite subchains are learned sequentially with dedicated movements. A second training scheme is provided to train composite chains simultaneously and online. Both schemes can be used together with many machine learning algorithms. In the simulations, an algorithm using Parameterized Self-Organizing Maps (PSOM) modified for online learning and Gaussian Mixture Models (GMM) were chosen to show the correctness of the approach. The experimental results show that, using a two-fold decomposition, the number of samples required to reach a given precision, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Outdoor view recognition based on landmark grouping and logistic regression

  • Todt, Eduardo
  • Torras, Carme|||0000-0002-2933-398X
Vision-based robot localization outdoors has remained more elusive than its indoors counterpart. Drastic illumination changes and the scarceness of suitable landmarks are the main difficulties. This paper attempts to surmount them by deviating from the main trend of using local features. Instead, a global descriptor called landmark-view is defined, which aggregates the most visually-salient landmarks present in each scene. Thus, landmark co-occurrence and spatial and saliency relationships between them are added to the single landmark characterization, based on saliency and color distribution. A suitable framework to compare landmark-views is developed, and it is shown how this remarkably enhances the recognition performance, compared against single landmark recognition. A view-matching model is constructed using logistic regression. Experimentation using 45 views, acquired outdoors, containing 273 landmarks, yielded good recognition results. The overall percentage of correct view classification obtained was 80.6%, indicating the adequacy of the approach., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Learning the semantics of object-action relations by observation

  • Aksoy, Eren Erdal
  • Abramov, Alexey
  • Dörr, Johannes
  • Ning, Kejun
  • Dellen, Babette
  • Wörgötter, Florentin
Recognizing manipulations performed by a human and the transfer and execution of this by a robot is a difficult problem. We address this in the current study by introducing a novel representation of the relations between objects at decisive time points during a manipulation. Thereby, we encode the essential changes in a visual scenery in a condensed way such that a robot can recognize and learn a manipulation without prior object knowledge. To achieve this we continuously track image segments in the video and construct a dynamic graph sequence. Topological transitions of those graphs occur whenever a spatial relation between some segments has changed in a discontinuous way and these moments are stored in a transition matrix called the semantic event chain (SEC). We demonstrate that these time points are highly descriptive for distinguishing between different manipulations. Employing simple sub-string search algorithms, SECs can be compared and type-similar manipulations can be recognized with high confidence. As the approach is generic, statistical learning can be used to find the archetypal SEC of a given manipulation class. The performance of the algorithm is demonstrated on a set of real videos showing hands manipulating various objects and performing different actions. In experiments with a robotic arm, we show that the SEC can be learned by observing human manipulations, transferred to a new scenario, and then reproduced by the machine., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Service robots for citizens of the future

  • Torras, Carme|||0000-0002-2933-398X
Robots are no longer confined to factories; they are progressively spreading to urban, social and assistive domains. In order to become handy co-workers and helpful assistants, they must be endowed with quite different abilities from their industrial ancestors. Research on service robots aims to make them intrinsically safe to people, easy to teach by non-experts, able to manipulate not only rigid but also deformable objects, and highly adaptable to non-predefined and dynamic environments. Robots worldwide will share object and environmental models, their acquired knowledge and experiences through global databases and, together with the internet of things, will strongly change the citizens’ way of life in so-called smart cities. This raises a number of social and ethical issues that are now being debated not only within the Robotics community but by society at large., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Growth signatures of rosette plants from time-lapse video

  • Dellen, Babette
  • Scharr, Hanno
  • Torras, Carme|||0000-0002-2933-398X
Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Robotized plant probing: leaf segmentation utilizing time-of-flight data

  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Dellen, Babette
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Torras, Carme|||0000-0002-2933-398X
Supervision of long-lasting extensive botanic experiments is a promising robotic application that some recent technological advances have made feasible. Plant modeling for this application has strong demands, particularly in what concerns three-dimensional (3-D) information gathering and speed., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Stochastic exploration of ambiguities for nonrigid shape recovery

  • Moreno-Noguer, Francesc
  • Fua, Pascal
Recovering the 3D shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult non-linear minimization problem. The advantages of our method are demonstrated on a variety of problems including both real and synthetic data., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Service robots for citizens of the future

  • Torras, Carme
Robots are no longer confined to factories; they are progressively spreading to urban, social and assistive domains. In order to become handy co-workers and helpful assistants, they must be endowed with quite different abilities from their industrial ancestors. Research on service robots aims to make them intrinsically safe to people, easy to teach by non-experts, able to manipulate not only rigid but also deformable objects, and highly adaptable to non-predefined and dynamic environments. Robots worldwide will share object and environmental models, their acquired knowledge and experiences through global databases and, together with the internet of things, will strongly change the citizens’ way of life in so-called smart cities. This raises a number of social and ethical issues that are now being debated not only within the Robotics community but by society at large., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

General robot kinematics decomposition without intermediate markers

  • Ulbrich, Stefan
  • Ruiz de Angulo García, Vicente
  • Asfour, Tamim
  • Torras, Carme
  • Dillmann, Rüdiger
The calibration of serial manipulators with high numbers of degrees of freedom by means of machine learning is a complex and time-consuming task. With the help of a simple strategy, this complexity can be drastically reduced and the speed of the learning procedure can be increased: When the robot is virtually divided into shorter kinematic chains, these subchains can be learned separately and, hence, much more efficiently than the complete kinematics. Such decompositions, however, require either the possibility to capture the poses of all endeffectors of all subchains at the same time, or they are limited to robots that fulfill special constraints. In this work, an alternative decomposition is presented that does not suffer from these limitations. An offline training algorithm is provided in which the composite subchains are learned sequentially with dedicated movements. A second training scheme is provided to train composite chains simultaneously and online. Both schemes can be used together with many machine learning algorithms. In the simulations, an algorithm using Parameterized Self-Organizing Maps (PSOM) modified for online learning and Gaussian Mixture Models (GMM) were chosen to show the correctness of the approach. The experimental results show that, using a two-fold decomposition, the number of samples required to reach a given precision, Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Learning the semantics of object-action relations by observation

  • Aksoy, Eren Erdal
  • Abramov, Alexey
  • Dörr, Johannes
  • Ning, Kejun
  • Dellen, Babette
  • Wörgötter, Florentin
Recognizing manipulations performed by a human and the transfer and execution of this by a robot is a difficult problem. We address this in the current study by introducing a novel representation of the relations between objects at decisive time points during a manipulation. Thereby, we encode the essential changes in a visual scenery in a condensed way such that a robot can recognize and learn a manipulation without prior object knowledge. To achieve this we continuously track image segments in the video and construct a dynamic graph sequence. Topological transitions of those graphs occur whenever a spatial relation between some segments has changed in a discontinuous way and these moments are stored in a transition matrix called the semantic event chain (SEC). We demonstrate that these time points are highly descriptive for distinguishing between different manipulations. Employing simple sub-string search algorithms, SECs can be compared and type-similar manipulations can be recognized with high confidence. As the approach is generic, statistical learning can be used to find the archetypal SEC of a given manipulation class. The performance of the algorithm is demonstrated on a set of real videos showing hands manipulating various objects and performing different actions. In experiments with a robotic arm, we show that the SEC can be learned by observing human manipulations, transferred to a new scenario, and then reproduced by the machine., Peer Reviewed, Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Robotized plant probing: leaf segmentation utilizing time-of-flight data

  • Alenyà Ribas, Guillem
  • Dellen, Babette
  • Foix Salmerón, Sergi
  • Torras, Carme
Supervision of long-lasting extensive botanic experiments is a promising robotic application that some recent technological advances have made feasible. Plant modeling for this application has strong demands, particularly in what concerns three-dimensional (3-D) information gathering and speed., Peer Reviewed, Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

ToF cameras for active vision in robotics

  • Alenyà Ribas, Guillem
  • Foix Salmerón, Sergi
  • Torras, Carme
ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited. After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Stochastic exploration of ambiguities for nonrigid shape recovery

  • Moreno-Noguer, Francesc
  • Fua, Pascal
Recovering the 3D shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult non-linear minimization problem. The advantages of our method are demonstrated on a variety of problems including both real and synthetic data., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Outdoor view recognition based on landmark grouping and logistic regression

  • Todt, Eduardo
  • Torras, Carme
Vision-based robot localization outdoors has remained more elusive than its indoors counterpart. Drastic illumination changes and the scarceness of suitable landmarks are the main difficulties. This paper attempts to surmount them by deviating from the main trend of using local features. Instead, a global descriptor called landmark-view is defined, which aggregates the most visually-salient landmarks present in each scene. Thus, landmark co-occurrence and spatial and saliency relationships between them are added to the single landmark characterization, based on saliency and color distribution. A suitable framework to compare landmark-views is developed, and it is shown how this remarkably enhances the recognition performance, compared against single landmark recognition. A view-matching model is constructed using logistic regression. Experimentation using 45 views, acquired outdoors, containing 273 landmarks, yielded good recognition results. The overall percentage of correct view classification obtained was 80.6%, indicating the adequacy of the approach., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Lock-in time-of-flight (ToF) cameras: a survey

  • Foix Salmerón, Sergi
  • Alenyà Ribas, Guillem
  • Torras, Carme
This paper reviews the state-of-the art in the field of lock-in ToF cameras, their advantages, their limitations, the existing calibration methods, and the way they are being used, sometimes in combination with other sensors. Even though lockin ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight and reduced power consumption have motivated their increasing usage in several research areas, such as computer graphics, machine vision and robotics., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: analysis and comparison

  • Kazmi, Wajahat
  • Foix Salmerón, Sergi
  • Alenyà Ribas, Guillem
  • Andersen, Hans Jørgen
In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger). (C) 2013 International Society for Photogrammetly and Remote Sensing, Inc. (ISPRS) Published by Elsevier B.V. All rights reserved., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Learning weakly correlated cause-effects for gardening with a cognitive system

  • Agostini, Alejandro Gabriel
  • Torras, Carme
  • Worgotter, Florentin
We propose a cognitive system that combines artificial intelligence techniques for planning and learning to execute tasks involving delayed and variable correlations between the actions executed and their expected effects. The system is applied to the task of controlling the growth of plants, where the evolution of the plant attributes strongly depends on different events taking place in the temporally distant past history of the plant. The main problem to tackle is how to efficiently detect these past events. This is very challenging since the inclusion of time could make the dimensionality of the search space extremely large and the collected training instances may only provide very limited information about the relevant combinations of events. To address this problem we propose a learning method that progressively identifies those events that are more likely to produce a sequence of changes under a plant treatment. Since the number of experiences is very limited compared to the size of the event space, we use a probabilistic estimate that takes into account the lack of experience to prevent biased estimations. Planning operators are generated from most accurately predicted sequences of changes. Planning and learning are integrated in a decision-making framework that operates without task interruptions by allowing a human gardener to instruct the treatments when the knowledge acquired so far is not enough to make a decision., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

  • Alenyà Ribas, Guillem
  • Foix Salmerón, Sergi
  • Torras, Carme
Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details., Peer Reviewed, Postprint (author’s final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Growth signatures of rosette plants from time-lapse video

  • Dellen, Babette
  • Scharr, Hanno
  • Torras, Carme
Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found., Peer Reviewed, Postprint (author's final draft)