Pasar al contenido principal

ESTIMACION Y CONTROL DE MOVIMIENTO CON CAMARAS DE EVENTOS

PID2020-119244GB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 14
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi
  • Andrade-Cetto, Juan
Trabajo presentado en la 33rd British Machine Vision Conference, celebrada en Londres (Gran Bretaña), del 21 al 24 de noviembre de 2022. Vídeo de la presentación disponible en: https://bmvc2022.mpi-inf.mpg.de/577/, Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020- 119244GB-I00 funded by MCIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Contribución de congreso (ConferenceOutput)

Knowledge representation to enable high-level planning in cloth manipulation tasks

  • García-Camacho, Irene
  • Borràs, Julia
  • Alenyà, Guillem
Trabajo presentado en la ICAPS Workshop on Knowledge Engineering for Planning and Scheduling, celebrada en Singapore, el 15 de junio de 2022, Cloth manipulation is very relevant for domestic robotic tasks, but it presents many challenges due to the complexity of representing, recognizing and predicting the behaviour of cloth under manipulation. In this work, we propose a generic, compact and simplified representation of the states of cloth manipulation that allows for representing tasks as sequences of states and transitions semantically. We also define a Cloth Manipulation Graph that encodes all the strategies to accomplish a task. Our novel representation is used to encode two different cloth manipulation tasks, learned from an experiment with human subjects manipulating clothes with video data. We show how our simplified representation allows to obtain a map of meaningful steps that can serve to describe cloth manipulation tasks as domain models in PDDL, enabling high-level planning. Finally, we discuss on the existing skills that could enable the sensory motor grounding and the low-level execution of the plan., The research leading to these results receives funding from the European Research Council (ERC) from the European Union Horizon 2020 Programme under grant agreement no. 741930 (CLOTHILDE: CLOTH manIpulation Learning from DEmonstrations); by MCIN/ AEI /10.13039/501100011033, Spain, under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033, Spain, and by the “European Union NextGenerationEU/PRTR, Spain, under the project COHERENT (PCI2020-120718-2); and by the European Commission- NextGenerationEU, through CSIC’s Thematic Platforms (PTI+ Neuro-Aging).
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi|||0000-0002-8673-6363
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020-119244GB-I00 funded by CIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

User interactions and negative examples to improve the learning of semantic rules in a cognitive exercise scenario

  • Suárez Hernández, Alejandro
  • Andriella, Antonio|||0000-0002-6641-6450
  • Torras, Carme|||0000-0002-2933-398X
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Enabling a robot to perform new tasks is a complex endeavor, usually beyond the reach of non-technical users. For this reason, research efforts that aim at empowering end-users to teach robots new abilities using intuitive modes of interaction are valuable. In this article, we present INtuitive PROgramming 2 (INPRO2), a learning framework that allows inferring planning actions from demonstrations given by a human teacher. INPRO2 operates in an assistive scenario, in which the robot may learn from a healthcare professional (a therapist or caregiver) new cognitive exercises that can be later administered to patients with cognitive impairment. INPRO2 features significant improvements over previous work, namely: (1) exploitation of negative examples; (2) proactive interaction with the teacher to ask questions about the legality of certain movements; and (3) learning goals in addition to legal actions. Through simulations, we show the performance of different proactive strategies for gathering negative examples. Real-world experiments with human teachers and a TIAGo robot are also presented to qualitatively illustrate INPRO2., The research leading to these results receives funding from the EU H2020 Programme under grant agreement ERC–2016–ADG–741930 (CLOTHILDE); by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR under the project COHERENT (PCI2020-120718-2); AA acknowledges the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801342 (Tecniospring INDUSTRY); ASH acknowledges the financial support of the Apadrina la Ciencia association and of the Ford Spanish branch., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Teaching a robot where doors and drawers are and how to handle them

  • Cupec, Robert
  • Vidovic, Ivan
  • Šimundic, Valentin
  • Pejic, P.
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state., This work has been partially supported by the Croatian Science Foundation under the project IP-2019-04-6819; by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Egomotion from event-based SNN optical flow

  • Tian, Yi|||0000-0002-8673-6363
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
We present a method for computing egomotion using event cameras with a pre-trained optical flow spiking neural network (SNN). To address the aperture problem encountered in the sparse and noisy normal flow of the initial SNN layers, our method includes a sliding-window bin-based pooling layer that computes a fused full flow estimate. To add robustness to noisy flow estimates, instead of computing the egomotion from vector averages, our method optimizes the intersection of constraints. The method also includes a RANSAC step to robustly deal with outlier flow estimates in the pooling layer. We validate our approach on both simulated and real scenes and compare our results favorably to the state-of-the-art methods. However, our method may be sensitive to datasets and motion speeds different from those used for training, limiting its generalizability., This work received support from projects EBCON (PID2020-119244GBI00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/ 501100011033 and by the "European Union NextGenerationEU/PRTR"; the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya; and by an FI AGAUR PhD grant to Yi Tian., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Event-IMU fusion strategies for faster-than-IMU estimation throughput

  • Chamorro Hernández, William Oswaldo
  • Solà Ortega, Joan|||0000-0002-2933-3381
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This study presents new methods for integrating event data and IMU readings to achieve ultra-fast camera pose estimates. The conventional predict-with-IMU-correct-with-vision approach is no longer optimal because events can be generated much more rapidly than IMU data. Therefore, two novel fusion schemes are proposed, which combine constant velocity and constant acceleration prediction models with ultra-fast (10 kHz) event-based updates and slower 1 kHz IMU updates. The first scheme uses IMU data as instantaneous measurements of acceleration and angular rate, while the second scheme considers these measurements as the average within the IMU sampling time. To provide a basis for comparison, the traditional method that predicts motion using IMU data and updates the estimates with event-feature matching over a 1ms time window is also implemented. All models are designed as Kalman filter variants, which act as the tracker module of a PTAM system in a human-made indoor scenario and are subjected to stress experiments to evaluate their capabilities. The models are also compared against an event-only estimator and a frame-based visual-inertial approach. The findings demonstrate superior performance at a throughput that is 100 times faster than the state-of-the-art., This work received support from projects EBCON (PID2020-119244GB-I00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR”, and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Adaptive human-robot collaboration: evolutionary learning of action costs using an action outcome simulator

  • Izquierdo Badiola, Silvia
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Rizzo, Carlos
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, One of the main challenges for successful human- robot collaborative applications lies in adapting the plan to the human agent’s changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents’ preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms., This work has been partially supported by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859), Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Event-IMU fusion strategies for faster-than-IMU estimation throughput

  • Chamorro Hernández, William Oswaldo
  • Solà Ortega, Joan
  • Andrade-Cetto, Juan
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This study presents new methods for integrating event data and IMU readings to achieve ultra-fast camera pose estimates. The conventional predict-with-IMU-correct-with-vision approach is no longer optimal because events can be generated much more rapidly than IMU data. Therefore, two novel fusion schemes are proposed, which combine constant velocity and constant acceleration prediction models with ultra-fast (10 kHz) event-based updates and slower 1 kHz IMU updates. The first scheme uses IMU data as instantaneous measurements of acceleration and angular rate, while the second scheme considers these measurements as the average within the IMU sampling time. To provide a basis for comparison, the traditional method that predicts motion using IMU data and updates the estimates with event-feature matching over a 1ms time window is also implemented. All models are designed as Kalman filter variants, which act as the tracker module of a PTAM system in a human-made indoor scenario and are subjected to stress experiments to evaluate their capabilities. The models are also compared against an event-only estimator and a frame-based visual-inertial approach. The findings demonstrate superior performance at a throughput that is 100 times faster than the state-of-the-art., This work received support from projects EBCON (PID2020-119244GB-I00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR”, and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi
  • Andrade-Cetto, Juan
Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020-119244GB-I00 funded by CIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Adaptive human-robot collaboration: evolutionary learning of action costs using an action outcome simulator

  • Izquierdo Badiola, Silvia
  • Alenyà Ribas, Guillem
  • Rizzo, Carlos
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, One of the main challenges for successful human- robot collaborative applications lies in adapting the plan to the human agent’s changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents’ preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms., This work has been partially supported by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Teaching a robot where doors and drawers are and how to handle them

  • Cupec, Robert
  • Vidovic, Ivan
  • Šimundic, Valentin
  • Pejic, P.
  • Foix Salmerón, Sergi
  • Alenyà Ribas, Guillem
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state., This work has been partially supported by the Croatian Science Foundation under the project IP-2019-04-6819; by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Egomotion from event-based SNN optical flow

  • Tian, Yi
  • Andrade-Cetto, Juan
We present a method for computing egomotion using event cameras with a pre-trained optical flow spiking neural network (SNN). To address the aperture problem encountered in the sparse and noisy normal flow of the initial SNN layers, our method includes a sliding-window bin-based pooling layer that computes a fused full flow estimate. To add robustness to noisy flow estimates, instead of computing the egomotion from vector averages, our method optimizes the intersection of constraints. The method also includes a RANSAC step to robustly deal with outlier flow estimates in the pooling layer. We validate our approach on both simulated and real scenes and compare our results favorably to the state-of-the-art methods. However, our method may be sensitive to datasets and motion speeds different from those used for training, limiting its generalizability., This work received support from projects EBCON (PID2020-119244GBI00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/ 501100011033 and by the "European Union NextGenerationEU/PRTR"; the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya; and by an FI AGAUR PhD grant to Yi Tian., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

User interactions and negative examples to improve the learning of semantic rules in a cognitive exercise scenario

  • Suárez Hernández, Alejandro
  • Andriella, Antonio
  • Torras, Carme
  • Alenyà Ribas, Guillem
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Enabling a robot to perform new tasks is a complex endeavor, usually beyond the reach of non-technical users. For this reason, research efforts that aim at empowering end-users to teach robots new abilities using intuitive modes of interaction are valuable. In this article, we present INtuitive PROgramming 2 (INPRO2), a learning framework that allows inferring planning actions from demonstrations given by a human teacher. INPRO2 operates in an assistive scenario, in which the robot may learn from a healthcare professional (a therapist or caregiver) new cognitive exercises that can be later administered to patients with cognitive impairment. INPRO2 features significant improvements over previous work, namely: (1) exploitation of negative examples; (2) proactive interaction with the teacher to ask questions about the legality of certain movements; and (3) learning goals in addition to legal actions. Through simulations, we show the performance of different proactive strategies for gathering negative examples. Real-world experiments with human teachers and a TIAGo robot are also presented to qualitatively illustrate INPRO2., The research leading to these results receives funding from the EU H2020 Programme under grant agreement ERC–2016–ADG–741930 (CLOTHILDE); by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR under the project COHERENT (PCI2020-120718-2); AA acknowledges the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801342 (Tecniospring INDUSTRY); ASH acknowledges the financial support of the Apadrina la Ciencia association and of the Ford Spanish branch., Peer Reviewed, Postprint (author's final draft)