Skip to main content

ESTIMACION Y CONTROL DE MOVIMIENTO CON CAMARAS DE EVENTOS

PID2020-119244GB-I00

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2020
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 37
Encontrada(s) 1 página(s)

RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Probabilistic Graph-Based Real-Time Ground Segmentation for Urban Robotics

  • del Pino, Iván
  • Santamaria-Navarro, Angel
  • Garrell Zulueta, Anaís
  • Torres, Fernando
  • Andrade-Cetto, Juan
  • Torres, Fernando|||/items/17ced7cb-5c1d-429d-b922-315f91fddc33
Terrain analysis is of paramount importance for the safe navigation of autonomous robots. In this study, we introduce GATA, a probabilistic real-time graph-based method for segmentation and traversability analysis of point clouds. In the method, we iteratively refine the parameters of a ground plane model and identify regions imaged by a LiDAR as traversable and non-traversable. The method excels in delivering rapid, high-precision obstacle detection, surpassing existing state-of-the-art methods. Furthermore, our method addresses the need to distinguish between surfaces with varying traversability, such as vegetation or unpaved roads, depending on the specific application. To achieve this, we integrate a shallow neural network, which operates on features extracted from the ground model. This enhancement not only boosts performance but also maintains real-time efficiency, without the need for GPUs. The method is rigorously evaluated using the SemanticKitti dataset and its practicality is showcased through real-world experiments with an urban last-mile delivery autonomous robot.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Event-Based Line SLAM in Real-Time

  • Chamorro Hernández, William Oswaldo
  • Solà, Joan
  • Andrade-Cetto, Juan
Event-based cameras generate asynchronous streams of events, triggered proportionally to the logarithmic change of brightness in the scene. These cameras have very low latency and high dynamic range suitable to address challenging motion scenarios in robotics. In this work, we explore a new event-based line-SLAM approach following a parallel tracking and mapping philosophy. Our fast tracking algorithm, produces accurate camera pose estimates at a high rate by minimizing the event-line reprojection error with an error-state Kalman filter formulated entirely with Lie theory. The mapping thread leverages the natural edge highlighting strength of events to recover and optimize straight lines in human-made scenarios. The proper manipulation of matrix sparsity as well as the information sharing between tracking and mapping nodes allow us to achieve real-time performance on a standard multi-core CPU. This system was tested on several scenarios rich in straight edge objects, and compared against, ground truth and frame and event based state-of-the-art approaches., This work was supported by the EU H2020 projects TERRINET INFRAI-2017-1-730994 and CANOPIES ICT-2020-2-101016906, by project EBCON PID2020- 119244GB-I00 funded by MCIN/AEI/10.13039/501100011033, and by a scholarship from SENESCYT, Republic of Ecuador to WCH.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

OCRA – An ontology for collaborative robotics and adaptation

  • Olivares-Alarcos, Alberto
  • Foix, Sergi
  • Borgo, Stefano
  • Alenyà, Guillem
Industrial collaborative robots will be used in unstructured scenarios and a large variety of tasks in the near future. These robots shall collaborate with humans, who will add uncertainty and safety constraints to the execution of industrial robotic tasks. Hence, trustworthy collaborative robots must be able to reason about their collaboration's requirements (e.g., safety), as well as the adaptation of their plans due to unexpected situations. A common approach to reasoning is to represent the knowledge of interest using logic-based formalisms, such as ontologies. However, there is not an established ontology defining notions such as collaboration or adaptation yet. In this article, we propose an Ontology for Collaborative Robotics and Adaptation (OCRA), which is built around two main notions: collaboration, and plan adaptation. OCRA ensures a reliable human-robot collaboration, since robots can formalize, and reason about their plan adaptations and collaborations in unstructured collaborative robotic scenarios. Furthermore, our ontology enhances the reusability of the domain's terminology, allowing robots to represent their knowledge about different collaborative and adaptive situations. We validate our formal model, first, by demonstrating that a robot may answer a set of competency questions using OCRA. Second, by studying the formalization's performance in limit cases that include instances with incongruent and incomplete axioms. For both validations, the example use case consists in a human and a robot collaborating on the filling of a tray., This work has been partially funded by MCIN/ AEI /10.13039/501100011033, Spain, under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033, Spain, and by the “European Union NextGenerationEU/PRTR, Spain, under the project COHERENT (PCI2020-120718-2); and by the European Commission- NextGenerationEU, through CSIC’s Thematic Platforms (PTI+ Neuro-Aging), Spain. S. Borgo is partially supported by the European project OntoCommons (GA 958371), Italy. A. Olivares-Alarcos is supported by the European Social Fund and the Ministry of Business and Knowledge of Catalonia, Spain, through the FI 2020 grant.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Wolf: A Modular Estimation Framework for Robotics Based on Factor Graphs

  • Solà, Joan
  • Vallvé, Joan
  • Casalsm Joaquim
  • Deray, Jeremie
  • Fourmy, Mederic
  • Atchuthan, Dinesh
  • Corominas Murtra, Andreu
  • Andrade-Cetto, Juan
This letter introduces Wolf, a C++ estimation framework based on factor graphs and targeted at mobile robotics. Wolf can be used beyond SLAM to handle self-calibration, model identification, or the observation of dynamic quantities other than localization. The architecture of Wolf allows for a modular yet tightly-coupled estimator. Modularity is enhanced via reusable plugins that are loaded at runtime depending on application setup. This setup is achieved conveniently through YAML files, allowing users to configure a wide range of applications without the need of writing or compiling code. Most procedures are coded as abstract algorithms in base classes with varying levels of specialization. Overall, all these assets allow for coherent processing and favor code re-usability and scalability. Wolf can be used with ROS, and is made publicly available and open to collaboration., This work was partially supported by the EU H2020 projects LOGIMATIC (Galileo- 2015-1-687534), GAUSS (Galileo-2017-1-776293), TERRINET (INFRAIA- 2017-1-730994), and MEMMO (ICT-25-2017-1-780684); and by the Spanish State Research Agency through projects EB-SLAM (DPI2017-89564- P), EBCON (PID2020-119244GB-I00), and the Mar´ıa de Maeztu Seal of Excellence to IRI (MDM-2016-0656).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

ACHORD: Communication-Aware Multi-Robot Coordination with Intermittent Connectivity

  • Saboia, María
  • Clark, Lillian
  • Thangavelu, Vivek
  • Edlund, Jeffrey A.
  • Otsu, Kyohei
  • Correa, Gustavo J.
  • Varadharajan, Vivek Shankar
  • Santamaria-Navarro, Àngel
  • Touma, Thomas
  • Bouman, Amanda
  • Melikyan, Hovhannes
  • Pailevanian, Torkom
  • Kim, Sung-Kyun
  • Archanian, Avak
  • Stegun Vaquero, Tiago
  • Beltrame, Giovanni
  • Napp, Nils
  • Pessin, Gustavo
  • Agha-Mohammadi, Ali Akbar
Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, otherwise mission-critical data may be lost. In this paper, we describe and analyze ACHORD (Autonomous & Collaborative High-Bandwidth Operations with Radio Droppables), a multi-layer networking solution which tightly co-designs the network architecture and high-level decision-making for improved comms. ACHORD provides bandwidth prioritization and timely and reliable data transfer despite intermittent connectivity. Furthermore, it exposes low-layer networking metrics to the application layer to enable robots to autonomously monitor, map, and extend the network via droppable radios, as well as restore connectivity to improve collaborative exploration. We evaluate our solution with respect to the comms performance in several challenging underground environments including the DARPA SubT Finals competition environment. Our findings support the use of data stratification and flow control to improve bandwidth-usage., This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004). This work was supported in part by FUNDEP, by NASA Space Technology Research Fellowship Grant No. 80NSSC19K1189, and by NSF Grant No. #1846340. Further, it was partially supported by the Spanish Ministry of Science and Innovation under the project EB-CON (PID2020-119244GB-I00, MCIN/AEI/10.13039/501100011033) and by the European Commission under the project CANOPIES (H2020-ICT-2020- 2-101016906).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi
  • Andrade-Cetto, Juan
Trabajo presentado en la 33rd British Machine Vision Conference, celebrada en Londres (Gran Bretaña), del 21 al 24 de noviembre de 2022. Vídeo de la presentación disponible en: https://bmvc2022.mpi-inf.mpg.de/577/, Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020- 119244GB-I00 funded by MCIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Knowledge representation to enable high-level planning in cloth manipulation tasks

  • García-Camacho, Irene
  • Borràs, Julia
  • Alenyà, Guillem
Trabajo presentado en la ICAPS Workshop on Knowledge Engineering for Planning and Scheduling, celebrada en Singapore, el 15 de junio de 2022, Cloth manipulation is very relevant for domestic robotic tasks, but it presents many challenges due to the complexity of representing, recognizing and predicting the behaviour of cloth under manipulation. In this work, we propose a generic, compact and simplified representation of the states of cloth manipulation that allows for representing tasks as sequences of states and transitions semantically. We also define a Cloth Manipulation Graph that encodes all the strategies to accomplish a task. Our novel representation is used to encode two different cloth manipulation tasks, learned from an experiment with human subjects manipulating clothes with video data. We show how our simplified representation allows to obtain a map of meaningful steps that can serve to describe cloth manipulation tasks as domain models in PDDL, enabling high-level planning. Finally, we discuss on the existing skills that could enable the sensory motor grounding and the low-level execution of the plan., The research leading to these results receives funding from the European Research Council (ERC) from the European Union Horizon 2020 Programme under grant agreement no. 741930 (CLOTHILDE: CLOTH manIpulation Learning from DEmonstrations); by MCIN/ AEI /10.13039/501100011033, Spain, under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033, Spain, and by the “European Union NextGenerationEU/PRTR, Spain, under the project COHERENT (PCI2020-120718-2); and by the European Commission- NextGenerationEU, through CSIC’s Thematic Platforms (PTI+ Neuro-Aging).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Time-to-contact for robot safety stop in close collaborative tasks

  • Olivares-Alarcos, Alberto
  • Foix, Sergi
  • Alenyà, Guillem
Within the next years, industrial collaborative robots will collaborate with human operators, whose safety might be compromised. To ensure a safe collaboration, robots should estimate the risk of collision with respect to the pose and motion of operators within the shared workspace. A common approach is to compute and maintain a minimum distance between humans and robots during tasks' execution. Nevertheless, separation-based solutions do not capture the real dynamics of the human-robot interaction and tend to be rather conservative, avoiding a close human-robot collaboration. In this chapter, we explore the concept of time-to-contact (TTC) as a softer trigger of safety stops in collaborative scenarios where humans and robots are in constant closeness. Particularly, we propose a TTC formulation and study its advantages with respect to two approaches based on the protective distance proposed by the ISO standards. We compared the three methods in some representative cases extracted from an example of a collaborative task. The evaluation is firstly done in simulation and then in a more realistic setup with a simulated human, aiming for repeatability, and a real robot. Furthermore, we showcased our approach in a demo of a complete collaborative task. TTC allows robots to operate closer to humans and for longer times before a safety stop is issued, which benefits long-term productivity. This later stop produces shorter human-robot distances, which might affect safety. However, the increment in time that the robot moves before stopping (productivity) is far greater than the reduction in the distance (safety). Hence, we can state that TTC greatly improves productivity while slightly compromising safety. In conclusion, our work demonstrates that TTC is a smoother, but still safe, collision risk estimator for close human-robot collaboration, This work has been partially funded by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by the ”European Union (EU) NextGenerationEU/PRTR and by MCIN/ AEI /10.13039/501100011033 under the project COHERENT (PCI2020-120718-2); and by the European Commission NextGenerationEU, through CSIC’s Thematic Platforms (PTI+ Neuro-Aging). A. Olivares-Alarcos is supported by the European Social Fund and the Ministry of Business and Knowledge of Catalonia through the FI 2020 grant.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Condition-based Design of Variable Impedance Controllers from User Demonstrations

  • San Miguel, Alberto
  • Puig, Vicenç
  • Alenyà, Guillem
This paper presents an approach to ensure conditions on Variable Impedance Controllers through the off-line tuning of the parameters involved in its description. In particular, we prove its application to term modulations defined by a Learning from Demonstration technique. This is performed through the assessment of conditions regarding safety and performance, which encompass heuristics and constraints in the form of Linear Matrix Inequalities. Latter ones allow to define a convex optimisation problem to analyse their fulfilment, and require a polytopic description of the VIC, in this case, obtained from its formulation as a discrete-time Linear Parameter Varying system. With respect to the current state-of-art, this approach only limits the term definition obtained by the Learning from Demonstration technique to be continuous and function of exogenous signals, i.e. external variables to the robot. Therefore, using a solution-search method, the most suitable set of parameters according to assessment criteria can be obtained. Using a 7-DoF KINOVAGEN3 manipulator, validation and comparison against solutions with relaxed conditions are performed. The method is applied to generate Variable Impedance Controllers for a pulley belt looping task, inspired by the Assembly Challenge for Industrial Robotics in World Robot Summit 2018, to reduce the exerted force with respect to a standard (constant) Impedance Controller. These controllers fulfil a set of safety constraints, namely stability, bounds on task variables and maximum response overshooting; and their performance is determined by the User Preference heuristic, which allows to intuitively define the desired compliant behaviour along the task. In the context of the task, this is used to generate new controllers for one-off modifications of the nominal belt looping task setup without new demonstrations., This work is partially supported by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR” under the project ROB-IN (PLEC2021-007859); and by MCIN/ AEI /10.13039/ 501100011033, Spain, under the project CHLOE-GRAPH (PID2020- 119244GB-I00). Authors also want to thank Adriá Colomé and Edoardo Caldarelli for their comments and help throughout this work., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Event-IMU fusion strategies for faster-than-IMU estimation throughput

  • Chamorro Hernández, William Oswaldo
  • Solà, Joan
  • Andrade-Cetto, Juan
This study presents new methods for integrating event data and IMU readings to achieve ultra-fast camera pose estimates. The conventional predict-with-IMU-correct-with-vision approach is no longer optimal because events can be generated much more rapidly than IMU data. Therefore, two novel fusion schemes are proposed, which combine constant velocity and constant acceleration prediction models with ultra-fast (10 kHz) event-based updates and slower 1 kHz IMU updates. The first scheme uses IMU data as instantaneous measurements of acceleration and angular rate, while the second scheme considers these measurements as the average within the IMU sampling time. To provide a basis for comparison, the traditional method that predicts motion using IMU data and updates the estimates with event-feature matching over a 1ms time window is also implemented. All models are designed as Kalman filter variants, which act as the tracker module of a PTAM system in a human-made indoor scenario and are subjected to stress experiments to evaluate their capabilities. The models are also compared against an event-only estimator and a frame-based visual-inertial approach. The findings demonstrate superior performance at a throughput that is 100 times faster than the state-of-the-art., This work received support from projects EBCON (PID2020-119244GB-I00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR”, and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Semantic State Prediction in Robotic Cloth Manipulation

  • Tzelepis, Georgios
  • Borràs, Julia
  • Eren Erdal, Aksoy
  • Alenyà, Guillem
State estimation of deformable objects such as textiles is notorious difficult due to its extreme high dimensionality and complexity. Lack of data and benchmarks is another challenge impeding progress in robotic cloth manipulation. In this paper, we make a first attempt to solve the problem of semantic state estimation through RGB-D data only in an end-to-end manner with the help of deep neural networks. Since neural networks require large amounts of labeled data, we introduce a novel Mujoco simulator to generate a large-scale fully annotated robotic textile manipulation dataset including bimanual actions. Finally, we provide a set of baseline deep neural networks and benchmark them on the problem of semantic state prediction on our proposed dataset., This work has been partially funded by MCIN/AEI/10.13039/501100011033, Spain, under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by the European Union’s Horizon 2020 under ERC Advanced Grant CLOTHILDE (no. 741930); and by MCIN/AEI/10.13039/501100011033, Spain, and by the European Union NextGenerationEU/ PRTR, Spain, under the project COHERENT (PCI2020-120718-2);, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Adaptive Human-Robot Collaboration: Evolutionary Learning of Action Costs Using an Action Outcome Simulator

  • Izquierdo-Badiola, Silvia
  • Alenyà, Guillem
  • Rizzo, Carlos
Trabajo presentado en el 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), celebrado en Busan (República de Corea), del 28 al 31 de agosto de 2023, One of the main challenges for successful human-robot collaborative applications lies in adapting the plan to the human agent's changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents' preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms., This work has been partially supported by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859), Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Teaching a Robot Where Doors and Drawers Are and How To Handle Them

  • Cupec, R.
  • Vidovic, I.
  • Simundic, V.
  • Pejic, P.
  • Foix, Sergi
  • Alenyà, Guillem
Trabajo presentado en el 32nd IEEE International Conference on Robot and Human Interactive Communication, celebrado en Busan (Corea del Sur), del 28 al 31 de agosto de 2023, We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state., This work has been partially supported by the Croatian Science Foundation under the project IP-2019-04-6819; by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Probabilistic Graph-based Real-Time Ground Segmentation for Urban Robotics

  • Pino, Iván del
  • Santamaria-Navarro, Àngel
  • Garrell, Anaís
  • Torres, Fernando
  • Andrade-Cetto, Juan
Terrain analysis is of paramount importance for the safe navigation of autonomous robots. In this study, we introduce GATA, a probabilistic real-time graph-based method for segmentation and traversability analysis of point clouds. In the method, we iteratively refine the parameters of a ground plane model and identify regions imaged by a LiDAR as traversable and non-traversable. The method excels in delivering rapid, high-precision obstacle detection, surpassing existing state-of-the-art methods. Furthermore, our method addresses the need to distinguish between surfaces with varying traversability, such as vegetation or unpaved roads, depending on the specific application. To achieve this, we integrate a shallow neural network, which operates on features extracted from the ground model. This enhancement not only boosts performance but also maintains real-time efficiency, without the need for GPUs. The method is rigorously evaluated using the SemanticKitti dataset and its practicality is showcased through real-world experiments with an urban last-mile delivery autonomous robot. The code is publicly available at <uri>https://gitlab.iri.upc.edu/idelpino/iri_ground_segmentation</uri>, This work was supported in part by the EIT Urban Mobility project LOGISMILE under Grant EIT-UM-2020-22140 and Grant EIT-UM-2023-23374, in part by the Spanish Projects EBCON under Grant PID2020-119244GB-I00, in part by MCIN/AEI/10.13039/501100011033, AUDEL under Grant TED2021- 131759A-I00, in part by MCIN/ AEI/10.13039/501100011033, in part by the “European Union NextGenerationEU/PRTR” and LENA under Grant PID2022- 142039NA-I00, in part by MCIN/ AEI/10.13039/501100011033, in part by “ERDF A way of making Europe”, in part by BotNet under Grant 23S06128-00, in part by the “Ajuntament de Barcelona” and “Fundació la Caixa”, in part by the Consolidated Research Group RAIG under Grant 2021 SGR 00510 of the Departament de Recerca i Universitats de la Generalitat de Catalunya, in part by aMargarita Salas Fellowship to IDP (MARSALAS21-08) funded by the Spanish Ministry of Universities, in part by the European Union-Next Generation, and in part by the University of Alicante., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

On Lie Group IMU and Linear Velocity Preintegration for Autonomous Navigation Considering the Earth Rotation Compensation

  • Vial, Pau
  • Solà, Joan
  • Palomeras, Narcís
  • Carreras, Marc
Robot localization is a fundamental task in achieving true autonomy. Recently, many graph-based navigators have been proposed that combine an inertial measurement unit (IMU) with an exteroceptive sensor applying IMU preintegration to synchronize both sensors. IMUs are affected by biases that also have to be estimated. To increase the navigator robustness when faults appear on the perception system, IMU preintegration can be complemented with linear velocity measurements obtained from visual odometry, leg odometry, or a Doppler Velocity Log (DVL), depending on the robotic application. Moreover, higher grade IMUs are sensitive to the Earth rotation rate, which must be compensated in the preintegrated measurements. In this article, we propose a general purpose preintegration methodology formulated on a compact Lie group to set motion constraints on graph simultaneous localization and mapping problems considering the Earth rotation effect. We introduce the SE_N(3) group to jointly preintegrate IMU data and linear velocity measurements to preserve all the existing correlation within the preintegrated quantity. Field experiments using an autonomous underwater vehicle equipped with a DVL and a navigational grade IMU are provided and results are benchmarked against a commercial filter-based inertial navigation system to prove the effectiveness of our methodology., The work of Pau Vial was supported by the PhD under Grant FPU19/03638. This work was supported in part by the BITER-AUV Project under Grant PID2020-114732RB-C33, in part by the PLOME Project under Grant PLEC2021-007525, in part by the EBCON Project under Grant PID2020119244GB-I00, and in part by the Consolidated Research Group RAIG under Grant 2021 SGR 00510 of the Departament de Recerca i Universitats de la Generalitat de Catalunya. Open Access funding was provided thanks to the CRUE-CSIC agreement with IEEE, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

SDformerFlow: Spiking Neural Network Transformer for Event-based Optical Flow

  • Tian, Yi
  • Andrade-Cetto, Juan
Event cameras produce asynchronous and sparse event streams capturing changes in light intensity. Overcoming limitations of conventional frame-based cameras, such as low dynamic range and data rate, event cameras prove advantageous, particularly in scenarios with fast motion or challenging illumination conditions. Leveraging similar asynchronous and sparse characteristics, Spiking Neural Networks (SNNs) emerge as natural counterparts for processing event camera data. Recent advancements in Visual Transformer architectures have demonstrated enhanced performance in both Artificial Neural Networks (ANNs) and SNNs across various computer vision tasks. Motivated by the potential of transformers and spikeformers, we propose two solutions for fast and robust optical flow estimation: STTFlowNet and SDformerFlow. STTFlowNet adopts a U-shaped ANN architecture with spatiotemporal Swin transformer encoders, while SDformerFlow presents its full spike counterpart with spike-driven Swin transformer encoders. Notably, our work marks the first utilization of spikeformer for dense optical flow estimation. We conduct end-to-end training for both models using supervised learning on the DSEC-flow Dataset. Our results indicate comparable performance with state-of-the-art SNNs and significant improvement in power consumption compared to the best-performing ANNs for the same task. Our code will be open-sourced at https://github.com/yitian97/SDformerFlow., This work received support from projects EBCON (PID2020-119244GB-I00) and RAADICAL (PLEC2021-007817) funded by MCIN/ AEI/ 10.13039/ 501100011033 and the "European Union NextGenerationEU/PRTR"; the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya; and by an FI AGAUR PhD grant to Yi Tian., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Semantic State Estimation in Robot Cloth Manipulations Using Domain Adaptation from Human Demonstrations

  • Tzelepis, Georgies
  • Aksoy, Eren Erdal
  • Borràs, Julia
  • Alenyà, Guillem
Deformable object manipulations, such as those involving textiles, present a significant challenge due to their high dimensionality and complexity. In this paper, we propose a solution for estimating semantic states in cloth manipulation tasks. To this end, we introduce a new, large-scale, fully-annotated RGB image dataset of semantic states featuring a diverse range of human demonstrations of various complex cloth manipulations. This effectively transforms the problem of action recognition into a classification task. We then evaluate the generalizability of our approach by employing domain adaptation techniques to transfer knowledge from human demonstrations to two distinct robotic platforms: Kinova and UR robots. Additionally, we further improve performance by utilizing a semantic state graph learned from human manipulation data., This work has been partially funded by MCIN/ AEI /10.13039/ 501100011033, Spain, under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by the European Union’s Horizon 2020 under ERC Advanced Grant CLOTHILDE (no. 741930); and by MCIN/ AEI /10.13039/501100011033, Spain, and by the “European Union NextGenerationEU/ PRTR, Spain, under the project COHERENT (PCI2020-120718-2);, Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

On Lie group IMU and linear velocity preintegration for autonomous navigation considering the Earth rotation compensation

  • Vial Serrat, Pau
  • Solà Ortega, Joan|||0000-0002-2933-3381
  • Palomeras Rovira, Narcís
  • Carreras Pérez, Marc
This work is licensed under a Creative Commons Attribution 4.0 License., Robot localization is a fundamental task in achieving true autonomy. Recently, many graph-based navigators have been proposed that combine an inertial measurement unit (IMU) with an exteroceptive sensor applying IMU preintegration to synchronize both sensors. IMUs are affected by biases that also have to be estimated. To increase the navigator robustness when faults appear on the perception system, IMU preintegration can be complemented with linear velocity measurements obtained from visual odometry, leg odometry, or a Doppler Velocity Log (DVL), depending on the robotic application. Moreover, higher grade IMUs are sensitive to the Earth rotation rate, which must be compensated in the preintegrated measurements. In this paper, we propose a general purpose preintegration methodology formulated on a compact Lie group to set motion constraints on Graph SLAM problems considering the Earth rotation effect. We introduce the SE N(3) group to jointly preintegrate IMU data and linear velocity measurements to preserve all the existing correlation within the preintegrated quantity. Field experiments using an Autonomous Underwater Vehicle (AUV) equipped with a DVL and a navigational grade IMU are provided and results are benchmarked against a commercial filter-based Inertial Navigation System (INS) to prove the effectiveness of our methodology., The work on this paper has been supported by the BITER-AUV Project (PID2020-114732RB-C33), the PLOME Project (PLEC2021-007525) and the PhD grant FPU19/03638 to Pau Vial. This work also received support from the EBCON Project (PID2020119244GB-I00) and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya. Open Access funding was provided thanks to the CRUE-CSIC agreement with IEEE., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

ACHORD: communication-aware multi-robot coordination with intermittent connectivity

  • Saboia Da Silva, Maira
  • Clack, Lillian
  • Thangavelu, Vivek
  • Edlund, Jeffrey
  • Kyohei, Otsu
  • Correa, Gustavo J
  • Varadharajan, Vivek Shankar
  • Santamaria Navarro, Àngel|||0000-0002-6328-1423
  • Touma, Thomas
  • Bouman, Amanda
  • Melikyan, Hovhannes
  • Pailevanian, Torkom
  • Kim, Sung-Kyun
  • Archanian, Avak
  • Vaquero, Tiago Stegun
  • Beltrame, Giovanni
  • Napp, Nils
  • Pessin, Gustavo
  • Agha-mohammadi, Ali-akbar
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, otherwise mission-critical data may be lost. In this paper, we describe and analyze ACHORD (Autonomous & Collaborative High-Bandwidth Operations with Radio Droppables), a multi-layer networking solution which tightly co-designs the network architecture and high-level decision-making for improved comms. ACHORD provides bandwidth prioritization and timely and reliable data transfer despite intermittent connectivity. Furthermore, it exposes low-layer networking metrics to the application layer to enable robots to autonomously monitor, map, and extend the network via droppable radios, as well as restore connectivity to improve collaborative exploration. We evaluate our solution with respect to the comms performance in several challenging underground environments including the DARPA SubT Finals competition environment. Our findings support the use of data stratification and flow control to improve bandwidth-usage., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Condition-based design of variable impedance controllers from user demonstrations

  • San Miguel Tello, Alberto|||0000-0001-5547-2564
  • Puig Cayuela, Vicenç|||0000-0002-6364-6429
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
This paper presents an approach to ensure conditions on Variable Impedance Controllers through the off-line tuning of the parameters involved in its description. In particular, we prove its application to term modulations defined by a Learning from Demonstration technique. This is performed through the assessment of conditions regarding safety and performance, which encompass heuristics and constraints in the form of Linear Matrix Inequalities. Latter ones allow to define a convex optimisation problem to analyse their fulfilment, and require a polytopic description of the VIC, in this case, obtained from its formulation as a discrete-time Linear Parameter Varying system. With respect to the current state-of-art, this approach only limits the term definition obtained by the Learning from Demonstration technique to be continuous and function of exogenous signals, i.e. external variables to the robot. Therefore, using a solution-search method, the most suitable set of parameters according to assessment criteria can be obtained. Using a 7-DoF KinovaGen3 manipulator, validation and comparison against solutions with relaxed conditions are performed. The method is applied to generate Variable Impedance Controllers for a pulley belt looping task, inspired by the Assembly Challenge for Industrial Robotics in World Robot Summit 2018, to reduce the exerted force with respect to a standard (constant) Impedance Controller. These controllers fulfil a set of safety constraints, namely stability, bounds on task variables and maximum response overshooting; and their performance is determined by the User Preference heuristic, which allows to intuitively define the desired compliant behaviour along the task. In the context of the task, this is used to generate new controllers for one-off modifications of the nominal belt looping task setup without new demonstrations., This work is partially supported by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR” under the project ROB-IN (PLEC2021-007859); and by MCIN/ AEI /10.13039/ 501100011033, Spain, under the project CHLOE-GRAPH (PID2020- 119244GB-I00). Authors also want to thank Adriá Colomé and Edoardo Caldarelli for their comments and help throughout this work., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi|||0000-0002-8673-6363
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020-119244GB-I00 funded by CIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

User interactions and negative examples to improve the learning of semantic rules in a cognitive exercise scenario

  • Suárez Hernández, Alejandro
  • Andriella, Antonio|||0000-0002-6641-6450
  • Torras, Carme|||0000-0002-2933-398X
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Enabling a robot to perform new tasks is a complex endeavor, usually beyond the reach of non-technical users. For this reason, research efforts that aim at empowering end-users to teach robots new abilities using intuitive modes of interaction are valuable. In this article, we present INtuitive PROgramming 2 (INPRO2), a learning framework that allows inferring planning actions from demonstrations given by a human teacher. INPRO2 operates in an assistive scenario, in which the robot may learn from a healthcare professional (a therapist or caregiver) new cognitive exercises that can be later administered to patients with cognitive impairment. INPRO2 features significant improvements over previous work, namely: (1) exploitation of negative examples; (2) proactive interaction with the teacher to ask questions about the legality of certain movements; and (3) learning goals in addition to legal actions. Through simulations, we show the performance of different proactive strategies for gathering negative examples. Real-world experiments with human teachers and a TIAGo robot are also presented to qualitatively illustrate INPRO2., The research leading to these results receives funding from the EU H2020 Programme under grant agreement ERC–2016–ADG–741930 (CLOTHILDE); by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR under the project COHERENT (PCI2020-120718-2); AA acknowledges the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801342 (Tecniospring INDUSTRY); ASH acknowledges the financial support of the Apadrina la Ciencia association and of the Ford Spanish branch., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Uncertainty quantification of first fix in a time-differenced carrier phase observation model

  • Cherfi, Hakim|||0009-0004-5893-9616
  • Lesouple, Julien
  • Solà Ortega, Joan|||0000-0002-2933-3381
  • Thevenon, Paul|||0000-0002-9600-1029
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), This paper presents an uncertainty quantification analysis of the first fix in a time-differenced carrier phase (TDCP) observation model. TDCP is a widely used method in GNSS-based odometry for precise positioning and displacement estimation. A key point in the TDCP modeling is the assumption that the GNSS receiver’s initial position is perfectly known, which is never exactly the case in real-world applications. This study assesses the impact of initial position errors on estimated displacement by formulating a correct TDCP model and a misspecified one, where the first position is not correct. Theoretical derivations provide a generic framework of estimation under the misspecified model and its associated mean squared error (MSE), as well as estimation performance bounds through the misspecified Cramér-Rao bound (MCRB) for the considered case. These theoretical considerations are then used to build an estimator of the receiver’s displacement, with comparisons to the MCRB for performance evaluation. Extensive simulations using realistic GNSS geometry assess the influence of a first-fix error under various conditions, including different time intervals, first-fix error norms, and first-fix error direction. As an example, it is shown that for the considered geometry, if a TDCP of $t_2 - t_1 = 1~\text{s}$ is built with an initial first fix error norm $\|\Delta \mathbf{r}_1\| = 10~\text{m}$, then it introduces an estimation of the displacement, with an error of norm equal to $1.3~\text{mm}$ at most. The results indicate that the displacement estimation error is linearly related to the initial position error and the time interval between observations, highlighting the importance of accurate first-fix estimation for reliable TDCP-based odometry. The findings contribute to highlighting the order of magnitude of errors on solutions as a function of the error on parameters., This work was supported in part by the EBCON Project under Grant PID2020119244GB-I00, and in part by the Consolidated Research Group RAIG under Grant 2021 SGR 00510 of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Teaching a robot where doors and drawers are and how to handle them

  • Cupec, Robert
  • Vidovic, Ivan
  • Šimundic, Valentin
  • Pejic, P.
  • Foix Salmerón, Sergi|||0000-0001-9249-6696
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state., This work has been partially supported by the Croatian Science Foundation under the project IP-2019-04-6819; by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Egomotion from event-based SNN optical flow

  • Tian, Yi|||0000-0002-8673-6363
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
We present a method for computing egomotion using event cameras with a pre-trained optical flow spiking neural network (SNN). To address the aperture problem encountered in the sparse and noisy normal flow of the initial SNN layers, our method includes a sliding-window bin-based pooling layer that computes a fused full flow estimate. To add robustness to noisy flow estimates, instead of computing the egomotion from vector averages, our method optimizes the intersection of constraints. The method also includes a RANSAC step to robustly deal with outlier flow estimates in the pooling layer. We validate our approach on both simulated and real scenes and compare our results favorably to the state-of-the-art methods. However, our method may be sensitive to datasets and motion speeds different from those used for training, limiting its generalizability., This work received support from projects EBCON (PID2020-119244GBI00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/ 501100011033 and by the "European Union NextGenerationEU/PRTR"; the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya; and by an FI AGAUR PhD grant to Yi Tian., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Event-IMU fusion strategies for faster-than-IMU estimation throughput

  • Chamorro Hernández, William Oswaldo
  • Solà Ortega, Joan|||0000-0002-2933-3381
  • Andrade-Cetto, Juan|||0000-0002-6354-8941
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This study presents new methods for integrating event data and IMU readings to achieve ultra-fast camera pose estimates. The conventional predict-with-IMU-correct-with-vision approach is no longer optimal because events can be generated much more rapidly than IMU data. Therefore, two novel fusion schemes are proposed, which combine constant velocity and constant acceleration prediction models with ultra-fast (10 kHz) event-based updates and slower 1 kHz IMU updates. The first scheme uses IMU data as instantaneous measurements of acceleration and angular rate, while the second scheme considers these measurements as the average within the IMU sampling time. To provide a basis for comparison, the traditional method that predicts motion using IMU data and updates the estimates with event-feature matching over a 1ms time window is also implemented. All models are designed as Kalman filter variants, which act as the tracker module of a PTAM system in a human-made indoor scenario and are subjected to stress experiments to evaluate their capabilities. The models are also compared against an event-only estimator and a frame-based visual-inertial approach. The findings demonstrate superior performance at a throughput that is 100 times faster than the state-of-the-art., This work received support from projects EBCON (PID2020-119244GB-I00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR”, and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Adaptive human-robot collaboration: evolutionary learning of action costs using an action outcome simulator

  • Izquierdo Badiola, Silvia
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
  • Rizzo, Carlos
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, One of the main challenges for successful human- robot collaborative applications lies in adapting the plan to the human agent’s changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents’ preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms., This work has been partially supported by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859), Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Event-IMU fusion strategies for faster-than-IMU estimation throughput

  • Chamorro Hernández, William Oswaldo
  • Solà Ortega, Joan
  • Andrade-Cetto, Juan
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., This study presents new methods for integrating event data and IMU readings to achieve ultra-fast camera pose estimates. The conventional predict-with-IMU-correct-with-vision approach is no longer optimal because events can be generated much more rapidly than IMU data. Therefore, two novel fusion schemes are proposed, which combine constant velocity and constant acceleration prediction models with ultra-fast (10 kHz) event-based updates and slower 1 kHz IMU updates. The first scheme uses IMU data as instantaneous measurements of acceleration and angular rate, while the second scheme considers these measurements as the average within the IMU sampling time. To provide a basis for comparison, the traditional method that predicts motion using IMU data and updates the estimates with event-feature matching over a 1ms time window is also implemented. All models are designed as Kalman filter variants, which act as the tracker module of a PTAM system in a human-made indoor scenario and are subjected to stress experiments to evaluate their capabilities. The models are also compared against an event-only estimator and a frame-based visual-inertial approach. The findings demonstrate superior performance at a throughput that is 100 times faster than the state-of-the-art., This work received support from projects EBCON (PID2020-119244GB-I00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR”, and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

On Lie group IMU and linear velocity preintegration for autonomous navigation considering the Earth rotation compensation

  • Vial Serrat, Pau
  • Solà Ortega, Joan
  • Palomeras Rovira, Narcís
  • Carreras Pérez, Marc
This work is licensed under a Creative Commons Attribution 4.0 License., Robot localization is a fundamental task in achieving true autonomy. Recently, many graph-based navigators have been proposed that combine an inertial measurement unit (IMU) with an exteroceptive sensor applying IMU preintegration to synchronize both sensors. IMUs are affected by biases that also have to be estimated. To increase the navigator robustness when faults appear on the perception system, IMU preintegration can be complemented with linear velocity measurements obtained from visual odometry, leg odometry, or a Doppler Velocity Log (DVL), depending on the robotic application. Moreover, higher grade IMUs are sensitive to the Earth rotation rate, which must be compensated in the preintegrated measurements. In this paper, we propose a general purpose preintegration methodology formulated on a compact Lie group to set motion constraints on Graph SLAM problems considering the Earth rotation effect. We introduce the SE N(3) group to jointly preintegrate IMU data and linear velocity measurements to preserve all the existing correlation within the preintegrated quantity. Field experiments using an Autonomous Underwater Vehicle (AUV) equipped with a DVL and a navigational grade IMU are provided and results are benchmarked against a commercial filter-based Inertial Navigation System (INS) to prove the effectiveness of our methodology., The work on this paper has been supported by the BITER-AUV Project (PID2020-114732RB-C33), the PLOME Project (PLEC2021-007525) and the PhD grant FPU19/03638 to Pau Vial. This work also received support from the EBCON Project (PID2020119244GB-I00) and the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya. Open Access funding was provided thanks to the CRUE-CSIC agreement with IEEE., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Uncertainty quantification of first fix in a time-differenced carrier phase observation model

  • Cherfi, Hakim
  • Lesouple, Julien
  • Solà Ortega, Joan
  • Thevenon, Paul
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/), This paper presents an uncertainty quantification analysis of the first fix in a time-differenced carrier phase (TDCP) observation model. TDCP is a widely used method in GNSS-based odometry for precise positioning and displacement estimation. A key point in the TDCP modeling is the assumption that the GNSS receiver’s initial position is perfectly known, which is never exactly the case in real-world applications. This study assesses the impact of initial position errors on estimated displacement by formulating a correct TDCP model and a misspecified one, where the first position is not correct. Theoretical derivations provide a generic framework of estimation under the misspecified model and its associated mean squared error (MSE), as well as estimation performance bounds through the misspecified Cramér-Rao bound (MCRB) for the considered case. These theoretical considerations are then used to build an estimator of the receiver’s displacement, with comparisons to the MCRB for performance evaluation. Extensive simulations using realistic GNSS geometry assess the influence of a first-fix error under various conditions, including different time intervals, first-fix error norms, and first-fix error direction. As an example, it is shown that for the considered geometry, if a TDCP of $t_2 - t_1 = 1~\text{s}$ is built with an initial first fix error norm $\|\Delta \mathbf{r}_1\| = 10~\text{m}$, then it introduces an estimation of the displacement, with an error of norm equal to $1.3~\text{mm}$ at most. The results indicate that the displacement estimation error is linearly related to the initial position error and the time interval between observations, highlighting the importance of accurate first-fix estimation for reliable TDCP-based odometry. The findings contribute to highlighting the order of magnitude of errors on solutions as a function of the error on parameters., This work was supported in part by the EBCON Project under Grant PID2020119244GB-I00, and in part by the Consolidated Research Group RAIG under Grant 2021 SGR 00510 of the Departament de Recerca i Universitats de la Generalitat de Catalunya., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Event transformer FlowNet for optical flow estimation

  • Tian, Yi
  • Andrade-Cetto, Juan
Event cameras are bioinspired sensors that produce asynchronous and sparse streams of events at image locations where intensity change is detected. They can detect fast motion with low latency, high dynamic range, and low power consumption. Over the past decade, efforts have been conducted in developing solutions with event cameras for robotics applications. In this work, we address their use for fast and robust computation of optical flow. We present ET-FlowNet, a hybrid RNN-ViT architecture for optical flow estimation. Visual transformers (ViTs) are ideal candidates for the learning of global context in visual tasks, and we argue that rigid body motion is a prime case for the use of ViTs since long-range dependencies in the image hold during rigid body motion. We perform end-to-end training with self-supervised learning method. Our results show comparable and in some cases exceeding performance with state-of-the-art coarse-to-fine event-based optical flow estimation., This work was supported by projects EBSLAM DPI2017-89564-P and EBCON PID2020-119244GB-I00 funded by CIN/AEI/10.13039/501100011033 and by an FI AGAUR PhD grant to Yi Tian., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Adaptive human-robot collaboration: evolutionary learning of action costs using an action outcome simulator

  • Izquierdo Badiola, Silvia
  • Alenyà Ribas, Guillem
  • Rizzo, Carlos
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, One of the main challenges for successful human- robot collaborative applications lies in adapting the plan to the human agent’s changing state and preferences. A promising solution is to bridge the gap between agent modelling and AI task planning, which can be done by integrating the agent state as action costs in the task planning domain. This allows for the plan to be adapted to different partners, by influencing the action allocation. The difficulty then lies in setting appropriate action costs. This paper presents a novel framework to learn a set of planning action costs considering the preferred actions for an agent based on their state. An evolutionary optimisation algorithm is used for this purpose, and an action outcome simulator is developed to act as the black-box function, based on both an agent model and an action type model. This addresses the challenge of collecting data in HRC real-world scenarios, accelerating the learning for posterior fine-tuning in real applications. The coherence of the models and the simulator is proven through a conducted survey, and the learning algorithm is shown to learn appropriate action costs, producing plans that satisfy both the agents’ preferences and the prioritised plan requisites. The resulting system is a generic learning framework integrating components that can be easily extended to a wide range of applications, models and planning formalisms., This work has been partially supported by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020- 119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Teaching a robot where doors and drawers are and how to handle them

  • Cupec, Robert
  • Vidovic, Ivan
  • Šimundic, Valentin
  • Pejic, P.
  • Foix Salmerón, Sergi
  • Alenyà Ribas, Guillem
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., We address the problem of teaching a service robot to detect doors and drawers in indoor environments. We propose a robust and accurate method in which a human demonstrates to the robot how to open doors and drawers that the robot is expected to operate in its future use. The proposed algorithm creates a model of a door or drawer from a sequence of RGB-D images and inserts it into an environment map. The model contains information about the size of the door panel or drawer front, as well as the position and orientation of the joint axis. This augmented environment map is then used by the robot to detect the target object in its environment and estimate its state., This work has been partially supported by the Croatian Science Foundation under the project IP-2019-04-6819; by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union (EU) NextGenerationEU/PRTR under the project ROB-IN (PLEC2021-007859)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Egomotion from event-based SNN optical flow

  • Tian, Yi
  • Andrade-Cetto, Juan
We present a method for computing egomotion using event cameras with a pre-trained optical flow spiking neural network (SNN). To address the aperture problem encountered in the sparse and noisy normal flow of the initial SNN layers, our method includes a sliding-window bin-based pooling layer that computes a fused full flow estimate. To add robustness to noisy flow estimates, instead of computing the egomotion from vector averages, our method optimizes the intersection of constraints. The method also includes a RANSAC step to robustly deal with outlier flow estimates in the pooling layer. We validate our approach on both simulated and real scenes and compare our results favorably to the state-of-the-art methods. However, our method may be sensitive to datasets and motion speeds different from those used for training, limiting its generalizability., This work received support from projects EBCON (PID2020-119244GBI00) and AUDEL (TED2021-131759A-I00) funded by MCIN/ AEI/ 10.13039/ 501100011033 and by the "European Union NextGenerationEU/PRTR"; the Consolidated Research Group RAIG (2021 SGR 00510) of the Departament de Recerca i Universitats de la Generalitat de Catalunya; and by an FI AGAUR PhD grant to Yi Tian., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Condition-based design of variable impedance controllers from user demonstrations

  • San Miguel Tello, Alberto
  • Puig Cayuela, Vicenç
  • Alenyà Ribas, Guillem
This paper presents an approach to ensure conditions on Variable Impedance Controllers through the off-line tuning of the parameters involved in its description. In particular, we prove its application to term modulations defined by a Learning from Demonstration technique. This is performed through the assessment of conditions regarding safety and performance, which encompass heuristics and constraints in the form of Linear Matrix Inequalities. Latter ones allow to define a convex optimisation problem to analyse their fulfilment, and require a polytopic description of the VIC, in this case, obtained from its formulation as a discrete-time Linear Parameter Varying system. With respect to the current state-of-art, this approach only limits the term definition obtained by the Learning from Demonstration technique to be continuous and function of exogenous signals, i.e. external variables to the robot. Therefore, using a solution-search method, the most suitable set of parameters according to assessment criteria can be obtained. Using a 7-DoF KinovaGen3 manipulator, validation and comparison against solutions with relaxed conditions are performed. The method is applied to generate Variable Impedance Controllers for a pulley belt looping task, inspired by the Assembly Challenge for Industrial Robotics in World Robot Summit 2018, to reduce the exerted force with respect to a standard (constant) Impedance Controller. These controllers fulfil a set of safety constraints, namely stability, bounds on task variables and maximum response overshooting; and their performance is determined by the User Preference heuristic, which allows to intuitively define the desired compliant behaviour along the task. In the context of the task, this is used to generate new controllers for one-off modifications of the nominal belt looping task setup without new demonstrations., This work is partially supported by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR” under the project ROB-IN (PLEC2021-007859); and by MCIN/ AEI /10.13039/ 501100011033, Spain, under the project CHLOE-GRAPH (PID2020- 119244GB-I00). Authors also want to thank Adriá Colomé and Edoardo Caldarelli for their comments and help throughout this work., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

User interactions and negative examples to improve the learning of semantic rules in a cognitive exercise scenario

  • Suárez Hernández, Alejandro
  • Andriella, Antonio
  • Torras, Carme
  • Alenyà Ribas, Guillem
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Enabling a robot to perform new tasks is a complex endeavor, usually beyond the reach of non-technical users. For this reason, research efforts that aim at empowering end-users to teach robots new abilities using intuitive modes of interaction are valuable. In this article, we present INtuitive PROgramming 2 (INPRO2), a learning framework that allows inferring planning actions from demonstrations given by a human teacher. INPRO2 operates in an assistive scenario, in which the robot may learn from a healthcare professional (a therapist or caregiver) new cognitive exercises that can be later administered to patients with cognitive impairment. INPRO2 features significant improvements over previous work, namely: (1) exploitation of negative examples; (2) proactive interaction with the teacher to ask questions about the legality of certain movements; and (3) learning goals in addition to legal actions. Through simulations, we show the performance of different proactive strategies for gathering negative examples. Real-world experiments with human teachers and a TIAGo robot are also presented to qualitatively illustrate INPRO2., The research leading to these results receives funding from the EU H2020 Programme under grant agreement ERC–2016–ADG–741930 (CLOTHILDE); by MCIN/ AEI /10.13039/501100011033 under the project CHLOE-GRAPH (PID2020-119244GB-I00); by MCIN/ AEI /10.13039/501100011033 and by the ”European Union NextGenerationEU/PRTR under the project COHERENT (PCI2020-120718-2); AA acknowledges the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801342 (Tecniospring INDUSTRY); ASH acknowledges the financial support of the Apadrina la Ciencia association and of the Ford Spanish branch., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

ACHORD: communication-aware multi-robot coordination with intermittent connectivity

  • Saboia Da Silva, Maira
  • Clack, Lillian
  • Thangavelu, Vivek
  • Edlund, Jeffrey
  • Kyohei, Otsu
  • Correa, Gustavo J
  • Varadharajan, Vivek Shankar
  • Santamaria Navarro, Àngel
  • Touma, Thomas
  • Bouman, Amanda
  • Melikyan, Hovhannes
  • Pailevanian, Torkom
  • Kim, Sung-Kyun
  • Archanian, Avak
  • Vaquero, Tiago Stegun
  • Beltrame, Giovanni
  • Napp, Nils
  • Pessin, Gustavo
  • Agha-mohammadi, Ali-akbar
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, otherwise mission-critical data may be lost. In this paper, we describe and analyze ACHORD (Autonomous & Collaborative High-Bandwidth Operations with Radio Droppables), a multi-layer networking solution which tightly co-designs the network architecture and high-level decision-making for improved comms. ACHORD provides bandwidth prioritization and timely and reliable data transfer despite intermittent connectivity. Furthermore, it exposes low-layer networking metrics to the application layer to enable robots to autonomously monitor, map, and extend the network via droppable radios, as well as restore connectivity to improve collaborative exploration. We evaluate our solution with respect to the comms performance in several challenging underground environments including the DARPA SubT Finals competition environment. Our findings support the use of data stratification and flow control to improve bandwidth-usage., Peer Reviewed, Postprint (author's final draft)