Pasar al contenido principal

MEJORAS EN COMPRENSION AUTOMATICA DE ESCENAS MEDIANTE MODALIDADES MULTIPLES DE SENSORES Y PERCEPCION ACTIVA

PID2021-125514NB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD DE ZARAGOZA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 15
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Comprensión automática de escenas en imágenes de entornos submarinos, Visual scene understanding in underwater environments

  • Borja, Cesar
  • Murillo, Ana C.
[EN] The utilization of Autonomous Underwater Vehicles (AUVs) represents a significant advancement in the field of seabed monitoring. However, image processing of data acquired from AUVs presents a unique challenge due to the inherent properties of the underwater environment, such as light attenuation and water turbidity. This work investigates techniques to enhance underwater scene understanding from monocular images. The proposed system leverages existing deep learning methods in conjunction with simple image processing algorithms, eliminating the need for additional supervised training. The system studies the combinatio of a pre-trained deep learning model, for depth estimation from monocular images, with the proposed algorithm to distinguish water regions from the rest of the scene elements. The presented study includes comprehensive comparison of various system alternatives and configuration options. The experimental validation shows how the presented system obtains richer segmentation results compared to baseline algorithms. Notably, the proposed system facilitates the accurate segmentation of water regions and enables the detection of other objects of interest, including suspended elements in the water, which can potentially correspond to fish or other mobile obstacles., [ES] La utilización de vehículos submarinos autónomos (AUV) representa un avance significativo en el campo de la monitorización del fondo marino. Sin embargo, el procesamiento de imágenes de datos adquiridos desde AUVs presenta un desafío único debido a las propiedades inherentes del entorno submarino, como la atenuación de la luz y la turbidez del agua. Este trabajo investiga técnicas para mejorar la comprensión automática del contenido de escenas submarinas a partir de imágenes monoculares. El sistema propuesto aprovecha modelos de aprendizaje profundo existentes junto con algoritmos simples de procesamiento de imágenes, eliminando la necesidad de entrenamiento supervisado adicional. El sistema estudia la combinación de un modelo de aprendizaje profundo pre-entrenado para la estimación de profundidad a partir de imágenes monoculares, con el algoritmo propuesto para distinguir regiones de agua del resto de elementos de la escena. El estudio presentado incluye una comparación detallada de la influencia en el resultado de varias alternativas y opciones de configuración del sistema. La validación experimental muestra cómo el sistema presentado obtiene resultados de segmentación más ricos en comparación con los algoritmos existentes utilizados como referencia. En particular, el sistema propuesto facilita la segmentación precisa de regiones de agua y facilita la detección de otros objetos de interés, incluyendo elementos suspendidos en el agua, que potencialmente pueden corresponder a peces u otros obstáculos móviles., Este trabajo ha sido financiado parcialmente por FEDER/Ministerio de Ciencia, Innovación y Universidades – Agencia Estatal de Investigación proyecto PID2021-125514NB-I00, y DGA T45 23R/FSE. Los autores agradecen el apoyo del Laboratorio VISEAON, de la Universidad de Haifa, Israel, a lo largo del trabajo. Además, el proyecto se ha desarrollado en el marco de la beca para Prácticas de Estudiantes de Grado Universitario en el marco del TFG del Instituto de Investigación en Ingeniería de Aragón (I3A) de la Universidad de Zaragoza.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Distributed outer approximation of the intersection of ellipsoids

  • Aldana-López, Rodrigo
  • Sebastián, Eduardo
  • Aragüés, Rosario
  • Montijano, Eduardo
  • Sagüés, Carlos
The outer Löwner-John method is widely used in sensor fusion applications to find the smallest ellipsoid that can approximate the intersection of a set of ellipsoids, described by positive definite covariance matrices modeling the quality of each sensor. We propose a distributed algorithm to solve this problem when these matrices are defined over the network’s nodes. This is of particular significance as it is the first decentralized algorithm capable of computing the covariance intersection ellipsoid by combining information from the entire network using only local interactions. The solution is based on a reformulation of the centralized problem, leading to a local protocol based on exact dynamic consensus tools. After reaching consensus, the protocol converges to an outer Löwner-John ellipsoid in finite time, and to the global optimum asymptotically. Formal convergence analysis and numerical experiments are provided to validate the proposal’s advantages.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Event transformer+. A multi-purpose solution for efficient event data processing

  • Sabater, Alberto
  • Montesano, Luis
  • Murillo, Ana C.
Event cameras record sparse illumination changes with high temporal resolution and high dynamic range. Thanks to their sparse recording and low consumption, they are increasingly used in applications such as AR/VR and autonomous driving. Current top-performing methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms, while event-aware methods do not perform as well. We propose Event Transformer++ , that improves our seminal work EvT with a refined patch-based event representation and a more robust backbone to achieve more accurate results, while still benefiting from event-data sparsity to increase its efficiency. Additionally, we show how our system can work with different data modalities and propose specific output heads, for event-stream classification (i.e. action recognition) and per-pixel predictions (dense depth estimation). Evaluation results show better performance to the state-of-the-art while requiring minimal computation resources, both on GPU and CPU.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Contribución de congreso (ConferenceOutput)

Robust fusion for bayesian semantic mapping

  • Morilla-Cabello, David
  • Mur-Labadia, Lorenzo
  • Martinez-Cantin, Ruben
  • Montijano, Eduardo
The integration of semantic information in a map allows robots to understand better their environment and make high-level decisions. In the last few years, neural networks have shown enormous progress in their perception capabilities. However, when fusing multiple observations from a neural network in a semantic map, its inherent overconfidence with unknown data gives too much weight to the outliers and decreases the robustness. To mitigate this issue we propose a novel robust fusion method to combine multiple Bayesian semantic predictions. Our method uses the uncertainty estimation provided by a Bayesian neural network to calibrate the way in which the measurements are fused. This is done by regularizing the observations to mitigate the problem of overconfident outlier predictions and using the epistemic uncertainty to weigh their influence in the fusion, resulting in a different formulation of the probability distributions. We validate our robust fusion strategy by performing experiments on photo-realistic simulated environments and real scenes. In both cases, we use a network trained on different data to expose the model to varying data distributions. The results show that considering the model's uncertainty and regularizing the probability distribution of the observations distribution results in a better semantic segmentation performance and more robustness to outliers, compared with other methods. Video - https://youtu.be/5xVGm7z9c-0
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Accelerated Alternating Direction Method of Multipliers Gradient Tracking for Distributed Optimization

  • Sebastián, Eduardo
  • Franceschelli, Mauro
  • Gasparri, Andrea
  • Montijano, Eduardo
  • Sagüés, Carlos
This letter presents a novel accelerated distributed algorithm for unconstrained consensus optimization over static undirected networks. The proposed algorithm combines the benefits of acceleration from momentum, the robustness of the alternating direction method of multipliers, and the computational efficiency of gradient tracking to surpass existing state-of-the-art methods in convergence speed, while preserving their computational and communication cost. First, we prove that, by applying momentum on the average dynamic consensus protocol over the estimates and gradient, we can study the algorithm as an interconnection of two singularly perturbed systems: the outer system connects the consensus variables and the optimization variables, and the inner system connects the estimates of the optimum and the auxiliary optimization variables. Next, we prove that, by adding momentum to the auxiliary dynamics, our algorithm always achieves faster convergence than the achievable linear convergence rate for the non-accelerated alternating direction method of multipliers gradient tracking algorithm case. Through simulations, we numerically show that our accelerated algorithm surpasses the existing accelerated and non-accelerated distributed consensus first-order optimization protocols in convergence speed.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Comprensión automática de escenas en imágenes de entornos submarinos

  • Borja, César
  • Murillo, Ana C.
La utilización de vehículos submarinos autónomos (AUV) representa un avance significativo en el campo de la monitorización del fondo marino. Sin embargo, el procesamiento de imágenes de datos adquiridos desde AUVs presenta un desafío único debido a las propiedades inherentes del entorno submarino, como la atenuación de la luz y la turbidez del agua. Este trabajo investiga técnicas para mejorar la comprensión automática del contenido de escenas submarinas a partir de imágenes monoculares. El sistema propuesto aprovecha modelos de aprendizaje profundo existentes junto con algoritmos simples de procesamiento de imágenes, eliminando la necesidad de entrenamiento supervisado adicional. El sistema estudia la combinación de un modelo de aprendizaje profundo pre-entrenado para la estimación de profundidad a partir de imágenes monoculares, con el algoritmo propuesto para distinguir regiones de agua del resto de elementos de la escena. El estudio presentado incluye una comparación detallada de la influencia en el resultado de varias alternativas y opciones de configuración del sistema. La validación experimental muestra cómo el sistema presentado obtiene resultados de segmentación más ricos en comparación con los algoritmos existentes utilizados como referencia. En particular, el sistema propuesto facilita la segmentación precisa de regiones de agua y facilita la detección de otros objetos de interés, incluyendo elementos suspendidos en el agua, que potencialmente pueden corresponder a peces u otros obstáculos móviles.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

On Multi-Robot Path Planning based on Petri Net Models and LTL Specifications

  • Hustiu, Sofia
  • Mahulea, Cristian
  • Kloetzer, Marius
  • Lesage, Jean-Jacques
This paper proposes a method exploiting the advantages of Petri net (PN) and Büchi automata models, by joining them in a newly defined Composed Petri net representation. Based on the latter model, collision-free trajectories are computed for a team of robots. The path planning algorithm is divided into two steps: computing a solution in a reduced PN model, and projecting it to the PN assigned to the environment. The results, given by a set of Mixed Integer Linear Programming (MILP) problems, yield lower computational complexity when compared with previous approaches.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

AVOCADO: Adaptive Optimal Collision Avoidance Driven by Opinion

  • Martinez-Baselga, Diego
  • Sebastián, Eduardo
  • Montijano, Eduardo
  • Riazuelo, Luis
  • Sagüés, Carlos
  • Montano, Luis
We present AdaptiVe Optimal Collision Avoidance Driven by Opinion (AVOCADO), a novel navigation approach to address holonomic robot collision avoidance when the robot does not know how cooperative the other agents in the environment are. AVOCADO departs from a velocity obstacle's (VO) formulation akin to the optimal reciprocal collision avoidance method. However, instead of assuming reciprocity, it poses an adaptive control problem to adapt to the cooperation level of other robots and agents in real time. This is achieved through a novel nonlinear opinion dynamics design that relies solely on sensor observations. As a by-product, we leverage tools from the opinion dynamics formulation to naturally avoid the deadlocks in geometrically symmetric scenarios that typically suffer VO-based planners. Extensive numerical simulations show that AVOCADO surpasses existing motion planners in mixed cooperative/noncooperative navigation environments in terms of success rate, time to goal and computational time. In addition, we conduct multiple real experiments that verify that AVOCADO is able to avoid collisions in environments crowded with other robots and humans.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation

  • Casao, Sara
  • Peña, Fernando
  • Sabater, Alberto
  • Castillón, Rosa
  • Suárez, Darío
  • Montijano, Eduardo
  • Murillo, Ana C.
The increase in non-biodegradable waste is a worldwide concern. Recycling facilities play a crucial role, but their automation is hindered by the complex characteristics of waste recycling lines like clutter or object deformation. In addition, the lack of publicly available labeled data for these environments makes developing robust perception systems challenging. Our work explores the benefits of multimodal perception for object segmentation in real waste management scenarios. First, we present SpectralWaste, the first dataset collected from an operational plastic waste sorting facility that provides synchronized hyperspectral and conventional RGB images. This dataset contains labels for several categories of objects that commonly appear in sorting plants and need to be detected and separated from the main trash flow for several reasons, such as security in the management line or reuse. Additionally, we propose a pipeline employing different object segmentation architectures and evaluate the alternatives on our dataset, conducting an extensive analysis for both multimodal and unimodal alternatives. Our evaluation pays special attention to efficiency and suitability for real-time processing and demonstrates how hyperspectral imaging can bring a boost to RGB-only perception in these realistic industrial settings without much computational overhead.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Physics-Informed Multiagent Reinforcement Learning for Distributed Multirobot Problems

  • Sebastián, Eduardo
  • Duong, Thai
  • Atanasov, Nikolay
  • Montijano, Eduardo
  • Sagüés, Carlos
The networked nature of multirobot systems presents challenges in the context of multiagent reinforcement learning. Centralized control policies do not scale with increasing numbers of robots, whereas independent control policies do not exploit the information provided by other robots, exhibiting poor performance in cooperative-competitive tasks. In this work, we propose a physics-informed reinforcement learning approach able to learn distributed multirobot control policies that are both scalable and make use of all the available information to each robot. Our approach has three key characteristics. First, it imposes a port-Hamiltonian structure on the policy representation, respecting energy conservation properties of physical robot systems and the networked nature of robot team interactions. Second, it uses self-attention to ensure a sparse policy representation able to handle time-varying information at each robot from the interaction graph. Third, we present a soft actor–critic reinforcement learning algorithm parameterized by our self-attention port-Hamiltonian control policy, which accounts for the correlation among robots during training while overcoming the need of value function factorization. Extensive simulations in different multirobot scenarios demonstrate the success of the proposed approach, surpassing previous multirobot reinforcement learning solutions in scalability, while achieving similar or superior performance (with averaged cumulative reward up to ×2 greater than the state-of-the-art with robot teams ×6 larger than the number of robots at training time). We also validate our approach on multiple real robots in the Georgia Tech Robotarium under imperfect communication, demonstrating zero-shot sim-to-real transfer and scalability across number of robots.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Automated overview of complete endoscopies with unsupervised learned descriptors

  • Barbed, O. León
  • Azagra, Pablo
  • Plo, Juan
  • Murillo, Ana C.
Purpose We aim to automate the initial analysis of complete endoscopy videos, identifying the sparse relevant content. This facilitates long procedure recording understanding, reduces the clinicians’ review time, and facilitates downstream tasks such as video summarization, event detection, and 3D reconstruction. Methods Our approach extracts endoscopic video frame representations with a learned embedding model. These descriptors are clustered to find visual patterns in the procedure, identifying key scene types (surgery, clear visibility frames, etc.) and enabling segmentation into informative and non-informative video parts. Results Evaluation on complete colonoscopy videos presents good performance identifying surgery segments and different visibility conditions. The method produces structured overviews that separate useful segments from irrelevant ones. We illustrate its suitability and benefits as preprocessing for other downstream tasks, such as 3D reconstruction or video summarization. Conclusion Our approach enables automated endoscopy overview generation, helping the clinicians focus on the relevant video content such as good visibility sections and surgery actions. The presented work facilitates faster recording reviewing for clinicians and effective video preprocessing for downstream tasks.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Probabilistic timed Petri nets for clinical pathway design and analysis: a case study

  • Le Moigne, Manon
  • Mahulea, Cristian
  • Faraut, Gregory
  • Bernardi, Simona
  • Albareda, Jorge
  • Castán, Lidia
The COVID-19 pandemic underscored the need for efficient hospital resource management and standardized patient care. Clinical pathways, structured plans for managing specific conditions, are critical but challenging to design and update. This paper introduces a methodology combining pattern mining and Probabilistic Timed Petri Nets (PTPN) to model, simulate, and evaluate clinical pathways, incorporating probabilistic transitions and continuous-time distributions for activity durations. In collaboration with medical professionals, the proposed methodology has been successfully applied to develop a clinical pathway for anterior cruciate ligament (ACL) ruptures using real data. Iterative refinement addressed data inconsistencies, producing a robust PTPN model that optimizes patient care and resource allocation. Validation using the CASAS database demonstrate the adaptability of the method to different healthcare contexts.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Temporal video segmentation with natural language using text–video cross attention and Bayesian order-priors

  • Plou, Carlos
  • Mur-Labadia, Lorenzo
  • Guerrero, Jose J.
  • Martinez-Cantin, Ruben
  • Murillo, Ana C.
Video is a crucial perception component in both robotics and wearable devices, two key technologies to enable innovative assistive applications, such as navigation and procedure execution assistance tools. Video understanding tasks are essential to enable these systems to interpret and execute complex instructions in realworld environments. One such task is step grounding, which involves identifying the temporal boundaries of activities based on natural language descriptions in long, untrimmed videos. This paper introduces BayesianVSLNet, a probabilistic formulation of step grounding that predicts a likelihood distribution over segments and refines it through Bayesian inference with temporal-order priors. These priors disambiguate cyclic and repeated actions that frequently appear in procedural tasks, enabling precise step localization in long videos. Our evaluations demonstrate superior performance over existing methods, achieving state-of-the-art results in the Ego4D Goal-Step dataset, winning the Goal Step challenge at the EgoVis 2024 CVPR. Furthermore, experiments on additional benchmarks confirm the generality of our approach beyond Ego4D. In addition, we present qualitative results in a real-world robotics scenario, illustrating the potential of this task to improve human–robot interaction in practical application.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

EventSleep2: Sleep activity recognition on complete night sleep recordings with an event camera

  • Gallego, Nerea
  • Plou, Carlos
  • Marcos, Miguel
  • Urcola, Pablo
  • Montesano, Luis
  • Montijano, Eduardo
  • Martinez-Cantin, Ruben
  • Murillo, Ana C.
Sleep is fundamental to health, and society is more and more aware of the impact and relevance of sleep disorders. Traditional diagnostic methods, like polysomnography, are intrusive and resource-intensive. Instead, research is focusing on developing novel, less intrusive or portable methods that combine intelligent sensors with activity recognition for diagnosis support and scoring. Event cameras offer a promising alternative for automated, in-home sleep activity recognition due to their excellent low-light performance and low power consumption. This work introduces EventSleep2-data, a significant extension to the EventSleep dataset, featuring 10 complete night recordings (around 7 h each) of volunteers sleeping in their homes. Unlike the original short and controlled recordings, this new dataset captures natural, full-night sleep sessions under realistic conditions. This new data incorporates challenging real-world scene variations, an efficient movement-triggered sparse data recording pipeline, and synchronized 2-channel EEG data for a subset of recordings. We also present EventSleep2-net, a novel event-based sleep activity recognition approach with a dual-head architecture to simultaneously analyze motion classes and static poses. The model is specifically designed to handle the motion-triggered, sparse nature of complete night recordings. Unlike the original EventSleep architecture, EventSleep2-net can predict both movement and static poses even during long periods with no events. We demonstrate state-of-the-art performance on both EventSleep1-data, the original dataset, and EventSleep2-data, with comprehensive ablation studies validating our design decisions. Together, EventSleep2-data and EventSleep2-net overcome the limitations of the previous setup and enable continuous, full-night analysis for real-world sleep monitoring, significantly advancing the potential of event-based vision for sleep disorder studies.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

A self-adaptive gallery construction method for open-world person re-identification

  • Casao, Sara
  • Azagra, Pablo
  • Murillo, Ana C.
  • Montijano, Eduardo
Person re-identification, or simply re-id, is the task of identifying again a person who has been seen in the past by a perception system. Multiple robotic applications, such as tracking or navigate-and-seek, use re-identification systems to perform their tasks. To solve the re-id problem, a common practice consists in using a gallery with relevant information about the people already observed. The construction of this gallery is a costly process, typically performed offline and only once because of the problems associated with labeling and storing new data as they arrive in the system. The resulting galleries from this process are static and do not acquire new knowledge from the scene, which is a limitation of the current re-id systems to work for open-world applications. Different from previous work, we overcome this limitation by presenting an unsupervised approach to automatically identify new people and incrementally build a gallery for open-world re-id that adapts prior knowledge with new information on a continuous basis. Our approach performs a comparison between the current person models and new unlabeled data to dynamically expand the gallery with new identities. We process the incoming information to maintain a small representative model of each person by exploiting concepts of information theory. The uncertainty and diversity of the new samples are analyzed to define which ones should be incorporated into the gallery. Experimental evaluation in challenging benchmarks includes an ablation study of the proposed framework, the assessment of different data selection algorithms that demonstrate the benefits of our approach, and a comparative analysis of the obtained results with other unsupervised and semi-supervised re-id methods.