Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 3
Encontrada(s) 1 página(s)

Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Random rotational embedding Bayesian optimization for human-in-the-loop personalized music generation

  • Marcos, Miguel
  • Mur-Labadia, Lorenzo
  • Martinez-Cantin, Ruben
Generative deep learning models, such as those used for music generation, can produce a wide variety of results based on perturbations of random points in their latent space. User preferences can be incorporated in the generative process by replacing this random sampling with a personalized query. Bayesian optimization, a sample-efficient nonlinear optimization method, is the gold standard for human-in-the-loop optimization problems, such as finding this query. In this paper, we present random rotational embedding Bayesian optimization (ROMBO). This novel method can efficiently sample and optimize high-dimensional spaces with rotational symmetries, like the Gaussian latent spaces found in generative models. ROMBO works by embedding a low-dimensional Gaussian search space into a high-dimensional one through random rotations. Our method outperforms several baselines, including other high-dimensional Bayesian optimization variants. We evaluate our algorithm through a music generation task. Our evaluation includes both simulated experiments and real user feedback. Our results show that ROMBO can perform efficient personalization of a generative deep learning model. The main contributions of our paper are: we introduce a novel embedding strategy for Bayesian optimization in high-dimensional Gaussian sample spaces; achieve a consistently better performance throughout optimization with respect to baselines, with a final loss reduction of 16%-31% in simulation; and complement our simulated evaluations with a study with human volunteers (n = 16). Users working with our music generation pipeline find new favorite pieces 40% more often, 16% faster, and spend 18% less time on pieces they dislike than when randomly querying the model. These results, along with a final survey, demonstrate great performance and satisfaction, even among users with particular tastes. © 2025 Marcos et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Temporal video segmentation with natural language using text–video cross attention and Bayesian order-priors

  • Plou, Carlos
  • Mur-Labadia, Lorenzo
  • Guerrero, Jose J.
  • Martinez-Cantin, Ruben
  • Murillo, Ana C.
Video is a crucial perception component in both robotics and wearable devices, two key technologies to enable innovative assistive applications, such as navigation and procedure execution assistance tools. Video understanding tasks are essential to enable these systems to interpret and execute complex instructions in realworld environments. One such task is step grounding, which involves identifying the temporal boundaries of activities based on natural language descriptions in long, untrimmed videos. This paper introduces BayesianVSLNet, a probabilistic formulation of step grounding that predicts a likelihood distribution over segments and refines it through Bayesian inference with temporal-order priors. These priors disambiguate cyclic and repeated actions that frequently appear in procedural tasks, enabling precise step localization in long videos. Our evaluations demonstrate superior performance over existing methods, achieving state-of-the-art results in the Ego4D Goal-Step dataset, winning the Goal Step challenge at the EgoVis 2024 CVPR. Furthermore, experiments on additional benchmarks confirm the generality of our approach beyond Ego4D. In addition, we present qualitative results in a real-world robotics scenario, illustrating the potential of this task to improve human–robot interaction in practical application.
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

EventSleep2: Sleep activity recognition on complete night sleep recordings with an event camera

  • Gallego, Nerea
  • Plou, Carlos
  • Marcos, Miguel
  • Urcola, Pablo
  • Montesano, Luis
  • Montijano, Eduardo
  • Martinez-Cantin, Ruben
  • Murillo, Ana C.
Sleep is fundamental to health, and society is more and more aware of the impact and relevance of sleep disorders. Traditional diagnostic methods, like polysomnography, are intrusive and resource-intensive. Instead, research is focusing on developing novel, less intrusive or portable methods that combine intelligent sensors with activity recognition for diagnosis support and scoring. Event cameras offer a promising alternative for automated, in-home sleep activity recognition due to their excellent low-light performance and low power consumption. This work introduces EventSleep2-data, a significant extension to the EventSleep dataset, featuring 10 complete night recordings (around 7 h each) of volunteers sleeping in their homes. Unlike the original short and controlled recordings, this new dataset captures natural, full-night sleep sessions under realistic conditions. This new data incorporates challenging real-world scene variations, an efficient movement-triggered sparse data recording pipeline, and synchronized 2-channel EEG data for a subset of recordings. We also present EventSleep2-net, a novel event-based sleep activity recognition approach with a dual-head architecture to simultaneously analyze motion classes and static poses. The model is specifically designed to handle the motion-triggered, sparse nature of complete night recordings. Unlike the original EventSleep architecture, EventSleep2-net can predict both movement and static poses even during long periods with no events. We demonstrate state-of-the-art performance on both EventSleep1-data, the original dataset, and EventSleep2-data, with comprehensive ablation studies validating our design decisions. Together, EventSleep2-data and EventSleep2-net overcome the limitations of the previous setup and enable continuous, full-night analysis for real-world sleep monitoring, significantly advancing the potential of event-based vision for sleep disorder studies.