Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 11
Encontrada(s) 1 página(s)

RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Multimodal pain assessment with transformers

  • Benavent-Lledó, Manuel
  • López Valle, María Dolores
  • Ortiz Pérez, David
  • Mulero Pérez, David
  • Garcia-Rodriguez, Jose
  • Psarrou, Alexandra
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
Pain assessment is a critical challenge in healthcare, requiring accurate and objective measurement to enhance patient care. Traditional methods rely on subjective self-reporting, which lacks reliability, particularly for patients with communication difficulties. This work presents PainFusion+, a multimodal transformer architecture for pain assessment that integrates physiological signals and facial expressions to improve pain assessment accuracy. For physiological signals, our approach first employs convolutional neural networks to extract local patterns from short signal fragments, capturing essential pain-related features. These localized representations are then processed by a transformer encoder, which models long-range dependencies to form a comprehensive global representation. For facial video data, we leverage a frozen video transformer to extract expressive features without requiring fine-tuning, significantly reducing computational costs. Finally, both feature spaces are fused using a transformer encoder, allowing effective cross-modal learning. Experiments on publicly available datasets demonstrate that PainFusion+ outperforms existing models. In biomedical signal processing, our method achieves over a 16 % improvement in accuracy. For multimodal pain estimation, it achieves 35.40 % accuracy on the BioVid dataset, setting a new state-of-the-art benchmark.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Optimizing IoT Video Data: Dimensionality Reduction for Efficient Deep Learning on Edge Computing

  • Ortiz Pérez, David
  • Ruiz Ponce, Pablo
  • Mulero Pérez, David
  • Benavent-Lledó, Manuel
  • Rodríguez Juan, Javier
  • Hernandez-Lopez, Hugo
  • Iarovikov, Anatoli
  • Krco, Srdjan
  • Nedić, Daliborka
  • Vukobratović, Dejan
  • Garcia-Rodriguez, Jose
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Ruiz Ponce, Pablo|||/items/bf90946b-d388-45f2-bbc1-53d580a02453
  • Rodríguez Juan, Javier|||/items/4f464549-7f4d-4a47-9602-a42f7e1b5975
The rapid loss of biodiversity significantly impacts birds’ environments and behaviors, highlighting the importance of analyzing bird behavior for ecological insights. With the growing adoption of Machine Learning (ML) algorithms in the Internet of Things (IoT) domain, edge computing has become essential to ensure data privacy and enable real-time predictions by processing high-dimensional data, such as video streams, efficiently. This paper introduces a set of dimensionality reduction techniques tailored for video sequences based on cutting-edge methods for this data representation. These methods drastically compress video data, reducing bandwidth and storage requirements while enabling the creation of compact ML models with faster inference speeds. Comprehensive experiments on bird behavior classification in rural environments demonstrate the effectiveness of the proposed techniques. The experiments incorporate state-of-the-art deep learning techniques, including pre-trained video vision models, Autoencoders, and single-frame feature extraction. These methods demonstrated superior performance to the baseline, achieving up to a 6000-fold reduction in data size while reaching a classification accuracy of 60.7% on the Visual WetlandBirds Dataset and obtaining state-of-the-art performance on this dataset. These findings underline the potential of using dimensionality reduction to enhance the scalability and efficiency of bird behavior analysis.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Text-driven online action detection

  • Benavent-Lledó, Manuel
  • Mulero Pérez, David
  • Ortiz Pérez, David
  • Garcia-Rodriguez, Jose
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
Detecting actions as they occur is essential for applications like video surveillance, autonomous driving, and humanrobot interaction. Known as online action detection, this task requires classifying actions in streaming videos, handling background noise, and coping with incomplete actions. Transformer architectures are the current state-of-the-art, yet the potential of recent advancements in computer vision, particularly vision-language models (VLMs), remains largely untapped for this problem, partly due to high computational costs. In this paper, we introduce TOAD: A Text-driven Online Action Detection architecture that supports zero-shot and few-shot learning. TOAD leverages CLIP (Contrastive Language-Image Pretraining) textual embeddings, enabling efficient use of VLMs without significant computational overhead. Our model achieves 82.46% mAP on the THUMOS14 dataset, outperforming existing methods, and sets new baselines for zero-shot and few-shot performance on the THUMOS14 and TVSeries datasets.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Deep insights into cognitive decline: A survey of leveraging non-intrusive modalities with deep learning techniques

  • Ortiz Pérez, David
  • Benavent-Lledó, Manuel
  • Garcia-Rodriguez, Jose
  • Tomás, David
  • Vizcaya-Moreno, M. Flores
  • Tomás, David|||/items/cf031fc3-3c30-454b-a6df-3762c29cc296
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Vizcaya-Moreno, M. Flores|||/items/47df53a0-4d4d-48fe-8f29-471e90417a09
Cognitive decline is a natural part of aging. However, under some circumstances, this decline is more pronounced than expected, typically due to disorders such as Alzheimer’s disease. Early detection of an anomalous decline is crucial, as it can facilitate timely professional intervention. While medical data can help, it often involves invasive procedures. An alternative approach is to employ non-intrusive techniques such as speech or handwriting analysis, which do not disturb daily activities. This survey reviews the most relevant non-intrusive methodologies that use deep learning techniques to automate the cognitive decline detection task, including audio, text, and visual processing. We discuss the key features and advantages of each modality and methodology, including state-of-the-art approaches like Transformer architecture and foundation models. In addition, we present studies that integrate different modalities to develop multimodal models. We also highlight the most significant datasets and the quantitative results from studies using these resources. From this review, several conclusions emerge. In most cases, text-based approaches consistently outperform other modalities. Furthermore, combining various approaches from individual modalities into a multimodal model consistently enhances performance across nearly all scenarios.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Face Recognition Bias Assessment through Quality Estimation Models

  • López Payá, Luis
  • Cordoba, Pedro
  • Sánchez Pérez, Ángela
  • Barrachina, Javier
  • Benavent-Lledó, Manuel
  • Mulero Pérez, David
  • Garcia-Rodriguez, Jose
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
Recent advances in facial recognition technology have achieved outstanding performance, but unconstrained face recognition remains an ongoing issue. Facial-image-quality-evaluation algorithms evaluate the quality of the input samples, providing crucial information about the accuracy of recognition decisions. By doing so, this can lead to improved results in challenging scenarios. In recent years, significant progress has been made in assessing the quality of facial images. The computation of quality scores has become highly precise and closely correlated with the model results. In this paper, we reviewed and analyzed the existing biases of cutting-edge quality-estimation techniques for face recognition. Our experimentation focused on the quality estimators developed by MagFace, FaceQNet, and SER-FIQ and were evaluated on the CelebA reference dataset. A study of bias in the face-recognition model was conducted by analyzing the quality scores presented in each article. This allowed for an examination of existing biases within both the quality estimators and the face-recognition models.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Holo4Care: a MR framework for assisting in activities of daily living by context-aware action recognition

  • Benavent-Lledó, Manuel
  • Mulero Pérez, David
  • Garcia-Rodriguez, Jose
  • Martinez-Martin, Ester
  • Vizcaya-Moreno, M. Flores
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Vizcaya-Moreno, M. Flores|||/items/47df53a0-4d4d-48fe-8f29-471e90417a09
  • Martinez-Martin, Ester|||/items/74a8fa4f-d7ad-4df1-9b91-355b474a0cd2
The evolution of virtual and augmented reality devices in recent years has encouraged researchers to develop new systems for different fields. This paper introduces Holo4Care, a context-aware mixed reality framework designed for assisting in activities of daily living (ADL) using the HoloLens 2. By leveraging egocentric cameras embedded in these devices, which offer a close-to-wearer perspective, our framework establishes a congruent relationship, facilitating a deeper understanding of user actions and enabling effective assistance. In our approach, we extend a previously established action estimation architecture after conducting a thorough review of state-of-the-art methods. The proposed architecture utilizes YOLO for hand and object detection, enabling action estimation based on these identified elements. We have trained new models on well-known datasets for object detection, incorporating action recognition annotations. The achieved mean Average Precision (mAP) is 33.2% in the EpicKitchens dataset and 26.4% on the ADL dataset. Leveraging the capabilities of the HoloLens 2, including spatial mapping and 3D hologram display, our system seamlessly presents the output of the action recognition architecture to the user. Unlike previous systems that focus primarily on user evaluation, Holo4Care emphasizes assistance by providing a set of global actions based on the user’s field of view and hand positions that reflect their intentions. Experimental results demonstrate Holo4Care’s ability to assist users in activities of daily living and other domains.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Enhancing action recognition by leveraging the hierarchical structure of actions and textual context

  • Benavent-Lledó, Manuel
  • Mulero Pérez, David
  • Ortiz Pérez, David
  • Garcia-Rodriguez, Jose
  • Argyros, Antonis
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
We propose a novel approach to improve action recognition by exploiting the hierarchical organization of actions and by incorporating contextualized textual information, including location and previous actions, to reflect the action’s temporal context. To achieve this, we introduce a transformer architecture tailored for action recognition that employs both visual and textual features. Visual features are obtained from RGB and optical flow data, while text embeddings represent contextual information. Furthermore, we define a joint loss function to simultaneously train the model for both coarse- and fine-grained action recognition, effectively exploiting the hierarchical nature of actions. To demonstrate the effectiveness of our method, we extend the Toyota Smarthome Untrimmed (TSU) dataset by incorporating action hierarchies, resulting in the Hierarchical TSU dataset, a hierarchical dataset designed for monitoring activities of the elderly in home environments. An ablation study assesses the performance impact of different strategies for integrating contextual and hierarchical data. Experimental results demonstrate that the proposed method consistently outperforms SOTA methods on the Hierarchical TSU dataset, Assembly101 and IkeaASM, achieving over a 17% improvement in top-1 accuracy.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

CogniAlign: Word-level multimodal speech alignment with gated cross-attention for Alzheimer’s detection

  • Ortiz Pérez, David
  • Benavent-Lledó, Manuel
  • Rodríguez Juan, Javier
  • Garcia-Rodriguez, Jose
  • Tomás, David
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Rodríguez Juan, Javier|||/items/4f464549-7f4d-4a47-9602-a42f7e1b5975
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Tomás, David|||/items/cf031fc3-3c30-454b-a6df-3762c29cc296
Early detection of cognitive disorders such as Alzheimer’s disease is critical for enabling timely clinical intervention and improving patient outcomes. In this work, we introduce CogniAlign, a multimodal architecture for Alzheimer’s detection that integrates audio and textual modalities, two non-intrusive sources of information that offer complementary insights into cognitive health. Unlike prior approaches that fuse modalities at a coarse level, CogniAlign leverages a word-level temporal alignment strategy that synchronizes audio embeddings with corresponding textual tokens based on transcription timestamps. This alignment supports the development of token-level fusion techniques, enabling more precise cross-modal interactions. To fully exploit this alignment, we propose a Gated Cross-Attention Fusion mechanism, where audio features attend over textual representations, guided by the superior unimodal performance of the text modality. In addition, we incorporate prosodic cues, specifically interword pauses, by inserting pause tokens into the text and generating audio embeddings for silent intervals, further enriching both streams. We evaluate CogniAlign on the ADReSSo dataset, where it achieves an accuracy of 87.35 % over a Leave-One-Subject-Out setup and of 90.36 % over a 5 fold Cross-Validation, outperforming existing state-of-the-art methods. A detailed ablation study confirms the advantages of our alignment strategy, attention-based fusion, and prosodic modeling. Finally, we perform a corpus analysis to assess the impact of the proposed prosodic features and apply Integrated Gradients to identify the most influential input segments used by the model in predicting cognitive health outcomes.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

A Comprehensive Study on Pain Assessment from Multimodal Sensor Data

  • Benavent-Lledó, Manuel
  • Mulero Pérez, David
  • Ortiz Pérez, David
  • Rodríguez Juan, Javier
  • Berenguer-Agullo, Adrian
  • Psarrou, Alexandra
  • Garcia-Rodriguez, Jose
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Rodríguez Juan, Javier|||/items/4f464549-7f4d-4a47-9602-a42f7e1b5975
Pain assessment is a critical aspect of healthcare, influencing timely interventions and patient well-being. Traditional pain evaluation methods often rely on subjective patient reports, leading to inaccuracies and disparities in treatment, especially for patients who present difficulties to communicate due to cognitive impairments. Our contributions are three-fold. Firstly, we analyze the correlations of the data extracted from biomedical sensors. Then, we use state-of-the-art computer vision techniques to analyze videos focusing on the facial expressions of the patients, both per-frame and using the temporal context. We compare them and provide a baseline for pain assessment methods using two popular benchmarks: UNBC-McMaster Shoulder Pain Expression Archive Database and BioVid Heat Pain Database. We achieved an accuracy of over 96% and over 94% for the F1 Score, recall and precision metrics in pain estimation using single frames with the UNBC-McMaster dataset, employing state-of-the-art computer vision techniques such as Transformer-based architectures for vision tasks. In addition, from the conclusions drawn from the study, future lines of work in this area are discussed.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Unrealgensyn: a framework for generating synthetic videos of Unfrequent human events

  • Mulero Pérez, David
  • Benavent-Lledó, Manuel
  • Garcia-Rodriguez, Jose
  • Vincze, Markus
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
Humans engage in a multitude of actions, some of which are rare but essential for data collection. Synthetic generation techniques are particularly effective in these scenarios, enriching the data for such uncommon actions. To address this need, we introduce a novel framework developed within Unreal Engine 5, designed to generate human action video data in hyper-realistic virtual environments. Our framework mitigates the scarcity and limited diversity of existing datasets for infrequent actions or routine tasks by utilizing synthetic motion generation through text-guided generative motion models, Gaussian splatting 3D reconstruction, and MetaHuman avatars. The utility of the framework is demonstrated by producing a synthetic video dataset depicting various human actions in diverse settings. To validate the effectiveness of the generated data, we trained VideoMAE, a state-of-the-art action recognition model, on the extended UCF101 dataset, incorporating both synthetic and real fall data, obtaining F1-scores of 0.95 and 0.97 when evaluated on the URFall and MCF datasets, respectively. The quality of the RGB-D videos generated represent a significant advance in the field. Additionally, a graph is generated from the rendered scene, detecting objects and their relationships, thus adding valuable contextual information to the video data. This capability to generate data across a wide range of actions and environments positions our framework as a valuable tool for broader applications, including digital twin creation and dataset augmentation.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

A Federated Learning Architecture for Bird Species Classification in Wetlands

  • Mulero Pérez, David
  • Rodríguez Juan, Javier
  • Ramírez, Tamai
  • Benavent-Lledó, Manuel
  • Ruiz Ponce, Pablo
  • Ortiz Pérez, David
  • Garcia-Rodriguez, Jose
  • Sebastián-González, Esther
  • Benavent-Lledó, Manuel|||/items/0d1bda20-de83-4080-97c9-1820faf481da
  • Garcia-Rodriguez, Jose|||/items/d5bc18f8-44db-4723-b291-5849321e9e69
  • Mulero Pérez, David|||/items/e3a342d3-f8d6-4010-bf87-a509394c0d36
  • Ortiz Pérez, David|||/items/e77e67d5-78fc-4170-a005-b6d3fd3ca79e
  • Rodríguez Juan, Javier|||/items/4f464549-7f4d-4a47-9602-a42f7e1b5975
  • Ruiz Ponce, Pablo|||/items/bf90946b-d388-45f2-bbc1-53d580a02453
  • Sebastián-González, Esther|||/items/c53fd431-7978-4887-bee5-296d259dab12
  • Ramírez, Tamai|||/items/893d04e3-ff41-417a-8deb-db50429d715a
Federated learning allows models to be trained on edge devices with local data, eliminating the need to share data with a central server. This significantly reduces the amount of data transferred from edge devices to central servers, which is particularly important in rural areas with limited bandwidth resources. Despite the potential of federated learning to fine-tune deep learning models using data collected from edge devices in low-resource environments, its application in the field of bird monitoring remains underexplored. This study proposes a federated learning pipeline tailored for bird species classification in wetlands. The proposed approach is based on lightweight convolutional neural networks optimized for use on resource-constrained devices. Since the performance of federated learning is strongly influenced by the models used and the experimental setting, this study conducts a comprehensive comparison of well-known lightweight models such as WideResNet, EfficientNetV2, MNASNet, GoogLeNet and ResNet in different training settings. The results demonstrate the importance of the training setting in federated learning architectures and the suitability of the different models for bird species recognition. This work contributes to the wider application of federated learning in ecological monitoring and highlights its potential to overcome challenges such as bandwidth limitations.