Pasar al contenido principal

INTELIGENCIA ARTIFICIAL RESPONSABLE PARA EL BIENESTAR HUMANO: PERCEPCION CONTEXTUALIZADA CENTRADA EN LA PERSONA

PID2022-138721NB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2022
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario FUNDACIO PER A LA UNIVERSITAT OBERTA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 3
Encontrada(s) 1 página(s)

O2, repositorio institucional de la UOC
Publicación Contribución de congreso (ConferenceOutput)

Enhancing facial expression recognition through dual-direction attention mixed feature networks: Application to 7th ABAW Challenge

  • Cabacas-Maso, Josep
  • Ortega Beltran, Elena
  • Benito Altamirano, Ismael
  • Ventura, Carles
We present our contribution to the 7th ABAW challenge at ECCV 2024, by utilizing a Dual-Direction Attention Mixed Feature Network (DDAMFN) for multitask facial expression recognition, we achieve results far beyond the proposed baseline for the Multi-Task ABAW challenge. Our proposal uses the well-known DDAMFN architecture as base to effectively predict valence-arousal, emotion recognition, and facial action units. We demonstrate the architecture ability to handle these tasks simultaneously, providing insights into its architecture and the rationale behind its design. Additionally, we compare our results for a multitask solution with independent single-task performance.
O2, repositorio institucional de la UOC
Publicación Artículo científico (JournalArticle)

Overlooked romance languages: in-the-wild emotion recognition in italian and spanish speakers

  • Ortega Beltran, Elena
  • Cabacas-Maso, Josep
  • Benito Altamirano, Ismael
  • Ventura, Carles
Within the context of developing new Socially Assistive Robots, emotion recognition has become a key factor, as it allows the robot to adapt to the user’s emotional state in real-world conditions. In this work, we focused on the analysis of Spanish and Italian voice-recording datasets, which, despite being widely spoken languages, do not receive the same level of attention as English. We selected five datasets: ELRA-S0329, EmoMatchSpanishDB, Emozionalmente, DEMoS, and EMOVO. Specifically, we centered our work on paralanguage—i.e., the vocal characteristics that accompany the message and clarify its meaning. We proposed the use of the DeepSpectrum method, which involves extracting a visual representation of the audio tracks and feeding them into a pretrained CNN model, as well as wav2vec 2.0, a self-supervised transformer module. We produced a total of four models and trained all of them on Spanish and Italian, finding that our DS-AM model (a VGG16-based model with two attention modules) and the wav2vec 2.0-based model outperformed the state-of-the-art (SOTA) models across all datasets. Finally, we performed an in-the-wild experiment, training the models on one dataset and testing them on another to simulate real-world conditions and assess how biased the models were toward specific datasets. We also included other datasets in the testing phase—CAVES (Cantonese), SUBESCO (Bangla), and RAVDESS (English)—to evaluate how the models perform in entirely different languages.
O2, repositorio institucional de la UOC
Publicación Contribución de congreso (ConferenceOutput)

Better Spanish emotion recognition in-the-wild: Bringing attention to deep spectrum voice analysis

  • Ortega Beltran, Elena
  • Cabacas-Maso, Josep
  • Benito Altamirano, Ismael
  • Ventura, Carles
Within the context of creating new Socially Assistive Robots, emotion recognition has become a key development factor, as it allows the robot to adapt to the user’s emotional state in the wild. In this work, we focused on the analysis of two voice recording Spanish datasets: ELRAS0329 and EmoMatchSpanishDB. Specifically, we centered our work in the paralanguage, e. g. the vocal characteristics that go along with the message and clarifies the meaning. We proposed the use of the DeepSpectrum method, which consists of extracting a visual representation of the audio tracks and feeding them to a pretrained CNN model. For the classification task, DeepSpectrum is often paired with a Support Vector Classifier –DS-SVC–, or a Fully-Connected deep-learning classifier –DS-FC–. We compared the results of the DS-SVC and DS-FC architectures with the state-of-the-art (SOTA) for ELRA-S0329 and EmoMatchSpanishDB. Moreover, we proposed our own classifier based upon Attention Mechanisms, namely DS-AM. We trained all models against both datasets, and we found that our DS-AM model outperforms the SOTA models for the datasets and the SOTA DeepSpectrum architectures. Finally, we trained our DS-AM model in one dataset and tested it in the other, to simulate real-world conditions on how biased is the model to the dataset.