Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 15
Encontrada(s) 1 página(s)

Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

First Workshop on Multimodal e-Coaches

  • Angelini, Leonardo
  • El Kamali, Mira
  • Mugellini, Elena
  • Khaled, Omar Abou
  • Dimitrov, Yordan
  • Veleva, Vera
  • Gospodinova, Zlatka
  • Miteva, Nadejda
  • Wheeler, Richard
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Griol Barres, David
  • Benghazi Akhlaki, Kawtar|||0000-0002-6677-7301
  • Noguera García, Manuel|||0000-0003-0503-6451
  • Bamidis, Panagiotis
  • Konstantinidis, Evdokimos
  • Petsani, Despoina
  • Beristain Iraola, Andoni
  • Fotiadis, Dimitrios I.
  • Chollet, Gérard
  • Torres, Inés
  • Esposito, Anna
  • Schlieter, Hannes
This preprint follows ACM policy: “Authors who publish with ACM have the freedom to post peer-reviewed pre-print versions of their papers to personal websites and institutional repositories. They can add a single-click link to their final published papers, and re-use any portion of their published work with the inclusion of a citation and DOI link. Authors can also post on any repository legally mandated by the agency funding the research on which the work is based, and on any non-commercial repository or aggregation that does not duplicate ACM tables of contents/substantially duplicate an ACM-copyrighted volume or issue” (https://authors.acm.org/author-resources/author-rights.) 2, e-Coaches are promising intelligent systems that aims at supporting human everyday life, dispatching advice through different interfaces, such as apps, conversational interfaces and augmented reality interfaces. This workshop aims at exploring how e-coaches might benefit from spatially and timemultiplexed interfaces and from different communication modalities (e.g., text, visual, audio, etc.) according to the context of the interaction., The NESTORE, SAAM, CAPTAIN, HOLOBALANCE, EMPATHIC, MENHIR, vCare projects are supported by the European Commission under the Horizon 2020 programmes SC1-PM-15-2017 H2020-MSCA-RISE-2018, and H2020-1.3.3, respectively through the project grants N.769643, 769661, 769830, 769574, 769872, 823907, 769807. The authors want to thank their respective Consortia. The opinions expressed in this paper are those of the authors and are not necessarily those of the project partners or the European Commission.
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

The Role of Trust in Proactive Conversational Assistants

  • Kraus, Matthias
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
This work was supported in part by the German Federal Ministry of Education and Research through the Project ``RobotKoop: Cooperative Interaction Strategies and Goal Negotiations with Learning Autonomous Robots,'' and in part by the European Union's Horizon 2020 Research and Innovation Program through MENHIR: Mental health monitoring through interactive conversations (https://menhir-project.eu) under Agreement No 823907., Humans and machines harmoniously collaborating and benefiting from each other is a long lasting dream for researchers in robotics and artificial intelligence. An important feature of efficient and rewarding cooperation is the ability to assume possible problematic situations and act in advance to prevent negative outcomes. This concept of assistance is known under the term proactivity. In this article, we investigate the development and implementation of proactive dialogues for fostering a trustworthy human-computer relationship and providing adequate and timely assistance. Here, we make several contributions. A formalisation of proactive dialogue in conversational assistants is provided. The formalisation forms a framework for integrating proactive dialogue in conversational applications. Additionally, we present a study showing the relations between proactive dialogue actions and several aspects of the perceived trustworthiness of a system as well as effects on the user experience. The results of the experiments provide significant contributions to the line of proactive dialogue research. Particularly, we provide insights on the effects of proactive dialogue on the human-computer trust relationship and dependencies between proactive dialogue and user specific and situational characteristics., Federal Ministry of Education & Research (BMBF), European Commission 823907
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Discriminative Power of Handwriting and Drawing Features in Depression

  • Greco, Claudia
  • Raimo, Gennaro
  • Amorese, Terry
  • Cuciniello, Marialucia
  • McConvey, Gavin
  • Cordasco, Gennaro
  • Faundez Zanuy, Marcos
  • Vinciarelli, Alessandro
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Esposito, Anna
This study contributes knowledge on the detection of depression through handwriting/drawing features, to identify quantitative and noninvasive indicators of the disorder for implementing algorithms for its automatic detection. For this purpose, an original online approach was adopted to provide a dynamic evaluation of handwriting/drawing performance of healthy participants with no history of any psychiatric disorders (n=28), and patients with a clinical diagnosis of depression (n=27). Both groups were asked to complete seven tasks requiring either the writing or drawing on a paper while five handwriting/drawing features’ categories (i.e. pressure on the paper, time, ductus, space among characters, and pen inclination) were recorded by using a digitalized tablet. The collected records were statistically analyzed. Results showed that, except for pressure, all the considered features, successfully discriminate between depressed and nondepressed subjects. In addition, it was observed that depression affects different writing/drawing functionalities. These findings suggest the adoption of writing/drawing tasks in the clinical practice as tools to support the current depression detection methods. This would have important repercussions on reducing the diagnostic times and treatment formulation., European Union Horizon 2020 research and innovation program under grant agreement No. 769872 (EMPATHIC), European Union Horizon 2020 research and innovation program under grant agreement No. 823907 (MENHIR), Project SIROBOTICS that received funding from Ministero dell'Istruzione, dell'Università, e della Ricerca (MIUR), PNR 2015- 2020, D.D. 1735/2017, Project ANDROIDS that received funding from Università della Campania \Luigi Vanvitelli" inside the program V:ALERE 2019, funded with D.R. 906/2019, Project SALICE that received funding from Università della Campania \Luigi Vanvitelli" inside the program Giovani Ricercatori (DR 509/2022), funded with DR 834/2022
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Architecting dietary intake monitoring as a service combining NLP and IoT

  • Benítez-Guijarro, Antonio
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Noguera García, Manuel|||0000-0003-0503-6451
  • Benghazi, Kawtar
Currently there exist many tools that support monitoring and encouragement of healthy nutrition habits in the context of wellness promotion. In this domain, interfaces based on natural language provide more flexibility for nutritional self-reporting than traditional form-based applications, allowing the users to provide richer and spontaneous descriptions. Nonetheless, in certain circumstances, natural language records may miss some important aspects, such as the quantity of food eaten, which results in incomplete recordings. In the Internet-of-Things (IoT) paradigm, smart home appliances can support and complement the recording process so as to make it more accurate. However, in order to build systems that support the semantic analysis of nutritional self-reports, it is necessary to integrate multiple inter-related components, possibly within complex e-health platforms. For this reason, these components should be designed and encapsulated avoiding monolithic approaches that derive in rigidity and dependency of particular technologies. Currently, there are no models or architectures that serve as a reference for developers towards this objective. In this paper, we present a service-based architecture that helps to contrast and complement the descriptions of food intakes by means of connected smart home devices, coordinating all the stages during the process of recognizing food records provided in natural language. Additionally, we aim to identify and design the essential services that are required to automate the recording and subsequent processing of natural language descriptions of nutritional intakes in association with smart home devices. The functionalities provided by each of these services are ready to work in isolation, just out of the box, or in downstream pipeline processes, bypassing the inconveniences of monolithic architectures., DEP2015-70980-R of the Spanish Ministry of Economy and Competitiveness (MINECO) and European Regional Development Fund (ERDF), European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907, COST Action IC1303 AAPELE
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Mobile Conversational Agents for Stroke Rehabilitation Therapy

  • Griol Barres, David
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
This “accepted article” follows IEEE post-publication policy: https://journals.ieeeauthorcenter. ieee.org/become-an-ieee-journal-author/publishing-ethics/guidelines-and-policies/ post-publication-policies/ ©2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Mobile health (m-Health) has emerged as a rapidly developing area that is transforming clinical research and health care on a global scale. In this paper, we describe a conversational app for the therapy of stroke rehabilitation. The main objective of the conversational app is to help recovering cognitive abilities of patients by means of a set of proposed exercises, which are divided into 8 categories focused on specific abilities. These categories have been defined after a detailed review of the guidelines for rehabilitation and training therapies. In addition, the application integrates a multimodal conversational interface to facilitate human-computer interaction, which has been specially designed for the elderly and patients with motor or visual or disabilities. The exercises provided by the application can be easily adapted to the specific users’ requirements and preferences by means of the incorporation, deletion or modification of routines stored into a specific database isolated from the logic of the application., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 823907 (MENHIR project) and from the Spanish Ministry of Economy, Industry and Competitiveness through the CAVIAR (MINECO, TEC2017-84593-C2-1-R) project.
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Fine-Tuning BERT Models for Intent Recognition Using a Frequency Cut-Off Strategy for Domain-Specific Vocabulary Extension

  • Fernández Martínez, Fernando
  • Griol Barres, David
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
The work leading to these results was supported by the Spanish Ministry of Science and Innovation through the R& D&i projects GOMINOLA (PID2020-118112RB-C21 and PID2020118112RB-C22, funded by MCIN/AEI/10.13039/501100011033), CAVIAR (TEC2017-84593-C2-1-R, funded by MCIN/ AEI/10.13039/501100011033/FEDER "Una manera de hacer Europa"), and AMICPoC (PDC2021-120846-C42, funded by MCIN/AEI/10.13039/501100011033 and by "the European Union "NextGenerationEU/PRTR"). This research also received funding from the European Union's Horizon2020 research and innovation program under grant agreement No 823907 (http://menhirproject.eu, accessed on 2 February 2022). Furthermore, R.K.'s research was supported by the Spanish Ministry of Education (FPI grant PRE2018-083225)., Intent recognition is a key component of any task-oriented conversational system. The intent recognizer can be used first to classify the user’s utterance into one of several predefined classes (intents) that help to understand the user’s current goal. Then, the most adequate response can be provided accordingly. Intent recognizers also often appear as a form of joint models for performing the natural language understanding and dialog management tasks together as a single process, thus simplifying the set of problems that a conversational system must solve. This happens to be especially true for frequently asked question (FAQ) conversational systems. In this work, we first present an exploratory analysis in which different deep learning (DL) models for intent detection and classification were evaluated. In particular, we experimentally compare and analyze conventional recurrent neural networks (RNN) and state-of-the-art transformer models. Our experiments confirmed that best performance is achieved by using transformers. Specifically, best performance was achieved by fine-tuning the so-called BETO model (a Spanish pretrained bidirectional encoder representations from transformers (BERT) model from the Universidad de Chile) in our intent detection task. Then, as the main contribution of the paper, we analyze the effect of inserting unseen domain words to extend the vocabulary of the model as part of the fine-tuning or domain-adaptation process. Particularly, a very simple word frequency cut-off strategy is experimentally shown to be a suitable method for driving the vocabulary learning decisions over unseen words. The results of our analysis show that the proposed method helps to effectively extend the original vocabulary of the pretrained models. We validated our approach with a selection of the corpus acquired with the Hispabot-Covid19 system obtaining satisfactory results., Spanish Ministry of Science and Innovation (MCIN/AEI) PID2020-118112RB-C21 PID2020118112RB-C22 PDC2021-120846-C42, Spanish Ministry of Science and Innovation (MCIN/AEI/FEDER "Una manera de hacer Europa") TEC2017-84593-C2-1-R, Spanish Ministry of Science and Innovation (European Union "NextGenerationEU/PRTR") PDC2021-120846-C42, European Commission 823907, German Research Foundation (DFG) PRE2018-083225
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Measuring and fostering engagement with mental health e-coaches

  • Chollet, Gérard
  • Torres, María Inés
  • Esposito, Anna
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Griol Barres, David
  • Benghazi Akhlaki, Kawtar|||0000-0002-6677-7301
  • Noguera García, Manuel|||0000-0003-0503-6451
This preprint follows ACM policy: “Authors who publish with ACM have the freedom to post peer-reviewed pre-print versions of their papers to personal websites and institutional repositories. They can add a single-click link to their final published papers, and re-use any portion of their published work with the inclusion of a citation and DOI link. Authors can also post on any repository legally mandated by the agency funding the research on which the work is based, and on any non-commercial repository or aggregation that does not duplicate ACM tables of contents/substantially duplicate an ACM-copyrighted volume or issue” (https://authors.acm.org/author-resources/author-rights.), Mental health e-coaches and technology-delivered services are showing considerable benefits to foster mental health literacy, monitor symptoms, favour self-management of different mental health conditions and scaffold positive behaviours. However, adherence to these systems is usually low and generally declines over time. There exists a recent body of work addressing engagement with mental health technology with the aim to understand the factors that influence sustained use and inform the design of systems that are able to generate sufficient engagement to attain their expected results. This paper explores the different facets of engagement in mental health e-coaches, including aspects related to the estimation of system use from log data, effective engagement, user experience, motivation, incentives, user expectations, peer support and the specific challenges of technologies addressed to mental health., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907 (MENHIR project https://menhir-project.eu) and grant agreement No 769872 (EMPATHIC project http://www.empathic-project.eu).
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Incorporating evidence into mental health Q&A: a novel method to use generative language models for validated clinical content extraction

  • Kharitonova, Ksenia
  • Pérez Fernández, David
  • Gutiérrez Hernando, Javier
  • Gutiérrez Fandiño, Asier
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Griol Barres, David
Generative language models have changed the way we interact with computers using natural language. With the release of increasingly advanced GPT models, systems are able to correctly respond to questions in various domains. However, they still have important limitations, such as hallucinations, lack of substance in answers, inability to justify responses, or showing high confidence with fabricated content. In digital mental health, every decision must be traceable and based on scientific evidence and these shortcomings are hindering the integration of LLMs into clinical practice. In this paper, we provide a novel automated method to develop evidencebased question answering systems. Powerful state-of-the-art generalist language models are used and forced to employ only contents in validated clinical guidelines, tracking the source of the evidence for each generated response. This way, the system is able to protect users from hallucinatory responses. As a proof of concept, we present the results obtained building question-answering systems circumscribed to the clinical practice guidelines of the Spanish National Health System about the management of depression and attention deficit hyperactivity disorder. The coherence, veracity, and evidence supporting the responses have been evaluated by human experts obtaining high reliability, clarity, completeness, and traceability of evidence results., ‘CONVERSA: Effective and efficient resources and models for transformative conversational AI in Spanish and co-official languages’ project with reference (Agencia Estatal de Investigación) TED2021- 132470B-I00, funded by MCIN/AEI/10.13039/501100011033 and by the European Union ‘NextGenerationEU/PRTR’, European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907 (MENHIR, https://menhir-project.eu)
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Adaptive dialogue management using intent clustering and fuzzy rules

  • Griol Barres, David
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Molina, José Manuel
  • Sanchis, Araceli
This is the peer reviewed version of the following article: Griol, D, Callejas, Z, Molina, JM, Sanchis, A. Adaptive dialogue management using intent clustering and fuzzy rules. Expert Systems. 2020; e12630. , which has been published in final form at https://doi.org/10.1111/exsy.12630. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions., Conversational systems have become an element of everyday life for billions of users who use speech-based interfaces to services, engage with personal digital assistants on smartphones, social media chatbots, or smart speakers. One of the most complex tasks in the development of these systems is to design the dialogue model, the logic that provided a user input selects the next answer. The dialogue model must also consider mechanisms to adapt the response of the system and the interaction style according to different groups and user profiles. Rule-based systems are difficult to adapt to phenomena that were not taken into consideration at design-time. However, many of the systems that are commercially available are based on rules, and so are the most widespread tools for the development of chatbots and speech interfaces. In this paper, we present a proposal to: i) automatically generate the dialogue rules from a dialogue corpus through the use of evolving algorithms, ii) adapt the rules according to the detected user intention. We have evaluated our proposal with several conversational systems of different application domains, from which our approach provided an efficient way for adapting a set of dialogue rules considering user utterance clusters., The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907 (MENHIR project: https://menhir-project.eu), and the Spanish project TEC2017-88048-C2-2-R.
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

  • Luna Jiménez, Cristina
  • Griol Barres, David
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
The work leading to these results was supported by the Spanish Ministry of Science and Innovation through the projects GOMINOLA (PID2020-118112RB-C21 and PID2020-118112RB-C22, funded by MCIN/AEI/10.13039/501100011033), CAVIAR (TEC2017-84593-C2-1-R, funded by MCIN/AEI/10.13039/501100011033/FEDER "Una manera de hacer Europa"), and AMIC-PoC (PDC2021-120846-C42, funded by MCIN/AEI/10.13039/501100011033 and by "the European Union "NextGenerationEU/PRTR"). This research also received funding from the European Union's Horizon2020 research and innovation program under grant agreement No 823907 (http://menhir-project.eu, accessed on 17 November 2021). Furthermore, R.K.'s research was supported by the Spanish Ministry of Education (FPI grant PRE2018-083225)., Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated a pre-trained xlsr-Wav2Vec2.0 transformer using two transfer-learning techniques: embedding extraction and fine-tuning. The best accuracy results were achieved when we fine-tuned the whole model by appending a multilayer perceptron on top of it, confirming that the training was more robust when it did not start from scratch and the previous knowledge of the network was similar to the task to adapt. Regarding the facial emotion recognizer, we extracted the Action Units of the videos and compared the performance between employing static models against sequential models. Results showed that sequential models beat static models by a narrow difference. Error analysis reported that the visual systems could improve with a detector of high-emotional load frames, which opened a new line of research to discover new ways to learn from videos. Finally, combining these two modalities with a late fusion strategy, we achieved 86.70% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. Results demonstrated that these modalities carried relevant information to detect users’ emotional state and their combination allowed to improve the final system performance., Spanish Government PID2020-118112RB-C21 PID2020-118112RB-C22 MCIN/AEI/10.13039/501100011033 TEC2017-84593-C2-1-R MCIN/AEI/10.13039/501100011033/FEDER PDC2021-120846-C42, European Union "NextGenerationEU/PRTR"), European Union's Horizon2020 research and innovation program 823907, German Research Foundation (DFG) PRE2018-083225
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

A Synthetic Voice for an Assistive Conversational Agent: A Survey to Discover Italian Preferences regarding Synthetic Voice’s Gender and Quality Level

  • Cuciniello, Marialucia
  • Amorese, Terry
  • Greco, Claudia
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Vogel, Carl
  • Cordasco, Gennaro
  • Esposito, Anna
Based on a previous investigation, a quantitative study aimed to identify user’ preferences towards four synthetic voices of two different quality levels (classified through the sophistication of the synthesizer: low vs. high) is proposed. The voices administered to participants were developed considering two main aspects: the voice quality (high/low) and their gender (male/female). 182 unpaid participants were recruited for the study, divided in four groups according to their age, and therefore classified as adolescents, young adults, middle-aged, and seniors. To collect data regarding each voice, randomly audited by participants, the shortened version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ) was exploited. Outcomes of the previous study revealed that the voices of high quality, regardless of their gender, received a higher acclaim by all participants examined rather than the corresponding two voices assessed as lower quality. Conversely, findings of the current study suggest that the four new groups of participants involved agreed in showing their strong preference towards the high-quality voice gendered as female compared to all the other considered voices. Regarding the two voices gendered as male, the high-quality one was considered as more original and capable to arouse positive emotional states than the low-quality one. Moreover, the high-quality male voice was judged as more natural than the female low-quality one. Results provide some insights for future directions in the user experience and design field., European Union Horizon 2020 research and innovation programme under grant agreement N.769872 (EMPATHIC) and N. 823907 (MENHIR), Project SIROBOTICS that received funding from Ministero dell’Istruzione, dell ‘Università, e della Ricerca (MIUR), PNR 2015-2020, D.D. 1735/2017, Project ANDROIDS that received funding from Università della Campania “Luigi Vanvitelli” inside the programme V:ALERE 2019, funded with D.R. 906/2019, Project SALICE that received funding from Università della Campania “Luigi Vanvitelli” inside the programme Giovani Ricercatori (DR 509/2022), funded with DR 834/2022
Archivo Digital UPM
Publicación Artículo científico (JournalArticle)

Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning

  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Griol, David|||0000-0001-6266-5321
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Kleinlein, Ricardo|||0000-0002-7313-7601
  • Montero Martínez, Juan Manuel|||0000-0002-7908-5400
  • Fernández Martí­nez, Fernando|||0000-0003-3877-0089
Archivo Digital UPM
Publicación Artículo científico (JournalArticle)

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

  • Luna Jiménez, Cristina|||0000-0001-5369-856X
  • Kleinlein, Ricardo|||0000-0002-7313-7601
  • Griol, David|||0000-0001-6266-5321
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Montero Martínez, Juan Manuel|||0000-0002-7908-5400
  • Fernández Martínez, Fernando|||0000-0003-3877-0089
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • López Zorrilla, Asier
  • Torres Barañano, María Inés
In this work, we present an approach to understand the computational methods and decision-making involved in the identification of emotions in spontaneous speech. The selected task consists of Spanish TV debates, which entail a high level of complexity as well as additional subjectivity in the human perception-based annotation procedure. A simple convolutional neural model is proposed, and its behaviour is analysed to explain its decision-making. The proposed model slightly outperforms commonly used CNN architectures such as VGG16, while being much lighter. Internal layer-by-layer transformations of the input spectrogram are visualised and analysed. Finally, a class model visualisation is proposed as a simple interpretation approach whose usefulness is assessed in the work., The research presented in this paper was conducted as part of the AMIC, AMIC-PdC, BEWORD and MENHIR projects, which received funding from the Spanish Minister of Science under grants TIN2017-85854-C4-3-R, PDC2021-120846-C43 and PID2021-126061OB-C42 and from the European Union’s H2020-MSCA-RISE-2018 Research and Innovation Staff Exchange, under Grant No. 823907. The first author also received a PhD scholarship from the University of the Basque Country UPV/EHU, PIF17/310.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Perceptual borderline for balancing multi-class spontaneous emotional data

  • Ben Letaifa Zouari, Leila
  • Torres Barañano, María Inés
Speech is a behavioural biometric signal that can provide important information to understand the human intends as well as their emotional status. The paper is centered on the speech-based identification of the seniors’s emotional status during their interaction with a virtual agent playing the role of a health professional coach. Under real conditions, we can just identify a small set of task-dependent spontaneous emotions. The number of identified samples is largely different for each emotion, which results in an imbalanced dataset problem. This research proposes the dimensional model of emotions as a perceptual representation space alternative to the generally used acoustic one. The main contribution of the paper is the definition of a perceptual borderline for the oversampling of minority emotion classes in this space. This limit, based on arousal and valence criteria, leads to two methods of balancing the data: the Perceptual Borderline oversampling and the Perceptual Borderline SMOTE (Synthetic Minority Oversampling Technique). Both methods are implemented and compared to state-of-the-art approaches of Random oversampling and SMOTE. The experimental evaluation was carried out on three imbalanced datasets of spontaneous emotions acquired in human-machine scenarios in three different cultures: Spain, France and Norway. The emotion recognition results obtained by neural networks classifiers show that the proposed perceptual oversampling methods led to significant improvements when compared with the state-of-the art, for all scenarios and languages., The research presented in this paper is conducted as partof the project EMPATHIC and of the MENHIR MSCAaction that have received funding from the European Union’s Horizon 2020 research and innovation program under grant agreements No 769872 an No 823907 respectivel