Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 30
Encontrada(s) 1 página(s)

Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

Discriminative Power of Handwriting and Drawing Features in Depression

  • Greco, Claudia
  • Raimo, Gennaro
  • Amorese, Terry
  • Cuciniello, Marialucia
  • McConvey, Gavin
  • Cordasco, Gennaro
  • Faundez Zanuy, Marcos
  • Vinciarelli, Alessandro
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Esposito, Anna
This study contributes knowledge on the detection of depression through handwriting/drawing features, to identify quantitative and noninvasive indicators of the disorder for implementing algorithms for its automatic detection. For this purpose, an original online approach was adopted to provide a dynamic evaluation of handwriting/drawing performance of healthy participants with no history of any psychiatric disorders (n=28), and patients with a clinical diagnosis of depression (n=27). Both groups were asked to complete seven tasks requiring either the writing or drawing on a paper while five handwriting/drawing features’ categories (i.e. pressure on the paper, time, ductus, space among characters, and pen inclination) were recorded by using a digitalized tablet. The collected records were statistically analyzed. Results showed that, except for pressure, all the considered features, successfully discriminate between depressed and nondepressed subjects. In addition, it was observed that depression affects different writing/drawing functionalities. These findings suggest the adoption of writing/drawing tasks in the clinical practice as tools to support the current depression detection methods. This would have important repercussions on reducing the diagnostic times and treatment formulation., European Union Horizon 2020 research and innovation program under grant agreement No. 769872 (EMPATHIC), European Union Horizon 2020 research and innovation program under grant agreement No. 823907 (MENHIR), Project SIROBOTICS that received funding from Ministero dell'Istruzione, dell'Università, e della Ricerca (MIUR), PNR 2015- 2020, D.D. 1735/2017, Project ANDROIDS that received funding from Università della Campania \Luigi Vanvitelli" inside the program V:ALERE 2019, funded with D.R. 906/2019, Project SALICE that received funding from Università della Campania \Luigi Vanvitelli" inside the program Giovani Ricercatori (DR 509/2022), funded with DR 834/2022
Digibug. Repositorio Institucional de la Universidad de Granada
Publicación Artículo científico (JournalArticle)

A Synthetic Voice for an Assistive Conversational Agent: A Survey to Discover Italian Preferences regarding Synthetic Voice’s Gender and Quality Level

  • Cuciniello, Marialucia
  • Amorese, Terry
  • Greco, Claudia
  • Callejas Carrión, Zoraida|||0000-0001-8891-5237
  • Vogel, Carl
  • Cordasco, Gennaro
  • Esposito, Anna
Based on a previous investigation, a quantitative study aimed to identify user’ preferences towards four synthetic voices of two different quality levels (classified through the sophistication of the synthesizer: low vs. high) is proposed. The voices administered to participants were developed considering two main aspects: the voice quality (high/low) and their gender (male/female). 182 unpaid participants were recruited for the study, divided in four groups according to their age, and therefore classified as adolescents, young adults, middle-aged, and seniors. To collect data regarding each voice, randomly audited by participants, the shortened version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ) was exploited. Outcomes of the previous study revealed that the voices of high quality, regardless of their gender, received a higher acclaim by all participants examined rather than the corresponding two voices assessed as lower quality. Conversely, findings of the current study suggest that the four new groups of participants involved agreed in showing their strong preference towards the high-quality voice gendered as female compared to all the other considered voices. Regarding the two voices gendered as male, the high-quality one was considered as more original and capable to arouse positive emotional states than the low-quality one. Moreover, the high-quality male voice was judged as more natural than the female low-quality one. Results provide some insights for future directions in the user experience and design field., European Union Horizon 2020 research and innovation programme under grant agreement N.769872 (EMPATHIC) and N. 823907 (MENHIR), Project SIROBOTICS that received funding from Ministero dell’Istruzione, dell ‘Università, e della Ricerca (MIUR), PNR 2015-2020, D.D. 1735/2017, Project ANDROIDS that received funding from Università della Campania “Luigi Vanvitelli” inside the programme V:ALERE 2019, funded with D.R. 906/2019, Project SALICE that received funding from Università della Campania “Luigi Vanvitelli” inside the programme Giovani Ricercatori (DR 509/2022), funded with DR 834/2022
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

A multilingual neural coaching model with enhanced long-term dialogue structure

  • López Zorrilla, Asier
  • Torres Barañano, María Inés
In this work we develop a fully data-driven conversational agent capable of carrying out motivational coach- ing sessions in Spanish, French, Norwegian, and English. Unlike the majority of coaching, and in general well-being related conversational agents that can be found in the literature, ours is not designed by hand- crafted rules. Instead, we directly model the coaching strategy of professionals with end users. To this end, we gather a set of virtual coaching sessions through a Wizard of Oz platform, and apply state of the art Natural Language Processing techniques. We employ a transfer learning approach, pretraining GPT2 neural language models and fine-tuning them on our corpus. However, since these only take as input a local dialogue history, a simple fine-tuning procedure is not capable of modeling the long-term dialogue strategies that appear in coaching sessions. To alleviate this issue, we first propose to learn dialogue phase and scenario embeddings in the fine-tuning stage. These indicate to the model at which part of the dialogue it is and which kind of coaching session it is carrying out. Second, we develop a global deep learning system which controls the long-term structure of the dialogue. We also show that this global module can be used to visualize and interpret the decisions taken by the the conversational agent, and that the learnt representations are comparable to dialogue acts. Automatic and human evaluation show that our proposals serve to improve the baseline models. Finally, interaction experiments with coaching experts indicate that the system is usable and gives rise to positive emotions in Spanish, French and English, while the results in Norwegian point out that there is still work to be done in fully data driven approaches with very low resource languages., This work has been partially funded by the Basque Government under grant PRE_2017_1_0357 and by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 769872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Dialogue Management and Language Generation for a Robust Conversational Virtual Coach: Validation and User Study

  • Vázquez Risco, Alain
  • López Zorrilla, Asier
  • Olaso Fernández, Javier Mikel
  • Torres Barañano, María Inés
Designing human–machine interactive systems requires cooperation between different disciplines is required. In this work, we present a Dialogue Manager and a Language Generator that are the core modules of a Voice-based Spoken Dialogue System (SDS) capable of carrying out challenging, long and complex coaching conversations. We also develop an efficient integration procedure of the whole system that will act as an intelligent and robust Virtual Coach. The coaching task significantly differs from the classical applications of SDSs, resulting in a much higher degree of complexity and difficulty. The Virtual Coach has been successfully tested and validated in a user study with independent elderly, in three different countries with three different languages and cultures: Spain, France and Norway., The research presented in this paper has been conducted as part of the project EMPATHIC that has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant No. 769872. Additionally, this work has been partially funded by projects BEWORD and AMIC-PC of the Minister of Science of Technology, under Grant Nos. PID2021-126061OB-C42 and PDC2021-120846-C43, respectively. Vázquez and López Zorrilla received a PhD scholarship from the Basque Government, with Grant Nos. PRE 2020 1 0274 and PRE 2017 1 0357, respectively.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Automatic Analysis of Emotions from the Voices/Speech in Spanish TV Debates

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • López Zorrilla, Asier
  • Torres Barañano, María Inés
The goal of this work is to automatically analyze the emotional status of speakers, in human-human interactions, carried out in TV debates, where controversial topics are often presented. Human observers provide their perception about the emotional status associated to the interventions of the participants. An analysis of the resulting annotation was carried out by using different models for representing the emotions. The obtained labeled corpus was used to build an automatic system capable of detecting the emotional status associated to each acoustic signal, making use of the deep learning paradigm. The use of a corpus, where the real emotions that appear in a Spanish TV debate (with subtleties and often closer to neutrality than acted ones), are represented is crucial for learning models properly. In fact, although the level of accuracy depends on the problem complexity and the model employed for representing the emotional status, F1 scores of 0.7 were attained., This work has been partially supported by the Spanish Government under grant TIN2017-85854-C4-3-R (MCIU/AEI/FEDER, UE), the Basque Government under grant and from the European Union’s Horizon 2020 research and innovation programme under grant agreement no 769872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Emotion Detection from Speech and Text

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • Antón, Josu
  • Carrilero, Mikel
  • Torres Barañano, María Inés
The main goal of this work is to carry out automatic emo-tion detection from speech by using both acoustic and textualinformation. For doing that a set of audios were extracted froma TV show were different guests discuss about topics of currentinterest. The selected audios were transcribed and annotatedin terms of emotional status using a crowdsourcing platform.A 3-dimensional model was used to define an specific emo-tional status in order to pick up the nuances in what the speakeris expressing instead of being restricted to a predefined set ofdiscrete categories. Different sets of acoustic parameters wereconsidered to obtain the input vectors for a neural network. Torepresent each sequence of words, a models based on word em-beddings was used. Different deep learning architectures weretested providing promising results, although having a corpus ofa limited size., This work has been partially founded by bythe Spanish Government (TIN2014-54288-C4-4-R and TIN2017-85854-C4-3-R), and bythe European Commission H2020 SC1-PM15program under RIA 7 grant 69872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Capítulo o parte de un libro (BookPart)

Regularized Neural User Model for Goal-Oriented Spoken Dialogue Systems

  • Serras Sáenz, Manex
  • Torres Barañano, María Inés
  • Del Pozo, Arantza
User simulation is widely used to generate artificial dialogues in order to train statistical spoken dialogue systems and perform evaluations. This paper presents a neural network approach for user modeling that exploits an encoder-decoder bidirectional architecture with a regularization layer for each dialogue act. In order to minimize the impact of data sparsity, the dialogue act space is compressed according to the user goal. Experiments on the Dialogue State Tracking Challenge 2 (DSTC2) dataset provide significant results at dialogue act and slot level predictions, outperforming previous neural user modeling approaches in terms of F1 score., Spanish Minister of Science under grants TIN2014-54288-C4-4-R and TIN2017-85854-C4-3-R and by the EU H2020 EMPATHIC project grant number 769872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Iruzurrezko portaeren detekzioa crowd motako etiketazioan

  • De Velasco Vázquez, Mikel
  • López Zorrilla, Asier
  • Justo Blanco, Raquel
This work aims at detecting low quality labels in crowdsourcing annotation tasks. We validate our proposal carrying out experiments in a difficult and subjective task: emotion recognition. We have developed several measures in order to detect fraudulent behaviour, including measures related to the labelling time, worker inter-agreement and the distribution of the answers. Not only do we show that each of the described measures is helpful but we also demonstrate that mixing them is the best way to go., Lan honek crowd motako etiketazioan agertu daitezkeen kalitate baxuko etiketak detektatzea du helburu. Proposatutako metodologia balioztatzeko, saiakuntzak ataza zail eta subjektibo batekin egin ditugu: emozioen de- tekzioarekin. Iruzurrezko langileak topatzeko zenbait neurri proposatu dira, etiketatze denboran, langileen arteko adostasunean eta langileen erantzunen banaketan oinarriturikoak. Neurri bakoitza baliagarria dela frogatu dugun arren, gure ondorio nagusia neurriak batzerakoan iruzurrezko langileak detektatzeko probabilitatea handitzen dela da., Egileok gure esker ona adierazi nahiko genioke Euskal Herriko Unibertsitateari, Espainako gobernuako TIN2017- 85854-C4-3-R zenbakidun diru laguntzari eta H2020 Europako Batzordeko SC1-PM15 programako RIA 7 deial- diko 769872 zenbakidun laguntzari, hurrenez hurren, ikerketa hau babesteagatik.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Detection of Sarcasm and Nastiness: New Resources for Spanish Language

  • Justo Blanco, Raquel
  • Alcaide Salinas, José María
  • Torres Barañano, María Inés
  • Walker, Marilyn
The main goal of this work is to provide the cognitive computing community with valuable resources to analyze and simulate the intentionality and/or emotions embedded in the language employed in social media. Specifically, it is focused on the Spanish language and online dialogues, leading to the creation of SOFOCO (Spanish Online Forums Corpus). It is the first Spanish corpus consisting of dialogic debates extracted from social media and it is annotated by means of crowdsourcing in order to carry out automatic analysis of subjective language forms, like sarcasm or nastiness. Furthermore, the annotators were also asked about the context need when taking a decision. In this way, the users’ intentions and their behavior inside social networks can be better understood and more accurate text analysis is possible. An analysis of the annotation results is carried out and the reliability of the annotations is also explored. Additionally, sarcasm and nastiness detection results (around 0.76 F-Measure in both cases) are also reported. The obtained results show the presented corpus as a valuable resource that might be used in very diverse future work., This study was partially funded by the Spanish Government (TIN2014-54288-C4-4-R and TIN2017-85854-C4-3-R) by the European Unions’s H2020 program under grant 769872 and by the National Science Foundation of USA (NSF CISE R1 #1202668)
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Corrective Focus Detection in Italian Speech Using Neural Networks

  • López Zorrilla, Asier
  • De Velasco Vázquez, Mikel
  • Cenceschi, Sonia
  • Torres Barañano, María Inés
The corrective focus is a particular kind of prosodic prominence where the speaker is intended to correct or to emphasize a concept. This work develops an Artificial Cognitive System (ACS) based on Recurrent Neural Networks that analyzes suitablefeatures of the audio channel in order to automatically identify the Corrective Focus on speech signals. Two different approaches to build the ACS have been developed. The first one addresses the detection of focused syllables within a given Intonational Unit whereas the second one identifies a whole IU as focused or not. The experimental evaluation over an Italian Corpus has shown the ability of the Artificial Cognitive System to identify the focus in the speaker IUs. This ability can lead to further important improvements in human-machine communication. The addressed problem is a good example of synergies between Humans and Artificial Cognitive Systems., The research leading to the results in this paper has been conducted in the project EMPATHIC (Grant N: 769872) that received funding from the European Union’s Horizon2020 research and innovation programme.Additionally, this work has been partially funded by the Spanish Minister of Science under grants TIN2014-54288-C4-4-R and TIN2017-85854-C4-3-R, by the Basque Government under grant PRE_2017_1_0357,andby the University of the Basque Country UPV/EHU under grantPIF17/310.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

A Differentiable Generative Adversarial Network for Open Domain Dialogue

  • López Zorrilla, Asier
  • De Velasco Vázquez, Mikel
  • Torres Barañano, María Inés
Presentado antes de su publicación por Springer en el IWSDS 2019: International Workshop on Spoken Dialogue Systems Technology, Siracusa, Italy, April 24-26, 2019, This work presents a novel methodology to train open domain neural dialogue systems within the framework of Generative Adversarial Networks with gradient-based optimization methods. We avoid the non-differentiability related to text-generating networks approximating the word vector corresponding to each generated token via a top-k softmax. We show that a weighted average of the word vectors of the most probable tokens computed from the probabilities resulting of the top-k softmax leads to a good approximation of the word vector of the generated token. Finally we demonstrate through a human evaluation process that training a neural dialogue system via adversarial learning with this method successfully discourages it from producing generic responses. Instead it tends to produce more informative and variate ones., This work has been partially funded by the Basque Government under grant PRE_2017_1_0357, by the University of the Basque Country UPV/EHU under grant PIF17/310, and by the H2020 RIA EMPATHIC (Grant N: 769872).
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

EMPATHIC: Empathic, Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly, EMPATHIC: Coach virtual empático, expresivo y avanzado para mejorar el bienestar de las personas de edad avanzada sanas e independientes

  • López Zorrilla, Asier
  • De Velasco Vázquez, Mikel
  • Irastorza Manso, Jon
  • Olaso Fernández, Javier Mikel
  • Justo Blanco, Raquel
  • Torres Barañano, María Inés
The EMPATHIC project will research, innovate, explore and validatenew paradigms and platforms, laying the foundation for future generations of Per-sonalised Virtual Coaches to assist elderly people living independently at and aroundtheir home. Innovative multimodal face analytics, adaptive spoken dialogue systemsand natural language interfaces are part of what the project will research and in-novate, in order to help dependent aging persons and their carers. The project willuse remote non-intrusive technologies to extract physiological markers of emotionalstates in real-time for online adaptive responses of the coach, and advance holisticmodelling of behavioural, computational, physical and social aspects of a person-alised expressive virtual coach. It will develop causal models of coach-user interac-tional exchanges that engage elders in emotionally believable interactions keeping offloneliness, sustaining health status, enhancing quality of life and simplifying accessto future telecare services, El proyecto EMPATHIC tiene como misi ́on investigar, explorar, innovary validar nuevos paradigmas y plataformas, sentando las bases para las futuras gen-eraciones decoachvirtuales personalizados para ayudar a personas de avanzada edadque viven de forma independiente en su hogar. Adem ́as, EMPATHIC investigar ́a einnovar ́a con el objetivo de ayudar a las personas de avanzada edad dependientes y asus cuidadores mediante an ́alisis facial multimodal, sistemas de di ́alogo adaptablese interfaces de lenguaje natural. El proyecto utilizar ́a tecnolog ́ıas remotas no in-trusivas para extraer marcadores fisiol ́ogicos de estados emocionales en tiempo real,las cuales influenciar ́an el comportamiento del coach. Tambi ́en se modelar ́an losaspectos computacionales, f ́ısicos y sociales de los coach virtuales expresivos desdeun punto de vista hol ́ıstico. Finalmente, se desarrollar ́an modelos que permitan lainteracci ́on entre el coach y el usuario, buscando as ́ı involucrar a ́estos en interac-ciones que contribuyan a evitar la soledad, mantener el estado de salud, mejorar lacalidad de vida y simplificar el acceso a futuros servicios de teleasistencia, This work has been founded by the European Commission H2020-SC1-2017-RIAgrant number 769872: “Empathic, Expressive, Advanced Virtual Coach to ImproveIndependent Healthy-Life-Years of the Elderly”.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly

  • Montenegro Portillo, César
  • López Zorrilla, Asier
  • Olaso Fernández, Javier Mikel
  • Santana Hermida, Roberto
  • Justo Blanco, Raquel
  • Lozano Alonso, José Antonio
  • Torres Barañano, María Inés
This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In contrast to other approaches such as task-oriented dialogue systems and chit-chat implementations, the agent should display a pro-active attitude, driving the conversation to reach a number of diverse coaching goals. Therefore, the main characteristic of the introduced dialogue act taxonomy is its capacity for supporting a communication based on the GROW model for coaching. In addition, the taxonomy has a hierarchical structure between the tags and it is multimodal. We use the taxonomy to annotate a Spanish dialogue corpus collected from a group of elder people. We also present a preliminary examination of the annotated corpus and discuss on the multiple possibilities it presents for further research., The research presented in this paper is conducted as part of the project EMPATHIC that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 769872. The authors would also like to thank the support by the Basque Government through the project IT-1244-19.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Capítulo o parte de un libro (BookPart)

The EMPATHIC Project: Mid-term Achievements

  • Torres Barañano, María Inés
  • Olaso Fernández, Javier Mikel
  • Montenegro Portillo, César
  • Santana Hermida, Roberto
  • Vázquez Risco, Alain
  • Justo Blanco, Raquel
  • Lozano Alonso, José Antonio
  • Schlögl, Stephan
  • Chollet, Gérard
  • Dugan, Nazim
  • Irvine, Matthew
  • Glackin, Neil
  • Pickard, Colin
  • Esposito, Anna
  • Cordasco, Gennaro
  • Troncone, A
  • Petrovska-Delacretaz, Dijana
  • Mtibaa, Aymen
  • Hmani, Mohamed
  • Korsnes, Maria S.
  • Martinussen, Liva J.
  • Escalera, Sergio
  • Palmero Cantariño, Cristina
  • Deroo, Olivier
  • Gordeeva, Olga
  • Tenorio-Laranga, Jofre
  • González Fraile, Eduardo
  • Fernández Ruanova, Begoña
  • González Pinto Arrillaga, Ana María
The goal of active aging is to promote changes in the elderly community so as to maintain an active, independent and socially-engaged lifestyle. Technological advancements currently provide the necessary tools to foster and monitor such processes. This paper reports on mid-term achievements of the European H2020 EMPATHIC project, which aims to research, innovate, explore and validate new interaction paradigms and platforms for future generations of personalized virtual coaches to assist the elderly and their carers to reach the active aging goal, in the vicinity of their home. The project focuses on evidence-based, user-validated research and integration of intelligent technology, and context sensing methods through automatic voice, eye and facial analysis, integrated with visual and spoken dialogue system capabilities. In this paper, we describe the current status of the system, with a special emphasis on its components and their integration, the creation of a Wizard of Oz platform, and findings gained from user interaction studies conducted throughout the first 18 months of the project., European Commission
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Capítulo o parte de un libro (BookPart)

Tracking the Expression of Annoyance in Call Centers

  • Irastorza Manso, Jon
  • Torres Barañano, María Inés
Machine learning researchers have dealt with the identification of emo- tional cues from speech since it is research domain showing a large number of po- tential applications. Many acoustic parameters have been analyzed when searching for cues to identify emotional categories. Then classical classifiers and also out- standing computational approaches have been developed. Experiments have been carried out mainly over induced emotions, even if recently research is shifting to work over spontaneous emotions. In such a framework, it is worth mentioning that the expression of spontaneous emotions depends on cultural factors, on the particu- lar individual and also on the specific situation. In this work, we were interested in the emotional shifts during conversation. In particular we were aimed to track the annoyance shifts appearing in phone conversations to complaint services. To this end we analyzed a set of audio files showing different ways to express annoyance. The call center operators found disappointment, impotence or anger as expression of annoyance. However, our experiments showed that variations of parameters derived from intensity combined with some spectral information and suprasegmental fea- tures are very robust for each speaker and annoyance rate. The work also discussed the annotation problem arising when dealing with human labelling of subjective events. In this work we proposed an extended rating scale in order to include anno- tators disagreements. Our frame classification results validated the chosen annota- tion procedure. Experimental results also showed that shifts in customer annoyance rates could be potentially tracked during phone calls, Spanish Mineco under grant TIN2014- 54288-C4-4-R H2020 EU under Empathic RIA action number 769872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Euskaraz hitz egiten ikasten duten makina autodidaktak

  • López Zorrilla, Asier
  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
Lan honetan sare neuronalen bidez euskaraz hitz egiten ikasten duen elkarrizketa sistema automatikoa aurkezten dugu. Horretarako, Turingen testaren ideia era konputazionalean inplementatzen duten sare neuronal sortzaile aurkariak erabili ditugu. Normalean erabiltzen diren ingelesezko corpusak baino bi magnitude ordena txikiagoa den euskarazko corpus batekin halako sareak entrenatzea badagoela frogatzen dugu. Amaitzeko, euskararen morfologia kontuan hartzen duen aurreprozesamendua erabiltzea komenigarria dela erakusten dugu. Dakigunaren arabera, sare neuronaletan oinarrituta dagoen euskarazko lehen elkarrizketa sistema aurkezten dugu., Lan honen egileok gure esker ona adierazi nahiko genioke Eusko Jaurlaritzari, Euskal Herriko Unibertsitateari eta baita Europar Batzordeari, PRE 2017 1 0357 eta PIF17/310 zenbakidun diru laguntzekin, eta H2020 SC1-PM15 programako RIA 7 deialdiko 769872 zenbakidun diru laguntzarekin, hurrenez hurren, ikerketa hau babesteagatik.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Can Spontaneous Emotions be Detected from Speech on TV Political Debates?

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • López Zorrilla, Asier
  • Torres Barañano, María Inés
Accepted paper, Decoding emotional states from multimodal signals is an increasingly active domain, within the framework of affective computing, which aims to a better understanding of Human-Human Communication as well as to improve Human- Computer Interaction. But the automatic recognition of sponta- neous emotions from speech is a very complex task due to the lack of a certainty of the speaker states as well as to the difficulty to identify a variety of emotions in real scenarios. In this work we explore the extent to which emotional states can be decoded from speech signals extracted from TV political debates. The labelling procedure was supported by perception experiments where only a small set of emotions has been identified. In addition, some scaled judgements of valence, arousal and dominance were also provided. In this framework the paper shows meaningful comparisons between both, the dimensional and the categorical models of emotions, which is a new con- tribution when dealing with spontaneous emotions. To this end Support Vector Machines (SVM) as well as Feedforward Neural Networks (FNN) have been proposed to develop classifiers and predictors. The experimental evaluation over a Spanish corpus has shown the ability of both models to be identified in speech segments by the proposed artificial systems., This work has been partially funded by the Spanish Government under grant TIN2017-85854-C4-3-R (AEI/FEDER,UE) and conducted in the project EMPATHIC (Grant n769872) funded by the European Union’s H2020 research andinnovation program.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Adding dimensional features for emotion recognition on speech

  • Ben Letaifa Zouari, Leila
  • Torres Barañano, María Inés
  • Justo Blanco, Raquel
Developing accurate emotion recognition systems requires extracting suitable features of these emotions. In this paper, we propose an original approach of parameters extraction based on the strong, theoretical and empirical, correlation between the emotion categories and the dimensional emotions parameters. More precisely, acoustic features and dimensional emotion parameters are combined for better speech emotion characterisation. The procedure consists in developing arousal and valence models by regression on the training data and estimating, by classification, their values in the test data. Hence, when classifying an unknown sample into emotion categories, these estimations could be integrated into the feature vectors. It is noted that the results using this new set of parameters show a significant improvement of the speech emotion recognition performance., Spanish Minister of Science under grant TIN2017-85854-C4-3-R European Union’s Horizon 2020 research and innovation program under grant agreement No 769872
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Perceptual borderline for balancing multi-class spontaneous emotional data

  • Ben Letaifa Zouari, Leila
  • Torres Barañano, María Inés
Speech is a behavioural biometric signal that can provide important information to understand the human intends as well as their emotional status. The paper is centered on the speech-based identification of the seniors’s emotional status during their interaction with a virtual agent playing the role of a health professional coach. Under real conditions, we can just identify a small set of task-dependent spontaneous emotions. The number of identified samples is largely different for each emotion, which results in an imbalanced dataset problem. This research proposes the dimensional model of emotions as a perceptual representation space alternative to the generally used acoustic one. The main contribution of the paper is the definition of a perceptual borderline for the oversampling of minority emotion classes in this space. This limit, based on arousal and valence criteria, leads to two methods of balancing the data: the Perceptual Borderline oversampling and the Perceptual Borderline SMOTE (Synthetic Minority Oversampling Technique). Both methods are implemented and compared to state-of-the-art approaches of Random oversampling and SMOTE. The experimental evaluation was carried out on three imbalanced datasets of spontaneous emotions acquired in human-machine scenarios in three different cultures: Spain, France and Norway. The emotion recognition results obtained by neural networks classifiers show that the proposed perceptual oversampling methods led to significant improvements when compared with the state-of-the art, for all scenarios and languages., The research presented in this paper is conducted as partof the project EMPATHIC and of the MENHIR MSCAaction that have received funding from the European Union’s Horizon 2020 research and innovation program under grant agreements No 769872 an No 823907 respectivel
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

First Steps to Develop a Corpus of Interactions between Elderly and Virtual Agents in Spanish with Emotion Labels

  • Ben Letaifa Zouari, Leila
  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • Torres Barañano, María Inés
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Contrasting the Emotions identified in Spanish TV debates and in Human-Machine Interactions

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • Ben Letaifa Zouari, Leila
  • Torres Barañano, María Inés
Conferencia presentada en IberSPEECH 2021, 24-25 March 2021, Valladolid, Spain, This work is aimed to contrast the similarities and differences for the emotions identified in two very different scenarios: human-to-human interaction on Spanish TV debates and human-machine interaction with a virtual agent in Spanish. To this end we developed a crowd annotation procedure to label the speech signal in terms of both, emotional categories and Valence-Arousal-Dominance models. The analysis of these data showed interesting findings that allowed to profile both the speakers and the task. Then, Convolutional Neural Networks were used for the automatic classification of the emotional samples in both tasks. Experimental results drew up a different human behavior in both tasks and outlined different speaker profiles., The research presented in this paper is conducted as part of the AMIC and EMPATHIC projects project that have received funding from the Spanish Minister of Science under grant TIN2017-85854-C4-3-R and from the European Union’s Horizon 2020 research and innovation program under grant agreement No 769872. First author has also received a PhD scholarship from the University of the Basque Country UPV/EHU, PIF17/310.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Analysis of the Sensitivity of the End-Of-Turn Detection Task to Errors Generated by the Automatic Speech Recognition Process

  • Montenegro Portillo, César
  • Santana Hermida, Roberto
  • Lozano Alonso, José Antonio
An End-Of-Turn Detection Module (EOTD-M) is an essential component of automatic Spoken Dialogue Systems. The capability of correctly detecting whether a user?s utterance has ended or not improves the accuracy in interpreting the meaning of the message and decreases the latency in the answer. Usually, in dialogue systems, an EOTD-M is coupled with an Automatic Speech Recognition Module (ASR-M) to transmit complete utterances to the Natural Language Understanding unit. Mistakes in the ASR-M transcription can have a strong effect on the performance of the EOTD-M. The actual extent of this effect depends on the particular combination of ASR M transcription errors and the sentence featurization techniques implemented as part of the EOTD-M. In this paper we investigate this important relationship for an EOTD-M based on semantic information and particular characteristics of the speakers (speech profiles). We introduce an Automatic Speech Recognition Simulator (ASR-SIM) that models different types of semantic mistakes in the ASR-M transcription as well as different speech profiles. We use the simulator to evaluate the sensitivity to ASR-M mistakes of a Long Short-Term Memory network classifier trained in EOTD with different featurization techniques. Our experiments reveal the different ways in which the performance of the model is influenced by the ASR-M errors. We corroborate that not only is the ASR-SIM useful to estimate the performance of an EOTD-M in customized noisy scenarios, but it can also be used to generate training datasets with the expected error rates of real working conditions, which leads to better performance., The research presented in this paper has been conducted as part of the project EMPATHIC that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 769872. Jose A. Lozano is partially supported by the Basque Government through the BERC 2018-2021 program, IT1244-19 and grant "Artificial Intelligence in BCAM number EXP. 2019/00432'' and by the Spanish Ministry of Science, Innovation and Universities: BCAM Severo Ochoa accreditation SEV-2017-0718, TIN2016-78365-R and PID2019-104966GB-I00. And R. Santana acknowledge support by the Spanish Ministry of Science, Innovation and Universities (Project TIN201678365-R and PID2019-104966GB-I00), and the Basque Government (IT1244-19 and ELKARTEK Programs)
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico antes de ser publicado, versión del autor (Preprint)

Ageing in the 21st century in Europe: social challenges and innovation opportunities to support elderly independency and wellbeing.

  • Torres Barañano, María Inés
  • Tenorio Laranga, Joffre
  • Cordasco, Gennaro
  • González Fraile, Eduardo
  • Esposito, Anna
  • Korsnes, Maria S.
  • Chollet, Gérard
  • Fernández Ruanova, Begoña
This chapter describes the vision of the EMPATHIC project of social challenges and innovation opportunities to support elderly independency and well- being. Asanintroduction,wefirstidentifythemainchallengesthatresultfromthe current demographic status in Europe. Then, we show the vision and approach pro- posed by the EMPATHIC project to deal with some of these challenges. Next sec- tion develops the main concepts, goals and outcomes of the EMPATHIC project. The following section reports the impact of the project and the final section de- scribes the exploitation of the results as well as the concluding remarks, The research leading to the results in this paper has been conducted in the project EMPATHIC (Grant N: 769872) that received funding from the European Union’s Horizon 2020 research and innovation programme.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

A Spanish Corpus for Talking to the Elderly

  • Justo Blanco, Raquel
  • Ben Letaifa Zouari, Leila
  • Olaso Fernández, Javier Mikel
  • López Zorrilla, Asier
  • De Velasco Vázquez, Mikel
  • Vázquez Risco, Alain
  • Torres Barañano, María Inés
Paper presented at 11th International Workshop on Spoken Dialogue Systems, IWSDS 2020; Madrid; Spain; 21 September 2020 through 23 September 2020, In this work, a Spanish corpus that was developed, within the EMPATHIC project (http://www.empathic-project.eu/) framework, is presented. It was designed for building a dialogue system capable of talking to elderly people and promoting healthy habits, through a coaching model. The corpus, that comprises audio, video an text channels, was acquired by using a Wizard of Oz strategy. It was annotated in terms of different labels according to the different models that are needed in a dialogue system, including an emotion based annotation that will be used to generate empathetic system reactions. The annotation at different levels along with the employed procedure are described and analysed.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Automatic Identification of Emotional Information in Spanish TV Debates and Human-Machine Interactions

  • De Velasco Vázquez, Mikel
  • Justo Blanco, Raquel
  • Torres Barañano, María Inés
Automatic emotion detection is a very attractive field of research that can help build more natural human–machine interaction systems. However, several issues arise when real scenarios are considered, such as the tendency toward neutrality, which makes it difficult to obtain balanced datasets, or the lack of standards for the annotation of emotional categories. Moreover, the intrinsic subjectivity of emotional information increases the difficulty of obtaining valuable data to train machine learning-based algorithms. In this work, two different real scenarios were tackled: human–human interactions in TV debates and human–machine interactions with a virtual agent. For comparison purposes, an analysis of the emotional information was conducted in both. Thus, a profiling of the speakers associated with each task was carried out. Furthermore, different classification experiments show that deep learning approaches can be useful for detecting speakers’ emotional information, mainly for arousal, valence, and dominance levels, reaching a 0.7F1-score., The research presented in this paper was conducted as part of the AMIC and EMPATHIC projects, which received funding from the Spanish Minister of Science under grants TIN2017-85854-C4-3-R and PDC2021-120846-C43 and from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 769872. The first author also received a PhD scholarship from the University of the Basque Country UPV/EHU, PIF17/310.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Artículo científico (JournalArticle)

Analysis of the interaction between elderly people and a simulated virtual coach

  • Justo Blanco, Raquel
  • Ben Letaifa Zouari, Leila
  • Palmero, Cristina
  • González Fraile, Eduardo
  • Torp Johansen, Anna
  • Vázquez Risco, Alain
  • Cordasco, Gennaro
  • Schlögl, Stephan
  • Fernández Ruanova, Begoña
  • Silva, Micaela
  • Escalera, Sergio
  • De Velasco Vázquez, Mikel
  • Tenorio Laranga, Joffre
  • Esposito, Anna
  • Korsnes, Maria
  • Torres Barañano, María Inés
The EMPATHIC project develops and validates new interaction paradigms for personalized virtual coaches (VC) to promote healthy and independent aging. To this end, the work presented in this paper is aimed to analyze the interaction between the EMPATHIC-VC and the users. One of the goals of the project is to ensure an end-user driven design, involving senior users from the beginning and during each phase of the project. Thus, the paper focuses on some sessions where the seniors carried out interactions with a Wizard of Oz driven, simulated system. A coaching strategy based on the GROW model was used throughout these sessions so as to guide interactions and engage the elderly with the goals of the project. In this interaction framework, both the human and the system behavior were analyzed. The way the wizard implements the GROW coaching strategy is a key aspect of the system behavior during the interaction. The language used by the virtual agent as well as his or her physical aspect are also important cues that were analyzed. Regarding the user behavior, the vocal communication provides information about the speaker's emotional status, that is closely related to human behavior and which can be extracted from the speech and language analysis. In the same way, the analysis of the facial expression, gazes and gestures can provide information on the non verbal human communication even when the user is not talking. In addition, in order to engage senior users, their preferences and likes had to be considered. To this end, the effect of the VC on the users was gathered by means of direct questionnaires. These analyses have shown a positive and calm behavior of users when interacting with the simulated virtual coach as well as some difficulties of the system to develop the proposed coaching strategy., The research presented in this paper is conducted as part of the project EMPATHIC that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no 769872.
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Transfer learning in hierarchical dialogue topic classification with neural networks

  • Montenegro Portillo, César
  • Santana Hermida, Roberto
  • Lozano Alonso, José Antonio
Knowledge transfer between tasks can significantly improve the efficiency of machine learning algorithms. In supervised natural language understanding problems, this sort of improvement is critical since the availability of labelled data is usually scarce. In this paper we address the question of transfer learning between related topic classification tasks. A characteristic of our problem is that the tasks have a hierarchical relationship. Therefore, we introduce and validate how to implement the transfer exploiting this hierarchical structure. Our results for a real-world topic classification task show that the transfer can produce improvements in the behavior of the classifiers for some particular problems., The research presented in this paper is conducted as part of the project EM-PATHIC that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 769872.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

EMPATHIC: Empathic, Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly, EMPATHIC: Coach virtual empático, expresivo y avanzado para mejorar el bienestar de las personas de edad avanzada sanas e independientes

  • López Zorrilla, Asier
  • Velasco Vázquez, Mikel de
  • Irastorza, Jon
  • Olaso Fernández, Javier Mikel
  • Justo Blanco, Raquel
  • Torres Barañano, María Inés
The EMPATHIC project will research, innovate, explore and validate new paradigms and platforms, laying the foundation for future generations of Personalised Virtual Coaches to assist elderly people living independently at and around their home. Innovative multimodal face analytics, adaptive spoken dialogue systems and natural language interfaces are part of what the project will research and innovate, in order to help dependent aging persons and their carers. The project will use remote non-intrusive technologies to extract physiological markers of emotional states in real-time for online adaptive responses of the coach, and advance holistic modelling of behavioural, computational, physical and social aspects of a personalised expressive virtual coach. It will develop causal models of coach-user interactional exchanges that engage elders in emotionally believable interactions keeping off loneliness, sustaining health status, enhancing quality of life and simplifying access to future telecare services.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Corpus for cyberbullying prevention

  • Moreno Bilbao, M. Asunción|||0000-0002-1823-5970
  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
  • Jauk, Igor
  • Tarrés, Laia
  • Pereira, Victor
Cyberbullying is the use of digital media to harass a person or group of people, through personal attacks, disclosure of confidential or false information, among other means. That is to say, it is considered cyberbullying, or cyber-aggression to everything that is done through electronic communication devices with the intended purpose of harming or attacking a person or a group.In this paper we present a starting project to prevent cyberbullying between kids and teenagers. The idea is to create a prevention system. A system which is installed in the mobile of a kid and, if a harassment is detected, some advice is given to the child. In case of serious or repeated behavior the parents are alerted. The focus of this paper is to describe the characteristics of the database to be used to train the system, Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Corpus for cyberbullying prevention

  • Moreno Bilbao, M. Asunción
  • Bonafonte Cávez, Antonio
  • Jauk, Igor
  • Tarrés, Laia
  • Pereira, Victor
Cyberbullying is the use of digital media to harass a person or group of people, through personal attacks, disclosure of confidential or false information, among other means. That is to say, it is considered cyberbullying, or cyber-aggression to everything that is done through electronic communication devices with the intended purpose of harming or attacking a person or a group.In this paper we present a starting project to prevent cyberbullying between kids and teenagers. The idea is to create a prevention system. A system which is installed in the mobile of a kid and, if a harassment is detected, some advice is given to the child. In case of serious or repeated behavior the parents are alerted. The focus of this paper is to describe the characteristics of the database to be used to train the system, Peer Reviewed, Postprint (published version)