Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 9
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Capítulo o parte de un libro (BookPart)

Web-based embodied conversational agents and older people

  • Llorach, Gerard
  • Agenjo, Javier
  • Blat, Josep
  • Sayago Barrantes, Sergio
Within Human-Computer Interaction, there has recently been an important turn to embodied and voice-based interaction. In this chapter, we discuss our ongoing research on building online Embodied Conversational Agents (ECAs), specifically, their interactive 3D web graphics aspects. We present ECAs based on our technological pipeline, which integrates a number of free online editors, such as Adobe Fuse CC or MakeHuman, and standards, mainly BML (Behaviour Markup Language). We claim that making embodiment available for online ECAs is attainable, and advantageous over current alternatives, mostly desktop-based. In this chapter we also report on initial results of activities aimed to explore the physical appearance of ECAs for older people. A group of them (N = 14) designed female ECAs. Designing them was easy and great fun. The perspective on older-adult HCI introduced in this chapter is mostly technological, allowing for rapid online experimentations to address key issues, such as anthropomorphic aspects, in the design of ECAs with, and for, older people., This work was funded by the EU’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 675324 (ENRICH) and under the contract number H2020-645012-RIA (KRISTINA).
Addi. Archivo Digital para la Docencia y la Investigación
Publication Artículo científico (JournalArticle)

Listeners’ Spectral Reallocation Preferences for Speech in Noise

  • Simantiraki, Olympia
  • Cooke, Martin
Modifying the spectrum of recorded or synthetic speech is an effective strategy for boosting intelligibility in noise without increasing the speech level. However, the wider impact of changes to the spectral energy distribution of speech is poorly understood. The present study explored the influence of spectral modifications using an experimental paradigm in which listeners were able to adjust speech parameters directly with real-time audio feedback, allowing the joint elicitation of preferences and word recognition scores. In two experiments involving full-bandwidth and bandwidth-limited speech, respectively, listeners adjusted one of eight features that altered the speech spectrum, and then immediately carried out a sentence-in-noise recognition task at the chosen setting. Listeners' preferred adjustments in most conditions involved the transfer of speech energy from the sub-1 kHz region to the 1–4 kHz range. Preferences were not random, even when intelligibility was at the ceiling or constant across a range of adjustment values, suggesting that listener choices encompass more than a desire to maintain comprehensibility., Olympia Simantiraki was funded by the European Commission under the Marie Curie European Training Network ENRICH (675324).
Addi. Archivo Digital para la Docencia y la Investigación
Publication Contribución de congreso (ConferenceOutput)

Parallel vs. Non-parallel Voice Conversion for Esophageal Speech

  • Serrano García, Luis
  • Raman, Sneha
  • Tavarez Arriba, David
  • Navas Cordón, Eva
  • Hernáez Rioja, Inmaculada
State of the art systems for voice conversion have been shown to generate highly natural sounding converted speech. Voice conversion techniques have also been applied to alaryngeal speech, with the aim of improving its quality or its intelligibility. In this paper, we present an attempt to apply a voice conversion strategy based on phonetic posteriorgrams (PPGs), which produces very high quality converted speech, to improve the characteristics of esophageal speech. The main advantage of this PPG based architecture lies in the fact that it is able to convert speech from any source, without the need to previously train the system with a parallel corpus. However, our results show that the PPG approach degrades the intelligibility of the converted speech considerably, especially when the input speech is already poorly intelligible. In this paper two systems are compared, an LSTM based one-to-one conversion system, which is referred to as the baseline, and the new system using phonetic posteriorgrams. Both spectral parameters and f0 are converted using DNN (Deep Neural Network) based architectures. Results from both objective and subjective evaluations are presented, showing that although ASR (Automated Speech Recognition) errors are reduced, original esophageal speech is still preferred by subjects., This work has been partially funded by the Spanish Ministry of Economy and Competitiveness with FEDER support (RESTORE project, TEC2015-67163-C2-1-R), the Basque Government (BerbaOla project, KK-2018/00014) and from the EUs H2020 research and innovation programme under the MSCA GA 67532*4 (the ENRICH network: www.enrich-etn.eu).
Addi. Archivo Digital para la Docencia y la Investigación
Publication Contribución de congreso (ConferenceOutput)

Listening to Laryngectomees: A study of Intelligibility and Self-reportedListening Effort of Spanish Oesophageal Speech

  • Raman, Sneha
  • Hernáez Rioja, Inmaculada
  • Navas Cordón, Eva
  • Serrano García, Luis
Oesophageal speakers face a multitude of challenges, such asdifficulty in basic everyday communication and inability to in-teract with digital voice assistants. We aim to quantify the diffi-culty involved in understanding oesophageal speech (in human-human and human-machine interactions) by measuring intel-ligibility and listening effort. We conducted a web-based lis-tening test to collect these metrics. Participants were asked totranscribe and then rate the sentences for listening effort on a5-point Likert scale. Intelligibility, calculated as Word ErrorRate (WER), showed significant correlation with user rated ef-fort. Speaker type (healthy or oesophageal) had a major effecton intelligibility and effort. Listeners familiar with oesophagealspeech did not have any advantage over non familiar listeners incorrectly understanding oesophageal speech. However, they re-ported lesser effort in listening to oesophageal speech comparedto non familiar listeners. Additionally, we calculated speaker-wise mean WERs and they were significantly lower when com-pared to an automatic speech recognition system, This project has received funding from the European UnionsH2020 research and innovation programme under the MarieCurie European Training Network ENRICH (675324).The work has been partially funded by the Spanish Min-istry of Economy and Competitiveness with FEDER support(MINECO/FEDER, UE) (RESTORE project, TEC2015-67163-C2-1-R
Addi. Archivo Digital para la Docencia y la Investigación
Publication Contribución de congreso (ConferenceOutput)

LSTM based voice conversion for laryngectomees

  • Serrano García, Luis
  • Tavarez Arriba, David
  • Sarasola Aramendia, Xabier
  • Raman, Sneha
  • Saratxaga Couceiro, Ibon
  • Navas Cordón, Eva
  • Hernáez Rioja, Inmaculada
This paper describes a voice conversion system designed withthe aim of improving the intelligibility and pleasantness of oe-sophageal voices. Two different systems have been built, oneto transform the spectral magnitude and another one for thefundamental frequency, both based on DNNs. Ahocoder hasbeen used to extract the spectral information (mel cepstral co-efficients) and a specific pitch extractor has been developed tocalculate the fundamental frequency of the oesophageal voices.The cepstral coefficients are converted by means of an LSTMnetwork. The conversion of the intonation curve is implementedthrough two different LSTM networks, one dedicated to thevoiced unvoiced detection and another one for the predictionof F0 from the converted cepstral coefficients. The experi-ments described here involve conversion from one oesophagealspeaker to a specific healthy voice. The intelligibility of thesignals has been measured with a Kaldi based ASR system. Apreference test has been implemented to evaluate the subjectivepreference of the obtained converted voices comparing themwith the original oesophageal voice. The results show that spec-tral conversion improves ASR while restoring the intonation ispreferred by human listeners, This work has been partially funded by the Spanish Ministryof Economy and Competitiveness with FEDER support (RE-STORE project, TEC2015-67163-C2-1-R), the Basque Govern-ment (BerbaOla project, KK-2018/00014) and from the Euro-pean Unions H2020 research and innovation programme un-der the Marie Curie European Training Network ENRICH(675324).
Addi. Archivo Digital para la Docencia y la Investigación
Publication Artículo científico (JournalArticle)

A Spanish multispeaker database of esophageal speech

  • Serrano García, Luis
  • Raman, Sneha
  • Hernáez Rioja, Inmaculada
  • Navas Cordón, Eva
  • Sánchez de la Fuente, Jon
  • Saratxaga Couceiro, Ibon
A laryngectomee is a person whose larynx has been removed by surgery, usually due to laryngeal cancer. After surgery, most laryngectomees are able to speak again, using techniques that are learned with the help of a speech therapist. This is termed as alaryngeal speech, and esophageal speech (ES) is one of the several alaryngeal speech production modes. A considerable amount of research has been dedicated to the study of alaryngeal speech, with a wide range of aims such as helping speech therapists with evaluation and diagnosis, and improving its quality and intelligibility using digital signal processing techniques. We present to you a database of Spanish ES voices, named AhoSLABI, which is designed to allow the development of new support technologies for this speech impairment. The database primarily consists of recordings of 31 laryngectomees (27 males and 4 females) pronouncing phonetically balanced sentences. Additionally, it includes parallel recordings of the sentences by 9 healthy speakers (6 males and 3 females) to facilitate speech processing tasks that require small parallel corpora, such as voice conversion or synthetic speech adaptation. Apart from the sentences, the database includes sustained vowels and a small set of isolated words, which can be valuable for research on ES analysis, diagnosis and evaluation. The paper describes the main contents of the database, the recording protocols and procedure, as well as the labeling process. The main acoustic characteristics of the voices, such as speaking rate, durations of the recordings, phones and silences, and other such characteristics are compared with those of a reduced set of healthy voices. In addition, we describe an experiment using the database to improve the performance of an ASR system for ES speakers. This new resource will be made available to the scientific community with the hope that it will be used to improve the quality of life of the laryngectomees., This work was partially funded by the Spanish Ministry of Economy and Competitiveness with FEDER support (RESTORE project, TEC2015-67163-C2-1-R), the Basque Government (PIBA-018-0035) and by the European Union’s H2020 research and innovation program under the Marie Curie European Training Network ENRICH (675324)
Addi. Archivo Digital para la Docencia y la Investigación
Publication Artículo científico (JournalArticle)

Enrichment of Oesophageal Speech: Voice Conversion with Duration-Matched Synthetic Speech as Target

  • Raman, Sneha
  • Sarasola Aramendia, Xabier
  • Navas Cordón, Eva
  • Hernáez Rioja, Inmaculada
Pathological speech such as Oesophageal Speech (OS) is difficult to understand due to the presence of undesired artefacts and lack of normal healthy speech characteristics. Modern speech technologies and machine learning enable us to transform pathological speech to improve intelligibility and quality. We have used a neural network based voice conversion method with the aim of improving the intelligibility and reducing the listening effort (LE) of four OS speakers of varying speaking proficiency. The novelty of this method is the use of synthetic speech matched in duration with the source OS as the target, instead of parallel aligned healthy speech. We evaluated the converted samples from this system using a collection of Automatic Speech Recognition systems (ASR), an objective intelligibility metric (STOI) and a subjective test. ASR evaluation shows that the proposed system had significantly better word recognition accuracy compared to unprocessed OS, and baseline systems which used aligned healthy speech as the target. There was an improvement of at least 15% on STOI scores indicating a higher intelligibility for the proposed system compared to unprocessed OS, and a higher target similarity in the proposed system compared to baseline systems. The subjective test reveals a significant preference for the proposed system compared to unprocessed OS for all OS speakers, except one who was the least proficient OS speaker in the data set., This project was supported by funding from the European Union’s H2020 research and innovation programme under the MSCA GA 675324 (the ENRICH network: www.enrich-etn.eu (accessed on 25 June 2021)), and the Basque Government (PIBA_2018_1_0035 and IT355-19).
Addi. Archivo Digital para la Docencia y la Investigación
Publication Artículo científico (JournalArticle)

Contributions of listening effort and intelligibility to cortical tracking of speech in adverse listening conditions

  • Ershaid, Hadeel
  • Lizarazu, Mikel
  • McLaughlin, Drew J.
  • Cooke, Martin
  • Simantiraki, Olympia
  • Koutsogiannaki, Maria
  • Lallier, Marie
Available online 27 December 2023, Cortical tracking of speech is vital for speech segmentation and is linked to speech intelligibility. However, there is no clear consensus as to whether reduced intelligibility leads to a decrease or an increase in cortical speech tracking, warranting further investigation of the factors influencing this relationship. One such factor is listening effort, defined as the cognitive resources necessary for speech comprehension, and reported to have a strong negative correlation with speech intelligibility. Yet, no studies have examined the relationship between speech intelligibility, listening effort, and cortical tracking of speech. The aim of the present study was thus to examine these factors in quiet and distinct adverse listening conditions. Forty-nine normal hearing adults listened to sentences produced casually, presented in quiet and two adverse listening conditions: cafeteria noise and reverberant speech. Electrophysiological responses were registered with electroencephalogram, and listening effort was estimated subjectively using self-reported scores and objectively using pupillometry. Results indicated varying impacts of adverse conditions on intelligibility, listening effort, and cortical tracking of speech, depending on the preservation of the speech temporal envelope. The more distorted envelope in the reverberant condition led to higher listening effort, as reflected in higher subjective scores, increased pupil diameter, and stronger cortical tracking of speech in the delta band. These findings suggest that using measures of listening effort in addition to those of intelligibility is useful for interpreting cortical tracking of speech results. Moreover, reading and phonological skills of participants were positively correlated with listening effort in the cafeteria condition, suggesting a special role of expert language skills in processing speech in this noisy condition. Implications for future research and theories linking atypical cortical tracking of speech and reading disorders are further discussed., This research was supported by the Basque Government through the BERC 2022-2025 program, the Spanish State Research Agency through BCBL Severo Ochoa excellence accreditation CEX2020- 001010-S; SEV-2015-0490, and by the Spanish Ministry of Science, Innovation and Universities (grant ref. PRE2021-098017; project CEX2020- 001010-S-21-3 funded by MCIN/AEI/10.13039/501100011033 and FSE+). Olympia Simantiraki was funded by the European Commission under the Marie Curie European Training Network ENRICH (675324).
Recercat. Dipósit de la Recerca de Catalunya
Publication Capítulo o parte de un libro (BookPart)

Web-based embodied conversational agents and older people

  • Llorach, Gerard
  • Agenjo, Javier
  • Blat, Josep
  • Sayago Barrantes, Sergio
Within Human-Computer Interaction, there has recently been an important turn to embodied and voice-based interaction. In this chapter, we discuss our ongoing research on building online Embodied Conversational Agents (ECAs), specifically, their interactive 3D web graphics aspects. We present ECAs based on our technological pipeline, which integrates a number of free online editors, such as Adobe Fuse CC or MakeHuman, and standards, mainly BML (Behaviour Markup Language). We claim that making embodiment available for online ECAs is attainable, and advantageous over current alternatives, mostly desktop-based. In this chapter we also report on initial results of activities aimed to explore the physical appearance of ECAs for older people. A group of them (N = 14) designed female ECAs. Designing them was easy and great fun. The perspective on older-adult HCI introduced in this chapter is mostly technological, allowing for rapid online experimentations to address key issues, such as anthropomorphic aspects, in the design of ECAs with, and for, older people., This work was funded by the EU’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 675324 (ENRICH) and under the contract number H2020-645012-RIA (KRISTINA).