Pasar al contenido principal

USING INFORMATION STRUCTURE, DISCOURSE STRUCTURE AND PARAGRAPH CUES FOR PROSODY GENERATION IN SPEECH SYNTHESIS APPLICATIONS

RYC-2015-17239

•
Nombre agencia financiadora Ministerio de Ciencia e Innovación
Acrónimo agencia financiadora MICINN
Programa Programa Estatal de Promoción del Talento y su Empleabilidad en I+D+i
Subprograma Subprograma Estatal de Incorporación
Convocatoria Ramón y Cajal
Año convocatoria 2015
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Unidad de gestión Secretaría de Estado de Investigación, Desarrollo e Innovación
Centro beneficiario UNIVERSITAT POMPEU FABRA CCT
Identificador persistente http://dx.doi.org/10.13039/501100004837

Resultados relacionados

Resultados totales (Incluyendo duplicados): 8
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

The Information structure–prosody interface in text-to-speech technologies: an empirical perspective

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Wanner, Leo
The correspondence between the communicative intention of a speaker in terms of Information Structure and the way this speaker reflects communicative aspects by means of prosody have been a fruitful field of study in Linguistics. However, text-to-speech applications still lack the variability and richness found in human speech in terms of how humans display their communication skills. Some attempts were made in the past to model one aspect of Information Structure, namely thematicity for its application to intonation generation in text-to-speech technologies. Yet these applications suffer from two limitations: (i) they draw upon a small number of made-up simple question-answer pairs rather than on real (spoken or written) corpus material; and (ii) they do not explore whether any other interpretation would better suit a wider range of textual genres beyond dialogues. In this paper, two different interpretations of thematicity in the field of speech technologies are examined: the state-of-art binary (and flat) theme-rheme, and the hierarchical thematicity defined by Igor Mel’ˇcuk within the Meaning-Text Theory. The outcome of the experiments on a corpus of native speakers of US English suggests that the latter interpretation of thematicity has a versatile implementation potential for text-to-speech applications of the Information Structure-–prosody interface, This work has been partially funded by the European Commission in the context of its H2020 Programme under the contract numbers H2020-645012-RIA (KRISTINA) and H2020-870930-IA (WELCOME). The second author has been funded by the Agencia Estatal de Investigaci´on (AEI), Ministerio de Ciencia, Innovaci´on y Universidades and the Fondo Social Europeo (FSE), grant RYC2015-17239 (AEI/FSE, UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Detection of speech events and speaker characteristics through photo-plethysmographic signal neural processing

  • Cámbara Ruiz, Guillermo
  • Luque, Jordi
  • Farrús, Mireia
Comunicació presentada a: 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), celebrat de manera virtual a Barcelona del 4 al 8 de maig de 2020., The use of photoplethysmogram signal (PPG) for heart and sleep monitoring is commonly found nowadays in smart-phones and wrist wearables. Besides common usages, it has been proposed and reported that person information can be extracted from PPG for other uses, like biometry tasks. In this work, we explore several end-to-end convolutional neural network architectures for detection of human’s characteristics such as gender or person identity. In addition, we evaluate whether speech/non-speech events may be inferred from PPG signal, where speech might translate in fluctuations into the pulse signal. The obtained results are promising and clearly show the potential of fully end-to-end topologies for automatic extraction of meaningful biomarkers, even from a noisy signal sampled by a low-cost PPG sensor. The AUCs for best architectures put forward PPG wave as biological discriminant, reaching 79% and 89.0%, respectively for gender and person verification tasks. Furthermore, speech detection experiments reporting AUCs around 69% encourage us for further exploration about the feasibility of PPG for speech processing tasks.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Acoustic and prosodic information for home monitoring of bipolar disorder

  • Farrús, Mireia
  • Codina Filbà, Joan
  • Escudero, Joan
Epidemiological studies suggest that bipolar disorder has a prevalence of about 1% in European countries, becoming one of the most disabling illnesses in working age adults, and often long-term and persistent with complex management and treatment. Therefore, the capacity of home monitoring for patients with this disorder is crucial for their quality of life. The current paper introduces the use of speech-based information as an easy-to-record, ubiquitous and non-intrusive health sensor suitable for home monitoring, and its application in the framework on the NYMPHA-MD project. Some preliminary results also show the potential of acoustic and prosodic features to detect and classify bipolar disorder, by predicting the values of the Hamilton Depression Rating Scale (HDRS) and the Young Mania Rating Scale (YMRS) from speech., The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is part of the NYMPHA-MD project, which has been funded by the European Union under Grant Agreement No. 610462. The first author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Data augmentation for low-resource Quechua ASR improvement

  • Zevallos, Rodolfo
  • Bel Rafecas, Núria
  • Cámbara Ruiz, Guillermo
  • Farrús, Mireia
  • Luque, Jordi
Comunicació presentada a INTERSPEECH 2022, celebrat del 18 al 22 de setembre de 2022 a Inchon, Corea del Sud., Automatic Speech Recognition (ASR) is a key element in new services that helps users to interact with an automated system. Deep learning methods have made it possible to deploy systems with word error rates below 5% for ASR of English. However, the use of these methods is only available for languages with hundreds or thousands of hours of audio and their corresponding transcriptions. For the so-called low-resource languages to speed up the availability of resources that can improve the performance of their ASR systems, methods of creating new resources on the basis of existing ones are being investigated. In this paper we describe our data augmentation approach to improve the results of ASR models for low-resource and agglutinative languages. We carry out experiments developing an ASR for Quechua using the wav2letter++ model. We reduced WER by 8.73% through our approach to the base model. The resulting ASR model obtained 22.75% WER and was trained with 99 hours of original resources and 99 hours of synthetic data obtained with a combination of text augmentation and synthetic speech generation., This work has been partially supported by the Project PID2019-104512GB-I00, Ministerio de Ciencia e Innovación and Agencia Estatal de Investigación (Spain), and the INGENIOUS project from the European Union’s Horizon 2020 Research and Innovation Program under grant numbers 833435. The third author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Data augmentation for low-resource Quechua ASR improvement

  • Zevallos, Rodolfo
  • Bel Rafecas, Núria
  • Cámbara Ruiz, Guillermo
  • Farrús, Mireia
  • Luque, Jordi
Comunicació presentada a INTERSPEECH 2022, celebrat del 18 al 22 de setembre de 2022 a Inchon, Corea del Sud., Automatic Speech Recognition (ASR) is a key element in new services that helps users to interact with an automated system. Deep learning methods have made it possible to deploy systems with word error rates below 5% for ASR of English. However, the use of these methods is only available for languages with hundreds or thousands of hours of audio and their corresponding transcriptions. For the so-called low-resource languages to speed up the availability of resources that can improve the performance of their ASR systems, methods of creating new resources on the basis of existing ones are being investigated. In this paper we describe our data augmentation approach to improve the results of ASR models for low-resource and agglutinative languages. We carry out experiments developing an ASR for Quechua using the wav2letter++ model. We reduced WER by 8.73% through our approach to the base model. The resulting ASR model obtained 22.75% WER and was trained with 99 hours of original resources and 99 hours of synthetic data obtained with a combination of text augmentation and synthetic speech generation., This work has been partially supported by the Project PID2019-104512GB-I00, Ministerio de Ciencia e Innovación and Agencia Estatal de Investigación (Spain), and the INGENIOUS project from the European Union’s Horizon 2020 Research and Innovation Program under grant numbers 833435. The third author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Detection of speech events and speaker characteristics through photo-plethysmographic signal neural processing

  • Cámbara Ruiz, Guillermo
  • Luque, Jordi
  • Farrús, Mireia
Comunicació presentada a: 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), celebrat de manera virtual a Barcelona del 4 al 8 de maig de 2020., The use of photoplethysmogram signal (PPG) for heart and sleep monitoring is commonly found nowadays in smart-phones and wrist wearables. Besides common usages, it has been proposed and reported that person information can be extracted from PPG for other uses, like biometry tasks. In this work, we explore several end-to-end convolutional neural network architectures for detection of human’s characteristics such as gender or person identity. In addition, we evaluate whether speech/non-speech events may be inferred from PPG signal, where speech might translate in fluctuations into the pulse signal. The obtained results are promising and clearly show the potential of fully end-to-end topologies for automatic extraction of meaningful biomarkers, even from a noisy signal sampled by a low-cost PPG sensor. The AUCs for best architectures put forward PPG wave as biological discriminant, reaching 79% and 89.0%, respectively for gender and person verification tasks. Furthermore, speech detection experiments reporting AUCs around 69% encourage us for further exploration about the feasibility of PPG for speech processing tasks.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Acoustic and prosodic information for home monitoring of bipolar disorder

  • Farrús, Mireia
  • Codina Filbà, Joan
  • Escudero, Joan
Epidemiological studies suggest that bipolar disorder has a prevalence of about 1% in European countries, becoming one of the most disabling illnesses in working age adults, and often long-term and persistent with complex management and treatment. Therefore, the capacity of home monitoring for patients with this disorder is crucial for their quality of life. The current paper introduces the use of speech-based information as an easy-to-record, ubiquitous and non-intrusive health sensor suitable for home monitoring, and its application in the framework on the NYMPHA-MD project. Some preliminary results also show the potential of acoustic and prosodic features to detect and classify bipolar disorder, by predicting the values of the Hamilton Depression Rating Scale (HDRS) and the Young Mania Rating Scale (YMRS) from speech., The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is part of the NYMPHA-MD project, which has been funded by the European Union under Grant Agreement No. 610462. The first author has been funded by the Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and the Fondo Social Europeo (FSE) under grant RYC-2015-17239 (AEI/FSE, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The Information structure–prosody interface in text-to-speech technologies: an empirical perspective

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Wanner, Leo
The correspondence between the communicative intention of a speaker in terms of Information Structure and the way this speaker reflects communicative aspects by means of prosody have been a fruitful field of study in Linguistics. However, text-to-speech applications still lack the variability and richness found in human speech in terms of how humans display their communication skills. Some attempts were made in the past to model one aspect of Information Structure, namely thematicity for its application to intonation generation in text-to-speech technologies. Yet these applications suffer from two limitations: (i) they draw upon a small number of made-up simple question-answer pairs rather than on real (spoken or written) corpus material; and (ii) they do not explore whether any other interpretation would better suit a wider range of textual genres beyond dialogues. In this paper, two different interpretations of thematicity in the field of speech technologies are examined: the state-of-art binary (and flat) theme-rheme, and the hierarchical thematicity defined by Igor Mel’ˇcuk within the Meaning-Text Theory. The outcome of the experiments on a corpus of native speakers of US English suggests that the latter interpretation of thematicity has a versatile implementation potential for text-to-speech applications of the Information Structure-–prosody interface, This work has been partially funded by the European Commission in the context of its H2020 Programme under the contract numbers H2020-645012-RIA (KRISTINA) and H2020-870930-IA (WELCOME). The second author has been funded by the Agencia Estatal de Investigaci´on (AEI), Ministerio de Ciencia, Innovaci´on y Universidades and the Fondo Social Europeo (FSE), grant RYC2015-17239 (AEI/FSE, UE).