Skip to main content

TECNOLOGIAS DE APRENDIZAJE PROFUNDO APLICADAS AL PROCESADO DE VOZ Y AUDIO

TEC2015-69266-P

•
Funding agency name Ministerio de Economía y Competitividad
Funding agency acronym MINECO
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2015
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Management unit Secretaría de Estado de Educación, Formación Profesional y Universidades
Beneficiary centre UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 60
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Towards large scale multimedia indexing: a case study on person discovery in broadcast news

  • Le, Nam
  • Bredin, Hervé
  • Sargent, Gabriel
  • India, Miquel
  • López Otero, Paula
  • Barras, Claude
  • Guinaudeau, Camille
  • Gravier, Guillaume
  • Barbosa da Fonseca, Gabriel
  • Lyon Freire, Izabela
  • Patrocínio, Zenilton
  • Guimarães, Silvio Jamil F.
  • Martí Juan, Gerard
  • Morros, Josep Ramon
  • Hernando, Javier
  • Docio Fernández, Laura
  • García Mateo, Carmen
  • Meignier, Sylvain
  • Odobez, Jean-Marc
Comunicació presentada a: the 15th International Workshop on Content-Based Multimedia Indexing (CBMI'17), celebrat a Florència, Itàlia, del 19 al 21 de juny de 2017, The rapid growth of multimedia databases and the human interest in their peers make indices representing the location and identity of people in audio-visual documents essential for searching archives. Person discovery in the absence of prior identity knowledge requires accurate association of audio-visual cues and detected names. To this end, we present 3 different strategies to approach this problem: clustering-based naming, verification-based naming, and graph-based naming. Each of these strategies utilizes different recent advances in unsupervised face / speech representation, verification, and optimization. To have a better understanding of the approaches, this paper also provides a quantitative and qualitative comparative study of these approaches using the associated corpus of the Person Discovery challenge at MediaEval 2016. From the results of our experiments, we can observe the pros and cons of each approach, thus paving the way for future promising research directions., This work was supported by the EU project EUMSSI (FP7-611057), ANR project MetaDaTV (ANR-14-CE24-0024) project, Camomile project (PCIN-2013-067), and the projects TEC2013-43935-R, TEC2015-69266-P, TEC2016-75976-R, TEC2015-65345-P financed by the Spanish government and ERDF.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

CATOTRON – A Neural text-to-speech system in catalan

  • Külebi, Baybars
  • Öktem, Alp
  • Peiró Lilja, Àlex
  • Pascual, Santiago
  • Farrús, Mireia
Comunicació presentada a Interspeech 2020 celebrat del 25 al 29 d'octubre de 2020 a Shanghai, Xina., We present Catotron, a neural network-based open-source speech synthesis system in Catalan. Catotron consists of a sequence-to-sequence model trained with two small opensource datasets based on semi-spontaneous and read speech. We demonstrate how a neural TTS can be built for languages with limited resources using found-data optimization and crosslingual transfer learning. We make the datasets, initial models and source code publicly available for both commercial and research purposes., This work was subsidised by the Catalan Department of Culture. A part of the funding comes from the financing administered by the inheritance board of the Generalitat de Catalunya. The last author has been funded by the Agencia Estatal de Investigaci ´on (AEI), Ministerio de Ciencia, Innovaci´on y Universidades and the Fondo Social Europeo (FSE) under grant RYC- 2015-17239 (AEI/FSE, UE). Part of this work has been carried out using an NVIDIA GPU Titan Xp generously provided by NVIDIA Company. This research was also partially supported by the project TEC2015-69266-P (MINECO/FEDER, UE).
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Auto-encoding nearest neighbor i-vectors for speaker verification

  • Khan, Umair
  • India Massana, Miquel Àngel|||0000-0002-3107-3662
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
In the last years, i-vectors followed by cosine or PLDA scoringtechniques were the state-of-the-art approach in speaker veri-fication. PLDA requires labeled background data, and thereexists a significant performance gap between the two scoringtechniques. In this work, we propose to reduce this gap by us-ing an autoencoder to transform i-vector into a new speaker vec-tor representation, which will be referred to as ae-vector. Theautoencoder will be trained to reconstruct neighbor i-vectors in-stead of the same training i-vectors, as usual. These neighbori-vectors will be selected in an unsupervised manner accordingto the highest cosine scores to the training i-vectors. The evalua-tion is performed on the speaker verification trials of VoxCeleb-1 database. The experiments show that our proposed ae-vectorsgain a relative improvement of 42% in terms of EER comparedto the conventional i-vectors using cosine scoring, which fillsthe performance gap between cosine and PLDA scoring tech-niques by 92%, but without using speaker labels, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A geometric proxy of economic uncertainty based on the disagreement in survey expectations

  • Claveria González, Oscar
  • Monte Moreno, Enrique|||0000-0002-4907-0494
  • Torra Porras, Salvador
In this study we present a geometric approach to proxy economic uncertainty. We design a positional indicator of disagreement among survey-based agents' expectations about the state of the economy. Previous dispersion-based uncertainty measures derived from business and consumer surveys exclusively make use of the two extreme pieces of information: the percentage of respondents expecting a variable to rise and to fall. With the aim of also incorporating the information coming from the share of respondents expecting a variable to remain constant, we propose a geometrical framework and use a barycentric coordinate system to generate a metric of disagreement, referred to as a discrepancy indicator. We assess its performance, both empirically and experimentally, by comparing it to the standard deviation of the share of positive and negative responses, which has been used by Bachman et al. (2013) as a proxy for economic uncertainty. When applied in sixteen European countries, we find that both time-varying metrics co-evolve in most countries for expectations about the country's overall economic situation in the present, but not in the future. Additionally, we obtain their simulated sampling distributions and we find that the proposed indicator gravitates uniformly towards the three vertices of the simplex representing the three answering categories, as opposed to the standard deviation, which tends to overestimate the level of uncertainty as a result of ignoring the no-change responses. Consequently, we find evidence that the information coming from agents expecting a variable to remain constant has an effect on the measurement of disagreement., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Spanish statistical parametric speech synthesis using a neural vocoder

  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
  • Pascual de la Puente, Santiago|||0000-0002-8365-7387
  • Dorca, G.
During the 2000s decade, unit-selection based text-to-speech was the dominant commercial technology. Meanwhile, the TTS research community has made a big effort to push statistical-parametric speech synthesis to get similar quality and more flexibility on the synthetically generated voice. During last years, deep learning advances applied to speech synthesis have filled the gap, specially when neural vocoders substitute traditional signal-processing based vocoders. In this paper we propose to substitute the waveform generation vocoder of MUSA, our Spanish TTS, with SampleRNN, a neural vocoder which was recently proposed as a deep autoregressive raw waveform generation model. MUSA uses recurrent neural networks to predict vocoder parameters (MFCC and logF0) from linguistic features. Then, the Ahocoder vocoder is used to recover the speech waveform out of the predicted parameters. In the first system SampleRNN is extended to generate speech conditioned on the Ahocoder generated parameters (mfcc and logF0), where two configurations have been considered to train the system. First, the parameters derived from the signal using Ahocoder are used. Secondly, the system is trained with the parameters predicted by MUSA, where SampleRNN and MUSA are jointly optimized. The subjective evaluation shows that the second system outperforms both the original Ahocoder and SampleRNN as an independent neural vocoder., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Evaluating the underlying gender bias in contextualized word embeddings

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Casas Manzanares, Noé
Gender bias is highly impacting natural language processing applications. Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources. Recently, contextualized word embeddings have enhanced previous word embedding techniques by computing word vector representations dependent on the sentence they appear in. In this paper, we study the impact of this conceptual change in the word embedding computation in relation with gender bias. Our analysis includes different measures previously applied in the literature to standard word embeddings. Our findings suggest that contextualized word embeddings are less biased than standard ones even when the latter are debiased., We want to thank Hila Gonen for her support dur-ing our research.This work is supported in part by the Catalan Agency for Management of University andResearch Grants (AGAUR) through the FI PhDScholarship and the Industrial PhD Grant. Thiswork is also supported in part by the Span-ish Ministerio de Economa y Competitividad, the European Regional Development Fund andthe Agencia Estatal de Investigacin, through thepostdoctoral senior grant Ramn y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU)and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Self multi-head attention for speaker recognition

  • India Massana, Miquel Àngel|||0000-0002-3107-3662
  • Safari, Pooyan
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
Most state-of-the-art Deep Learning (DL) approaches forspeaker recognition work on a short utterance level. Given thespeech signal, these algorithms extract a sequence of speakerembeddings from short segments and those are averaged to ob-tain an utterance level speaker representation. In this work wepropose the use of an attention mechanism to obtain a discrim-inative speaker embedding given non fixed length speech utter-ances. Our system is based on a Convolutional Neural Network(CNN) that encodes short-term speaker features from the spec-trogram and a self multi-head attention model that maps theserepresentations into a long-term speaker embedding. The atten-tion model that we propose produces multiple alignments fromdifferent subsegments of the CNN encoded states over the se-quence. Hence this mechanism works as a pooling layer whichdecides the most discriminative features over the sequence toobtain an utterance level representation. We have tested thisapproach for the verification task for the VoxCeleb1 dataset.The results show that self multi-head attention outperforms bothtemporal and statistical pooling methods with a18%of rela-tive EER. Obtained results show a58%relative improvementin EER compared to i-vector+PLDA, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A novel approach to real-time range estimation of underwater acoustic sources using supervised machine learning

  • Houégnigan, Ludwig|||0000-0003-2903-3140
  • Safari, Pooyan
  • Nadeu Camprubí, Climent|||0000-0002-5863-0983
  • Van der Schaar, Mike Connor Roger Malcolm
  • André, Michel|||0000-0002-0091-7279
© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The proposed paper introduces a novel method for range estimation of acoustic sources, both cetaceans and industrial sources, in deep sea environments using supervised learning with neural networks in the contex of a single sensor, a compact array, or a small aperture towed array. The presented results have potential both for industrial impact and for the conservation and density estimation of cetaceans. With an average error of 4.3% for ranges up to 8 kilometers and typically below 300 meters, those results are challenging and to our knowledge they are unprecedented for an automated real-time solution., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Byte-based neural machine translation

  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
This paper presents experiments compar- ing character-based and byte-based neural machine translation systems. The main motivation of the byte-based neural ma- chine translation system is to build multi- lingual neural machine translation systems that can share the same vocabulary. We compare the performance of both systems in several language pairs and we see that the performance in test is similar for most language pairs while the training time is slightly reduced in the case of byte-based neural machine translation.
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Terminology-aware segmentation and domain feature for the WMT19 biomedical translation task

  • Carrino, Casimiro Pio
  • Rafieian, Bardia
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
In this work, we give a description of the TALP-UPC systems submitted for the WMT19 Biomedical Translation Task. Our proposed strategy is NMT model-independent and relies only on one ingredient, a biomedical terminology list. We first extracted such a terminology list by labelling biomedical words in our training dataset using the BabelNet API. Then, we designed a data preparation strategy to insert the terms information at a token level. Finally, we trained the Transformer model with this terms-informed data. Our best-submitted system ranked 2nd and 3rd for Spanish-English and English-Spanish translation directions, respectively., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

DNN speaker embeddings using autoencoder pre-training

  • Khan, Umair
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
Over the last years, i-vectors have been the state-of-the-art approach in speaker recognition. Recent improvements in deep learning have increased the discriminative quality of i-vectors. However, deep learning architectures require a large amount of labeled background data which is difficult in practice. The aim of this paper is to propose an alternative scheme in order to reduce the need of labeled data. We propose the use of autoencoder pre-training in a speaker verification task. First, we train an autoencoder in an unsupervised way, using a large amount of unlabeled background data. Then, we train a Deep Neural Network (DNN) initialized with the parameters of the pre-trained autoencoder. The DNN training is carried out in a supervised way using relatively small labeled background data. In the testing phase, we extract speaker embeddings as the output of an intermediate layer of the DNN. The training and evaluation were performed on VoxCeleb-2 and VoxCeleb1 databases, respectively. The experimental results have shown that by initializing DNN with the parameters of the pre-trained autoencoder, we have achieved a relative improvement of 21%, in terms of Equal Error Rate (EER), over the baseline i-vector/PLDA system., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

I-vector transformation using k-nearest neighbors for speaker verification

  • Khan, Umair
  • India Massana, Miquel Àngel|||0000-0002-3107-3662
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. https://dx.doi.org/10.1109/ICASSP40776.2020.9053504, Probabilistic Linear Discriminant Analysis (PLDA) is the most efficient backend for i-vectors. However, it requires labeled background data which can be difficult to access in practice. Unlike PLDA, cosine scoring avoids speaker-labels at the cost of degrading the performance. In this work, we propose a post processing of i-vectors using a Deep Neural Network (DNN) to transform i-vectors into a new speaker vector representation. The DNN will be trained using i-vectors that are similar to the training i-vectors. These similar i-vectors will be selected in an unsupervised manner. Using the new vector representation, we will score the experimental trials using cosine scoring. The evaluation was performed on the speaker verification trials of VoxCeleb-1 database. The experiments have shown that with the help of the similar i-vectors the new vectors become more discriminative than the original i-vectors. The new vectors have gained a relative improvement of 53% in terms of EER, compared to the conventional i-vector/PLDA system, but without using speaker labels., This work has been developed in the framework of DeepVoice Project (TEC2015-69266-P), funded by Spanish Ministry., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Prosodic and spectral iVectors for expressive speech synthesis

  • Jauk, Igor
  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
This work presents a study on the suitability of prosodic andacoustic features, with a special focus on i-vectors, in expressivespeech analysis and synthesis. For each utterance of two dif-ferent databases, a laboratory recorded emotional acted speech,and an audiobook, several prosodic and acoustic features are ex-tracted. Among them, i-vectors are built not only on the MFCCbase, but also on F0, power and syllable durations. Then, un-supervised clustering is performed using different feature com-binations. The resulting clusters are evaluated calculating clus-ter entropy for labeled portions of the databases. Additionally,synthetic voices are trained, applying speaker adaptive training,from the clusters built from the audiobook. The voices are eval-uated in a perceptual test where the participants have to edit anaudiobook paragraph using the synthetic voices.The objective results suggest that i-vectors are very use-ful for the audiobook, where different speakers (book charac-ters) are imitated. On the other hand, for the laboratory record-ings, traditional prosodic features outperform i-vectors. Also,a closer analysis of the created clusters suggest that differentspeakers use different prosodic and acoustic means to conveyemotions. The perceptual results suggest that the proposed i-vector based feature combinations can be used for audiobookclustering and voice training., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Automatic speech recognition with deep neural networks for impaired speech

  • España-i-Bonet, Cristina
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
The final publication is available at https://link.springer.com/chapter/10.1007%2F978-3-319-49169-1_10, Automatic Speech Recognition has reached almost human performance in some controlled scenarios. However, recognition of impaired speech is a difficult task for two main reasons: data is (i) scarce and (ii) heterogeneous. In this work we train different architectures on a database of dysarthric speech. A comparison between architectures shows that, even with a small database, hybrid DNN-HMM models outperform classical GMM-HMM according to word error rate measures. A DNN is able to improve the recognition word error rate a 13% for subjects with dysarthria with respect to the best classical architecture. This improvement is higher than the one given by other deep neural networks such as CNNs, TDNNs and LSTMs. All the experiments have been done with the Kaldi toolkit for speech recognition for which we have adapted several recipes to deal with dysarthric speech and work on the TORGO database. These recipes are publicly available., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Multilingual, multi-scale and multi-layer visualization of sequence-based intermediate representations

  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Lacroux, Elora
  • Vázquez Alcocer, Pere Pau|||0000-0003-4638-4065
The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation., This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

From bilingual to multilingual neural machine translation by incremental training

  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
Multilingual Neural Machine Translation approaches are based on the use of task specific models and the addition of one more language can only be done by retraining the whole system. In this work, we propose a new training schedule that allows the system to scale to more languages without modification of the previous components based on joint training and language-independent encoder/decoder modules allowing for zero-shot translation. This work in progress shows close results to state-of-the-art in the WMT task., This work is supported in part by a Google Faculty Research Award. This work is also supported in part by the Spanish Ministerio de Economa y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacin, through the postdoctoral senior grant Ramon y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017- 079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Self-attention encoding and pooling for speaker recognition

  • Safari, Pooyan
  • India Massana, Miquel Àngel|||0000-0002-3107-3662
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
The computing power of mobile devices limits the end-user applications in terms of storage size, processing, memory and energy consumption. These limitations motivate researchers for the design of more efficient deep models. On the other hand, self-attention networks based on Transformer architecture have attracted remarkable interests due to their high parallelization capabilities and strong performance on a variety of Natural Language Processing (NLP) applications. Inspired by the Transformer, we propose a tandem Self-Attention Encoding and Pooling (SAEP) mechanism to obtain a discriminative speaker embedding given non-fixed length speech utterances. SAEP is a stack of identical blocks solely relied on self-attention and position-wise feed-forward networks to create vector representation of speakers. This approach encodes short-term speaker spectral features into speaker embeddings to be used in text-independent speaker verification. We have evaluated this approach on both VoxCeleb1 & 2 datasets. The proposed architecture is able to outperform the baseline x-vector, and shows competitive performance to some other benchmarks based on convolutions, with a significant reduction in model size. It employs 94%, 95%, and 73% less parameters compared to ResNet-34, ResNet-50, and x-vector, respectively. This indicates that the proposed fully attention based architecture is more efficient in extracting time-invariant features from speaker utterances., This work was supported in part by the Spanish Project DeepVoice (TEC2015-69266-P)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Problem-agnostic speech embeddings for multi-speaker text-to-speech with SampleRNN

  • Álvarez Robert, David
  • Pascual de la Puente, Santiago|||0000-0002-8365-7387
  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
Text-to-speech (TTS) acoustic models map linguistic features into an acoustic representation out of which an audible waveform is generated. The latest and most natural TTS systems build a direct mapping between linguistic and waveform domains, like SampleRNN. This way, possible signal naturalness losses are avoided as intermediate acoustic representations are discarded. Another important dimension of study apart from naturalness is their adaptability to generate voice from new speakers that were unseen during training. In this paper we first propose the use of problem-agnostic speech embeddings in a multi-speaker acoustic model for TTS based on SampleRNN. This way, we feed the acoustic model with speaker acousticallydependent representations that enrich the waveform generation more than embeddings unrelated to these factors. Our first results suggest that the proposed embeddings lead to better quality voices than those obtained with one-hot embeddings. Furthermore, as we can use any speech segment as an encoded representation during inference, the model is capable to generalize to new speaker identities without retraining the network. We finally show that, with a small increase of speech duration in the embedding extractor, we dramatically reduce the spectral distortion to close the gap towards the target identities., This research was supported by the project TEC2015-69266-P (MINECO/FEDER, UE)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Restricted Boltzmann Machine vectors for speaker clustering

  • Khan, Umair
  • Safari, Pooyan
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
Restricted Boltzmann Machines (RBMs) have been used both in the front-end and backend of speaker verification systems. In this work, we apply RBMs as a front-end in the context of speaker clustering. Speakers' utterances are transformed into a vector representation by means of RBMs. These vectors, referred to as RBM vectors, have shown to preserve speaker-specific information and are used for the task of speaker clustering. In this work, we perform the traditional bottom-up Agglomerative Hierarchical Clustering (AHC). Using the RBM vector representation of speakers, the performance of speaker clustering is improved. The evaluation has been performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed system outperforms the baseline i-vectors system in terms of Equal Impurity (EI). Using cosine scoring, a relative improvement of 11% and 12% are achieved for average and single linkage clustering algorithms respectively. Using PLDA scoring, the RBM vectors achieve a relative improvement of 11% compared to i-vectors for the single linkage algorithm., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

A differentiable BLEU loss. Analysis and first results

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
In natural language generation tasks, like neural machine translation and image captioning, there is usually a mismatch between the optimized loss and the de facto evaluation criterion, namely token-level maximum likelihood and corpus-level BLEU score. This article tries to reduce this gap by defining differentiable computations of the BLEU and GLEU scores. We test this approach on simple tasks, obtaining valuable lessons on its potential applications but also its pitfalls, mainly that these loss functions push each token in the hypothesis sequence toward the average of the tokens in the reference, resulting in a poor training signal., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

UPC multimodal speaker diarization system for the 2018 Albayzin challenge

  • India Massana, Miquel Àngel|||0000-0002-3107-3662
  • Sagastiberri, Itziar
  • Palau Puigdevall, Ponç
  • Sayrol Clols, Elisa|||0000-0002-0526-9733
  • Morros Rubió, Josep Ramon|||0000-0002-1395-487X
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
This paper presents the UPC system proposed for the Multimodal Speaker Diarization task of the 2018 Albayzin Challenge. This approach works by processing individually the speech and the image signal. In the speech domain, speaker diarization is performed using identity embeddings created by a triplet loss DNN that uses i-vectors as input. The triplet DNN is trained with an additional regularization loss that minimizes the variance of both positive and negative distances. A sliding windows is then used to compare speech segments with enrollment speaker targets using cosine distance between the embeddings. To detect identities from the face modality, a face detector followed by a face tracker has been used on the videos. For each cropped face a feature vector is obtained using a Deep Neural Network based on the ResNet 34 architecture, trained using a metric learning triplet loss (available from dlib library). For each track the face feature vector is obtained by averaging the features obtained for each one of the frames of that track. Then, this feature vector is compared with the features extracted from the images of the enrollment identities. The proposed system is evaluated on the RTVE2018 database., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Refinement of unsupervised cross-lingual word embeddings

  • Biesialska, Magdalena Marta|||0000-0001-7890-3523
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
Cross-lingual word embeddings aim to bridge the gap between high-resource and low-resource languages by allowing to learn multilingual word representations even without using any direct bilingual signal. The lion's share of the methods are projection-based approaches that map pre-trained embeddings into a shared latent space. These methods are mostly based on the orthogonal transformation, which assumes language vector spaces to be isomorphic. However, this criterion does not necessarily hold, especially for morphologically-rich languages. In this paper, we propose a self-supervised method to refine the alignment of unsupervised bilingual word embeddings. The proposed model moves vectors of words and their corresponding translations closer to each other as well as enforces length- and center-invariance, thus allowing to better align cross-lingual embeddings. The experimental results demonstrate the effectiveness of our approach, as in most cases it outperforms state-of-the-art methods in a bilingual lexicon induction task., We thank anonymous reviewers for their helpful comments. This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, the contract TEC2015-69266-P (MINECO/FEDER,EU) and the contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Multi-output RNN-LSTM for multiple speaker speech synthesis with a-interpolation model

  • Pascual, Santiago
  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
Deep Learning has been applied successfully to speech processing. In this paper we propose an architecture for speech synthesis using multiple speakers. Some hidden layers are shared by all the speakers, while there is a specific output layer for each speaker. Objective and perceptual experiments prove that this scheme produces much better results in comparison with sin- gle speaker model. Moreover, we also tackle the problem of speaker interpolation by adding a new output layer (a-layer) on top of the multi-output branches. An identifying code is injected into the layer together with acoustic features of many speakers. Experiments show that the a-layer can effectively learn to interpolate the acoustic features between speakers., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Towards mitigating gender bias in a decoder-based neural machine translation model by adding contextual information

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
Gender bias negatively impacts many natural language processing applications, including ma-chine translation (MT). The motivation behind this work is to study whether recent proposedMT techniques are significantly contributing to attenuate biases in document-level and gender-balanced data. For the study, we consider approaches of adding the previous sentence and thespeaker information, implemented in a decoder-based neural MT system. We show improve-ments both in translation quality (+1 BLEU point) as well as in gender bias mitigation onWinoMT (+5% accuracy)., This work is supported in part by the Catalan Agency for Management of University and Research Grants (AGAUR) through the FI PhD Scholarship. This work is also supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacion, through the postdoctoral senior grant Ramón y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

The TALP-UPC machine translation systems for WMT19 news translation task: pivoting techniques for low resource MT

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
In this article, we describe the TALP-UPC research group participation in the WMT19 news translation shared task for Kazakh-English. Given the low amount of parallel training data, we resort to using Russian as pivot language, training subword-based statistical translation systems for Russian-Kazakh and Russian-English that were then used to create two synthetic pseudo-parallel corpora for Kazakh-English and English-Kazakh respectively. Finally, a self-attention model based on the decoder part of the Transformer architecture was trained on the two pseudo-parallel corpora., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Unsupervised training of siamese networks for speaker verification

  • Khan, Umair
  • Hernando Pericás, Francisco Javier|||0000-0002-1730-8154
Speaker labeled background data is an essential requirement for most state-of-the-art approaches in speaker recognition, e.g., xvectors and i-vector/PLDA. However, in reality it is difficult to access large amount of labeled data. In this work, we propose siamese networks for speaker verification without using speaker labels. We propose two different siamese networks having two and three branches, respectively, where each branch is a CNN encoder. Since the goal is to avoid speaker labels, we propose to generate the training pairs in an unsupervised manner. The client samples are selected within one database according to highest cosine scores with the anchor in i-vector space. The impostor samples are selected in the same way but from another database. Our double-branch siamese performs binary classification using cross entropy loss during training. In testing phase, we obtain speaker verification scores directly from its output layer. Whereas, our triple-branch siamese is trained to learn speaker embeddings using triplet loss. During testing, we extract speaker embeddings from its output layer, which are scored in the experiments using cosine scoring. The evaluation is performed on VoxCeleb-1 database, which show that using the proposed unsupervised systems, solely or in fusion, the results get closer to supervised baseline, This work has been developed in the framework of DeepVoice Project (TEC2015-69266-P), funded by Spanish Ministry, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Wav2Pix: speech-conditioned face generation using generative adversarial networks

  • Cardoso Duarte, Amanda
  • Roldan, Francisco
  • Tubau, Miquel
  • Escur, Janna
  • Pascual de la Puente, Santiago|||0000-0002-8365-7387
  • Salvador Aguilera, Amaia|||0000-0002-9908-1685
  • Mohedano, Eva
  • McGuinness, Kevin
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Giró Nieto, Xavier|||0000-0002-9935-5332
Speech is a rich biometric signal that contains information about the identity, gender and emotional state of the speaker. In this work, we explore its potential to generate face images of a speaker by conditioning a Generative Adversarial Network (GAN) with raw speech input. We propose a deep neural network that is trained from scratch in an end-to-end fashion, generating a face directly from the raw speech waveform without any additional identity information (e.g reference image or one-hot encoding). Our model is trained in a self-supervised approach by exploiting the audio and visual signals naturally aligned in videos. With the purpose of training from video data, we present a novel dataset collected for this work, with high-quality videos of youtubers with notable expressiveness in both the speech and visual signals., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Multi-speaker neural vocoder

  • Barbany Mayor, Oriol|||0000-0002-1379-9152
  • Bonafonte Cávez, Antonio|||0000-0002-6240-9915
  • Pascual de la Puente, Santiago|||0000-0002-8365-7387
Statistical Parametric Speech Synthesis (SPSS) offers more f lexibility than unit-selection based speech synthesis, which was the dominant commercial technology during the 2000s decade. However, classical SPSS systems generate speech with lower naturalness than unit-selection methods. Deep learning based SPSS, thanks to recurrent architectures, surpasses classical SPSS limits. These architectures offer high quality speech while preserving the desired flexibility in choosing the parameters such as the speaker, the intonation, etc. This paper exposes two proposals conceived to improve deep learning-based text-to-speech systems. First a baseline model, obtained by adapting SampleRNN, making it as a speaker-independent neural vocoder that generates the speech waveform from acoustic parameters. Then two approaches are proposed to improve the quality, applying speaker dependent normalization of the acoustic features, and the look ahead, consisting on feeding acoustic features of future frames to the network with the aim of better modeling the present waveform and avoiding possible discontinuities. Human listeners prefer the system that combines both techniques, which reaches a rate of 4 in the mean opinion score scale (MOS) with the balanced dataset and outperforms the other models., This research was supported by the project TEC2015-69266-P (MINECO/FEDER, UE)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

The TALP-UPC system for the WMT similar language task: statistical vs neural machine translation

  • Biesialska, Magdalena Marta|||0000-0001-7890-3523
  • Guàrdia Fernández, Lluís
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
Although the problem of similar language translation has been an area of research interest for many years, yet it is still far from being solved. In this paper, we study the performance of two popular approaches: statistical and neural. We conclude that both methods yield similar results; however, the performance varies depending on the language pair. While the statistical approach outperforms the neural one by a difference of 6 BLEU points for the Spanish-Portuguese language pair, the proposed neural model surpasses the statistical one by a difference of 2 BLEU points for Czech-Polish. In the former case, the language similarity (based on perplexity) is much higher than in the latter case. Additionally, we report negative results for the system combination with back-translation. Our TALP-UPC system submission won 1st place for Czech->Polish and 2nd place for Spanish->Portuguese in the official evaluation of the 1st WMT Similar Language Translation task., This work is supported in part by the Span-ish Ministerio de Economía y Competitividad,the European Regional Development Fund andthe Agencia Estatal de Investigación, through thepostdoctoral senior grant Ramón y Cajal, the con-tract TEC2015-69266-P (MINECO/FEDER,EU)and the contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Machine and deep learning approaches to localization and range estimation of underwater acoustic sources

  • Houégnigan, Ludwig|||0000-0003-2903-3140
  • Safari, Pooyan
  • Nadeu Camprubí, Climent|||0000-0002-5863-0983
  • André, Michel|||0000-0002-0091-7279
  • Van der Schaar, Mike Connor Roger Malcolm
This paper introduces ongoing experiments and early results for the underwater localization and range estimation of acoustic sources. Beyond classical results obtained for direction of arrival estimation, results concerning range estimation using supervised learning with neural networks having both shallow and deep architectures are presented. The developed method is applicable in the context of a single sensor, a compact array, or a small aperture towed array and provided results with great potential both for industrial impact and for the conservation and density estimation of cetaceans. With an average error of 4.3% and 3.5%-respectively for a shallow and for a deep pre-trained architecture-for ranges up to 8 kilometers and consistently below 300 meters, the system provides robust estimates suitable for an automated real-time solution., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Machine and deep learning approaches to localization and range estimation of underwater acoustic sources

  • Houégnigan, Ludwig
  • Safari, Pooyan
  • Nadeu Camprubí, Climent
  • André, Michel
  • Van der Schaar, Mike Connor Roger Malcolm
This paper introduces ongoing experiments and early results for the underwater localization and range estimation of acoustic sources. Beyond classical results obtained for direction of arrival estimation, results concerning range estimation using supervised learning with neural networks having both shallow and deep architectures are presented. The developed method is applicable in the context of a single sensor, a compact array, or a small aperture towed array and provided results with great potential both for industrial impact and for the conservation and density estimation of cetaceans. With an average error of 4.3% and 3.5%-respectively for a shallow and for a deep pre-trained architecture-for ranges up to 8 kilometers and consistently below 300 meters, the system provides robust estimates suitable for an automated real-time solution., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Prosodic and spectral iVectors for expressive speech synthesis

  • Jauk, Igor
  • Bonafonte Cávez, Antonio
This work presents a study on the suitability of prosodic andacoustic features, with a special focus on i-vectors, in expressivespeech analysis and synthesis. For each utterance of two dif-ferent databases, a laboratory recorded emotional acted speech,and an audiobook, several prosodic and acoustic features are ex-tracted. Among them, i-vectors are built not only on the MFCCbase, but also on F0, power and syllable durations. Then, un-supervised clustering is performed using different feature com-binations. The resulting clusters are evaluated calculating clus-ter entropy for labeled portions of the databases. Additionally,synthetic voices are trained, applying speaker adaptive training,from the clusters built from the audiobook. The voices are eval-uated in a perceptual test where the participants have to edit anaudiobook paragraph using the synthetic voices.The objective results suggest that i-vectors are very use-ful for the audiobook, where different speakers (book charac-ters) are imitated. On the other hand, for the laboratory record-ings, traditional prosodic features outperform i-vectors. Also,a closer analysis of the created clusters suggest that differentspeakers use different prosodic and acoustic means to conveyemotions. The perceptual results suggest that the proposed i-vector based feature combinations can be used for audiobookclustering and voice training., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A novel approach to real-time range estimation of underwater acoustic sources using supervised machine learning

  • Houégnigan, Ludwig
  • Safari, Pooyan
  • Nadeu Camprubí, Climent
  • Van der Schaar, Mike Connor Roger Malcolm
  • André, Michel
© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., The proposed paper introduces a novel method for range estimation of acoustic sources, both cetaceans and industrial sources, in deep sea environments using supervised learning with neural networks in the contex of a single sensor, a compact array, or a small aperture towed array. The presented results have potential both for industrial impact and for the conservation and density estimation of cetaceans. With an average error of 4.3% for ranges up to 8 kilometers and typically below 300 meters, those results are challenging and to our knowledge they are unprecedented for an automated real-time solution., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Problem-agnostic speech embeddings for multi-speaker text-to-speech with SampleRNN

  • Álvarez Robert, David
  • Pascual de la Puente, Santiago
  • Bonafonte Cávez, Antonio
Text-to-speech (TTS) acoustic models map linguistic features into an acoustic representation out of which an audible waveform is generated. The latest and most natural TTS systems build a direct mapping between linguistic and waveform domains, like SampleRNN. This way, possible signal naturalness losses are avoided as intermediate acoustic representations are discarded. Another important dimension of study apart from naturalness is their adaptability to generate voice from new speakers that were unseen during training. In this paper we first propose the use of problem-agnostic speech embeddings in a multi-speaker acoustic model for TTS based on SampleRNN. This way, we feed the acoustic model with speaker acousticallydependent representations that enrich the waveform generation more than embeddings unrelated to these factors. Our first results suggest that the proposed embeddings lead to better quality voices than those obtained with one-hot embeddings. Furthermore, as we can use any speech segment as an encoded representation during inference, the model is capable to generalize to new speaker identities without retraining the network. We finally show that, with a small increase of speech duration in the embedding extractor, we dramatically reduce the spectral distortion to close the gap towards the target identities., This research was supported by the project TEC2015-69266-P (MINECO/FEDER, UE)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Wav2Pix: speech-conditioned face generation using generative adversarial networks

  • Cardoso Duarte, Amanda
  • Roldan, Francisco
  • Tubau, Miquel
  • Escur, Janna
  • Pascual de la Puente, Santiago
  • Salvador Aguilera, Amaia
  • Mohedano, Eva
  • McGuinness, Kevin
  • Torres Viñals, Jordi
  • Giró Nieto, Xavier
Speech is a rich biometric signal that contains information about the identity, gender and emotional state of the speaker. In this work, we explore its potential to generate face images of a speaker by conditioning a Generative Adversarial Network (GAN) with raw speech input. We propose a deep neural network that is trained from scratch in an end-to-end fashion, generating a face directly from the raw speech waveform without any additional identity information (e.g reference image or one-hot encoding). Our model is trained in a self-supervised approach by exploiting the audio and visual signals naturally aligned in videos. With the purpose of training from video data, we present a novel dataset collected for this work, with high-quality videos of youtubers with notable expressiveness in both the speech and visual signals., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Multi-speaker neural vocoder

  • Barbany Mayor, Oriol
  • Bonafonte Cávez, Antonio
  • Pascual de la Puente, Santiago
Statistical Parametric Speech Synthesis (SPSS) offers more f lexibility than unit-selection based speech synthesis, which was the dominant commercial technology during the 2000s decade. However, classical SPSS systems generate speech with lower naturalness than unit-selection methods. Deep learning based SPSS, thanks to recurrent architectures, surpasses classical SPSS limits. These architectures offer high quality speech while preserving the desired flexibility in choosing the parameters such as the speaker, the intonation, etc. This paper exposes two proposals conceived to improve deep learning-based text-to-speech systems. First a baseline model, obtained by adapting SampleRNN, making it as a speaker-independent neural vocoder that generates the speech waveform from acoustic parameters. Then two approaches are proposed to improve the quality, applying speaker dependent normalization of the acoustic features, and the look ahead, consisting on feeding acoustic features of future frames to the network with the aim of better modeling the present waveform and avoiding possible discontinuities. Human listeners prefer the system that combines both techniques, which reaches a rate of 4 in the mean opinion score scale (MOS) with the balanced dataset and outperforms the other models., This research was supported by the project TEC2015-69266-P (MINECO/FEDER, UE)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The TALP-UPC machine translation systems for WMT19 news translation task: pivoting techniques for low resource MT

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián
  • Escolano Peinado, Carlos
  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
In this article, we describe the TALP-UPC research group participation in the WMT19 news translation shared task for Kazakh-English. Given the low amount of parallel training data, we resort to using Russian as pivot language, training subword-based statistical translation systems for Russian-Kazakh and Russian-English that were then used to create two synthetic pseudo-parallel corpora for Kazakh-English and English-Kazakh respectively. Finally, a self-attention model based on the decoder part of the Transformer architecture was trained on the two pseudo-parallel corpora., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Self-attention encoding and pooling for speaker recognition

  • Safari, Pooyan
  • India Massana, Miquel Àngel
  • Hernando Pericás, Francisco Javier
The computing power of mobile devices limits the end-user applications in terms of storage size, processing, memory and energy consumption. These limitations motivate researchers for the design of more efficient deep models. On the other hand, self-attention networks based on Transformer architecture have attracted remarkable interests due to their high parallelization capabilities and strong performance on a variety of Natural Language Processing (NLP) applications. Inspired by the Transformer, we propose a tandem Self-Attention Encoding and Pooling (SAEP) mechanism to obtain a discriminative speaker embedding given non-fixed length speech utterances. SAEP is a stack of identical blocks solely relied on self-attention and position-wise feed-forward networks to create vector representation of speakers. This approach encodes short-term speaker spectral features into speaker embeddings to be used in text-independent speaker verification. We have evaluated this approach on both VoxCeleb1 & 2 datasets. The proposed architecture is able to outperform the baseline x-vector, and shows competitive performance to some other benchmarks based on convolutions, with a significant reduction in model size. It employs 94%, 95%, and 73% less parameters compared to ResNet-34, ResNet-50, and x-vector, respectively. This indicates that the proposed fully attention based architecture is more efficient in extracting time-invariant features from speaker utterances., This work was supported in part by the Spanish Project DeepVoice (TEC2015-69266-P)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

UPC multimodal speaker diarization system for the 2018 Albayzin challenge

  • India Massana, Miquel Àngel
  • Sagastiberri, Itziar
  • Palau Puigdevall, Ponç
  • Sayrol Clols, Elisa
  • Morros Rubió, Josep Ramon
  • Hernando Pericás, Francisco Javier
This paper presents the UPC system proposed for the Multimodal Speaker Diarization task of the 2018 Albayzin Challenge. This approach works by processing individually the speech and the image signal. In the speech domain, speaker diarization is performed using identity embeddings created by a triplet loss DNN that uses i-vectors as input. The triplet DNN is trained with an additional regularization loss that minimizes the variance of both positive and negative distances. A sliding windows is then used to compare speech segments with enrollment speaker targets using cosine distance between the embeddings. To detect identities from the face modality, a face detector followed by a face tracker has been used on the videos. For each cropped face a feature vector is obtained using a Deep Neural Network based on the ResNet 34 architecture, trained using a metric learning triplet loss (available from dlib library). For each track the face feature vector is obtained by averaging the features obtained for each one of the frames of that track. Then, this feature vector is compared with the features extracted from the images of the enrollment identities. The proposed system is evaluated on the RTVE2018 database., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Restricted Boltzmann Machine vectors for speaker clustering

  • Khan, Umair
  • Safari, Pooyan
  • Hernando Pericás, Francisco Javier
Restricted Boltzmann Machines (RBMs) have been used both in the front-end and backend of speaker verification systems. In this work, we apply RBMs as a front-end in the context of speaker clustering. Speakers' utterances are transformed into a vector representation by means of RBMs. These vectors, referred to as RBM vectors, have shown to preserve speaker-specific information and are used for the task of speaker clustering. In this work, we perform the traditional bottom-up Agglomerative Hierarchical Clustering (AHC). Using the RBM vector representation of speakers, the performance of speaker clustering is improved. The evaluation has been performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed system outperforms the baseline i-vectors system in terms of Equal Impurity (EI). Using cosine scoring, a relative improvement of 11% and 12% are achieved for average and single linkage clustering algorithms respectively. Using PLDA scoring, the RBM vectors achieve a relative improvement of 11% compared to i-vectors for the single linkage algorithm., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

I-vector transformation using k-nearest neighbors for speaker verification

  • Khan, Umair
  • India Massana, Miquel Àngel
  • Hernando Pericás, Francisco Javier
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. https://dx.doi.org/10.1109/ICASSP40776.2020.9053504, Probabilistic Linear Discriminant Analysis (PLDA) is the most efficient backend for i-vectors. However, it requires labeled background data which can be difficult to access in practice. Unlike PLDA, cosine scoring avoids speaker-labels at the cost of degrading the performance. In this work, we propose a post processing of i-vectors using a Deep Neural Network (DNN) to transform i-vectors into a new speaker vector representation. The DNN will be trained using i-vectors that are similar to the training i-vectors. These similar i-vectors will be selected in an unsupervised manner. Using the new vector representation, we will score the experimental trials using cosine scoring. The evaluation was performed on the speaker verification trials of VoxCeleb-1 database. The experiments have shown that with the help of the similar i-vectors the new vectors become more discriminative than the original i-vectors. The new vectors have gained a relative improvement of 53% in terms of EER, compared to the conventional i-vector/PLDA system, but without using speaker labels., This work has been developed in the framework of DeepVoice Project (TEC2015-69266-P), funded by Spanish Ministry., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A differentiable BLEU loss. Analysis and first results

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián
  • Ruiz Costa-Jussà, Marta
In natural language generation tasks, like neural machine translation and image captioning, there is usually a mismatch between the optimized loss and the de facto evaluation criterion, namely token-level maximum likelihood and corpus-level BLEU score. This article tries to reduce this gap by defining differentiable computations of the BLEU and GLEU scores. We test this approach on simple tasks, obtaining valuable lessons on its potential applications but also its pitfalls, mainly that these loss functions push each token in the hypothesis sequence toward the average of the tokens in the reference, resulting in a poor training signal., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Byte-based neural machine translation

  • Ruiz Costa-Jussà, Marta
  • Escolano Peinado, Carlos
  • Rodríguez Fonollosa, José Adrián
This paper presents experiments compar- ing character-based and byte-based neural machine translation systems. The main motivation of the byte-based neural ma- chine translation system is to build multi- lingual neural machine translation systems that can share the same vocabulary. We compare the performance of both systems in several language pairs and we see that the performance in test is similar for most language pairs while the training time is slightly reduced in the case of byte-based neural machine translation., Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The TALP-UPC system for the WMT similar language task: statistical vs neural machine translation

  • Biesialska, Magdalena Marta
  • Guàrdia Fernández, Lluís
  • Ruiz Costa-Jussà, Marta
Although the problem of similar language translation has been an area of research interest for many years, yet it is still far from being solved. In this paper, we study the performance of two popular approaches: statistical and neural. We conclude that both methods yield similar results; however, the performance varies depending on the language pair. While the statistical approach outperforms the neural one by a difference of 6 BLEU points for the Spanish-Portuguese language pair, the proposed neural model surpasses the statistical one by a difference of 2 BLEU points for Czech-Polish. In the former case, the language similarity (based on perplexity) is much higher than in the latter case. Additionally, we report negative results for the system combination with back-translation. Our TALP-UPC system submission won 1st place for Czech->Polish and 2nd place for Spanish->Portuguese in the official evaluation of the 1st WMT Similar Language Translation task., This work is supported in part by the Span-ish Ministerio de Economía y Competitividad,the European Regional Development Fund andthe Agencia Estatal de Investigación, through thepostdoctoral senior grant Ramón y Cajal, the con-tract TEC2015-69266-P (MINECO/FEDER,EU)and the contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A geometric proxy of economic uncertainty based on the disagreement in survey expectations

  • Claveria González, Oscar
  • Monte Moreno, Enrique
  • Torra Porras, Salvador
In this study we present a geometric approach to proxy economic uncertainty. We design a positional indicator of disagreement among survey-based agents' expectations about the state of the economy. Previous dispersion-based uncertainty measures derived from business and consumer surveys exclusively make use of the two extreme pieces of information: the percentage of respondents expecting a variable to rise and to fall. With the aim of also incorporating the information coming from the share of respondents expecting a variable to remain constant, we propose a geometrical framework and use a barycentric coordinate system to generate a metric of disagreement, referred to as a discrepancy indicator. We assess its performance, both empirically and experimentally, by comparing it to the standard deviation of the share of positive and negative responses, which has been used by Bachman et al. (2013) as a proxy for economic uncertainty. When applied in sixteen European countries, we find that both time-varying metrics co-evolve in most countries for expectations about the country's overall economic situation in the present, but not in the future. Additionally, we obtain their simulated sampling distributions and we find that the proposed indicator gravitates uniformly towards the three vertices of the simplex representing the three answering categories, as opposed to the standard deviation, which tends to overestimate the level of uncertainty as a result of ignoring the no-change responses. Consequently, we find evidence that the information coming from agents expecting a variable to remain constant has an effect on the measurement of disagreement., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Multilingual, multi-scale and multi-layer visualization of sequence-based intermediate representations

  • Escolano Peinado, Carlos
  • Ruiz Costa-Jussà, Marta
  • Lacroux, Elora
  • Vázquez Alcocer, Pere Pau
The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation., This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

From bilingual to multilingual neural machine translation by incremental training

  • Escolano Peinado, Carlos
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
Multilingual Neural Machine Translation approaches are based on the use of task specific models and the addition of one more language can only be done by retraining the whole system. In this work, we propose a new training schedule that allows the system to scale to more languages without modification of the previous components based on joint training and language-independent encoder/decoder modules allowing for zero-shot translation. This work in progress shows close results to state-of-the-art in the WMT task., This work is supported in part by a Google Faculty Research Award. This work is also supported in part by the Spanish Ministerio de Economa y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacin, through the postdoctoral senior grant Ramon y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017- 079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Auto-encoding nearest neighbor i-vectors for speaker verification

  • Khan, Umair
  • India Massana, Miquel Àngel
  • Hernando Pericás, Francisco Javier
In the last years, i-vectors followed by cosine or PLDA scoringtechniques were the state-of-the-art approach in speaker veri-fication. PLDA requires labeled background data, and thereexists a significant performance gap between the two scoringtechniques. In this work, we propose to reduce this gap by us-ing an autoencoder to transform i-vector into a new speaker vec-tor representation, which will be referred to as ae-vector. Theautoencoder will be trained to reconstruct neighbor i-vectors in-stead of the same training i-vectors, as usual. These neighbori-vectors will be selected in an unsupervised manner accordingto the highest cosine scores to the training i-vectors. The evalua-tion is performed on the speaker verification trials of VoxCeleb-1 database. The experiments show that our proposed ae-vectorsgain a relative improvement of 42% in terms of EER comparedto the conventional i-vectors using cosine scoring, which fillsthe performance gap between cosine and PLDA scoring tech-niques by 92%, but without using speaker labels, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

CATOTRON – A Neural text-to-speech system in catalan

  • Külebi, Baybars
  • Öktem, Alp
  • Peiró Lilja, Àlex
  • Pascual, Santiago
  • Farrús, Mireia
Comunicació presentada a Interspeech 2020 celebrat del 25 al 29 d'octubre de 2020 a Shanghai, Xina., We present Catotron, a neural network-based open-source speech synthesis system in Catalan. Catotron consists of a sequence-to-sequence model trained with two small opensource datasets based on semi-spontaneous and read speech. We demonstrate how a neural TTS can be built for languages with limited resources using found-data optimization and crosslingual transfer learning. We make the datasets, initial models and source code publicly available for both commercial and research purposes., This work was subsidised by the Catalan Department of Culture. A part of the funding comes from the financing administered by the inheritance board of the Generalitat de Catalunya. The last author has been funded by the Agencia Estatal de Investigaci ´on (AEI), Ministerio de Ciencia, Innovaci´on y Universidades and the Fondo Social Europeo (FSE) under grant RYC- 2015-17239 (AEI/FSE, UE). Part of this work has been carried out using an NVIDIA GPU Titan Xp generously provided by NVIDIA Company. This research was also partially supported by the project TEC2015-69266-P (MINECO/FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Multi-output RNN-LSTM for multiple speaker speech synthesis with a-interpolation model

  • Pascual, Santiago
  • Bonafonte Cávez, Antonio
Deep Learning has been applied successfully to speech processing. In this paper we propose an architecture for speech synthesis using multiple speakers. Some hidden layers are shared by all the speakers, while there is a specific output layer for each speaker. Objective and perceptual experiments prove that this scheme produces much better results in comparison with sin- gle speaker model. Moreover, we also tackle the problem of speaker interpolation by adding a new output layer (a-layer) on top of the multi-output branches. An identifying code is injected into the layer together with acoustic features of many speakers. Experiments show that the a-layer can effectively learn to interpolate the acoustic features between speakers., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Terminology-aware segmentation and domain feature for the WMT19 biomedical translation task

  • Carrino, Casimiro Pio
  • Rafieian, Bardia
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
In this work, we give a description of the TALP-UPC systems submitted for the WMT19 Biomedical Translation Task. Our proposed strategy is NMT model-independent and relies only on one ingredient, a biomedical terminology list. We first extracted such a terminology list by labelling biomedical words in our training dataset using the BabelNet API. Then, we designed a data preparation strategy to insert the terms information at a token level. Finally, we trained the Transformer model with this terms-informed data. Our best-submitted system ranked 2nd and 3rd for Spanish-English and English-Spanish translation directions, respectively., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Evaluating the underlying gender bias in contextualized word embeddings

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
  • Casas Manzanares, Noé
Gender bias is highly impacting natural language processing applications. Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources. Recently, contextualized word embeddings have enhanced previous word embedding techniques by computing word vector representations dependent on the sentence they appear in. In this paper, we study the impact of this conceptual change in the word embedding computation in relation with gender bias. Our analysis includes different measures previously applied in the literature to standard word embeddings. Our findings suggest that contextualized word embeddings are less biased than standard ones even when the latter are debiased., We want to thank Hila Gonen for her support dur-ing our research.This work is supported in part by the Catalan Agency for Management of University andResearch Grants (AGAUR) through the FI PhDScholarship and the Industrial PhD Grant. Thiswork is also supported in part by the Span-ish Ministerio de Economa y Competitividad, the European Regional Development Fund andthe Agencia Estatal de Investigacin, through thepostdoctoral senior grant Ramn y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU)and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Self multi-head attention for speaker recognition

  • India Massana, Miquel Àngel
  • Safari, Pooyan
  • Hernando Pericás, Francisco Javier
Most state-of-the-art Deep Learning (DL) approaches forspeaker recognition work on a short utterance level. Given thespeech signal, these algorithms extract a sequence of speakerembeddings from short segments and those are averaged to ob-tain an utterance level speaker representation. In this work wepropose the use of an attention mechanism to obtain a discrim-inative speaker embedding given non fixed length speech utter-ances. Our system is based on a Convolutional Neural Network(CNN) that encodes short-term speaker features from the spec-trogram and a self multi-head attention model that maps theserepresentations into a long-term speaker embedding. The atten-tion model that we propose produces multiple alignments fromdifferent subsegments of the CNN encoded states over the se-quence. Hence this mechanism works as a pooling layer whichdecides the most discriminative features over the sequence toobtain an utterance level representation. We have tested thisapproach for the verification task for the VoxCeleb1 dataset.The results show that self multi-head attention outperforms bothtemporal and statistical pooling methods with a18%of rela-tive EER. Obtained results show a58%relative improvementin EER compared to i-vector+PLDA, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards large scale multimedia indexing: a case study on person discovery in broadcast news

  • Le, Nam
  • Bredin, Hervé
  • Sargent, Gabriel
  • India, Miquel
  • López Otero, Paula
  • Barras, Claude
  • Guinaudeau, Camille
  • Gravier, Guillaume
  • Barbosa da Fonseca, Gabriel
  • Lyon Freire, Izabela
  • Patrocínio, Zenilton
  • Guimarães, Silvio Jamil F.
  • Martí Juan, Gerard
  • Morros, Josep Ramon
  • Hernando, Javier
  • Docio Fernández, Laura
  • García Mateo, Carmen
  • Meignier, Sylvain
  • Odobez, Jean-Marc
Comunicació presentada a: the 15th International Workshop on Content-Based Multimedia Indexing (CBMI'17), celebrat a Florència, Itàlia, del 19 al 21 de juny de 2017, The rapid growth of multimedia databases and the human interest in their peers make indices representing the location and identity of people in audio-visual documents essential for searching archives. Person discovery in the absence of prior identity knowledge requires accurate association of audio-visual cues and detected names. To this end, we present 3 different strategies to approach this problem: clustering-based naming, verification-based naming, and graph-based naming. Each of these strategies utilizes different recent advances in unsupervised face / speech representation, verification, and optimization. To have a better understanding of the approaches, this paper also provides a quantitative and qualitative comparative study of these approaches using the associated corpus of the Person Discovery challenge at MediaEval 2016. From the results of our experiments, we can observe the pros and cons of each approach, thus paving the way for future promising research directions., This work was supported by the EU project EUMSSI (FP7-611057), ANR project MetaDaTV (ANR-14-CE24-0024) project, Camomile project (PCIN-2013-067), and the projects TEC2013-43935-R, TEC2015-69266-P, TEC2016-75976-R, TEC2015-65345-P financed by the Spanish government and ERDF.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

DNN speaker embeddings using autoencoder pre-training

  • Khan, Umair
  • Hernando Pericás, Francisco Javier
Over the last years, i-vectors have been the state-of-the-art approach in speaker recognition. Recent improvements in deep learning have increased the discriminative quality of i-vectors. However, deep learning architectures require a large amount of labeled background data which is difficult in practice. The aim of this paper is to propose an alternative scheme in order to reduce the need of labeled data. We propose the use of autoencoder pre-training in a speaker verification task. First, we train an autoencoder in an unsupervised way, using a large amount of unlabeled background data. Then, we train a Deep Neural Network (DNN) initialized with the parameters of the pre-trained autoencoder. The DNN training is carried out in a supervised way using relatively small labeled background data. In the testing phase, we extract speaker embeddings as the output of an intermediate layer of the DNN. The training and evaluation were performed on VoxCeleb-2 and VoxCeleb1 databases, respectively. The experimental results have shown that by initializing DNN with the parameters of the pre-trained autoencoder, we have achieved a relative improvement of 21%, in terms of Equal Error Rate (EER), over the baseline i-vector/PLDA system., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Unsupervised training of siamese networks for speaker verification

  • Khan, Umair
  • Hernando Pericás, Francisco Javier
Speaker labeled background data is an essential requirement for most state-of-the-art approaches in speaker recognition, e.g., xvectors and i-vector/PLDA. However, in reality it is difficult to access large amount of labeled data. In this work, we propose siamese networks for speaker verification without using speaker labels. We propose two different siamese networks having two and three branches, respectively, where each branch is a CNN encoder. Since the goal is to avoid speaker labels, we propose to generate the training pairs in an unsupervised manner. The client samples are selected within one database according to highest cosine scores with the anchor in i-vector space. The impostor samples are selected in the same way but from another database. Our double-branch siamese performs binary classification using cross entropy loss during training. In testing phase, we obtain speaker verification scores directly from its output layer. Whereas, our triple-branch siamese is trained to learn speaker embeddings using triplet loss. During testing, we extract speaker embeddings from its output layer, which are scored in the experiments using cosine scoring. The evaluation is performed on VoxCeleb-1 database, which show that using the proposed unsupervised systems, solely or in fusion, the results get closer to supervised baseline, This work has been developed in the framework of DeepVoice Project (TEC2015-69266-P), funded by Spanish Ministry, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Spanish statistical parametric speech synthesis using a neural vocoder

  • Bonafonte Cávez, Antonio
  • Pascual de la Puente, Santiago
  • Dorca, G.
During the 2000s decade, unit-selection based text-to-speech was the dominant commercial technology. Meanwhile, the TTS research community has made a big effort to push statistical-parametric speech synthesis to get similar quality and more flexibility on the synthetically generated voice. During last years, deep learning advances applied to speech synthesis have filled the gap, specially when neural vocoders substitute traditional signal-processing based vocoders. In this paper we propose to substitute the waveform generation vocoder of MUSA, our Spanish TTS, with SampleRNN, a neural vocoder which was recently proposed as a deep autoregressive raw waveform generation model. MUSA uses recurrent neural networks to predict vocoder parameters (MFCC and logF0) from linguistic features. Then, the Ahocoder vocoder is used to recover the speech waveform out of the predicted parameters. In the first system SampleRNN is extended to generate speech conditioned on the Ahocoder generated parameters (mfcc and logF0), where two configurations have been considered to train the system. First, the parameters derived from the signal using Ahocoder are used. Secondly, the system is trained with the parameters predicted by MUSA, where SampleRNN and MUSA are jointly optimized. The subjective evaluation shows that the second system outperforms both the original Ahocoder and SampleRNN as an independent neural vocoder., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Automatic speech recognition with deep neural networks for impaired speech

  • España-i-Bonet, Cristina
  • Rodríguez Fonollosa, José Adrián
The final publication is available at https://link.springer.com/chapter/10.1007%2F978-3-319-49169-1_10, Automatic Speech Recognition has reached almost human performance in some controlled scenarios. However, recognition of impaired speech is a difficult task for two main reasons: data is (i) scarce and (ii) heterogeneous. In this work we train different architectures on a database of dysarthric speech. A comparison between architectures shows that, even with a small database, hybrid DNN-HMM models outperform classical GMM-HMM according to word error rate measures. A DNN is able to improve the recognition word error rate a 13% for subjects with dysarthria with respect to the best classical architecture. This improvement is higher than the one given by other deep neural networks such as CNNs, TDNNs and LSTMs. All the experiments have been done with the Kaldi toolkit for speech recognition for which we have adapted several recipes to deal with dysarthric speech and work on the TORGO database. These recipes are publicly available., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards mitigating gender bias in a decoder-based neural machine translation model by adding contextual information

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
Gender bias negatively impacts many natural language processing applications, including ma-chine translation (MT). The motivation behind this work is to study whether recent proposedMT techniques are significantly contributing to attenuate biases in document-level and gender-balanced data. For the study, we consider approaches of adding the previous sentence and thespeaker information, implemented in a decoder-based neural MT system. We show improve-ments both in translation quality (+1 BLEU point) as well as in gender bias mitigation onWinoMT (+5% accuracy)., This work is supported in part by the Catalan Agency for Management of University and Research Grants (AGAUR) through the FI PhD Scholarship. This work is also supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacion, through the postdoctoral senior grant Ramón y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Refinement of unsupervised cross-lingual word embeddings

  • Biesialska, Magdalena Marta
  • Ruiz Costa-Jussà, Marta
Cross-lingual word embeddings aim to bridge the gap between high-resource and low-resource languages by allowing to learn multilingual word representations even without using any direct bilingual signal. The lion's share of the methods are projection-based approaches that map pre-trained embeddings into a shared latent space. These methods are mostly based on the orthogonal transformation, which assumes language vector spaces to be isomorphic. However, this criterion does not necessarily hold, especially for morphologically-rich languages. In this paper, we propose a self-supervised method to refine the alignment of unsupervised bilingual word embeddings. The proposed model moves vectors of words and their corresponding translations closer to each other as well as enforces length- and center-invariance, thus allowing to better align cross-lingual embeddings. The experimental results demonstrate the effectiveness of our approach, as in most cases it outperforms state-of-the-art methods in a bilingual lexicon induction task., We thank anonymous reviewers for their helpful comments. This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, the contract TEC2015-69266-P (MINECO/FEDER,EU) and the contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)