Pasar al contenido principal

AUTONOMOUS LIFELONG LEARNING INTELLIGENT SYSTEMS

PCIN-2017-079

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos Programación Conjunta Internacional
Año convocatoria 2017
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 22
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Syntax-driven iterative expansion language models for controllable text generation

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a new paradigm for introducing a syntactic inductive bias into neural text generation, where the dependency parse tree is used to drive the Transformer model to generate sentences iteratively. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity, requiring less than half their decoding steps, and its generation process allows direct control over the syntactic constructions of the generated text, enabling the induction of stylistic variations., This work is partially supported by Lucy Software / United Language Group (ULG) and the Catalan Agency for Management of University and Research Grants (AGAUR) through an Industrial Ph.D. Grant. This work also is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramón y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Automatic Spanish translation of SQuAD dataset for multi-lingual question answering

  • Carrino, Casimiro Pio
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
Recently, multilingual question answering became a crucial research topic, and it is receiving increased interest in the NLP community.However, the unavailability of large-scale datasets makes it challenging to train multilingual QA systems with performance comparableto the English ones. In this work, we develop the Translate Align Retrieve (TAR) method to automatically translate the Stanford QuestionAnswering Dataset (SQuAD) v1.1 to Spanish. We then used this dataset to train Spanish QA systems by fine-tuning a Multilingual-BERTmodel. Finally, we evaluated our QA models with the recently proposed MLQA and XQuAD benchmarks for cross-lingual ExtractiveQA. Experimental results show that our models outperform the previous Multilingual-BERT baselines achieving the new state-of-the-artvalues of 68.1 F1 on the Spanish MLQA corpus and 77.6 F1 on the Spanish XQuAD corpus. The resulting, synthetically generatedSQuAD-es v1.1 corpora, with almost 100% of data contained in the original English version, to the best of our knowledge, is the firstlarge-scale QA training resource for Spanish., This work is supported in part by the Spanish Ministe-rio de Econom ́ıa y Competitividad, the European RegionalDevelopment Fund and the Agencia Estatal de Investi-gaci ́on, through the postdoctoral senior grant Ram ́on y Ca-jal (FEDER/MINECO) amd the project PCIN-2017-079(AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

GeBioToolkit: automatic extraction of gender-balanced multilingual corpus of Wikipedia biographies

  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Li Lin, Pau
  • España Bonet, Cristina
We introduce GeBioToolkit, a tool for extracting multilingual parallel corpora at sentence level, with document and gender information from Wikipedia biographies. Despite the gender inequalities present in Wikipedia, the toolkit has been designed to extract corpus balanced in gender. While our toolkit is customizable to any number of languages (and different domains), in this work we present a corpus of 2,000 sentences in English, Spanish and Catalan, which has been post-edited by native speakers to become a high-quality dataset for machine translation evaluation. While GeBioCorpus aims at being one of the first non-synthetic gender-balanced test datasets, GeBioToolkit aims at paving the path to standardize procedures to produce gender-balanced datasets., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819 and PCIN-2017- 079. It is also partially funded by the German Federal Ministry of Education and Research under the funding code 01IW17001 (Deeplee)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the underlying gender bias in contextualized word embeddings

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Casas Manzanares, Noé
Gender bias is highly impacting natural language processing applications. Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources. Recently, contextualized word embeddings have enhanced previous word embedding techniques by computing word vector representations dependent on the sentence they appear in. In this paper, we study the impact of this conceptual change in the word embedding computation in relation with gender bias. Our analysis includes different measures previously applied in the literature to standard word embeddings. Our findings suggest that contextualized word embeddings are less biased than standard ones even when the latter are debiased., We want to thank Hila Gonen for her support dur-ing our research.This work is supported in part by the Catalan Agency for Management of University andResearch Grants (AGAUR) through the FI PhDScholarship and the Industrial PhD Grant. Thiswork is also supported in part by the Span-ish Ministerio de Economa y Competitividad, the European Regional Development Fund andthe Agencia Estatal de Investigacin, through thepostdoctoral senior grant Ramn y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU)and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Continual lifelong learning in natural language processing: a survey

  • Biesialska, Magdalena Marta|||0000-0001-7890-3523
  • Biesialska, Katarzyna|||0000-0002-2865-7990
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. However, it is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. Furthermore, CL is particularly challenging for language learning, as natural language is ambiguous: it is discrete, compositional, and its meaning is context-dependent. In this work, we look at the problem of CL through the lens of various NLP tasks. Our survey discusses major challenges in CL and current methods applied in neural network models. We also provide a critical review of the existing CL evaluation methods and datasets in NLP. Finally, we present our outlook on future research directions., This work is supported in part by the Catalan Agencia de Gestión de Ayudas Universitarias y de Investigación (AGAUR) through the FI PhD grant; the Spanish Ministerio de Ciencia e Innovación and by the Agencia Estatal de Investigación through the Ramón y Cajal grant and the project PCIN-2017-079; and by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 947657)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

A differentiable BLEU loss. Analysis and first results

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
In natural language generation tasks, like neural machine translation and image captioning, there is usually a mismatch between the optimized loss and the de facto evaluation criterion, namely token-level maximum likelihood and corpus-level BLEU score. This article tries to reduce this gap by defining differentiable computations of the BLEU and GLEU scores. We test this approach on simple tasks, obtaining valuable lessons on its potential applications but also its pitfalls, mainly that these loss functions push each token in the hypothesis sequence toward the average of the tokens in the reference, resulting in a poor training signal., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Towards mitigating gender bias in a decoder-based neural machine translation model by adding contextual information

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
Gender bias negatively impacts many natural language processing applications, including ma-chine translation (MT). The motivation behind this work is to study whether recent proposedMT techniques are significantly contributing to attenuate biases in document-level and gender-balanced data. For the study, we consider approaches of adding the previous sentence and thespeaker information, implemented in a decoder-based neural MT system. We show improve-ments both in translation quality (+1 BLEU point) as well as in gender bias mitigation onWinoMT (+5% accuracy)., This work is supported in part by the Catalan Agency for Management of University and Research Grants (AGAUR) through the FI PhD Scholarship. This work is also supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacion, through the postdoctoral senior grant Ramón y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Enhancing word embeddings with knowledge extracted from lexical resources

  • Biesialska, Magdalena Marta|||0000-0001-7890-3523
  • Rafieian, Bardia
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
In this work, we present an effective method for semantic specialization of word vector representations. To this end, we use traditional word embeddings and apply specialization methods to better capture semantic relations between words. In our approach, we leverage external knowledge from rich lexical resources such as BabelNet. We also show that our proposed post-specialization method based on an adversarial neural network with the Wasserstein distance allows to gain improvements over state-of-the-art methods on two tasks: word similarity and dialog state tracking., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819 and PCIN-2017-079., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Multilingual neural machine translation: case-study for Catalan, Spanish and Portuguese romance languages

  • Vergés Boncompte, Pere
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
In this paper, we describe the TALP-UPC participation in the WMT Similar Language Translation task between Catalan, Spanish, and Portuguese, all of them, Romance languages. We made use of different techniques to improve the translation between these languages. The multilingual shared encoder/decoder has been used for all of them. Additionally, we applied back-translation to take advantage of the monolingual data. Finally, we have applied fine-tuning to improve the in-domain data. Each of these techniques brings improvements over the previous one. In the official evaluation, our system was ranked 1st in the Portuguese-to-Spanish direction, 2nd in the opposite direction, and 3rd in the Catalan-Spanish pair., This work is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

The TALP-UPC system description for WMT20 news translation task: multilingual adaptation for low resource MT

  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Rodríguez Fonollosa, José Adrián|||0000-0001-9513-7939
In this article, we describe the TALP-UPC participation in the WMT20 news translation shared task for Tamil-English. Given the low amount of parallel training data, we resort to adapt the task to a multilingual system to benefit from the positive transfer from high resource languages. We use iterative backtranslation to fine-tune the system and benefit from the monolingual data available. In order to measure the effectivity of such methods, we compare our results to a bilingual baseline system., This work is supported in part by the Google Faculty Research Award 2019, the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019- 107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Findings of the first shared task on lifelong learning machine translation

  • Barrault, Loïc
  • Biesialska, Magdalena Marta|||0000-0001-7890-3523
  • Ruiz Costa-Jussà, Marta|||0000-0002-5703-520X
  • Bougares, Fethi
  • Galibert, Olivier
A lifelong learning system can adapt to new data without forgetting previously acquired knowledge. In this paper, we introduce the first benchmark for lifelong learning machine translation. For this purpose, we provide training, lifelong and test data sets for two language pairs: English-German and English-French. Additionally, we report the results of our baseline systems, which we make available to the public. The goal of this shared task is to encourage research on the emerging topic of lifelong learning machine translation., This work is is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Enhancing word embeddings with knowledge extracted from lexical resources

  • Biesialska, Magdalena Marta
  • Rafieian, Bardia
  • Ruiz Costa-Jussà, Marta
In this work, we present an effective method for semantic specialization of word vector representations. To this end, we use traditional word embeddings and apply specialization methods to better capture semantic relations between words. In our approach, we leverage external knowledge from rich lexical resources such as BabelNet. We also show that our proposed post-specialization method based on an adversarial neural network with the Wasserstein distance allows to gain improvements over state-of-the-art methods on two tasks: word similarity and dialog state tracking., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819 and PCIN-2017-079., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A differentiable BLEU loss. Analysis and first results

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián
  • Ruiz Costa-Jussà, Marta
In natural language generation tasks, like neural machine translation and image captioning, there is usually a mismatch between the optimized loss and the de facto evaluation criterion, namely token-level maximum likelihood and corpus-level BLEU score. This article tries to reduce this gap by defining differentiable computations of the BLEU and GLEU scores. We test this approach on simple tasks, obtaining valuable lessons on its potential applications but also its pitfalls, mainly that these loss functions push each token in the hypothesis sequence toward the average of the tokens in the reference, resulting in a poor training signal., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Findings of the first shared task on lifelong learning machine translation

  • Barrault, Loïc
  • Biesialska, Magdalena Marta
  • Ruiz Costa-Jussà, Marta
  • Bougares, Fethi
  • Galibert, Olivier
A lifelong learning system can adapt to new data without forgetting previously acquired knowledge. In this paper, we introduce the first benchmark for lifelong learning machine translation. For this purpose, we provide training, lifelong and test data sets for two language pairs: English-German and English-French. Additionally, we report the results of our baseline systems, which we make available to the public. The goal of this shared task is to encourage research on the emerging topic of lifelong learning machine translation., This work is is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Multilingual neural machine translation: case-study for Catalan, Spanish and Portuguese romance languages

  • Vergés Boncompte, Pere
  • Ruiz Costa-Jussà, Marta
In this paper, we describe the TALP-UPC participation in the WMT Similar Language Translation task between Catalan, Spanish, and Portuguese, all of them, Romance languages. We made use of different techniques to improve the translation between these languages. The multilingual shared encoder/decoder has been used for all of them. Additionally, we applied back-translation to take advantage of the monolingual data. Finally, we have applied fine-tuning to improve the in-domain data. Each of these techniques brings improvements over the previous one. In the official evaluation, our system was ranked 1st in the Portuguese-to-Spanish direction, 2nd in the opposite direction, and 3rd in the Catalan-Spanish pair., This work is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Syntax-driven iterative expansion language models for controllable text generation

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián
  • Ruiz Costa-Jussà, Marta
The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a new paradigm for introducing a syntactic inductive bias into neural text generation, where the dependency parse tree is used to drive the Transformer model to generate sentences iteratively. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity, requiring less than half their decoding steps, and its generation process allows direct control over the syntactic constructions of the generated text, enabling the induction of stylistic variations., This work is partially supported by Lucy Software / United Language Group (ULG) and the Catalan Agency for Management of University and Research Grants (AGAUR) through an Industrial Ph.D. Grant. This work also is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramón y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic Spanish translation of SQuAD dataset for multi-lingual question answering

  • Carrino, Casimiro Pio
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
Recently, multilingual question answering became a crucial research topic, and it is receiving increased interest in the NLP community.However, the unavailability of large-scale datasets makes it challenging to train multilingual QA systems with performance comparableto the English ones. In this work, we develop the Translate Align Retrieve (TAR) method to automatically translate the Stanford QuestionAnswering Dataset (SQuAD) v1.1 to Spanish. We then used this dataset to train Spanish QA systems by fine-tuning a Multilingual-BERTmodel. Finally, we evaluated our QA models with the recently proposed MLQA and XQuAD benchmarks for cross-lingual ExtractiveQA. Experimental results show that our models outperform the previous Multilingual-BERT baselines achieving the new state-of-the-artvalues of 68.1 F1 on the Spanish MLQA corpus and 77.6 F1 on the Spanish XQuAD corpus. The resulting, synthetically generatedSQuAD-es v1.1 corpora, with almost 100% of data contained in the original English version, to the best of our knowledge, is the firstlarge-scale QA training resource for Spanish., This work is supported in part by the Spanish Ministe-rio de Econom ́ıa y Competitividad, the European RegionalDevelopment Fund and the Agencia Estatal de Investi-gaci ́on, through the postdoctoral senior grant Ram ́on y Ca-jal (FEDER/MINECO) amd the project PCIN-2017-079(AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

GeBioToolkit: automatic extraction of gender-balanced multilingual corpus of Wikipedia biographies

  • Ruiz Costa-Jussà, Marta
  • Li Lin, Pau
  • España Bonet, Cristina
We introduce GeBioToolkit, a tool for extracting multilingual parallel corpora at sentence level, with document and gender information from Wikipedia biographies. Despite the gender inequalities present in Wikipedia, the toolkit has been designed to extract corpus balanced in gender. While our toolkit is customizable to any number of languages (and different domains), in this work we present a corpus of 2,000 sentences in English, Spanish and Catalan, which has been post-edited by native speakers to become a high-quality dataset for machine translation evaluation. While GeBioCorpus aims at being one of the first non-synthetic gender-balanced test datasets, GeBioToolkit aims at paving the path to standardize procedures to produce gender-balanced datasets., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819 and PCIN-2017- 079. It is also partially funded by the German Federal Ministry of Education and Research under the funding code 01IW17001 (Deeplee)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Evaluating the underlying gender bias in contextualized word embeddings

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
  • Casas Manzanares, Noé
Gender bias is highly impacting natural language processing applications. Word embeddings have clearly been proven both to keep and amplify gender biases that are present in current data sources. Recently, contextualized word embeddings have enhanced previous word embedding techniques by computing word vector representations dependent on the sentence they appear in. In this paper, we study the impact of this conceptual change in the word embedding computation in relation with gender bias. Our analysis includes different measures previously applied in the literature to standard word embeddings. Our findings suggest that contextualized word embeddings are less biased than standard ones even when the latter are debiased., We want to thank Hila Gonen for her support dur-ing our research.This work is supported in part by the Catalan Agency for Management of University andResearch Grants (AGAUR) through the FI PhDScholarship and the Industrial PhD Grant. Thiswork is also supported in part by the Span-ish Ministerio de Economa y Competitividad, the European Regional Development Fund andthe Agencia Estatal de Investigacin, through thepostdoctoral senior grant Ramn y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU)and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The TALP-UPC system description for WMT20 news translation task: multilingual adaptation for low resource MT

  • Escolano Peinado, Carlos
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
In this article, we describe the TALP-UPC participation in the WMT20 news translation shared task for Tamil-English. Given the low amount of parallel training data, we resort to adapt the task to a multilingual system to benefit from the positive transfer from high resource languages. We use iterative backtranslation to fine-tune the system and benefit from the monolingual data available. In order to measure the effectivity of such methods, we compare our results to a bilingual baseline system., This work is supported in part by the Google Faculty Research Award 2019, the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019- 107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Continual lifelong learning in natural language processing: a survey

  • Biesialska, Magdalena Marta
  • Biesialska, Katarzyna
  • Ruiz Costa-Jussà, Marta
Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. However, it is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. Furthermore, CL is particularly challenging for language learning, as natural language is ambiguous: it is discrete, compositional, and its meaning is context-dependent. In this work, we look at the problem of CL through the lens of various NLP tasks. Our survey discusses major challenges in CL and current methods applied in neural network models. We also provide a critical review of the existing CL evaluation methods and datasets in NLP. Finally, we present our outlook on future research directions., This work is supported in part by the Catalan Agencia de Gestión de Ayudas Universitarias y de Investigación (AGAUR) through the FI PhD grant; the Spanish Ministerio de Ciencia e Innovación and by the Agencia Estatal de Investigación through the Ramón y Cajal grant and the project PCIN-2017-079; and by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 947657)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Towards mitigating gender bias in a decoder-based neural machine translation model by adding contextual information

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
Gender bias negatively impacts many natural language processing applications, including ma-chine translation (MT). The motivation behind this work is to study whether recent proposedMT techniques are significantly contributing to attenuate biases in document-level and gender-balanced data. For the study, we consider approaches of adding the previous sentence and thespeaker information, implemented in a decoder-based neural MT system. We show improve-ments both in translation quality (+1 BLEU point) as well as in gender bias mitigation onWinoMT (+5% accuracy)., This work is supported in part by the Catalan Agency for Management of University and Research Grants (AGAUR) through the FI PhD Scholarship. This work is also supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigacion, through the postdoctoral senior grant Ramón y Cajal, contract TEC2015-69266-P (MINECO/FEDER,EU) and contract PCIN-2017-079 (AEI/MINECO)., Peer Reviewed, Postprint (published version)