Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Enhancing word embeddings with knowledge extracted from lexical resources

  • Biesialska, Magdalena Marta
  • Rafieian, Bardia
  • Ruiz Costa-Jussà, Marta
In this work, we present an effective method for semantic specialization of word vector representations. To this end, we use traditional word embeddings and apply specialization methods to better capture semantic relations between words. In our approach, we leverage external knowledge from rich lexical resources such as BabelNet. We also show that our proposed post-specialization method based on an adversarial neural network with the Wasserstein distance allows to gain improvements over state-of-the-art methods on two tasks: word similarity and dialog state tracking., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819 and PCIN-2017-079., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

AMALEU: una representación universal del lenguaje basada en aprendizaje automático, AMALEU: a machine-learned universal language representation

  • Ruiz Costa-Jussà, Marta
El objetivo del proyecto AMALEU es aprender una representación común para diferentes idiomas. Se pretende tener una representación común para la lengua oral y una para la lengua escrita. AMALEU, de dos años de duración, está financiado por el MINECO dentro del programa de Europa Excelencia., The objective of AMALEU’s project is learning a multilingual common representation for speech and a different multilingual common representation for text. AMALEU is a two-year project funded by the MINECO., Este trabajo está financiado por el Ministerio de Economía y Competitividad a través del projecto EUR2019-103819 y el programa Ramón y Cajal que incluye financiamiento del European Regional Development Fund., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Findings of the first shared task on lifelong learning machine translation

  • Barrault, Loïc
  • Biesialska, Magdalena Marta
  • Ruiz Costa-Jussà, Marta
  • Bougares, Fethi
  • Galibert, Olivier
A lifelong learning system can adapt to new data without forgetting previously acquired knowledge. In this paper, we introduce the first benchmark for lifelong learning machine translation. For this purpose, we provide training, lifelong and test data sets for two language pairs: English-German and English-French. Additionally, we report the results of our baseline systems, which we make available to the public. The goal of this shared task is to encourage research on the emerging topic of lifelong learning machine translation., This work is is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Multilingual neural machine translation: case-study for Catalan, Spanish and Portuguese romance languages

  • Vergés Boncompte, Pere
  • Ruiz Costa-Jussà, Marta
In this paper, we describe the TALP-UPC participation in the WMT Similar Language Translation task between Catalan, Spanish, and Portuguese, all of them, Romance languages. We made use of different techniques to improve the translation between these languages. The multilingual shared encoder/decoder has been used for all of them. Additionally, we applied back-translation to take advantage of the monolingual data. Finally, we have applied fine-tuning to improve the in-domain data. Each of these techniques brings improvements over the previous one. In the official evaluation, our system was ranked 1st in the Portuguese-to-Spanish direction, 2nd in the opposite direction, and 3rd in the Catalan-Spanish pair., This work is supported in part by the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Combining subword representations into word-level representations in the transformer architecture

  • Casas Manzanares, Noé
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
In Neural Machine Translation, using word-level tokens leads to degradation in translation quality. The dominant approaches use subword-level tokens, but this increases the length of the sequences and makes it difficult to profit from word-level information such as POS tags or semantic dependencies. We propose a modification to the Transformer model to combine subword-level representations into word-level ones in the first layers of the encoder, reducing the effective length of the sequences in the following layers and providing a natural point to incorporate extra word-level information. Our experiments show that this approach maintains the translation quality with respect to the normal Transformer model when no extra word-level information is injected and that it is superior to the currently dominant method for incorporating word-level source language information to models based on subword-level vocabularies., This work is partially supported by Lucy Software / United Language Group (ULG) and the Catalan Agency for Management of University and Research Grants (AGAUR) through an Industrial PhD Grant. This work is also supported in part by the the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramón y Cajal and by the Agencia Estatal de Investigación through the project EUR2019-103819., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Extensive study on the underlying gender bias in contextualized word embeddings

  • Basta, Christine Raouf Saad
  • Ruiz Costa-Jussà, Marta
  • Casas Manzanares, Noé
Gender bias is affecting many natural language processing applications. While we are still far from proposing debiasing methods that will solve the problem, we are making progress analyzing the impact of this bias in current algorithms. This paper provides an extensive study of the underlying gender bias in popular contextualized word embeddings. Our study provides an insightful analysis of evaluation measures applied to several English data domains and the layers of the contextualized word embeddings. It is also adapted and extended to the Spanish language. Our study points out the advantages and limitations of the various evaluation measures that we are using and aims to standardize the evaluation of gender bias in contextualized word embeddings., This work is supported in part by the Catalan Agency for Management of University and Research Grants (AGAUR) through the FI PhD Scholarship and the Industrial PhD Grant. This work also is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund, the Agencia Estatal de Investigación through the postdoctoral senior grant Ramón y Cajal and the Projects EUR2019-103819, PCIN-2017-079 and PID2019-107579RB-I00., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Syntax-driven iterative expansion language models for controllable text generation

  • Casas Manzanares, Noé
  • Rodríguez Fonollosa, José Adrián
  • Ruiz Costa-Jussà, Marta
The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a new paradigm for introducing a syntactic inductive bias into neural text generation, where the dependency parse tree is used to drive the Transformer model to generate sentences iteratively. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity, requiring less than half their decoding steps, and its generation process allows direct control over the syntactic constructions of the generated text, enabling the induction of stylistic variations., This work is partially supported by Lucy Software / United Language Group (ULG) and the Catalan Agency for Management of University and Research Grants (AGAUR) through an Industrial Ph.D. Grant. This work also is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramón y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

GeBioToolkit: automatic extraction of gender-balanced multilingual corpus of Wikipedia biographies

  • Ruiz Costa-Jussà, Marta
  • Li Lin, Pau
  • España Bonet, Cristina
We introduce GeBioToolkit, a tool for extracting multilingual parallel corpora at sentence level, with document and gender information from Wikipedia biographies. Despite the gender inequalities present in Wikipedia, the toolkit has been designed to extract corpus balanced in gender. While our toolkit is customizable to any number of languages (and different domains), in this work we present a corpus of 2,000 sentences in English, Spanish and Catalan, which has been post-edited by native speakers to become a high-quality dataset for machine translation evaluation. While GeBioCorpus aims at being one of the first non-synthetic gender-balanced test datasets, GeBioToolkit aims at paving the path to standardize procedures to produce gender-balanced datasets., This work is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819 and PCIN-2017- 079. It is also partially funded by the German Federal Ministry of Education and Research under the funding code 01IW17001 (Deeplee)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The TALP-UPC system description for WMT20 news translation task: multilingual adaptation for low resource MT

  • Escolano Peinado, Carlos
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
In this article, we describe the TALP-UPC participation in the WMT20 news translation shared task for Tamil-English. Given the low amount of parallel training data, we resort to adapt the task to a multilingual system to benefit from the positive transfer from high resource languages. We use iterative backtranslation to fine-tune the system and benefit from the monolingual data available. In order to measure the effectivity of such methods, we compare our results to a bilingual baseline system., This work is supported in part by the Google Faculty Research Award 2019, the Spanish Ministerio de Ciencia e Innovacion, through the postdoctoral senior grant Ramon y Cajal and by the Agencia Estatal de Investigacion through the projects EUR2019-103819, PCIN-2017-079 and PID2019- 107579RB-I00 / AEI / 10.13039/501100011033, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

From bilingual to multilingual neural-based machine translation by incremental training

  • Escolano Peinado, Carlos
  • Ruiz Costa-Jussà, Marta
  • Rodríguez Fonollosa, José Adrián
A common intermediate language representation in neural machine translation can be used to extend bilingual systems by incremental training. We propose a new architecture based on introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we can train multiple encoders and decoders for each language, sharing among them a common intermediate representation. Translation results on the low-resource tasks (Turkish-English and Kazakh-English tasks) show a BLEU improvement of up to 2.8 points. However, results on a larger dataset (Russian-English and Kazakh-English) show BLEU losses of a similar amount. While our system provides improvements only for the low-resource tasks in terms of translation quality, our system is capable of quickly deploying new language pairs without the need to retrain the rest of the system, which may be a game changer in some situations. Specifically, what is most relevant regarding our architecture is that it is capable of: reducing the number of production systems, with respect to the number of languages, from quadratic to linear; incrementally adding a new language to the system without retraining the languages already there; and allowing for translations from the new language to all the others present in the system., This work also is supported in part by the Google Faculty Research Awards and, also by, the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund, the Agencia Estatal de Investigación through the postdoctoral senior grant Ramón y Cajal and the projects EUR2019-103819, PCIN-2017-079 andPID2019-107579RB-I00., Peer Reviewed, Postprint (published version)