Skip to main content

TECNOLOGIAS NEURONALES LIGERAS PARA LA TRADUCCION DE IDIOMAS CON POCOS RECURSOS

PID2021-127999NB-I00

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2021
State Plan 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSIDAD DE ALICANTE

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Non-Fluent Synthetic Target-Language Data Improve Neural Machine Translation

  • Sánchez-Cartagena, Víctor M.
  • Esplà-Gomis, Miquel
  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Esplà-Gomis, Miquel|||/items/6614b776-cf96-43c9-b300-de4c8cf17c43
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
When the amount of parallel sentences available to train a neural machine translation is scarce, a common practice is to generate new synthetic training samples from them. A number of approaches have been proposed to produce synthetic parallel sentences that are similar to those in the parallel data available. These approaches work under the assumption that non-fluent target-side synthetic training samples can be harmful and may deteriorate translation performance. Even so, in this paper we demonstrate that synthetic training samples with non-fluent target sentences can improve translation performance if they are used in a multilingual machine translation framework as if they were sentences in another language. We conducted experiments on ten low-resource and four high-resource translation tasks and found out that this simple approach consistently improves translation performance as compared to state-of-the-art methods for generating synthetic training samples similar to those found in corpora. Furthermore, this improvement is independent of the size of the original training corpus, the resulting systems are much more robust against domain shift and produce less hallucinations.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Data-centric strategies for machine translation: efficient acquisition, cleaning, and exploitation of parallel resources

  • García Romero, Cristian
  • García Romero, Cristian|||/items/4fad4894-8a55-40f8-852a-d21c46a9f92a
High-quality parallel corpora are an essential resource for crosslingual natural language processing. Particularly in machine translation (MT), dedicated systems heavily rely on massive amounts of this data for training, while large language models (LLMs) require less parallel data to be adapted for translation through fine-tuning and in-context learning, where quality is prioritized over quantity. However, acquiring high-quality parallel corpora from the web presents significant efficiency and quality challenges. This thesis proposes a comprehensive framework for the acquisition, cleaning, and exploitation of parallel data. First, we tackle the inefficiency of traditional web crawling with a novel smart bilingual focused crawler. By utilizing URL-based models to infer language and document parallelness, our approach reduces wasted bandwidth and accelerates the discovery of parallel content. Second, to ensure data quality, we address the growing proliferation of synthetic, machine-translated text on the web, which can cause model degradation. We introduce an automatic machine translation detection method that leverages the internal representations of a surrogate multilingual MT model to accurately filter out synthetic data, outperforming state-of-the-art techniques, with particularly strong results for non-English language pairs. Finally, we explore the optimal exploitation of high-quality parallel data for MT using LLMs. We propose an in-context learning approach driven by reinforcement learning to dynamically select the most effective parallel examples, maximizing the final translation quality.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Smart bilingual focused crawling of parallel documents

  • García Romero, Cristian|||/items/4fad4894-8a55-40f8-852a-d21c46a9f92a
  • Esplà-Gomis, Miquel
  • Sánchez-Martínez, Felipe
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
  • Esplà-Gomis, Miquel|||/items/6614b776-cf96-43c9-b300-de4c8cf17c43
Crawling parallel texts—texts that are mutual translations—from the Internet is usually done following a brute-force approach: documents are massively downloaded in an unguided process, and only a fraction of them end up leading to actual parallel content. In this work, we propose a smart crawling method that guides the crawl towards finding parallel content more rapidly. We follow a neural approach that consists in adapting a pre-trained multilingual language model based on the encoder of the Transformer architecture by fine-tuning it for two new tasks: inferring the language of a document from its Uniform Resource Locator (URL) and inferring whether a pair of URLs link to parallel documents. We evaluate both models in isolation and their integration into a crawling tool. The results demonstrate the individual effectiveness of both models and highlight that their combination enables us to address a practical engineering challenge: the early discovery of parallel content during web crawling in a given language pair. This leads to a reduction in the amount of downloaded documents deemed useless and yields a greater quantity of parallel documents compared to conventional crawling approaches.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Curated Datasets and Neural Models for Machine Translation of Informal Registers between Mayan and Spanish Vernaculars

  • Lou, Andrés
  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Sánchez-Cartagena, Víctor M.
  • Lou, Andrés|||/items/813d3f73-92fd-42ae-a626-ba9868e5239d
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
The Mayan languages comprise a language family with an ancient history, millions of speakers, and immense cultural value, that, nevertheless, remains severely underrepresented in terms of resources and global exposure. In this paper we develop, curate, and publicly release a set of corpora in several Mayan languages spoken in Guatemala and Southern Mexico, which we call MayanV. The datasets are parallel with Spanish, the dominant language of the region, and are taken from official native sources focused on representing informal, day-to-day, and non-domain-specific language. As such, and according to our dialectometric analysis, they differ in register from most other available resources. Additionally, we present neural machine translation models, trained on as many resources and Mayan languages as possible, and evaluated exclusively on our datasets. We observe lexical divergences between the dialects of Spanish in our resources and the more widespread written standard of Spanish, and that resources other than the ones we present do not seem to improve translation performance, indicating that many such resources may not accurately capture common, real-life language usage. The MayanV dataset is available at https://github.com/transducens/mayanv.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Expanding the FLORES+ Multilingual Benchmark with Translations for Aragonese, Aranese, Asturian, and Valencian

  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Sánchez-Cartagena, Víctor M.
  • Esplà-Gomis, Miquel
  • Galiano Jiménez, Aarón
  • Oliver, Antoni
  • Aventín-Boya, Claudi
  • Pardos, Alejandro
  • Valdés, Cristina
  • Sans Socasau, Jusèp Loís
  • Martínez, Juan Pablo
  • Esplà-Gomis, Miquel|||/items/6614b776-cf96-43c9-b300-de4c8cf17c43
  • Galiano Jiménez, Aarón|||/items/ff07f91f-bf9b-4ef1-b2f1-8457faff7914
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
In this paper, we describe the process of creating the FLORES+ datasets for several Romance languages spoken in Spain, namely Aragonese, Aranese, Asturian, and Valencian. The Aragonese and Aranese datasets are entirely new additions to the FLORES+ multilingual benchmark. An initial version of the Asturian dataset was already available in FLORES+, and our work focused on a thorough revision. Similarly, FLORES+ included a Catalan dataset, which we adapted to the Valencian variety spoken in the Valencian Community. The development of the Aragonese, Aranese, and revised Asturian FLORES+ datasets was undertaken as part of a WMT24 shared task on translation into low-resource languages of Spain.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model

  • García Romero, Cristian
  • Esplà-Gomis, Miquel
  • Sánchez-Martínez, Felipe
  • García Romero, Cristian|||/items/4fad4894-8a55-40f8-852a-d21c46a9f92a
  • Esplà-Gomis, Miquel|||/items/6614b776-cf96-43c9-b300-de4c8cf17c43
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
Modern machine translation (MT) systems depend on large parallel corpora, often collected from the Internet. However, recent evidence indicates that (i) a substantial portion of these texts are machine-generated translations, and (ii) an overreliance on such synthetic content in training data can significantly degrade translation quality. As a result, filtering out non-human translations is becoming an essential pre-processing step in building high-quality MT systems. In this work, we propose a novel approach that directly exploits the internal representations of a surrogate multilingual MT model to distinguish between human and machine-translated sentences. Experimental results show that our method outperforms current state-of-the-art techniques, particularly for non-English language pairs, achieving gains of at least 5 percentage points of accuracy.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Universitat d’Alacant’s Submission to the WMT 2024 Shared Task on Translation into Low-Resource Languages of Spain

  • Galiano Jiménez, Aarón
  • Sánchez-Cartagena, Víctor M.
  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Galiano Jiménez, Aarón|||/items/ff07f91f-bf9b-4ef1-b2f1-8457faff7914
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
This paper describes the submissions of the Transducens group of the Universitat d’Alacant to the WMT 2024 Shared Task on Translation into Low-Resource Languages of Spain; in particular, the task focuses on the translation from Spanish into Aragonese, Aranese and Asturian. Our submissions use parallel and monolingual data to fine-tune the NLLB-1.3B model and to investigate the effectiveness of synthetic corpora and transfer-learning between related languages such as Catalan, Galician and Valencian. We also present a many-to-many multilingual neural machine translation model focused on the Romance languages of Spain.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Beyond the Mode: Sequence-Level Distillation of Multilingual Translation Models for Low-Resource Language Pairs

  • Galiano Jiménez, Aarón
  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Sánchez-Cartagena, Víctor M.
  • Galiano Jiménez, Aarón|||/items/ff07f91f-bf9b-4ef1-b2f1-8457faff7914
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
This paper delves into sequence-level knowledge distillation (KD) of multilingual pre-trained translation models. We posit that, beyond the approximated mode obtained via beam search, the whole output distribution of the teacher contains valuable insights for students. We explore the potential of n-best lists from beam search to guide student’s learning and then investigate alternative decoding methods to address observed issues like low variability and under-representation of infrequent tokens. Our research in data-limited scenarios reveals that although sampling methods can slightly compromise the translation quality of the teacher output compared to beam search based methods, they enrich the generated corpora with increased variability and lexical richness, ultimately enhancing student model performance and reducing the gender bias amplification commonly associated with KD.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Exploiting large pre-trained models for low-resource neural machine translation

  • Galiano Jiménez, Aarón
  • Sánchez-Martínez, Felipe
  • Sánchez-Cartagena, Víctor M.
  • Pérez-Ortiz, Juan Antonio
  • Galiano Jiménez, Aarón|||/items/ff07f91f-bf9b-4ef1-b2f1-8457faff7914
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
Pre-trained models have revolutionized the natural language processing field by leveraging large-scale language representations for various tasks. Some pre-trained models offer general-purpose representations, while others are specialized in particular tasks, like neural machine translation (NMT). Multilingual NMT-targeted systems are often fine-tuned for specific language pairs, but there is a lack of evidence-based best-practice recommendations to guide this process. Additionally, deploying these large pre-trained models in computationally restricted environments, typically found in developing regions where low-resource languages are spoken, has become challenging. We propose a pipeline to tune the mBART50 pre-trained model to 8 diverse low-resource language pairs, and then distill the resulting system to obtain lightweight and more sustainable NMT models. Our pipeline conveniently exploits back-translation, synthetic corpus filtering, and knowledge distillation to deliver efficient bilingual translation models that are 13 times smaller, while maintaining a close BLEU performance.
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Cross-lingual neural fuzzy matching for exploiting target-language monolingual corpora in computer-aided translation

  • Esplà-Gomis, Miquel
  • Sánchez-Cartagena, Víctor M.
  • Pérez-Ortiz, Juan Antonio
  • Sánchez-Martínez, Felipe
  • Esplà-Gomis, Miquel|||/items/6614b776-cf96-43c9-b300-de4c8cf17c43
  • Pérez-Ortiz, Juan Antonio|||/items/a13b860a-9e21-4163-a290-337d45dce50e
  • Sánchez-Cartagena, Víctor M.|||/items/782c1de3-587b-40f4-87f0-d214fe789104
  • Sánchez-Martínez, Felipe|||/items/117378d0-8f7d-40ef-9b4a-5646b1af1999
Computer-aided translation (CAT) tools based on translation memories (MT) play a prominent role in the translation workflow of professional translators. However, the reduced availability of in-domain TMs, as compared to in-domain monolingual corpora, limits its adoption for a number of translation tasks. In this paper, we introduce a novel neural approach aimed at overcoming this limitation by exploiting not only TMs, but also in-domain target-language (TL) monolingual corpora, and still enabling a similar functionality to that offered by conventional TM-based CAT tools. Our approach relies on cross-lingual sentence embeddings to retrieve translation proposals from TL monolingual corpora, and on a neural model to estimate their post-editing effort. The paper presents an automatic evaluation of these techniques on four language pairs that shows that our approach can successfully exploit monolingual texts in a TM-based CAT environment, increasing the amount of useful translation proposals, and that our neural model for estimating the post-editing effort enables the combination of translation proposals obtained from monolingual corpora and from TMs in the usual way. A human evaluation performed on a single language pair confirms the results of the automatic evaluation and seems to indicate that the translation proposals retrieved with our approach are more useful than what the automatic evaluation shows.