Skip to main content

MULTILINGUAL TRANSFER LEARNING FOR THE INCLUSION OF VULNERABLE SOCIAL GROUPS

PID2021-123988OB-C33

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2021
State Plan 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre BARCELONA SUPERCOMPUTING CENTER CENTRO NACIONAL DE SUPERCOMPUTACION

Resultados relacionados

Resultados totales (Incluyendo duplicados): 8
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

MT-LENS: An all-in-one toolkit for better machine translation evaluation

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Mash, Audrey
  • Liao, Xixian
  • Melero Nogues, Maite
We introduce MT-Lens, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxicity, and robustness to misspellings. While several toolkits have become very popular for benchmarking the capabilities of Large Language Models (LLMs), existing evaluation tools often lack the ability to thoroughly assess the diverse aspects of MT performance. MT-Lens addresses these limitations by extending the capabilities of LM-eval-harness for MT, supporting state-of-the-art datasets and a wide range of evaluation metrics. It also offers a user-friendly platform to compare systems and analyze translations with interactive visualizations. MT-Lens aims to broaden access to evaluation strategies that go beyond traditional translation quality evaluation, enabling researchers and engineers to better understand the performance of a NMT model and also easily measure system’s biases., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by DeepR3 (TED2021-130295B-C32) funded MCIN/AEI/10.13039/501100011033 European Union NextGeneration EU/PRTR. This work was supported by EU Horizon 2020 project ELOQUENCE13 (grant number 101070558)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Terminology-constrained translation from monolingual data using GRPO

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Liao, Xixian
  • Melero Nogues, Maite
Terminology consistency is essential for high-quality machine translation, especially in domain-specific and professional contexts, where accurate term translation directly impacts usability. This paper presents the submission from the BSC team to the WMT25 Terminology-Aware Translation Task. We propose the use of GRPO (Group Relative Policy Optimization) to adapt translation models using monolingual data only, without requiring parallel corpora. Our reward function jointly optimizes for terminology adherence and overall translation quality, leveraging quality-estimation metrics. Experimental results demonstrate that our method consistently improves terminology translation across three language directions—English to Spanish, German, and Russian—by up to +0.36 Tₚ points across all evaluated models., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by MLLM4TRA (PID2024-158157OB-C32) funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública- Funded by EU– NextGenerationEU within the framework of the ILENIA Project with reference 2022/TL22/00215337. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Investigating the translation capabilities of Large Language Models trained on parallel data only

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Sant Savall, Aleix
  • De Luca Fornaciari, Francesca
  • Mash, Audrey
  • Liao, Xixian
  • Melero Nogues, Maite
In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Translation. However, previous methods predominantly relied on iterative processes such as instruction fine-tuning or continual pre-training, leaving unexplored the challenges of training LLMs solely on parallel data. In this work, we introduce Plume (Parallel Language Model), a collection of three 2B LLMs featuring varying vocabulary sizes (32k, 128k, and 256k) trained exclusively on Catalan-centric parallel examples. These models perform comparably to previous encoder-decoder architectures on 16 supervised translation directions and 56 zero-shot ones. Utilizing this set of models, we conduct a thorough investigation into the translation capabilities of LLMs, probing their performance, the role of vocabulary size, the impact of the different elements of the prompt, and their cross-lingual representation space. We find that larger vocabulary sizes improve zero-shot performance and that different layers specialize in distinct aspects of the prompt, such as language-specific tags. We further show that as the vocabulary size grows, a larger number of attention heads can be pruned with minimal loss in translation quality, achieving a reduction of over 64.7% in attention heads., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by DeepR3 (TED2021-130295B-C32) funded by MCIN/AEI/10.13039/501100011033 and European Union NextGeneration EU/PRTR. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

From SALAMANDRA to SALAMANDRATA: BSC submission for WMT25 general machine translation shared task

  • García Gilabert, Javier
  • Liao, Xixian
  • Da Dalt, Severino
  • Bohman, Ella
  • Mash, Audrey
  • De Luca Fornaciari, Francesca
  • Baucells de la Peña, Irene
  • Llop Palao, Joan
  • Claramunt Argote, Miguel
  • Escolano Peinado, Carlos|||0000-0001-6657-673X
  • Melero Nogues, Maite
In this paper, we present the SalamandraTA family of models, an improved iteration of Salamandra LLMs (Gonzalez-Agirre et al., 2025) specifically trained to achieve strong performance in translation-related tasks for 38 European languages. SalamandraTA comes in two scales: 2B and 7B parameters. For both versions, we applied the same training recipe with a first step of continual pre-training on parallel data, and a second step of supervised fine-tuning on high-quality instructions.The BSC submission to the WMT25 General Machine Translation shared task is based on the 7B variant of SalamandraTA. We first extended the model vocabulary to support the additional non-European languages included in the task. This was followed by a second phase of continual pretraining and supervised fine-tuning, carefully designed to optimize performance across all translation directions for this year’s shared task. For decoding, we employed two quality-aware strategies: Minimum Bayes Risk Decoding and Translation Reranking using Comet and Comet-kiwi.We publicly release both the 2B and 7B versions of SalamandraTA, along with the newer SalamandraTA-v2 model, on Hugging Face., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by MLLM4TRA (PID2024-158157OB-C32) funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública- Funded by EU– NextGenerationEU within the framework of ILENIA Project with reference 2022/TL22/00215337. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Investigating the translation capabilities of Large Language Models trained on parallel data only

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos
  • Sant Savall, Aleix
  • De Luca Fornaciari, Francesca
  • Mash, Audrey
  • Liao, Xixian
  • Melero Nogues, Maite
In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Translation. However, previous methods predominantly relied on iterative processes such as instruction fine-tuning or continual pre-training, leaving unexplored the challenges of training LLMs solely on parallel data. In this work, we introduce Plume (Parallel Language Model), a collection of three 2B LLMs featuring varying vocabulary sizes (32k, 128k, and 256k) trained exclusively on Catalan-centric parallel examples. These models perform comparably to previous encoder-decoder architectures on 16 supervised translation directions and 56 zero-shot ones. Utilizing this set of models, we conduct a thorough investigation into the translation capabilities of LLMs, probing their performance, the role of vocabulary size, the impact of the different elements of the prompt, and their cross-lingual representation space. We find that larger vocabulary sizes improve zero-shot performance and that different layers specialize in distinct aspects of the prompt, such as language-specific tags. We further show that as the vocabulary size grows, a larger number of attention heads can be pruned with minimal loss in translation quality, achieving a reduction of over 64.7% in attention heads., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by DeepR3 (TED2021-130295B-C32) funded by MCIN/AEI/10.13039/501100011033 and European Union NextGeneration EU/PRTR. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

From SALAMANDRA to SALAMANDRATA: BSC submission for WMT25 general machine translation shared task

  • García Gilabert, Javier
  • Liao, Xixian
  • Da Dalt, Severino
  • Bohman, Ella
  • Mash, Audrey
  • De Luca Fornaciari, Francesca
  • Baucells de la Peña, Irene
  • Llop Palao, Joan
  • Claramunt Argote, Miguel
  • Escolano Peinado, Carlos
  • Melero Nogues, Maite
In this paper, we present the SalamandraTA family of models, an improved iteration of Salamandra LLMs (Gonzalez-Agirre et al., 2025) specifically trained to achieve strong performance in translation-related tasks for 38 European languages. SalamandraTA comes in two scales: 2B and 7B parameters. For both versions, we applied the same training recipe with a first step of continual pre-training on parallel data, and a second step of supervised fine-tuning on high-quality instructions.The BSC submission to the WMT25 General Machine Translation shared task is based on the 7B variant of SalamandraTA. We first extended the model vocabulary to support the additional non-European languages included in the task. This was followed by a second phase of continual pretraining and supervised fine-tuning, carefully designed to optimize performance across all translation directions for this year’s shared task. For decoding, we employed two quality-aware strategies: Minimum Bayes Risk Decoding and Translation Reranking using Comet and Comet-kiwi.We publicly release both the 2B and 7B versions of SalamandraTA, along with the newer SalamandraTA-v2 model, on Hugging Face., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by MLLM4TRA (PID2024-158157OB-C32) funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública- Funded by EU– NextGenerationEU within the framework of ILENIA Project with reference 2022/TL22/00215337. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Terminology-constrained translation from monolingual data using GRPO

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos
  • Liao, Xixian
  • Melero Nogues, Maite
Terminology consistency is essential for high-quality machine translation, especially in domain-specific and professional contexts, where accurate term translation directly impacts usability. This paper presents the submission from the BSC team to the WMT25 Terminology-Aware Translation Task. We propose the use of GRPO (Group Relative Policy Optimization) to adapt translation models using monolingual data only, without requiring parallel corpora. Our reward function jointly optimizes for terminology adherence and overall translation quality, leveraging quality-estimation metrics. Experimental results demonstrate that our method consistently improves terminology translation across three language directions—English to Spanish, German, and Russian—by up to +0.36 Tₚ points across all evaluated models., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by MLLM4TRA (PID2024-158157OB-C32) funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública- Funded by EU– NextGenerationEU within the framework of the ILENIA Project with reference 2022/TL22/00215337. This work is funded by the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia- Funded by EU– NextGenerationEU within the framework of the project Desarrollo Modelos ALIA., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

MT-LENS: An all-in-one toolkit for better machine translation evaluation

  • Garcia Gilabert, Javier
  • Escolano Peinado, Carlos
  • Mash, Audrey
  • Liao, Xixian
  • Melero Nogues, Maite
We introduce MT-Lens, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxicity, and robustness to misspellings. While several toolkits have become very popular for benchmarking the capabilities of Large Language Models (LLMs), existing evaluation tools often lack the ability to thoroughly assess the diverse aspects of MT performance. MT-Lens addresses these limitations by extending the capabilities of LM-eval-harness for MT, supporting state-of-the-art datasets and a wide range of evaluation metrics. It also offers a user-friendly platform to compare systems and analyze translations with interactive visualizations. MT-Lens aims to broaden access to evaluation strategies that go beyond traditional translation quality evaluation, enabling researchers and engineers to better understand the performance of a NMT model and also easily measure system’s biases., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by DeepR3 (TED2021-130295B-C32) funded MCIN/AEI/10.13039/501100011033 European Union NextGeneration EU/PRTR. This work was supported by EU Horizon 2020 project ELOQUENCE13 (grant number 101070558)., Peer Reviewed, Postprint (published version)