UPCommons. Portal del coneixement obert de la UPC
Publication
Contribución de congreso (ConferenceOutput)
MT-LENS: An all-in-one toolkit for better machine translation evaluation
We introduce MT-Lens, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxicity, and robustness to misspellings. While several toolkits have become very popular for benchmarking the capabilities of Large Language Models (LLMs), existing evaluation tools often lack the ability to thoroughly assess the diverse aspects of MT performance. MT-Lens addresses these limitations by extending the capabilities of LM-eval-harness for MT, supporting state-of-the-art datasets and a wide range of evaluation metrics. It also offers a user-friendly platform to compare systems and analyze translations with interactive visualizations. MT-Lens aims to broaden access to evaluation strategies that go beyond traditional translation quality evaluation, enabling researchers and engineers to better understand the performance of a NMT model and also easily measure system’s biases., This work has been promoted and financed by the Generalitat de Catalunya through the Aina Project. This work has been supported by the Spanish project PID2021-123988OB-C33 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work is partially supported by DeepR3 (TED2021-130295B-C32) funded MCIN/AEI/10.13039/501100011033 European Union NextGeneration EU/PRTR. This work was supported by EU Horizon 2020 project ELOQUENCE13 (grant number 101070558)., Peer Reviewed