Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Reference bias in monolingual machine translation evaluation

  • Fomicheva, Marina
  • Specia, Lucia
Comunicació presentada a la 54th Annual Meeting of the Association for Computational Linguistics, celebrada del 7 al 10 d'agost de 2016 a Berlín, Alemanya., In the translation industry, human translations are assessed by comparison with the source texts. In the Machine Translation (MT) research community, however, it is a common practice to perform quality assessment using a reference translation instead of the source text. In this paper we show that this practice has a serious issue – annotators are strongly biased by the reference translation provided, and this can have a negative impact on the assessment of MT quality., Marina Fomicheva was supported by funding from IULA (UPF) and the FI-DGR grant program of the Generalitat de Catalunya. Lucia Specia was supported by the QT21 project (H2020 No. 645452).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

CobaltF: a fluent metric for MT evaluation

  • Fomicheva, Marina
  • Bel Rafecas, Núria
  • Specia, Lucia
  • da Cunha Fanego, Iria
  • Malinovskiy, Anton
Comunicació presentada a la First Conference on Machine Translation (WMT), que es va dur a terme durant el 54th Annual Meeting of the Association for Computational Linguistics, els dies 7 a 12 d'agost de 2016 a Berlín, Alemanya., The vast majority of Machine Translation (MT) evaluation approaches are based on the idea that the closer the MT output is to a human reference translation, the higher its quality. While translation quality has two important aspects, adequacy and fluency, the existing referencebased metrics are largely focused on the former. In this work we combine our metric UPF-Cobalt, originally presented at the WMT15 Metrics Task, with a number of features intended to capture translation fluency. Experiments show that the integration of fluency-oriented features significantly improves the results, rivalling the best-performing evaluation metrics on the WMT15 data., This work was partially funded by TUNER (TIN2015-65308-C5-5-R) and MINECO/FEDER, UE. Marina Fomicheva was supported by funding from the FI-DGR grant program of the Generalitat de Catalunya. Iria da Cunha was supported by a Ramon y Cajal contract (RYC-2014-16935). Lucia Specia was supported by the QT21 project (H2020 No. 645452).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Reference bias in monolingual machine translation evaluation

  • Fomicheva, Marina
  • Specia, Lucia
Comunicació presentada a la 54th Annual Meeting of the Association for Computational Linguistics, celebrada del 7 al 10 d'agost de 2016 a Berlín, Alemanya., In the translation industry, human translations are assessed by comparison with the source texts. In the Machine Translation (MT) research community, however, it is a common practice to perform quality assessment using a reference translation instead of the source text. In this paper we show that this practice has a serious issue – annotators are strongly biased by the reference translation provided, and this can have a negative impact on the assessment of MT quality., Marina Fomicheva was supported by funding from IULA (UPF) and the FI-DGR grant program of the Generalitat de Catalunya. Lucia Specia was supported by the QT21 project (H2020 No. 645452).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

CobaltF: a fluent metric for MT evaluation

  • Fomicheva, Marina
  • Bel Rafecas, Núria
  • Specia, Lucia
  • da Cunha Fanego, Iria
  • Malinovskiy, Anton
Comunicació presentada a la First Conference on Machine Translation (WMT), que es va dur a terme durant el 54th Annual Meeting of the Association for Computational Linguistics, els dies 7 a 12 d'agost de 2016 a Berlín, Alemanya., The vast majority of Machine Translation (MT) evaluation approaches are based on the idea that the closer the MT output is to a human reference translation, the higher its quality. While translation quality has two important aspects, adequacy and fluency, the existing referencebased metrics are largely focused on the former. In this work we combine our metric UPF-Cobalt, originally presented at the WMT15 Metrics Task, with a number of features intended to capture translation fluency. Experiments show that the integration of fluency-oriented features significantly improves the results, rivalling the best-performing evaluation metrics on the WMT15 data., This work was partially funded by TUNER (TIN2015-65308-C5-5-R) and MINECO/FEDER, UE. Marina Fomicheva was supported by funding from the FI-DGR grant program of the Generalitat de Catalunya. Iria da Cunha was supported by a Ramon y Cajal contract (RYC-2014-16935). Lucia Specia was supported by the QT21 project (H2020 No. 645452).
Addi. Archivo Digital para la Docencia y la Investigación
Publicación Contribución de congreso (ConferenceOutput)

Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation

  • Cer, Daniel
  • Diab, Mona
  • Agirre Bengoa, Eneko
  • López Gazpio, Iñigo
  • Specia, Lucia
Semantic Textual Similarity (STS) measures the meaning similarity of sentences. Applications include machine translation (MT), summarization, generation, question answering (QA), short answer grading, semantic search, dialog and conversational systems. The STS shared task is a venue for assessing the current state-of-the-art. The 2017 task focuses on multilingual and cross-lingual pairs with one sub-track exploring MT quality estimation (MTQE) data. The task obtained strong participation from 31 teams, with 17 participating in all language tracks. We summarize performance and review a selection of well performing methods. Analysis highlights common errors, providing insight into the limitations of existing models. To support ongoing work on semantic representations, the STS Benchmark is introduced as a new shared training and evaluation set carefully selected from the corpus of English STS shared task data (2012-2017)., This material is based in part upon work supported by QNRF-NPRP 6 - 1020-1-199 OPTDIAC that funded Arabic translation, and by a grant from the Spanish MINECO (projects TUNER TIN2015-65308-C5-1-R and MUSTER PCIN-2015-226 cofunded by EU FEDER) that funded STS label annotation and by the QT21 EU project (H2020 No. 645452) that funded STS labels and data preparation for machine translation pairs. I˜nigo Lopez-Gazpio is supported by the Spanish MECD. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of QNRF-NPRP, Spanish MINECO, QT21 EU, or the Spanish MECD.