RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication
Capítulo o parte de un libro (BookPart)
A Comparison of Approaches for Measuring Cross-Lingual Similarity of Wikipedia Articles
Wikipedia has been used as a source of comparable texts
for a range of tasks, such as Statistical Machine Translation and CrossLanguage
Information Retrieval. Articles written in different languages
on the same topic are often connected through inter-language-links. However,
the extent to which these articles are similar is highly variable and
this may impact on the use of Wikipedia as a comparable resource. In this
paper we compare various language-independent methods for measuring
cross-lingual similarity: character n-grams, cognateness, word count ratio,
and an approach based on outlinks. These approaches are compared
against a baseline utilising MT resources. Measures are also compared
to human judgements of similarity using a manually created resource
containing 700 pairs of Wikipedia articles (in 7 language pairs). Results
indicate that a combination of language-independent models (char-ngrams,
outlinks and word-count ratio) is highly effective for identifying
cross-lingual similarity and performs comparably to language-dependent
models (translation and monolingual analysis)., The work of the first author was in the framework of the Tacardi research project (TIN2012-38523-C02-00). The work of the fourth author was in the framework of the DIANA-Applications (TIN2012-38603-C02-01) and WIQ-EI IRSES (FP7 Marie Curie No. 269180) research projects.