Skip to main content

TRADUCCION DE TEXTOS Y TRANSCRIPCION DE VOZ INTERACTIVAS

TIN2009-14511

•
Funding agency name Ministerio de Ciencia e Innovación
Funding agency acronym MICINN
Program Programa Nacional de Proyectos de Investigación Fundamental
Subprogram Subprograma de Proyectos de Investigación Fundamental no Orientada
Call Proyectos I+D
Call year 2009
State Plan 2008-2011 Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica
Management unit Subdirección General de Proyectos de Investigación
Beneficiary centre UNIVERSITAT POLITÈCNICA DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 27
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Passive-aggressive for on-line learning in statistical machine translation

  • Martínez Gómez, Pascual
  • Sanchis Trilles, Germán
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
New variations on the application of the passive-aggressive algorithm to statistical machine translation are developed and compared to previously existing approaches. In online adaptation, the system needs to adapt to real-world changing scenarios, where training and tuning only take place when the system is set-up for the first time. Post-edit information, as described by a given quality measure, is used as valuable feedback within the passive-aggressive framework, adapting the statistical models on-line. First, by modifying the translation model parameters, and alternatively, by adapting the scaling factors present in stateof- the-art SMT systems. Experimental results show improvements in translation quality by allowing the system to learn on a sentence-by-sentence basis., This paper is based upon work supported by the EC (FEDER/FSE) and the Spanish MICINN under projects MIPRCV “Consolider Ingenio 2010” (CSD2007-00018) and iTrans2 (TIN2009-14511). Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project, by the Generalitat Valenciana under grant Prometeo/2009/014 and scholarship GV/2010/067 and by the UPV under grant 20091027.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Arabic Printed Word Recognition Using Windowed Bernoulli HMMs

  • Alkhoury, Ihab
  • Andrés Ferrer, Jesús
  • Giménez Pastor, Adrián
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Hidden Markov Models (HMMs) are now widely used for off-line text recognition in many languages and, in particular, Arabic. In previous work, we proposed to directly use columns of raw, binary image pixels, which are directly fed into embedded Bernoulli (mixture) HMMs, that is, embedded HMMs in which the emission probabilities are modeled with Bernoulli mixtures. The idea was to by-pass feature extraction and to ensure that no discriminative information is filtered out during feature extraction, which in some sense is integrated into the recognition model. More recently, we extended the column bit vectors by means of a sliding window of adequate width to better capture image context at each horizontal position of the word image. However, these models might have limited capability to properly model vertical image distortions. In this paper, we have considered three methods of window repositioning after window extraction to overcome this limitation. Each sliding window is translated (repositioned) to align its center to the center of mass. Using this approach, state-of-art results are reported on the Arabic Printed Text Recognition (APTI) database., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no 287755. Also supported by the Spanish Government (Plan E, iTrans2 TIN2009-14511 and AECID 2011/2012 grant).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

GREAT: open source software for statistical machine translation

  • González Mollá, Jorge|||0000-0002-0129-0981
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
[EN] In this article, the first public release of GREAT as an open-source, statistical machine translation (SMT) software toolkit is described. GREAT is based on a bilingual language modelling approach for SMT, which is so far implemented for n-gram models based on the framework of stochastic finite-state transducers. The use of finite-state models is motivated by their simplicity, their versatility, and the fact that they present a lower computational cost, if compared with other more expressive models. Moreover, if translation is assumed to be a subsequential process, finite-state models are enough for modelling the existing relations between a source and a target language. GREAT includes some characteristics usually present in state-of-the-art SMT, such as phrase-based translation models or a log-linear framework for local features. Experimental results on a well-known corpus such as Europarl are reported in order to validate this software. A competitive translation quality is achieved, yet using both a lower number of model parameters and a lower response time than the widely-used, state-of-the-art SMT system Moses. © 2011 Springer Science+Business Media B.V., Study was supported by the EC (FEDER, FSE), the Spanish government (MICINN, MITyC, “Plan E”, under Grants MIPRCV “Consolider Ingenio 2010”, iTrans2 TIN2009-14511, and erudito.com TSI-020110-2009-439), and the Generalitat Valenciana (Grant Prometeo/2009/014).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Window repositioning for Printed Arabic Recognition

  • Alkhoury, Ihab
  • Andrés Ferrer, Jesús
  • Giménez Pastor, Adrián
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Bernoulli HMMs are conventional HMMs in which the emission probabilities are modeled with Bernoulli mixtures. They have recently been applied, with good results, in off-line text recognition in many languages, in particular, Arabic. A key idea that has proven to be very effective in this application of Bernoulli HMMs is the use of a sliding window of adequate width for feature extraction. This idea has allowed us to obtain very competitive results in the recognition of both Arabic handwriting and printed text. Indeed, a system based on it ranked first at the ICDAR 2011 Arabic recognition competition on the Arabic Printed Text Image (APTI) database. More recently, this idea has been refined by using repositioning techniques for extracted windows, leading to further improvements in Arabic handwriting recognition. In the case of printed text, this refinement led to an improved system which ranked second at the ICDAR 2013 second competition on APTI, only at a marginal distance from the best system. In this work, we describe the development of this improved system. Following evaluation protocols similar to those of the competitions on APTI, exhaustive experiments are detailed from which state-of-the-art results are obtained., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ICT-287755) under grant agreement no. 287755. The research is also supported by the Spanish Government (Plan E, iTrans2 TIN2009-14511 and AECID 2011/2012 grant).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Integrating a State-of-the-Art ASR System into the Opencast Matterhorn Platform

  • Valor Miró, Juan Daniel
  • Pérez González de Martos, Alejandro Manuel
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
[EN] In this paper we present the integration of a state-of-the-art ASR system into the Opencast Matterhorn platform, a free, open-source platform to support the management of educational audio and video content. The ASR system was trained on a novel large speech corpus, known as poliMedia, that was manually transcribed for the European project transLectures. This novel corpus contains more than 115 hours of transcribed speech that will be available for the research community. Initial results on the poliMedia corpus are also reported to compare the performance of different ASR systems based on the linear interpolation of language models. To this purpose, the in-domain poliMedia corpus was linearly interpolated with an external large-vocabulary dataset, the well-known Google N-Gram corpus. WER figures reported denote the notable improvement over the baseline performance as a result of incorporating the vast amount of data represented by the Google N-Gram corpus., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no 287755. Also supported by the Spanish Government (MIPRCV ”Consolider Ingenio 2010” and iTrans2 TIN2009-14511) and the Generalitat Valenciana (Prometeo/2009/014).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

An iterative multimodal framework for the transcription of handwritten historical documents

  • Alabau, Vicent
  • Romero Gómez, Verónica
  • Lagarda Arroyo, Antonio Luís
  • Martínez-Hinarejos, Carlos-D.|||0000-0002-6139-2891
[EN] The transcription of historical documents is one of the most interesting tasks in which Handwritten Text Recognition can be applied, due to its interest in humanities research. One alternative for transcribing the ancient manuscripts is the use of speech dictation by using Automatic Speech Recognition techniques. In the two alternatives similar models (Hidden Markov Models and n-grams) and decoding processes (Viterbi decoding) are employed, which allows a possible combination of the two modalities with little diffi- culties. In this work, we explore the possibility of using recognition results of one modality to restrict the decoding process of the other modality, and apply this process iteratively. Results of these multimodal iterative alternatives are significantly better than the baseline uni-modal systems and better than the non-iterative alternatives. 2012 Elsevier B.V. All rights reserved., Work supported by the EC (FEDER/FSE) and the Spanish MEC/MICINN under the MIPRCV ’’Consolider Ingenio 2010’’ program (CSD2007-00018), iTrans2 (TIN2009–14511) and MITTRAL (TIN2009-14633-C03–01) projects. Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project and by the Generalitat Valenciana under grant GV/2010/067, and by the UPV under project PAID-05-11-2779 and grant UPV/2009/2851.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Discriminative Bernoulli HMMs for isolated handwritten word recognition

  • Giménez Pastor, Adrián
  • Andrés Ferrer, Jesús
  • Juan, Alfons
[EN] Bernoulli HMMs (BHMMs) have been successfully applied to handwritten text recognition (HTR) tasks such as continuous and isolated handwritten words. BHMMs belong to the generative model family and, hence, are usually trained by (joint) maximum likelihood estimation (MLE) by means of the Baum-Welch algorithm. Despite the good properties of the MLE criterion, there are better training criteria such as maximum mutual information (MM!). The MMI is the most widespread criterion to train discriminative models such as log-linear (or maximum entropy) models. Inspired by a BHMM classifier, in this work, a log-linear HMM (LLHMM) for binary data is proposed. The proposed model is proved to be equivalent to the BHMM classifier, and, in this way, a discriminative training framework for BHMM classifiers is defined. The behavior of the proposed discriminative training framework is deeply studied in a well known task of isolated word recognition, the RIMES database. (C) 2013 Elsevier B.V. All rights reserved., Work supported by the EC (FEDER/FSE) and the Spanish MEC/MICINN under the MIPRCV ‘‘Consolider Ingenio 2010’’ program (CSD2007-00018), iTrans2 (TIN2009-14511) and MITTRAL (TIN2009-14633-C03-01) projects. Also supported by the IST Programme of the European Community, under the PASCAL2 Network of Excellence, IST-2007-216886, and by the Spanish MITyC under the erudito.com (TSI-020110-2009-439).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

TransLectures

  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
  • Giménez Pastor, Adrián
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
  • Del Agua Teba, Miguel Angel
  • Gascó Mora, Guillem
  • Martínez-Villaronga, Adrià Agustí
  • Pérez González de Martos, Alejandro Manuel
  • Sánchez-Cortina, Isaías
  • Serrano Martínez-Santos, Nicolás
  • Spencer, Rachel Nadine
  • Valor Miró, Juan Daniel
  • Andrés Ferrer, Jesús
transLectures (Transcription and Translation of Video Lectures) is an EU STREP project in which advanced automatic speech recognition and machine translation techniques are being tested on large video lecture repositories. The project began in November 2011 and will run for three years. This paper will outline the project¿s main motivation and objectives, and give a brief description of the two main repositories being considered: VideoLectures.NET and poliMedia. The first results obtained by the UPV group for the poliMedia repository will also be provided., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 287755. Funding was also provided by the Spanish Government (iTrans2 project, TIN2009-14511; FPI scholarship BES-2010-033005; FPU scholarship AP2010-4349)
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Does more data always yield better translations?

  • Gascó Mora, Guillem
  • Rocha Sánchez, Martha Alicia
  • Sanchis Trilles, Germán
  • Andrés Ferrer, Jesús
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
Nowadays, there are large amounts of data available to train statistical machine translation systems. However, it is not clear whether all the training data actually help or not. A system trained on a subset of such huge bilingual corpora might outperform the use of all the bilingual data. This paper studies such issues by analysing two training data selection techniques: one based on approximating the probability of an indomain corpus; and another based on infrequent n-gram occurrence. Experimental results not only report significant improvements over random sentence selection but also an improvement over a system trained with the whole available data. Surprisingly, the improvements are obtained with just a small fraction of the data that accounts for less than 0.5% of the sentences. Afterwards, we show that a much larger room for improvement exists, although this is done under non-realistic conditions., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement nr. 287755. This work was also supported by the Spanish MEC/MICINN under the MIPRCV ”Consolider Ingenio 2010” program (CSD2007-00018), and iTrans2 (TIN2009-14511) project. Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project and Instituto Tecnológico de León, DGEST-PROMEP y CONACYT, México.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Minimum Bayes’ risk subsequence combination for machine translation

  • Gonzalez Rubio, Jesus
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
System combination has proved to be a successful technique in the pattern recognition field. However, several difficulties arise when combining the outputs of tasks, e.g. machine translation, that generate structured patterns. So far, machine translation system combination approaches either implement sophisticated classifiers to select one of the provided translations, or generate new sentences by combining the "best" subsequences of the provided translations. We present minimum Bayes' risk system combination (MBRSC), a system combination method for machine translation that gathers together the advantages of sentence-selection and subsequence-combination methods. MBRSC is able to detect and utilize the "best" subsequences of the provided translations to generate the optimal consensus translation with respect to a particular performance met- ric. Experiments show that MBRSC yields significant improvements in translation quality., Work supported by the EC (FEDER/FSE) and the Spanish MEC/MICINN under the MIPRCV "Consolider Ingenio 2010'' program (CSD2007-00018), the iTrans2 (TIN2009-14511) project, the UPV under Grant 20091027, the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project and by the General-itat Valenciana under grant Prometeo/2009/014.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Explicit length modelling for statistical machine translation

  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Andrés Ferrer, Jesús
Explicit length modelling has been previously explored in statistical pattern recognition with successful results. In this paper, two length models along with two parameter estimation methods for statistical machine translation (SMT) are presented. More precisely, we incorporate explicit length modelling in a state-of-the-art log-linear SMT system as an additional feature function in order to prove the contribution of length information. Finally, promising experimental results are reported on a reference SMT task. © 2011 Springer-Verlag., Work supported by the EC (FEDER/FSE) and the Spanish MEC/MICINN under the MIPRCV “Consolider Ingenio 2010” program (CSD2007-00018) and iTrans2 (TIN2009-14511) projects. Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project and by the Generalitat Valenciana under grant Prometeo/2009/014 and GV/2010/067, and by the “Vicerrectorado de Investigaci´on de la UPV” under grant 20091027.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

On the optimal decision rule for sequential interactive structured prediction

  • Alabau, Vicent
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
[EN] Interactive structured prediction (ISP) is an emerging framework for structured prediction (SP) where the user and the system collaborate to produce a high quality output. Typically, search algorithms applied to ISP problems have been based on the algorithms for fully-automatic SP systems. However, the decision rule applied should not be considered as optimal since the goal in ISP is to reduce human effort instead of output errors. In this work, we present some insight into the theory of the sequential ISP search problem. First, it is formulated as a decision theory problem from which a general analytical formulation of the opti- mal decision rule is derived. Then, it is compared with the standard formulation to establish under what conditions the standard algorithm should perform similarly to the optimal decision rule. Finally, a general and practical implementation is given and evaluated against three classical ISP problems: interactive machine translation, interactive handwritten text recognition, and interactive speech recognition., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under Grant agreement no. 287576 (CasMaCat), and from the Spanish MEC/MICINN under the MIPRCV "Consolider Ingenio 2010" program (CSD2007-00018) and iTrans2 (TIN2009-14511) project. It is also supported by the Generalitat Valenciana under grant ALMPR (Prometeo/2009/01) and GV/2010/067. The authors thank the anonymous reviewers for their criticisms and suggestions.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Explicit length modelling for statistical machine translation

  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Andrés Ferrer, Jesús
[EN] Explicit length modelling has been previously explored in statistical pattern recognition with successful results. In this paper, two length models along with two parameter estimation methods and two alternative parametrisations for statistical machine translation (SMT) are presented. More precisely, we incorporate explicit bilingual length modelling in a state-of-the-art log-linear SMT system as an additional feature function in order to prove the contribution of length information. Finally, a systematic evaluation on reference SMT tasks considering different language pairs proves the benefits of explicit length modelling., Work supported by the EC (FEDER/FSE) under the transLectures project (FP7-ICT-2011-7-287755) and the Spanish MEC/MICINN under the MIPRCV "Consolider Ingenio 2010" program (CSD2007-00018) and iTrans2 (TIN2009-14511) projects and FPU grant (AP2010-4349). Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project and by the Generalitat Valenciana under grants Prometeo/2009/014 and GV/2010/067, and by the UPV under the AdInTAO (20091027) project. The authors wish to thank the anonymous reviewers for their criticisms and suggestions.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Speaker-adapted confidence measures for speech recognition of video lectures

  • Sanchez-Cortina, Isaias
  • Andrés Ferrer, Jesús
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Automatic speech recognition applications can benefit from a confidence measure (CM) to predict the reliability of the output. Previous works showed that a word-dependent native Bayes (NB) classifier outperforms the conventional word posterior probability as a CM. However, a discriminative formulation usually renders improved performance due to the available training techniques. Taking this into account, we propose a logistic regression (LR) classifier defined with simple input functions to approximate to the NB behaviour. Additionally, as a main contribution, we propose to adapt the CM to the speaker in cases in which it is possible to identify the speakers, such as online lecture repositories. The experiments have shown that speaker-adapted models outperform their non-adapted counterparts on two difficult tasks from English (videoLectures.net) and Spanish (poliMedia) educational lectures. They have also shown that the NB model is clearly superseded by the proposed LR classifier., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no 287755. Also supported by the Spanish MINECO (iTrans2 TIN2009-14511 and Active2Trans TIN2012-31723) research projects and the FPI Scholarship BES-2010-033005.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Online adaptation strategies for statistical machine translation in post-editing scenarios

  • Martínez Gómez, Pascual
  • Sanchis Trilles, Germán
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
[EN] One of the most promising approaches to machine translation consists in formulating the problem by means of a pattern recognition approach. By doing so, there are some tasks in which online adapta- tion is needed in order to adapt the system to changing scenarios. In the present work, we perform an exhaustive comparison of four online learning algorithms when combined with two adaptation strategies for the task of online adaptation in statistical machine translation. Two of these algorithms are already well-known in the pattern recognition community, such as the perceptron and passive- aggressive algorithms, but here they are thoroughly analyzed for their applicability in the statistical machine translation task. In addition, we also compare them with two novel methods, i.e., Bayesian predictive adaptation and discriminative ridge regression. In statistical machine translation, the most successful approach is based on a log-linear approximation to a posteriori distribution. According to experimental results, adapting the scaling factors of this log-linear combination of models using discriminative ridge regression or Bayesian predictive adaptation yields the best performance., This paper is based upon work supported by the EC (FP7) under CasMaCat (287576) project and the EC (FEDER/FSE) and the Spanish MICINN under projects MIPRCV "Consolider Ingenio 2010" (CSD2007-00018) and iTrans2 (TIN2009-14511). This work is also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project, by the Generalitat Valenciana under Grant Prometeo/2009/014, and by the UPV under Grant 20091027. The authors would like to thank the anonymous reviewers for their useful and constructive comments.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Extracción de Corpus Paralelos de la Wikipedia basada en la Obtención de Alineamientos Bilingües a Nivel de Frase, Extracting Parallel Corpora from Wikipedia on the basis of Phrase Level Bilingual Alignment

  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Rosso, Paolo|||0000-0002-8922-1242
  • Garcia Martinez, Maria Mercedes
  • Barrón Cedeño, Luis Alberto
[EN] This paper presents a proposal for extracting parallel corpora from Wikipedia on the basis of statistical machine translation techniques. We have used word-level alignment models from IBM in order to obtain phrase-level bilingual alignments between documents pairs. We have manually annotated a set of test English-Spanish comparable documents in order to evaluate the model. The obtained results are encouraging., [ES] Este art'¿culo presenta una nueva t'ecnica de extracci'on de corpus paralelos de la Wikipedia mediante la aplicaci'on de t'ecnicas de traducci'on autom'atica estad'¿stica. En concreto, se han utilizado los modelos de alineamiento basados en palabras de IBM para obtener alineamientos biling¿ues a nivel de frase entre pares de documentos. Para su evaluaci'on se ha generado manualmente un conjunto de test formado por pares de documentos ingl'es-espa¿nol, obteni'endose resultados prometedores., Este trabajo se ha llevado a cabo en el marco del VLC/CAMPUS Microcluster on Multimodal Interaction in Intelligent Systems, financiado parcialmente por parte de la EC (FEDER/FSE; WIQEI IRSES no. 269180 / FP 7 Marie Curie People), por el MICINN como parte del proyecto Text-Enterprise 2.0 (TIN2009-13391-C04-03) en el Plan I+D+i, y por la beca 192021 del CONACyT. Tambi´en ha recibido apoyo por parte del EC (FEDER/FSE) y del MEC/MICINN bajo el programa MIPRCV “Consolider Ingenio 2010” (CSD2007-00018) y el proyecto iTrans2 (TIN2009-14511), por el MITyC en el marco del proyecto erudito.com (TSI-020110-2009-439), por la Generalitat Valenciana con las ayudas Prometeo/2009/014 y GV/2010/067, y por el “Vicerrectorado de Investigaci´on de la UPV” con la ayuda 20091027.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Discriminative Bernoulli Mixture Models for Handwritten Digit Recognition

  • Giménez Pastor, Adrián
  • Juan, Alfons|||0000-0002-9984-4072
  • Andrés Ferrer, Jesús
  • Serrano Martinez Santos, Nicolas
Bernoulli-based models such as Bernoulli mixtures or Bernoulli HMMs (BHMMs), have been successfully applied to several handwritten text recognition (HTR) tasks which range from character recognition to continuous and isolated handwritten words. All these models belong to the generative model family and, hence, are usually trained by (joint) maximum likelihood estimation (MLE). Despite the good properties of the MLE criterion, there are better training criteria such as maximum mutual information (MMI). The MMI is a widespread criterion that is mainly employed to train discriminative models such as log-linear (or maximum entropy) models. Inspired by the Bernoulli mixture classifier, in this work a log-linear model for binary data is proposed, the so-called mixture of multiclass logistic regression. The proposed model is proved to be equivalent to the Bernoulli mixture classifier. In this way, we give a discriminative training framework for Bernoulli mixture models. The proposed discriminative training framework is applied to a well-known Indian digit recognition task., Work supported by the EC (FEDER/FSE) and the Spanish MEC/MICINN under the MIPRCV “Consolider Ingenio 2010” program (CSD2007-00018), iTrans2 (TIN2009-14511) and MITTRAL (TIN2009-14633-C03-01) projects. Also supported by the IST Programme of the European Community, under the PASCAL2 Network of Excellence, IST-2007-216886, and by the Spanish MITyC under the erudito.com (TSI-020110-2009-439).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

A word-based naïve bayes classifier for confidence estimation in speech recognition

  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
  • Vidal, Enrique|||0000-0003-4579-5196
Confidence estimation has been largely used in speech recognition to detect words in the recognized sentence that have been likely misrecognized. Confidence estimation can be seen as a conventional pattern classification problem in which a set of features is obtained for each hypothesized word in order to classify it as either correct or incorrect. We propose a smoothed naïve Bayes classification model to profitably combine these features. The model itself is a combination of word-dependent (specific) and word-independent (generalized) naïve Bayes models. As in statistical language modeling, the purpose of the generalized model is to smooth the (class posterior) estimates given by the specific models. Our classification model is empirically compared with confidence estimation based on posterior probabilities computed on word graphs. Empirical results clearly show that the good performance of word graph-based posterior probabilities can be improved by using the naïve Bayes combination of features., This work was supported by the EC (FEDER/FSE), the Spanish Government (MICINN, MITyC, "Plan E," under grants MIPRCV "Consolider Ingenio 2010" CSD2007-00018, iTrans2 TIN2009-14511, and erudito.com TSI-020110-2009-439), the Generalitat Valenciana (grants Prometeo/2009/014 and GV/2010/067), and the Universitat Politecnica de Valencia (grant 20091027). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Mark J. F. Gales.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Online learning via dynamic reranking for Computer Assisted Translation

  • Martínez Gómez, Pascual
  • Sanchis Trilles, Germán
  • Casacuberta Nolla, Francisco|||0000-0002-8497-5598
New techniques for online adaptation in computer assisted translation are explored and compared to previously existing approaches. Under the online adaptation paradigm, the translation system needs to adapt itself to real-world changing scenarios, where training and tuning may only take place once, when the system is set-up for the first time. For this purpose, post-edit information, as described by a given quality measure, is used as valuable feedback within a dynamic reranking algorithm. Two possible approaches are presented and evaluated. The first one relies on the well-known perceptron algorithm, whereas the second one is a novel approach using the Ridge regression in order to compute the optimum scaling factors within a state-of-the-art SMT system. Experimental results show that such algorithms are able to improve translation quality by learning from the errors produced by the system on a sentence-by-sentence basis., This paper is based upon work supported by the EC (FEDER/FSE) and the Spanish MICINN under projects MIPRCV “Consolider Ingenio 2010” (CSD2007-00018) and iTrans2 (TIN2009-14511). Also supported by the Spanish MITyC under the erudito.com (TSI-020110-2009-439) project, by the Generalitat Valenciana under grant Prometeo/2009/014 and scholarship GV/2010/067 and by the UPV under grant 20091027
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Effective balancing error and user effort in interactive handwriting recognition

  • Serrano Martinez Santos, Nicolas
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Transcription of handwritten text documents is an expensive and time-consuming task. Unfortunately, the accuracy of current state-of-the-art handwriting recognition systems cannot guarantee fully-automatic high quality transcriptions, so we need to revert to the computer assisted approach. Although this approach reduces the user effort needed to transcribe a given document, the transcription of handwriting text documents still requires complete manual supervision. An especially appealing scenario is the interactive transcription of handwriting documents, in which the user defines the amount of errors that can be tolerated in the final transcribed document. Under this scenario, the transcription of a handwriting text document could be obtained efficiently, supervising only a certain number of incorrectly recognised words. In this work, we develop a new method for predicting the error rate in a block of automatically recognised words, and estimate how much effort is required to correct a transcription to a certain user-defined error rate. The proposed method is included in an interactive approach to transcribing handwritten text documents, which efficiently employs user interactions by means of active and semi-supervised learning techniques, along with a hypothesis recomputation algorithm based on constrained Viterbi search. Transcription results, in terms of trade-off between user effort and transcription accuracy, are reported for two real handwritten documents, and prove the effectiveness of the proposed approach., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under Grant agreement No 287755 (transLectures). Also supported by the EC (FEDER, FSE), the Spanish Government (MICINN, MITyC, "Plan E", under grants MIPRCV "Consolider Ingenio 2010", MITTRAL (TIN2009-14633-C03-01), iTrans2 (TIN2009-14511), and FPU (AP2007-02867), and the Generalitat Valenciana (Grants Prometeo/2009/014 and GV/2010/067). Special thanks to Jesus Andres for his fruitful discussions.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

The UPV Handwriting Recognition and Translation System for OpenHaRT 2013

  • Alkhoury, Ihab
  • Andrés Ferrer, Jesús
  • Giménez Pastor, Adrián
  • Juan, Alfons|||0000-0002-9984-4072
  • Sánchez Peiró, Joan Andreu|||0000-0003-0423-2020
The NIST Open Handwriting Recognition and Translation Evaluation 2013 (NIST OpenHaRT’13) is a performance evaluation assessing technologies that transcribe and translate text in document images. This evaluation is focused on recognizing Arabic text images and translating them into English. A Handwriting Recognition and Translation system typically consists of a combination of two systems: a Text Recognition system and a Machine Translation system. In this paper, we present the UPV participation in the NIST OpenHaRT 2013 evaluation. For the Text Recognition system we used the TL toolkit for training and recognition. For the Machine Translation system we used the Moses toolkit for training and decoding. Results in this evaluation are challenging and they significantly outperform our previous results in the OpenHaRT 2010 evaluation.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Tesis de maestría (MasterThesis)

Machine Translation of Open Educational Resources: Evaluating Translation Quality and the Transition to Neural Machine Translation, Traducción automática de recursos educativos abiertos: Evaluación de la calidad de la traducción y la transición hacia la traducción automáticaneuronal

  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
[EN] Open Education has become a revolutionary approach towards the future of education, enabling worldwide free access to a huge volume of Open Educational Resources (OER). The rapid growth of OER and MOOCs has not gone unnoticed by governments and international organizations, as is demonstrated by the UNESCO-sponsored 2012 Paris OER Declaration and the 2017 Ljubljana OER Action Plan, the latter addressing ve strategic actions to support the mainstreaming of OER around UN Sustainable Development Goal 4 (SDG4) on Quality Education . In the EU, the European Commission's 2013 Opening up Education agenda recognized that the EU lacks a critical mass of good quality educational content in multiple languages. Although there is a clear need for multilingual services in Open Education, current OER platforms and MOOCs do not o er multilingual communication and seldom o er multilingual content. Based on this evidence, this master's thesis aims to foster Open Education with contributions on machine translation for the provision of multilingual access to OER and MOOC platforms. With the trans-disciplinary tools of automatic speech recognition (ASR), machine translation (MT), text-to-speech synthesis (TTS), and dialogue, multilingual access to OER will be possible for everyone regardless of their mother tongue or learning abilities. Firstly, we present work on the evaluation of MT, including intelligent interaction approaches to post-editing, as carried out in the framework of EU project transLectures. Evaluating MT is still an open question, and so exploring di erent ways to address this e ectively and applying them in real scenarios has a clear interest. The results obtained con rm that the intelligent interaction approach can make postediting automatic transcriptions and translations even more cost-e ective. Secondly, we present work on developing state-of-the-art neural machine translation (NMT) systems to transition from the previous phrase-based MT models. The new NMT paradigms provide signi cant improvements in MT quality, and so at this point it is key to move towards them, research on how to improve them, and, again, apply them in real scenarios to verify their usefulness. Our work resulted in a rstrank classi cation in an international evaluation campaign on MT, and we show the impact that these new NMT systems have in real OER scenarios. A structured comparison shows that our results are on par with the high quality of recent Google Translate results, and also shows that it is possible to go beyond Google Translate's quality through domain adaptation of MT systems., [ES] La Educación Abierta es cada vez más un modelo revolucionario de cara al futuro de la educación, permitiendo el acceso libre a nivel global a un enorme volumen de Recursos Educativos Abiertos (REA). El rápido crecimiento de los REA y de los cursos MOOC no ha pasado desapercibido para los gobiernos y las organizaciones internacionales, como se comprueba por la Declaración de París sobre los REA de 2012 y el Plan de Acción de Liubliana sobre los REA de 2017. Este último propone cinco acciones estratégicas para la implantación de los REA en torno al 4.o Objetivo de Desarrollo Sostenible de la ONU (ODS4) sobre Educación de Calidad . En la UE, la agenda Apertura de la educación de la Comisión Europea (2013) reconoce que la UE carece de una masa crítica de contenidos educativos de alta calidad en múltiples lenguas. A pesar de la clara necesidad de servicios multilingües para la Educación Abierta, las plataformas actuales de REA y MOOC no ofrecen comunicación multilingüe y raramente ofrecen contenidos multilingües. En base a esta evidencia, este Trabajo de Fin de Máster propone fomentar la Educación Abierta con contribuciones sobre la traducción automática (TA) para la provisión de acceso multilingüe a plataformas de REA y MOOC. Las herramientas interdisciplinares del reconocimiento automático del habla, la traducción automática, la síntesis de texto a voz y las tecnologías del diálogo harán posible el acceso multilingüe a los REA para todos, independientemente de la lengua materna o de las capacidades de aprendizaje de cada uno. En primer lugar, presentamos un trabajo sobre la evaluación de la TA, incluyendo métodos de interacción inteligente para la postedición, en el marco del proyecto europeo transLectures. La evaluación de la TA es aún una cuestión abierta, de forma que explorar diferentes vías para abordarla de forma efectiva y aplicarlas en escenarios reales tiene un claro interés. Mostraremos resultados que con rman que la interacción inteligente reduce el esfuerzo necesario para la postedición de transcripciones y traducciones automáticas. En segundo lugar, presentamos un esfuerzo de desarrollo de sistemas de traducción automática neuronal (TAN) punteros para superar los anteriores modelos de TA basada en frases. Los nuevos paradigmas de la TAN aportan mejoras signi cativas en la calidad de la TA, y es fundamental adoptarlos, investigar sobre cómo mejorarlos, y, de nuevo, aplicarlos en escenarios reales para veri car su utilidad. El sistema de TAN desarrollado se ha clasi cado entre los mejores en una competición internacional de TA, y aquí mostramos la repercusión que tienen estos nuevos sistemas de TAN en escenarios reales de REA. Nos hemos comparado y hemos comprobado que nuestros resultados compiten con la alta calidad de los resultados más recientes de Google Translate, y que es posible ir más allá de esta calidad con la adaptación al dominio de los sistemas de TA., [CA] L'Educació Oberta ha esdevingut un model revolucionari de cara al futur de l'educació, permetent l'accés lliure a nivell global a un enorme volum de Recursos Educatius Oberts (REO). El ràpid creixement dels REO i dels cursos MOOC no ha passat desapercebut per als governs i les organitzacions internacionals, com es comprova per la Declaració de París sobre els REO de 2012 i el Pla d'Acció de Ljubljana sobre els REO de 2017. Aquest últim proposa cinc accions estratègiques per a la implantació dels REO entorn del 4t Objectiu de Desenvolupament Sostenible de l'ONU (ODS4) sobre Educació de Qualitat . A l'UE, l'agenda Obertura de l'educació de la Comissió Europea (2013) reconeix que a l'UE manca una massa crítica de continguts educatius d'alta qualitat en múltiples llengües. Malgrat la clara necessitat de serveis multilingües per a l'Educació Oberta, les plataformes actuals de REO i MOOC no ofereixen comunicació multilingüe i rarament ofereixen continguts multilingües. Sobre la base d'aquesta evidència, aquest Treball de Fi de Màster es proposa fomentar l'Educació Oberta amb contribucions sobre la traducció automàtica (TA) per a la provisió d'accés multilingüe a plataformes de REO i MOOC. Les eines interdisciplinàries del reconeixement automàtic de la parla, la traducció automàtica, la síntesi de text a veu i les tecnologies del diàleg faran possible l'accés multilingüe als REO per a tots, independentment de la llengua materna o de les destreses d'aprenentatge. En primer lloc, presentem un treball sobre l'avaluació de la TA, incloent-hi mètodes d'interacció intel·ligent per a la postedició, en el marc del projecte europeu transLectures. L'avaluació de la TA és encara una qüestió oberta, de manera que explorar diferents vies per a abordar-la de forma efectiva i aplicar-les en escenaris reals té un interés clar. Els resultats obtinguts con rmen que la interacció intel·ligent redueix l'esforç necessari per a la postedició de transcripcions i traduccions automàtiques. En segon lloc, presentem el desenvolupament de sistemes de traducció automàtica neuronal (TAN) punters per a superar els anteriors models de TA basada en frases. Els nous paradigmes de la TAN aporten millores signi catives en la qualitat de la TA, i és fonamental abraçar-los, investigar sobre com millorar-los, i, novament, aplicarlos en escenaris reals per a veri car-ne la utilitat. El sistema de TAN desenvolupat s'ha classi cat entre els millors en una competició internacional de TA, i ací mostrem la repercussió que tenen aquests nous sistemes de TAN en escenaris reals de REO. Ens hem comparat i hem comprovat que els nostres resultats competeixen amb l'alta qualitat dels resultats més recents de Google Translate, i que és possible anar més enllà d'aquesta qualitat amb l'adaptació al domini dels sistemes de TA., The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 287755 (transLectures), Competitiveness and Innovation Framework Programme (CIP) under grant agreement no. 621030 (EMMA) and Horizon 2020 research and innovation programme under grant agreement no. 761758 (X5gon); from the Government of Spain's research projects iTrans2 (ref. TIN2009-14511, MICINN/FEDER EU) and MORE (ref. TIN2015-68326-R, MINECO/FEDER EU); and from the Universitat Politècnica de València's PAID-01-17 R&D support programme.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Interactive handwriting recognition with limited user effort

  • Serrano Martinez Santos, Nicolas
  • Giménez Pastor, Adrián
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Transcription of handwritten text in (old) documents is an important, time-consuming task for digital libraries. Although post-editing automatic recognition of handwritten text is feasible, it is not clearly better than simply ignoring it and transcribing the document from scratch. A more effective approach is to follow an interactive approach in which both the system is guided by the user, and the user is assisted by the system to complete the transcription task as efficiently as possible. Nevertheless, in some applications, the user effort available to transcribe documents is limited and fully supervision of the system output is not realistic. To circumvent these problems, we propose a novel interactive approach which efficiently employs user effort to transcribe a document by improving three different aspects. Firstly, the system employs a limited amount of effort to solely supervise recognised words that are likely to be incorrect. Thus, user effort is efficiently focused on the supervision of words for which the system is not confident enough. Secondly, it refines the initial transcription provided to the user by recomputing it constrained to user supervisions. In this way, incorrect words in unsupervised parts can be automatically amended without user supervision. Finally, it improves the underlying system models by retraining the system from partially supervised transcriptions. In order to prove these statements, empirical results are presented on two real databases showing that the proposed approach can notably reduce user effort in the transcription of handwritten text in (old) documents., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No 287755 (transLectures). Also supported by the Spanish Government (MICINN, MITyC, "Plan E", under Grants MIPRCV "Consolider Ingenio 2010", MITTRAL (TIN2009-14633-C03-01), erudito.com (TSI-020110-2009-439), iTrans2 (TIN2009-14511), and FPU (AP2007-02867), and the Generalitat Valenciana (Grants Prometeo/2009/014 and GV/2010/067).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Character-Based Handwritten Text Recognition of Multilingual Documents

  • Del Agua Teba, Miguel Angel
  • Serrano Martinez Santos, Nicolas
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
[EN] An effective approach to transcribe handwritten text documents is to follow a sequential interactive approach. During the supervision phase, user corrections are incorporated into the system through an ongoing retraining process. In the case of multilingual documents with a high percentage of out-of-vocabulary (OOV) words, two principal issues arise. On the one hand, a minor yet important matter for this interactive approach is to identify the language of the current text line image to be transcribed, as a language dependent recognisers typically performs better than a monolingual recogniser. On the other hand, word-based language models suffer from data scarcity in the presence of a large number of OOV words, degrading their estimation and affecting the performance of the transcription system. In this paper, we successfully tackle both issues deploying character-based language models combined with language identification techniques on an entire 764-page multilingual document. The results obtained significantly reduce previously reported results in terms of transcription error on the same task, but showed that a language dependent approach is not effective on top of character-based recognition of similar languages., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n◦ 287755. Also supported by the Spanish Government (MIPRCV ”Consolider Ingenio 2010”, iTrans2 TIN2009-14511, MITTRAL TIN2009-14633-C03-01 and FPU AP2007-0286) and the Generalitat Valenciana (Prometeo/2009/014).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Tesis doctoral (DoctoralThesis)

Automatic Speech Recognition and Machine Translation with Deep Neural Networks for Open Educational Resources, Parliamentary Contents and Broadcast Media

  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
[ES] En la última década, el reconocimiento automático del habla (RAH) y la traducción automática (TA) han mejorado enormemente mediante el uso de modelos de redes neuronales profundas (RNP) en constante evolución. Si a principios de los 2010 los sistemas de RAH y TA previos a las RNP llegaron a afrontar con éxito algunas aplicaciones reales como la transcripción y traducción de vídeos docentes pregrabados, ahora en los 2020 son abordables aplicaciones que suponen un reto mucho mayor, como la subtitulación de retransmisiones audiovisuales en directo. En este mismo período, se están invirtiendo cada vez mayores esfuerzos en la accesibilidad a los medios audiovisuales para todos, incluidas las personas sordas. El RAH y la TA, en su estado actual, son grandes herramientas para aumentar la disponibilidad de medidas de accesibilidad como subtítulos, transcripciones y traducciones, y también para proporcionar acceso multilingüe a todo tipo de contenidos. En esta tesis doctoral presentamos resultados de investigación sobre RAH y TA basadas en RNP en tres campos muy activos: los recursos educativos abiertos, los contenidos parlamentarios y los medios audiovisuales. En el área de los recursos educativos abiertos (REA), presentamos primeramente trabajos sobre la evaluación y postedición de RAH y TA con métodos de interacción inteligente, en el marco del proyecto de investigación europeo "transLectures: Transcripción y Traducción de Vídeos Docentes". Los resultados obtenidos confirman que la interacción inteligente puede reducir aún más el esfuerzo de postedición de transcripciones y traducciones automáticas. Seguidamente, en el contexto del posterior proyecto europeo X5gon, presentamos una investigación sobre el desarrollo de sistemas de TA neuronal basados en RNP, y sobre sacar el máximo partido de corpus de TA masivos mediante filtrado automático de datos. Este trabajo dio como resultado sistemas de TA neuronal clasificados entre los mejores en una competición internacional de TA, y mostramos cómo estos nuevos sistemas mejoraron la calidad de los subtítulos multilingües en casos reales de REA. En el ámbito también en crecimiento de las tecnologías del lenguaje para contenidos parlamentarios, describimos una investigación sobre técnicas de filtrado de datos de habla para el RAH en tiempo real en el contexto de debates del Parlamento Europeo. Esta investigación permitió la publicación de Europarl-ASR, un nuevo y amplio corpus de habla para entrenamiento y evaluación de sistemas de RAH en continuo, así como para la evaluación comparativa de técnicas de filtrado de datos de habla. Finalmente, presentamos un trabajo en un ámbito en la vanguardia tecnológica del RAH y de la TA: la subtitulación de retransmisiones audiovisuales en directo, en el marco del Convenio de colaboración I+D+i 2020-2023 entre la radiotelevisión pública valenciana À Punt y la Universitat Politècnica de València para la subtitulación asistida por ordenador de contenidos audiovisuales en tiempo real. Esta investigación ha resultado en la implantación de sistemas de RAH en tiempo real, de alta precisión y baja latencia, para una lengua no mayoritaria en el mundo (el catalán) y una de las lenguas más habladas del mundo (el castellano) en un medio audiovisual real., [CA] En l'última dècada, el reconeixement automàtic de la parla (RAP) i la traducció automàtica (TA) han millorat enormement mitjançant l'ús de models de xarxes neuronals profundes (XNP) en constant evolució. Si a principis dels 2010 els sistemes de RAP i TA previs a les XNP van arribar a afrontar amb èxit algunes aplicacions reals com la transcripció i traducció de vídeos docents pregravats, ara en els 2020 són abordables aplicacions que suposen un repte molt major, com la subtitulació de retransmissions audiovisuals en directe. En aquest mateix període, s'estan invertint cada vegada majors esforços en l'accessibilitat als mitjans audiovisuals per a tots, incloses les persones sordes. El RAP i la TA, en el seu estat actual, són grans eines per a incrementar la disponibilitat de mesures d'accessibilitat com subtítols, transcripcions i traduccions, també com una manera de proporcionar accés multilingüe a tota classe de continguts. En aquesta tesi doctoral presentem resultats d'investigació sobre RAP i TA basades en XNP en tres camps molt actius: els recursos educatius oberts, els continguts parlamentaris i els mitjans audiovisuals. En l'àrea dels recursos educatius oberts (REO), presentem primerament treballs sobre l'avaluació i postedició de RAP i TA amb mètodes d'interacció intel·ligent, en el marc del projecte d'investigació europeu "transLectures: Transcripció i traducció de vídeos docents". Els resultats obtinguts confirmen que la interacció intel·ligent pot reduir encara més l'esforç de postedició de transcripcions i traduccions automàtiques. Seguidament, en el context del posterior projecte europeu X5gon, presentem una investigació sobre el desenvolupament de sistemes de TA neuronal basats en XNP, i sobre traure el màxim partit de corpus de TA massius mitjançant filtratge automàtic de dades. Aquest treball va donar com a resultat sistemes de TA neuronal classificats entre els millors en una competició internacional de TA, i mostrem com aquests nous sistemes milloren la qualitat dels subtítols multilingües en casos reals de REO. En l'àmbit també en creixement de les tecnologies del llenguatge per a continguts parlamentaris, descrivim una investigació sobre tècniques de filtratge de dades de parla per al RAP en temps real en el context de debats del Parlament Europeu. Aquesta investigació va permetre la publicació d'Europarl-ASR, un corpus de parla nou i ampli per a l'entrenament i l'avaluació de sistemes de RAP en continu, així com per a l'avaluació comparativa de tècniques de filtratge de dades de parla. Finalment, presentem un treball en un àmbit en l'avantguarda tecnològica del RAP i de la TA: la subtitulació de retransmissions audiovisuals en directe, en el context del Conveni de col·laboració R+D+i 2020-2023 entre la radiotelevisió pública valenciana À Punt i la Universitat Politècnica de València per a la subtitulació assistida per ordinador de continguts audiovisuals en temps real. Aquesta investigació ha donat com a resultat la implantació de sistemes de RAP en temps real, amb alta precisió i baixa latència, per a una llengua no majoritària en el món (el català) i una de les llengües més parlades del món (el castellà) en un mitjà audiovisual real., [EN] In the last decade, automatic speech recognition (ASR) and machine translation (MT) have improved enormously through the use of constantly evolving deep neural network (DNN) models. If at the beginning of the 2010s the then pre-DNN ASR and MT systems were ready to tackle with success some real-life applications such as offline video lecture transcription and translation, now in the 2020s much more challenging applications are within grasp, such as live broadcast media subtitling. At the same time in this period, media accessibility for everyone, including deaf and hard-of-hearing people, is being given more and more importance. ASR and MT, in their current state, are powerful tools to increase the coverage of accessibility measures such as subtitles, transcriptions and translations, also as a way of providing multilingual access to all types of content. In this PhD thesis, we present research results on automatic speech recognition and machine translation based on deep neural networks in three very active domains: open educational resources, parliamentary contents and broadcast media. Regarding open educational resources (OER), we first present work on the evaluation and post-editing of ASR and MT with intelligent interaction approaches, as carried out in the framework of EU project transLectures: Transcription and Translation of Video Lectures. The results obtained confirm that the intelligent interaction approach can make post-editing automatic transcriptions and translations even more cost-effective. Then, in the context of subsequent EU project X5gon, we present research on developing DNN-based neural MT systems, and making the most of larger MT corpora through automatic data filtering. This work resulted in a first-rank classification in an international evaluation campaign on MT, and we show how these new NMT systems improved the quality of multilingual subtitles in real OER scenarios. In the also growing domain of language technologies for parliamentary contents, we describe research on speech data curation techniques for streaming ASR in the context of European Parliament debates. This research resulted in the release of Europarl-ASR, a new, large speech corpus for streaming ASR system training and evaluation, as well as for the benchmarking of speech data curation techniques. Finally, we present work in a domain on the edge of the state of the art for ASR and MT: the live subtitling of broadcast media, in the context of the 2020-2023 R&D collaboration agreement between the Valencian public broadcaster À Punt and the Universitat Politècnica de València for real-time computer assisted subtitling of media contents. This research has resulted in the deployment of high-quality, low-latency, real-time streaming ASR systems for a less-spoken language (Catalan) and a widely spoken language (Spanish) in a real broadcast use case., The research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 287755 (transLectures), Competitiveness and Innovation Framework Programme (CIP) under grant agreement no. 621030 (EMMA), Horizon 2020 research and innovation programme under grant agreements no. 761758 (X5gon) and no. 952215 (TAILOR), and EU4Health Programme 2021–2027 as part of Europe’s Beating Cancer Plan under grant agreements no. 101056995 (INTERACT-EUROPE) and no. 101129375 (INTERACT-EUROPE 100); from the Government of Spain’s research projects iTrans2 (ref. TIN2009-14511, MICINN/ERDF EU), MORE (ref. TIN2015-68326-R,MINECO/ERDF EU), Multisub (ref. RTI2018-094879-B-I00, MCIN/AEI/10.13039/501100011033 ERDF “A way of making Europe”), and XLinDub (ref. PID2021-122443OB-I00, MCIN/AEI/10.13039/501100011033 ERDF “A way of making Europe”); from the Generalitat Valenciana’s “R&D collaboration agreement between the Corporació Valenciana de Mitjans de Comunicació (À Punt Mèdia) and the Universitat Politècnica de València (UPV) for real-time computer assisted subtitling of audiovisual contents based on artificial intelligence”, and research project Classroom Activity Recognition (PROMETEO/2019/111); and from the Universitat Politècnica de València’s PAID-01-17 R&D support programme. This work uses data from the RTVE 2018 and 2020 Databases. This set of data has been provided by RTVE Corporation to help develop Spanish-language speech technologies.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Language model adaptation for video lectures transcription

  • Martínez-Villaronga, Adrià
  • Del Agua Teba, Miguel Angel
  • Andrés Ferrer, Jesús
  • Juan, Alfons|||0000-0002-9984-4072
Videolectures are currently being digitised all over the world for its enormous value as reference resource. Many of these lectures are accompanied with slides. The slides offer a great opportunity for improving ASR systems performance. We propose a simple yet powerful extension to the linear interpolation of language models for adapting language models with slide information. Two types of slides are considered, correct slides, and slides automatic extracted from the videos with OCR. Furthermore, we compare both time aligned and unaligned slides. Results report an improvement of up to 3.8 % absolute WER points when using correct slides. Surprisingly, when using automatic slides obtained with poor OCR quality, the ASR system still improves up to 2.2 absolute WER points., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no 287755 (transLectures). Also supported by the Spanish Government (Plan E, iTrans2 TIN2009-14511).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Capítulo o parte de un libro (BookPart)

Albayzin Evaluation: The PRHLT-UPV Audio Segmentation System

  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Giménez Pastor, Adrián
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
  • Andrés Ferrer, Jesús
This paper describes the audio segmentation system developed by the PRHLT research group at the UPV for the Albayzin Audio Segmentation Evaluation 2012. The PRHLT-UPV audio segmentation system is based on a conventional GMM-HMM speech recognition approach in which the vocabulary set is defined by the power set of segment classes. MFCC features were extracted to represent the acoustic signal and the AK toolkit was used for both, training acoustic models and performing audio segmentation. Experimental results reveals that our system provides an excellent performance on speech detection, so it could be successfully employed to provide speech segments to a diarization or speech recognition system., The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 287755. Funding was also provided by the Spanish Government (iTrans2 project, TIN2009-14511; FPU scholarship AP2010-4349).