Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 50
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe di erent or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a mul- timodal query, requires the combination of several sources of information in an e cient and scalable way. Towards this direction, we provide a novel unsuper- vised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fu- sion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when com- pared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Neural architectures for named entity recognition

  • Lample, Guillaume
  • Ballesteros, Miguel
  • Subramanian, Sandeep
  • Kawakami, Kazuya
  • Dyer, Chris
Comunicació presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 de juny 2016., State-of-the-art named entity recognition systems/nrely heavily on hand-crafted features and/ndomain-specific knowledge in order to learn/neffectively from the small, supervised training/ncorpora that are available. In this paper, we/nintroduce two new neural architectures—one/nbased on bidirectional LSTMs and conditional/nrandom fields, and the other that constructs/nand labels segments using a transition-based/napproach inspired by shift-reduce parsers./nOur models rely on two sources of information/nabout words: character-based word/nrepresentations learned from the supervised/ncorpus and unsupervised word representations/nlearned from unannotated corpora. Our/nmodels obtain state-of-the-art performance in/nNER in four languages without resorting to/nany language-specific knowledge or resources/nsuch as gazetteers., This work was sponsored in part by the Defense/nAdvanced Research Projects Agency (DARPA)/nInformation Innovation Office (I2O) under the/nLow Resource Languages for Emergent Incidents/n(LORELEI) program issued by DARPA/I2O under/nContract No. HR0011-15-C-0114. Miguel Ballesteros/nis supported by the European Commission under/nthe contract numbers FP7-ICT-610411 (project/nMULTISENSOR) and H2020-RIA-645012 (project/nKRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Exploiting visual similarities for ontology alignment

  • Doulaverakis, Charalampos
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Ontology alignment is the process where two different ontologies that usually describe similar domains are ’aligned’, i.e. a set of correspondences between their entities, regarding semantic equivalence, is determined. In order to identify these correspondences several methods and metrics that measure semantic equivalence have been proposed in literature. The most common features that these metrics employ are string-, lexical-, structure- and semantic-based similarities for which several approaches have been developed. However, what hasn’t been investigated is the usage of visual-based features for determining entity similarity in cases where images are associated with concepts. Nowadays the existence of several resources (e.g. ImageNet) that map lexical concepts onto images allows for exploiting visual similarities for this purpose. In this paper, a novel approach for ontology matching based on visual similarity is presented. Each ontological entity is associated with sets of images, retrieved through ImageNet or web-based search, and state of the art visual feature extraction, clustering and indexing for computing the similarity between entities is employed. An adaptation of a popular Wordnet-based matching algorithm to exploit the visual similarity is also proposed. Our method is compared with traditional metrics against a standard ontology alignment benchmark dataset and demonstrates promising results., This work was supported by MULTISENSOR (contract no. FP7-610411) and KRISTINA (contract no./nH2020-645012) projects, partially funded by the European Commission.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Retrieval of multimedia objects by fusing multiple modalities

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: ICMR'16. International Conference on Multimedia Retrieval 2016, celebrat a Nova York del 6 al 9 de juny de 2016, Searching for multimedia objects with heterogeneous modalities is critical for the construction of effective multimedia retrieval systems. Towards this direction, we propose a framework for the multimodal fusion of visual and textual similarities, based on visual features, visual concepts and textual concepts. Our method is compared to the baseline method that only fuses two modalities but integrates all early, late, linearly weighted, diffusion and graph-based models in one unifying framework. Our framework integrates more than two modalities and high-level information, so as to retrieve multimedia objects enriched with high-level textual and visual concepts, in response to a multimodal query. The experimental comparison is done under the same memory complexity, in two multimedia collections in the multimedia retrieval task. The results have shown that we outperform the baseline method, in terms of Mean Average Precision., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Hybrid graph-based and non-linear late fusion approach for multimedia retrieval

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 14th International Workshop on Content-Based Multimedia Indexing (CBMI 2016) celebrat del 15 al 17 de juny de 2016 a Bucarest, Romania., Nowadays, multimedia retrieval has become a task of high importance, due to the need for efficient and fast access to very large and heterogeneous multimedia collections. An interesting challenge within the aforementioned task is the efficient combination of different modalities in a multimedia object and especially the fusion between textual and visual information. The fusion of multiple modalities for retrieval in an unsupervised way has been mostly based on early, weighted linear, graph-based and diffusion-based techniques. In contrast, we present a strategy for fusing textual and visual modalities, through the combination of a non-linear fusion model and a graph-based late fusion approach. The fusion strategy is based on the construction of a uniform multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. The proposed late fusion approach is evaluated in the multimedia retrieval task, by applying it to two multimedia collections, namely the WIKI11 and IAPR-TC12. The experimental results indicate its superiority over the baseline method in terms of Mean Average Precision for both considered datasets., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A neural network architecture for multilingual punctuation generation

  • Ballesteros, Miguel
  • Wanner, Leo
Even syntactically correct sentences are perceived as awkward if they do not contain correct punctuation. Still, the problem of automatic generation of punctuation marks has been largely neglected for a long time. We/npresent a novel model that introduces punctuation marks into raw text material with transition-based algorithm using LSTMs. Unlike the state-of-the-art approaches, our model is language-independent and also neutral with respect to the intended use of the punctuation. Multilingual experiments show that it achieves high accuracy on the full range of punctuation marks across languages., This work was supported by the European Commission under the contract numbers FP7-ICT-/n610411 (MULTISENSOR) and H2020-RIA-645012 (KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Visualizing deep-syntactic parser output

  • Soler Company, Juan
  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
Comunicació presentada a la 2015 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), celebrada del 31 de maig al 5 de juny 2015 a Denver (CO, EUA)., “Deep-syntactic” dependency structures bridge the gap between the surface-syntactic structures as produced by state-of-the-art dependency parsers and semantic logical forms in that they abstract away from surfacesyntactic idiosyncrasies, but still keep the linguistic structure of a sentence. They have thus a great potential for such downstream applications as machine translation and summarization. In this demo paper, we propose an online version of a deep-syntactic parser that outputs deep-syntactic structures from plain sentences and visualizes them using the Brat tool. Along with the deep syntactic structures, the user can also inspect the visual presentation of the surface-syntactic structures that serve as input to the deep-syntactic parser and that are produced by the joint tagger and syntactic transition-based parser ran in the pipeline before deep-syntactic parsing takes place., This work has been partially funded by the European Union’s Seventh Framework and Horizon 2020 Research and Innovation Programmes under the Grant Agreement numbers FP7-ICT-610411, FP7-SME- 606163, and H2020-RIA-645012.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Transition-based dependency parsing with stack long short-term memory

  • Dyer, Chris
  • Ballesteros, Miguel
  • Ling, W
  • Matthews, A
  • Smith, Noah A.
We propose a technique for learning representations of parser states in transitionbased dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks— the stack LSTM. Like the conventional stack data structures used in transitionbased parsing, elements can be pushed to/nor popped from the top of the stack in constant time, but, in addition, an LSTM maintains a continuous space embedding of the stack contents. This lets us formulate an efficient parsing model that captures three facets of a parser’s state: (i) unbounded look-ahead into the buffer of incoming words, (ii) the complete history of actions taken by the parser, and (iii) the complete contents of the stack of partially built tree fragments, including their internal/nstructures. Standard backpropagation techniques are used for training and yield state-of-the-art parsing performance., This work was sponsored in part by the U. S. Army Research Laboratory and the U. S. Army Research Office/nunder contract/grant number W911NF-10-1-0533, and in part by NSF CAREER grant IIS-1054319./nMiguel Ballesteros is supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

FORGe at SemEval-2017 Task 9: deep sentence generation based on a sequence of graph transducers

  • Mille, Simon
  • Carlini, Roberto
  • Burga Díaz, Alicia
  • Wanner, Leo
Comunicació presentada a: The 11th International Workshop on Semantic Evaluations (SemEval-2017), celebrada del 3 al 4 d'agost de 2017 a Vancouver, Canadà., We present the contribution of Universitat Pompeu Fabra’s NLP group to the Sem-Eval Task 9.2 (AMR-to-English Generation). The proposed generation pipeline comprises: (i) a series of rule-based graphtransducers for the syntacticization of the input graphs and the resolution of morphological agreements, and (ii) an off-theshelf statistical linearization component., The work described in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411, H2020- 645012-RIA, H2020-700024-RIA, and H2020-DRS-01-2015.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Data-driven sentence generation with non-isomorphic trees

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
Comunicació presentada a la 2015 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), celebrada del 31 de maig al 5 de juny 2015 a Denver (CO, EUA)., Abstract structures from which the generation naturally starts often do not contain any functional nodes, while surface-syntactic structures or a chain of tokens in a linearized tree contain all of them. Therefore, data-driven linguistic generation needs to be able to cope with the projection between non-isomorphic structures that differ in their topology and number of nodes. So far, such a projection has been a challenge in data-driven generation/nand was largely avoided. We present a fully stochastic generator that is able to cope with projection between non-isomorphic structures. The generator, which starts from PropBank-like structures, consists of a cascade/nof SVM-classifier based submodules that map in a series of transitions the input structures onto sentences. The generator has been evaluated for English on the Penn-Treebank and for Spanish on the multi-layered AncoraUPF corpus., Our work on deep stochastic sentence generation is partially supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Extending WordNet with fine-grained collocational information via supervised distributional learning

  • Espinosa-Anke, Luis
  • Camacho-Collados, Jose
  • Rodríguez Fernández, Sara
  • Saggion, Horacio
  • Wanner, Leo
Comunicació presentada a COLING 2016: the 26th International Conference on Computational Linguistics; 2016 Dec. 11-16; Osaka (Japan), WordNet is probably the best known lexical resource in Natural Language Processing. While it/nis widely regarded as a high quality repository of concepts and semantic relations, updating and/nextending it manually is costly. One important type of relation which could potentially add enormous/nvalue toWordNet is the inclusion of collocational information, which is paramount in tasks/nsuch as Machine Translation, Natural Language Generation and Second Language Learning. In/nthis paper, we present ColWordNet (CWN), an extended WordNet version with fine-grained collocational/ninformation, automatically introduced thanks to a method exploiting linear relations/nbetween analogous sense-level embeddings spaces. We perform both intrinsic and extrinsic evaluations,/nand release CWN for the use and scrutiny of the community., This work was partially funded by the following projects: MULTISENSOR (FP7-ICT-610411),/nKRISTINA (H2020-645012-RIA), HARENES (FFI201130219-CO2-02), through a predoctoral grant/n(BES-2012-057036), the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and TUNER/n(TIN2015-65308-C5-5-R, MINECO/FEDER, UE). Jose Camacho-Collados is supported by a Google/nDoctoral Fellowship in Natural Language Processing.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Towards automatic extraction of prosodic patterns for speech synthesis

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Burga Díaz, Alicia
  • Wanner, Leo
Paper presented at: 7th international conference on Speech Prosody (SP2014); 2014 20-23 May; Dublin, Ireland., This paper deals with the adaptation of AuToBI annotation for speech synthesis purposes. AuToBI is a tool that automatically determines and classifies the standard ToBI labels for American English. AuToBI annotation is performed word-by-word. However, for speech synthesis applications that use various layers of linguistic annotation (syntax, semantic information and prosody structures) and, in particular, for the detection of the correlation between the information structure and prosody, a labeling of intonation patterns at the intonational phrase level is essential. We present a rule-based procedure for initial AuToBI annotation and its adaptation a phrase-based annotation, avoiding thus a post-processing stage of the extracted labels. To validate our/nproposal, the outcome of the procedure is compared with manual annotation and with patterns prognosticated by information structure–prosody correlation argued for by main stream theories., Parts of this work have been funded by a grant from the European/nCommission under the contract number FP7-ICT-610411./nThe second author is partially funded by a grant from the Spanish/nMinistry of Economy and Competitivity in the framework/nof the Juan de la Cierva fellowship program (JCI-2012-12272).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Recurrent neural network grammars

  • Dyer, Chris
  • Kuncoro, Adhiguna
  • Ballesteros, Miguel
  • Smith, Noah A.
Comunicació presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 de juny 2016., We introduce recurrent neural network grammars,/nprobabilistic models of sentences with/nexplicit phrase structure. We explain efficient/ninference procedures that allow application to/nboth parsing and language modeling. Experiments/nshow that they provide better parsing in/nEnglish than any single previously published/nsupervised generative model and better language/nmodeling than state-of-the-art sequential/nRNNs in English and Chinese., This work was sponsored in part by the Defense/nAdvanced Research Projects Agency (DARPA)/nInformation Innovation Office (I2O) under the/nLow Resource Languages for Emergent Incidents/n(LORELEI) program issued by DARPA/I2O under/nContract No. HR0011-15-C-0114; it was also supported/nin part by Contract No. W911NF-15-1-0543/nwith the DARPA and the Army Research Office/n(ARO). Approved for public release, distribution/nunlimited. The views expressed are those of the authors/nand do not reflect the official policy or position/nof the Department of Defense or the U.S. Government./nMiguel Ballesteros was supported by the/nEuropean Commission under the contract numbers/nFP7-ICT-610411 (project MULTISENSOR) and/nH2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Query-based topic detection using concepts and named entities

  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 1st International Workshop on Multimodal Media Data Analytics, celebrat juntament amb 22nd European Conference on Artificial Intelligence (ECAI 2016), el 30 d'agost de 2016 a La Haia, Holanda., In this paper, we present a framework for topic detection in news articles. The framework receives as input the results retrieved from a query-based search and clusters them by topic. To this end, the recently introduced “DBSCAN-Martingale” method for automatically estimating the number of topics and the well-established Latent Dirichlet Allocation topic modelling approach for the assignment of news articles into topics of interest, are utilized. Furthermore, the proposed query-based topic detection framework works on high-level textual features (such as concepts and named entities) that are extracted from news articles. Our topic detection approach is tackled as a text clustering task, without knowing the number of clusters and compares favorably to several text clustering approaches, in a public dataset of retrieved results, with respect to four representative queries., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Data-driven deep-syntactic dependency parsing

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
‘Deep-syntactic’ dependency structures that capture the argumentative, attributive and co-/nordinative relations between full words of a sentence have a great potential for a number/nof NLP-applications. The abstraction degree of these structures is in between the output/nof a syntactic dependency parser (connected trees defined over all words of a sentence and/nlanguage-specific grammatical functions) and the output of a semantic parser (forests of trees/ndefined over individual lexemes or phrasal chunks and abstract semantic role labels which/ncapture the frame structures of predicative elements and drop all attributive and coordinative/ndependencies). We propose a parser that provides deep-syntactic structures. The parser has/nbeen tested on Spanish, English and Chinese, The work reported on in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411 (MULTISENSOR) and H2020-645012-RIA (KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Transition-based spinal parsing

  • Ballesteros, Miguel
  • Carreras, Xavier
Comunicació presentada a la 19th Conference on Computational Language Learning, celebrada els dies 30 i 31 de juliol 2015 a Beijing (China) i organitzada per l'ACL Special Interest Group on Natural Language Learning., We present a transition-based arc-eager model to parse spinal trees, a dependencybased representation that includes phrasestructure information in the form of constituent spines assigned to tokens. As a main advantage, the arc-eager model can use a rich set of features combining dependency and constituent information, while parsing in linear time. We describe a set of conditions for the arc-eager system to produce valid spinal structures. In experiments using beam search we show that the model obtains a good trade-off between speed and accuracy, and yields state of the art performance for both dependency and constituent parsing measures., Miguel Ballesteros is supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

  • Vrochidis, Stefanos
  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Casamayor, Gerard
  • Wanner, Leo
  • Heise, Nicolaus
  • Wagner, Tilman
  • Bilous, Andriy
  • Jamin, Emmanuel
  • Simeonov, Boyan
  • Alexiev, Vladimir
  • Busch, Reinhard
  • Arapakis, Ioannis
  • Kompatsiaris, Ioannis
Analysts and journalists face the problem of having to deal with very large, heterogeneous, and multilingual data volumes that need to be analyzed, understood, and aggregated. Automated and simplified editorial and authoring process could significantly reduce time, labor, and costs. Therefore, there is a need for unified access to multilingual and multicultural news story material, beyond the level of a nation, ensuring context-aware, spatiotemporal, and semantic interpretation, correlating also and summarizing the interpreted material into a coherent gist. In this paper, we present a platform integrating multimodal analytics techniques, which are able to support journalists in handling large streams of real-time and diverse information. Specifically, the platform automatically crawls and indexes multilingual and multimedia information from heterogeneous resources. Textual information is automatically summarized and can be translated (on demand) into the language of the journalist. High-level information is extracted from both textual and multimedia content for fast inspection using concept clouds. The textual and multimedia content is semantically integrated and indexed using a common representation, to be accessible through a web-based search engine. The evaluation of the proposed platform was performed by several groups of journalists revealing satisfaction from the user side., This work was xsupported by the EC funded projects MULTISENSOR (FP7-610411) and V4Design (H2020-779962)x.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Fast visual vocabulary construction for image retrieval using skewed-split k-d trees

  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a la 22nd International Conference on MultiMedia Modeling (MMM16), celebrada els dies 4 a 6 de gener 2016 a Miami (FL, EUA)., Most of the image retrieval approaches nowadays are based on the Bag-of-Words (BoW) model, which allows for representing an image efficiently and quickly. The efficiency of the BoW model is related to the efficiency of the visual vocabulary. In general, visual vocabularies are created by clustering all available visual features, formulating specific patterns. Clustering techniques are k-means oriented and they are replaced by approximate k-means methods for very large datasets. In this work, we propose a faster construction of visual vocabularies compared to the existing method in the case of SIFT descriptors, based on our observation that the values of the 128-dimensional SIFT descriptors follow the exponential distribution. The application of our method to image retrieval in specific image datasets showed that the mean Average Precision is not reduced by our approximation, despite that the visual vocabulary has been constructed significantly faster compared to the state of the art methods., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

MULTISENSOR: development of multimedia content integration technologies for journalism, media monitoring and international exporting decision support

  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
  • Casamayor, Gerard
  • Arapakis, Ioannis
  • Busch, Reinhard
  • Alexiev, Vladimir
  • Jamin, Emmanuel
  • Jugov, Michael
  • Heise, Nicolaus
  • Forrellat, Teresa
  • Liparas, Dimitris
  • Wanner, Leo
This paper presents an overview and the first results of the FP7 MULTISENSOR project, which deals with multidimensional content integration of multimedia content for intelligent sentiment enriched and context oriented interpretation. MULTISENSOR aims at providing unified access to multilingual, multimedia and multicultural economic, news story material across borders in order to support journalism and media monitoring tasks and provide decision support for internationalisation of companies., This work was supported by MULTISENSOR project 1 , partially funded by the European Commission, under the contract number FP7-610411.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Distilling an ensemble of greedy dependency parsers into one MST parser

  • Kuncoro, Adhiguna
  • Ballesteros, Miguel
  • Kong, Lingpeng
  • Dyer, Chris
  • Smith, Noah A.
Comunicació presentada a la 2016 Conference on Empirical Methods in Natural Language Processing, celebrada de l'1 al 5 de novembre de 2016 a Austin, Texas., We introduce two first-order graph-based dependency parsers achieving a new state of the art. The first is a consensus parser built from an ensemble of independently trained greedy LSTM transition-based parsers with different random initializations. We cast this approach as minimum Bayes risk decoding (under the Hamming cost) and argue that weaker consensus within the ensemble is a useful signal of difficulty or ambiguity. The second parser is a “distillation” of the ensemble into a single model. We train the distillation parser using a structured hinge loss objective with a novel cost that incorporates ensemble uncertainty estimates for each possible attachment, thereby avoiding the intractable crossentropy computations required by applying standard distillation objectives to problems with structured outputs. The first-order distillation parser matches or surpasses the state of the art on English, Chinese, and German., We thank Swabha Swayamdipta, Sam Thomson, Jesse Dodge, Dallas Card, Yuichiro Sawai, Graham Neubig, and the anonymous reviewers for useful feedback. We also thank Juntao Yu and Bernd Bohnet for re-running the parser of Bohnet and Nivre (2012) on Chinese with gold tags. This work was sponsored in part by the Defense Advanced Research Projects Agency (DARPA) Information Innovation Office (I2O) under the Low Resource Languages for Emergent Incidents (LORELEI) program issued by DARPA/I2O under Contract No. HR0011-15-C-0114; it was also supported in part by Contract No. W911NF-15-1-0543 with the DARPA and the Army Research Office (ARO). Approved for public release, distribution unlimited. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. Miguel Ballesteros was supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Multimedia interactive search engine based on graph-based and non-linear multimodal fusion

  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Mironidis, Theodoros
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 14th International Workshop on Content-Based Multimedia Indexing (CBMI 2016) celebrat del 15 al 17 de juny de 2016 a Bucarest, Romania., This paper presents an interactive multimedia search engine, which is capable of searching into multimedia collections by fusing textual and visual information. Apart from multimedia search, the engine is able to perform text search and image retrieval independently using both high-level and lowlevel information. The images of the multimedia collection are organized by color, offering fast browsing in the image collection., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411), HOMER (FP7-312883) and KRISTINA (H2020-645012).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Improved transition-based parsing by modeling characters instead of words with LSTMs

  • Ballesteros, Miguel
  • Dyer, Chris
  • Smith, Noah A.
Comunicació presentada a la 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), organitzada pel SIGDAT i celebrada els dies 17 a 21 de setembre 2015 a Lisboa (Portugal)., We present extensions to a continuousstate dependency parsing method that makes it applicable to morphologically rich languages. Starting with a highperformance transition-based parser that uses long short-term memory (LSTM) recurrent neural networks to learn representations of the parser state, we replace lookup-based word representations with representations constructed from the orthographic representations of the words, also using LSTMs. This allows statistical sharing across word forms that are similar on the surface. Experiments for morphologically rich languages show that the parsing model benefits from incorporating the character-based encodings of words., MB was supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA). This research was supported by the U.S. Army Research Laboratory and the U.S. Army Research Office under contract/grant number W911NF-10-1-0533 and NSF IIS-1054319. This work was completed while NAS was at CMU.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

The Information structure-prosody interface revisited

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Burga Díaz, Alicia
  • Wanner, Leo
Paper presented at: 7th international conference on Speech Prosody (SP2014); 2014 20-23 May; Dublin, Ireland., Several grammar theories relate information structure and prosody, highlighting a major correspondence between theme and rheme, and intonation patterns. Although these theories have been successfully exploited in some specific speech synthesis applications, they are mainly based on short default-order sentences, which limits their expressiveness for real discourse with longer sentences and complex structures. This paper revises/nthese theories, identifying cases in which they are valid, and providing a new proposal for cases in which a more complex model is needed. Specifically, our experiments performed on real discourse from the Wall Street Journal corpus show that we need a model that: (1) foresees a hierarchical theme/rheme structure, and (2) introduces, apart from the traditional theme and rheme, a new element—the specifier., Parts of this work have been funded by a grant from the European Commission under the contract number FP7-ICT-610411. The second author is partially funded by a grant from the Spanish Ministry of Economy and Competitivity in the framework of the Juan de la Cierva fellowship program (JCI-2012-12272).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Transition-based dependency parsing with heuristic backtracking

  • Buckman, Jacob
  • Ballesteros, Miguel
  • Dyer, Chris
Comunicació presentada a Conference on Empirical Methods in Natural Language Processing, We introduce a novel approach to the decoding problem in transition-based parsing: heuristic backtracking. This algorithm uses a series of partial parses on the sentence to locate the best candidate parse, using confidence estimates/nof transition decisions as a heuristic to guide the starting points of the search. This allows us to achieve a parse accuracy comparable to beam search, despite using fewer transitions. When used to augment a Stack-LSTM transition-based parser, the parser shows an unlabeled attachment score of up to 93.30% for English and 87.61% for Chinese., Miguel Ballesteros was supported by the European Commission under the contract numbers FP7-/nICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A hybrid framework for news clustering based on the DBSCAN-Martingale and LDA

  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
  • Wanner, Leo
Nowadays there is an important need by journalists and media monitoring companies to cluster news in large amounts of web articles, in order to ensure fast access to their topics or events of interest. Our aim in this work is to identify groups of news articles that share a common topic or event, without a priori knowledge of the number of clusters. The estimation of the correct number of topics is a challenging issue, due to the existence of “noise”, i.e. news articles which are irrelevant to all other topics. In this context, we introduce a novel density-based news clustering framework, in which the assignment of news articles to topics is done by the well-established Latent Dirichlet Allocation, but the estimation of the number of clusters is performed by our novel method, called “DBSCAN-Martingale”, which allows for extracting noise from the dataset and progressively extracts clusters from an OPTICS reachability plot. We evaluate our framework and the DBSCAN-Martingale on the 20newsgroups-mini dataset and on 220 web news articles, which are references to specific Wikipedia pages. Among twenty methods for news clustering, without knowing the number of clusters k, the framework of DBSCAN-Martingale provides the correct number of clusters and the highest Normalized Mutual Information., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Transition-based dependency parsing with heuristic backtracking

  • Buckman, Jacob
  • Ballesteros, Miguel
  • Dyer, Chris
Comunicació presentada a Conference on Empirical Methods in Natural Language Processing, We introduce a novel approach to the decoding problem in transition-based parsing: heuristic backtracking. This algorithm uses a series of partial parses on the sentence to locate the best candidate parse, using confidence estimates/nof transition decisions as a heuristic to guide the starting points of the search. This allows us to achieve a parse accuracy comparable to beam search, despite using fewer transitions. When used to augment a Stack-LSTM transition-based parser, the parser shows an unlabeled attachment score of up to 93.30% for English and 87.61% for Chinese., Miguel Ballesteros was supported by the European Commission under the contract numbers FP7-/nICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Transition-based spinal parsing

  • Ballesteros, Miguel
  • Carreras, Xavier
Comunicació presentada a la 19th Conference on Computational Language Learning, celebrada els dies 30 i 31 de juliol 2015 a Beijing (China) i organitzada per l'ACL Special Interest Group on Natural Language Learning., We present a transition-based arc-eager model to parse spinal trees, a dependencybased representation that includes phrasestructure information in the form of constituent spines assigned to tokens. As a main advantage, the arc-eager model can use a rich set of features combining dependency and constituent information, while parsing in linear time. We describe a set of conditions for the arc-eager system to produce valid spinal structures. In experiments using beam search we show that the model obtains a good trade-off between speed and accuracy, and yields state of the art performance for both dependency and constituent parsing measures., Miguel Ballesteros is supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Visualizing deep-syntactic parser output

  • Soler Company, Juan
  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
Comunicació presentada a la 2015 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), celebrada del 31 de maig al 5 de juny 2015 a Denver (CO, EUA)., “Deep-syntactic” dependency structures bridge the gap between the surface-syntactic structures as produced by state-of-the-art dependency parsers and semantic logical forms in that they abstract away from surfacesyntactic idiosyncrasies, but still keep the linguistic structure of a sentence. They have thus a great potential for such downstream applications as machine translation and summarization. In this demo paper, we propose an online version of a deep-syntactic parser that outputs deep-syntactic structures from plain sentences and visualizes them using the Brat tool. Along with the deep syntactic structures, the user can also inspect the visual presentation of the surface-syntactic structures that serve as input to the deep-syntactic parser and that are produced by the joint tagger and syntactic transition-based parser ran in the pipeline before deep-syntactic parsing takes place., This work has been partially funded by the European Union’s Seventh Framework and Horizon 2020 Research and Innovation Programmes under the Grant Agreement numbers FP7-ICT-610411, FP7-SME- 606163, and H2020-RIA-645012.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

MULTISENSOR: development of multimedia content integration technologies for journalism, media monitoring and international exporting decision support

  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
  • Casamayor, Gerard
  • Arapakis, Ioannis
  • Busch, Reinhard
  • Alexiev, Vladimir
  • Jamin, Emmanuel
  • Jugov, Michael
  • Heise, Nicolaus
  • Forrellat, Teresa
  • Liparas, Dimitris
  • Wanner, Leo
This paper presents an overview and the first results of the FP7 MULTISENSOR project, which deals with multidimensional content integration of multimedia content for intelligent sentiment enriched and context oriented interpretation. MULTISENSOR aims at providing unified access to multilingual, multimedia and multicultural economic, news story material across borders in order to support journalism and media monitoring tasks and provide decision support for internationalisation of companies., This work was supported by MULTISENSOR project 1 , partially funded by the European Commission, under the contract number FP7-610411.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A Multimedia interactive search engine based on graph-based and non-linear multimodal fusion

  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Mironidis, Theodoros
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 14th International Workshop on Content-Based Multimedia Indexing (CBMI 2016) celebrat del 15 al 17 de juny de 2016 a Bucarest, Romania., This paper presents an interactive multimedia search engine, which is capable of searching into multimedia collections by fusing textual and visual information. Apart from multimedia search, the engine is able to perform text search and image retrieval independently using both high-level and lowlevel information. The images of the multimedia collection are organized by color, offering fast browsing in the image collection., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411), HOMER (FP7-312883) and KRISTINA (H2020-645012).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Neural architectures for named entity recognition

  • Lample, Guillaume
  • Ballesteros, Miguel
  • Subramanian, Sandeep
  • Kawakami, Kazuya
  • Dyer, Chris
Comunicació presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 de juny 2016., State-of-the-art named entity recognition systems/nrely heavily on hand-crafted features and/ndomain-specific knowledge in order to learn/neffectively from the small, supervised training/ncorpora that are available. In this paper, we/nintroduce two new neural architectures—one/nbased on bidirectional LSTMs and conditional/nrandom fields, and the other that constructs/nand labels segments using a transition-based/napproach inspired by shift-reduce parsers./nOur models rely on two sources of information/nabout words: character-based word/nrepresentations learned from the supervised/ncorpus and unsupervised word representations/nlearned from unannotated corpora. Our/nmodels obtain state-of-the-art performance in/nNER in four languages without resorting to/nany language-specific knowledge or resources/nsuch as gazetteers., This work was sponsored in part by the Defense/nAdvanced Research Projects Agency (DARPA)/nInformation Innovation Office (I2O) under the/nLow Resource Languages for Emergent Incidents/n(LORELEI) program issued by DARPA/I2O under/nContract No. HR0011-15-C-0114. Miguel Ballesteros/nis supported by the European Commission under/nthe contract numbers FP7-ICT-610411 (project/nMULTISENSOR) and H2020-RIA-645012 (project/nKRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

  • Vrochidis, Stefanos
  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Casamayor, Gerard
  • Wanner, Leo
  • Heise, Nicolaus
  • Wagner, Tilman
  • Bilous, Andriy
  • Jamin, Emmanuel
  • Simeonov, Boyan
  • Alexiev, Vladimir
  • Busch, Reinhard
  • Arapakis, Ioannis
  • Kompatsiaris, Ioannis
Analysts and journalists face the problem of having to deal with very large, heterogeneous, and multilingual data volumes that need to be analyzed, understood, and aggregated. Automated and simplified editorial and authoring process could significantly reduce time, labor, and costs. Therefore, there is a need for unified access to multilingual and multicultural news story material, beyond the level of a nation, ensuring context-aware, spatiotemporal, and semantic interpretation, correlating also and summarizing the interpreted material into a coherent gist. In this paper, we present a platform integrating multimodal analytics techniques, which are able to support journalists in handling large streams of real-time and diverse information. Specifically, the platform automatically crawls and indexes multilingual and multimedia information from heterogeneous resources. Textual information is automatically summarized and can be translated (on demand) into the language of the journalist. High-level information is extracted from both textual and multimedia content for fast inspection using concept clouds. The textual and multimedia content is semantically integrated and indexed using a common representation, to be accessible through a web-based search engine. The evaluation of the proposed platform was performed by several groups of journalists revealing satisfaction from the user side., This work was xsupported by the EC funded projects MULTISENSOR (FP7-610411) and V4Design (H2020-779962)x.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Data-driven deep-syntactic dependency parsing

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
‘Deep-syntactic’ dependency structures that capture the argumentative, attributive and co-/nordinative relations between full words of a sentence have a great potential for a number/nof NLP-applications. The abstraction degree of these structures is in between the output/nof a syntactic dependency parser (connected trees defined over all words of a sentence and/nlanguage-specific grammatical functions) and the output of a semantic parser (forests of trees/ndefined over individual lexemes or phrasal chunks and abstract semantic role labels which/ncapture the frame structures of predicative elements and drop all attributive and coordinative/ndependencies). We propose a parser that provides deep-syntactic structures. The parser has/nbeen tested on Spanish, English and Chinese, The work reported on in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411 (MULTISENSOR) and H2020-645012-RIA (KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Transition-based dependency parsing with stack long short-term memory

  • Dyer, Chris
  • Ballesteros, Miguel
  • Ling, W
  • Matthews, A
  • Smith, Noah A.
We propose a technique for learning representations of parser states in transitionbased dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks— the stack LSTM. Like the conventional stack data structures used in transitionbased parsing, elements can be pushed to/nor popped from the top of the stack in constant time, but, in addition, an LSTM maintains a continuous space embedding of the stack contents. This lets us formulate an efficient parsing model that captures three facets of a parser’s state: (i) unbounded look-ahead into the buffer of incoming words, (ii) the complete history of actions taken by the parser, and (iii) the complete contents of the stack of partially built tree fragments, including their internal/nstructures. Standard backpropagation techniques are used for training and yield state-of-the-art parsing performance., This work was sponsored in part by the U. S. Army Research Laboratory and the U. S. Army Research Office/nunder contract/grant number W911NF-10-1-0533, and in part by NSF CAREER grant IIS-1054319./nMiguel Ballesteros is supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A Hybrid graph-based and non-linear late fusion approach for multimedia retrieval

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 14th International Workshop on Content-Based Multimedia Indexing (CBMI 2016) celebrat del 15 al 17 de juny de 2016 a Bucarest, Romania., Nowadays, multimedia retrieval has become a task of high importance, due to the need for efficient and fast access to very large and heterogeneous multimedia collections. An interesting challenge within the aforementioned task is the efficient combination of different modalities in a multimedia object and especially the fusion between textual and visual information. The fusion of multiple modalities for retrieval in an unsupervised way has been mostly based on early, weighted linear, graph-based and diffusion-based techniques. In contrast, we present a strategy for fusing textual and visual modalities, through the combination of a non-linear fusion model and a graph-based late fusion approach. The fusion strategy is based on the construction of a uniform multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. The proposed late fusion approach is evaluated in the multimedia retrieval task, by applying it to two multimedia collections, namely the WIKI11 and IAPR-TC12. The experimental results indicate its superiority over the baseline method in terms of Mean Average Precision for both considered datasets., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Retrieval of multimedia objects by fusing multiple modalities

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: ICMR'16. International Conference on Multimedia Retrieval 2016, celebrat a Nova York del 6 al 9 de juny de 2016, Searching for multimedia objects with heterogeneous modalities is critical for the construction of effective multimedia retrieval systems. Towards this direction, we propose a framework for the multimodal fusion of visual and textual similarities, based on visual features, visual concepts and textual concepts. Our method is compared to the baseline method that only fuses two modalities but integrates all early, late, linearly weighted, diffusion and graph-based models in one unifying framework. Our framework integrates more than two modalities and high-level information, so as to retrieve multimedia objects enriched with high-level textual and visual concepts, in response to a multimodal query. The experimental comparison is done under the same memory complexity, in two multimedia collections in the multimedia retrieval task. The results have shown that we outperform the baseline method, in terms of Mean Average Precision., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Extending WordNet with fine-grained collocational information via supervised distributional learning

  • Espinosa-Anke, Luis
  • Camacho-Collados, Jose
  • Rodríguez Fernández, Sara
  • Saggion, Horacio
  • Wanner, Leo
Comunicació presentada a COLING 2016: the 26th International Conference on Computational Linguistics; 2016 Dec. 11-16; Osaka (Japan), WordNet is probably the best known lexical resource in Natural Language Processing. While it/nis widely regarded as a high quality repository of concepts and semantic relations, updating and/nextending it manually is costly. One important type of relation which could potentially add enormous/nvalue toWordNet is the inclusion of collocational information, which is paramount in tasks/nsuch as Machine Translation, Natural Language Generation and Second Language Learning. In/nthis paper, we present ColWordNet (CWN), an extended WordNet version with fine-grained collocational/ninformation, automatically introduced thanks to a method exploiting linear relations/nbetween analogous sense-level embeddings spaces. We perform both intrinsic and extrinsic evaluations,/nand release CWN for the use and scrutiny of the community., This work was partially funded by the following projects: MULTISENSOR (FP7-ICT-610411),/nKRISTINA (H2020-645012-RIA), HARENES (FFI201130219-CO2-02), through a predoctoral grant/n(BES-2012-057036), the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and TUNER/n(TIN2015-65308-C5-5-R, MINECO/FEDER, UE). Jose Camacho-Collados is supported by a Google/nDoctoral Fellowship in Natural Language Processing.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Recurrent neural network grammars

  • Dyer, Chris
  • Kuncoro, Adhiguna
  • Ballesteros, Miguel
  • Smith, Noah A.
Comunicació presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 de juny 2016., We introduce recurrent neural network grammars,/nprobabilistic models of sentences with/nexplicit phrase structure. We explain efficient/ninference procedures that allow application to/nboth parsing and language modeling. Experiments/nshow that they provide better parsing in/nEnglish than any single previously published/nsupervised generative model and better language/nmodeling than state-of-the-art sequential/nRNNs in English and Chinese., This work was sponsored in part by the Defense/nAdvanced Research Projects Agency (DARPA)/nInformation Innovation Office (I2O) under the/nLow Resource Languages for Emergent Incidents/n(LORELEI) program issued by DARPA/I2O under/nContract No. HR0011-15-C-0114; it was also supported/nin part by Contract No. W911NF-15-1-0543/nwith the DARPA and the Army Research Office/n(ARO). Approved for public release, distribution/nunlimited. The views expressed are those of the authors/nand do not reflect the official policy or position/nof the Department of Defense or the U.S. Government./nMiguel Ballesteros was supported by the/nEuropean Commission under the contract numbers/nFP7-ICT-610411 (project MULTISENSOR) and/nH2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Towards automatic extraction of prosodic patterns for speech synthesis

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Burga Díaz, Alicia
  • Wanner, Leo
Paper presented at: 7th international conference on Speech Prosody (SP2014); 2014 20-23 May; Dublin, Ireland., This paper deals with the adaptation of AuToBI annotation for speech synthesis purposes. AuToBI is a tool that automatically determines and classifies the standard ToBI labels for American English. AuToBI annotation is performed word-by-word. However, for speech synthesis applications that use various layers of linguistic annotation (syntax, semantic information and prosody structures) and, in particular, for the detection of the correlation between the information structure and prosody, a labeling of intonation patterns at the intonational phrase level is essential. We present a rule-based procedure for initial AuToBI annotation and its adaptation a phrase-based annotation, avoiding thus a post-processing stage of the extracted labels. To validate our/nproposal, the outcome of the procedure is compared with manual annotation and with patterns prognosticated by information structure–prosody correlation argued for by main stream theories., Parts of this work have been funded by a grant from the European/nCommission under the contract number FP7-ICT-610411./nThe second author is partially funded by a grant from the Spanish/nMinistry of Economy and Competitivity in the framework/nof the Juan de la Cierva fellowship program (JCI-2012-12272).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Distilling an ensemble of greedy dependency parsers into one MST parser

  • Kuncoro, Adhiguna
  • Ballesteros, Miguel
  • Kong, Lingpeng
  • Dyer, Chris
  • Smith, Noah A.
Comunicació presentada a la 2016 Conference on Empirical Methods in Natural Language Processing, celebrada de l'1 al 5 de novembre de 2016 a Austin, Texas., We introduce two first-order graph-based dependency parsers achieving a new state of the art. The first is a consensus parser built from an ensemble of independently trained greedy LSTM transition-based parsers with different random initializations. We cast this approach as minimum Bayes risk decoding (under the Hamming cost) and argue that weaker consensus within the ensemble is a useful signal of difficulty or ambiguity. The second parser is a “distillation” of the ensemble into a single model. We train the distillation parser using a structured hinge loss objective with a novel cost that incorporates ensemble uncertainty estimates for each possible attachment, thereby avoiding the intractable crossentropy computations required by applying standard distillation objectives to problems with structured outputs. The first-order distillation parser matches or surpasses the state of the art on English, Chinese, and German., We thank Swabha Swayamdipta, Sam Thomson, Jesse Dodge, Dallas Card, Yuichiro Sawai, Graham Neubig, and the anonymous reviewers for useful feedback. We also thank Juntao Yu and Bernd Bohnet for re-running the parser of Bohnet and Nivre (2012) on Chinese with gold tags. This work was sponsored in part by the Defense Advanced Research Projects Agency (DARPA) Information Innovation Office (I2O) under the Low Resource Languages for Emergent Incidents (LORELEI) program issued by DARPA/I2O under Contract No. HR0011-15-C-0114; it was also supported in part by Contract No. W911NF-15-1-0543 with the DARPA and the Army Research Office (ARO). Approved for public release, distribution unlimited. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. Miguel Ballesteros was supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Fast visual vocabulary construction for image retrieval using skewed-split k-d trees

  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a la 22nd International Conference on MultiMedia Modeling (MMM16), celebrada els dies 4 a 6 de gener 2016 a Miami (FL, EUA)., Most of the image retrieval approaches nowadays are based on the Bag-of-Words (BoW) model, which allows for representing an image efficiently and quickly. The efficiency of the BoW model is related to the efficiency of the visual vocabulary. In general, visual vocabularies are created by clustering all available visual features, formulating specific patterns. Clustering techniques are k-means oriented and they are replaced by approximate k-means methods for very large datasets. In this work, we propose a faster construction of visual vocabularies compared to the existing method in the case of SIFT descriptors, based on our observation that the values of the 128-dimensional SIFT descriptors follow the exponential distribution. The application of our method to image retrieval in specific image datasets showed that the mean Average Precision is not reduced by our approximation, despite that the visual vocabulary has been constructed significantly faster compared to the state of the art methods., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Improved transition-based parsing by modeling characters instead of words with LSTMs

  • Ballesteros, Miguel
  • Dyer, Chris
  • Smith, Noah A.
Comunicació presentada a la 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), organitzada pel SIGDAT i celebrada els dies 17 a 21 de setembre 2015 a Lisboa (Portugal)., We present extensions to a continuousstate dependency parsing method that makes it applicable to morphologically rich languages. Starting with a highperformance transition-based parser that uses long short-term memory (LSTM) recurrent neural networks to learn representations of the parser state, we replace lookup-based word representations with representations constructed from the orthographic representations of the words, also using LSTMs. This allows statistical sharing across word forms that are similar on the surface. Experiments for morphologically rich languages show that the parsing model benefits from incorporating the character-based encodings of words., MB was supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA). This research was supported by the U.S. Army Research Laboratory and the U.S. Army Research Office under contract/grant number W911NF-10-1-0533 and NSF IIS-1054319. This work was completed while NAS was at CMU.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The Information structure-prosody interface revisited

  • Domínguez Bajo, Mónica
  • Farrús, Mireia
  • Burga Díaz, Alicia
  • Wanner, Leo
Paper presented at: 7th international conference on Speech Prosody (SP2014); 2014 20-23 May; Dublin, Ireland., Several grammar theories relate information structure and prosody, highlighting a major correspondence between theme and rheme, and intonation patterns. Although these theories have been successfully exploited in some specific speech synthesis applications, they are mainly based on short default-order sentences, which limits their expressiveness for real discourse with longer sentences and complex structures. This paper revises/nthese theories, identifying cases in which they are valid, and providing a new proposal for cases in which a more complex model is needed. Specifically, our experiments performed on real discourse from the Wall Street Journal corpus show that we need a model that: (1) foresees a hierarchical theme/rheme structure, and (2) introduces, apart from the traditional theme and rheme, a new element—the specifier., Parts of this work have been funded by a grant from the European Commission under the contract number FP7-ICT-610411. The second author is partially funded by a grant from the Spanish Ministry of Economy and Competitivity in the framework of the Juan de la Cierva fellowship program (JCI-2012-12272).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Data-driven sentence generation with non-isomorphic trees

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
Comunicació presentada a la 2015 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), celebrada del 31 de maig al 5 de juny 2015 a Denver (CO, EUA)., Abstract structures from which the generation naturally starts often do not contain any functional nodes, while surface-syntactic structures or a chain of tokens in a linearized tree contain all of them. Therefore, data-driven linguistic generation needs to be able to cope with the projection between non-isomorphic structures that differ in their topology and number of nodes. So far, such a projection has been a challenge in data-driven generation/nand was largely avoided. We present a fully stochastic generator that is able to cope with projection between non-isomorphic structures. The generator, which starts from PropBank-like structures, consists of a cascade/nof SVM-classifier based submodules that map in a series of transitions the input structures onto sentences. The generator has been evaluated for English on the Penn-Treebank and for Spanish on the multi-layered AncoraUPF corpus., Our work on deep stochastic sentence generation is partially supported by the European Commission under the contract numbers FP7-ICT-610411 (project MULTISENSOR) and H2020-RIA-645012 (project KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe di erent or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a mul- timodal query, requires the combination of several sources of information in an e cient and scalable way. Towards this direction, we provide a novel unsuper- vised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fu- sion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when com- pared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Query-based topic detection using concepts and named entities

  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Comunicació presentada a: 1st International Workshop on Multimodal Media Data Analytics, celebrat juntament amb 22nd European Conference on Artificial Intelligence (ECAI 2016), el 30 d'agost de 2016 a La Haia, Holanda., In this paper, we present a framework for topic detection in news articles. The framework receives as input the results retrieved from a query-based search and clusters them by topic. To this end, the recently introduced “DBSCAN-Martingale” method for automatically estimating the number of topics and the well-established Latent Dirichlet Allocation topic modelling approach for the assignment of news articles into topics of interest, are utilized. Furthermore, the proposed query-based topic detection framework works on high-level textual features (such as concepts and named entities) that are extracted from news articles. Our topic detection approach is tackled as a text clustering task, without knowing the number of clusters and compares favorably to several text clustering approaches, in a public dataset of retrieved results, with respect to four representative queries., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Exploiting visual similarities for ontology alignment

  • Doulaverakis, Charalampos
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Ontology alignment is the process where two different ontologies that usually describe similar domains are ’aligned’, i.e. a set of correspondences between their entities, regarding semantic equivalence, is determined. In order to identify these correspondences several methods and metrics that measure semantic equivalence have been proposed in literature. The most common features that these metrics employ are string-, lexical-, structure- and semantic-based similarities for which several approaches have been developed. However, what hasn’t been investigated is the usage of visual-based features for determining entity similarity in cases where images are associated with concepts. Nowadays the existence of several resources (e.g. ImageNet) that map lexical concepts onto images allows for exploiting visual similarities for this purpose. In this paper, a novel approach for ontology matching based on visual similarity is presented. Each ontological entity is associated with sets of images, retrieved through ImageNet or web-based search, and state of the art visual feature extraction, clustering and indexing for computing the similarity between entities is employed. An adaptation of a popular Wordnet-based matching algorithm to exploit the visual similarity is also proposed. Our method is compared with traditional metrics against a standard ontology alignment benchmark dataset and demonstrates promising results., This work was supported by MULTISENSOR (contract no. FP7-610411) and KRISTINA (contract no./nH2020-645012) projects, partially funded by the European Commission.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

FORGe at SemEval-2017 Task 9: deep sentence generation based on a sequence of graph transducers

  • Mille, Simon
  • Carlini, Roberto
  • Burga Díaz, Alicia
  • Wanner, Leo
Comunicació presentada a: The 11th International Workshop on Semantic Evaluations (SemEval-2017), celebrada del 3 al 4 d'agost de 2017 a Vancouver, Canadà., We present the contribution of Universitat Pompeu Fabra’s NLP group to the Sem-Eval Task 9.2 (AMR-to-English Generation). The proposed generation pipeline comprises: (i) a series of rule-based graphtransducers for the syntacticization of the input graphs and the resolution of morphological agreements, and (ii) an off-theshelf statistical linearization component., The work described in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411, H2020- 645012-RIA, H2020-700024-RIA, and H2020-DRS-01-2015.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A neural network architecture for multilingual punctuation generation

  • Ballesteros, Miguel
  • Wanner, Leo
Even syntactically correct sentences are perceived as awkward if they do not contain correct punctuation. Still, the problem of automatic generation of punctuation marks has been largely neglected for a long time. We/npresent a novel model that introduces punctuation marks into raw text material with transition-based algorithm using LSTMs. Unlike the state-of-the-art approaches, our model is language-independent and also neutral with respect to the intended use of the punctuation. Multilingual experiments show that it achieves high accuracy on the full range of punctuation marks across languages., This work was supported by the European Commission under the contract numbers FP7-ICT-/n610411 (MULTISENSOR) and H2020-RIA-645012 (KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A hybrid framework for news clustering based on the DBSCAN-Martingale and LDA

  • Gialampoukidis, Ilias
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
  • Wanner, Leo
Nowadays there is an important need by journalists and media monitoring companies to cluster news in large amounts of web articles, in order to ensure fast access to their topics or events of interest. Our aim in this work is to identify groups of news articles that share a common topic or event, without a priori knowledge of the number of clusters. The estimation of the correct number of topics is a challenging issue, due to the existence of “noise”, i.e. news articles which are irrelevant to all other topics. In this context, we introduce a novel density-based news clustering framework, in which the assignment of news articles to topics is done by the well-established Latent Dirichlet Allocation, but the estimation of the number of clusters is performed by our novel method, called “DBSCAN-Martingale”, which allows for extracting noise from the dataset and progressively extracts clusters from an OPTICS reachability plot. We evaluate our framework and the DBSCAN-Martingale on the 20newsgroups-mini dataset and on 220 web news articles, which are references to specific Wikipedia pages. Among twenty methods for news clustering, without knowing the number of clusters k, the framework of DBSCAN-Martingale provides the correct number of clusters and the highest Normalized Mutual Information., This work was supported by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012), funded by the European Commission.