Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe di erent or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a mul- timodal query, requires the combination of several sources of information in an e cient and scalable way. Towards this direction, we provide a novel unsuper- vised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fu- sion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when com- pared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Data-driven deep-syntactic dependency parsing

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
‘Deep-syntactic’ dependency structures that capture the argumentative, attributive and co-/nordinative relations between full words of a sentence have a great potential for a number/nof NLP-applications. The abstraction degree of these structures is in between the output/nof a syntactic dependency parser (connected trees defined over all words of a sentence and/nlanguage-specific grammatical functions) and the output of a semantic parser (forests of trees/ndefined over individual lexemes or phrasal chunks and abstract semantic role labels which/ncapture the frame structures of predicative elements and drop all attributive and coordinative/ndependencies). We propose a parser that provides deep-syntactic structures. The parser has/nbeen tested on Spanish, English and Chinese, The work reported on in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411 (MULTISENSOR) and H2020-645012-RIA (KRISTINA).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

  • Vrochidis, Stefanos
  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Casamayor, Gerard
  • Wanner, Leo
  • Heise, Nicolaus
  • Wagner, Tilman
  • Bilous, Andriy
  • Jamin, Emmanuel
  • Simeonov, Boyan
  • Alexiev, Vladimir
  • Busch, Reinhard
  • Arapakis, Ioannis
  • Kompatsiaris, Ioannis
Analysts and journalists face the problem of having to deal with very large, heterogeneous, and multilingual data volumes that need to be analyzed, understood, and aggregated. Automated and simplified editorial and authoring process could significantly reduce time, labor, and costs. Therefore, there is a need for unified access to multilingual and multicultural news story material, beyond the level of a nation, ensuring context-aware, spatiotemporal, and semantic interpretation, correlating also and summarizing the interpreted material into a coherent gist. In this paper, we present a platform integrating multimodal analytics techniques, which are able to support journalists in handling large streams of real-time and diverse information. Specifically, the platform automatically crawls and indexes multilingual and multimedia information from heterogeneous resources. Textual information is automatically summarized and can be translated (on demand) into the language of the journalist. High-level information is extracted from both textual and multimedia content for fast inspection using concept clouds. The textual and multimedia content is semantically integrated and indexed using a common representation, to be accessible through a web-based search engine. The evaluation of the proposed platform was performed by several groups of journalists revealing satisfaction from the user side., This work was xsupported by the EC funded projects MULTISENSOR (FP7-610411) and V4Design (H2020-779962)x.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A multimodal analytics platform for journalists analyzing large-scale, heterogeneous multilingual, and multimedia content

  • Vrochidis, Stefanos
  • Moumtzidou, Anastasia
  • Gialampoukidis, Ilias
  • Liparas, Dimitris
  • Casamayor, Gerard
  • Wanner, Leo
  • Heise, Nicolaus
  • Wagner, Tilman
  • Bilous, Andriy
  • Jamin, Emmanuel
  • Simeonov, Boyan
  • Alexiev, Vladimir
  • Busch, Reinhard
  • Arapakis, Ioannis
  • Kompatsiaris, Ioannis
Analysts and journalists face the problem of having to deal with very large, heterogeneous, and multilingual data volumes that need to be analyzed, understood, and aggregated. Automated and simplified editorial and authoring process could significantly reduce time, labor, and costs. Therefore, there is a need for unified access to multilingual and multicultural news story material, beyond the level of a nation, ensuring context-aware, spatiotemporal, and semantic interpretation, correlating also and summarizing the interpreted material into a coherent gist. In this paper, we present a platform integrating multimodal analytics techniques, which are able to support journalists in handling large streams of real-time and diverse information. Specifically, the platform automatically crawls and indexes multilingual and multimedia information from heterogeneous resources. Textual information is automatically summarized and can be translated (on demand) into the language of the journalist. High-level information is extracted from both textual and multimedia content for fast inspection using concept clouds. The textual and multimedia content is semantically integrated and indexed using a common representation, to be accessible through a web-based search engine. The evaluation of the proposed platform was performed by several groups of journalists revealing satisfaction from the user side., This work was xsupported by the EC funded projects MULTISENSOR (FP7-610411) and V4Design (H2020-779962)x.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Data-driven deep-syntactic dependency parsing

  • Ballesteros, Miguel
  • Bohnet, Bernd
  • Mille, Simon
  • Wanner, Leo
‘Deep-syntactic’ dependency structures that capture the argumentative, attributive and co-/nordinative relations between full words of a sentence have a great potential for a number/nof NLP-applications. The abstraction degree of these structures is in between the output/nof a syntactic dependency parser (connected trees defined over all words of a sentence and/nlanguage-specific grammatical functions) and the output of a semantic parser (forests of trees/ndefined over individual lexemes or phrasal chunks and abstract semantic role labels which/ncapture the frame structures of predicative elements and drop all attributive and coordinative/ndependencies). We propose a parser that provides deep-syntactic structures. The parser has/nbeen tested on Spanish, English and Chinese, The work reported on in this paper has been partially funded by the European Commission under the contract numbers FP7-ICT-610411 (MULTISENSOR) and H2020-645012-RIA (KRISTINA).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Multimedia retrieval based on non-linear graph-based fusion and partial least squares regression

  • Gialampoukidis, Ilias
  • Moumtzidou, Anastasia
  • Liparas, Dimitris
  • Tsikrika, Theodora
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Heterogeneous sources of information, such as images, videos, text and metadata are often used to describe di erent or complementary views of the same multimedia object, especially in the online news domain and in large annotated image collections. The retrieval of multimedia objects, given a mul- timodal query, requires the combination of several sources of information in an e cient and scalable way. Towards this direction, we provide a novel unsuper- vised framework for multimodal fusion of visual and textual similarities, which are based on visual features, visual concepts and textual metadata, integrating non-linear graph-based fusion and Partial Least Squares Regression. The fu- sion strategy is based on the construction of a multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. Our framework can employ more than two modalities and high-level information, without increase in memory complexity, when com- pared to state-of-the-art baseline methods. The experimental comparison is done in three public multimedia collections in the multimedia retrieval task. The results have shown that the proposed method outperforms the baseline methods, in terms of Mean Average Precision and Precision@20., This work was partially supported by the European Commission by the projects MULTISENSOR (FP7-610411) and KRISTINA (H2020-645012).