Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

SICK through the SemEval glasses: lesson learned from the evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment

  • Bentivogli, Luisa
  • Bernardi, Raffaella
  • Marelli, Marco
  • Menini, Stefano
  • Baroni, Marco
  • Zamparelli, Roberto
This paper is an extended description of SemEval-2014 Task 1, the task on the evaluation of Compositional Distributional Semantics Models on full sentences. Systems participating in the task were presented with pairs of sentences and were evaluated on their ability to predict human judgments on (1) semantic relatedness and (2) entailment. Training and testing data were subsets of the SICK (Sentences Involving Compositional Knowledge) data set. SICK was developed with the aim of providing a proper benchmark to evaluate compositional semantic systems, though task participation was open to systems based on any approach. Taking advantage of the SemEval experience, in this paper we analyze the SICK data set, in order to evaluate the extent to which it meets its design goal and to shed light on the linguistic phenomena that are still challenging for state-of-the-art computational semantic systems. Qualitative and quantitative error analyses show that many systems are quite sensitive to changes in the proportion of sentence pair types, and degrade in the presence of additional lexico-syntactic complexities which do not affect human judgements. More compositional systems seem to perform better when the task proportions are changed, but the effect needs further confirmation., We thank the creators of the ImageFlickr, MSR-Video, and SemEval-2012 STS data sets for granting us permission to use their data for the task. The University of Trento authors were supported by ERC 2011 Starting Independent Research Grant No. 283554 (COMPOSES).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

When the whole is less than the sum of its parts: how composition affects PMI values in distributional semantic vectors

  • Paperno, Denis
  • Baroni, Marco
Distributional semantic models, deriving vector-based word representations from patterns of word usage in corpora, have many useful applications (Turney and Pantel 2010). Recently, there has been interest in compositional distributional models, which derive vectors for phrases from representations of their constituent words (Mitchell and Lapata 2010). Often, the values of distributional vectors are pointwise mutual information (PMI) scores obtained from raw co-occurrence counts. In this article we study the relation between the PMI dimensions of a phrase vector and its components in order to gain insights into which operations an adequate composition model should perform. We show mathematically that the difference between the PMI dimension of a phrase vector and the sum of PMIs in the corresponding dimensions of the phrase's parts is an independently interpretable value, namely, a quantification of the impact of the context associated with the relevant dimension on the phrase's internal cohesion, as also measured by PMI. We then explore this quantity empirically, through an analysis of adjective–noun composition., We would like to thank the Computational Linguistics editor and reviewers: Yoav Goldberg, Omer Levy, Katya Tentori, Germán Kruszewski, Nghia Pham, and the other members of the Composes team for useful feedback. Our work is funded by ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Formal Distributional Semantics: Introduction to the Special Issue

  • Boleda, Gemma
  • Herbelot, Aurélie
Formal Semantics and Distributional Semantics are two very influential semantic frameworks in Computational Linguistics. Formal Semantics is based on a symbolic tradition and centered around the inferential properties of language. Distributional Semantics is statistical and data-driven, and focuses on aspects of meaning related to descriptive content. The two frameworks are complementary in their strengths, and this has motivated interest in combining them into an overarching semantic framework: a “Formal Distributional Semantics.” Given the fundamentally different natures of the two paradigms, however, building an integrative framework poses significant theoretical and engineering challenges. The present issue of Computational Linguistics advances the state of the art in Formal Distributional Semantics; this introductory article explains the motivation behind it and summarizes the contributions of previous work on the topic, providing the necessary background for the articles that follow., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no 655577 (LOVe) as well as the ERC 2011 Starting Independent Research Grant no. 283554 (COMPOSES).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Multimodal word meaning induction from minimal exposure to natural text

  • Lazaridou, Angeliki
  • Marelli, Marco
  • Baroni, Marco
By the time they reach early adulthood, English speakers are familiar with the meaning of thousands of words. In the last decades, computational simulations known as distributional semantic models (DSMs) have demonstrated that it is possible to induce word meaning representations solely from word co‐occurrence statistics extracted from a large amount of text. However, while these models learn in batch mode from large corpora, human word learning proceeds incrementally after minimal exposure to new words. In this study, we run a set of experiments investigating whether minimal distributional evidence from very short passages suffices to trigger successful word learning in subjects, testing their linguistic and visual intuitions about the concepts associated with new words. After confirming that subjects are indeed very efficient distributional learners even from small amounts of evidence, we test a DSM on the same multimodal task, finding that it behaves in a remarkable human‐like way. We conclude that DSMs provide a convincing computational account of word learning even at the early stages in which a word is first encountered, and the way they build meaning representations can offer new insights into human language acquisition., We thank the Cognitive Science editor and reviewers for constructive criticism. We also received useful feedback from the audience at *SEM 2015 and the International Meeting of the Psychonomic Society 2016. We acknowledge ERC 2011 Starting Independent Research Grant number 283554 (COMPOSES project). Marco Marelli conducted most of the work reported in this article while employed by the University of Trento. All authors equally contributed to the reported work.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

There is no logical negation here, but there are alternatives: modeling conversational negation with distributional semantics

  • Kruszewski, German
  • Paperno, Denis
  • Bernardi, Raffaella
  • Baroni, Marco
Logical negation is a challenge for distributional semantics, because predicates and their negations tend to occur in very similar contexts, and consequently their distributional vectors are very similar. Indeed, it is not even clear what properties a “negated” distributional vector should possess. However, when linguistic negation is considered in its actual discourse usage, it often performs a role that is quite different from straightforward logical negation. If someone states, in the middle of a conversation, that “This is not a dog,” the negation strongly suggests a restricted set of alternative predicates that might hold true of the object being talked about. In particular, other canids and middle-sized mammals are plausible alternatives, birds are less likely, skyscrapers and other large buildings virtually impossible. Conversational negation acts like a graded similarity function, of the sort that distributional semantics might be good at capturing. In this article, we introduce a large data set of alternative plausibility ratings for conversationally negated nominal predicates, and we show that simple similarity in distributional semantic space provides an excellent fit to subject data. On the one hand, this fills a gap in the literature on conversational negation, proposing distributional semantics as the right tool to make explicit predictions about potential alternatives of negated predicates. On the other hand, the results suggest that negation, when addressed from a broader pragmatic perspective, far from being a nuisance, is an ideal application domain for distributional semantic methods., Jacopo Romoli and Roberto Zamparelli first made us aware of the possible link between alternative semantics and DS. We got the inspiration to work on whether DS can capture alternatives under conversational negation from an informal talk given by Mark Steedman at the 2013 Dagstuhl Seminar on Computational Models of Language Meaning in Context, and from some remarks made by Hans Kamp in the same occasion. Hinrich Schütze first introduced the idea of dynamic DS at the same seminar. We thank Uri Hasson and Raquel Fernandez for important bibliographic advice. We had many illuminating discussions on related topics with Roberto Zamparelli, Gemma Boleda, Nick Asher, the Rovereto Composers, and the members of the intercontinental FLOSS reading group. Finally, we thank the reviewers for helpful comments. Our work is funded by ERC 2011 Starting Independent Research Grant no. 283554 (COMPOSES).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Multimodal word meaning induction from minimal exposure to natural text

  • Lazaridou, Angeliki
  • Marelli, Marco
  • Baroni, Marco
By the time they reach early adulthood, English speakers are familiar with the meaning of thousands of words. In the last decades, computational simulations known as distributional semantic models (DSMs) have demonstrated that it is possible to induce word meaning representations solely from word co‐occurrence statistics extracted from a large amount of text. However, while these models learn in batch mode from large corpora, human word learning proceeds incrementally after minimal exposure to new words. In this study, we run a set of experiments investigating whether minimal distributional evidence from very short passages suffices to trigger successful word learning in subjects, testing their linguistic and visual intuitions about the concepts associated with new words. After confirming that subjects are indeed very efficient distributional learners even from small amounts of evidence, we test a DSM on the same multimodal task, finding that it behaves in a remarkable human‐like way. We conclude that DSMs provide a convincing computational account of word learning even at the early stages in which a word is first encountered, and the way they build meaning representations can offer new insights into human language acquisition., We thank the Cognitive Science editor and reviewers for constructive criticism. We also received useful feedback from the audience at *SEM 2015 and the International Meeting of the Psychonomic Society 2016. We acknowledge ERC 2011 Starting Independent Research Grant number 283554 (COMPOSES project). Marco Marelli conducted most of the work reported in this article while employed by the University of Trento. All authors equally contributed to the reported work.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

SICK through the SemEval glasses: lesson learned from the evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment

  • Bentivogli, Luisa
  • Bernardi, Raffaella
  • Marelli, Marco
  • Menini, Stefano
  • Baroni, Marco
  • Zamparelli, Roberto
This paper is an extended description of SemEval-2014 Task 1, the task on the evaluation of Compositional Distributional Semantics Models on full sentences. Systems participating in the task were presented with pairs of sentences and were evaluated on their ability to predict human judgments on (1) semantic relatedness and (2) entailment. Training and testing data were subsets of the SICK (Sentences Involving Compositional Knowledge) data set. SICK was developed with the aim of providing a proper benchmark to evaluate compositional semantic systems, though task participation was open to systems based on any approach. Taking advantage of the SemEval experience, in this paper we analyze the SICK data set, in order to evaluate the extent to which it meets its design goal and to shed light on the linguistic phenomena that are still challenging for state-of-the-art computational semantic systems. Qualitative and quantitative error analyses show that many systems are quite sensitive to changes in the proportion of sentence pair types, and degrade in the presence of additional lexico-syntactic complexities which do not affect human judgements. More compositional systems seem to perform better when the task proportions are changed, but the effect needs further confirmation., We thank the creators of the ImageFlickr, MSR-Video, and SemEval-2012 STS data sets for granting us permission to use their data for the task. The University of Trento authors were supported by ERC 2011 Starting Independent Research Grant No. 283554 (COMPOSES).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Formal Distributional Semantics: Introduction to the Special Issue

  • Boleda, Gemma
  • Herbelot, Aurélie
Formal Semantics and Distributional Semantics are two very influential semantic frameworks in Computational Linguistics. Formal Semantics is based on a symbolic tradition and centered around the inferential properties of language. Distributional Semantics is statistical and data-driven, and focuses on aspects of meaning related to descriptive content. The two frameworks are complementary in their strengths, and this has motivated interest in combining them into an overarching semantic framework: a “Formal Distributional Semantics.” Given the fundamentally different natures of the two paradigms, however, building an integrative framework poses significant theoretical and engineering challenges. The present issue of Computational Linguistics advances the state of the art in Formal Distributional Semantics; this introductory article explains the motivation behind it and summarizes the contributions of previous work on the topic, providing the necessary background for the articles that follow., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no 655577 (LOVe) as well as the ERC 2011 Starting Independent Research Grant no. 283554 (COMPOSES).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

When the whole is less than the sum of its parts: how composition affects PMI values in distributional semantic vectors

  • Paperno, Denis
  • Baroni, Marco
Distributional semantic models, deriving vector-based word representations from patterns of word usage in corpora, have many useful applications (Turney and Pantel 2010). Recently, there has been interest in compositional distributional models, which derive vectors for phrases from representations of their constituent words (Mitchell and Lapata 2010). Often, the values of distributional vectors are pointwise mutual information (PMI) scores obtained from raw co-occurrence counts. In this article we study the relation between the PMI dimensions of a phrase vector and its components in order to gain insights into which operations an adequate composition model should perform. We show mathematically that the difference between the PMI dimension of a phrase vector and the sum of PMIs in the corresponding dimensions of the phrase's parts is an independently interpretable value, namely, a quantification of the impact of the context associated with the relevant dimension on the phrase's internal cohesion, as also measured by PMI. We then explore this quantity empirically, through an analysis of adjective–noun composition., We would like to thank the Computational Linguistics editor and reviewers: Yoav Goldberg, Omer Levy, Katya Tentori, Germán Kruszewski, Nghia Pham, and the other members of the Composes team for useful feedback. Our work is funded by ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

There is no logical negation here, but there are alternatives: modeling conversational negation with distributional semantics

  • Kruszewski, German
  • Paperno, Denis
  • Bernardi, Raffaella
  • Baroni, Marco
Logical negation is a challenge for distributional semantics, because predicates and their negations tend to occur in very similar contexts, and consequently their distributional vectors are very similar. Indeed, it is not even clear what properties a “negated” distributional vector should possess. However, when linguistic negation is considered in its actual discourse usage, it often performs a role that is quite different from straightforward logical negation. If someone states, in the middle of a conversation, that “This is not a dog,” the negation strongly suggests a restricted set of alternative predicates that might hold true of the object being talked about. In particular, other canids and middle-sized mammals are plausible alternatives, birds are less likely, skyscrapers and other large buildings virtually impossible. Conversational negation acts like a graded similarity function, of the sort that distributional semantics might be good at capturing. In this article, we introduce a large data set of alternative plausibility ratings for conversationally negated nominal predicates, and we show that simple similarity in distributional semantic space provides an excellent fit to subject data. On the one hand, this fills a gap in the literature on conversational negation, proposing distributional semantics as the right tool to make explicit predictions about potential alternatives of negated predicates. On the other hand, the results suggest that negation, when addressed from a broader pragmatic perspective, far from being a nuisance, is an ideal application domain for distributional semantic methods., Jacopo Romoli and Roberto Zamparelli first made us aware of the possible link between alternative semantics and DS. We got the inspiration to work on whether DS can capture alternatives under conversational negation from an informal talk given by Mark Steedman at the 2013 Dagstuhl Seminar on Computational Models of Language Meaning in Context, and from some remarks made by Hans Kamp in the same occasion. Hinrich Schütze first introduced the idea of dynamic DS at the same seminar. We thank Uri Hasson and Raquel Fernandez for important bibliographic advice. We had many illuminating discussions on related topics with Roberto Zamparelli, Gemma Boleda, Nick Asher, the Rovereto Composers, and the members of the intercontinental FLOSS reading group. Finally, we thank the reviewers for helpful comments. Our work is funded by ERC 2011 Starting Independent Research Grant no. 283554 (COMPOSES).