Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 86
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Attentional pragmatics: a pragmatic approach to exhaustivity

  • Westera, Matthijs
The neo-Gricean approach to exhaustivity is based on the idea that exhaustivity arises when relevant propositions are not asserted. This paper presents a new pragmatic approach based on the idea that exhaustivity arises when relevant propositions are not mentioned, or more precisely, when the speaker did not intend to draw attention to them. This seemingly subtle shift from information to attention results in different predictions on a range of challenges for the neo-Gricean approach, some of which have been brought up in support of the grammatical approach to exhaustivity. This paper discusses three such challenges: exhaustivity on the hints of a quizmaster, exhaustivity on questions, and exhaustivity without an opinionatedness assumption. The two pragmatic approaches are compared on these puzzles along with the grammatical approach.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Similarity or deeper understanding?: analyzing the TED-Q dataset of evoked questions

  • Westera, Matthijs
  • Amidei, Jacopo
  • Mayol, Laia
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We take a close look at a recent dataset of TED-talks annotated with the questions they implicitly evoke, TED-Q (Westera et al., 2020). We test to what extent the relation between a discourse and the questions it evokes is merely one of similarity or association, as opposed to deeper semantic/pragmatic interpretation. We do so by turning the TED-Q dataset into a binary classification task, constructing an analogous task from explicit questions we extract from the BookCorpus (Zhu et al., 2015), and fitting a BERT-based classifier alongside models based on different notions of similarity. The BERT-based classifier, achieving close to human performance, outperforms all similarity-based models, suggesting that there is more to identifying true evoked questions than plain similarity., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Living a discrete life in a continuous world: reference with distributed representations

  • Boleda, Gemma
  • Padó, Sebastian
  • Pham, Nghia The
  • Baroni, Marco
Comunicació presentada a IWCS 2017, celebrat a Montpellier (França) del 19 al 22 de setembre de 2017., Reference is the crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. We propose a fully data-driven, end-toend trainable model that, while operating on continuous multimodal representations, learns to organize them into a discrete-like entity library. We also introduce a referential task to test it, cross-modal tracking. Our model beats standard neural network architectures, but is outperformed by some parametrizations of Memory Networks, another model with external memory., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE); Marie Sklodowska-Curie grant agreement No 655577 (LOVe); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Modeling word interpretation with deep language models: the interaction between expectations and lexical information

  • Aina, Laura
  • Brochhagen, Thomas
  • Boleda, Gemma
How a word is interpreted depends on the context it appears in. We study word interpretation leveraging deep language models, tracing the contribution and interaction of two sources of information that have been shown to be central to it: contextinvariant lexical information, represented by the word embeddings of a model, and a listener’s contextual expectations, represented by its predictions. We define operations to combine these components to obtain representations of word interpretations. We instantiate our framework using two English language models, and evaluate the resulting representations in the extent by which they reflect contextual word substitutes provided by human subjects. Our results suggest that both lexi cal information and expectations codify information pivotal to word interpretation; however, their combination is better than either on its own. Moreover, the division of labor between expectations and the lexicon appears to change across contexts, This project has re ceived funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and inno vation programme (grant agreement No 715154), and from the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research, and the computer resources at CTE-POWER and the technical support pro vided by Barcelona Supercomputing Center (RES-IM2019-3-0006). This paper reflects the authors view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Idioms and the syntax/semantics interface of descriptive content vs. reference

  • McNally, Louise, 1965-
  • Gehrke, Berit
The syntactic literature on idioms contains some proposals that are surprising from a compositional perspective. For example, Sportiche (2005) and Cecchetto and Donati (2015) propose that, in the case of verb-object idioms, the verb combines directly with the noun inside its DP complement, and the determiner is introduced higher up in the syntactic structure, or is late-adjoined. This seems to violate compositionality insofar as it is generally assumed that the semantic role of the determiner is to convert a noun to the appropriate semantic type to serve as the argument to the function denoted by the verb. In this paper, we establish a connection between this line of analysis and lines of work in semantics that have developed outside of the domain of idioms, particularly work on incorporation and a mixed formal and distributional semantic model developed in McNally (2017); McNally and Boleda (2017). This semantic work separates the composition of descriptive content from that of discourse referent introducing material; our proposal shows that this separation offers a particularly promising way to handle the compositional difficulties posed by idioms, including certain patterns of variation in intervening determiners and modifiers., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No 715154), grant FFI2016-76045-P (AEI/MINEICO/FEDER, UE), a Salvador de Madariaga mobility grant from the Spanish Ministry of Education, Culture and Sports, an ICREA Academia award, and the Alexander von Humboldt Foundation.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Controlled tasks for model analysis: retrieving discrete information from sequences

  • Sorodoc, Ionut-Teodor
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada al 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP celebrat l'11 de novembre de 2021 de manera virtual., In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

​What do entity-centric models learn? Insights from entity linking in multi-party dialogue

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why. In this paper we analyze the behavior of two recently proposed entity-centric models in a referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entitycentric naturally fosters good architectural decisions. However, we also show that these models do not really build entity representations and that they make poor use of linguistic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when deployed., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Deep daxes: mutual exclusivity arises through both learning biases and pragmatic strategies in neural networks

  • Gulordava, Kristina
  • Brochhagen, Thomas
  • Boleda, Gemma
Children’s tendency to associate novel words with novel referents has been taken to reflect a bias toward mutual exclusivity. This tendency may be advantageous both as (1) an ad-hoc referent selection heuristic to single out referents lacking a label and as (2) an organizing principle of lexical acquisition. This paper investigates under which circumstances cross-situational neural models can come to exhibit analogous behavior to children, focusing on these two possibilities and their interaction. To this end, we evaluate neural networks’ on both symbolic data and, as a first, on large-scale image data. We find that constraints in both learning and selection can foster mutual exclusivity, as long as they put words in competition for lexical meaning. For computational models, these findings clarify the role of available options for better performance in tasks where mutual exclusivity is advantageous. For cognitive research, they highlight latent interactions between word learning, referent selection mechanisms, and the structure of stimuli of varying complexity: symbolic and visual, This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramon´y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM-2019-3-0006). We are grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

When do languages use the same word for different meanings? The Goldilocks principle in colexification

  • Brochhagen, Thomas
  • Boleda, Gemma
Lexical ambiguity is pervasive in language, and often systematic. For instance, the Spanish word dedo can refer to a toe or a finger, that is, these two meanings colexify in Spanish; and they do so as well in over one hundred other languages. Previous work shows that related meanings are more likely to colexify. This is attributed to cognitive pressure towards simplicity in language, as it makes lexicons easier to learn and use. The present study examines the interplay between this pressure and the competing pressure for languages to support accurate information transfer. We hypothesize that colexification follows a Goldilocks principle that balances the two pressures: meanings are more likely to attach to the same word when they are related to an optimal degree—neither too much, nor too little. We find support for this principle in data from over 1200 languages and 1400 meanings. Our results thus suggest that universal forces shape the lexicons of natural languages. More broadly, they contribute to the growing body of evidence suggesting that languages evolve to strike a balance between competing functional and cognitive pressures., This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Grounding semantic roles in images

  • Silberer, Carina
  • Pinkal, Manfred
Comunicació presentada a la 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), celebrada del 31 d'octubre al 4 de novembre de 2018 a Brussel·les, Bèlgica., We address the task of visual semantic role labeling (vSRL), the identification of the participants of a situation or event in a visual scene, and their labeling with their semantic relations to the event or situation. We render candidate participants as image regions of objects, and train a model which learns to ground roles in the regions which depict the corresponding participant. Experimental results demonstrate that we can train a vSRL model without reliance on prohibitive image-based role annotations, by utilizing noisy data which we extract automatically from image captions using a linguistic SRL system. Furthermore, our model induces frame–semantic visual representations, and their comparison to previous work on supervised visual verb sense disambiguation yields overall better results., We thank the anonymous reviewers, Leonie Harter, Christine Sch¨afer, Michael Roth, Gemma Boleda, Anna Rohrbach and Bernt Schiele. This research was supported by the German Research Foundation (DFG EXC 285), by the European Research Council (ERC Horizon 2020 grant agreement No 715154), and the Spanish Ramon y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Effects of task and visual context on referring expressions using natural scenes

  • Mädebach, Andreas
  • Torubarova, Ekaterina
  • Gualdoni, Eleonora
  • Boleda, Gemma
We explore contextual adaptation of referring expressions with respect to referential ambiguity and communicative intention. We focus not only on whether people adapt, but also on how by contrasting lexical specification (e.g., “batter”) and syntactic modification (e.g., “man in white pants”) when discriminating between objects in natural scenes (e.g., a batter wearing white pants and a referee). There are three main results. First, we replicate that speakers adapt their expressions to avoid ambiguity. Second, communicative intention has an effect: participants tended to use more specific names in a discrimination task than in a descriptive task, even without referential ambiguity in the context. Third, when given the choice, participants tended to prefer more specific words over adding modification – that is, using lexical rather than syntactic means to resolve ambiguity. This suggests that it may be less demanding to increase informativity of referring expressions with lexical specification than syntactic modification., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la Conference on Empirical Methods in Natural Language Processing, celebrada els dies 31 d'octubre a 4 de novembre de 2020 a Brussel·les, Bèlgica., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., We thank German Kruszewski and the AMORE ´ team for the helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramon y Cajal programme (grant RYC-2015- ´ 18907) and the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Negated adjectives and antonyms in distributional semantics: not similar?

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
We investigate the relation between negated adjectives and antonyms pairs in English (e.g., not cold vs. hot - cold) using Distributional Semantics. We build vector representations of a set of antonyms and their negations on the basis of their contexts of use, and compare thensimilarities of the negated adjectives to each of the adjective in their antonym pair. We findnthat in a distributional semantic model a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot). The effect is less strong for antonyms that share their lexical root (morphological; e.g., happy - unhappy). No difference is observed between simple and double negations (e.g., not happy, not unhappy), and contrary and contradictory antonyms (e.g., hot - cold, dead - alive). Our results provides insights on negated adjectives, and in general the type of similarity captured by Distributional Semantics., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Distributed prediction of relations for entities: the Easy, the Difficult, and the impossible

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a la 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), celebrat els dies 3 i 4 d'agost de 2017 a Vancouver, Canada., Word embeddings are supposed to provide easy access to semantic relations such as “male of” (man–woman). While this claim has been investigated for concepts, little is known about the distributional behavior of relations of (Named) Entities. We describe two word embedding-based models that predict values for relational attributes of entities, and analyse them. The task is challenging, with major performance differences between relations. Contrary to many NLP tasks, high difficulty for a relation does not result from low frequency, but from (a) one-to-many mappings; and (b) lack of context patterns expressing the relation that are easy to pick up by word embeddings., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and the DFG (SFB 732, Project D10).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

“Show me the cup”: reference with continuous representations

  • Boleda, Gemma
  • Baroni, Marco
  • Padó, Sebastian
Comunicació presentada a CICLing 2017, celebrat a Budapest (Hongria) del 17 al 23 d'abril de 2017., One of the most basic functions of language is to refer to objects in a shared scene. Modeling reference with continuous representations is challenging because it requires individuation, i.e., tracking and distinguishing an arbitrary number of referents. We introduce a neural network model that, given a de nite description and a set of objects represented by natural images, points to the intended object if the expression has a unique referent, or indicates a failure, if it does not. The model, directly trained on reference acts, is competitive with a pipeline manually engineered to perform the same task, both when referents are purely visual, and when they are characterized by a combination of visual and linguistic properties., This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and ERC grant agreement No 715154 (AMORE); ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

AMORE-UPF at SemEval-2018 Task 4: BiLSTM with entity library

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada al 12th International Workshop on Semantic Evaluation (SemEval-2018), celebrat els dies 5 i 6 de juny de 2018 a Nova Orleans, EUA., This paper describes our winning contribution to SemEval 2018 Task 4: Character Identification on Multiparty Dialogues. It is a simple, standard model with one key innovation, an entity library. Our results show that this innovation greatly facilitates the identification of infrequent characters. Because of the generic nature of our model, this finding is potentially relevant to any task that requires effective learning from sparse or unbalanced data., This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

What’s in a name? A large-scale computational study on how competition between names affects naming variation

  • Gualdoni, Eleonora
  • Brochhagen, Thomas
  • Mädebach, Andreas
  • Boleda, Gemma
Different speakers often use different names to refer to the same entity (e.g., “woman” vs. “tennis player” for a given woman playing tennis). We study how visual typicality affects variation in naming behavior. We use a novel computational approach to estimate visual typicality from images, and analyze a large dataset containing naming data for realistic images. In contrast to previous work, we take into account the visual properties of both the object and the scene in which it appears; and factor in multiple candidate names. We show that visual typicality mediates competition between candidate names: high competition, induced by the relationship between the visual properties of the object and the visual representations associated to names, predicts higher naming variation. On a methodological level, we demonstrate the potential of using large-scale datasets with realistic images in conjunction with computational methods to shed light on how people name objects., The authors thank the editors Adrian Staub and Kathleen Rastle, the reviewers Fritz Günther and Marc Brysbaert, and the COLT research group for their useful feedback, as well as Carina Silberer for advice regarding Computer Vision models. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain; ref. PID2020-112602GB-I00/MICIN/AEI/10.13039/501100011033). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Challenges in including extra-linguistic context in pre-trained language models

  • Sorodoc, Ionut-Teodor
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a Third Workshop on Insights from Negative Results in NLP (Insights 2022), celebrat el 26 de maig de 2022 a Dublín, Irlanda., To successfully account for language, computational models need to take into account both the linguistic context (the content of the utterances) and the extra-linguistic context (for instance, the participants in a dialogue). We focus on a referential task that asks models to link entity mentions in a TV show to the corresponding characters, and design an architecture that attempts to account for both kinds of context. In particular, our architecture combines a previously proposed specialized module (an “entity library”) for character representation with transfer learning from a pre-trained language model. We find that, although the model does improve linguistic contextualization, it fails to successfully integrate extra-linguistic information about the participants in the dialogue. Our work shows that it is very challenging to incorporate extra-linguistic information into pretrained language models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Asking between the lines: elicitation of evoked questions in text

  • Westera, Matthijs
  • Rohde, Hannah
Comunicació presentada al Amsterdam Colloquium 2019, celebrat del 18 al 20 de desembre de 2019 a Amsterdam, Holanda., We introduce a novel, scalable method aimed at annotating potential and actual Questions Under Discussion (QUDs) in naturalistic discourse. It consists of asking naive participants first what questions a certain portion of the discourse evokes for them and subsequently which of those end up being answered as the discourse proceeds. This paper outlines the method and design decisions that went into it and on characterizing highlevel properties of the resulting data. We highlight ways in which the data gathered via our method could inform our understanding of QUD-driven phenomena and QUD models themselves. We also provide access to a visualization tool for viewing the evoked questions we gathered using this method (N=4765 from 111 crowdsourced annotators)., This work was supported in part by a Leverhulme Trust Prize in Languages and Literatures to H. Rohde. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A distributional study of negated adjectives and antonyms

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
Comunicació presentada al Fifth Italian Conference on Computational Linguistics (CLiC-it 2018); celebrada del 10 al 12 de desembre de 2018 a Torí, Itàlia., In this paper, we investigate the relation between negated adjectives and antonyms in English using Distributional Semantics methods. Results show that, on the basis of contexts of use, a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot); such effect is less strong for antonyms derived by affixation (e.g., happy - unhappy). In questo lavoro, analizziamo la relazione fra aggettivi negati e antonimi in inglese utilizzando metodi di Semantica Distribuzionale. I risultati mostrano che, sulla base dei contesti di uso, la negazione di un aggettivo (ad es. “not cold”; it.: “non freddo”) e tipicamente ` piu simile all’aggettivo stesso (“cold”; it.: ` “freddo”) che al suo antonimo (“hot”; it.: “caldo”). Tale effetto e meno accentuato ` per antonimi derivati tramite affissi (ad es. “happy”-“unhappy”; it.: “felice”- “infelice”)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Probing for referential information in language models

  • Sorodoc, Ionut-Teodor
  • Gulordava, Kristina
  • Boleda, Gemma
Comunicació presentada al 58th Annual Meeting of the Association for Computational Linguistics celebrat del 5 al 10 de juliol de 2020 de manera virtual., Language models keep track of complex information about the preceding context – including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, via probe tasks and analysis on a coreference annotated corpus. The Transformer outperforms the LSTM in all analyses. Our results suggest that language models are more successful at learning grammatical constraints than they are at learning truly referential information, in the sense of capturing the fact that we use language to refer to entities in the world. However, we find traces of the latter aspect, too., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM2019-3-0006).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Woman or tennis player? Visual typicality and lexical frequency affect variation in object naming

  • Gualdoni, Eleonora
  • Brochhagen, Thomas
  • Mädebach, Andreas
  • Boleda, Gemma
Speakers often use different names to refer to the same entity (e.g., “woman” vs. “tennis player”). We here explore factors that affect naming variation for visually presented objects. We analyze a large dataset of object names with realistic images and focus on two factors: visual typicality (of both objects and the contexts they appear in) and name frequency. We develop a novel computational approach to estimate visual typicality, using image representations from Computer Vision models. Specifically, we compute visual typicality as similarity between the representation of an object/context to the average representation of other objects/contexts of its nominal class. In contrast to previous studies, we not only study the name used by most annotators for a given object (top name), but also the second most frequently used (alternative name). Our results show that the top name and the alternative name pull in opposite directions. People’s naming choices are more varied for objects that are less typical for their top name, or more typical for their alternative name. They are also more varied when the top name has relatively low frequency (for alternative names, the opposite effect may be present but the data are not conclusive). Context typicality instead does not show a general effect in our analysis. Overall, our results show that visual and lexical characteristics relating to name candidates beyond the top name are informative for predicting variability in object naming. On a methodological level, we demonstrate the potential of using large scale datasets with realistic images in conjunction with computational methods to inform models of human object naming., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

TED-Q: TED talks and the questions they evoke

  • Westera, Matthijs
  • Mayol, Laia
  • Rohde, Hannah
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., We present a new dataset of TED-talks annotated with the questions they evoke and, where available, the answers to these questions. Evoked questions represent a hitherto mostly unexplored type of linguistic data, which promises to open up important new lines of research, especially related to the Question Under Discussion (QUD)-based approach to discourse structure. In this paper we introduce the method and open the first installment of our data to the public. We summarize and explore the current dataset, illustrate its potential by providing new evidence for the relation between predictability and implicitness – capitalizing on the already existing PDTB-style annotations for the texts we use – and outline its potential for future research. The dataset should be of interest, at its current scale, to researchers on formal and experimental pragmatics, discourse coherence, information structure, discourse expectations and processing. Our data-gathering procedure is designed to scale up, relying on crowdsourcing by non-expert annotators, with its utility for Natural Language Processing in mind (e.g., dialogue systems, conversational question answering)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Comparatives, quantifiers, proportions: a multi-task model for the learning of quantities from vision

  • Sorodoc, Ionut-Teodor
  • Pezzelle, Sandro
  • Bernardi, Raffaella
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), celebrada els dies 1 a 6 de juny de 2018 a Nova Orleans, Estats Units d'Amèrica., The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual scenes by a multi-task computational model. The motivation is that, in humans, these processes underlie the same cognitive, nonsymbolic ability, which allows an automatic estimation and comparison of set magnitudes. We show that when information about lowercomplexity tasks is available, the higher-level proportional task becomes more accurate than when performed in isolation. Moreover, the multi-task model is able to generalize to unseen combinations of target/non-target objects. Consistently with behavioral evidence showing the interference of absolute number in the proportional task, the multi-task model no longer works when asked to provide the number of target objects in the scene., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A closer look at scalar diversity using contextualized semantic similarity

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a la 24th Sinn und Bedeutung Conference, SuB24, celebrada del 4 al 6 de setembre de 2019 a la Universitat d'Osnabrück, Alemania., We take a closer look at van Tiel et al.’s (2016) experimental results on diversity in scalar inference rates. In contrast to their finding that semantic similarity had no significant effect on scalar inference rates, we show that a sufficiently fine-grained notion of semantic similarity does have an effect: the more similar the two terms on a scale, the lower the scalar inference rate. Moreover, we show that a context-sensitive notion of semantic similarity (in particular ELMo; Peters et al., 2018) can explain more of the variance in the data, but only modestly, only for stimuli that contain informative context words, and only when the scalar terms themselves are sufficiently context-sensitive., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains. This project has also received funding from the Ramon y Cajal programme (grant RYC-2015-18907) and from the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Humans meet models on object naming: a new dataset and analysis

  • Silberer, Carina
  • Zarrieß, Sina
  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We release ManyNames v2 (MN v2), a verified version of an object naming dataset that contains dozens of valid names per object for 25K images. We analyze issues in the data collection method originally employed, standard in Language & Vision (L&V), and find that the main source of noise in the data comes from simulating a naming context solely from an image with a target object marked with a bounding box, which causes subjects to sometimes disagree regarding which object is the target. We also find that both the degree of this uncertainty in the original data and the amount of true naming variation in MN v2 differs substantially across object domains. We use MN v2 to analyze a popular L&V model and demonstrate its effectiveness on the task of object naming. However, our fine-grained analysis reveals that what appears to be human-like model behavior is not stable across domains, e.g., the model confuses people and clothing objects much more frequently than humans do. We also find that standard evaluations underestimate the actual effectiveness of the naming model: on the single-label names of the original dataset (Visual Genome), it obtains −27% accuracy points than on MN v2, that includes all valid object names., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publicación Capítulo o parte de un libro (BookPart)

Conceptual versus referential affordance in concept composition

  • McNally, Louise, 1965-
  • Boleda, Gemma
One of the defining traits of language is its capacity to mediate between concepts in our mind, which encapsulate generalizations, and the things they refer to in a given communicative act, with all their idiosyncratic properties. This article examines precisely this interplay between conceptual and referential aspects of meaning, and proposes that concept composition (or concept combination, a term more commonly used in Psychology) exploits both: Conceptually afforded composition is at play when a modifier and its head fit as could be expected given the properties of the two concepts involved, whereas in referentially afforded composition the result of the composition depends on specific, independently available properties of the referent. For instance, red box tends to be applied to boxes whose surface is red, but, given the appropriate context, it can also be applied to e.g. a brown box that contains red objects. We support our proposal with data from nominal modification, and explore a way to formally distinguish the two kinds of composition and integrate them into a more general framework for semantic analysis. Along the way, we recover the classically Fregean notion of sense as including conceptual information, and show the potential of distributional semantics, a framework that has become very influential in Cognitive Science and Computational Linguistics, to address research questions from a theoretical linguistic perspective., This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154, AMORE); Spanish MINECO grants FFI2010-15006, FFI2010-09464-E, and FFI2013-41301, as well as an ICREA Academia award to the first author.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Rising declaratives of the quality-suspending kind

  • Westera, Matthijs
The theory of Intonational Compliance Marking (ICM) maintains that speakers of English use final rising intonation to indicate a suspension (potential violation) of a conversational maxim (Westera 2013; 2014). This paper aims to show that a certain kind of rising declarative, one which has been prominent in the literature (e.g., Gunlogson 2008), can be adequately understood in ICM’s terms as involving a suspension of the maxim of Quality. By explicating certain minimal assumptions about pragmatics, this understanding accounts for three core features of such rising declaratives: their question-likeness, the speaker bias they express and their badness out of the blue. In a nutshell, their question-likeness is derived from principles of general cooperative discourse, their bias from the relative importance of the maxim of Quality, and their badness out of the blue from a competition between rising declaratives and interrogatives. The account is compared in detail to various existing accounts of rising declaratives of the relevant sort, highlighting explanatory and empirical differences., This work has benefited from detailed commentary by four anonymous reviewers for Glossa, as well as Jeroen Groenendijk and Floris Roelofsen. Any remaining errors are of course my own. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Hearing “birch” hampers saying “duck”: an event-related potential study on phonological interference in immediate and delayed word production

  • Mädebach, Andreas
  • Widmann, Andreas
  • Posch, Melina
  • Schröger, Erich
  • Jescheniak, Jörg D.
When speakers name a picture (e.g., “duck”), a distractor word phonologically related to an alternative name (e.g., “birch” related to “bird”) slows down naming responses compared with an unrelated distractor word. This interference effect obtained with the picture–word interference task is assumed to reflect the phonological coactivation of close semantic competitors and is critical for evaluating contemporary models of word production. In this study, we determined the ERP signature of this effect in immediate and delayed versions of the picture–word interference task. ERPs revealed a differential processing of related and unrelated distractors: an early (305–436 msec) and a late (537–713 msec) negativity for related as compared with unrelated distractors. In the behavioral data, the interference effect was only found in immediate naming, whereas its ERP signature was also present in delayed naming. The time window of the earlier ERP effect suggests that the behavioral interference effect indeed emerges at a phonological processing level, whereas the functional significance of the later ERP effect is as yet not clear. The finding of a robust ERP correlate of phonological coactivation might facilitate future research on lexical processing in word production., This work was supported by the German Research Foundation under grant DFG JE229/11-2. A. M. received funding from a Beatriu-de-Pinós postdoctoral grant (2017 BP 00180) of the Catalan Agency for Management of University and Research Grants and from the European Research Council under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 715154).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Run Like a Girl! Sport-Related Gender Bias in Language and Vision

  • Harrison, Sophie
  • Gualdoni, Eleonora
  • Boleda, Gemma
Gender bias in Language and Vision datasets and models has the potential to perpetuate harmful stereotypes and discrimination. We analyze gender bias in two Language and Vision datasets. Consistent with prior work, we find that both datasets underrepresent women, which promotes their invisibilization. Moreover, we hypothesize and find that a bias affects human naming choices for people playing sports: speakers produce names indicating the sport (e.g. ‘tennis player’ or ‘surfer’) more often when it is a man or a boy participating in the sport than when it is a woman or a girl, with an average of 46% vs. 35% of sports-related names for each gender. A computational model trained on these naming data reproduces the bias. We argue that both the data and the model result in representational harm against women., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain) and of grant agreement No. 715154 funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Brief at the risk of being misunderstood: consolidating population- and individual-level tendencies

  • Brochhagen, Thomas
Communicative pressures can give rise to regular patterns of language use. These patterns, in turn, can come to shape a language’s structure over time. In a recent study, Kanwal et al. (Cognition, 165:45–52, 2017) investigate whether an interaction of such pressures may underlie the cross-linguistic tendency of frequent forms to be shorter. Using a miniature artificial language, they show that speakers follow this tendency if pressured for brevity and accuracy. In this study, we use probabilistic models of varying complexity to shed light on the individual-level factors behind this trend. We find that a hierarchical model that accommodates for subjects’ heterogeneous beliefs about object frequencies best explains the data. At the population level, this model predicts an association of short forms with frequent meanings, in line with past research. At the individual level, however, it reveals a number of patterns that systematically deviate from this trend. On the one hand, these findings support the hypothesis that individual-level pressures may underlie natural languages’ relationship between frequency and brevity. On the other, by characterizing the individual-level dynamics on which this relationship rests, they highlight the importance of consolidating multiple strata of analysis and of understanding where and why they might diverge., This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovarion programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Object naming in language and vision: a survey and a new dataset

  • Silberer, Carina
  • Zarrieß, Sina
  • Boleda, Gemma
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., People choose particular names for objects, such as dog or puppy for a given dog. Object naming has been studied in Psycholinguistics, but has received relatively little attention in Computational Linguistics. We review resources from Language and Vision that could be used to study object naming on a large scale, discuss their shortcomings, and create a new dataset that affords more opportunities for analysis and modeling. Our dataset, ManyNames, provides 36 name annotations for each of 25K objects in images selected from VisualGenome. We highlight the challenges involved and provide a preliminary analysis of the ManyNames data, showing that there is a high level of agreement in naming, on average. At the same time, the average number of name types associated with an object is much higher in our dataset than in existing corpora for Language and Vision, such that ManyNames provides a rich resource for studying phenomena like hierarchical variation (chihuahua vs. dog), which has been discussed at length in the theoretical literature, and other less well studied phenomena like cross-classification (cake vs. dessert)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Putting words in context: LSTM language models and lexical ambiguity

  • Boleda, Gemma
  • Gulordava, Kristina
  • Aina, Laura
Comunicació presentada al 57th Annual Meeting of the Association for Computational Linguistic (ACL 2019), celebrat els dies 28 de juliol a 2 d'agost de 2019 a Florència, Itàlia., In neural network models of language, words are commonly represented using context invariant representations (word embeddings) which are then put in context in the hidden layers. Since words are often ambiguous, representing the contextually relevant information is not trivial. We investigate how an LSTM language model deals with lexical ambiguity in English, designing a method to probe its hidden representations for lexical and contextual information about words. We find that both types of information are represented to a large extent, but also that there is room for improvement for contextual information., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Distributional models of category concepts based on names of category members

  • Westera, Matthijs
  • Gupta, Abhijee
  • Boleda, Gemma
  • Padó, Sebastian
Cognitive scientists have long used distributional semantic representations of categories. The predominant approach uses distributional representations of category-denoting nouns, such as “city” for the category city. We propose a novel scheme that represents categories as prototypes over representations of names of its members, such as “Barcelona,” “Mumbai,” and “Wuhan” for the category city. This name-based representation empirically outperforms the noun-based representation on two experiments (modeling human judgments of category relatedness and predicting category membership) with particular improvements for ambiguous nouns. We discuss the model complexity of both classes of models and argue that the name-based model has superior explanatory potential with regard to concept acquisition., Abhijeet Gupta and Sebastian Padó received funding from Deutsche Forschungsgemeinschaft (DFG) through Sonderforschungsbereich 732, project D10. Gemma Boleda and Matthijs Westera received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 715154) and the Catalan government (SGR 2017 1575). Gemma Boleda also received funding from the Ramón y Cajal program (grant RYC-2015-18907). Matthijs Westera also received funding from Leiden University (LUCL, SAILS). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Does referent predictability affect the choice of referential form? A computational approach using masked coreference resolution

  • Aina, Laura
  • Liao, Xixian
  • Boleda, Gemma
  • Westera, Matthijs
Comunicació presentada a 25th Conference on Computational Natural Language Learning (CoNLL 2021) celebrada en línia del 10 al 11 de 2021, It is often posited that more predictable parts of a speaker’s meaning tend to be made less explicit, for instance using shorter, less informative words. Studying these dynamics in the domain of referring expressions has proven difficult, with existing studies, both psycholinguistic and corpus-based, providing contradictory results. We test the hypothesis that speakers produce less informative referring expressions (e.g., pronouns vs. full noun phrases) when the context is more informative about the referent, using novel computational estimates of referent predictability. We obtain these estimates training an existing coreference resolution system for English on a new task, masked coreference resolution, giving us a probability distribution over referents that is conditioned on the context but not the referring expression. The resulting system retains standard coreference resolution performance while yielding a better estimate of human-derived referent predictability than previous attempts. A statistical analysis of the relationship between model output and mention form supports the hypothesis that predictability affects the form of a mention, both its morphosyntactic type and its length., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Instances and concepts in distributional space

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a l'European Chapter of the Association for Computational Linguistics (EACL) celebrat a València, Espanya, del 2 al 6 de juny de 2017., Instances (“Mozart”) are ontologically distinct from concepts or classes (“composer”). Natural language encompasses both, but instances have received comparatively little attention in distributional semantics. Our results show that instances and concepts differ in their distributional properties. We also establish that instantiation detection (“Mozart – composer”) is generally easier than hypernymy detection (“chemist – scientist”), and that results on the influence of input representation do not transfer from hyponymy to instantiation., The authors have received funding from DFG (SFB 732, project B9). and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE), as well as under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Quantifying informativeness of names in visual space

  • Gualdoni, Eleonora
  • Kemp, Charles
  • Xu, Yang
  • Boleda, Gemma
Comunicació presentada a 45th Annual Conference of the Cognitive Science Society (CogSci 2023), celebrada a Sydney (Austràlia) del 26 al 29 de juliol de 2023., The human lexicon expresses a wide array of concepts with a limited set of words. Previous work has suggested that semantic categories are structured compactly to enable informative communication. Informativeness is typically quantified with respect to an entire semantic domain and not at the level of individual names. We develop a measure of name informativeness using an information-theoretic framework grounded in visual object representations derived from natural images. Our approach uses computer vision models to characterize informativeness of individual names with respect to large-scale data in a naturalistic setting. We show that our informativeness measure predicts degrees of specificity in lexical categories more precisely than alternative measures based on entropy and frequency. We also show that name informativeness jointly captures within-category similarity and distinctiveness across categories. Our analyses suggest how the variability of names from a broad part of the lexicon may be understood through the lens of information theory., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovacion and the Agencia Estatal ´ de Investigacion (Spain), of grant agreement No. 715154 ´ funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme, and of NSERC Alliance International-Catalyst Grant ALLRP 576149-22. The second author was supported by grant FT190100200 from the Australian Research Council.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Don’t blame distributional semantics if it can’t do entailment

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a 13th International Conference on Computational Semantics (IWCS 2019), celebrat del 23 al 27 de maig de 2019 a Göteborg, Suècia., Distributional semantics has had enormous empirical success in Computational Linguistics and Cognitive Science in modeling various semantic phenomena, such as semantic similarity, and distributional models are widely used in state-of-the-art Natural Language Processing systems. However, the theoretical status of distributional semantics within a broader theory of language and cognition is still unclear: What does distributional semantics model? Can it be, on its own, a fully adequate model of the meanings of linguistic expressions? The standard answer is that distributional semantics is not fully adequate in this regard, because it falls short on some of the central aspects of formal semantic approaches: truth conditions, entailment, reference, and certain aspects of compositionality. We argue that this standard answer rests on a misconception: These aspects do not belong in a theory of expression meaning, they are instead aspects of speaker meaning, i.e., communicative intentions in a particular context. In a slogan: words do not refer, speakers do. Clearing this up enables us to argue that distributional semantics on its own is an adequate model of expression meaning. Our proposal sheds light on the role of distributional semantics in a broader theory of language and cognition, its relationship to formal semantics, and its place in computational models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

QUDs, brevity, and the asymmetry of alternatives

  • Westera, Matthijs
Comunicació presentada al Amsterdam Colloquium 2017, celebrat a Amsterdam (Holanda) del 20 a 22 de desembre de 2017, Exhaustivity is typically explained in terms of the exclusion of unmentioned alternatives. For this to work, the set of alternatives must be asymmetrical, lest both a proposition and its negation get excluded, yielding a contradiction (the Symmetry Problem). Since exhaustivity is regularly observed, these alternative sets must tend to be asymmetrical, and this requires an explanation. Existing explanations are based on considerations of brevity, but these run into certain problems. A new solution is proposed, explaining the asymmetry of alternatives in terms of the fact that discourse strategies with asymmetrical questions under discussion (Quds) are favored because they allow part of the answer to be communicated implicitly, namely as an exhaustivity implicature., Many thanks to Floris Roelofsen and Jeroen Groenendijk for their comments on many iterations of this work. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Short-term meaning shift: a distributional exploration

  • Boleda, Gemma
  • Del Tredici, Marco
  • Fernández, Raquel
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., We present the first exploration of meaning shift over short periods of time in online communities using distributional representations. We create a small annotated dataset and use it to assess the performance of a standard model for meaning shift detection on shortterm meaning shift. We find that the model has problems distinguishing meaning shift from referential phenomena, and propose a measure of contextual variability to remedy this., The research carried out by the Amsterdam section of the team was partially funded by the Netherlands Organisation for Scientific Research (NWO) under VIDI grant no. 276-89-008, Asymmetry in Conversation. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Quantifying informativeness of names in visual space

  • Gualdoni, Eleonora
  • Kemp, Charles
  • Xu, Yang
  • Boleda, Gemma
Comunicació presentada a 45th Annual Conference of the Cognitive Science Society (CogSci 2023), celebrada a Sydney (Austràlia) del 26 al 29 de juliol de 2023., The human lexicon expresses a wide array of concepts with a limited set of words. Previous work has suggested that semantic categories are structured compactly to enable informative communication. Informativeness is typically quantified with respect to an entire semantic domain and not at the level of individual names. We develop a measure of name informativeness using an information-theoretic framework grounded in visual object representations derived from natural images. Our approach uses computer vision models to characterize informativeness of individual names with respect to large-scale data in a naturalistic setting. We show that our informativeness measure predicts degrees of specificity in lexical categories more precisely than alternative measures based on entropy and frequency. We also show that name informativeness jointly captures within-category similarity and distinctiveness across categories. Our analyses suggest how the variability of names from a broad part of the lexicon may be understood through the lens of information theory., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovacion and the Agencia Estatal ´ de Investigacion (Spain), of grant agreement No. 715154 ´ funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme, and of NSERC Alliance International-Catalyst Grant ALLRP 576149-22. The second author was supported by grant FT190100200 from the Australian Research Council.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Don’t blame distributional semantics if it can’t do entailment

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a 13th International Conference on Computational Semantics (IWCS 2019), celebrat del 23 al 27 de maig de 2019 a Göteborg, Suècia., Distributional semantics has had enormous empirical success in Computational Linguistics and Cognitive Science in modeling various semantic phenomena, such as semantic similarity, and distributional models are widely used in state-of-the-art Natural Language Processing systems. However, the theoretical status of distributional semantics within a broader theory of language and cognition is still unclear: What does distributional semantics model? Can it be, on its own, a fully adequate model of the meanings of linguistic expressions? The standard answer is that distributional semantics is not fully adequate in this regard, because it falls short on some of the central aspects of formal semantic approaches: truth conditions, entailment, reference, and certain aspects of compositionality. We argue that this standard answer rests on a misconception: These aspects do not belong in a theory of expression meaning, they are instead aspects of speaker meaning, i.e., communicative intentions in a particular context. In a slogan: words do not refer, speakers do. Clearing this up enables us to argue that distributional semantics on its own is an adequate model of expression meaning. Our proposal sheds light on the role of distributional semantics in a broader theory of language and cognition, its relationship to formal semantics, and its place in computational models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Comparatives, quantifiers, proportions: a multi-task model for the learning of quantities from vision

  • Sorodoc, Ionut-Teodor
  • Pezzelle, Sandro
  • Bernardi, Raffaella
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), celebrada els dies 1 a 6 de juny de 2018 a Nova Orleans, Estats Units d'Amèrica., The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual scenes by a multi-task computational model. The motivation is that, in humans, these processes underlie the same cognitive, nonsymbolic ability, which allows an automatic estimation and comparison of set magnitudes. We show that when information about lowercomplexity tasks is available, the higher-level proportional task becomes more accurate than when performed in isolation. Moreover, the multi-task model is able to generalize to unseen combinations of target/non-target objects. Consistently with behavioral evidence showing the interference of absolute number in the proportional task, the multi-task model no longer works when asked to provide the number of target objects in the scene., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Modeling word interpretation with deep language models: the interaction between expectations and lexical information

  • Aina, Laura
  • Brochhagen, Thomas
  • Boleda, Gemma
How a word is interpreted depends on the context it appears in. We study word interpretation leveraging deep language models, tracing the contribution and interaction of two sources of information that have been shown to be central to it: contextinvariant lexical information, represented by the word embeddings of a model, and a listener’s contextual expectations, represented by its predictions. We define operations to combine these components to obtain representations of word interpretations. We instantiate our framework using two English language models, and evaluate the resulting representations in the extent by which they reflect contextual word substitutes provided by human subjects. Our results suggest that both lexi cal information and expectations codify information pivotal to word interpretation; however, their combination is better than either on its own. Moreover, the division of labor between expectations and the lexicon appears to change across contexts, This project has re ceived funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and inno vation programme (grant agreement No 715154), and from the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research, and the computer resources at CTE-POWER and the technical support pro vided by Barcelona Supercomputing Center (RES-IM2019-3-0006). This paper reflects the authors view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Does referent predictability affect the choice of referential form? A computational approach using masked coreference resolution

  • Aina, Laura
  • Liao, Xixian
  • Boleda, Gemma
  • Westera, Matthijs
Comunicació presentada a 25th Conference on Computational Natural Language Learning (CoNLL 2021) celebrada en línia del 10 al 11 de 2021, It is often posited that more predictable parts of a speaker’s meaning tend to be made less explicit, for instance using shorter, less informative words. Studying these dynamics in the domain of referring expressions has proven difficult, with existing studies, both psycholinguistic and corpus-based, providing contradictory results. We test the hypothesis that speakers produce less informative referring expressions (e.g., pronouns vs. full noun phrases) when the context is more informative about the referent, using novel computational estimates of referent predictability. We obtain these estimates training an existing coreference resolution system for English on a new task, masked coreference resolution, giving us a probability distribution over referents that is conditioned on the context but not the referring expression. The resulting system retains standard coreference resolution performance while yielding a better estimate of human-derived referent predictability than previous attempts. A statistical analysis of the relationship between model output and mention form supports the hypothesis that predictability affects the form of a mention, both its morphosyntactic type and its length., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Living a discrete life in a continuous world: reference with distributed representations

  • Boleda, Gemma
  • Padó, Sebastian
  • Pham, Nghia The
  • Baroni, Marco
Comunicació presentada a IWCS 2017, celebrat a Montpellier (França) del 19 al 22 de setembre de 2017., Reference is the crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. We propose a fully data-driven, end-toend trainable model that, while operating on continuous multimodal representations, learns to organize them into a discrete-like entity library. We also introduce a referential task to test it, cross-modal tracking. Our model beats standard neural network architectures, but is outperformed by some parametrizations of Memory Networks, another model with external memory., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE); Marie Sklodowska-Curie grant agreement No 655577 (LOVe); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Probing for referential information in language models

  • Sorodoc, Ionut-Teodor
  • Gulordava, Kristina
  • Boleda, Gemma
Comunicació presentada al 58th Annual Meeting of the Association for Computational Linguistics celebrat del 5 al 10 de juliol de 2020 de manera virtual., Language models keep track of complex information about the preceding context – including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, via probe tasks and analysis on a coreference annotated corpus. The Transformer outperforms the LSTM in all analyses. Our results suggest that language models are more successful at learning grammatical constraints than they are at learning truly referential information, in the sense of capturing the fact that we use language to refer to entities in the world. However, we find traces of the latter aspect, too., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM2019-3-0006).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Deep daxes: mutual exclusivity arises through both learning biases and pragmatic strategies in neural networks

  • Gulordava, Kristina
  • Brochhagen, Thomas
  • Boleda, Gemma
Children’s tendency to associate novel words with novel referents has been taken to reflect a bias toward mutual exclusivity. This tendency may be advantageous both as (1) an ad-hoc referent selection heuristic to single out referents lacking a label and as (2) an organizing principle of lexical acquisition. This paper investigates under which circumstances cross-situational neural models can come to exhibit analogous behavior to children, focusing on these two possibilities and their interaction. To this end, we evaluate neural networks’ on both symbolic data and, as a first, on large-scale image data. We find that constraints in both learning and selection can foster mutual exclusivity, as long as they put words in competition for lexical meaning. For computational models, these findings clarify the role of available options for better performance in tasks where mutual exclusivity is advantageous. For cognitive research, they highlight latent interactions between word learning, referent selection mechanisms, and the structure of stimuli of varying complexity: symbolic and visual, This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramon´y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM-2019-3-0006). We are grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Capítulo o parte de un libro (BookPart)

Conceptual versus referential affordance in concept composition

  • McNally, Louise, 1965-
  • Boleda, Gemma
One of the defining traits of language is its capacity to mediate between concepts in our mind, which encapsulate generalizations, and the things they refer to in a given communicative act, with all their idiosyncratic properties. This article examines precisely this interplay between conceptual and referential aspects of meaning, and proposes that concept composition (or concept combination, a term more commonly used in Psychology) exploits both: Conceptually afforded composition is at play when a modifier and its head fit as could be expected given the properties of the two concepts involved, whereas in referentially afforded composition the result of the composition depends on specific, independently available properties of the referent. For instance, red box tends to be applied to boxes whose surface is red, but, given the appropriate context, it can also be applied to e.g. a brown box that contains red objects. We support our proposal with data from nominal modification, and explore a way to formally distinguish the two kinds of composition and integrate them into a more general framework for semantic analysis. Along the way, we recover the classically Fregean notion of sense as including conceptual information, and show the potential of distributional semantics, a framework that has become very influential in Cognitive Science and Computational Linguistics, to address research questions from a theoretical linguistic perspective., This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154, AMORE); Spanish MINECO grants FFI2010-15006, FFI2010-09464-E, and FFI2013-41301, as well as an ICREA Academia award to the first author.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

QUDs, brevity, and the asymmetry of alternatives

  • Westera, Matthijs
Comunicació presentada al Amsterdam Colloquium 2017, celebrat a Amsterdam (Holanda) del 20 a 22 de desembre de 2017, Exhaustivity is typically explained in terms of the exclusion of unmentioned alternatives. For this to work, the set of alternatives must be asymmetrical, lest both a proposition and its negation get excluded, yielding a contradiction (the Symmetry Problem). Since exhaustivity is regularly observed, these alternative sets must tend to be asymmetrical, and this requires an explanation. Existing explanations are based on considerations of brevity, but these run into certain problems. A new solution is proposed, explaining the asymmetry of alternatives in terms of the fact that discourse strategies with asymmetrical questions under discussion (Quds) are favored because they allow part of the answer to be communicated implicitly, namely as an exhaustivity implicature., Many thanks to Floris Roelofsen and Jeroen Groenendijk for their comments on many iterations of this work. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Humans meet models on object naming: a new dataset and analysis

  • Silberer, Carina
  • Zarrieß, Sina
  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We release ManyNames v2 (MN v2), a verified version of an object naming dataset that contains dozens of valid names per object for 25K images. We analyze issues in the data collection method originally employed, standard in Language & Vision (L&V), and find that the main source of noise in the data comes from simulating a naming context solely from an image with a target object marked with a bounding box, which causes subjects to sometimes disagree regarding which object is the target. We also find that both the degree of this uncertainty in the original data and the amount of true naming variation in MN v2 differs substantially across object domains. We use MN v2 to analyze a popular L&V model and demonstrate its effectiveness on the task of object naming. However, our fine-grained analysis reveals that what appears to be human-like model behavior is not stable across domains, e.g., the model confuses people and clothing objects much more frequently than humans do. We also find that standard evaluations underestimate the actual effectiveness of the naming model: on the single-label names of the original dataset (Visual Genome), it obtains −27% accuracy points than on MN v2, that includes all valid object names., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Challenges in including extra-linguistic context in pre-trained language models

  • Sorodoc, Ionut-Teodor
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a Third Workshop on Insights from Negative Results in NLP (Insights 2022), celebrat el 26 de maig de 2022 a Dublín, Irlanda., To successfully account for language, computational models need to take into account both the linguistic context (the content of the utterances) and the extra-linguistic context (for instance, the participants in a dialogue). We focus on a referential task that asks models to link entity mentions in a TV show to the corresponding characters, and design an architecture that attempts to account for both kinds of context. In particular, our architecture combines a previously proposed specialized module (an “entity library”) for character representation with transfer learning from a pre-trained language model. We find that, although the model does improve linguistic contextualization, it fails to successfully integrate extra-linguistic information about the participants in the dialogue. Our work shows that it is very challenging to incorporate extra-linguistic information into pretrained language models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Distributed prediction of relations for entities: the Easy, the Difficult, and the impossible

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a la 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), celebrat els dies 3 i 4 d'agost de 2017 a Vancouver, Canada., Word embeddings are supposed to provide easy access to semantic relations such as “male of” (man–woman). While this claim has been investigated for concepts, little is known about the distributional behavior of relations of (Named) Entities. We describe two word embedding-based models that predict values for relational attributes of entities, and analyse them. The task is challenging, with major performance differences between relations. Contrary to many NLP tasks, high difficulty for a relation does not result from low frequency, but from (a) one-to-many mappings; and (b) lack of context patterns expressing the relation that are easy to pick up by word embeddings., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and the DFG (SFB 732, Project D10).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Run Like a Girl! Sport-Related Gender Bias in Language and Vision

  • Harrison, Sophie
  • Gualdoni, Eleonora
  • Boleda, Gemma
Gender bias in Language and Vision datasets and models has the potential to perpetuate harmful stereotypes and discrimination. We analyze gender bias in two Language and Vision datasets. Consistent with prior work, we find that both datasets underrepresent women, which promotes their invisibilization. Moreover, we hypothesize and find that a bias affects human naming choices for people playing sports: speakers produce names indicating the sport (e.g. ‘tennis player’ or ‘surfer’) more often when it is a man or a boy participating in the sport than when it is a woman or a girl, with an average of 46% vs. 35% of sports-related names for each gender. A computational model trained on these naming data reproduces the bias. We argue that both the data and the model result in representational harm against women., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain) and of grant agreement No. 715154 funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Object naming in language and vision: a survey and a new dataset

  • Silberer, Carina
  • Zarrieß, Sina
  • Boleda, Gemma
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., People choose particular names for objects, such as dog or puppy for a given dog. Object naming has been studied in Psycholinguistics, but has received relatively little attention in Computational Linguistics. We review resources from Language and Vision that could be used to study object naming on a large scale, discuss their shortcomings, and create a new dataset that affords more opportunities for analysis and modeling. Our dataset, ManyNames, provides 36 name annotations for each of 25K objects in images selected from VisualGenome. We highlight the challenges involved and provide a preliminary analysis of the ManyNames data, showing that there is a high level of agreement in naming, on average. At the same time, the average number of name types associated with an object is much higher in our dataset than in existing corpora for Language and Vision, such that ManyNames provides a rich resource for studying phenomena like hierarchical variation (chihuahua vs. dog), which has been discussed at length in the theoretical literature, and other less well studied phenomena like cross-classification (cake vs. dessert)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Instances and concepts in distributional space

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a l'European Chapter of the Association for Computational Linguistics (EACL) celebrat a València, Espanya, del 2 al 6 de juny de 2017., Instances (“Mozart”) are ontologically distinct from concepts or classes (“composer”). Natural language encompasses both, but instances have received comparatively little attention in distributional semantics. Our results show that instances and concepts differ in their distributional properties. We also establish that instantiation detection (“Mozart – composer”) is generally easier than hypernymy detection (“chemist – scientist”), and that results on the influence of input representation do not transfer from hyponymy to instantiation., The authors have received funding from DFG (SFB 732, project B9). and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE), as well as under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Putting words in context: LSTM language models and lexical ambiguity

  • Boleda, Gemma
  • Gulordava, Kristina
  • Aina, Laura
Comunicació presentada al 57th Annual Meeting of the Association for Computational Linguistic (ACL 2019), celebrat els dies 28 de juliol a 2 d'agost de 2019 a Florència, Itàlia., In neural network models of language, words are commonly represented using context invariant representations (word embeddings) which are then put in context in the hidden layers. Since words are often ambiguous, representing the contextually relevant information is not trivial. We investigate how an LSTM language model deals with lexical ambiguity in English, designing a method to probe its hidden representations for lexical and contextual information about words. We find that both types of information are represented to a large extent, but also that there is room for improvement for contextual information., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

What’s in a name? A large-scale computational study on how competition between names affects naming variation

  • Gualdoni, Eleonora
  • Brochhagen, Thomas
  • Mädebach, Andreas
  • Boleda, Gemma
Different speakers often use different names to refer to the same entity (e.g., “woman” vs. “tennis player” for a given woman playing tennis). We study how visual typicality affects variation in naming behavior. We use a novel computational approach to estimate visual typicality from images, and analyze a large dataset containing naming data for realistic images. In contrast to previous work, we take into account the visual properties of both the object and the scene in which it appears; and factor in multiple candidate names. We show that visual typicality mediates competition between candidate names: high competition, induced by the relationship between the visual properties of the object and the visual representations associated to names, predicts higher naming variation. On a methodological level, we demonstrate the potential of using large-scale datasets with realistic images in conjunction with computational methods to shed light on how people name objects., The authors thank the editors Adrian Staub and Kathleen Rastle, the reviewers Fritz Günther and Marc Brysbaert, and the COLT research group for their useful feedback, as well as Carina Silberer for advice regarding Computer Vision models. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain; ref. PID2020-112602GB-I00/MICIN/AEI/10.13039/501100011033). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Negated adjectives and antonyms in distributional semantics: not similar?

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
We investigate the relation between negated adjectives and antonyms pairs in English (e.g., not cold vs. hot - cold) using Distributional Semantics. We build vector representations of a set of antonyms and their negations on the basis of their contexts of use, and compare thensimilarities of the negated adjectives to each of the adjective in their antonym pair. We findnthat in a distributional semantic model a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot). The effect is less strong for antonyms that share their lexical root (morphological; e.g., happy - unhappy). No difference is observed between simple and double negations (e.g., not happy, not unhappy), and contrary and contradictory antonyms (e.g., hot - cold, dead - alive). Our results provides insights on negated adjectives, and in general the type of similarity captured by Distributional Semantics., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Hearing “birch” hampers saying “duck”: an event-related potential study on phonological interference in immediate and delayed word production

  • Mädebach, Andreas
  • Widmann, Andreas
  • Posch, Melina
  • Schröger, Erich
  • Jescheniak, Jörg D.
When speakers name a picture (e.g., “duck”), a distractor word phonologically related to an alternative name (e.g., “birch” related to “bird”) slows down naming responses compared with an unrelated distractor word. This interference effect obtained with the picture–word interference task is assumed to reflect the phonological coactivation of close semantic competitors and is critical for evaluating contemporary models of word production. In this study, we determined the ERP signature of this effect in immediate and delayed versions of the picture–word interference task. ERPs revealed a differential processing of related and unrelated distractors: an early (305–436 msec) and a late (537–713 msec) negativity for related as compared with unrelated distractors. In the behavioral data, the interference effect was only found in immediate naming, whereas its ERP signature was also present in delayed naming. The time window of the earlier ERP effect suggests that the behavioral interference effect indeed emerges at a phonological processing level, whereas the functional significance of the later ERP effect is as yet not clear. The finding of a robust ERP correlate of phonological coactivation might facilitate future research on lexical processing in word production., This work was supported by the German Research Foundation under grant DFG JE229/11-2. A. M. received funding from a Beatriu-de-Pinós postdoctoral grant (2017 BP 00180) of the Catalan Agency for Management of University and Research Grants and from the European Research Council under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Attentional pragmatics: a pragmatic approach to exhaustivity

  • Westera, Matthijs
The neo-Gricean approach to exhaustivity is based on the idea that exhaustivity arises when relevant propositions are not asserted. This paper presents a new pragmatic approach based on the idea that exhaustivity arises when relevant propositions are not mentioned, or more precisely, when the speaker did not intend to draw attention to them. This seemingly subtle shift from information to attention results in different predictions on a range of challenges for the neo-Gricean approach, some of which have been brought up in support of the grammatical approach to exhaustivity. This paper discusses three such challenges: exhaustivity on the hints of a quizmaster, exhaustivity on questions, and exhaustivity without an opinionatedness assumption. The two pragmatic approaches are compared on these puzzles along with the grammatical approach.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

“Show me the cup”: reference with continuous representations

  • Boleda, Gemma
  • Baroni, Marco
  • Padó, Sebastian
Comunicació presentada a CICLing 2017, celebrat a Budapest (Hongria) del 17 al 23 d'abril de 2017., One of the most basic functions of language is to refer to objects in a shared scene. Modeling reference with continuous representations is challenging because it requires individuation, i.e., tracking and distinguishing an arbitrary number of referents. We introduce a neural network model that, given a de nite description and a set of objects represented by natural images, points to the intended object if the expression has a unique referent, or indicates a failure, if it does not. The model, directly trained on reference acts, is competitive with a pipeline manually engineered to perform the same task, both when referents are purely visual, and when they are characterized by a combination of visual and linguistic properties., This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and ERC grant agreement No 715154 (AMORE); ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Woman or tennis player? Visual typicality and lexical frequency affect variation in object naming

  • Gualdoni, Eleonora
  • Brochhagen, Thomas
  • Mädebach, Andreas
  • Boleda, Gemma
Speakers often use different names to refer to the same entity (e.g., “woman” vs. “tennis player”). We here explore factors that affect naming variation for visually presented objects. We analyze a large dataset of object names with realistic images and focus on two factors: visual typicality (of both objects and the contexts they appear in) and name frequency. We develop a novel computational approach to estimate visual typicality, using image representations from Computer Vision models. Specifically, we compute visual typicality as similarity between the representation of an object/context to the average representation of other objects/contexts of its nominal class. In contrast to previous studies, we not only study the name used by most annotators for a given object (top name), but also the second most frequently used (alternative name). Our results show that the top name and the alternative name pull in opposite directions. People’s naming choices are more varied for objects that are less typical for their top name, or more typical for their alternative name. They are also more varied when the top name has relatively low frequency (for alternative names, the opposite effect may be present but the data are not conclusive). Context typicality instead does not show a general effect in our analysis. Overall, our results show that visual and lexical characteristics relating to name candidates beyond the top name are informative for predicting variability in object naming. On a methodological level, we demonstrate the potential of using large scale datasets with realistic images in conjunction with computational methods to inform models of human object naming., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Short-term meaning shift: a distributional exploration

  • Boleda, Gemma
  • Del Tredici, Marco
  • Fernández, Raquel
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., We present the first exploration of meaning shift over short periods of time in online communities using distributional representations. We create a small annotated dataset and use it to assess the performance of a standard model for meaning shift detection on shortterm meaning shift. We find that the model has problems distinguishing meaning shift from referential phenomena, and propose a measure of contextual variability to remedy this., The research carried out by the Amsterdam section of the team was partially funded by the Netherlands Organisation for Scientific Research (NWO) under VIDI grant no. 276-89-008, Asymmetry in Conversation. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Controlled tasks for model analysis: retrieving discrete information from sequences

  • Sorodoc, Ionut-Teodor
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada al 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP celebrat l'11 de novembre de 2021 de manera virtual., In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Distributional models of category concepts based on names of category members

  • Westera, Matthijs
  • Gupta, Abhijee
  • Boleda, Gemma
  • Padó, Sebastian
Cognitive scientists have long used distributional semantic representations of categories. The predominant approach uses distributional representations of category-denoting nouns, such as “city” for the category city. We propose a novel scheme that represents categories as prototypes over representations of names of its members, such as “Barcelona,” “Mumbai,” and “Wuhan” for the category city. This name-based representation empirically outperforms the noun-based representation on two experiments (modeling human judgments of category relatedness and predicting category membership) with particular improvements for ambiguous nouns. We discuss the model complexity of both classes of models and argue that the name-based model has superior explanatory potential with regard to concept acquisition., Abhijeet Gupta and Sebastian Padó received funding from Deutsche Forschungsgemeinschaft (DFG) through Sonderforschungsbereich 732, project D10. Gemma Boleda and Matthijs Westera received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 715154) and the Catalan government (SGR 2017 1575). Gemma Boleda also received funding from the Ramón y Cajal program (grant RYC-2015-18907). Matthijs Westera also received funding from Leiden University (LUCL, SAILS). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

​What do entity-centric models learn? Insights from entity linking in multi-party dialogue

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why. In this paper we analyze the behavior of two recently proposed entity-centric models in a referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entitycentric naturally fosters good architectural decisions. However, we also show that these models do not really build entity representations and that they make poor use of linguistic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when deployed., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Idioms and the syntax/semantics interface of descriptive content vs. reference

  • McNally, Louise, 1965-
  • Gehrke, Berit
The syntactic literature on idioms contains some proposals that are surprising from a compositional perspective. For example, Sportiche (2005) and Cecchetto and Donati (2015) propose that, in the case of verb-object idioms, the verb combines directly with the noun inside its DP complement, and the determiner is introduced higher up in the syntactic structure, or is late-adjoined. This seems to violate compositionality insofar as it is generally assumed that the semantic role of the determiner is to convert a noun to the appropriate semantic type to serve as the argument to the function denoted by the verb. In this paper, we establish a connection between this line of analysis and lines of work in semantics that have developed outside of the domain of idioms, particularly work on incorporation and a mixed formal and distributional semantic model developed in McNally (2017); McNally and Boleda (2017). This semantic work separates the composition of descriptive content from that of discourse referent introducing material; our proposal shows that this separation offers a particularly promising way to handle the compositional difficulties posed by idioms, including certain patterns of variation in intervening determiners and modifiers., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No 715154), grant FFI2016-76045-P (AEI/MINEICO/FEDER, UE), a Salvador de Madariaga mobility grant from the Spanish Ministry of Education, Culture and Sports, an ICREA Academia award, and the Alexander von Humboldt Foundation.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Effects of task and visual context on referring expressions using natural scenes

  • Mädebach, Andreas
  • Torubarova, Ekaterina
  • Gualdoni, Eleonora
  • Boleda, Gemma
We explore contextual adaptation of referring expressions with respect to referential ambiguity and communicative intention. We focus not only on whether people adapt, but also on how by contrasting lexical specification (e.g., “batter”) and syntactic modification (e.g., “man in white pants”) when discriminating between objects in natural scenes (e.g., a batter wearing white pants and a referee). There are three main results. First, we replicate that speakers adapt their expressions to avoid ambiguity. Second, communicative intention has an effect: participants tended to use more specific names in a discrimination task than in a descriptive task, even without referential ambiguity in the context. Third, when given the choice, participants tended to prefer more specific words over adding modification – that is, using lexical rather than syntactic means to resolve ambiguity. This suggests that it may be less demanding to increase informativity of referring expressions with lexical specification than syntactic modification., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A distributional study of negated adjectives and antonyms

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
Comunicació presentada al Fifth Italian Conference on Computational Linguistics (CLiC-it 2018); celebrada del 10 al 12 de desembre de 2018 a Torí, Itàlia., In this paper, we investigate the relation between negated adjectives and antonyms in English using Distributional Semantics methods. Results show that, on the basis of contexts of use, a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot); such effect is less strong for antonyms derived by affixation (e.g., happy - unhappy). In questo lavoro, analizziamo la relazione fra aggettivi negati e antonimi in inglese utilizzando metodi di Semantica Distribuzionale. I risultati mostrano che, sulla base dei contesti di uso, la negazione di un aggettivo (ad es. “not cold”; it.: “non freddo”) e tipicamente ` piu simile all’aggettivo stesso (“cold”; it.: ` “freddo”) che al suo antonimo (“hot”; it.: “caldo”). Tale effetto e meno accentuato ` per antonimi derivati tramite affissi (ad es. “happy”-“unhappy”; it.: “felice”- “infelice”)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Asking between the lines: elicitation of evoked questions in text

  • Westera, Matthijs
  • Rohde, Hannah
Comunicació presentada al Amsterdam Colloquium 2019, celebrat del 18 al 20 de desembre de 2019 a Amsterdam, Holanda., We introduce a novel, scalable method aimed at annotating potential and actual Questions Under Discussion (QUDs) in naturalistic discourse. It consists of asking naive participants first what questions a certain portion of the discourse evokes for them and subsequently which of those end up being answered as the discourse proceeds. This paper outlines the method and design decisions that went into it and on characterizing highlevel properties of the resulting data. We highlight ways in which the data gathered via our method could inform our understanding of QUD-driven phenomena and QUD models themselves. We also provide access to a visualization tool for viewing the evoked questions we gathered using this method (N=4765 from 111 crowdsourced annotators)., This work was supported in part by a Leverhulme Trust Prize in Languages and Literatures to H. Rohde. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Rising declaratives of the quality-suspending kind

  • Westera, Matthijs
The theory of Intonational Compliance Marking (ICM) maintains that speakers of English use final rising intonation to indicate a suspension (potential violation) of a conversational maxim (Westera 2013; 2014). This paper aims to show that a certain kind of rising declarative, one which has been prominent in the literature (e.g., Gunlogson 2008), can be adequately understood in ICM’s terms as involving a suspension of the maxim of Quality. By explicating certain minimal assumptions about pragmatics, this understanding accounts for three core features of such rising declaratives: their question-likeness, the speaker bias they express and their badness out of the blue. In a nutshell, their question-likeness is derived from principles of general cooperative discourse, their bias from the relative importance of the maxim of Quality, and their badness out of the blue from a competition between rising declaratives and interrogatives. The account is compared in detail to various existing accounts of rising declaratives of the relevant sort, highlighting explanatory and empirical differences., This work has benefited from detailed commentary by four anonymous reviewers for Glossa, as well as Jeroen Groenendijk and Floris Roelofsen. Any remaining errors are of course my own. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

TED-Q: TED talks and the questions they evoke

  • Westera, Matthijs
  • Mayol, Laia
  • Rohde, Hannah
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., We present a new dataset of TED-talks annotated with the questions they evoke and, where available, the answers to these questions. Evoked questions represent a hitherto mostly unexplored type of linguistic data, which promises to open up important new lines of research, especially related to the Question Under Discussion (QUD)-based approach to discourse structure. In this paper we introduce the method and open the first installment of our data to the public. We summarize and explore the current dataset, illustrate its potential by providing new evidence for the relation between predictability and implicitness – capitalizing on the already existing PDTB-style annotations for the texts we use – and outline its potential for future research. The dataset should be of interest, at its current scale, to researchers on formal and experimental pragmatics, discourse coherence, information structure, discourse expectations and processing. Our data-gathering procedure is designed to scale up, relying on crowdsourcing by non-expert annotators, with its utility for Natural Language Processing in mind (e.g., dialogue systems, conversational question answering)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Brief at the risk of being misunderstood: consolidating population- and individual-level tendencies

  • Brochhagen, Thomas
Communicative pressures can give rise to regular patterns of language use. These patterns, in turn, can come to shape a language’s structure over time. In a recent study, Kanwal et al. (Cognition, 165:45–52, 2017) investigate whether an interaction of such pressures may underlie the cross-linguistic tendency of frequent forms to be shorter. Using a miniature artificial language, they show that speakers follow this tendency if pressured for brevity and accuracy. In this study, we use probabilistic models of varying complexity to shed light on the individual-level factors behind this trend. We find that a hierarchical model that accommodates for subjects’ heterogeneous beliefs about object frequencies best explains the data. At the population level, this model predicts an association of short forms with frequent meanings, in line with past research. At the individual level, however, it reveals a number of patterns that systematically deviate from this trend. On the one hand, these findings support the hypothesis that individual-level pressures may underlie natural languages’ relationship between frequency and brevity. On the other, by characterizing the individual-level dynamics on which this relationship rests, they highlight the importance of consolidating multiple strata of analysis and of understanding where and why they might diverge., This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovarion programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Grounding semantic roles in images

  • Silberer, Carina
  • Pinkal, Manfred
Comunicació presentada a la 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), celebrada del 31 d'octubre al 4 de novembre de 2018 a Brussel·les, Bèlgica., We address the task of visual semantic role labeling (vSRL), the identification of the participants of a situation or event in a visual scene, and their labeling with their semantic relations to the event or situation. We render candidate participants as image regions of objects, and train a model which learns to ground roles in the regions which depict the corresponding participant. Experimental results demonstrate that we can train a vSRL model without reliance on prohibitive image-based role annotations, by utilizing noisy data which we extract automatically from image captions using a linguistic SRL system. Furthermore, our model induces frame–semantic visual representations, and their comparison to previous work on supervised visual verb sense disambiguation yields overall better results., We thank the anonymous reviewers, Leonie Harter, Christine Sch¨afer, Michael Roth, Gemma Boleda, Anna Rohrbach and Bernt Schiele. This research was supported by the German Research Foundation (DFG EXC 285), by the European Research Council (ERC Horizon 2020 grant agreement No 715154), and the Spanish Ramon y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la Conference on Empirical Methods in Natural Language Processing, celebrada els dies 31 d'octubre a 4 de novembre de 2020 a Brussel·les, Bèlgica., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., We thank German Kruszewski and the AMORE ´ team for the helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramon y Cajal programme (grant RYC-2015- ´ 18907) and the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

AMORE-UPF at SemEval-2018 Task 4: BiLSTM with entity library

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada al 12th International Workshop on Semantic Evaluation (SemEval-2018), celebrat els dies 5 i 6 de juny de 2018 a Nova Orleans, EUA., This paper describes our winning contribution to SemEval 2018 Task 4: Character Identification on Multiparty Dialogues. It is a simple, standard model with one key innovation, an entity library. Our results show that this innovation greatly facilitates the identification of infrequent characters. Because of the generic nature of our model, this finding is potentially relevant to any task that requires effective learning from sparse or unbalanced data., This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Similarity or deeper understanding?: analyzing the TED-Q dataset of evoked questions

  • Westera, Matthijs
  • Amidei, Jacopo
  • Mayol, Laia
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We take a close look at a recent dataset of TED-talks annotated with the questions they implicitly evoke, TED-Q (Westera et al., 2020). We test to what extent the relation between a discourse and the questions it evokes is merely one of similarity or association, as opposed to deeper semantic/pragmatic interpretation. We do so by turning the TED-Q dataset into a binary classification task, constructing an analogous task from explicit questions we extract from the BookCorpus (Zhu et al., 2015), and fitting a BERT-based classifier alongside models based on different notions of similarity. The BERT-based classifier, achieving close to human performance, outperforms all similarity-based models, suggesting that there is more to identifying true evoked questions than plain similarity., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A closer look at scalar diversity using contextualized semantic similarity

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a la 24th Sinn und Bedeutung Conference, SuB24, celebrada del 4 al 6 de setembre de 2019 a la Universitat d'Osnabrück, Alemania., We take a closer look at van Tiel et al.’s (2016) experimental results on diversity in scalar inference rates. In contrast to their finding that semantic similarity had no significant effect on scalar inference rates, we show that a sufficiently fine-grained notion of semantic similarity does have an effect: the more similar the two terms on a scale, the lower the scalar inference rate. Moreover, we show that a context-sensitive notion of semantic similarity (in particular ELMo; Peters et al., 2018) can explain more of the variance in the data, but only modestly, only for stimuli that contain informative context words, and only when the scalar terms themselves are sufficiently context-sensitive., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains. This project has also received funding from the Ramon y Cajal programme (grant RYC-2015-18907) and from the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

When do languages use the same word for different meanings? The Goldilocks principle in colexification

  • Brochhagen, Thomas
  • Boleda, Gemma
Lexical ambiguity is pervasive in language, and often systematic. For instance, the Spanish word dedo can refer to a toe or a finger, that is, these two meanings colexify in Spanish; and they do so as well in over one hundred other languages. Previous work shows that related meanings are more likely to colexify. This is attributed to cognitive pressure towards simplicity in language, as it makes lexicons easier to learn and use. The present study examines the interplay between this pressure and the competing pressure for languages to support accurate information transfer. We hypothesize that colexification follows a Goldilocks principle that balances the two pressures: meanings are more likely to attach to the same word when they are related to an optimal degree—neither too much, nor too little. We find support for this principle in data from over 1200 languages and 1400 meanings. Our results thus suggest that universal forces shape the lexicons of natural languages. More broadly, they contribute to the growing body of evidence suggesting that languages evolve to strike a balance between competing functional and cognitive pressures., This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 715154).