Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 60
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Similarity or deeper understanding?: analyzing the TED-Q dataset of evoked questions

  • Westera, Matthijs
  • Amidei, Jacopo
  • Mayol, Laia
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We take a close look at a recent dataset of TED-talks annotated with the questions they implicitly evoke, TED-Q (Westera et al., 2020). We test to what extent the relation between a discourse and the questions it evokes is merely one of similarity or association, as opposed to deeper semantic/pragmatic interpretation. We do so by turning the TED-Q dataset into a binary classification task, constructing an analogous task from explicit questions we extract from the BookCorpus (Zhu et al., 2015), and fitting a BERT-based classifier alongside models based on different notions of similarity. The BERT-based classifier, achieving close to human performance, outperforms all similarity-based models, suggesting that there is more to identifying true evoked questions than plain similarity., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Living a discrete life in a continuous world: reference with distributed representations

  • Boleda, Gemma
  • Padó, Sebastian
  • Pham, Nghia The
  • Baroni, Marco
Comunicació presentada a IWCS 2017, celebrat a Montpellier (França) del 19 al 22 de setembre de 2017., Reference is the crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. We propose a fully data-driven, end-toend trainable model that, while operating on continuous multimodal representations, learns to organize them into a discrete-like entity library. We also introduce a referential task to test it, cross-modal tracking. Our model beats standard neural network architectures, but is outperformed by some parametrizations of Memory Networks, another model with external memory., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE); Marie Sklodowska-Curie grant agreement No 655577 (LOVe); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Modeling word interpretation with deep language models: the interaction between expectations and lexical information

  • Aina, Laura
  • Brochhagen, Thomas
  • Boleda, Gemma
How a word is interpreted depends on the context it appears in. We study word interpretation leveraging deep language models, tracing the contribution and interaction of two sources of information that have been shown to be central to it: contextinvariant lexical information, represented by the word embeddings of a model, and a listener’s contextual expectations, represented by its predictions. We define operations to combine these components to obtain representations of word interpretations. We instantiate our framework using two English language models, and evaluate the resulting representations in the extent by which they reflect contextual word substitutes provided by human subjects. Our results suggest that both lexi cal information and expectations codify information pivotal to word interpretation; however, their combination is better than either on its own. Moreover, the division of labor between expectations and the lexicon appears to change across contexts, This project has re ceived funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and inno vation programme (grant agreement No 715154), and from the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research, and the computer resources at CTE-POWER and the technical support pro vided by Barcelona Supercomputing Center (RES-IM2019-3-0006). This paper reflects the authors view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Controlled tasks for model analysis: retrieving discrete information from sequences

  • Sorodoc, Ionut-Teodor
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada al 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP celebrat l'11 de novembre de 2021 de manera virtual., In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

​What do entity-centric models learn? Insights from entity linking in multi-party dialogue

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why. In this paper we analyze the behavior of two recently proposed entity-centric models in a referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entitycentric naturally fosters good architectural decisions. However, we also show that these models do not really build entity representations and that they make poor use of linguistic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when deployed., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Deep daxes: mutual exclusivity arises through both learning biases and pragmatic strategies in neural networks

  • Gulordava, Kristina
  • Brochhagen, Thomas
  • Boleda, Gemma
Children’s tendency to associate novel words with novel referents has been taken to reflect a bias toward mutual exclusivity. This tendency may be advantageous both as (1) an ad-hoc referent selection heuristic to single out referents lacking a label and as (2) an organizing principle of lexical acquisition. This paper investigates under which circumstances cross-situational neural models can come to exhibit analogous behavior to children, focusing on these two possibilities and their interaction. To this end, we evaluate neural networks’ on both symbolic data and, as a first, on large-scale image data. We find that constraints in both learning and selection can foster mutual exclusivity, as long as they put words in competition for lexical meaning. For computational models, these findings clarify the role of available options for better performance in tasks where mutual exclusivity is advantageous. For cognitive research, they highlight latent interactions between word learning, referent selection mechanisms, and the structure of stimuli of varying complexity: symbolic and visual, This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramon´y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM-2019-3-0006). We are grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Grounding semantic roles in images

  • Silberer, Carina
  • Pinkal, Manfred
Comunicació presentada a la 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), celebrada del 31 d'octubre al 4 de novembre de 2018 a Brussel·les, Bèlgica., We address the task of visual semantic role labeling (vSRL), the identification of the participants of a situation or event in a visual scene, and their labeling with their semantic relations to the event or situation. We render candidate participants as image regions of objects, and train a model which learns to ground roles in the regions which depict the corresponding participant. Experimental results demonstrate that we can train a vSRL model without reliance on prohibitive image-based role annotations, by utilizing noisy data which we extract automatically from image captions using a linguistic SRL system. Furthermore, our model induces frame–semantic visual representations, and their comparison to previous work on supervised visual verb sense disambiguation yields overall better results., We thank the anonymous reviewers, Leonie Harter, Christine Sch¨afer, Michael Roth, Gemma Boleda, Anna Rohrbach and Bernt Schiele. This research was supported by the German Research Foundation (DFG EXC 285), by the European Research Council (ERC Horizon 2020 grant agreement No 715154), and the Spanish Ramon y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la Conference on Empirical Methods in Natural Language Processing, celebrada els dies 31 d'octubre a 4 de novembre de 2020 a Brussel·les, Bèlgica., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., We thank German Kruszewski and the AMORE ´ team for the helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramon y Cajal programme (grant RYC-2015- ´ 18907) and the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Distributed prediction of relations for entities: the Easy, the Difficult, and the impossible

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a la 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), celebrat els dies 3 i 4 d'agost de 2017 a Vancouver, Canada., Word embeddings are supposed to provide easy access to semantic relations such as “male of” (man–woman). While this claim has been investigated for concepts, little is known about the distributional behavior of relations of (Named) Entities. We describe two word embedding-based models that predict values for relational attributes of entities, and analyse them. The task is challenging, with major performance differences between relations. Contrary to many NLP tasks, high difficulty for a relation does not result from low frequency, but from (a) one-to-many mappings; and (b) lack of context patterns expressing the relation that are easy to pick up by word embeddings., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and the DFG (SFB 732, Project D10).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

“Show me the cup”: reference with continuous representations

  • Boleda, Gemma
  • Baroni, Marco
  • Padó, Sebastian
Comunicació presentada a CICLing 2017, celebrat a Budapest (Hongria) del 17 al 23 d'abril de 2017., One of the most basic functions of language is to refer to objects in a shared scene. Modeling reference with continuous representations is challenging because it requires individuation, i.e., tracking and distinguishing an arbitrary number of referents. We introduce a neural network model that, given a de nite description and a set of objects represented by natural images, points to the intended object if the expression has a unique referent, or indicates a failure, if it does not. The model, directly trained on reference acts, is competitive with a pipeline manually engineered to perform the same task, both when referents are purely visual, and when they are characterized by a combination of visual and linguistic properties., This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and ERC grant agreement No 715154 (AMORE); ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

AMORE-UPF at SemEval-2018 Task 4: BiLSTM with entity library

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada al 12th International Workshop on Semantic Evaluation (SemEval-2018), celebrat els dies 5 i 6 de juny de 2018 a Nova Orleans, EUA., This paper describes our winning contribution to SemEval 2018 Task 4: Character Identification on Multiparty Dialogues. It is a simple, standard model with one key innovation, an entity library. Our results show that this innovation greatly facilitates the identification of infrequent characters. Because of the generic nature of our model, this finding is potentially relevant to any task that requires effective learning from sparse or unbalanced data., This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Challenges in including extra-linguistic context in pre-trained language models

  • Sorodoc, Ionut-Teodor
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a Third Workshop on Insights from Negative Results in NLP (Insights 2022), celebrat el 26 de maig de 2022 a Dublín, Irlanda., To successfully account for language, computational models need to take into account both the linguistic context (the content of the utterances) and the extra-linguistic context (for instance, the participants in a dialogue). We focus on a referential task that asks models to link entity mentions in a TV show to the corresponding characters, and design an architecture that attempts to account for both kinds of context. In particular, our architecture combines a previously proposed specialized module (an “entity library”) for character representation with transfer learning from a pre-trained language model. We find that, although the model does improve linguistic contextualization, it fails to successfully integrate extra-linguistic information about the participants in the dialogue. Our work shows that it is very challenging to incorporate extra-linguistic information into pretrained language models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Asking between the lines: elicitation of evoked questions in text

  • Westera, Matthijs
  • Rohde, Hannah
Comunicació presentada al Amsterdam Colloquium 2019, celebrat del 18 al 20 de desembre de 2019 a Amsterdam, Holanda., We introduce a novel, scalable method aimed at annotating potential and actual Questions Under Discussion (QUDs) in naturalistic discourse. It consists of asking naive participants first what questions a certain portion of the discourse evokes for them and subsequently which of those end up being answered as the discourse proceeds. This paper outlines the method and design decisions that went into it and on characterizing highlevel properties of the resulting data. We highlight ways in which the data gathered via our method could inform our understanding of QUD-driven phenomena and QUD models themselves. We also provide access to a visualization tool for viewing the evoked questions we gathered using this method (N=4765 from 111 crowdsourced annotators)., This work was supported in part by a Leverhulme Trust Prize in Languages and Literatures to H. Rohde. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A distributional study of negated adjectives and antonyms

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
Comunicació presentada al Fifth Italian Conference on Computational Linguistics (CLiC-it 2018); celebrada del 10 al 12 de desembre de 2018 a Torí, Itàlia., In this paper, we investigate the relation between negated adjectives and antonyms in English using Distributional Semantics methods. Results show that, on the basis of contexts of use, a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot); such effect is less strong for antonyms derived by affixation (e.g., happy - unhappy). In questo lavoro, analizziamo la relazione fra aggettivi negati e antonimi in inglese utilizzando metodi di Semantica Distribuzionale. I risultati mostrano che, sulla base dei contesti di uso, la negazione di un aggettivo (ad es. “not cold”; it.: “non freddo”) e tipicamente ` piu simile all’aggettivo stesso (“cold”; it.: ` “freddo”) che al suo antonimo (“hot”; it.: “caldo”). Tale effetto e meno accentuato ` per antonimi derivati tramite affissi (ad es. “happy”-“unhappy”; it.: “felice”- “infelice”)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Probing for referential information in language models

  • Sorodoc, Ionut-Teodor
  • Gulordava, Kristina
  • Boleda, Gemma
Comunicació presentada al 58th Annual Meeting of the Association for Computational Linguistics celebrat del 5 al 10 de juliol de 2020 de manera virtual., Language models keep track of complex information about the preceding context – including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, via probe tasks and analysis on a coreference annotated corpus. The Transformer outperforms the LSTM in all analyses. Our results suggest that language models are more successful at learning grammatical constraints than they are at learning truly referential information, in the sense of capturing the fact that we use language to refer to entities in the world. However, we find traces of the latter aspect, too., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM2019-3-0006).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

TED-Q: TED talks and the questions they evoke

  • Westera, Matthijs
  • Mayol, Laia
  • Rohde, Hannah
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., We present a new dataset of TED-talks annotated with the questions they evoke and, where available, the answers to these questions. Evoked questions represent a hitherto mostly unexplored type of linguistic data, which promises to open up important new lines of research, especially related to the Question Under Discussion (QUD)-based approach to discourse structure. In this paper we introduce the method and open the first installment of our data to the public. We summarize and explore the current dataset, illustrate its potential by providing new evidence for the relation between predictability and implicitness – capitalizing on the already existing PDTB-style annotations for the texts we use – and outline its potential for future research. The dataset should be of interest, at its current scale, to researchers on formal and experimental pragmatics, discourse coherence, information structure, discourse expectations and processing. Our data-gathering procedure is designed to scale up, relying on crowdsourcing by non-expert annotators, with its utility for Natural Language Processing in mind (e.g., dialogue systems, conversational question answering)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Comparatives, quantifiers, proportions: a multi-task model for the learning of quantities from vision

  • Sorodoc, Ionut-Teodor
  • Pezzelle, Sandro
  • Bernardi, Raffaella
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), celebrada els dies 1 a 6 de juny de 2018 a Nova Orleans, Estats Units d'Amèrica., The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual scenes by a multi-task computational model. The motivation is that, in humans, these processes underlie the same cognitive, nonsymbolic ability, which allows an automatic estimation and comparison of set magnitudes. We show that when information about lowercomplexity tasks is available, the higher-level proportional task becomes more accurate than when performed in isolation. Moreover, the multi-task model is able to generalize to unseen combinations of target/non-target objects. Consistently with behavioral evidence showing the interference of absolute number in the proportional task, the multi-task model no longer works when asked to provide the number of target objects in the scene., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Humans meet models on object naming: a new dataset and analysis

  • Silberer, Carina
  • Zarrieß, Sina
  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We release ManyNames v2 (MN v2), a verified version of an object naming dataset that contains dozens of valid names per object for 25K images. We analyze issues in the data collection method originally employed, standard in Language & Vision (L&V), and find that the main source of noise in the data comes from simulating a naming context solely from an image with a target object marked with a bounding box, which causes subjects to sometimes disagree regarding which object is the target. We also find that both the degree of this uncertainty in the original data and the amount of true naming variation in MN v2 differs substantially across object domains. We use MN v2 to analyze a popular L&V model and demonstrate its effectiveness on the task of object naming. However, our fine-grained analysis reveals that what appears to be human-like model behavior is not stable across domains, e.g., the model confuses people and clothing objects much more frequently than humans do. We also find that standard evaluations underestimate the actual effectiveness of the naming model: on the single-label names of the original dataset (Visual Genome), it obtains −27% accuracy points than on MN v2, that includes all valid object names., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Run Like a Girl! Sport-Related Gender Bias in Language and Vision

  • Harrison, Sophie
  • Gualdoni, Eleonora
  • Boleda, Gemma
Gender bias in Language and Vision datasets and models has the potential to perpetuate harmful stereotypes and discrimination. We analyze gender bias in two Language and Vision datasets. Consistent with prior work, we find that both datasets underrepresent women, which promotes their invisibilization. Moreover, we hypothesize and find that a bias affects human naming choices for people playing sports: speakers produce names indicating the sport (e.g. ‘tennis player’ or ‘surfer’) more often when it is a man or a boy participating in the sport than when it is a woman or a girl, with an average of 46% vs. 35% of sports-related names for each gender. A computational model trained on these naming data reproduces the bias. We argue that both the data and the model result in representational harm against women., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain) and of grant agreement No. 715154 funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Object naming in language and vision: a survey and a new dataset

  • Silberer, Carina
  • Zarrieß, Sina
  • Boleda, Gemma
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., People choose particular names for objects, such as dog or puppy for a given dog. Object naming has been studied in Psycholinguistics, but has received relatively little attention in Computational Linguistics. We review resources from Language and Vision that could be used to study object naming on a large scale, discuss their shortcomings, and create a new dataset that affords more opportunities for analysis and modeling. Our dataset, ManyNames, provides 36 name annotations for each of 25K objects in images selected from VisualGenome. We highlight the challenges involved and provide a preliminary analysis of the ManyNames data, showing that there is a high level of agreement in naming, on average. At the same time, the average number of name types associated with an object is much higher in our dataset than in existing corpora for Language and Vision, such that ManyNames provides a rich resource for studying phenomena like hierarchical variation (chihuahua vs. dog), which has been discussed at length in the theoretical literature, and other less well studied phenomena like cross-classification (cake vs. dessert)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Putting words in context: LSTM language models and lexical ambiguity

  • Boleda, Gemma
  • Gulordava, Kristina
  • Aina, Laura
Comunicació presentada al 57th Annual Meeting of the Association for Computational Linguistic (ACL 2019), celebrat els dies 28 de juliol a 2 d'agost de 2019 a Florència, Itàlia., In neural network models of language, words are commonly represented using context invariant representations (word embeddings) which are then put in context in the hidden layers. Since words are often ambiguous, representing the contextually relevant information is not trivial. We investigate how an LSTM language model deals with lexical ambiguity in English, designing a method to probe its hidden representations for lexical and contextual information about words. We find that both types of information are represented to a large extent, but also that there is room for improvement for contextual information., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Does referent predictability affect the choice of referential form? A computational approach using masked coreference resolution

  • Aina, Laura
  • Liao, Xixian
  • Boleda, Gemma
  • Westera, Matthijs
Comunicació presentada a 25th Conference on Computational Natural Language Learning (CoNLL 2021) celebrada en línia del 10 al 11 de 2021, It is often posited that more predictable parts of a speaker’s meaning tend to be made less explicit, for instance using shorter, less informative words. Studying these dynamics in the domain of referring expressions has proven difficult, with existing studies, both psycholinguistic and corpus-based, providing contradictory results. We test the hypothesis that speakers produce less informative referring expressions (e.g., pronouns vs. full noun phrases) when the context is more informative about the referent, using novel computational estimates of referent predictability. We obtain these estimates training an existing coreference resolution system for English on a new task, masked coreference resolution, giving us a probability distribution over referents that is conditioned on the context but not the referring expression. The resulting system retains standard coreference resolution performance while yielding a better estimate of human-derived referent predictability than previous attempts. A statistical analysis of the relationship between model output and mention form supports the hypothesis that predictability affects the form of a mention, both its morphosyntactic type and its length., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Instances and concepts in distributional space

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a l'European Chapter of the Association for Computational Linguistics (EACL) celebrat a València, Espanya, del 2 al 6 de juny de 2017., Instances (“Mozart”) are ontologically distinct from concepts or classes (“composer”). Natural language encompasses both, but instances have received comparatively little attention in distributional semantics. Our results show that instances and concepts differ in their distributional properties. We also establish that instantiation detection (“Mozart – composer”) is generally easier than hypernymy detection (“chemist – scientist”), and that results on the influence of input representation do not transfer from hyponymy to instantiation., The authors have received funding from DFG (SFB 732, project B9). and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE), as well as under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Quantifying informativeness of names in visual space

  • Gualdoni, Eleonora
  • Kemp, Charles
  • Xu, Yang
  • Boleda, Gemma
Comunicació presentada a 45th Annual Conference of the Cognitive Science Society (CogSci 2023), celebrada a Sydney (Austràlia) del 26 al 29 de juliol de 2023., The human lexicon expresses a wide array of concepts with a limited set of words. Previous work has suggested that semantic categories are structured compactly to enable informative communication. Informativeness is typically quantified with respect to an entire semantic domain and not at the level of individual names. We develop a measure of name informativeness using an information-theoretic framework grounded in visual object representations derived from natural images. Our approach uses computer vision models to characterize informativeness of individual names with respect to large-scale data in a naturalistic setting. We show that our informativeness measure predicts degrees of specificity in lexical categories more precisely than alternative measures based on entropy and frequency. We also show that name informativeness jointly captures within-category similarity and distinctiveness across categories. Our analyses suggest how the variability of names from a broad part of the lexicon may be understood through the lens of information theory., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovacion and the Agencia Estatal ´ de Investigacion (Spain), of grant agreement No. 715154 ´ funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme, and of NSERC Alliance International-Catalyst Grant ALLRP 576149-22. The second author was supported by grant FT190100200 from the Australian Research Council.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Don’t blame distributional semantics if it can’t do entailment

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a 13th International Conference on Computational Semantics (IWCS 2019), celebrat del 23 al 27 de maig de 2019 a Göteborg, Suècia., Distributional semantics has had enormous empirical success in Computational Linguistics and Cognitive Science in modeling various semantic phenomena, such as semantic similarity, and distributional models are widely used in state-of-the-art Natural Language Processing systems. However, the theoretical status of distributional semantics within a broader theory of language and cognition is still unclear: What does distributional semantics model? Can it be, on its own, a fully adequate model of the meanings of linguistic expressions? The standard answer is that distributional semantics is not fully adequate in this regard, because it falls short on some of the central aspects of formal semantic approaches: truth conditions, entailment, reference, and certain aspects of compositionality. We argue that this standard answer rests on a misconception: These aspects do not belong in a theory of expression meaning, they are instead aspects of speaker meaning, i.e., communicative intentions in a particular context. In a slogan: words do not refer, speakers do. Clearing this up enables us to argue that distributional semantics on its own is an adequate model of expression meaning. Our proposal sheds light on the role of distributional semantics in a broader theory of language and cognition, its relationship to formal semantics, and its place in computational models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

QUDs, brevity, and the asymmetry of alternatives

  • Westera, Matthijs
Comunicació presentada al Amsterdam Colloquium 2017, celebrat a Amsterdam (Holanda) del 20 a 22 de desembre de 2017, Exhaustivity is typically explained in terms of the exclusion of unmentioned alternatives. For this to work, the set of alternatives must be asymmetrical, lest both a proposition and its negation get excluded, yielding a contradiction (the Symmetry Problem). Since exhaustivity is regularly observed, these alternative sets must tend to be asymmetrical, and this requires an explanation. Existing explanations are based on considerations of brevity, but these run into certain problems. A new solution is proposed, explaining the asymmetry of alternatives in terms of the fact that discourse strategies with asymmetrical questions under discussion (Quds) are favored because they allow part of the answer to be communicated implicitly, namely as an exhaustivity implicature., Many thanks to Floris Roelofsen and Jeroen Groenendijk for their comments on many iterations of this work. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Short-term meaning shift: a distributional exploration

  • Boleda, Gemma
  • Del Tredici, Marco
  • Fernández, Raquel
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., We present the first exploration of meaning shift over short periods of time in online communities using distributional representations. We create a small annotated dataset and use it to assess the performance of a standard model for meaning shift detection on shortterm meaning shift. We find that the model has problems distinguishing meaning shift from referential phenomena, and propose a measure of contextual variability to remedy this., The research carried out by the Amsterdam section of the team was partially funded by the Netherlands Organisation for Scientific Research (NWO) under VIDI grant no. 276-89-008, Asymmetry in Conversation. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Quantifying informativeness of names in visual space

  • Gualdoni, Eleonora
  • Kemp, Charles
  • Xu, Yang
  • Boleda, Gemma
Comunicació presentada a 45th Annual Conference of the Cognitive Science Society (CogSci 2023), celebrada a Sydney (Austràlia) del 26 al 29 de juliol de 2023., The human lexicon expresses a wide array of concepts with a limited set of words. Previous work has suggested that semantic categories are structured compactly to enable informative communication. Informativeness is typically quantified with respect to an entire semantic domain and not at the level of individual names. We develop a measure of name informativeness using an information-theoretic framework grounded in visual object representations derived from natural images. Our approach uses computer vision models to characterize informativeness of individual names with respect to large-scale data in a naturalistic setting. We show that our informativeness measure predicts degrees of specificity in lexical categories more precisely than alternative measures based on entropy and frequency. We also show that name informativeness jointly captures within-category similarity and distinctiveness across categories. Our analyses suggest how the variability of names from a broad part of the lexicon may be understood through the lens of information theory., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovacion and the Agencia Estatal ´ de Investigacion (Spain), of grant agreement No. 715154 ´ funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme, and of NSERC Alliance International-Catalyst Grant ALLRP 576149-22. The second author was supported by grant FT190100200 from the Australian Research Council.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Don’t blame distributional semantics if it can’t do entailment

  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada a 13th International Conference on Computational Semantics (IWCS 2019), celebrat del 23 al 27 de maig de 2019 a Göteborg, Suècia., Distributional semantics has had enormous empirical success in Computational Linguistics and Cognitive Science in modeling various semantic phenomena, such as semantic similarity, and distributional models are widely used in state-of-the-art Natural Language Processing systems. However, the theoretical status of distributional semantics within a broader theory of language and cognition is still unclear: What does distributional semantics model? Can it be, on its own, a fully adequate model of the meanings of linguistic expressions? The standard answer is that distributional semantics is not fully adequate in this regard, because it falls short on some of the central aspects of formal semantic approaches: truth conditions, entailment, reference, and certain aspects of compositionality. We argue that this standard answer rests on a misconception: These aspects do not belong in a theory of expression meaning, they are instead aspects of speaker meaning, i.e., communicative intentions in a particular context. In a slogan: words do not refer, speakers do. Clearing this up enables us to argue that distributional semantics on its own is an adequate model of expression meaning. Our proposal sheds light on the role of distributional semantics in a broader theory of language and cognition, its relationship to formal semantics, and its place in computational models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Comparatives, quantifiers, proportions: a multi-task model for the learning of quantities from vision

  • Sorodoc, Ionut-Teodor
  • Pezzelle, Sandro
  • Bernardi, Raffaella
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), celebrada els dies 1 a 6 de juny de 2018 a Nova Orleans, Estats Units d'Amèrica., The present work investigates whether different quantification mechanisms (set comparison, vague quantification, and proportional estimation) can be jointly learned from visual scenes by a multi-task computational model. The motivation is that, in humans, these processes underlie the same cognitive, nonsymbolic ability, which allows an automatic estimation and comparison of set magnitudes. We show that when information about lowercomplexity tasks is available, the higher-level proportional task becomes more accurate than when performed in isolation. Moreover, the multi-task model is able to generalize to unseen combinations of target/non-target objects. Consistently with behavioral evidence showing the interference of absolute number in the proportional task, the multi-task model no longer works when asked to provide the number of target objects in the scene., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Modeling word interpretation with deep language models: the interaction between expectations and lexical information

  • Aina, Laura
  • Brochhagen, Thomas
  • Boleda, Gemma
How a word is interpreted depends on the context it appears in. We study word interpretation leveraging deep language models, tracing the contribution and interaction of two sources of information that have been shown to be central to it: contextinvariant lexical information, represented by the word embeddings of a model, and a listener’s contextual expectations, represented by its predictions. We define operations to combine these components to obtain representations of word interpretations. We instantiate our framework using two English language models, and evaluate the resulting representations in the extent by which they reflect contextual word substitutes provided by human subjects. Our results suggest that both lexi cal information and expectations codify information pivotal to word interpretation; however, their combination is better than either on its own. Moreover, the division of labor between expectations and the lexicon appears to change across contexts, This project has re ceived funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and inno vation programme (grant agreement No 715154), and from the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research, and the computer resources at CTE-POWER and the technical support pro vided by Barcelona Supercomputing Center (RES-IM2019-3-0006). This paper reflects the authors view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Does referent predictability affect the choice of referential form? A computational approach using masked coreference resolution

  • Aina, Laura
  • Liao, Xixian
  • Boleda, Gemma
  • Westera, Matthijs
Comunicació presentada a 25th Conference on Computational Natural Language Learning (CoNLL 2021) celebrada en línia del 10 al 11 de 2021, It is often posited that more predictable parts of a speaker’s meaning tend to be made less explicit, for instance using shorter, less informative words. Studying these dynamics in the domain of referring expressions has proven difficult, with existing studies, both psycholinguistic and corpus-based, providing contradictory results. We test the hypothesis that speakers produce less informative referring expressions (e.g., pronouns vs. full noun phrases) when the context is more informative about the referent, using novel computational estimates of referent predictability. We obtain these estimates training an existing coreference resolution system for English on a new task, masked coreference resolution, giving us a probability distribution over referents that is conditioned on the context but not the referring expression. The resulting system retains standard coreference resolution performance while yielding a better estimate of human-derived referent predictability than previous attempts. A statistical analysis of the relationship between model output and mention form supports the hypothesis that predictability affects the form of a mention, both its morphosyntactic type and its length., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Living a discrete life in a continuous world: reference with distributed representations

  • Boleda, Gemma
  • Padó, Sebastian
  • Pham, Nghia The
  • Baroni, Marco
Comunicació presentada a IWCS 2017, celebrat a Montpellier (França) del 19 al 22 de setembre de 2017., Reference is the crucial property of language that allows us to connect linguistic expressions to the world. Modeling it requires handling both continuous and discrete aspects of meaning. Data-driven models excel at the former, but struggle with the latter, and the reverse is true for symbolic models. We propose a fully data-driven, end-toend trainable model that, while operating on continuous multimodal representations, learns to organize them into a discrete-like entity library. We also introduce a referential task to test it, cross-modal tracking. Our model beats standard neural network architectures, but is outperformed by some parametrizations of Memory Networks, another model with external memory., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE); Marie Sklodowska-Curie grant agreement No 655577 (LOVe); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la EMNLP 2018 Conference on Empirical Methods in Natural Language Processing, celebrada a Brussel·les (Bèlgica), del 31 d'octubre al 4 de novembre de 2018., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., This project as received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907) and the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Probing for referential information in language models

  • Sorodoc, Ionut-Teodor
  • Gulordava, Kristina
  • Boleda, Gemma
Comunicació presentada al 58th Annual Meeting of the Association for Computational Linguistics celebrat del 5 al 10 de juliol de 2020 de manera virtual., Language models keep track of complex information about the preceding context – including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, via probe tasks and analysis on a coreference annotated corpus. The Transformer outperforms the LSTM in all analyses. Our results suggest that language models are more successful at learning grammatical constraints than they are at learning truly referential information, in the sense of capturing the fact that we use language to refer to entities in the world. However, we find traces of the latter aspect, too., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM2019-3-0006).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Deep daxes: mutual exclusivity arises through both learning biases and pragmatic strategies in neural networks

  • Gulordava, Kristina
  • Brochhagen, Thomas
  • Boleda, Gemma
Children’s tendency to associate novel words with novel referents has been taken to reflect a bias toward mutual exclusivity. This tendency may be advantageous both as (1) an ad-hoc referent selection heuristic to single out referents lacking a label and as (2) an organizing principle of lexical acquisition. This paper investigates under which circumstances cross-situational neural models can come to exhibit analogous behavior to children, focusing on these two possibilities and their interaction. To this end, we evaluate neural networks’ on both symbolic data and, as a first, on large-scale image data. We find that constraints in both learning and selection can foster mutual exclusivity, as long as they put words in competition for lexical meaning. For computational models, these findings clarify the role of available options for better performance in tasks where mutual exclusivity is advantageous. For cognitive research, they highlight latent interactions between word learning, referent selection mechanisms, and the structure of stimuli of varying complexity: symbolic and visual, This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramon´y Cajal programme (grant RYC-2015-18907). We thankfully acknowledge the computer resources at CTE-POWER and the technical support provided by Barcelona Supercomputing Center (RES-IM-2019-3-0006). We are grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

QUDs, brevity, and the asymmetry of alternatives

  • Westera, Matthijs
Comunicació presentada al Amsterdam Colloquium 2017, celebrat a Amsterdam (Holanda) del 20 a 22 de desembre de 2017, Exhaustivity is typically explained in terms of the exclusion of unmentioned alternatives. For this to work, the set of alternatives must be asymmetrical, lest both a proposition and its negation get excluded, yielding a contradiction (the Symmetry Problem). Since exhaustivity is regularly observed, these alternative sets must tend to be asymmetrical, and this requires an explanation. Existing explanations are based on considerations of brevity, but these run into certain problems. A new solution is proposed, explaining the asymmetry of alternatives in terms of the fact that discourse strategies with asymmetrical questions under discussion (Quds) are favored because they allow part of the answer to be communicated implicitly, namely as an exhaustivity implicature., Many thanks to Floris Roelofsen and Jeroen Groenendijk for their comments on many iterations of this work. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154). This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Humans meet models on object naming: a new dataset and analysis

  • Silberer, Carina
  • Zarrieß, Sina
  • Westera, Matthijs
  • Boleda, Gemma
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We release ManyNames v2 (MN v2), a verified version of an object naming dataset that contains dozens of valid names per object for 25K images. We analyze issues in the data collection method originally employed, standard in Language & Vision (L&V), and find that the main source of noise in the data comes from simulating a naming context solely from an image with a target object marked with a bounding box, which causes subjects to sometimes disagree regarding which object is the target. We also find that both the degree of this uncertainty in the original data and the amount of true naming variation in MN v2 differs substantially across object domains. We use MN v2 to analyze a popular L&V model and demonstrate its effectiveness on the task of object naming. However, our fine-grained analysis reveals that what appears to be human-like model behavior is not stable across domains, e.g., the model confuses people and clothing objects much more frequently than humans do. We also find that standard evaluations underestimate the actual effectiveness of the naming model: on the single-label names of the original dataset (Visual Genome), it obtains −27% accuracy points than on MN v2, that includes all valid object names., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Challenges in including extra-linguistic context in pre-trained language models

  • Sorodoc, Ionut-Teodor
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a Third Workshop on Insights from Negative Results in NLP (Insights 2022), celebrat el 26 de maig de 2022 a Dublín, Irlanda., To successfully account for language, computational models need to take into account both the linguistic context (the content of the utterances) and the extra-linguistic context (for instance, the participants in a dialogue). We focus on a referential task that asks models to link entity mentions in a TV show to the corresponding characters, and design an architecture that attempts to account for both kinds of context. In particular, our architecture combines a previously proposed specialized module (an “entity library”) for character representation with transfer learning from a pre-trained language model. We find that, although the model does improve linguistic contextualization, it fails to successfully integrate extra-linguistic information about the participants in the dialogue. Our work shows that it is very challenging to incorporate extra-linguistic information into pretrained language models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Distributed prediction of relations for entities: the Easy, the Difficult, and the impossible

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a la 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), celebrat els dies 3 i 4 d'agost de 2017 a Vancouver, Canada., Word embeddings are supposed to provide easy access to semantic relations such as “male of” (man–woman). While this claim has been investigated for concepts, little is known about the distributional behavior of relations of (Named) Entities. We describe two word embedding-based models that predict values for relational attributes of entities, and analyse them. The task is challenging, with major performance differences between relations. Contrary to many NLP tasks, high difficulty for a relation does not result from low frequency, but from (a) one-to-many mappings; and (b) lack of context patterns expressing the relation that are easy to pick up by word embeddings., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and the DFG (SFB 732, Project D10).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Run Like a Girl! Sport-Related Gender Bias in Language and Vision

  • Harrison, Sophie
  • Gualdoni, Eleonora
  • Boleda, Gemma
Gender bias in Language and Vision datasets and models has the potential to perpetuate harmful stereotypes and discrimination. We analyze gender bias in two Language and Vision datasets. Consistent with prior work, we find that both datasets underrepresent women, which promotes their invisibilization. Moreover, we hypothesize and find that a bias affects human naming choices for people playing sports: speakers produce names indicating the sport (e.g. ‘tennis player’ or ‘surfer’) more often when it is a man or a boy participating in the sport than when it is a woman or a girl, with an average of 46% vs. 35% of sports-related names for each gender. A computational model trained on these naming data reproduces the bias. We argue that both the data and the model result in representational harm against women., This research is an output of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain) and of grant agreement No. 715154 funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Object naming in language and vision: a survey and a new dataset

  • Silberer, Carina
  • Zarrieß, Sina
  • Boleda, Gemma
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., People choose particular names for objects, such as dog or puppy for a given dog. Object naming has been studied in Psycholinguistics, but has received relatively little attention in Computational Linguistics. We review resources from Language and Vision that could be used to study object naming on a large scale, discuss their shortcomings, and create a new dataset that affords more opportunities for analysis and modeling. Our dataset, ManyNames, provides 36 name annotations for each of 25K objects in images selected from VisualGenome. We highlight the challenges involved and provide a preliminary analysis of the ManyNames data, showing that there is a high level of agreement in naming, on average. At the same time, the average number of name types associated with an object is much higher in our dataset than in existing corpora for Language and Vision, such that ManyNames provides a rich resource for studying phenomena like hierarchical variation (chihuahua vs. dog), which has been discussed at length in the theoretical literature, and other less well studied phenomena like cross-classification (cake vs. dessert)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No715154) and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Instances and concepts in distributional space

  • Boleda, Gemma
  • Gupta, Abhijeet
  • Padó, Sebastian
Comunicació presentada a l'European Chapter of the Association for Computational Linguistics (EACL) celebrat a València, Espanya, del 2 al 6 de juny de 2017., Instances (“Mozart”) are ontologically distinct from concepts or classes (“composer”). Natural language encompasses both, but instances have received comparatively little attention in distributional semantics. Our results show that instances and concepts differ in their distributional properties. We also establish that instantiation detection (“Mozart – composer”) is generally easier than hypernymy detection (“chemist – scientist”), and that results on the influence of input representation do not transfer from hyponymy to instantiation., The authors have received funding from DFG (SFB 732, project B9). and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154; AMORE), as well as under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Putting words in context: LSTM language models and lexical ambiguity

  • Boleda, Gemma
  • Gulordava, Kristina
  • Aina, Laura
Comunicació presentada al 57th Annual Meeting of the Association for Computational Linguistic (ACL 2019), celebrat els dies 28 de juliol a 2 d'agost de 2019 a Florència, Itàlia., In neural network models of language, words are commonly represented using context invariant representations (word embeddings) which are then put in context in the hidden layers. Since words are often ambiguous, representing the contextually relevant information is not trivial. We investigate how an LSTM language model deals with lexical ambiguity in English, designing a method to probe its hidden representations for lexical and contextual information about words. We find that both types of information are represented to a large extent, but also that there is room for improvement for contextual information., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

“Show me the cup”: reference with continuous representations

  • Boleda, Gemma
  • Baroni, Marco
  • Padó, Sebastian
Comunicació presentada a CICLing 2017, celebrat a Budapest (Hongria) del 17 al 23 d'abril de 2017., One of the most basic functions of language is to refer to objects in a shared scene. Modeling reference with continuous representations is challenging because it requires individuation, i.e., tracking and distinguishing an arbitrary number of referents. We introduce a neural network model that, given a de nite description and a set of objects represented by natural images, points to the intended object if the expression has a unique referent, or indicates a failure, if it does not. The model, directly trained on reference acts, is competitive with a pipeline manually engineered to perform the same task, both when referents are purely visual, and when they are characterized by a combination of visual and linguistic properties., This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 655577 (LOVe) and ERC grant agreement No 715154 (AMORE); ERC 2011 Starting Independent Research Grant n. 283554 (COMPOSES); DFG (SFB 732, Project D10); and Spanish MINECO (grant FFI2013-41301-P).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Short-term meaning shift: a distributional exploration

  • Boleda, Gemma
  • Del Tredici, Marco
  • Fernández, Raquel
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., We present the first exploration of meaning shift over short periods of time in online communities using distributional representations. We create a small annotated dataset and use it to assess the performance of a standard model for meaning shift detection on shortterm meaning shift. We find that the model has problems distinguishing meaning shift from referential phenomena, and propose a measure of contextual variability to remedy this., The research carried out by the Amsterdam section of the team was partially funded by the Netherlands Organisation for Scientific Research (NWO) under VIDI grant no. 276-89-008, Asymmetry in Conversation. This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Controlled tasks for model analysis: retrieving discrete information from sequences

  • Sorodoc, Ionut-Teodor
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada al 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP celebrat l'11 de novembre de 2021 de manera virtual., In recent years, the NLP community has shown increasing interest in analysing how deep learning models work. Given that large models trained on complex tasks are difficult to inspect, some of this work has focused on controlled tasks that emulate specific aspects of language. We propose a new set of such controlled tasks to explore a crucial aspect of natural language processing that has not received enough attention: the need to retrieve discrete information from sequences. We also study model behavior on the tasks with simple instantiations of Transformers and LSTMs. Our results highlight the beneficial role of decoder attention and its sometimes unexpected interaction with other components. Moreover, we show that, for most of the tasks, these simple models still show significant difficulties. We hope that the community will take up the analysis possibilities that our tasks afford, and that a clearer understanding of model behavior on the tasks will lead to better and more transparent models., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154). We are also grateful to the NVIDIA Corporation for the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

​What do entity-centric models learn? Insights from entity linking in multi-party dialogue

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada a la Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), celebrada els dies 2 a 7 de juny de 2019 a Minneapolis, Estats Units d'Amèrica., Humans use language to refer to entities in the external world. Motivated by this, in recent years several models that incorporate a bias towards learning entity representations have been proposed. Such entity-centric models have shown empirical success, but we still know little about why. In this paper we analyze the behavior of two recently proposed entity-centric models in a referential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4). We show that these models outperform the state of the art on this task, and that they do better on lower frequency entities than a counterpart model that is not entity-centric, with the same model size. We argue that making models entitycentric naturally fosters good architectural decisions. However, we also show that these models do not really build entity representations and that they make poor use of linguistic context. These negative results underscore the need for model analysis, to test whether the motivations for particular architectures are borne out in how models behave when deployed., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A distributional study of negated adjectives and antonyms

  • Aina, Laura
  • Bernardi, Raffaella
  • Fernández, Raquel
Comunicació presentada al Fifth Italian Conference on Computational Linguistics (CLiC-it 2018); celebrada del 10 al 12 de desembre de 2018 a Torí, Itàlia., In this paper, we investigate the relation between negated adjectives and antonyms in English using Distributional Semantics methods. Results show that, on the basis of contexts of use, a negated adjective (e.g., not cold) is typically more similar to the adjective itself (cold) than to its antonym (hot); such effect is less strong for antonyms derived by affixation (e.g., happy - unhappy). In questo lavoro, analizziamo la relazione fra aggettivi negati e antonimi in inglese utilizzando metodi di Semantica Distribuzionale. I risultati mostrano che, sulla base dei contesti di uso, la negazione di un aggettivo (ad es. “not cold”; it.: “non freddo”) e tipicamente ` piu simile all’aggettivo stesso (“cold”; it.: ` “freddo”) che al suo antonimo (“hot”; it.: “caldo”). Tale effetto e meno accentuato ` per antonimi derivati tramite affissi (ad es. “happy”-“unhappy”; it.: “felice”- “infelice”)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and by the Catalan government (SGR 2017 1575).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Asking between the lines: elicitation of evoked questions in text

  • Westera, Matthijs
  • Rohde, Hannah
Comunicació presentada al Amsterdam Colloquium 2019, celebrat del 18 al 20 de desembre de 2019 a Amsterdam, Holanda., We introduce a novel, scalable method aimed at annotating potential and actual Questions Under Discussion (QUDs) in naturalistic discourse. It consists of asking naive participants first what questions a certain portion of the discourse evokes for them and subsequently which of those end up being answered as the discourse proceeds. This paper outlines the method and design decisions that went into it and on characterizing highlevel properties of the resulting data. We highlight ways in which the data gathered via our method could inform our understanding of QUD-driven phenomena and QUD models themselves. We also provide access to a visualization tool for viewing the evoked questions we gathered using this method (N=4765 from 111 crowdsourced annotators)., This work was supported in part by a Leverhulme Trust Prize in Languages and Literatures to H. Rohde. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

TED-Q: TED talks and the questions they evoke

  • Westera, Matthijs
  • Mayol, Laia
  • Rohde, Hannah
Comunicació presentada al 12th Language Resources and Evaluation Conference celebrat del 13 al 15 de maig de 2020 de manera virtual., We present a new dataset of TED-talks annotated with the questions they evoke and, where available, the answers to these questions. Evoked questions represent a hitherto mostly unexplored type of linguistic data, which promises to open up important new lines of research, especially related to the Question Under Discussion (QUD)-based approach to discourse structure. In this paper we introduce the method and open the first installment of our data to the public. We summarize and explore the current dataset, illustrate its potential by providing new evidence for the relation between predictability and implicitness – capitalizing on the already existing PDTB-style annotations for the texts we use – and outline its potential for future research. The dataset should be of interest, at its current scale, to researchers on formal and experimental pragmatics, discourse coherence, information structure, discourse expectations and processing. Our data-gathering procedure is designed to scale up, relying on crowdsourcing by non-expert annotators, with its utility for Natural Language Processing in mind (e.g., dialogue systems, conversational question answering)., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Grounding semantic roles in images

  • Silberer, Carina
  • Pinkal, Manfred
Comunicació presentada a la 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), celebrada del 31 d'octubre al 4 de novembre de 2018 a Brussel·les, Bèlgica., We address the task of visual semantic role labeling (vSRL), the identification of the participants of a situation or event in a visual scene, and their labeling with their semantic relations to the event or situation. We render candidate participants as image regions of objects, and train a model which learns to ground roles in the regions which depict the corresponding participant. Experimental results demonstrate that we can train a vSRL model without reliance on prohibitive image-based role annotations, by utilizing noisy data which we extract automatically from image captions using a linguistic SRL system. Furthermore, our model induces frame–semantic visual representations, and their comparison to previous work on supervised visual verb sense disambiguation yields overall better results., We thank the anonymous reviewers, Leonie Harter, Christine Sch¨afer, Michael Roth, Gemma Boleda, Anna Rohrbach and Bernt Schiele. This research was supported by the German Research Foundation (DFG EXC 285), by the European Research Council (ERC Horizon 2020 grant agreement No 715154), and the Spanish Ramon y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

How to represent a word and predict it, too: improving tied architectures for language modelling

  • Gulordava, Kristina
  • Aina, Laura
  • Boleda, Gemma
Comunicació presentada a la Conference on Empirical Methods in Natural Language Processing, celebrada els dies 31 d'octubre a 4 de novembre de 2020 a Brussel·les, Bèlgica., Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks., We thank German Kruszewski and the AMORE ´ team for the helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Ramon y Cajal programme (grant RYC-2015- ´ 18907) and the Catalan government (SGR 2017 1575). We gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research. This paper reflects the authors’ view only, and the EU is not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

AMORE-UPF at SemEval-2018 Task 4: BiLSTM with entity library

  • Boleda, Gemma
  • Aina, Laura
  • Silberer, Carina
  • Sorodoc, Ionut-Teodor
  • Westera, Matthijs
Comunicació presentada al 12th International Workshop on Semantic Evaluation (SemEval-2018), celebrat els dies 5 i 6 de juny de 2018 a Nova Orleans, EUA., This paper describes our winning contribution to SemEval 2018 Task 4: Character Identification on Multiparty Dialogues. It is a simple, standard model with one key innovation, an entity library. Our results show that this innovation greatly facilitates the identification of infrequent characters. Because of the generic nature of our model, this finding is potentially relevant to any task that requires effective learning from sparse or unbalanced data., This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant agreement No 715154), and from the Spanish Ramón y Cajal programme (grant RYC-2015-18907).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Similarity or deeper understanding?: analyzing the TED-Q dataset of evoked questions

  • Westera, Matthijs
  • Amidei, Jacopo
  • Mayol, Laia
Comunicació presentada al 28th International Conference on Computational Linguistics celebrat del 8 al 13 de desembre de 2020 de manera virtual., We take a close look at a recent dataset of TED-talks annotated with the questions they implicitly evoke, TED-Q (Westera et al., 2020). We test to what extent the relation between a discourse and the questions it evokes is merely one of similarity or association, as opposed to deeper semantic/pragmatic interpretation. We do so by turning the TED-Q dataset into a binary classification task, constructing an analogous task from explicit questions we extract from the BookCorpus (Zhu et al., 2015), and fitting a BERT-based classifier alongside models based on different notions of similarity. The BERT-based classifier, achieving close to human performance, outperforms all similarity-based models, suggesting that there is more to identifying true evoked questions than plain similarity., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 715154) and from the Spanish State Research Agency (AEI) and the European Regional Development Fund (FEDER, UE) (project PGC2018-094029-A-I00).