Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 20
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Franzon, Francesca
  • Dessì, Roberto
  • Baroni, Marco
Comunicació presentada a la 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023), celebrada a Londres del 29 de maig al 2 de juny de 2023., As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we explore the task of referential communication in a community of state-of-the-art pre-trained visual networks, showing that they can develop a shared protocol to refer to a target image among a set of candidates. Such shared protocol, induced in a self-supervised way, can to some extent be used to communicate about previously unseen object categories. Finally, we show that a new neural network can learn the shared protocol developed in a community with remarkable ease, and the process of integrating a new agent into a community more stably succeeds when the original community includes a larger set of heterogeneous networks., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Not a nuisance but a useful heuristic: outlier dimensions favor frequent tokens in language models

  • Macocco, Iuri
  • Graichen, Nora
  • Boleda, Gemma
  • Baroni, Marco
We study last-layer outlier dimensions, i.e. dimensions that display extreme activations for the majority of inputs. We show that outlier dimensions arise in many different modern language models, and trace their function back to the heuristic of constantly predicting frequent words. We further show how a model can block this heuristic when it is not contextually appropriate, by assigning a counterbalancing weight mass to the remaining dimensions, and we investigate which model parameters boost outlier dimensions and when they arise during training. We conclude that outlier dimensions are a specialized mechanism discovered by many distinct models to implement a useful token prediction heuristic., We thank Beatrix Miranda Ginn Nielsen, Santiago Acevedo, Diego Doimo, Javier Ferrando, Alessandro Laio and the members of the UPF COLT group for feedback. Our work was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). NG also received the support of a fellowship from Fundación Ramón Areces. GB also received the support of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Can discrete information extraction prompts generalize across language models?

  • Rakotonirina, Nathanael Carraz
  • Dessì, Roberto
  • Petroni, Fabio
  • Riedel, Sebastian
  • Baroni, Marco
Comunicació presentada a: ICLR 2023. 11th Conference on Learning Representations (ICLR) celebrada a Kigali, Rwanda, del 1 al 5 de maig a de 2023., We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information. After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, we demonstrate a drop in performance for AutoPrompt prompts learned on a model and tested on another. We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models. We conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens. Our work provides preliminary evidence that it’s possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts., UPF has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291)., https://creativecommons.org/licenses/by-nc-nd/4.0/deed.es_ES
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Emergence of a high-dimensional abstraction phase in language transformers

  • Cheng, Emily
  • Doimo, Diego
  • Kervadec, Corentin
  • Macocco, Iuri
  • Yu, Lei
  • Laio, Alessandro
  • Baroni, Marco
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate to its function. We take a high-level geometric approach to its analysis, observing, across five pre-trained transformer-based LMs and three input datasets, a distinct phase characterized by high intrinsic dimensionality. During this phase, representations (1) correspond to the first full linguistic abstraction of the input; (2) are the first to viably transfer to downstream tasks; (3) predict each other across different LMs. Moreover, we find that an earlier onset of the phase strongly predicts better language modelling performance. In short, our results suggest that a central high-dimensionality phase underlies core linguistic processing in many common LM architectures., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 101019291). This paper reflects the authors’ view only, and the funding agency is not responsible for any use that may be made of the information it contains. Additionally, D.D. received support from the project “Supporto alla diagnosi di malattie rare tramite l’intelligenza artificiale”(CUP: F53C22001770002).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Evil twins are not that evil: qualitative insights into machine-generated prompts

  • Carraz Rakotonirina, Nathanaël
  • Kervadec, Corentin
  • Franzon, Francesca
  • Baroni, Marco
It has been widely observed that language models (LMs) respond in predictable ways to algorithmically generated prompts that are seemingly unintelligible. This is both a sign that we lack a full understanding of how LMs work, and a practical challenge, because opaqueness can be exploited for harmful uses of LMs, such as jailbreaking. We present the first thorough analysis of opaque machine-generated prompts, or autoprompts, pertaining to 6 LMs of different sizes and families. We find that machinegenerated prompts are characterized by a last token that is often intelligible and strongly affects the generation. A small but consistent proportion of the previous tokens are prunable, probably appearing in the prompt as a by-product of the fact that the optimization process fixes the number of tokens. The remaining tokens fall into two categories: filler tokens, which can be replaced with semantically unrelated substitutes, and keywords, that tend to have at least a loose semantic relation with the generation, although they do not engage in wellformed syntactic relations with it. Additionally, human experts can reliably identify the most influential tokens in an autoprompt a posteriori, suggesting these prompts are not entirely opaque. Finally, some of the ablations we applied to autoprompts yield similar effects in natural language inputs, suggesting that autoprompts emerge naturally from the way LMs process linguistic inputs in general., We thank Emmanuel Chemla, Mor Geva and audiences at SISA, CIMeC and at the online HiTZ seminar series for feedback. Our work was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Dessì, Roberto
  • Franzon, Francesca
  • Baroni, Marco
Data de publicació electrònica: 25-04-2025, As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we systematically explore the task of referential communication in a community of heterogeneous state-of-the-art pre-trained visual networks, showing that they can develop, in a self-supervised way, a shared protocol to refer to a target object among a set of candidates. This shared protocol can also be used, to some extent, to communicate about previously unseen object categories of different granularity. Moreover, a visual network that was not initially part of an existing community can learn the community's protocol with remarkable ease. Finally, we study, both qualitatively and quantitatively, the properties of the emergent protocol, providing some evidence that it is capturing high-level semantic features of objects., We thank Agnieszka Słowik for early contributions to this work. We are also grateful to Thomas Brochhagen, Dan Dediu, Rahma Chaabouni, Emily Cheng, Valentino Maiorca, Luca Moschella, Clément Moulin Frier, the members of the COLT group and the TMLR editor and reviewers for useful feedback. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

MemoryPrompt: a light wrapper to improve context tracking in pre-trained language models

  • Carraz Rakotonirina, Nathanaël
  • Baroni, Marco
Comunicació presentada a Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), celebrada a Turí del 20 al 25 de maig de 2024., Transformer-based language models (LMs) track contextual information through large, hard-coded input windows. We introduce MemoryPrompt, a leaner approach in which the LM is complemented by a small auxiliary recurrent network that passes information to the LM by prefixing its regular input with a sequence of vectors, akin to soft prompts, without requiring LM finetuning. Tested on a task designed to probe a LM’s ability to keep track of multiple fact updates, a MemoryPrompt-augmented LM outperforms much larger LMs that have access to the full input history. We also test MemoryPrompt on a long-distance dialogue dataset, where its performance is comparable to that of a model conditioned on the entire conversation history. In both experiments we also observe that, unlike full-finetuning approaches, MemoryPrompt does not suffer from catastrophic forgetting when adapted to new tasks, thus not disrupting the generalist capabilities of the underlying LM.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Prediction hubs are context-informed frequent tokens in LLMs

  • Nielsen, Beatrix MG
  • Macocco, Iuri
  • Baroni, Marco
Hubness, the tendency for a few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to high-dimensional data, often negatively impacting distance-based analysis. As autoregressive large language models (LLMs) operate on high-dimensional representations, we ask whether they are also affected by hubness. We first prove that the only large-scale representation comparison operation performed by LLMs, namely that between context and unembedding vectors to determine continuation probabilities, is not characterized by the concentration of distances phenomenon that typically causes the appearance of nuisance hubness. We then empirically show that this comparison still leads to a high degree of hubness, but the hubs in this case do not constitute a disturbance. They are rather the result of context-modulated frequent tokens often appearing in the pool of likely candidates for next token prediction. However, when other distances are used to compare LLM representations, we do not have the same theoretical guarantees, and, indeed, we see nuisance hubs appear. There are two main takeaways. First, hubness, while omnipresent in high-dimensional spaces, is not a negative property that needs to be mitigated when LLMs are being used for next token prediction. Second, when comparing representations from LLMs using Euclidean or cosine distance, there is a high risk of nuisance hubs and practitioners should use mitigation techniques if relevant., We thank Santiago Acevedo, Luca Moschella, the members of the COLT group at Universitat Pompeu Fabra and the ARR reviewers for feedback and advice. Beatrix M. G. Nielsen was supported by the Danish Pioneer Centre for AI, DNRF grant number P1. Iuri Macocco and Marco Baroni received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 101019291) and from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Cross-domain image captioning with discriminative finetuning

  • Dessì, Roberto
  • Bevilacqua, Michele
  • Gualdoni, Eleonora
  • Rakotonirina, Nathanael Carraz
  • Franzon, Francesca
  • Baroni, Marco
Comunicació presentada a: CVPR 2023. IEEE/CVF Conference on Computer Vision and Pattern Recognition celebrat del 18 al 22 de juny a Vancouver, Canada., Neural captioners are typically trained to mimic humangenerated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show that fine-tuning an out-of-the-box neural captioner with a selfsupervised discriminative communication objective helps to recover a plain, visually descriptive language that is more informative about image contents. Given a target image, the system must learn to produce a description that enables an out-of-the-box text-conditioned image retriever to identify such image among a set of candidates. We experiment with the popular ClipCap captioner, also replicating the main results with BLIP. In terms of similarity to groundtruth human descriptions, the captions emerging from discriminative finetuning lag slightly behind those generated by the non-finetuned model, when the latter is trained and tested on the same caption dataset. However, when the model is used without further tuning to generate captions for out-of-domain datasets, our discriminatively-finetuned captioner generates descriptions that resemble human references more than those produced by the same captioner without finetuning. We further show that, on the Conceptual Captions dataset, discriminatively finetuned captions are more helpful than either vanilla ClipCap captions or ground-truth captions for human annotators tasked with an image discrimination task, EG, NCR, FF and MB received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Emergence of a high-dimensional abstraction phase in language transformers

  • Cheng, Emily
  • Doimo, Diego
  • Kervadec, Corentin
  • Macocco, Iuri
  • Yu, Lei
  • Laio, Alessandro
  • Baroni, Marco
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate to its function. We take a high-level geometric approach to its analysis, observing, across five pre-trained transformer-based LMs and three input datasets, a distinct phase characterized by high intrinsic dimensionality. During this phase, representations (1) correspond to the first full linguistic abstraction of the input; (2) are the first to viably transfer to downstream tasks; (3) predict each other across different LMs. Moreover, we find that an earlier onset of the phase strongly predicts better language modelling performance. In short, our results suggest that a central high-dimensionality phase underlies core linguistic processing in many common LM architectures., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 101019291). This paper reflects the authors’ view only, and the funding agency is not responsible for any use that may be made of the information it contains. Additionally, D.D. received support from the project “Supporto alla diagnosi di malattie rare tramite l’intelligenza artificiale”(CUP: F53C22001770002).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Prediction hubs are context-informed frequent tokens in LLMs

  • Nielsen, Beatrix MG
  • Macocco, Iuri
  • Baroni, Marco
Hubness, the tendency for a few points to be among the nearest neighbours of a disproportionate number of other points, commonly arises when applying standard distance measures to high-dimensional data, often negatively impacting distance-based analysis. As autoregressive large language models (LLMs) operate on high-dimensional representations, we ask whether they are also affected by hubness. We first prove that the only large-scale representation comparison operation performed by LLMs, namely that between context and unembedding vectors to determine continuation probabilities, is not characterized by the concentration of distances phenomenon that typically causes the appearance of nuisance hubness. We then empirically show that this comparison still leads to a high degree of hubness, but the hubs in this case do not constitute a disturbance. They are rather the result of context-modulated frequent tokens often appearing in the pool of likely candidates for next token prediction. However, when other distances are used to compare LLM representations, we do not have the same theoretical guarantees, and, indeed, we see nuisance hubs appear. There are two main takeaways. First, hubness, while omnipresent in high-dimensional spaces, is not a negative property that needs to be mitigated when LLMs are being used for next token prediction. Second, when comparing representations from LLMs using Euclidean or cosine distance, there is a high risk of nuisance hubs and practitioners should use mitigation techniques if relevant., We thank Santiago Acevedo, Luca Moschella, the members of the COLT group at Universitat Pompeu Fabra and the ARR reviewers for feedback and advice. Beatrix M. G. Nielsen was supported by the Danish Pioneer Centre for AI, DNRF grant number P1. Iuri Macocco and Marco Baroni received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 101019291) and from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Dessì, Roberto
  • Franzon, Francesca
  • Baroni, Marco
Data de publicació electrònica: 25-04-2025, As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we systematically explore the task of referential communication in a community of heterogeneous state-of-the-art pre-trained visual networks, showing that they can develop, in a self-supervised way, a shared protocol to refer to a target object among a set of candidates. This shared protocol can also be used, to some extent, to communicate about previously unseen object categories of different granularity. Moreover, a visual network that was not initially part of an existing community can learn the community's protocol with remarkable ease. Finally, we study, both qualitatively and quantitatively, the properties of the emergent protocol, providing some evidence that it is capturing high-level semantic features of objects., We thank Agnieszka Słowik for early contributions to this work. We are also grateful to Thomas Brochhagen, Dan Dediu, Rahma Chaabouni, Emily Cheng, Valentino Maiorca, Luca Moschella, Clément Moulin Frier, the members of the COLT group and the TMLR editor and reviewers for useful feedback. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Communication breakdown: on the low mutual intelligibility between human and neural captioning

  • Dessì, Roberto
  • Gualdoni, Eleonora
  • Franzon, Francesca
  • Boleda, Gemma
  • Baroni, Marco
Comunicació presentada a la 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), celebrada del 7 a l'11 de desembre de 2022 a Abu Dhabi, Emirats Àrabs Units., We compare the 0-shot performance of a neural caption-based image retriever when given as input either human-produced captions or captions generated by a neural captioner. We conduct this comparison on the recently introduced IMAGECODE data-set (Krojer et al., 2022), which contains hard distractors nearly identical to the images to be retrieved. We find that the neural retriever has much higher performance when fed neural rather than human captions, despite the fact that the former, unlike the latter, were generated without awareness of the distractors that make the task hard. Even more remarkably, when the same neural captions are given to human subjects, their retrieval performance is almost at chance level. Our results thus add to the growing body of evidence that, even when the “language” of neural models resembles English, this superficial resemblance might be deeply misleading., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreements No. 715154 and No. 101019291) and the Spanish Research Agency (ref. PID2020-112602GB-I00).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Evil twins are not that evil: qualitative insights into machine-generated prompts

  • Carraz Rakotonirina, Nathanaël
  • Kervadec, Corentin
  • Franzon, Francesca
  • Baroni, Marco
It has been widely observed that language models (LMs) respond in predictable ways to algorithmically generated prompts that are seemingly unintelligible. This is both a sign that we lack a full understanding of how LMs work, and a practical challenge, because opaqueness can be exploited for harmful uses of LMs, such as jailbreaking. We present the first thorough analysis of opaque machine-generated prompts, or autoprompts, pertaining to 6 LMs of different sizes and families. We find that machinegenerated prompts are characterized by a last token that is often intelligible and strongly affects the generation. A small but consistent proportion of the previous tokens are prunable, probably appearing in the prompt as a by-product of the fact that the optimization process fixes the number of tokens. The remaining tokens fall into two categories: filler tokens, which can be replaced with semantically unrelated substitutes, and keywords, that tend to have at least a loose semantic relation with the generation, although they do not engage in wellformed syntactic relations with it. Additionally, human experts can reliably identify the most influential tokens in an autoprompt a posteriori, suggesting these prompts are not entirely opaque. Finally, some of the ablations we applied to autoprompts yield similar effects in natural language inputs, suggesting that autoprompts emerge naturally from the way LMs process linguistic inputs in general., We thank Emmanuel Chemla, Mor Geva and audiences at SISA, CIMeC and at the online HiTZ seminar series for feedback. Our work was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

MemoryPrompt: a light wrapper to improve context tracking in pre-trained language models

  • Carraz Rakotonirina, Nathanaël
  • Baroni, Marco
Comunicació presentada a Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), celebrada a Turí del 20 al 25 de maig de 2024., Transformer-based language models (LMs) track contextual information through large, hard-coded input windows. We introduce MemoryPrompt, a leaner approach in which the LM is complemented by a small auxiliary recurrent network that passes information to the LM by prefixing its regular input with a sequence of vectors, akin to soft prompts, without requiring LM finetuning. Tested on a task designed to probe a LM’s ability to keep track of multiple fact updates, a MemoryPrompt-augmented LM outperforms much larger LMs that have access to the full input history. We also test MemoryPrompt on a long-distance dialogue dataset, where its performance is comparable to that of a model conditioned on the entire conversation history. In both experiments we also observe that, unlike full-finetuning approaches, MemoryPrompt does not suffer from catastrophic forgetting when adapted to new tasks, thus not disrupting the generalist capabilities of the underlying LM.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Can discrete information extraction prompts generalize across language models?

  • Rakotonirina, Nathanael Carraz
  • Dessì, Roberto
  • Petroni, Fabio
  • Riedel, Sebastian
  • Baroni, Marco
Comunicació presentada a: ICLR 2023. 11th Conference on Learning Representations (ICLR) celebrada a Kigali, Rwanda, del 1 al 5 de maig a de 2023., We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information. After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, we demonstrate a drop in performance for AutoPrompt prompts learned on a model and tested on another. We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models. We conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens. Our work provides preliminary evidence that it’s possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts., UPF has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291)., https://creativecommons.org/licenses/by-nc-nd/4.0/deed.es_ES
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Not a nuisance but a useful heuristic: outlier dimensions favor frequent tokens in language models

  • Macocco, Iuri
  • Graichen, Nora
  • Boleda, Gemma
  • Baroni, Marco
We study last-layer outlier dimensions, i.e. dimensions that display extreme activations for the majority of inputs. We show that outlier dimensions arise in many different modern language models, and trace their function back to the heuristic of constantly predicting frequent words. We further show how a model can block this heuristic when it is not contextually appropriate, by assigning a counterbalancing weight mass to the remaining dimensions, and we investigate which model parameters boost outlier dimensions and when they arise during training. We conclude that outlier dimensions are a specialized mechanism discovered by many distinct models to implement a useful token prediction heuristic., We thank Beatrix Miranda Ginn Nielsen, Santiago Acevedo, Diego Doimo, Javier Ferrando, Alessandro Laio and the members of the UPF COLT group for feedback. Our work was funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). NG also received the support of a fellowship from Fundación Ramón Areces. GB also received the support of grant PID2020-112602GBI00/MICIN/AEI/10.13039/501100011033, funded by the Ministerio de Ciencia e Innovación and the Agencia Estatal de Investigación (Spain). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Franzon, Francesca
  • Dessì, Roberto
  • Baroni, Marco
Comunicació presentada a la 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023), celebrada a Londres del 29 de maig al 2 de juny de 2023., As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we explore the task of referential communication in a community of state-of-the-art pre-trained visual networks, showing that they can develop a shared protocol to refer to a target image among a set of candidates. Such shared protocol, induced in a self-supervised way, can to some extent be used to communicate about previously unseen object categories. Finally, we show that a new neural network can learn the shared protocol developed in a community with remarkable ease, and the process of integrating a new agent into a community more stably succeeds when the original community includes a larger set of heterogeneous networks., This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291).