Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 2
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Dessì, Roberto
  • Franzon, Francesca
  • Baroni, Marco
Data de publicació electrònica: 25-04-2025, As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we systematically explore the task of referential communication in a community of heterogeneous state-of-the-art pre-trained visual networks, showing that they can develop, in a self-supervised way, a shared protocol to refer to a target object among a set of candidates. This shared protocol can also be used, to some extent, to communicate about previously unseen object categories of different granularity. Moreover, a visual network that was not initially part of an existing community can learn the community's protocol with remarkable ease. Finally, we study, both qualitatively and quantitatively, the properties of the emergent protocol, providing some evidence that it is capturing high-level semantic features of objects., We thank Agnieszka Słowik for early contributions to this work. We are also grateful to Thomas Brochhagen, Dan Dediu, Rahma Chaabouni, Emily Cheng, Valentino Maiorca, Luca Moschella, Clément Moulin Frier, the members of the COLT group and the TMLR editor and reviewers for useful feedback. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Referential communication in heterogeneous communities of pre-trained visual deep networks

  • Mahaut, Matéo
  • Dessì, Roberto
  • Franzon, Francesca
  • Baroni, Marco
Data de publicació electrònica: 25-04-2025, As large pre-trained image-processing neural networks are being embedded in autonomous agents such as self-driving cars or robots, the question arises of how such systems can communicate with each other about the surrounding world, despite their different architectures and training regimes. As a first step in this direction, we systematically explore the task of referential communication in a community of heterogeneous state-of-the-art pre-trained visual networks, showing that they can develop, in a self-supervised way, a shared protocol to refer to a target object among a set of candidates. This shared protocol can also be used, to some extent, to communicate about previously unseen object categories of different granularity. Moreover, a visual network that was not initially part of an existing community can learn the community's protocol with remarkable ease. Finally, we study, both qualitatively and quantitatively, the properties of the emergent protocol, providing some evidence that it is capturing high-level semantic features of objects., We thank Agnieszka Słowik for early contributions to this work. We are also grateful to Thomas Brochhagen, Dan Dediu, Rahma Chaabouni, Emily Cheng, Valentino Maiorca, Luca Moschella, Clément Moulin Frier, the members of the COLT group and the TMLR editor and reviewers for useful feedback. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). We also received funding from the Catalan government (AGAUR grant SGR 2021 00470). This paper reflects the authors’ view only, and the funding agencies are not responsible for any use that may be made of the information it contains.