Pasar al contenido principal

ALINEAMIENTO DE REDES NEURONALES PROFUN- DAS CON EL SISTEMA VISUAL HUMANO

PID2023-152133NB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2023
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 2
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Contrast sensitivity in multimodal large language models: A psychophysics-inspired evaluation

  • Hernández-Cámara, Pablo
  • Gomez-Villa, Alexandra
  • Jaén-Lorites, José Manuel
  • Vila-Tomás, Jorge
  • Laparra, Valero
  • Malo, Jesús
[EN] Understanding how Multimodal Large Language Models (MLLMs) process low-level visual features is critical for evaluating their perceptual abilities and has not been systematically characterized. Inspired by human psychophysics, we introduce a behavioural method for estimating the Contrast Sensitivity Function (CSF) in MLLMs by treating them as end-to-end observers. Models are queried with structured prompts while viewing noise-based stimuli filtered at specific spatial frequencies. Psychometric functions are derived from the binary verbal responses. Therefore, contrast thresholds (and CSFs) are obtained without relying on internal activations or classifier-based proxies, as opposed to previous reports on artificial networks. Our results reveal that some models resemble human CSFs in shape or scale, but none capture both. We also find that CSF estimates are highly sensitive to prompt phrasing, indicating limited linguistic robustness. Finally, we show that CSFs predict model performance under frequency-filtered and adversarial conditions. These findings highlight systematic differences in frequency tuning across MLLMs and establish this CSF estimation as a scalable diagnostic tool for multimodal perception., This work was supported in part by MICIIN/FEDER/UE under Grant PID2023-152133NB-I00; in part by Spanish MIU under Grant FPU21/02256; in part by Generalitat Valenciana under Project GV/2021/074, CIPROM/2021/056; and in part by the grant BBVA Foundations of Science program: Maths, Stats, Comp. Sci. and AI (VIS4NN) . The authors gratefully acknowledge the computer resources at Artemisa and the technical support provided by the European Union through the 2014-2020 ERDF Operative Programme of Comunitat Valenciana, project IDIFEDER/2018/048.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Plaid masking explained with input-dependent dendritic nonlinearities

  • Bertalmio, Marcelo
  • Durán-Vizcaíno, A.
  • Malo, Jesús
  • Wichmann, F.A.
7 pages, 3 figures, A serious obstacle for understanding early spatial vision comes from the failure of the so-called standard model (SM) to predict the perception of plaid masking. But the SM originated from a major oversimplification of single neuron computations, ignoring fundamental properties of dendrites. Here we show that a spatial vision model including computations mimicking the input-dependent nature of dendritic nonlinearities, i.e. including nonlinear neural summation, has the potential to explain plaid masking data., All authors acknowledge funding from VIS4NN, Programa Fundamentos 2022, Fundación BBVA. M.B. and A.D.V. were also supported by PID2021-127373NB-I00, Ministerio de Ciencia e Innovación (Spain). J.M was supported by grants PDC2021-121522-C21 and PID2023-152133NB-I00, Ministerio de Ciencia e Innovacion (Spain) (MICIIN/FEDER/EU). Felix A. Wichmann is a member of the Machine Learning Cluster of Excellence, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC number 2064/1 – project number 390727645. In addition this work was supported by the German Research Foundation (DFG) SFB 1233, Robust Vision: Inference Principles and Neural Mechanisms, project number 276693517. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.