RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación
Artículo científico (JournalArticle)
Contrast sensitivity in multimodal large language models: A psychophysics-inspired evaluation
[EN] Understanding how Multimodal Large Language Models (MLLMs) process low-level visual features is critical for evaluating their perceptual abilities and has not been systematically characterized. Inspired by human psychophysics, we introduce a behavioural method for estimating the Contrast Sensitivity Function (CSF) in MLLMs by treating them as end-to-end observers. Models are queried with structured prompts while viewing noise-based stimuli filtered at specific spatial frequencies. Psychometric functions are derived from the binary verbal responses. Therefore, contrast thresholds (and CSFs) are obtained without relying on internal activations or classifier-based proxies, as opposed to previous reports on artificial networks. Our results reveal that some models resemble human CSFs in shape or scale, but none capture both. We also find that CSF estimates are highly sensitive to prompt phrasing, indicating limited linguistic robustness. Finally, we show that CSFs predict model performance under frequency-filtered and adversarial conditions. These findings highlight systematic differences in frequency tuning across MLLMs and establish this CSF estimation as a scalable diagnostic tool for multimodal perception., This work was supported in part by MICIIN/FEDER/UE under Grant PID2023-152133NB-I00; in part by Spanish MIU under Grant FPU21/02256; in part by Generalitat Valenciana under Project GV/2021/074, CIPROM/2021/056; and in part by the grant BBVA Foundations of Science program: Maths, Stats, Comp. Sci. and AI (VIS4NN) . The authors gratefully acknowledge the computer resources at Artemisa and the technical support provided by the European Union through the 2014-2020 ERDF Operative Programme of Comunitat Valenciana, project IDIFEDER/2018/048.