Pasar al contenido principal

(SUBPROYECTO 2) "DEEP NETWORKS" EN EL CEREBRO VISUAL: DE LA NEUROIMAGEN A LA DECODIFICACION DE LA RESPUESTA VISUAL

DPI2017-89867-C2-2-R

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2017
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 7
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Visual information flow in Wilson–Cowan networks

  • Gómez Villa, Alexander
  • Bertalmío, Marcelo
  • Malo, Jesús
In this paper, we study the communication efficiency of a psychophysically tuned cascade of Wilson–Cowan and divisive normalization layers that simulate the retina-V1 pathway. This is the first analysis of Wilson–Cowan networks in terms of multivariate total correlation. The parameters of the cortical model have been derived through the relation between the steady state of the Wilson–Cowan model and the divisive normalization model. The communication efficiency has been analyzed in two ways: First, we provide an analytical expression for the reduction of the total correlation among the responses of a V1-like population after the application of the Wilson–Cowan interaction. Second, we empirically study the efficiency with visual stimuli and statistical tools that were not available before 1) we use a recent, radiometrically calibrated, set of natural scenes, and 2) we use a recent technique to estimate the multivariate total correlation in bits from sets of visual responses, which only involves univariate operations, thus giving better estimates of the redundancy. The theoretical and the empirical results show that, although this cascade of layers was not optimized for statistical independence in any way, the redundancy between the responses gets substantially reduced along the neural pathway. Specifically, we show that 1) the efficiency of a Wilson–Cowan network is similar to its equivalent divisive normalization model; 2) while initial layers (Von Kries adaptation and Weber-like brightness) contribute to univariate equalization, and the bigger contributions to the reduction in total correlation come from the computation of nonlinear local contrast and the application of local oriented filters; and 3) psychophysically tuned models are more efficient (reduce more total correlation) in the more populated regions of the luminance-contrast plane. These results are an alternative confirmation of the efficient coding hypothesis for the Wilson–Cowan systems, and, from an applied perspective, they suggest that neural field models could be an option in image coding to perform image compression., This work was partially funded by the Spanish Government and EU FEDER fund through the MINECO grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Contrast sensitivity functions in autoencoders

  • Li, Qiang
  • Gomez-Villa, Alex
  • Bertalmío, Marcelo
  • Malo, Jesús
Three decades ago, Atick et al. suggested that human frequency sensitivity may emerge from the enhancement required for a more efficient analysis of retinal images. Here we reassess the relevance of low-level vision tasks in the explanation of the contrast sensitivity functions (CSFs) in light of 1) the current trend of using artificial neural networks for studying vision, and 2) the current knowledge of retinal image representations. As a first contribution, we show that a very popular type of convolutional neural networks (CNNs), called autoencoders, may develop human-like CSFs in the spatiotemporal and chromatic dimensions when trained to perform some basic low-level vision tasks (like retinal noise and optical blur removal), but not others (like chromatic) adaptation or pure reconstruction after simple bottlenecks). As an illustrative example, the best CNN (in the considered set of simple architectures for enhancement of the retinal signal) reproduces the CSFs with a root mean square error of 11% of the maximum sensitivity. As a second contribution, we provide experimental evidence of the fact that, for some functional goals (at low abstraction level), deeper CNNs that are better in reaching the quantitative goal are actually worse in replicating human-like phenomena (such as the CSFs). This low-level result (for the explored networks) is not necessarily in contradiction with other works that report advantages of deeper nets in modeling higher level vision goals. However, in line with a growing body of literature, our results suggests another word of caution about CNNs in vision science because the use of simplified units or unrealistic architectures in goal optimization may be a limitation for the modeling and understanding of human vision., Partially funded by these grants from GVA/AEI/FEDER/EU: MICINN DPI2017- 89867-C2-2-R, MICINN PID2020-118071GB-I00, and GVA Grisolía-P/2019/035 (for JM and QL), and MICINN PGC2018-099651-B-I00 (for A.G.V. and M.B.).
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

In praise of artifice reloaded: Caution with natural image databases in modeling vision

  • Martinez-Garcia, Marina
  • Bertalmío, Marcelo
  • Malo, Jesús
Subjective image quality databases are a major source of raw data on how the visual system works in naturalistic environments. These databases describe the sensitivity of many observers to a wide range of distortions of different nature and intensity seen on top of a variety of natural images. Data of this kind seems to open a number of possibilities for the vision scientist to check the models in realistic scenarios. However, while these natural databases are great benchmarks for models developed in some other way (e.g., by using the well-controlled artificial stimuli of traditional psychophysics), they should be carefully used when trying to fit vision models. Given the high dimensionality of the image space, it is very likely that some basic phenomena are under-represented in the database. Therefore, a model fitted on these large-scale natural databases will not reproduce these under-represented basic phenomena that could otherwise be easily illustrated with well selected artificial stimuli. In this work we study a specific example of the above statement. A standard cortical model using wavelets and divisive normalization tuned to reproduce subjective opinion on a large image quality dataset fails to reproduce basic cross-masking. Here we outline a solution for this problem by using artificial stimuli and by proposing a modification that makes the model easier to tune. Then, we show that the modified model is still competitive in the large-scale database. Our simulations with these artificial stimuli show that when using steerable wavelets, the conventional unit norm Gaussian kernels in divisive normalization should be multiplied by high-pass filters to reproduce basic trends in masking. Basic visual phenomena may be misrepresented in large natural image datasets but this can be solved with model-interpretable stimuli. This is an additional argument in praise of artifice in line with Rust and Movshon (2005)., This work was partially funded by the Spanish and EU FEDER fund through the MINECO/FEDER/EU grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Contrast sensitivity functions in autoencoders

  • Li, Qiang
  • Gómez-Villa, Alex
  • Bertalmío, Marcelo
  • Malo, Jesús
45 pags., 24 figs., 4 tabs., 3 apps., The human contrast sensitivity function (CSF) characterizes the psychophysical response to visual gratings of different frequency (Campbell & Robson, 1968). Filter characterizations in the Fourier domain are complete only for linear, shift-invariant systems. Human vision certainly is more complicated than that, however, this simple measure of the bandwidth of the system is still of paramount significance in biological vision: the CSF filter is an image-computable model that roughly describes the kind of visual information that is available for humans (Watson & Ahumada, 2016). Moreover, although it is defined for threshold conditions, there are many examples that illustrate the relevance of the CSF in more general situations (Watson et al., 1986; Watson & Malo, 2002; Watson & Ahumada, 2005), so it has shaped image engineering over decades (Mannos & Sakrison, 1974; Hunt, 1975; Wallace, 1992; Taubman & Marcellin, 2001). This theoretical and practical relevance motivated the measurement of CSFs, not only for spatial gratings (Campbell & Robson, 1968), but also for moving gratings (Kelly, 1979), chromatic gratings (Mullen, 1985), spatiotemporal chromatic gratings (Díez-Ajenjo et al., 2011), at different luminance levels (Wuerger et al., 2020), and for alternative basis of the image space (Malo et al., 1997), Partially funded by these grants from GVA/AEI/FEDER/EU: MICINN DPI2017-89867-C2-2-R, MICINN PID2020-118071GB-I00, and GVA Grisolía-P/2019/035 (for JM and QL), and MICINN PGC2018-099651-B-I00 (for A.G.V. and M.B.)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Contrast sensitivity functions in autoencoders

  • Li, Qiang
  • Gomez-Villa, Alex
  • Bertalmío, Marcelo
  • Malo, Jesús
Three decades ago, Atick et al. suggested that human frequency sensitivity may emerge from the enhancement required for a more efficient analysis of retinal images. Here we reassess the relevance of low-level vision tasks in the explanation of the contrast sensitivity functions (CSFs) in light of 1) the current trend of using artificial neural networks for studying vision, and 2) the current knowledge of retinal image representations. As a first contribution, we show that a very popular type of convolutional neural networks (CNNs), called autoencoders, may develop human-like CSFs in the spatiotemporal and chromatic dimensions when trained to perform some basic low-level vision tasks (like retinal noise and optical blur removal), but not others (like chromatic) adaptation or pure reconstruction after simple bottlenecks). As an illustrative example, the best CNN (in the considered set of simple architectures for enhancement of the retinal signal) reproduces the CSFs with a root mean square error of 11% of the maximum sensitivity. As a second contribution, we provide experimental evidence of the fact that, for some functional goals (at low abstraction level), deeper CNNs that are better in reaching the quantitative goal are actually worse in replicating human-like phenomena (such as the CSFs). This low-level result (for the explored networks) is not necessarily in contradiction with other works that report advantages of deeper nets in modeling higher level vision goals. However, in line with a growing body of literature, our results suggests another word of caution about CNNs in vision science because the use of simplified units or unrealistic architectures in goal optimization may be a limitation for the modeling and understanding of human vision., Partially funded by these grants from GVA/AEI/FEDER/EU: MICINN DPI2017- 89867-C2-2-R, MICINN PID2020-118071GB-I00, and GVA Grisolía-P/2019/035 (for JM and QL), and MICINN PGC2018-099651-B-I00 (for A.G.V. and M.B.).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Visual information flow in Wilson–Cowan networks

  • Gómez Villa, Alexander
  • Bertalmío, Marcelo
  • Malo, Jesús
In this paper, we study the communication efficiency of a psychophysically tuned cascade of Wilson–Cowan and divisive normalization layers that simulate the retina-V1 pathway. This is the first analysis of Wilson–Cowan networks in terms of multivariate total correlation. The parameters of the cortical model have been derived through the relation between the steady state of the Wilson–Cowan model and the divisive normalization model. The communication efficiency has been analyzed in two ways: First, we provide an analytical expression for the reduction of the total correlation among the responses of a V1-like population after the application of the Wilson–Cowan interaction. Second, we empirically study the efficiency with visual stimuli and statistical tools that were not available before 1) we use a recent, radiometrically calibrated, set of natural scenes, and 2) we use a recent technique to estimate the multivariate total correlation in bits from sets of visual responses, which only involves univariate operations, thus giving better estimates of the redundancy. The theoretical and the empirical results show that, although this cascade of layers was not optimized for statistical independence in any way, the redundancy between the responses gets substantially reduced along the neural pathway. Specifically, we show that 1) the efficiency of a Wilson–Cowan network is similar to its equivalent divisive normalization model; 2) while initial layers (Von Kries adaptation and Weber-like brightness) contribute to univariate equalization, and the bigger contributions to the reduction in total correlation come from the computation of nonlinear local contrast and the application of local oriented filters; and 3) psychophysically tuned models are more efficient (reduce more total correlation) in the more populated regions of the luminance-contrast plane. These results are an alternative confirmation of the efficient coding hypothesis for the Wilson–Cowan systems, and, from an applied perspective, they suggest that neural field models could be an option in image coding to perform image compression., This work was partially funded by the Spanish Government and EU FEDER fund through the MINECO grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE).
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Contrast sensitivity functions in autoencoders

  • Li, Qiang
  • Gómez Villa, Alexandra|||0000-0003-0469-3425
  • Bertalmío, Marcelo|||0000-0002-1023-8325
  • Malo, Jesús|||0000-0002-5684-8591
Three decades ago, Atick et al. suggested that human frequency sensitivity may emerge from the enhancement required for a more efficient analysis of retinal images. Here we reassess the relevance of low-level vision tasks in the explanation of the contrast sensitivity functions (CSFs) in light of 1) the current trend of using artificial neural networks for studying vision, and 2) the current knowledge of retinal image representations. As a first contribution, we show that a very popular type of convolutional neural networks (CNNs), called autoencoders, may develop human-like CSFs in the spatiotemporal and chromatic dimensions when trained to perform some basic low-level vision tasks (like retinal noise and optical blur removal), but not others (like chromatic) adaptation or pure reconstruction after simple bottlenecks). As an illustrative example, the best CNN (in the considered set of simple architectures for enhancement of the retinal signal) reproduces the CSFs with a root mean square error of 11% of the maximum sensitivity. As a second contribution, we provide experimental evidence of the fact that, for some functional goals (at low abstraction level), deeper CNNs that are better in reaching the quantitative goal are actually worse in replicating human-like phenomena (such as the CSFs). This low-level result (for the explored networks) is not necessarily in contradiction with other works that report advantages of deeper nets in modeling higher level vision goals. However, in line with a growing body of literature, our results suggests another word of caution about CNNs in vision science because the use of simplified units or unrealistic architectures in goal optimization may be a limitation for the modeling and understanding of human vision.