Pasar al contenido principal

INTRINSICALLY NONLINEAR RECEPTIVE FIELDS: REPLACING THE STANDARD MODEL OF VISION

PID2021-127373NB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

State-of-the-art image and video quality assessment with a metric based on an intrinsically non-linear neural summation model

  • Luna, Raúl
  • Zabaleta, Itziar
  • Bertalmío, Marcelo
11 pags., 1 fig., 5 tabs., The development of automatic methods for image and video quality assessment that correlate well with the perception of human observers is a very challenging open problem in vision science, with numerous practical applications in disciplines such as image processing and computer vision, as well as in the media industry. In the past two decades, the goal of image quality research has been to improve upon classical metrics by developing models that emulate some aspects of the visual system, and while the progress has been considerable, state-of-the-art quality assessment methods still share a number of shortcomings, like their performance dropping considerably when they are tested on a database that is quite different from the one used to train them, or their significant limitations in predicting observer scores for high framerate videos. In this work we propose a novel objective method for image and video quality assessment that is based on the recently introduced Intrinsically Non-linear Receptive Field (INRF) formulation, a neural summation model that has been shown to be better at predicting neural activity and visual perception phenomena than the classical linear receptive field. Here we start by optimizing, on a classic image quality database, the four parameters of a very simple INRF-based metric, and proceed to test this metric on three other databases, showing that its performance equals or surpasses that of the state-of-the-art methods, some of them having millions of parameters. Next, we extend to the temporal domain this INRF image quality metric, and test it on several popular video quality datasets; again, the results of our proposed INRF-based video quality metric are shown to be very competitive., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 952027 (project AdMiRe) and the Spanish Ministry of Science, grant reference PID2021-127373NB-I00. RL was supported by a Juan de la Cierva-Formación fellowship (FJC2020-044084-I) funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Color matching in the wild

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmio, Marcelo
  • Finlayson, Graham D.
11 pags., 9 figs., 7 tabs., We present a method that, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, corrects the colors of one of the images making it look as if it was captured under the other camera settings. Our method is able to deal with any standard non-linear encoded images (gamma-corrected, logarithmic-encoded, or any other) without requiring any previous knowledge of the encoding. To this end, our method makes use of two important observations. First, the camera imaging pipeline from RAW to sRGB can be well approximated by considering just a per-pixel shading and a color transformation matrix, and second, for correcting the images we only need to estimate a single matrix –that will contain information from both of the original images– and an approximation of the shading term (that emulates the non-linearity). Our proposed method is fast and the results have no spurious artifacts. The method outperforms the state-of-the-art when compared with other methods that do not require knowledge of the encoding used. It is also able to compete with –and even surpass in some cases– methods that consider information about image encoding., This work has received funding from different sources. RGR was supported by ERC Advanced Grant, Color3.0. JVC was supported by Grant PID2021-128178OB-I00 funded by MCIN/AEI/10.13039/ 501100011033 and by ERDF “A way of making Europe”, and also by the Departament de Recerca i Universitats from Generalitat de Catalunya with reference 2021SGR01499 . MB was supported by grant PID2021-127373NB-I00 from the Spanish Ministry of Science, Innovation and Universities (MICINN). GF was funded by EPSRC, United Kingdom Grant EP/S028730/1 ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Overcoming the limitations of motion sensor models by considering dendritic computations

  • Luna, Raúl
  • Serrano-Pedraza, Ignacio
  • Bertalmío, Marcelo
11 pages, 8 figures, The estimation of motion is an essential process for any sighted animal. Computational models of motion sensors have a long and successful history but they still suffer from basic shortcomings, as they disagree with physiological evidence and each model is dedicated to a specific type of motion, which is controversial from a biological standpoint. In this work, we propose a new approach to modeling motion sensors that considers dendritic computations, a key aspect for predicting single-neuron responses that had previously been absent from motion models. We show how, by taking into account the dynamic and input-dependent nature of dendritic nonlinearities, our motion sensor model is able to overcome the fundamental limitations of standard approaches., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. R.L. was supported by a Juan de la Cierva-Formación fellowship (FJC2020-044084-I) funded by Ministerio de Ciencia e Innovación/Agencia Estatal de Investigación (Spain) and by the European Union NextGenerationEU/PRTR. I.S-P was supported by grant PID2021-122245NB-I00, from Ministerio de Ciencia e Innovación (Spain). M.B. was supported by project VIS4NN, Programa Fundamentos 2022, Fundación BBVA, and by grant PID2021-127373NB-I00, Ministerio de Ciencia e Innovación (Spain)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Plaid masking explained with input-dependent dendritic nonlinearities

  • Bertalmio, Marcelo
  • Durán-Vizcaíno, A.
  • Malo, Jesús
  • Wichmann, F.A.
7 pages, 3 figures, A serious obstacle for understanding early spatial vision comes from the failure of the so-called standard model (SM) to predict the perception of plaid masking. But the SM originated from a major oversimplification of single neuron computations, ignoring fundamental properties of dendrites. Here we show that a spatial vision model including computations mimicking the input-dependent nature of dendritic nonlinearities, i.e. including nonlinear neural summation, has the potential to explain plaid masking data., All authors acknowledge funding from VIS4NN, Programa Fundamentos 2022, Fundación BBVA. M.B. and A.D.V. were also supported by PID2021-127373NB-I00, Ministerio de Ciencia e Innovación (Spain). J.M was supported by grants PDC2021-121522-C21 and PID2023-152133NB-I00, Ministerio de Ciencia e Innovacion (Spain) (MICIIN/FEDER/EU). Felix A. Wichmann is a member of the Machine Learning Cluster of Excellence, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC number 2064/1 – project number 390727645. In addition this work was supported by the German Research Foundation (DFG) SFB 1233, Robust Vision: Inference Principles and Neural Mechanisms, project number 276693517. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A Neural Model for V1 That Incorporates Dendritic Nonlinearities and Backpropagating Action Potentials

  • Rentzeperis, Ilias
  • Prandi, Dario
  • Bertalmío, Marcelo
23 pages, 7 figures, 1 table, The work of Hubel and Wiesel has been instrumental in shaping our understanding of V1, leading to modeling neural responses as cascades of linear and nonlinear processes in what is known as the "standard model" of vision. Under this formulation, however, some dendritic properties cannot be represented in a practical manner, while evidence from both experimental and theoretical work indicates that dendritic processes are an indispensable element of key neural behaviors. As a result, current V1 models fail to explain neural responses in a number of scenarios. In this work, we propose an implicit model for V1 that considers nonlinear dendritic integration and backpropagation of action potentials from the soma to the dendrites. Our model can be viewed as an extension of the standard model that minimizes an energy function, allows for a better conceptual understanding of neural processes, and explains several neurophysiological phenomena that have challenged classical approaches., For this project, we acknowledge the funding of PID2021-127373NB-I00, Ministry of Science and Innovation (Spain); VIS4NN - Programa Fundamentos Fundacion BBVA 2022 (Spain); IDEALCV-CM (ref. TEC 2024/COM-322), Comunidad de Madrid (Spain); ANR-20-CE48-0003 and ANR-11-IDEX-0003-02, National Research Agency (France)., Peer reviewed
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Color matching in the wild

  • Gil Rodríguez, Raquel|||0000-0001-5561-1990
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Bertalmío, Marcelo|||0000-0002-1023-8325
  • Finlayson, Graham D.|||0000-0003-4040-0554
We present a method that, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, corrects the colors of one of the images making it look as if it was captured under the other camera settings. Our method is able to deal with any standard non-linear encoded images (gamma-corrected, logarithmic-encoded, or any other) without requiring any previous knowledge of the encoding. To this end, our method makes use of two important observations. First, the camera imaging pipeline from RAW to sRGB can be well approximated by considering just a per-pixel shading and a color transformation matrix, and second, for correcting the images we only need to estimate a single matrix -that will contain information from both of the original images- and an approximation of the shading term (that emulates the non-linearity). Our proposed method is fast and the results have no spurious artifacts. The method outperforms the state-of-the-art when compared with other methods that do not require knowledge of the encoding used. It is also able to compete with -and even surpass in some cases- methods that consider information about image encoding.