Skip to main content

MODELOS DE VISION DE CONTRASTE Y COLOR PARA CINE

TIN2015-71537-P

•
Funding agency name Ministerio de Economía y Competitividad
Funding agency acronym MINECO
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2015
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Management unit Secretaría de Estado de Educación, Formación Profesional y Universidades
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 38
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Color Matching Images with Unknown Non-Linear Encodings

  • Gil Rodríguez, Raquel|||0000-0001-5561-1990
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Bertalmío, Marcelo|||0000-0002-1023-8325
We present a color matching method that deals with different non-linear encodings. In particular, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, and encoded with unknown non-linear curves, our method is able to correct the colors of one of the images making it look as if it was captured under the other camera's settings. Our method is based on treating the in-camera color processing pipeline as a concatenation of a matrix multiplication on the linear image followed by a non-linearity. This allows us to model a color stabilization transformation among the two shots by estimating a single matrix -that will contain information from both of the original images- and an extra parameter that complies with the non-linearity. The method is fast and the results have no spurious colors. It outperforms the state-of-the-art both visually and according to several metrics, and can handle HDR encodings and very challenging real-life examples.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

System gamma as a function of image- and monitor-dynamic range

  • Kane, David
  • Bertalmío, Marcelo
System gamma is the end-to-end exponent that describes the relationship between the relative luminance values at capture and the reproduced image. The system gamma preferred by subjects is known to vary with the background luminance condition and the image in question. We confirm the previous two findings using an image database with both high and low dynamic range images (from 102 to 107), but also find that the preferred system gamma varies with the dynamic range of the monitor (CRT, LCD, or OLED). We find that the preferred system gamma can be predicted in all conditions and for all images by a simple model that searches for the value that best flattens the lightness distribution, where lightness is modeled as a power law of onscreen luminance. To account for the data, the exponent must vary with the viewing conditions. The method presented allows the inference of lightness perception in natural scenes without direct measurement and makes testable predictions for how lightness perception varies with the viewing condition and the distribution of luminance values in a scene. The data from this paper has been made available online., This work was supported by the European Research Council, Starting (Grant ref. 306337), the Spanish government (grant ref. TIN2015-71537-P), and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Analysis of retinal and cortical components of Retinex algorithms

  • Yeonan-Kim, Jihyun
  • Bertalmío, Marcelo
Following Land and McCann’s first proposal of the Retinex theory, numerous Retinex algorithms that differ considerably both algorithmically and functionally have been developed. We clarify the relationships among various Retinex families by associating their spatial processing structures to the neural organizations in the retina and the primary visual cortex in the brain. Some of the Retinex algorithms have a retina-like processing structure (Land’s designator idea and NASA Retinex), and some show a close connection with the cortical structures in the primary visual area of the brain (two-dimensional L&M Retinex). A third group of Retinexes (the variational Retinex) manifests an explicit algorithmic relation to Wilson–Cowan’s physiological model. We intend to overview these three groups of Retinexes with the frame of reference in the biological visual mechanisms., This work was supported by the European Research Council, Starting Grant Ref. 306337, by the Spanish government and FEDER Fund, grant Ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The authors declare that no conflicting interests exist.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Gamut mapping for visual attention retargeting

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Visual attention retargeting attempts to modify an image such that the viewer’s attention is directed to specific regions. Goals include highlighting a particular object or hiding possible problems in the image. In this work, we show that we can pose the visual retargeting problem in terms of gamut mapping. In short, visual attention retargeting can be achieved by performing gamut extension in those regions that we want to highlight and gamut reduction in the other regions., Authors are supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014- 19516.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Gamut reduction through local saturation reduction

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award. The work of Javier Vazquez-Corral was supported by the Spanish government grant IJCI-2014-19516.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Retinal lateral inhibition provides the biological basis of long-range spatial induction

  • Yeonan-Kim, Jihyun
  • Bertalmío, Marcelo
Retinal lateral inhibition is one of the conventional efficient coding mechanisms in the visual system that is produced by interneurons that pool signals over a neighborhood of presynaptic feedforward cells and send inhibitory signals back to them. Thus, the receptive-field (RF) of a retinal ganglion cell has a center-surround receptive-field (RF) profile that is classically represented as a difference-of-Gaussian (DOG) adequate for efficient spatial contrast coding. The DOG RF profile has been attributed to produce the psychophysical phenomena of brightness induction, in which the perceived brightness of an object is affected by that of its vicinity, either shifting away from it (brightness contrast) or becoming more similar to it (brightness assimilation) depending on the size of the surfaces surrounding the object. While brightness contrast can be modeled using a DOG with a narrow surround, brightness assimilation requires a wide suppressive surround. Early retinal studies determined that the suppressive surround of a retinal ganglion cell is narrow (< 100–300 μm; ‘classic RF’), which led researchers to postulate that brightness assimilation must originate at some post-retinal, possibly cortical, stage where long-range interactions are feasible. However, more recent studies have reported that the retinal interneurons also exhibit a spatially wide component (> 500–1000 μm). In the current study, we examine the effect of this wide interneuron RF component in two biophysical retinal models and show that for both of the retinal models it explains the long-range effect evidenced in simultaneous brightness induction phenomena and that the spatial extent of this long-range effect of the retinal model responses matches that of perceptual data. These results suggest that the retinal lateral inhibition mechanism alone can regulate local as well as long-range spatial induction through the narrow and wide RF components of retinal interneurons, arguing against the existing view that spatial induction is operated by two separate local vs. long-range mechanisms., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Issues with common assumptions about the camera pipeline and their impact in hdr imaging from multiple exposures

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Multiple-exposure approaches for high dynamic range (HDR) image generation share a set of building assumptions: that color channels are independent and that the camera response function (CRF) remains constant while changing the exposure. The first contribution of this paper is to highlight how these assumptions, which were correct for film photography, do not hold in general for digital cameras. As a consequence, results of multiexposure HDR methods are less accurate, and when tonemapped they often present problems like hue shifts and color artifacts. The second contribution is to propose a method to stabilize the CRF while coupling all color channels, which can be applied to both static and dynamic scenes, and yield artifact-free results that are more accurate than those obtained with state-of-the-art methods according to several image metrics., The work of the authors was supported by the European Unions Horizon 2020 research and innovation programme grant 761544 (project HDR4EU) and grant 780470 (project SAUCE) and by the Spanish government and FEDER fund grant TIN2015-71537-P (MINECO/FEDER,UE). The work of the second author was supported by the Spanish government under grant IJCI-2014-19516.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A reevaluation of Whittle (1986, 1992) reveals the link between detection thresholds, discrimination thresholds, and brightness perception

  • Kane, David
  • Bertalmío, Marcelo
In 1986, Paul Whittle investigated the ability to discriminate between the luminance of two small patches viewed upon a uniform background. In 1992, Paul Whittle asked subjects to manipulate the luminance of a number of patches on a uniform background until their brightness appeared to vary from black to white with even steps. The data from the discrimination experiment almost perfectly predicted the gradient of the function obtained in the brightness experiment, indicating that the two experimental methodologies were probing the same underlying mechanism. Whittle introduced a model that was able to capture the pattern of discrimination thresholds and, in turn, the brightness data; however, there were a number of features in the data set that the model couldn't capture. In this paper, we demonstrate that the models of Kane and Bertalmío (2017) and Kingdom and Moulden (1991) may be adapted to predict all the data but only by incorporating an accurate model of detection thresholds. Additionally, we show that a divisive gain model may also capture the data but only by considering polarity-dependent, nonlinear inputs following the underlying pattern of detection thresholds. In summary, we conclude that these models provide a simple link between detection thresholds, discrimination thresholds, and brightness perception., This work has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant reference TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Visual information flow in Wilson–Cowan networks

  • Gómez Villa, Alexander
  • Bertalmío, Marcelo
  • Malo, Jesús
In this paper, we study the communication efficiency of a psychophysically tuned cascade of Wilson–Cowan and divisive normalization layers that simulate the retina-V1 pathway. This is the first analysis of Wilson–Cowan networks in terms of multivariate total correlation. The parameters of the cortical model have been derived through the relation between the steady state of the Wilson–Cowan model and the divisive normalization model. The communication efficiency has been analyzed in two ways: First, we provide an analytical expression for the reduction of the total correlation among the responses of a V1-like population after the application of the Wilson–Cowan interaction. Second, we empirically study the efficiency with visual stimuli and statistical tools that were not available before 1) we use a recent, radiometrically calibrated, set of natural scenes, and 2) we use a recent technique to estimate the multivariate total correlation in bits from sets of visual responses, which only involves univariate operations, thus giving better estimates of the redundancy. The theoretical and the empirical results show that, although this cascade of layers was not optimized for statistical independence in any way, the redundancy between the responses gets substantially reduced along the neural pathway. Specifically, we show that 1) the efficiency of a Wilson–Cowan network is similar to its equivalent divisive normalization model; 2) while initial layers (Von Kries adaptation and Weber-like brightness) contribute to univariate equalization, and the bigger contributions to the reduction in total correlation come from the computation of nonlinear local contrast and the application of local oriented filters; and 3) psychophysically tuned models are more efficient (reduce more total correlation) in the more populated regions of the luminance-contrast plane. These results are an alternative confirmation of the efficient coding hypothesis for the Wilson–Cowan systems, and, from an applied perspective, they suggest that neural field models could be an option in image coding to perform image compression., This work was partially funded by the Spanish Government and EU FEDER fund through the MINECO grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Statistics of natural images as a function of dynamic range

  • Grimaldi, Antoine Vincent
  • Kane, David
  • Bertalmío, Marcelo
The statistics of real world images have been extensively investigated, but in virtually all cases using only low dynamic range image databases. The few studies that have considered high dynamic range (HDR) images have performed statistical analyses categorizing images as HDR according to their creation technique, and not to the actual dynamic range of the underlying scene. In this study we demonstrate, using a recent HDR dataset of natural images, that the statistics of the image as received at the camera sensor change dramatically with dynamic range, with particularly strong correlations with dynamic range being observed for the median, standard deviation, skewness, and kurtosis, while the one over frequency relationship for the power spectrum breaks down for images with a very high dynamic range, in practice making HDR images not scale invariant. Effects are also noted in the derivative statistics, the single pixel histograms, and the Haar wavelet analysis. However, we also show that after some basic early transforms occurring within the eye (light scatter, nonlinear photoreceptor response, center-surround modulation) the statistics of the resulting images become virtually independent from the dynamic range, which would allow them to be processed more efficiently by the human visual system., This work has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement Number 761544 (project HDR4EU) and under Grant Agreement Number 780470 (project SAUCE); and by the Spanish government and FEDER Fund, Grantref. TIN2015- 71537-P(MINECO/ FEDER, UE).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Vision models for wide color gamut imaging in cinema

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Gamut mapping is the problem of transforming the colors of image or video content so as to fully exploit the color palette ofthe display device where the content will be shown, while preserving the artistic intent of the original content’s creator. In particular in thecinema industry, the rapid advancement in display technologies has created a pressing need to develop automatic and fast gamutmapping algorithms. In this paper we propose a novel framework that is based on vision science models, performs both gamut reductionand gamut extension, is of low computational complexity, produces results that are free from artifacts and outperforms state-of-the-artmethods according to psychophysical tests. Our experiments also highlight the limitations of existing objective metrics for the gamutmapping problem., Thiswork has received funding from the European Union’sHorizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) andunder grant agreement number 780470 (project SAUCE),and by the Spanish government and FEDER Fund, grantref. TIN2015-71537-P (MINECO/FEDER,UE). The work of J.Vazquez-Corral was supported by the Spanish governmentunder Grant IJCI-2014-19516
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Fusion-based variational image dehazing

  • Galdran, Adrian
  • Vazquez-Corral, Javier
  • Pardo, David
  • Bertalmío, Marcelo
We propose a novel image-dehazing technique based on the minimization of two energy functionals and a fusion scheme to combine the output of both optimizations. The proposed fusion-based variational image-dehazing (FVID) method is a spatially varying image enhancement process that first minimizes a previously proposed variational formulation that maximizes contrast and saturation on the hazy input. The iterates produced by this minimization are kept, and a second energy that shrinks faster intensity values of well-contrasted regions is minimized, allowing to generate a set of difference-of-saturation (DiffSat) maps by observing the shrinking rate. The iterates produced in the first minimization are then fused with these DiffSat maps to produce a haze-free version of the degraded input. The FVID method does not rely on a physical model from which to estimate a depth map, nor it needs a training stage on a database of human-labeled examples. Experimental results on a wide set of hazy images demonstrate that FVID better preserves the image structure on nearby regions that are less affected by fog, and it is successfully compared with other current methods in the task of removing haze degradation from faraway regions., The work of J. Vazquez-Corral and M. Bertalmío was supported by the ERC Starting Grant 306337, by the FEDER Fund under Grant TIN2015-71537-P(MINECO/FEDER,UE), by the ICREA Academia Award, and by the Spanish government under Grant IJCI-2014-19516. The work of D. Pardo was supported by the Spanish government under Grant MTM2013-40824-P, by the BCAM Severo Ochoa accreditation of excellence SEV-2013-0323, and the Basque Government CRG Grant IT649-13.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Color matching images with unknown non-linear encodings

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
We present a color matching method that, given two different views of the same scene taken by two cameras with unknown settings and unknown internal parameter values, and encoded with unknown non-linear curves, is able to correct the colors of one of the images making it look as if it was captured under the other camera’s settings. Our method is based on treating the in-camera color processing pipeline as a matrix multiplication followed by a non-linearity. This allows us to model a color stabilization transformation among the two shots by estimating several parameters. The method is fast and the results have no spurious colors. It outperforms the state-of-the-art both visually and according to several metrics, and can handle very challenging real-life examples., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Spatial gamut mapping among non-inclusive gamuts

  • Bertalmío, Marcelo
  • Vazquez-Corral, Javier
Gamut mapping transforms the color gamut of an image to that of a target device. Two cases are usually considered: gamut reduction (target gamut smaller than source gamut), and gamut extension (target gamut larger than the source gamut). Less attention is devoted to the more general case, when neither gamut is fully included in the other. In this work we unify and expand two recent methods for gamut extension and reduction, so as to simultaneously perform both forms of gamut mapping in di erent regions of the same image without introducing color artifacts or halos. We demonstrate the usefulness of this approach for the traditional gamut mapping problem, and also how the proposed method can be used to adapt the color palette of an image so that it is closer to that of a given reference image. Results are compared with the state-of-the-art and validated through user tests and objective metrics., This work is supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A geometric model of brightness perception and its application to color images correction

  • Batard, Thomas
  • Bertalmío, Marcelo
Human perception involves many features like contours, shapes, textures, and colors to name a few. Whereas several geometric models for contours, shapes and textures perception have been proposed, the geometry of color perception has received very little attention, possibly due to the fact that our perception of colors is still not fully understood. Nonetheless, there exists a class of mathematical models, gathered under the name Retinex, that aim at modeling the color perception of an image, that are inspired by psychophysical/physiological knowledge about color perception, and that can geometrically be viewed as the averaging of perceptual distances between image pixels. Some of the Retinex models turn out to be associated to an efficient image processing technique for the correction of camera output images. The aim of this paper is to show that this image processing technique can be improved by including more properties of the human visual system. To that purpose, we first present a generalization of the perceptual distance between image pixels by considering the parallel transport map associated to a covariant derivative on a vector bundle, and from which can be derived a new image processing model for color images correction. Then, we show that the family of covariant derivatives constructed in [T. Batard and N. Sochen, J. Math. Imaging Vision, 48(3) (2014), pp. 517-543] can model some color appearance phenomena related to brightness perception. Finally, we conduct experiments in which we show that the image processing techniques induced by these covariant derivatives outperform the original approach., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Derivatives and inverse of cascaded linear + nonlinear neural models

  • Martínez García, Marina
  • Cyriac, Praveen
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Malo, Jesús
In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina)., This work was partially funded by the Spanish Ministerio de Economia y Competitividad projects CICYT TEC2013-50520-EXP and CICYT BFU2014-59776-R, by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015- 71537-P(MINECO/FEDER,UE), 1021, and by the ICREA Academia Award.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Issues with common assumptions about the camera pipeline, and their impact in HDR imaging from multiple exposures

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Multiple-exposure approaches for high dynamic range (HDR) image generation share a set of building assumptions: that color channels are independent and that the camera response function (CRF) remains constant while changing the exposure. The first contribution of this paper is to highlight how these assumptions, which were correct for film photography, do not hold in general for digital cameras. As a consequence, results of multiexposure HDR methods are less accurate, and when tone-mapped they often present problems like hue shifts and color artifacts. The second contribution is to propose a method to stabilize the CRF while coupling all color channels, which can be applied to both static and dynamic scenes, and yield artifact-free results that are more accurate than those obtained with state-of-the-art methods according to several image metrics., This work has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Improved high dynamic range video coding with a nonlinearity based on natural image statistics

  • Sugito, Yasuko
  • Cyriac, Praveen
  • Kane, David
  • Bertalmío, Marcelo
High Dynamic Range (HDR) technologies support the capture and presentation of a wider range of luminance values than conventional systems. An important element of video processing is the transfer function which should emulate human perception and this needs to be revisited for HDR content and displays. In the paper, we adapt a nonlinearity designed for the tone-mapping problem to the problem of video coding. We test the nonlinearity using the Motion Picture Experts Group methodology and find it can outperform existing methods in terms of HDR video quality measure., This work was partially supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

In praise of artifice reloaded: Caution with natural image databases in modeling vision

  • Martinez-Garcia, Marina
  • Bertalmío, Marcelo
  • Malo, Jesús
Subjective image quality databases are a major source of raw data on how the visual system works in naturalistic environments. These databases describe the sensitivity of many observers to a wide range of distortions of different nature and intensity seen on top of a variety of natural images. Data of this kind seems to open a number of possibilities for the vision scientist to check the models in realistic scenarios. However, while these natural databases are great benchmarks for models developed in some other way (e.g., by using the well-controlled artificial stimuli of traditional psychophysics), they should be carefully used when trying to fit vision models. Given the high dimensionality of the image space, it is very likely that some basic phenomena are under-represented in the database. Therefore, a model fitted on these large-scale natural databases will not reproduce these under-represented basic phenomena that could otherwise be easily illustrated with well selected artificial stimuli. In this work we study a specific example of the above statement. A standard cortical model using wavelets and divisive normalization tuned to reproduce subjective opinion on a large image quality dataset fails to reproduce basic cross-masking. Here we outline a solution for this problem by using artificial stimuli and by proposing a modification that makes the model easier to tune. Then, we show that the modified model is still competitive in the large-scale database. Our simulations with these artificial stimuli show that when using steerable wavelets, the conventional unit norm Gaussian kernels in divisive normalization should be multiplied by high-pass filters to reproduce basic trends in masking. Basic visual phenomena may be misrepresented in large natural image datasets but this can be solved with model-interpretable stimuli. This is an additional argument in praise of artifice in line with Rust and Movshon (2005)., This work was partially funded by the Spanish and EU FEDER fund through the MINECO/FEDER/EU grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Derivatives and inverse of cascaded linear +nonlinear neural models

  • Martinez-Garcia, Marina
  • Cyriac, P.
  • Batard, T.
  • Bertalmío, Marcelo
  • Malo, Jesús
In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina)., This work was partially funded by the Spanish Ministerio de Economia y Competitividad projects CICYT TEC2013-50520-EXP and CICYT BFU2014-59776-R, by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), 1021, and by the ICREA Academia Award., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Spatial gamut mapping among non-inclusive gamuts

  • Bertalmío, Marcelo
  • Vazquez-Corral, Javier
Gamut mapping transforms the color gamut of an image to that of a target device. Two cases are usually considered: gamut reduction (target gamut smaller than source gamut), and gamut extension (target gamut larger than the source gamut). Less attention is devoted to the more general case, when neither gamut is fully included in the other. In this work we unify and expand two recent methods for gamut extension and reduction, so as to simultaneously perform both forms of gamut mapping in di erent regions of the same image without introducing color artifacts or halos. We demonstrate the usefulness of this approach for the traditional gamut mapping problem, and also how the proposed method can be used to adapt the color palette of an image so that it is closer to that of a given reference image. Results are compared with the state-of-the-art and validated through user tests and objective metrics., This work is supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A reevaluation of Whittle (1986, 1992) reveals the link between detection thresholds, discrimination thresholds, and brightness perception

  • Kane, David
  • Bertalmío, Marcelo
In 1986, Paul Whittle investigated the ability to discriminate between the luminance of two small patches viewed upon a uniform background. In 1992, Paul Whittle asked subjects to manipulate the luminance of a number of patches on a uniform background until their brightness appeared to vary from black to white with even steps. The data from the discrimination experiment almost perfectly predicted the gradient of the function obtained in the brightness experiment, indicating that the two experimental methodologies were probing the same underlying mechanism. Whittle introduced a model that was able to capture the pattern of discrimination thresholds and, in turn, the brightness data; however, there were a number of features in the data set that the model couldn't capture. In this paper, we demonstrate that the models of Kane and Bertalmío (2017) and Kingdom and Moulden (1991) may be adapted to predict all the data but only by incorporating an accurate model of detection thresholds. Additionally, we show that a divisive gain model may also capture the data but only by considering polarity-dependent, nonlinear inputs following the underlying pattern of detection thresholds. In summary, we conclude that these models provide a simple link between detection thresholds, discrimination thresholds, and brightness perception., This work has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant reference TIN2015-71537-P (MINECO/FEDER,UE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Issues with common assumptions about the camera pipeline and their impact in hdr imaging from multiple exposures

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Multiple-exposure approaches for high dynamic range (HDR) image generation share a set of building assumptions: that color channels are independent and that the camera response function (CRF) remains constant while changing the exposure. The first contribution of this paper is to highlight how these assumptions, which were correct for film photography, do not hold in general for digital cameras. As a consequence, results of multiexposure HDR methods are less accurate, and when tonemapped they often present problems like hue shifts and color artifacts. The second contribution is to propose a method to stabilize the CRF while coupling all color channels, which can be applied to both static and dynamic scenes, and yield artifact-free results that are more accurate than those obtained with state-of-the-art methods according to several image metrics., The work of the authors was supported by the European Unions Horizon 2020 research and innovation programme grant 761544 (project HDR4EU) and grant 780470 (project SAUCE) and by the Spanish government and FEDER fund grant TIN2015-71537-P (MINECO/FEDER,UE). The work of the second author was supported by the Spanish government under grant IJCI-2014-19516.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Issues with common assumptions about the camera pipeline, and their impact in HDR imaging from multiple exposures

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Multiple-exposure approaches for high dynamic range (HDR) image generation share a set of building assumptions: that color channels are independent and that the camera response function (CRF) remains constant while changing the exposure. The first contribution of this paper is to highlight how these assumptions, which were correct for film photography, do not hold in general for digital cameras. As a consequence, results of multiexposure HDR methods are less accurate, and when tone-mapped they often present problems like hue shifts and color artifacts. The second contribution is to propose a method to stabilize the CRF while coupling all color channels, which can be applied to both static and dynamic scenes, and yield artifact-free results that are more accurate than those obtained with state-of-the-art methods according to several image metrics., This work has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Fusion-based variational image dehazing

  • Galdran, Adrian
  • Vazquez-Corral, Javier
  • Pardo, David
  • Bertalmío, Marcelo
We propose a novel image-dehazing technique based on the minimization of two energy functionals and a fusion scheme to combine the output of both optimizations. The proposed fusion-based variational image-dehazing (FVID) method is a spatially varying image enhancement process that first minimizes a previously proposed variational formulation that maximizes contrast and saturation on the hazy input. The iterates produced by this minimization are kept, and a second energy that shrinks faster intensity values of well-contrasted regions is minimized, allowing to generate a set of difference-of-saturation (DiffSat) maps by observing the shrinking rate. The iterates produced in the first minimization are then fused with these DiffSat maps to produce a haze-free version of the degraded input. The FVID method does not rely on a physical model from which to estimate a depth map, nor it needs a training stage on a database of human-labeled examples. Experimental results on a wide set of hazy images demonstrate that FVID better preserves the image structure on nearby regions that are less affected by fog, and it is successfully compared with other current methods in the task of removing haze degradation from faraway regions., The work of J. Vazquez-Corral and M. Bertalmío was supported by the ERC Starting Grant 306337, by the FEDER Fund under Grant TIN2015-71537-P(MINECO/FEDER,UE), by the ICREA Academia Award, and by the Spanish government under Grant IJCI-2014-19516. The work of D. Pardo was supported by the Spanish government under Grant MTM2013-40824-P, by the BCAM Severo Ochoa accreditation of excellence SEV-2013-0323, and the Basque Government CRG Grant IT649-13.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Retinal lateral inhibition provides the biological basis of long-range spatial induction

  • Yeonan-Kim, Jihyun
  • Bertalmío, Marcelo
Retinal lateral inhibition is one of the conventional efficient coding mechanisms in the visual system that is produced by interneurons that pool signals over a neighborhood of presynaptic feedforward cells and send inhibitory signals back to them. Thus, the receptive-field (RF) of a retinal ganglion cell has a center-surround receptive-field (RF) profile that is classically represented as a difference-of-Gaussian (DOG) adequate for efficient spatial contrast coding. The DOG RF profile has been attributed to produce the psychophysical phenomena of brightness induction, in which the perceived brightness of an object is affected by that of its vicinity, either shifting away from it (brightness contrast) or becoming more similar to it (brightness assimilation) depending on the size of the surfaces surrounding the object. While brightness contrast can be modeled using a DOG with a narrow surround, brightness assimilation requires a wide suppressive surround. Early retinal studies determined that the suppressive surround of a retinal ganglion cell is narrow (< 100–300 μm; ‘classic RF’), which led researchers to postulate that brightness assimilation must originate at some post-retinal, possibly cortical, stage where long-range interactions are feasible. However, more recent studies have reported that the retinal interneurons also exhibit a spatially wide component (> 500–1000 μm). In the current study, we examine the effect of this wide interneuron RF component in two biophysical retinal models and show that for both of the retinal models it explains the long-range effect evidenced in simultaneous brightness induction phenomena and that the spatial extent of this long-range effect of the retinal model responses matches that of perceptual data. These results suggest that the retinal lateral inhibition mechanism alone can regulate local as well as long-range spatial induction through the narrow and wide RF components of retinal interneurons, arguing against the existing view that spatial induction is operated by two separate local vs. long-range mechanisms., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Improved high dynamic range video coding with a nonlinearity based on natural image statistics

  • Sugito, Yasuko
  • Cyriac, Praveen
  • Kane, David
  • Bertalmío, Marcelo
High Dynamic Range (HDR) technologies support the capture and presentation of a wider range of luminance values than conventional systems. An important element of video processing is the transfer function which should emulate human perception and this needs to be revisited for HDR content and displays. In the paper, we adapt a nonlinearity designed for the tone-mapping problem to the problem of video coding. We test the nonlinearity using the Motion Picture Experts Group methodology and find it can outperform existing methods in terms of HDR video quality measure., This work was partially supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A geometric model of brightness perception and its application to color images correction

  • Batard, Thomas
  • Bertalmío, Marcelo
Human perception involves many features like contours, shapes, textures, and colors to name a few. Whereas several geometric models for contours, shapes and textures perception have been proposed, the geometry of color perception has received very little attention, possibly due to the fact that our perception of colors is still not fully understood. Nonetheless, there exists a class of mathematical models, gathered under the name Retinex, that aim at modeling the color perception of an image, that are inspired by psychophysical/physiological knowledge about color perception, and that can geometrically be viewed as the averaging of perceptual distances between image pixels. Some of the Retinex models turn out to be associated to an efficient image processing technique for the correction of camera output images. The aim of this paper is to show that this image processing technique can be improved by including more properties of the human visual system. To that purpose, we first present a generalization of the perceptual distance between image pixels by considering the parallel transport map associated to a covariant derivative on a vector bundle, and from which can be derived a new image processing model for color images correction. Then, we show that the family of covariant derivatives constructed in [T. Batard and N. Sochen, J. Math. Imaging Vision, 48(3) (2014), pp. 517-543] can model some color appearance phenomena related to brightness perception. Finally, we conduct experiments in which we show that the image processing techniques induced by these covariant derivatives outperform the original approach., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Color matching images with unknown non-linear encodings

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
We present a color matching method that, given two different views of the same scene taken by two cameras with unknown settings and unknown internal parameter values, and encoded with unknown non-linear curves, is able to correct the colors of one of the images making it look as if it was captured under the other camera’s settings. Our method is based on treating the in-camera color processing pipeline as a matrix multiplication followed by a non-linearity. This allows us to model a color stabilization transformation among the two shots by estimating several parameters. The method is fast and the results have no spurious colors. It outperforms the state-of-the-art both visually and according to several metrics, and can handle very challenging real-life examples., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Vision models for wide color gamut imaging in cinema

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Gamut mapping is the problem of transforming the colors of image or video content so as to fully exploit the color palette ofthe display device where the content will be shown, while preserving the artistic intent of the original content’s creator. In particular in thecinema industry, the rapid advancement in display technologies has created a pressing need to develop automatic and fast gamutmapping algorithms. In this paper we propose a novel framework that is based on vision science models, performs both gamut reductionand gamut extension, is of low computational complexity, produces results that are free from artifacts and outperforms state-of-the-artmethods according to psychophysical tests. Our experiments also highlight the limitations of existing objective metrics for the gamutmapping problem., Thiswork has received funding from the European Union’sHorizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) andunder grant agreement number 780470 (project SAUCE),and by the Spanish government and FEDER Fund, grantref. TIN2015-71537-P (MINECO/FEDER,UE). The work of J.Vazquez-Corral was supported by the Spanish governmentunder Grant IJCI-2014-19516
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

System gamma as a function of image- and monitor-dynamic range

  • Kane, David
  • Bertalmío, Marcelo
System gamma is the end-to-end exponent that describes the relationship between the relative luminance values at capture and the reproduced image. The system gamma preferred by subjects is known to vary with the background luminance condition and the image in question. We confirm the previous two findings using an image database with both high and low dynamic range images (from 102 to 107), but also find that the preferred system gamma varies with the dynamic range of the monitor (CRT, LCD, or OLED). We find that the preferred system gamma can be predicted in all conditions and for all images by a simple model that searches for the value that best flattens the lightness distribution, where lightness is modeled as a power law of onscreen luminance. To account for the data, the exponent must vary with the viewing conditions. The method presented allows the inference of lightness perception in natural scenes without direct measurement and makes testable predictions for how lightness perception varies with the viewing condition and the distribution of luminance values in a scene. The data from this paper has been made available online., This work was supported by the European Research Council, Starting (Grant ref. 306337), the Spanish government (grant ref. TIN2015-71537-P), and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Gamut mapping for visual attention retargeting

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Visual attention retargeting attempts to modify an image such that the viewer’s attention is directed to specific regions. Goals include highlighting a particular object or hiding possible problems in the image. In this work, we show that we can pose the visual retargeting problem in terms of gamut mapping. In short, visual attention retargeting can be achieved by performing gamut extension in those regions that we want to highlight and gamut reduction in the other regions., Authors are supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014- 19516.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Visual information flow in Wilson–Cowan networks

  • Gómez Villa, Alexander
  • Bertalmío, Marcelo
  • Malo, Jesús
In this paper, we study the communication efficiency of a psychophysically tuned cascade of Wilson–Cowan and divisive normalization layers that simulate the retina-V1 pathway. This is the first analysis of Wilson–Cowan networks in terms of multivariate total correlation. The parameters of the cortical model have been derived through the relation between the steady state of the Wilson–Cowan model and the divisive normalization model. The communication efficiency has been analyzed in two ways: First, we provide an analytical expression for the reduction of the total correlation among the responses of a V1-like population after the application of the Wilson–Cowan interaction. Second, we empirically study the efficiency with visual stimuli and statistical tools that were not available before 1) we use a recent, radiometrically calibrated, set of natural scenes, and 2) we use a recent technique to estimate the multivariate total correlation in bits from sets of visual responses, which only involves univariate operations, thus giving better estimates of the redundancy. The theoretical and the empirical results show that, although this cascade of layers was not optimized for statistical independence in any way, the redundancy between the responses gets substantially reduced along the neural pathway. Specifically, we show that 1) the efficiency of a Wilson–Cowan network is similar to its equivalent divisive normalization model; 2) while initial layers (Von Kries adaptation and Weber-like brightness) contribute to univariate equalization, and the bigger contributions to the reduction in total correlation come from the computation of nonlinear local contrast and the application of local oriented filters; and 3) psychophysically tuned models are more efficient (reduce more total correlation) in the more populated regions of the luminance-contrast plane. These results are an alternative confirmation of the efficient coding hypothesis for the Wilson–Cowan systems, and, from an applied perspective, they suggest that neural field models could be an option in image coding to perform image compression., This work was partially funded by the Spanish Government and EU FEDER fund through the MINECO grants TIN2015-71537-P and DPI2017-89867-C2-2-R; and by the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Analysis of retinal and cortical components of Retinex algorithms

  • Yeonan-Kim, Jihyun
  • Bertalmío, Marcelo
Following Land and McCann’s first proposal of the Retinex theory, numerous Retinex algorithms that differ considerably both algorithmically and functionally have been developed. We clarify the relationships among various Retinex families by associating their spatial processing structures to the neural organizations in the retina and the primary visual cortex in the brain. Some of the Retinex algorithms have a retina-like processing structure (Land’s designator idea and NASA Retinex), and some show a close connection with the cortical structures in the primary visual area of the brain (two-dimensional L&M Retinex). A third group of Retinexes (the variational Retinex) manifests an explicit algorithmic relation to Wilson–Cowan’s physiological model. We intend to overview these three groups of Retinexes with the frame of reference in the biological visual mechanisms., This work was supported by the European Research Council, Starting Grant Ref. 306337, by the Spanish government and FEDER Fund, grant Ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The authors declare that no conflicting interests exist.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Derivatives and inverse of cascaded linear + nonlinear neural models

  • Martínez García, Marina
  • Cyriac, Praveen
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Malo, Jesús
In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina)., This work was partially funded by the Spanish Ministerio de Economia y Competitividad projects CICYT TEC2013-50520-EXP and CICYT BFU2014-59776-R, by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015- 71537-P(MINECO/FEDER,UE), 1021, and by the ICREA Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Statistics of natural images as a function of dynamic range

  • Grimaldi, Antoine Vincent
  • Kane, David
  • Bertalmío, Marcelo
The statistics of real world images have been extensively investigated, but in virtually all cases using only low dynamic range image databases. The few studies that have considered high dynamic range (HDR) images have performed statistical analyses categorizing images as HDR according to their creation technique, and not to the actual dynamic range of the underlying scene. In this study we demonstrate, using a recent HDR dataset of natural images, that the statistics of the image as received at the camera sensor change dramatically with dynamic range, with particularly strong correlations with dynamic range being observed for the median, standard deviation, skewness, and kurtosis, while the one over frequency relationship for the power spectrum breaks down for images with a very high dynamic range, in practice making HDR images not scale invariant. Effects are also noted in the derivative statistics, the single pixel histograms, and the Haar wavelet analysis. However, we also show that after some basic early transforms occurring within the eye (light scatter, nonlinear photoreceptor response, center-surround modulation) the statistics of the resulting images become virtually independent from the dynamic range, which would allow them to be processed more efficiently by the human visual system., This work has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement Number 761544 (project HDR4EU) and under Grant Agreement Number 780470 (project SAUCE); and by the Spanish government and FEDER Fund, Grantref. TIN2015- 71537-P(MINECO/ FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Gamut reduction through local saturation reduction

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award. The work of Javier Vazquez-Corral was supported by the Spanish government grant IJCI-2014-19516.
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Enhancing spatio-chromatic representation with more-than-three color coding for image description

  • Rafegas Fonoll, Ivet
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Benavente i Vidal, Robert|||0000-0001-9819-4445
  • Vanrell i Martorell, Maria Isabel|||0000-0002-1567-9293
  • Álvarez Fernandez, Susana|||0000-0002-1376-2034
The extraction of spatio-chromatic features from color images is usually performed independently on each color channel. Usual 3D color spaces, such as RGB, present a high inter-channel correlation for natural images. This correlation can be reduced using color-opponent representations, but the spatial structure of regions with small color differences is not fully captured in two generic Red-Green and Blue-Yellow channels. To overcome these problems, we propose new color coding that is adapted to the specific content of each image. Our proposal is based on two steps: (a) setting the number of channels to the number of distinctive colors we find in each image (avoiding the problem of channel correlation), and (b) building a channel representation that maximizes contrast differences within each color channel (avoiding the problem of low local contrast). We call this approach more-than-three color coding (MTT) to emphasize the fact that the number of channels is adapted to the image content. The higher the color complexity of an image, the more channels can be used to represent it. Here we select distinctive colors as the most predominant in the image, which we call color pivots, and we build the new color coding strategy using these color pivots as a basis. To evaluate the proposed approach, we measure the efficiency in an image categorization task. We show how a generic descriptor improves performance at the description level when applied to the MTT coding.