Pasar al contenido principal

PROCESAMIENTO DE IMAGENES PARA REALCE DE FOTOGRAFIA CINEMATOGRAFICA

TIN2012-38112

•
Nombre agencia financiadora Ministerio de Economía y Competitividad
Acrónimo agencia financiadora MINECO
Programa Programa Nacional de Proyectos de Investigación Fundamental
Subprograma Subprograma de Proyectos de Investigación Fundamental no Orientada
Convocatoria Proyectos I+D
Año convocatoria 2012
Plan Estatal 2008-2011 Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica
Unidad de gestión Secretaría de Estado de Investigación, Desarrollo e Innovación
Centro beneficiario UNIVERSITAT POMPEU FABRA CCT
Identificador persistente http://dx.doi.org/10.13039/501100003329

Resultados relacionados

Resultados totales (Incluyendo duplicados): 44
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Connections between Retinex, neural models and variational methods

  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., This paper explores the interrelations between Retinex, neural models and variational methods by making an overview of relevant related works in the past few years. Taking all the essential elements of the Retinex theory as postulated by Land and McCann (channel independence, the ratio reset mechanism, local averages, non-linear correction), it has been shown that we can obtain a Retinex algorithm implementation that is intrinsically 2D, whose results comply with all the expected properties of the original, one-dimensional path-based Retinex algorithm (such as approximating color constancy while being unable to deal with overexposed images) but don't suffer from common shortcomings such as sensitivity to noise, appearance of halos, etc. An iterative application of this 2D Retinex algorithm takes the form of a partial differential equation (PDE) that it's proven not to minimize any energy functional, and this fact is linked to its limitations regarding over-exposed pictures. It was proposed to modify in this regard the iterative method in a certain way so that it is able to handle both under and over-exposed images, the resulting PDE now has a number of very relevant properties which allow to connect Retinex with variational models, histogram equalization and efficient coding, perceptual color correction algorithms, and computational neuroscience models of cortical activity and retinal models., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Image processing applications through a variational perceptually-based color correction related to Retinex

  • Vazquez-Corral, Javier
  • Zamir, Syed Waqas
  • Galdran, Adrian
  • Pardo, David
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., The retinex theory of color vision has been a light of inspiration for color researchers since its inception 50 years ago. It has been adapted to work for different goals such as shadow removal, high dynamic range imaging, or computational color constancy. Back in 2007 a variational perceptually-based color correction model related to Retinex was presented by Bertalmío and colleagues. In this paper we first comment on this model and later we review different image processing applications that have been obtained by performing small modifications to it, namely color gamut mapping and image dehazing., J. Vazquez-Corral. S.W. Zamir, and M. Bertalmío were supported by European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award. D. Pardo was partially funded by the Project of the Spanish Ministry of Economy and Competitiveness with reference MTM2013-40824-P, the BCAM ”Severo Ochoa” accreditation of excellence SEV-2013-0323, the CYTED 2011 project 712RT0449, and the Basque Government Consolidated Research Group Grant IT649-13 on “Mathematical Modeling, Simulation, and Industrial Applications (M2SI)”.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Simultaneous blind gamma estimation

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Blind gamma estimation is the problem of estimating the gamma function that is applied to a linear image both for perceptual reasons and for the compensation of the non-linear behavior of displays. Gamma values change both inter- and intra-camera. In the latter case, the change comes from the use of different scene settings. In this letter we propose a new approach that relies on the use of more than a single image from the same scene. We estimate the gammas for all the different images at the same time with a method based on exploiting the structure of the standard in-camera processing pipeline. Our results improve over the state-of-the-art., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Spectral sharpening of color sensors: diagonal color constancy and beyond

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
It has now been 20 years since the seminal work by Finlayson et al. on the use/nof spectral sharpening of sensors to achieve diagonal color constancy. Spectral sharpening is/nstill used today by numerous researchers for different goals unrelated to the original goal/nof diagonal color constancy e.g., multispectral processing, shadow removal, location of/nunique hues. This paper reviews the idea of spectral sharpening through the lens of what/nis known today in color constancy, describes the different methods used for obtaining a set/nof sharpening sensors and presents an overview of the many different uses that have been/nfound for spectral sharpening over the years., This work was supported by European Research Council, Starting Grant ref. 306337, and by Spanish/ngrants ref. TIN2011-15954-E, and ref. TIN2012-38112
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A class of nonlocal variational problems on a vector bundle for color image local contrast reduction/enhancement

  • Bertalmío, Marcelo
  • Batard, Thomas
We extend two existing variational models from the Euclidean space to a vector bundle over a Riemannian manifold. The Euclidean models, dedicated to regularize or enhance some color image features, are based on the concept of nonlocal gradient operator acting on a function of the Euclidean space. We then extend these models by generalizing this operator to a vector bundle over a Riemannian manifold with the help of the parallel transport map associated to some class of covariant derivatives. Through the dual formulations of the proposed models, we obtain the expressions of their solutions, which exhibit the functional spaces that describe the image features. Finally, for a well-chosen covariant derivative and its nonlocal extension, the proposed models perform local contrast modification (reduction or enhancement) and experiments show that they preserve more the aspect of the original image than the Euclidean models do while modifying equally its contrast., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Color stabilization along time and across shots of the same scene, for one or several cameras of unknown specifications

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
We propose a method for color stabilization of shots of the same scene, taken under the same illumination, where one image is chosen as reference and one or several other images are modified so that their colors match those of the reference. We make use of two crucial but often overlooked observations: first, that the core of the color correction chain in a digital camera is simply a multiplication by a 3 × 3 matrix; second, that to color-match a source image to a reference image we do not need to compute their two color correction matrices, it is enough to compute the operation that transforms one matrix into the other. This operation is a 3 × 3 matrix as well, which we call H. Once we have H, we just multiply by it each pixel value of the source and obtain an image which matches in color the reference. To compute H we only require a set of pixel correspondences, we do not need any information about the cameras used, neither models nor specifications or parameter values. We propose an implementation of our framework which is very simple and fast, and show how it can be successfully employed in a number of situations, comparing favorably with the state of the art. There is a wide range of applications of our technique, both for amateur and professional photography and video: color matching for multicamera TV broadcasts, color matching for 3D cinema, color stabilization for amateur video, etc., This work was supported in part by the European Research Council under Grant 306337 and in part by the Spanish Research under Grant TIN2011-15954-E and Grant TIN2012-38112.
Repositorio Digital de la UPF
Publicación Capítulo o parte de un libro (BookPart)

Light and color

  • Bertalmío, Marcelo
Image Processing for Cinema presents a detailed overview of image processing techniques that are used in practice in digital cinema. The book shows how image processing has become ubiquitous in movie-making, from shooting to exhibition. It covers all the ways in which image processing algorithms are used to enhance, restore, adapt, and convert moving images. These techniques and algorithms make the images look as good as possible while exploiting the capabilities of cameras, projectors, and displays. The author focuses on the ideas behind the methods, rather than proofs and derivations. The first part of the text presents fundamentals on optics and color. The second part explains how cameras work and details all the image processing algorithms that are applied in-camera. With an emphasis on state-of-the-art methods that are actually used in practice, the last part describes image processing algorithms that are applied offline to solve a variety of problems. The book is designed for advanced undergraduate and graduate students in applied mathematics, image processing, computer science, and related fields. It is also suitable for academic researchers and professionals in the movie industry., The author would like to acknowledge the support of the European Research Council through the Starting Grant ref. 306337, of ICREA through their ICREA Acadèmia Award, and of the Spanish government through the grants TIN2012-38112 and TIN2011-1594-E.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A decomposition framework for image denoising algorithms

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Levine, Stacey
In this paper, we consider an image decomposition model that provides a novel framework for image denoising. The model computes the components of the image to be processed in a moving frame that encodes its local geometry (directions of gradients and level lines). Then, the strategy we develop is to denoise the components of the image in the moving frame in order to preserve its local geometry, which would have been more affected if processing the image directly. Experiments on a whole image database tested with several denoising methods show that this framework can provide better results than denoising the image directly, both in terms of Peak signal-to-noise ratio and Structural similarity index metrics., The work of G. Ghimpeteanu, T. Batard, and/nM. Bertalmío was supported in part by the Spanish Government under/nGrant TIN2012-38112, in part by the Icrea Academia Award, and in/npart by the European Research Council under Grant 306337. The work/nof S. Levine was supported by the Na/ntional Science Foundation under/nGrant NSF-DMS 1320829.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A tone mapping operator based on neural and psychophysical models of visual perception

  • Cyriac, Praveen
  • Bertalmío, Marcelo
  • Kane, David
  • Vazquez-Corral, Javier
High dynamic range imaging techniques involve capturing and storing real world radiance values that span many orders of magnitude. However, common display devices can usually reproduce intensity ranges only up to two to three orders of magnitude. Therefore, in order to display a high dynamic range image on a low dynamic range screen, the dynamic range of the image needs to be compressed without losing details or introducing artefacts, and this process is called tone mapping. A good tone mapping operator must be able to produce a low dynamic range image that matches as much as possible the perception of the real world scene. We propose a two stage tone mapping approach, in which the first stage is a global method for range compression based on a gamma curve that equalizes the lightness histogram the best, and the second stage performs local contrast enhancement and color induction using neural activity models for the visual cortex. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

From image processing to computational neuroscience: A neural model based on histogram equalization

  • Bertalmío, Marcelo
There are many ways in which the human visual system works to reduce the inherent redundancy of the visual information in natural scenes, coding it in an efficient way. The non-linear response curves of photoreceptors and the spatial organization of the receptive fields of visual neurons both work toward this goal of efficient coding. A related, very important aspect is that of the existence of post-retinal mechanisms for contrast enhancement that compensate for the blurring produced in early stages of the visual process. And alongside mechanisms for coding and wiring efficiency, there is neural activity in the human visual cortex that correlates with the perceptual phenomenon of lightness induction. In this paper we propose a neural model that is derived from an image processing technique for histogram equalization, and that is able to deal with all the aspects just mentioned: this new model is able to predict lightness induction phenomena, and improves the efficiency of the representation by flattening both the histogram and the power spectrum of the image signal., This work was supported by European Research Council, Starting Grant ref. 306337, by the ICREA Acadèemia Award and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Color matching for stereoscopic cinema

  • Bertalmío, Marcelo
  • Levine, Stacey
In a stereo camera set-up for 3D cinema, much care is taken to ensure/nthat the cameras are both the same model and use identical/nparameter values. But an imperfect adjustment, the use of beam/nsplitters, as well as unavoidable differences in the characteristics/nof the cameras and lens systems, often cause visible differences in/ncolor among the two views. Many of these differences appear to/nbe local and therefore cannot be fully removed by common color/nmatching approaches, which are typically global methods. We propose/na simple yet effective method for local color matching, that/noperates in three steps. First, one view is chosen as the target and/nmorphed so that it becomes locally registered with the source view,/nthe one whose colors will be modified. Next, color matching is performed/non the source view by aligning each local histogram with/nthe corresponding local histogram in the warped target view. Finally,/nPoisson editing [9] is applied on the source regions which/nhave no correspondence in the target. For several professional, high/nquality examples of 3D sequences, we show that our algorithm outperforms/nfour different conventional color matching approaches., We would like to thank Pierre Jasmin of RE:Vision Effects Inc./nfor his unwavering support and his help running tests, and Clark/nDunbar of Mammoth HD Inc. for supplying us with professional/n3D footage. M. Bertalmío acknowledges support by European Research/nCouncil, Starting Grant ref. 306337, and Spanish grants AACC, ref. TIN2011-15954-E, and Plan Nacional, ref. TIN2012-/n38112. S. Levine is funded in part by NSF-DMS #0915219.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A variational method for the optimization of tone mapping operators

  • Cyriac, Praveen
  • Batard, Thomas
  • Bertalmío, Marcelo
Comunicació presentada a: 6th Pacific-Rim Symposium on Image and Video Technology, celebrat del 28 d'octubre a 1 de novembre de 2013 a Guanajuato, Mèxic., Given any metric that compares images of di erent dynamic range, we propose a method to reduce their distance with respect to this metric. The key idea is to consider the metric as a non local operator. Then, we transform the problem of distance reduction into a non local variational problem. In this context, the low dynamic range image having the smallest distance with a given high dynamic range is the minimum of a suitable energy, and can be reached through a gradient descent algorithm. Dealing with an appropriate metric, we present an application to Tone Mapping Operator (TMO) optimization. We apply our gradient descent algorithm, where the initial conditions are Tone Mapped (TM) images. Experiments show that our algorithm does reduce the distance of the TM images with the high dynamic range source images, meaning that our method improves the corresponding TMOs., This work was supported by the European Research Council, Starting Grant ref. 306337, and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

High quality video in high dynamic range scenes from interlaced dual-ISO footage

  • Gil Rodríguez, Raquel
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., In this paper we present a simple and affordable method to generate high quality video from a high dynamic range scene. It is performed without utilizing extra lighting, neither alternating exposures, nor operating with dual-camera set-ups. Our input is an interlaced video alternating row pairs with different ISO values, as some DSLR camera models can provide. The proposed algorithm involves two main steps: first the computation of two single-ISO full-frame images (one for each ISO value) using an inpainting-based deinterlacing method, followed by their combination into a single frame by a weighted average. This results in a high dynamic range frame, containing all the details in bright and dark areas at the same time, that is finally tone-mapped into a low dynamic range frame for display purposes. Current results show this is a practical and cost-effective method that produces outputs free of ghosting artifacts and with very little noise., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Denoising an image by denoising its components in a moving frame

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Levine, Stacey
Comunicació presentada a: the 6th International Conference, (ICISP 2014), celebrada del 30. de juny al 2 de juliol de 2014 a Cherbourg, França., In this paper, we provide a new non-local method for image denoising. The key idea we develop is to denoise the components of the image in a well-chosen moving frame instead of the image itself.We prove the relevance of our approach by showing that the PSNR of a grayscale noisy image is lower than the PSNR of its components. Experiments show that applying the Non Local Means algorithm of Buades et al. on the components provides better results than applying it directly on the image., This work was supported by European Research Council, Starting Grant ref. 306337, and by Spanish grants AACC, ref. TIN2011-15954-E, and Plan Na- cional, ref. TIN2012-38112. S. Levine acknowledges partial support by NSF-DMS #0915219.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Perceptual color characterization of cameras

  • Vazquez-Corral, Javier
  • Connah, David
  • Bertalmío, Marcelo
Color camera characterization, mapping outputs from the camera sensors to an/nindependent color space, such as XY Z, is an important step in the camera processing/npipeline. Until now, this procedure has been primarily solved by using a 3 × 3 matrix/nobtained via a least-squares optimization. In this paper, we propose to use the spherical/nsampling method, recently published by Finlayson et al., to perform a perceptual color/ncharacterization. In particular, we search for the 3 × 3 matrix that minimizes three different/nperceptual errors, one pixel based and two spatially based. For the pixel-based case,/nwe minimize the CIE ∆E error, while for the spatial-based case, we minimize both the/nS-CIELAB error and the CID error measure. Our results demonstrate an improvement of/napproximately 3% for the ∆E error, 7% for the S-CIELAB error and 13% for the CID/nerror measures, This work was supported by European Research Council, Starting Grant Ref. 306337, and by Spanish/ngrants AACC, Ref. TIN2011-15954-E, and Plan Nacional, Ref. TIN2012-38112.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A variational framework for single image dehazing

  • Galdran, Adrian
  • Vazquez-Corral, Javier
  • Pardo, David
  • Bertalmío, Marcelo
Comunicació presentada a: European Conference on Computer Vision Workshops (ECCV 2014), celebrada del 6 al 7 de setembre de 2014 a Zurich, Suïssa., Images captured under adverse weather conditions, such as haze or fog, typically exhibit low contrast and faded colors, which may severely limit the visibility within the scene. Unveiling the image struc- ture under the haze layer and recovering vivid colors out of a single image remains a challenging task, since the degradation is depth-dependent and conventional methods are unable to handle this problem. We propose to extend a well-known perception-inspired variational frame- work [1] for the task of single image dehazing. The main modification consists on the replacement of the value used by this framework for the grey-world hypothesis by an estimation of the mean of the clean image. This allows us to devise a variational method that requires no estimate of the depth structure of the scene, performing a spatially-variant contrast enhancement that effectively removes haze from far away regions. Experimental results show that our method competes well with other state- of-the-art methods in typical benchmark images, while outperforming current image dehazing methods in more challenging scenarios., JVC and MB were supported by European Research Council, Starting Grant ref. 306337, and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Gamut extension for cinema: psychophysical evaluation of the state of the art and a new algorithm

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Wide gamut digital display technology, in order to show its full potential in terms of colors, is creating an opportunity to develop gamut extension algorithms (GEAs). To this end, in this work we present two contributions. First we report a psychophysical evaluation of GEAs specifically for cinema using a digital cinema projector under cinematic (low ambient light) conditions; to the best of our knowledge this is the first evaluation of this kind reported in the literature. Second, we propose a new GEA by introducing simple but key modifications to the algorithm of Zamir et al. This new algorithm performs well in terms of skin tones and memory colors, with results that look natural and which are free from artifacts. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only., This work was supported by the European Research Council, Starting Grant ref. 306337, by Spanish grant ref. TIN2012-38112 and by ICREA Academia
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Perceptually inspired gamut mapping between any gamuts with any intersection

  • Bertalmío, Marcelo
  • Vazquez-Corral, Javier
Comunicació presentada a la AIC 2015 Tokyo Midterm Meeting, celebrada a Tòquio (Japó) del 19 a 22 de maig de 2015., Gamut mapping transforms the color of an input image within the range of a target device. A huge amount of research has been devoted to two subproblems that arise from this general one: gamut reduction and gamut extension. Gamut reduction algorithms convert the input image to a new gamut that fits inside the one of the image , i.e. the gamuts’ intersection is equal to the target gamut, while gamut extension algorithms convert the input image to a gamut that embodies the original image gamut, i.e. the gamuts’ intersection is equal to the source gamut. In contrast to the two aforementioned cases, very little attention has been paid to the most general problem, where the intersection of source and target gamut is not equal to one of the two gamuts. In this paper we address this most general problem of gamut mapping between any two gamuts presenting any possible intersection. To deal with this problem we unify the gamut extension and gamut reduction algorithms presented in Zamir –et al- (Zamir 2014), which are based in the perceptually inspired variational framework of Bertalmío –et al- (Bertalmío 2007) that presents three competing terms; an attachment to the original data, a term for not-modifying the perchannel image mean (i.e. not modifying the white point), and a contrast enhancement term. In particular, in this paper we show how by defining a smooth transition on the contrast enhancement parameter over the chromaticity diagram we can simultaneously reduce the input gamut in some chromatic areas while increasing it in some other without introducing neither color artifacts nor halos., This work was supported by the European Research Council, Starting Grant ref.306337, by the Spanish Government, grant red.TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Local denoising applied to RAW images may outperform non-local patch-based methods applied to the camera output

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Seybold, Tamara
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., State-of-the-art denoising methods achieve impressive results, even for large noise levels. However, they can not be implemented in camera hardware, mainly due to the fact that they are computationally too intensive. The aim of this paper is then to show that we can obtain comparable denoising results to the ones obtained with state-of-art methods by inserting a well-chosen fast denoising method at the right location in the camera processing pipeline. We evaluate our results visually and with respect to objective measures., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Optimized tone curve for in-camera image processing

  • Cyriac, Praveen
  • Kane, David
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., The sensor values captured by a digital camera are transformed in a non-linear manner prior to quantization in order to make the quantization rate approximately proportional to the sensitivity of the human visual system. We propose an image dependent non-linear transformation that can accurately reproduce the detail and contrast visible in the original scene. The principles underpinning the transform stem from an understanding of natural image statistics, as well as recent experimental and neurophysiological findings. To optimize the parameters of the model we collect user-feedback and develop a method that can predict the user defined parameters. The method we have developed has an extremely low computational complexity, therefore it operates almost instantaneously making it suitable for in-camera operations. The final image looks natural, without any halos, spurious colors or artifacts. It can also be applied to video sequences, after imposing temporal coherence on the parameter values by smoothing them over time. The proposed approach is validated through psychophysical tests that confirm that it outperforms other state of the art algorithms in terms of users' preference., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Duality principle for image regularization and perceptual color correction models

  • Bertalmío, Marcelo
  • Batard, Thomas
Comunicació presentada a la 5th International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2015), celebrada a Lège-Cap Ferret (França) del dia 31 de maig al 4 de juny de 2015., In this paper, we show that the anisotropic nonlocal total variation involved in the image regularization model of Gilboa and Osher as well as in the perceptual color correction model of Bertalmío et al. possesses a dual formulation. We then obtain novel formulations of their solutions, which provide new insights on these models. In particular, we show that the model of Bertalmío et al. can be split into two steps: first, it performs global color constancy, then local contrast enhancement. We also extend these two channel-wise variational models in a vectorial way by extending the anisotropic nonlocal total variation to vector-valued functions., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Harmonic flow for histogram matching

  • Batard, Thomas
  • Bertalmío, Marcelo
Comunicació presentada a: 6th Pacific-Rim Symposium on Image and Video Technology Workshop, celebrat del 28-29 d'octubre de 2013 a Guanajuato, Mèxic., We present a method to perform histogram matching be- tween two color images based on the concept of harmonic mapping be- tween Riemannian manifolds. The key idea is to associate the histogram of a color image to a Riemannian manifold. In this context, the energy of the matching between the two images is measured by the Dirichlet energy of the mapping between the Riemannian manifolds. Then, we assimilate optimal matchings to critical points of the Dirichlet energy. Such points are called harmonic maps. As there is no explicit expression for harmonic maps in general, we use a gradient descent ow with boundary condition to reach them, that we call harmonic ow. We present an application to color transfer, however many others applications can be envisaged using this general framework., This work was supported by European Research Council, Starting Grant ref. 306337. The second author acknowledges partial support by Spanish grants AACC, ref. TIN2011-15954-E, and Plan Nacional, ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Duality principle for image regularization and perceptual color correction models

  • Bertalmío, Marcelo
  • Batard, Thomas
Comunicació presentada a la 5th International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2015), celebrada a Lège-Cap Ferret (França) del dia 31 de maig al 4 de juny de 2015., In this paper, we show that the anisotropic nonlocal total variation involved in the image regularization model of Gilboa and Osher as well as in the perceptual color correction model of Bertalmío et al. possesses a dual formulation. We then obtain novel formulations of their solutions, which provide new insights on these models. In particular, we show that the model of Bertalmío et al. can be split into two steps: first, it performs global color constancy, then local contrast enhancement. We also extend these two channel-wise variational models in a vectorial way by extending the anisotropic nonlocal total variation to vector-valued functions., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Spectral sharpening of color sensors: diagonal color constancy and beyond

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
It has now been 20 years since the seminal work by Finlayson et al. on the use/nof spectral sharpening of sensors to achieve diagonal color constancy. Spectral sharpening is/nstill used today by numerous researchers for different goals unrelated to the original goal/nof diagonal color constancy e.g., multispectral processing, shadow removal, location of/nunique hues. This paper reviews the idea of spectral sharpening through the lens of what/nis known today in color constancy, describes the different methods used for obtaining a set/nof sharpening sensors and presents an overview of the many different uses that have been/nfound for spectral sharpening over the years., This work was supported by European Research Council, Starting Grant ref. 306337, and by Spanish/ngrants ref. TIN2011-15954-E, and ref. TIN2012-38112
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

From image processing to computational neuroscience: A neural model based on histogram equalization

  • Bertalmío, Marcelo
There are many ways in which the human visual system works to reduce the inherent redundancy of the visual information in natural scenes, coding it in an efficient way. The non-linear response curves of photoreceptors and the spatial organization of the receptive fields of visual neurons both work toward this goal of efficient coding. A related, very important aspect is that of the existence of post-retinal mechanisms for contrast enhancement that compensate for the blurring produced in early stages of the visual process. And alongside mechanisms for coding and wiring efficiency, there is neural activity in the human visual cortex that correlates with the perceptual phenomenon of lightness induction. In this paper we propose a neural model that is derived from an image processing technique for histogram equalization, and that is able to deal with all the aspects just mentioned: this new model is able to predict lightness induction phenomena, and improves the efficiency of the representation by flattening both the histogram and the power spectrum of the image signal., This work was supported by European Research Council, Starting Grant ref. 306337, by the ICREA Acadèemia Award and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Gamut extension for cinema: psychophysical evaluation of the state of the art and a new algorithm

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Wide gamut digital display technology, in order to show its full potential in terms of colors, is creating an opportunity to develop gamut extension algorithms (GEAs). To this end, in this work we present two contributions. First we report a psychophysical evaluation of GEAs specifically for cinema using a digital cinema projector under cinematic (low ambient light) conditions; to the best of our knowledge this is the first evaluation of this kind reported in the literature. Second, we propose a new GEA by introducing simple but key modifications to the algorithm of Zamir et al. This new algorithm performs well in terms of skin tones and memory colors, with results that look natural and which are free from artifacts. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only., This work was supported by the European Research Council, Starting Grant ref. 306337, by Spanish grant ref. TIN2012-38112 and by ICREA Academia
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A variational method for the optimization of tone mapping operators

  • Cyriac, Praveen
  • Batard, Thomas
  • Bertalmío, Marcelo
Comunicació presentada a: 6th Pacific-Rim Symposium on Image and Video Technology, celebrat del 28 d'octubre a 1 de novembre de 2013 a Guanajuato, Mèxic., Given any metric that compares images of di erent dynamic range, we propose a method to reduce their distance with respect to this metric. The key idea is to consider the metric as a non local operator. Then, we transform the problem of distance reduction into a non local variational problem. In this context, the low dynamic range image having the smallest distance with a given high dynamic range is the minimum of a suitable energy, and can be reached through a gradient descent algorithm. Dealing with an appropriate metric, we present an application to Tone Mapping Operator (TMO) optimization. We apply our gradient descent algorithm, where the initial conditions are Tone Mapped (TM) images. Experiments show that our algorithm does reduce the distance of the TM images with the high dynamic range source images, meaning that our method improves the corresponding TMOs., This work was supported by the European Research Council, Starting Grant ref. 306337, and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Perceptually inspired gamut mapping between any gamuts with any intersection

  • Bertalmío, Marcelo
  • Vazquez-Corral, Javier
Comunicació presentada a la AIC 2015 Tokyo Midterm Meeting, celebrada a Tòquio (Japó) del 19 a 22 de maig de 2015., Gamut mapping transforms the color of an input image within the range of a target device. A huge amount of research has been devoted to two subproblems that arise from this general one: gamut reduction and gamut extension. Gamut reduction algorithms convert the input image to a new gamut that fits inside the one of the image , i.e. the gamuts’ intersection is equal to the target gamut, while gamut extension algorithms convert the input image to a gamut that embodies the original image gamut, i.e. the gamuts’ intersection is equal to the source gamut. In contrast to the two aforementioned cases, very little attention has been paid to the most general problem, where the intersection of source and target gamut is not equal to one of the two gamuts. In this paper we address this most general problem of gamut mapping between any two gamuts presenting any possible intersection. To deal with this problem we unify the gamut extension and gamut reduction algorithms presented in Zamir –et al- (Zamir 2014), which are based in the perceptually inspired variational framework of Bertalmío –et al- (Bertalmío 2007) that presents three competing terms; an attachment to the original data, a term for not-modifying the perchannel image mean (i.e. not modifying the white point), and a contrast enhancement term. In particular, in this paper we show how by defining a smooth transition on the contrast enhancement parameter over the chromaticity diagram we can simultaneously reduce the input gamut in some chromatic areas while increasing it in some other without introducing neither color artifacts nor halos., This work was supported by the European Research Council, Starting Grant ref.306337, by the Spanish Government, grant red.TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Harmonic flow for histogram matching

  • Batard, Thomas
  • Bertalmío, Marcelo
Comunicació presentada a: 6th Pacific-Rim Symposium on Image and Video Technology Workshop, celebrat del 28-29 d'octubre de 2013 a Guanajuato, Mèxic., We present a method to perform histogram matching be- tween two color images based on the concept of harmonic mapping be- tween Riemannian manifolds. The key idea is to associate the histogram of a color image to a Riemannian manifold. In this context, the energy of the matching between the two images is measured by the Dirichlet energy of the mapping between the Riemannian manifolds. Then, we assimilate optimal matchings to critical points of the Dirichlet energy. Such points are called harmonic maps. As there is no explicit expression for harmonic maps in general, we use a gradient descent ow with boundary condition to reach them, that we call harmonic ow. We present an application to color transfer, however many others applications can be envisaged using this general framework., This work was supported by European Research Council, Starting Grant ref. 306337. The second author acknowledges partial support by Spanish grants AACC, ref. TIN2011-15954-E, and Plan Nacional, ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A decomposition framework for image denoising algorithms

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Levine, Stacey
In this paper, we consider an image decomposition model that provides a novel framework for image denoising. The model computes the components of the image to be processed in a moving frame that encodes its local geometry (directions of gradients and level lines). Then, the strategy we develop is to denoise the components of the image in the moving frame in order to preserve its local geometry, which would have been more affected if processing the image directly. Experiments on a whole image database tested with several denoising methods show that this framework can provide better results than denoising the image directly, both in terms of Peak signal-to-noise ratio and Structural similarity index metrics., The work of G. Ghimpeteanu, T. Batard, and/nM. Bertalmío was supported in part by the Spanish Government under/nGrant TIN2012-38112, in part by the Icrea Academia Award, and in/npart by the European Research Council under Grant 306337. The work/nof S. Levine was supported by the Na/ntional Science Foundation under/nGrant NSF-DMS 1320829.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A variational framework for single image dehazing

  • Galdran, Adrian
  • Vazquez-Corral, Javier
  • Pardo, David
  • Bertalmío, Marcelo
Comunicació presentada a: European Conference on Computer Vision Workshops (ECCV 2014), celebrada del 6 al 7 de setembre de 2014 a Zurich, Suïssa., Images captured under adverse weather conditions, such as haze or fog, typically exhibit low contrast and faded colors, which may severely limit the visibility within the scene. Unveiling the image struc- ture under the haze layer and recovering vivid colors out of a single image remains a challenging task, since the degradation is depth-dependent and conventional methods are unable to handle this problem. We propose to extend a well-known perception-inspired variational frame- work [1] for the task of single image dehazing. The main modification consists on the replacement of the value used by this framework for the grey-world hypothesis by an estimation of the mean of the clean image. This allows us to devise a variational method that requires no estimate of the depth structure of the scene, performing a spatially-variant contrast enhancement that effectively removes haze from far away regions. Experimental results show that our method competes well with other state- of-the-art methods in typical benchmark images, while outperforming current image dehazing methods in more challenging scenarios., JVC and MB were supported by European Research Council, Starting Grant ref. 306337, and by Spanish grants ref. TIN2011-15954-E and ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Simultaneous blind gamma estimation

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Blind gamma estimation is the problem of estimating the gamma function that is applied to a linear image both for perceptual reasons and for the compensation of the non-linear behavior of displays. Gamma values change both inter- and intra-camera. In the latter case, the change comes from the use of different scene settings. In this letter we propose a new approach that relies on the use of more than a single image from the same scene. We estimate the gammas for all the different images at the same time with a method based on exploiting the structure of the standard in-camera processing pipeline. Our results improve over the state-of-the-art., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Color matching for stereoscopic cinema

  • Bertalmío, Marcelo
  • Levine, Stacey
In a stereo camera set-up for 3D cinema, much care is taken to ensure/nthat the cameras are both the same model and use identical/nparameter values. But an imperfect adjustment, the use of beam/nsplitters, as well as unavoidable differences in the characteristics/nof the cameras and lens systems, often cause visible differences in/ncolor among the two views. Many of these differences appear to/nbe local and therefore cannot be fully removed by common color/nmatching approaches, which are typically global methods. We propose/na simple yet effective method for local color matching, that/noperates in three steps. First, one view is chosen as the target and/nmorphed so that it becomes locally registered with the source view,/nthe one whose colors will be modified. Next, color matching is performed/non the source view by aligning each local histogram with/nthe corresponding local histogram in the warped target view. Finally,/nPoisson editing [9] is applied on the source regions which/nhave no correspondence in the target. For several professional, high/nquality examples of 3D sequences, we show that our algorithm outperforms/nfour different conventional color matching approaches., We would like to thank Pierre Jasmin of RE:Vision Effects Inc./nfor his unwavering support and his help running tests, and Clark/nDunbar of Mammoth HD Inc. for supplying us with professional/n3D footage. M. Bertalmío acknowledges support by European Research/nCouncil, Starting Grant ref. 306337, and Spanish grants AACC, ref. TIN2011-15954-E, and Plan Nacional, ref. TIN2012-/n38112. S. Levine is funded in part by NSF-DMS #0915219.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

High quality video in high dynamic range scenes from interlaced dual-ISO footage

  • Gil Rodríguez, Raquel
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., In this paper we present a simple and affordable method to generate high quality video from a high dynamic range scene. It is performed without utilizing extra lighting, neither alternating exposures, nor operating with dual-camera set-ups. Our input is an interlaced video alternating row pairs with different ISO values, as some DSLR camera models can provide. The proposed algorithm involves two main steps: first the computation of two single-ISO full-frame images (one for each ISO value) using an inpainting-based deinterlacing method, followed by their combination into a single frame by a weighted average. This results in a high dynamic range frame, containing all the details in bright and dark areas at the same time, that is finally tone-mapped into a low dynamic range frame for display purposes. Current results show this is a practical and cost-effective method that produces outputs free of ghosting artifacts and with very little noise., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A class of nonlocal variational problems on a vector bundle for color image local contrast reduction/enhancement

  • Bertalmío, Marcelo
  • Batard, Thomas
We extend two existing variational models from the Euclidean space to a vector bundle over a Riemannian manifold. The Euclidean models, dedicated to regularize or enhance some color image features, are based on the concept of nonlocal gradient operator acting on a function of the Euclidean space. We then extend these models by generalizing this operator to a vector bundle over a Riemannian manifold with the help of the parallel transport map associated to some class of covariant derivatives. Through the dual formulations of the proposed models, we obtain the expressions of their solutions, which exhibit the functional spaces that describe the image features. Finally, for a well-chosen covariant derivative and its nonlocal extension, the proposed models perform local contrast modification (reduction or enhancement) and experiments show that they preserve more the aspect of the original image than the Euclidean models do while modifying equally its contrast., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Connections between Retinex, neural models and variational methods

  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., This paper explores the interrelations between Retinex, neural models and variational methods by making an overview of relevant related works in the past few years. Taking all the essential elements of the Retinex theory as postulated by Land and McCann (channel independence, the ratio reset mechanism, local averages, non-linear correction), it has been shown that we can obtain a Retinex algorithm implementation that is intrinsically 2D, whose results comply with all the expected properties of the original, one-dimensional path-based Retinex algorithm (such as approximating color constancy while being unable to deal with overexposed images) but don't suffer from common shortcomings such as sensitivity to noise, appearance of halos, etc. An iterative application of this 2D Retinex algorithm takes the form of a partial differential equation (PDE) that it's proven not to minimize any energy functional, and this fact is linked to its limitations regarding over-exposed pictures. It was proposed to modify in this regard the iterative method in a certain way so that it is able to handle both under and over-exposed images, the resulting PDE now has a number of very relevant properties which allow to connect Retinex with variational models, histogram equalization and efficient coding, perceptual color correction algorithms, and computational neuroscience models of cortical activity and retinal models., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Perceptual color characterization of cameras

  • Vazquez-Corral, Javier
  • Connah, David
  • Bertalmío, Marcelo
Color camera characterization, mapping outputs from the camera sensors to an/nindependent color space, such as XY Z, is an important step in the camera processing/npipeline. Until now, this procedure has been primarily solved by using a 3 × 3 matrix/nobtained via a least-squares optimization. In this paper, we propose to use the spherical/nsampling method, recently published by Finlayson et al., to perform a perceptual color/ncharacterization. In particular, we search for the 3 × 3 matrix that minimizes three different/nperceptual errors, one pixel based and two spatially based. For the pixel-based case,/nwe minimize the CIE ∆E error, while for the spatial-based case, we minimize both the/nS-CIELAB error and the CID error measure. Our results demonstrate an improvement of/napproximately 3% for the ∆E error, 7% for the S-CIELAB error and 13% for the CID/nerror measures, This work was supported by European Research Council, Starting Grant Ref. 306337, and by Spanish/ngrants AACC, Ref. TIN2011-15954-E, and Plan Nacional, Ref. TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A tone mapping operator based on neural and psychophysical models of visual perception

  • Cyriac, Praveen
  • Bertalmío, Marcelo
  • Kane, David
  • Vazquez-Corral, Javier
High dynamic range imaging techniques involve capturing and storing real world radiance values that span many orders of magnitude. However, common display devices can usually reproduce intensity ranges only up to two to three orders of magnitude. Therefore, in order to display a high dynamic range image on a low dynamic range screen, the dynamic range of the image needs to be compressed without losing details or introducing artefacts, and this process is called tone mapping. A good tone mapping operator must be able to produce a low dynamic range image that matches as much as possible the perception of the real world scene. We propose a two stage tone mapping approach, in which the first stage is a global method for range compression based on a gamma curve that equalizes the lightness histogram the best, and the second stage performs local contrast enhancement and color induction using neural activity models for the visual cortex. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Denoising an image by denoising its components in a moving frame

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Bertalmío, Marcelo
  • Levine, Stacey
Comunicació presentada a: the 6th International Conference, (ICISP 2014), celebrada del 30. de juny al 2 de juliol de 2014 a Cherbourg, França., In this paper, we provide a new non-local method for image denoising. The key idea we develop is to denoise the components of the image in a well-chosen moving frame instead of the image itself.We prove the relevance of our approach by showing that the PSNR of a grayscale noisy image is lower than the PSNR of its components. Experiments show that applying the Non Local Means algorithm of Buades et al. on the components provides better results than applying it directly on the image., This work was supported by European Research Council, Starting Grant ref. 306337, and by Spanish grants AACC, ref. TIN2011-15954-E, and Plan Na- cional, ref. TIN2012-38112. S. Levine acknowledges partial support by NSF-DMS #0915219.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Local denoising applied to RAW images may outperform non-local patch-based methods applied to the camera output

  • Ghimpeteanu, Gabriela
  • Batard, Thomas
  • Seybold, Tamara
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., State-of-the-art denoising methods achieve impressive results, even for large noise levels. However, they can not be implemented in camera hardware, mainly due to the fact that they are computationally too intensive. The aim of this paper is then to show that we can obtain comparable denoising results to the ones obtained with state-of-art methods by inserting a well-chosen fast denoising method at the right location in the camera processing pipeline. We evaluate our results visually and with respect to objective measures., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Optimized tone curve for in-camera image processing

  • Cyriac, Praveen
  • Kane, David
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., The sensor values captured by a digital camera are transformed in a non-linear manner prior to quantization in order to make the quantization rate approximately proportional to the sensitivity of the human visual system. We propose an image dependent non-linear transformation that can accurately reproduce the detail and contrast visible in the original scene. The principles underpinning the transform stem from an understanding of natural image statistics, as well as recent experimental and neurophysiological findings. To optimize the parameters of the model we collect user-feedback and develop a method that can predict the user defined parameters. The method we have developed has an extremely low computational complexity, therefore it operates almost instantaneously making it suitable for in-camera operations. The final image looks natural, without any halos, spurious colors or artifacts. It can also be applied to video sequences, after imposing temporal coherence on the parameter values by smoothing them over time. The proposed approach is validated through psychophysical tests that confirm that it outperforms other state of the art algorithms in terms of users' preference., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Color stabilization along time and across shots of the same scene, for one or several cameras of unknown specifications

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
We propose a method for color stabilization of shots of the same scene, taken under the same illumination, where one image is chosen as reference and one or several other images are modified so that their colors match those of the reference. We make use of two crucial but often overlooked observations: first, that the core of the color correction chain in a digital camera is simply a multiplication by a 3 × 3 matrix; second, that to color-match a source image to a reference image we do not need to compute their two color correction matrices, it is enough to compute the operation that transforms one matrix into the other. This operation is a 3 × 3 matrix as well, which we call H. Once we have H, we just multiply by it each pixel value of the source and obtain an image which matches in color the reference. To compute H we only require a set of pixel correspondences, we do not need any information about the cameras used, neither models nor specifications or parameter values. We propose an implementation of our framework which is very simple and fast, and show how it can be successfully employed in a number of situations, comparing favorably with the state of the art. There is a wide range of applications of our technique, both for amateur and professional photography and video: color matching for multicamera TV broadcasts, color matching for 3D cinema, color stabilization for amateur video, etc., This work was supported in part by the European Research Council under Grant 306337 and in part by the Spanish Research under Grant TIN2011-15954-E and Grant TIN2012-38112.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Capítulo o parte de un libro (BookPart)

Light and color

  • Bertalmío, Marcelo
Image Processing for Cinema presents a detailed overview of image processing techniques that are used in practice in digital cinema. The book shows how image processing has become ubiquitous in movie-making, from shooting to exhibition. It covers all the ways in which image processing algorithms are used to enhance, restore, adapt, and convert moving images. These techniques and algorithms make the images look as good as possible while exploiting the capabilities of cameras, projectors, and displays. The author focuses on the ideas behind the methods, rather than proofs and derivations. The first part of the text presents fundamentals on optics and color. The second part explains how cameras work and details all the image processing algorithms that are applied in-camera. With an emphasis on state-of-the-art methods that are actually used in practice, the last part describes image processing algorithms that are applied offline to solve a variety of problems. The book is designed for advanced undergraduate and graduate students in applied mathematics, image processing, computer science, and related fields. It is also suitable for academic researchers and professionals in the movie industry., The author would like to acknowledge the support of the European Research Council through the Starting Grant ref. 306337, of ICREA through their ICREA Acadèmia Award, and of the Spanish government through the grants TIN2012-38112 and TIN2011-1594-E.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Image processing applications through a variational perceptually-based color correction related to Retinex

  • Vazquez-Corral, Javier
  • Zamir, Syed Waqas
  • Galdran, Adrian
  • Pardo, David
  • Bertalmío, Marcelo
Comunicació presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology., The retinex theory of color vision has been a light of inspiration for color researchers since its inception 50 years ago. It has been adapted to work for different goals such as shadow removal, high dynamic range imaging, or computational color constancy. Back in 2007 a variational perceptually-based color correction model related to Retinex was presented by Bertalmío and colleagues. In this paper we first comment on this model and later we review different image processing applications that have been obtained by performing small modifications to it, namely color gamut mapping and image dehazing., J. Vazquez-Corral. S.W. Zamir, and M. Bertalmío were supported by European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-38112, and by the Icrea Academia Award. D. Pardo was partially funded by the Project of the Spanish Ministry of Economy and Competitiveness with reference MTM2013-40824-P, the BCAM ”Severo Ochoa” accreditation of excellence SEV-2013-0323, the CYTED 2011 project 712RT0449, and the Basque Government Consolidated Research Group Grant IT649-13 on “Mathematical Modeling, Simulation, and Industrial Applications (M2SI)”.