Skip to main content

MODELOS DE VISION DE CONTRASTE Y COLOR PARA CINE

TIN2015-71537-P

•
Funding agency name Ministerio de Economía y Competitividad
Funding agency acronym MINECO
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2015
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Management unit Secretaría de Estado de Educación, Formación Profesional y Universidades
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 20
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

In-camera, photorealistic style transfer for on-set automatic grading

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a la SMPTE 2018 Annual Technical Conference of the Society of Motion Picture and Television Engineers, celebrada a Los Angeles (Estats Units d'Amèrica) del 22 a 25 de octubre de 2018., In professional cinema, the intended artistic look of the movie informs the creation of a static 3D LUT that is applied on set, where further manual modifications to the image appearance are registered as 10-parameter transforms in a color decision list (CDL). The original RAW footage and its corresponding LUT and CDL are passedon to the post-production stage where the fine-tuning of the final look is performed during color grading. — In many cases, the director wants to emulate the style and look present in a reference image, e.g. a still from an existing movie, or a photograph, or a painting, or even a frame from a previously shot sequence in the current movie. The manual creation of a LUT and CDL for this purpose may require a significant amount of work from very skilled artists and technicians, while the state of the art in the academic literature offers promising but partial solutions to the photorealistic style transfer problem, with limitations regarding artifacts, speed and manual interaction. — In this paper, we propose a method that automatically transfers the style, in terms of luminance, color palette and contrast, from a reference image to the source raw footage. It consists of three separable operations: global luminance matching, global color transfer and local contrast matching. As it just takes into account the statistics of source and reference images, no training is required. The total transform is not static but adapts to the changes in the source footage. The computational complexity of the procedure is extremely low and allows for real-time implementation in-camera, for on-set monitoring. While the method is proposed as a substitute for the need to specify a LUT and a CDL, it's compatible with further refinements performed via LUTs, CDLs and grading, both on-set and in post-production. The results are free from artifacts and provide an excellent approximation to the intended look, bringing savings in pre-production, shooting and post-production time., This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Perceptually-based gamut extension algorithm for emerging wide color gamut display and projection technologies

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2016 Annual Technical Conference and Exhibition, celebrada a Los Angeles, Estats Units d'Amèrica, del 25 al 27 d'octubre de 2016., While wide color gamut (WCG) capabilities are a key element of emerging display and projection technologies, at present most image content is recorded using standards such as DCI-P3 for cinema or BT.709 for TV that have a reduced color gamut. Therefore, there is a need for gamut extension methods that process regular content and allow to appreciate the full color potential of new displays, improving user experience. We present a gamut extension algorithm that is based on visual perception models and explicitly takes into account low chromatic colors such as skin tones. It produces results that look natural, are free of artifacts of any kind, and outperform the state of the art. The method is fast, allowing for operation interaction if needed., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Performance evaluation of objective quality metrics on HLG-based HDR image coding

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), celebrada a Anaheim, Estats Units d'Amèrica, del 26 al 29 de novembre de 2018., We evaluate the performance of objective quality metrics for high dynamic range (HDR) image coding that uses the transfer function (TF) of the Hybrid Log-Gamma (HLG) method. Previous evaluations of objective metrics for HDR image coding have studied which of them are reliable predictors of perceived quality; however, in those tests, all the non-linear transforms used both for encoding and by the best-performing metrics are essentially very similar and based on visual perception data of detection thresholds for lightness variations. The HLG non-linearity on the other hand is very different, as it is designed for backward compatibility with standard dynamic range (SDR) displays. We test a variety of options for objective metrics, including HLG-based. Our results show that the ranking of metrics for HDR coding changes drastically depending on the TF used for compression., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER, UE)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Photorealistic style transfer for cinema shoots

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 Colour and Visual Computing Symposium (CVCS), celebrat del 19 a 20 de setembre de 2018 a Gjøvik, Noruega, Color grading is the process of subtly mixing andadjusting the color and tonal balance of a movie to achievea specific visuallook. This manual editing task may requirea significant amount of work from very skilled artists andtechnicians. In many cases the director wants to emulate thestyle and look present in a reference image, e.g. a still froman existing movie, a photograph, or even a previously shotsequence in the current movie. In this paper we propose amethod that automatically transfers the style, in terms of tone,color palette and contrast, from a reference image to the sourceRAW image. It consists of three separable operations: globalluminance matching, global color transfer and local contrastmatching. The computational complexity of the procedureis extremely low and allows for real-time implementationin-camera. As it just takes into account the statistics of sourceand reference images, no training is required. The results arefree from artifacts and provide an excellent approximation tothe intended look, bringing savings in pre-production, shootingand post-production time., This work has received funding from the European Union’s Horizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Color-matching shots from different cameras having unknown gamma or logarithmic encoding curves

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2017 Annual Technical Conference and Exhibition, celebrat del 23 al 26 d'octubre de 2017 a Los Angeles, Estats Units d'Amèrica., In cinema and TV it is quite usual to have to work with footage coming from several cameras, which show noticeable color differences among them even if they are all the same model. In TV broadcasts, technicians work in camera control units so as to ensure color consistency when cutting from one camera to another. In cinema post-production, colorists need to manually colormatch images coming from different sources. Aiming to help perform this task automatically, the Academy Color Encoding System (ACES) introduced a color management framework to work within the same color space and be able to use different cameras and displays; however, the ACES pipeline requires to have the cameras characterized previously, and therefore does not allow to work ‘in the wild’, a situation which is very common. We present a color stabilization method that, given two images of the same scene taken by two cameras with unknown settings and unknown internal parameter values, and encoded with unknown non-linear curves (logarithmic or gamma), is able to correct the colors of one of the images making it look as if it was captured with the other camera. Our method is based on treating the in-camera color processing pipeline as a combination of a 3x3 matrix followed by a non-linearity, which allows us to model a color stabilization transformation among two shots as a linear-nonlinear function with several parameters. We find corresponding points between the two images, compute the error (color difference) over them, and determine the transformation parameters that minimize this error, all automatically without any user input. The method is fast and the results have no spurious colors or spatio-temporal artifacts of any kind. It outperforms the state of the art both visually and according to several metrics, and can handle very challenging real-life examples., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic, fast and perceptually accurate gamut mapping based on vision science models

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2017 Annual Technical Conference and Exhibition, celebrat del 23 al 26 d'octubre de 2017 a Los Angeles, Estats Units d'Amèrica., Gamut mapping transforms colors of the original (image or video) content to the color palette of the display device with the simultaneous goals of (a) reproducing content accurately while preserving the artistic intent of the original content’s creator and (b) exploiting the full color rendering potential of the target display device. The rapid advancement in display technologies has created a pressing need to develop automatic and fast gamut mapping algorithms that can deal with imagery intended for both conventional and emerging displays. In this paper, we propose a novel framework based on retinal and color perception models from vision science that offers a functionality to perform both gamut reduction and gamut extension, while preserving hue and taking into account the analysis of the colors of the input image. We evaluate the performance of the proposed framework visually and by using a perceptually-based error metric, according to which the gamut-mapped results of our framework outperform those of the state-of-the-art methods., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award. The work of Javier Vazquez-Corral was supported by the Spanish government grant IJCI-2014-19516.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Practical use suggests a re-evaluation of HDR objective quality metrics

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: Eleventh International Conference on Quality of Multimedia Experience (QoMEX, 2019), celebrada a Berlín (Alemanya) del 5 al 7 de juny de 2019., Full-reference objective quality metrics for high dynamic range (HDR) images must be validated in terms of their consistency with subjective evaluation results, and previous studies have found a number of metrics that appear to correlate well with the preference of observers. However, those conclusions were based on experiments that do not correspond to the practical use of a current day HDR professional production scenario. In this work, we carry out subjective evaluation experiments for the popular HDR standards of perceptual quantization (PQ) and Hybrid Log-Gamma (HLG), with a state-of-the-art HDR reference monitor used in broadcasting and post-production, and where all the observers participating in the tests are video experts. We find that the ranking of HDR metrics is now substantially different from what was reported earlier, and also that a simple standard dynamic range (SDR) metric can be applied directly to PQ or HLG encoded signals providing excellent results that surpass those of HDR metrics., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER, UE)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Local denoising based on curvature smoothing can visually outperform non-local methods on photographs with actual noise

  • Ghimpeteanu, Gabriela
  • Kane, David
  • Batard, Thomas
  • Levine, Stacey
  • Bertalmío, Marcelo
Comunicació presentada a: IEEE International Conference on Image Processing 2016, celebrada del 25 l 28 de setembre de 2016 a Phoenix, Estats Units d'Amèrica., We propose a fast, local denoising method where the Euclidean curvature of the noisy image is approximated in a regularizing manner and a clean image is reconstructed from this smoothed curvature. User preference tests show that when denoising real photographs with actual noise our method produces results with the same visual quality as the more sophisticated, nonlocal algorithms Non-local Means and BM3D, but at a fraction of their computational cost. These tests also highlight the limitations of objective image quality metrics like PSNR and SSIM, which correlate poorly with user preference., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2015-71537-P, and by the Icrea Academia Award.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Convolutional neural networks deceived by visual illusions

  • Gómez Villa, Alexander
  • Martín, Adrian
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada al CVPR 2019: Conference on Computer Vision and Pattern Recognition, celebrat del 16 al 20 de juny de 2019 a California, Estats Units d'Amèrica., Visual illusions teach us that what we see is not always what is represented in the physical world. Their special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear and non-linear operations. The similarity of this structure with the operations present in Convolutional Neural Networks (CNNs) has motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs trained for image denoising, image deblurring, and computational color constancy are able to replicate the human response to visual illusions, and that the extent of this replication varies with respect to variation in architecture and spatial pattern size. These results suggest that in order to obtain CNNs that better replicate human behaviour, we may need to start aiming for them to better replicate visual illusions., This work has received funding from the EU Horizon 2020 programme under grant agreement 761544 (project HDR4EU) and under grant agreement 780470 (project SAUCE) and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Log-encoding estimation for color stabilization of cinematic footage

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: IEEE International Conference on Image Processing 2016, celebrada del 25 l 28 de setembre de 2016 a Phoenix, Estats Units d'Amèrica., We propose a method for the color stabilization of cinema shots coming from different cameras that use unknown logarithmic encoding curves. The log-encoding curves are approximated by a concatenation of gamma-curves, whose values are accurately computed using image matches. The color stabilization procedure, based on the generic color processing pipeline of a digital camera, can be performed after the estimation of the encoding curves, and it also requires the existence of image matches. Our work can be applied in different scenarios such as multi-camera shoots, native-3D cinema, or color grading in post-production., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2015-71537-P, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Performance evaluation of objective quality metrics on HLG-based HDR image coding

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), celebrada a Anaheim, Estats Units d'Amèrica, del 26 al 29 de novembre de 2018., We evaluate the performance of objective quality metrics for high dynamic range (HDR) image coding that uses the transfer function (TF) of the Hybrid Log-Gamma (HLG) method. Previous evaluations of objective metrics for HDR image coding have studied which of them are reliable predictors of perceived quality; however, in those tests, all the non-linear transforms used both for encoding and by the best-performing metrics are essentially very similar and based on visual perception data of detection thresholds for lightness variations. The HLG non-linearity on the other hand is very different, as it is designed for backward compatibility with standard dynamic range (SDR) displays. We test a variety of options for objective metrics, including HLG-based. Our results show that the ranking of metrics for HDR coding changes drastically depending on the TF used for compression., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER, UE)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Photorealistic style transfer for cinema shoots

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 Colour and Visual Computing Symposium (CVCS), celebrat del 19 a 20 de setembre de 2018 a Gjøvik, Noruega, Color grading is the process of subtly mixing andadjusting the color and tonal balance of a movie to achievea specific visuallook. This manual editing task may requirea significant amount of work from very skilled artists andtechnicians. In many cases the director wants to emulate thestyle and look present in a reference image, e.g. a still froman existing movie, a photograph, or even a previously shotsequence in the current movie. In this paper we propose amethod that automatically transfers the style, in terms of tone,color palette and contrast, from a reference image to the sourceRAW image. It consists of three separable operations: globalluminance matching, global color transfer and local contrastmatching. The computational complexity of the procedureis extremely low and allows for real-time implementationin-camera. As it just takes into account the statistics of sourceand reference images, no training is required. The results arefree from artifacts and provide an excellent approximation tothe intended look, bringing savings in pre-production, shootingand post-production time., This work has received funding from the European Union’s Horizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Color-matching shots from different cameras having unknown gamma or logarithmic encoding curves

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2017 Annual Technical Conference and Exhibition, celebrat del 23 al 26 d'octubre de 2017 a Los Angeles, Estats Units d'Amèrica., In cinema and TV it is quite usual to have to work with footage coming from several cameras, which show noticeable color differences among them even if they are all the same model. In TV broadcasts, technicians work in camera control units so as to ensure color consistency when cutting from one camera to another. In cinema post-production, colorists need to manually colormatch images coming from different sources. Aiming to help perform this task automatically, the Academy Color Encoding System (ACES) introduced a color management framework to work within the same color space and be able to use different cameras and displays; however, the ACES pipeline requires to have the cameras characterized previously, and therefore does not allow to work ‘in the wild’, a situation which is very common. We present a color stabilization method that, given two images of the same scene taken by two cameras with unknown settings and unknown internal parameter values, and encoded with unknown non-linear curves (logarithmic or gamma), is able to correct the colors of one of the images making it look as if it was captured with the other camera. Our method is based on treating the in-camera color processing pipeline as a combination of a 3x3 matrix followed by a non-linearity, which allows us to model a color stabilization transformation among two shots as a linear-nonlinear function with several parameters. We find corresponding points between the two images, compute the error (color difference) over them, and determine the transformation parameters that minimize this error, all automatically without any user input. The method is fast and the results have no spurious colors or spatio-temporal artifacts of any kind. It outperforms the state of the art both visually and according to several metrics, and can handle very challenging real-life examples., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), and by the Icrea Academia Award. The work of J. Vazquez-Corral was supported by the Spanish government under Grant IJCI-2014-19516.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Practical use suggests a re-evaluation of HDR objective quality metrics

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: Eleventh International Conference on Quality of Multimedia Experience (QoMEX, 2019), celebrada a Berlín (Alemanya) del 5 al 7 de juny de 2019., Full-reference objective quality metrics for high dynamic range (HDR) images must be validated in terms of their consistency with subjective evaluation results, and previous studies have found a number of metrics that appear to correlate well with the preference of observers. However, those conclusions were based on experiments that do not correspond to the practical use of a current day HDR professional production scenario. In this work, we carry out subjective evaluation experiments for the popular HDR standards of perceptual quantization (PQ) and Hybrid Log-Gamma (HLG), with a state-of-the-art HDR reference monitor used in broadcasting and post-production, and where all the observers participating in the tests are video experts. We find that the ranking of HDR metrics is now substantially different from what was reported earlier, and also that a simple standard dynamic range (SDR) metric can be applied directly to PQ or HLG encoded signals providing excellent results that surpass those of HDR metrics., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER, UE)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Automatic, fast and perceptually accurate gamut mapping based on vision science models

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2017 Annual Technical Conference and Exhibition, celebrat del 23 al 26 d'octubre de 2017 a Los Angeles, Estats Units d'Amèrica., Gamut mapping transforms colors of the original (image or video) content to the color palette of the display device with the simultaneous goals of (a) reproducing content accurately while preserving the artistic intent of the original content’s creator and (b) exploiting the full color rendering potential of the target display device. The rapid advancement in display technologies has created a pressing need to develop automatic and fast gamut mapping algorithms that can deal with imagery intended for both conventional and emerging displays. In this paper, we propose a novel framework based on retinal and color perception models from vision science that offers a functionality to perform both gamut reduction and gamut extension, while preserving hue and taking into account the analysis of the colors of the input image. We evaluate the performance of the proposed framework visually and by using a perceptually-based error metric, according to which the gamut-mapped results of our framework outperform those of the state-of-the-art methods., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award. The work of Javier Vazquez-Corral was supported by the Spanish government grant IJCI-2014-19516.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Perceptually-based gamut extension algorithm for emerging wide color gamut display and projection technologies

  • Zamir, Syed Waqas
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: SMPTE 2016 Annual Technical Conference and Exhibition, celebrada a Los Angeles, Estats Units d'Amèrica, del 25 al 27 d'octubre de 2016., While wide color gamut (WCG) capabilities are a key element of emerging display and projection technologies, at present most image content is recorded using standards such as DCI-P3 for cinema or BT.709 for TV that have a reduced color gamut. Therefore, there is a need for gamut extension methods that process regular content and allow to appreciate the full color potential of new displays, improving user experience. We present a gamut extension algorithm that is based on visual perception models and explicitly takes into account low chromatic colors such as skin tones. It produces results that look natural, are free of artifacts of any kind, and outperform the state of the art. The method is fast, allowing for operation interaction if needed., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE), and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Local denoising based on curvature smoothing can visually outperform non-local methods on photographs with actual noise

  • Ghimpeteanu, Gabriela
  • Kane, David
  • Batard, Thomas
  • Levine, Stacey
  • Bertalmío, Marcelo
Comunicació presentada a: IEEE International Conference on Image Processing 2016, celebrada del 25 l 28 de setembre de 2016 a Phoenix, Estats Units d'Amèrica., We propose a fast, local denoising method where the Euclidean curvature of the noisy image is approximated in a regularizing manner and a clean image is reconstructed from this smoothed curvature. User preference tests show that when denoising real photographs with actual noise our method produces results with the same visual quality as the more sophisticated, nonlocal algorithms Non-local Means and BM3D, but at a fraction of their computational cost. These tests also highlight the limitations of objective image quality metrics like PSNR and SSIM, which correlate poorly with user preference., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2015-71537-P, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Log-encoding estimation for color stabilization of cinematic footage

  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: IEEE International Conference on Image Processing 2016, celebrada del 25 l 28 de setembre de 2016 a Phoenix, Estats Units d'Amèrica., We propose a method for the color stabilization of cinema shots coming from different cameras that use unknown logarithmic encoding curves. The log-encoding curves are approximated by a concatenation of gamma-curves, whose values are accurately computed using image matches. The color stabilization procedure, based on the generic color processing pipeline of a digital camera, can be performed after the estimation of the encoding curves, and it also requires the existence of image matches. Our work can be applied in different scenarios such as multi-camera shoots, native-3D cinema, or color grading in post-production., This work was supported by the European Research Council, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2015-71537-P, and by the Icrea Academia Award.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Convolutional neural networks deceived by visual illusions

  • Gómez Villa, Alexander
  • Martín, Adrian
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada al CVPR 2019: Conference on Computer Vision and Pattern Recognition, celebrat del 16 al 20 de juny de 2019 a California, Estats Units d'Amèrica., Visual illusions teach us that what we see is not always what is represented in the physical world. Their special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear and non-linear operations. The similarity of this structure with the operations present in Convolutional Neural Networks (CNNs) has motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs trained for image denoising, image deblurring, and computational color constancy are able to replicate the human response to visual illusions, and that the extent of this replication varies with respect to variation in architecture and spatial pattern size. These results suggest that in order to obtain CNNs that better replicate human behaviour, we may need to start aiming for them to better replicate visual illusions., This work has received funding from the EU Horizon 2020 programme under grant agreement 761544 (project HDR4EU) and under grant agreement 780470 (project SAUCE) and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

In-camera, photorealistic style transfer for on-set automatic grading

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a la SMPTE 2018 Annual Technical Conference of the Society of Motion Picture and Television Engineers, celebrada a Los Angeles (Estats Units d'Amèrica) del 22 a 25 de octubre de 2018., In professional cinema, the intended artistic look of the movie informs the creation of a static 3D LUT that is applied on set, where further manual modifications to the image appearance are registered as 10-parameter transforms in a color decision list (CDL). The original RAW footage and its corresponding LUT and CDL are passedon to the post-production stage where the fine-tuning of the final look is performed during color grading. — In many cases, the director wants to emulate the style and look present in a reference image, e.g. a still from an existing movie, or a photograph, or a painting, or even a frame from a previously shot sequence in the current movie. The manual creation of a LUT and CDL for this purpose may require a significant amount of work from very skilled artists and technicians, while the state of the art in the academic literature offers promising but partial solutions to the photorealistic style transfer problem, with limitations regarding artifacts, speed and manual interaction. — In this paper, we propose a method that automatically transfers the style, in terms of luminance, color palette and contrast, from a reference image to the source raw footage. It consists of three separable operations: global luminance matching, global color transfer and local contrast matching. As it just takes into account the statistics of source and reference images, no training is required. The total transform is not static but adapts to the changes in the source footage. The computational complexity of the procedure is extremely low and allows for real-time implementation in-camera, for on-set monitoring. While the method is proposed as a substitute for the need to specify a LUT and a CDL, it's compatible with further refinements performed via LUTs, CDLs and grading, both on-set and in post-production. The results are free from artifacts and provide an excellent approximation to the intended look, bringing savings in pre-production, shooting and post-production time., This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).