Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 18
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

The Potential of light fields in media productions

  • Trottnow, Jonas
  • Spielmann, Simon
  • Lange, Tobias
  • Chelli, Kelvin
  • Solony, Marek
  • Smrž, Pavel
  • Zemčík, Pavel
  • Aenchbacher, Weston
  • Grogan, Mairéad
  • Alain, Martin
  • Smolic, Aljosa
  • Canham, Trevor
  • Vu-Thanh, Olivier
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: The 12th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, celebrada del 17 al 20 de novembre de 2019 a Brisbane, Austràlia., One aspect of the EU funded project SAUCE is to explore the pos-sibilities and challenges of integrating light field capturing andprocessing into media productions. A special light field camera wasbuild by Saarland University [Herfet et al.2018] and is first testedunder production conditions in the test production “Unfolding” aspart of the SAUCE project. Filmakademie Baden-Württemberg de-veloped the contentual frame, executed the post-production and pre-pared a complete previsualization. Calibration and post-processingalgorithms are developed by the Trinity College Dublin and theBrno University of Technology. This document describes challengesduring building and shooting with the light field camera array, aswell as its potential and challenges for the post-production., The project has received funding from the European Union’s Hori-zon 2020 Research and Innovation Programme under Grant Agree-ment No. 780470.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

In-camera, photorealistic style transfer for on-set automatic grading

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a la SMPTE 2018 Annual Technical Conference of the Society of Motion Picture and Television Engineers, celebrada a Los Angeles (Estats Units d'Amèrica) del 22 a 25 de octubre de 2018., In professional cinema, the intended artistic look of the movie informs the creation of a static 3D LUT that is applied on set, where further manual modifications to the image appearance are registered as 10-parameter transforms in a color decision list (CDL). The original RAW footage and its corresponding LUT and CDL are passedon to the post-production stage where the fine-tuning of the final look is performed during color grading. — In many cases, the director wants to emulate the style and look present in a reference image, e.g. a still from an existing movie, or a photograph, or a painting, or even a frame from a previously shot sequence in the current movie. The manual creation of a LUT and CDL for this purpose may require a significant amount of work from very skilled artists and technicians, while the state of the art in the academic literature offers promising but partial solutions to the photorealistic style transfer problem, with limitations regarding artifacts, speed and manual interaction. — In this paper, we propose a method that automatically transfers the style, in terms of luminance, color palette and contrast, from a reference image to the source raw footage. It consists of three separable operations: global luminance matching, global color transfer and local contrast matching. As it just takes into account the statistics of source and reference images, no training is required. The total transform is not static but adapts to the changes in the source footage. The computational complexity of the procedure is extremely low and allows for real-time implementation in-camera, for on-set monitoring. While the method is proposed as a substitute for the need to specify a LUT and a CDL, it's compatible with further refinements performed via LUTs, CDLs and grading, both on-set and in post-production. The results are free from artifacts and provide an excellent approximation to the intended look, bringing savings in pre-production, shooting and post-production time., This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Performance evaluation of objective quality metrics on HLG-based HDR image coding

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), celebrada a Anaheim, Estats Units d'Amèrica, del 26 al 29 de novembre de 2018., We evaluate the performance of objective quality metrics for high dynamic range (HDR) image coding that uses the transfer function (TF) of the Hybrid Log-Gamma (HLG) method. Previous evaluations of objective metrics for HDR image coding have studied which of them are reliable predictors of perceived quality; however, in those tests, all the non-linear transforms used both for encoding and by the best-performing metrics are essentially very similar and based on visual perception data of detection thresholds for lightness variations. The HLG non-linearity on the other hand is very different, as it is designed for backward compatibility with standard dynamic range (SDR) displays. We test a variety of options for objective metrics, including HLG-based. Our results show that the ranking of metrics for HDR coding changes drastically depending on the TF used for compression., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER, UE)
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Photorealistic style transfer for cinema shoots

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 Colour and Visual Computing Symposium (CVCS), celebrat del 19 a 20 de setembre de 2018 a Gjøvik, Noruega, Color grading is the process of subtly mixing andadjusting the color and tonal balance of a movie to achievea specific visuallook. This manual editing task may requirea significant amount of work from very skilled artists andtechnicians. In many cases the director wants to emulate thestyle and look present in a reference image, e.g. a still froman existing movie, a photograph, or even a previously shotsequence in the current movie. In this paper we propose amethod that automatically transfers the style, in terms of tone,color palette and contrast, from a reference image to the sourceRAW image. It consists of three separable operations: globalluminance matching, global color transfer and local contrastmatching. The computational complexity of the procedureis extremely low and allows for real-time implementationin-camera. As it just takes into account the statistics of sourceand reference images, no training is required. The results arefree from artifacts and provide an excellent approximation tothe intended look, bringing savings in pre-production, shootingand post-production time., This work has received funding from the European Union’s Horizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

The persistent influence of viewing environment illumination color on displayed image appearance

  • Canham, Trevor
  • Bertalmío, Marcelo
Comunicació presentada a la IS&T London Imaging Meeting, celebrada online del 29 de setembre a l'1 d'octubre de 2020., Chromatic adaptation considering competing influences from emissive displays and ambient illumination is a little studied topic in the context of color management in proportion to its influence on displayed image appearance. An experiment was conducted to identify the degree to which observers adapt to the white point of natural images on an emissive display versus the color of ambient illumination in the room. The responses of observers had no significant difference from those of a previous experiment which was conducted with roughly the same procedure and conditions on a mobile display with a significantly smaller viewing angle. A model is proposed to predict the degree of adaptation values reported by observers. This model has a form such that it can be re-optimized to fit additional data sets for different viewing scenarios and can be used in conjunction with a number of chromatic adaptation transforms., The authors would like to thank all observers for their participation, in particular the colorists from Deluxe Spain who generously lent their time and spectrophotometer to the experiment. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Practical use suggests a re-evaluation of HDR objective quality metrics

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: Eleventh International Conference on Quality of Multimedia Experience (QoMEX, 2019), celebrada a Berlín (Alemanya) del 5 al 7 de juny de 2019., Full-reference objective quality metrics for high dynamic range (HDR) images must be validated in terms of their consistency with subjective evaluation results, and previous studies have found a number of metrics that appear to correlate well with the preference of observers. However, those conclusions were based on experiments that do not correspond to the practical use of a current day HDR professional production scenario. In this work, we carry out subjective evaluation experiments for the popular HDR standards of perceptual quantization (PQ) and Hybrid Log-Gamma (HLG), with a state-of-the-art HDR reference monitor used in broadcasting and post-production, and where all the observers participating in the tests are video experts. We find that the ranking of HDR metrics is now substantially different from what was reported earlier, and also that a simple standard dynamic range (SDR) metric can be applied directly to PQ or HLG encoded signals providing excellent results that surpass those of HDR metrics., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER, UE)
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Color stabilization for multi-camera light-field imaging

  • Vu-Thanh, Olivier
  • Canham, Trevor
  • Vazquez-Corral, Javier
  • Gil Rodríguez, Raquel
  • Bertalmío, Marcelo
By capturing a more complete rendition of scene light than standard 2D cameras, light-field technology represents an important step towards closing the gap between live action cinematography and computer graphics. Light-field cameras accomplish this by simultaneously capturing the same scene under different angular configurations, providing directional information that allows for a multitude of post-production effects. Among the practical challenges related to capturing multiple images simultaneously, a very important problem is the fact that the different images do not perfectly match in terms of color, which severely complicates all further processing. In this work we adapt and extend to the light-field scenario a color stabilization method previously proposed for standard multi-camera shoots, and demonstrate experimentally that it provides an improvement over the state-of-the-art techniques for light-field imaging., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Convolutional neural networks deceived by visual illusions

  • Gómez Villa, Alexander
  • Martín, Adrian
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada al CVPR 2019: Conference on Computer Vision and Pattern Recognition, celebrat del 16 al 20 de juny de 2019 a California, Estats Units d'Amèrica., Visual illusions teach us that what we see is not always what is represented in the physical world. Their special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear and non-linear operations. The similarity of this structure with the operations present in Convolutional Neural Networks (CNNs) has motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs trained for image denoising, image deblurring, and computational color constancy are able to replicate the human response to visual illusions, and that the extent of this replication varies with respect to variation in architecture and spatial pattern size. These results suggest that in order to obtain CNNs that better replicate human behaviour, we may need to start aiming for them to better replicate visual illusions., This work has received funding from the EU Horizon 2020 programme under grant agreement 761544 (project HDR4EU) and under grant agreement 780470 (project SAUCE) and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Non-experts or experts? Statistical analyses of MOS using DSIS method

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), celebrada a Barcelona (Espanya) del 4 a 8 de maig de 2020., In image quality assessments, the results of subjective evaluation experiments that use the double-stimulus impairment scale (DSIS) method are often expressed in terms of the mean opinion score (MOS), which is the average score of all subjects for each test condition. Some MOS values are used to derive image quality criteria, and it has been assumed that it is preferable to perform tests with non-expert subjects rather than with experts. In this study, we analyze the results of several subjective evaluation experiments using the DSIS method. Our first contribution is to discuss the statistical meaning of the MOS values, which has not been previously addressed in the literature. Second, our results show that, contrary to the established belief, there are advantages when performing subjective tests with experts, in that they allow experiments to be performed with fewer subjects, and to better determine the lower threshold of image quality., This work received partial funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE) and from the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Performance evaluation of objective quality metrics on HLG-based HDR image coding

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), celebrada a Anaheim, Estats Units d'Amèrica, del 26 al 29 de novembre de 2018., We evaluate the performance of objective quality metrics for high dynamic range (HDR) image coding that uses the transfer function (TF) of the Hybrid Log-Gamma (HLG) method. Previous evaluations of objective metrics for HDR image coding have studied which of them are reliable predictors of perceived quality; however, in those tests, all the non-linear transforms used both for encoding and by the best-performing metrics are essentially very similar and based on visual perception data of detection thresholds for lightness variations. The HLG non-linearity on the other hand is very different, as it is designed for backward compatibility with standard dynamic range (SDR) displays. We test a variety of options for objective metrics, including HLG-based. Our results show that the ranking of metrics for HDR coding changes drastically depending on the TF used for compression., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER, UE)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Color stabilization for multi-camera light-field imaging

  • Vu-Thanh, Olivier
  • Canham, Trevor
  • Vazquez-Corral, Javier
  • Gil Rodríguez, Raquel
  • Bertalmío, Marcelo
By capturing a more complete rendition of scene light than standard 2D cameras, light-field technology represents an important step towards closing the gap between live action cinematography and computer graphics. Light-field cameras accomplish this by simultaneously capturing the same scene under different angular configurations, providing directional information that allows for a multitude of post-production effects. Among the practical challenges related to capturing multiple images simultaneously, a very important problem is the fact that the different images do not perfectly match in terms of color, which severely complicates all further processing. In this work we adapt and extend to the light-field scenario a color stabilization method previously proposed for standard multi-camera shoots, and demonstrate experimentally that it provides an improvement over the state-of-the-art techniques for light-field imaging., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The Potential of light fields in media productions

  • Trottnow, Jonas
  • Spielmann, Simon
  • Lange, Tobias
  • Chelli, Kelvin
  • Solony, Marek
  • Smrž, Pavel
  • Zemčík, Pavel
  • Aenchbacher, Weston
  • Grogan, Mairéad
  • Alain, Martin
  • Smolic, Aljosa
  • Canham, Trevor
  • Vu-Thanh, Olivier
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada a: The 12th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, celebrada del 17 al 20 de novembre de 2019 a Brisbane, Austràlia., One aspect of the EU funded project SAUCE is to explore the pos-sibilities and challenges of integrating light field capturing andprocessing into media productions. A special light field camera wasbuild by Saarland University [Herfet et al.2018] and is first testedunder production conditions in the test production “Unfolding” aspart of the SAUCE project. Filmakademie Baden-Württemberg de-veloped the contentual frame, executed the post-production and pre-pared a complete previsualization. Calibration and post-processingalgorithms are developed by the Trinity College Dublin and theBrno University of Technology. This document describes challengesduring building and shooting with the light field camera array, aswell as its potential and challenges for the post-production., The project has received funding from the European Union’s Hori-zon 2020 Research and Innovation Programme under Grant Agree-ment No. 780470.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Photorealistic style transfer for cinema shoots

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a: 2018 Colour and Visual Computing Symposium (CVCS), celebrat del 19 a 20 de setembre de 2018 a Gjøvik, Noruega, Color grading is the process of subtly mixing andadjusting the color and tonal balance of a movie to achievea specific visuallook. This manual editing task may requirea significant amount of work from very skilled artists andtechnicians. In many cases the director wants to emulate thestyle and look present in a reference image, e.g. a still froman existing movie, a photograph, or even a previously shotsequence in the current movie. In this paper we propose amethod that automatically transfers the style, in terms of tone,color palette and contrast, from a reference image to the sourceRAW image. It consists of three separable operations: globalluminance matching, global color transfer and local contrastmatching. The computational complexity of the procedureis extremely low and allows for real-time implementationin-camera. As it just takes into account the statistics of sourceand reference images, no training is required. The results arefree from artifacts and provide an excellent approximation tothe intended look, bringing savings in pre-production, shootingand post-production time., This work has received funding from the European Union’s Horizon 2020 research and innovation programme undergrant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Practical use suggests a re-evaluation of HDR objective quality metrics

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: Eleventh International Conference on Quality of Multimedia Experience (QoMEX, 2019), celebrada a Berlín (Alemanya) del 5 al 7 de juny de 2019., Full-reference objective quality metrics for high dynamic range (HDR) images must be validated in terms of their consistency with subjective evaluation results, and previous studies have found a number of metrics that appear to correlate well with the preference of observers. However, those conclusions were based on experiments that do not correspond to the practical use of a current day HDR professional production scenario. In this work, we carry out subjective evaluation experiments for the popular HDR standards of perceptual quantization (PQ) and Hybrid Log-Gamma (HLG), with a state-of-the-art HDR reference monitor used in broadcasting and post-production, and where all the observers participating in the tests are video experts. We find that the ranking of HDR metrics is now substantially different from what was reported earlier, and also that a simple standard dynamic range (SDR) metric can be applied directly to PQ or HLG encoded signals providing excellent results that surpass those of HDR metrics., This work has partially received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER, UE)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The persistent influence of viewing environment illumination color on displayed image appearance

  • Canham, Trevor
  • Bertalmío, Marcelo
Comunicació presentada a la IS&T London Imaging Meeting, celebrada online del 29 de setembre a l'1 d'octubre de 2020., Chromatic adaptation considering competing influences from emissive displays and ambient illumination is a little studied topic in the context of color management in proportion to its influence on displayed image appearance. An experiment was conducted to identify the degree to which observers adapt to the white point of natural images on an emissive display versus the color of ambient illumination in the room. The responses of observers had no significant difference from those of a previous experiment which was conducted with roughly the same procedure and conditions on a mobile display with a significantly smaller viewing angle. A model is proposed to predict the degree of adaptation values reported by observers. This model has a form such that it can be re-optimized to fit additional data sets for different viewing scenarios and can be used in conjunction with a number of chromatic adaptation transforms., The authors would like to thank all observers for their participation, in particular the colorists from Deluxe Spain who generously lent their time and spectrophotometer to the experiment. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Non-experts or experts? Statistical analyses of MOS using DSIS method

  • Sugito, Yasuko
  • Bertalmío, Marcelo
Comunicació presentada a: 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), celebrada a Barcelona (Espanya) del 4 a 8 de maig de 2020., In image quality assessments, the results of subjective evaluation experiments that use the double-stimulus impairment scale (DSIS) method are often expressed in terms of the mean opinion score (MOS), which is the average score of all subjects for each test condition. Some MOS values are used to derive image quality criteria, and it has been assumed that it is preferable to perform tests with non-expert subjects rather than with experts. In this study, we analyze the results of several subjective evaluation experiments using the DSIS method. Our first contribution is to discuss the statistical meaning of the MOS values, which has not been previously addressed in the literature. Second, our results show that, contrary to the established belief, there are advantages when performing subjective tests with experts, in that they allow experiments to be performed with fewer subjects, and to better determine the lower threshold of image quality., This work received partial funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE) and from the Spanish government and FEDER Fund, grant ref. PGC2018-099651-B-I00 (MCIU/AEI/FEDER, UE).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Convolutional neural networks deceived by visual illusions

  • Gómez Villa, Alexander
  • Martín, Adrian
  • Vazquez-Corral, Javier
  • Bertalmío, Marcelo
Comunicació presentada al CVPR 2019: Conference on Computer Vision and Pattern Recognition, celebrat del 16 al 20 de juny de 2019 a California, Estats Units d'Amèrica., Visual illusions teach us that what we see is not always what is represented in the physical world. Their special nature make them a fascinating tool to test and validate any new vision model proposed. In general, current vision models are based on the concatenation of linear and non-linear operations. The similarity of this structure with the operations present in Convolutional Neural Networks (CNNs) has motivated us to study if CNNs trained for low-level visual tasks are deceived by visual illusions. In particular, we show that CNNs trained for image denoising, image deblurring, and computational color constancy are able to replicate the human response to visual illusions, and that the extent of this replication varies with respect to variation in architecture and spatial pattern size. These results suggest that in order to obtain CNNs that better replicate human behaviour, we may need to start aiming for them to better replicate visual illusions., This work has received funding from the EU Horizon 2020 programme under grant agreement 761544 (project HDR4EU) and under grant agreement 780470 (project SAUCE) and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

In-camera, photorealistic style transfer for on-set automatic grading

  • Zabaleta, Itziar
  • Bertalmío, Marcelo
Comunicació presentada a la SMPTE 2018 Annual Technical Conference of the Society of Motion Picture and Television Engineers, celebrada a Los Angeles (Estats Units d'Amèrica) del 22 a 25 de octubre de 2018., In professional cinema, the intended artistic look of the movie informs the creation of a static 3D LUT that is applied on set, where further manual modifications to the image appearance are registered as 10-parameter transforms in a color decision list (CDL). The original RAW footage and its corresponding LUT and CDL are passedon to the post-production stage where the fine-tuning of the final look is performed during color grading. — In many cases, the director wants to emulate the style and look present in a reference image, e.g. a still from an existing movie, or a photograph, or a painting, or even a frame from a previously shot sequence in the current movie. The manual creation of a LUT and CDL for this purpose may require a significant amount of work from very skilled artists and technicians, while the state of the art in the academic literature offers promising but partial solutions to the photorealistic style transfer problem, with limitations regarding artifacts, speed and manual interaction. — In this paper, we propose a method that automatically transfers the style, in terms of luminance, color palette and contrast, from a reference image to the source raw footage. It consists of three separable operations: global luminance matching, global color transfer and local contrast matching. As it just takes into account the statistics of source and reference images, no training is required. The total transform is not static but adapts to the changes in the source footage. The computational complexity of the procedure is extremely low and allows for real-time implementation in-camera, for on-set monitoring. While the method is proposed as a substitute for the need to specify a LUT and a CDL, it's compatible with further refinements performed via LUTs, CDLs and grading, both on-set and in post-production. The results are free from artifacts and provide an excellent approximation to the intended look, bringing savings in pre-production, shooting and post-production time., This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement number 761544 (project HDR4EU) and under grant agreement number 780470 (project SAUCE), and by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P (MINECO/FEDER,UE).