DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación
Artículo científico (JournalArticle)
Using Decoupled Features for Photorealistic Style Transfer
45 pags., 23 figs., 8 algorithms, 1 tab. -- AMS subject classifications. 68Q25, 68R10, 68U05, In this work we propose a photo-realistic style transfer method for image and video that is based on
vision science principles and on a recent mathematical formulation for the deterministic decoupling of
sample statistics. The novel aspects of our approach include matching decoupled moments of higher
order than in common style transfer approaches, and matching a descriptor of the power spectrum
so as to characterize and transfer diffusion effects between source and target, which is something
that has not been considered before in the literature. The results are of high visual quality, without
spatio-temporal artifacts, and validation tests in the form of observer preference experiments show
that our method compares very well with the state-of-the-art. The computational complexity of the
algorithm is low, and we propose a numerical implementation that is amenable for real-time video
application. Finally, another contribution of our work is to point out that current deep learning
approaches for photo-realistic style transfer don’t really achieve photo-realistic quality outside of
limited examples, because the results too often show unacceptable visual artifacts., This work has received funding from the Spanish Government grants PID2020-118071GB-I00,
PID2020-113596GB-I00 and PID2021-127373NB-I00, and from the European Union’s Horizon 2020 research and
innovation program under grant agreement number 952027 (project AdMiRe)., Peer reviewed