Pasar al contenido principal

METODOS EXPLICABLES E INTERACTIVOS PARA LA MEJORA DE LA INFORMACION VISUAL

PID2021-128178OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario CENTRO DE VISION POR COMPUTADOR

Resultados relacionados

Resultados totales (Incluyendo duplicados): 14
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Image Quality Evaluation in Professional HDR/WCG Production Questions the Need for HDR Metrics

  • Sugito, Yasuko|||0000-0002-7549-1073
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Canham, Trevor|||0000-0003-4830-4786
  • Bertalmío, Marcelo|||0000-0002-1023-8325
In the quality evaluation of high dynamic range and wide color gamut (HDR/WCG) images, a number of works have concluded that native HDR metrics, such as HDR visual difference predictor (HDR-VDP), HDR video quality metric (HDR-VQM), or convolutional neural network (CNN)-based visibility metrics for HDR content, provide the best results. These metrics consider only the luminance component, but several color difference metrics have been specifically developed for, and validated with, HDR/WCG images. In this paper, we perform subjective evaluation experiments in a professional HDR/WCG production setting, under a real use case scenario. The results are quite relevant in that they show, firstly, that the performance of HDR metrics is worse than that of a classic, simple standard dynamic range (SDR) metric applied directly to the HDR content; and secondly, that the chrominance metrics specifically developed for HDR/WCG imaging have poor correlation with observer scores and are also outperformed by an SDR metric. Based on these findings, we show how a very simple framework for creating color HDR metrics, that uses only luminance SDR metrics, transfer functions, and classic color spaces, is able to consistently outperform, by a considerable margin, state-of-the-art HDR metrics on a varied set of HDR content, for both perceptual quantization (PQ) and Hybrid Log-Gamma (HLG) encoding, luminance and chroma distortions, and on different color spaces of common use.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Trust Your Good Friends, Source-Free Domain Adaptation by Reciprocal Neighborhood Clustering

  • Yang, Shiqi|||0000-0002-4141-7190
  • Wang, Yaxing|||0000-0002-6055-7164
  • Weijer, Joost van de|||0000-0002-9656-9706
  • Herranz, Luis|||0000-0002-7022-3395
  • Jui, Shangling|||0000-0002-1047-4264
  • Yang, Jian|||0000-0003-4800-832X
Domain adaptation (DA) aims to alleviate the domain shift between source domain and target domain. Most DA methods require access to the source data, but often that is not possible (e.g., due to data privacy or intellectual property). In this paper, we address the challenging source-free domain adaptation (SFDA) problem, where the source pretrained model is adapted to the target domain in the absence of source data. Our method is based on the observation that target data, which might not align with the source domain classifier, still forms clear clusters. We capture this intrinsic structure by defining local affinity of the target data, and encourage label consistency among data with high local affinity. We observe that higher affinity should be assigned to reciprocal neighbors. To aggregate information with more context, we consider expanded neighborhoods with small affinity values. Furthermore, we consider the density around each target sample, which can alleviate the negative impact of potential outliers. In the experimental results we verify that the inherent structure of the target features is an important source of information for domain adaptation. We demonstrate that this local structure can be efficiently captured by considering the local neighbors, the reciprocal neighbors, and the expanded neighborhood. Finally, we achieve state-of-the-art performance on several 2D image and 3D point cloud recognition datasets.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Casting a BAIT for offline and online source-free domain adaptation

  • Yang, Shiqi|||0000-0002-4141-7190
  • Wang, Yaxing|||0000-0002-6055-7164
  • Herranz, Luis|||0000-0002-7022-3395
  • Jui, Shangling|||0000-0002-1047-4264
  • van de Weijer, Joost
We address the source-free domain adaptation (SFDA) problem, where only the source model is available during adaptation to the target domain. We consider two settings: the offline setting where all target data can be visited multiple times (epochs) to arrive at a prediction for each target sample, and the online setting where the target data needs to be directly classified upon arrival. Inspired by diverse classifier based domain adaptation methods, in this paper we introduce a second classifier, but with another classifier head fixed. When adapting to the target domain, the additional classifier initialized from source classifier is expected to find misclassified features. Next, when updating the feature extractor, those features will be pushed towards the right side of the source decision boundary, thus achieving source-free domain adaptation. Experimental results show that the proposed method achieves competitive results for offline SFDA on several benchmark datasets compared with existing DA and SFDA methods, and our method surpasses by a large margin other SFDA methods under online source-free domain adaptation setting.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Generalized portrait quality assessment

  • Chahine, Nicolas|||0000-0002-8798-2814
  • Ferradans, Sira
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Ponce, Jean
Automated and robust portrait quality assessment (PQA) is of paramount importance in high-impact applications such as smartphone photography. This paper presents FHIQA, a learning-based approach to PQA that introduces a simple but effective quality score rescaling method based on image semantics, to enhance the precision of fine-grained image quality metrics while ensuring robust generalization to various scene settings beyond the training dataset. The proposed approach is validated by extensive experiments on the PIQ23 benchmark and comparisons with the current state of the art. The source code of FHIQA will be made publicly available on the PIQ23 GitHub repository at https://github.com/DXOMARK-Research/PIQ2023.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Leveraging Color Naming for Image Enhancement

  • Serrano Lozano, David|||0009-0009-1224-8453
  • Herranz, Luis|||0000-0002-7022-3395
  • Brown, Michael S.|||0000-0002-9840-0795
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
Enhancing images to make them visually appealing is a persistent challenge in computer vision. Many deep-learning methods train models on paired datasets to replicate expert editing styles. However, these approaches struggle with two key issues: (1) interpretability and (2) a parametrization suitable for user adjustments. To address these challenges, we present NamedCurves+, an approach inspired by the concept of Color Naming, a universal set of familiar colors widely used in software tools for intuitive editing. Our method integrates color names into a learning-based framework, enabling global adjustments for each named color through tone curves. To address local image variations, we incorporate a transformer block that captures spatial dependencies, enabling context-aware edits across the image. NamedCurves+ enhances the retouching process's interpretability and supports user interaction, allowing flexible modifications of individual tone curves to refine the retouched image according to personal preferences. Extensive experiments on tasks such as image retouching, tone mapping, and exposure correction demonstrate that NamedCurves+ outperforms state-of-the-art methods. Notably, our approach is both explainable, as the tone curves explicitly represent how each color name contributes to the enhancement, and interactive, allowing users to customize the retouching process and achieve results tailored to their liking. Source code and models will be publicly available at: https://namedcurves.github.io.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Color matching in the wild

  • Gil Rodríguez, Raquel
  • Vazquez-Corral, Javier
  • Bertalmio, Marcelo
  • Finlayson, Graham D.
11 pags., 9 figs., 7 tabs., We present a method that, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, corrects the colors of one of the images making it look as if it was captured under the other camera settings. Our method is able to deal with any standard non-linear encoded images (gamma-corrected, logarithmic-encoded, or any other) without requiring any previous knowledge of the encoding. To this end, our method makes use of two important observations. First, the camera imaging pipeline from RAW to sRGB can be well approximated by considering just a per-pixel shading and a color transformation matrix, and second, for correcting the images we only need to estimate a single matrix –that will contain information from both of the original images– and an approximation of the shading term (that emulates the non-linearity). Our proposed method is fast and the results have no spurious artifacts. The method outperforms the state-of-the-art when compared with other methods that do not require knowledge of the encoding used. It is also able to compete with –and even surpass in some cases– methods that consider information about image encoding., This work has received funding from different sources. RGR was supported by ERC Advanced Grant, Color3.0. JVC was supported by Grant PID2021-128178OB-I00 funded by MCIN/AEI/10.13039/ 501100011033 and by ERDF “A way of making Europe”, and also by the Departament de Recerca i Universitats from Generalitat de Catalunya with reference 2021SGR01499 . MB was supported by grant PID2021-127373NB-I00 from the Spanish Ministry of Science, Innovation and Universities (MICINN). GF was funded by EPSRC, United Kingdom Grant EP/S028730/1 ., Peer reviewed
Dipòsit Digital de Documents de la UAB
Publicación Artículo preliminar (WorkingPaper)

Nilut, conditional neural implicit 3d lookup tables for image enhancement

  • Conde, Marcos V.|||0000-0002-5823-4964
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Brown, Michael S.|||0000-0002-9840-0795
  • Timofte, Radu|||0000-0002-1478-0402
3D lookup tables (3D LUTs) are a key component for image enhancement. Modern image signal processors (ISPs) have dedicated support for these as part of the camera rendering pipeline. Cameras typically provide multiple options for picture styles, where each style is usually obtained by applying a unique handcrafted 3D LUT. Current approaches for learning and applying 3D LUTs are notably fast, yet not so memoryeffcient, as storing multiple 3D LUTs is required. For this reason and other implementation limitations, their use on mobile devices is less popular. In this work, we propose a Neural Implicit LUT (NILUT), an implicitly defned continuous 3D color transformation parameterized by a neural network. We show that NILUTs are capable of accurately emulating real 3D LUTs. Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly. Our novel approach is memory-effcient, controllable and can complement previous methods, including learned ISPs. Code at https://github.com/mv-lab/nilut.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Noise prism, a novel multispectral visualization technique

  • Canham, Trevor|||0000-0003-4830-4786
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Long, David L.
  • Murray, Richard F.
  • Brown, Michael S.|||0000-0002-9840-0795
A novel technique for visualizing multispectral images is proposed. Inspired by how prisms work, our method spreads spectral information over a chromatic noise pattern. This is accomplished by populating the pattern with pixels representing each measurement band at a count proportional to its measured intensity. The method is advantageous because it allows for lightweight encoding and visualization of spectral information while maintaining the color appearance of the stimulus. A four alternative forced choice (4AFC) experiment was conducted to validate the method's information-carrying capacity in displaying metameric stimuli of varying colors and spectral basis functions. The scores ranged from 100% to 20% (less than chance given the 4AFC task), with many conditions falling somewhere in between at statistically significant intervals. Using this data, color and texture difference metrics can be evaluated and optimized to predict the legibility of the visualization technique.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Burst perception-distortion tradeoff, analysis and evaluation

  • Xue, Danna|||0000-0002-4280-706X
  • Herranz, Luis|||0000-0002-7022-3395
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Zhang, Yanning|||0000-0002-2977-8057
Burst image restoration attempts to effectively utilize the complementary cues appearing in sequential images to produce a high-quality image. Most current methods use all the available images to obtain the reconstructed image. However, using more images for burst restoration is not always the best option regarding reconstruction quality and efficiency, as the images acquired by handheld imaging devices suffer from degradation and misalignment caused by the camera noise and shake. In this paper, we extend the perception-distortion tradeoff theory by introducing multiple-frame information. We propose the area of the unattainable region as a new metric for perception-distortion tradeoff evaluation and comparison. Based on this metric, we analyse the performance of burst restoration from the perspective of the perception-distortion tradeoff under both aligned bursts and misaligned bursts situations. Our analysis reveals the importance of inter-frame alignment for burst restoration and shows that the optimal burst length for the restoration model depends both on the degree of degradation and misalignment.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Improving the perception of low-light enhanced images

  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Finlayson, Graham D.|||0000-0003-4040-0554
  • Herranz, Luis|||0000-0002-7022-3395
Improving images captured under low-light conditions has become an important topic in computational color imaging, as it has a wide range of applications. Most current methods are either based on handcrafted features or on end-to-end training of deep neural networks that mostly focus on minimizing some distortion metric -such as PSNR or SSIM-on a set of training images. However, the minimization of distortion metrics does not mean that the results are optimal in terms of perception (i.e. perceptual quality). As an example, the perception-distortion trade-off states that, close to the optimal results, improving distortion results in worsening perception. This means that current low-light image enhancement methods -that focus on distortion minimization- cannot be optimal in the sense of obtaining a good image in terms of perception errors. In this paper, we propose a post-processing approach in which, given the original low-light image and the result of a specific method, we are able to obtain a result that resembles as much as possible that of the original method, but, at the same time, giving an improvement in the perception of the final image. More in detail, our method follows the hypothesis that in order to minimally modify the perception of an input image, any modification should be a combination of a local change in the shading across a scene and a global change in illumination color. We demonstrate the ability of our method quantitatively using perceptual blind image metrics such as BRISQUE, NIQE, or UNIQUE, and through user preference tests.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Integrating High-Level Features for Consistent Palette-based Multi-image Recoloring

  • Xue, Danna|||0000-0002-4280-706X
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Herranz, Luis|||0000-0002-7022-3395
  • Zhang, Yanning|||0000-0002-2977-8057
  • Brown, Michael S.|||0000-0002-9840-0795
Achieving visually consistent colors across multiple images is important when images are used in photo albums, websites, and brochures. Unfortunately, only a handful of methods address multi-image color consistency compared to one-to-one color transfer techniques. Furthermore, existing methods do not incorporate high-level features that can assist graphic designers in their work. To address these limitations, we introduce a framework that builds upon a previous palette-based color consistency method and incorporates three high-level features: white balance, saliency, and color naming. We show how these features overcome the limitations of the prior multi-consistency workflow and showcase the user-friendly nature of our framework.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Perceptual image enhancement for smartphone real-time applications

  • Conde, Marcos V.|||0000-0002-5823-4964
  • Vasluianu, Florin
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Timofte, Radu|||0000-0002-1478-0402
Recent advances in camera designs and imaging pipelines allow us to capture high-quality images using smartphones. However, due to the small size and lens limitations of the smartphone cameras, we commonly find artifacts or degradation in the processed images. The most common unpleasant effects are noise artifacts, diffraction artifacts, blur, and HDR overexposure. Deep learning methods for image restoration can successfully remove these artifacts. However, most approaches are not suitable for real-time applications on mobile devices due to their heavy computation and memory requirements.In this paper, we propose LPIENet, a lightweight network for perceptual image enhancement, with the focus on deploying it on smartphones. Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks. Moreover, to prove the efficiency and reliability of our approach, we deployed the model directly on commercial smartphones and evaluated its performance. Our model can process 2K resolution images under 1 second in mid-level commercial smartphones.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Color matching in the wild

  • Gil Rodríguez, Raquel|||0000-0001-5561-1990
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Bertalmío, Marcelo|||0000-0002-1023-8325
  • Finlayson, Graham D.|||0000-0003-4040-0554
We present a method that, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, corrects the colors of one of the images making it look as if it was captured under the other camera settings. Our method is able to deal with any standard non-linear encoded images (gamma-corrected, logarithmic-encoded, or any other) without requiring any previous knowledge of the encoding. To this end, our method makes use of two important observations. First, the camera imaging pipeline from RAW to sRGB can be well approximated by considering just a per-pixel shading and a color transformation matrix, and second, for correcting the images we only need to estimate a single matrix -that will contain information from both of the original images- and an approximation of the shading term (that emulates the non-linearity). Our proposed method is fast and the results have no spurious artifacts. The method outperforms the state-of-the-art when compared with other methods that do not require knowledge of the encoding used. It is also able to compete with -and even surpass in some cases- methods that consider information about image encoding.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Palette-Based Color Harmonization via Color Naming

  • Xue, Danna|||0000-0002-4280-706X
  • Vazquez-Corral, Javier|||0000-0003-0414-7096
  • Herranz, Luis|||0000-0002-7022-3395
  • Zhang, Yanning|||0000-0002-2977-8057
  • Brown, Michael S.|||0000-0002-9840-0795
Color harmony refers to combinations of colors that look pleasing together. We present a novel strategy to harmonize an image's colors using color-palette manipulation and color naming. Palette-based color manipulation is a method that extracts a few colors to represent the image. Modifying the palette colors modifies the color appearance of the image. A color-naming model is a mechanism to categorize colors into a fixed number of basic color terms. Working from a color-naming model, we derive a set of prototype colors and demonstrate that mapping an image's extracted color palette to the nearest prototype colors effectively harmonizes the image's colors. This straightforward approach yields visually compelling, outperforming more complex color harmony methods.