Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 15
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Image rain removal and illumination enhancement done in one go

  • Wan, Yecong
  • Cheng, Yuanshuo
  • Shao, Ming-Wen|||0000-0001-7323-5896
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Rain removal plays an important role in the restoration of degraded images. Recently, CNN-based methods have achieved remarkable success. However, these approaches neglect that the appearance of real-world rain is often accompanied by low light conditions, which will further degrade the image quality, thereby hindering the restoration mission. Therefore, it is very indispensable to jointly remove the rain and enhance illumination for real-world rain image restoration. To this end, we proposed a novel spatially-adaptive network, dubbed SANet, which can remove the rain and enhance illumination in one go with the guidance of degradation mask. Meanwhile, to fully utilize negative samples, a contrastive loss is proposed to preserve more natural textures and consistent illumination. In addition, we present a new synthetic dataset, named DarkRain, to boost the development of rain image restoration algorithms in practical scenarios. DarkRain not only contains different degrees of rain, but also considers different lighting conditions, and more realistically simulates real-world rainfall scenarios. SANet is extensively evaluated on the proposed dataset and attains new state-of-the-art performance against other combining methods. Moreover, after a simple transformation, our SANet surpasses existing the state-of-the-art algorithms in both rain removal and low-light image enhancement.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Frequency-Based Enhancement Network for Efficient Super-Resolution

  • Behjati, Parichehr|||0000-0003-4266-545X
  • Rodríguez López, Pau|||0000-0002-1689-8084
  • Fernández Tena, Carles|||0000-0001-6185-3427
  • Mehri, Armin|||0000-0003-3472-2530
  • Roca, F. Xavier|||0000-0002-7043-7334
  • Ozawa, Seiichi|||0000-0002-0965-0064
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Recently, deep convolutional neural networks (CNNs) have provided outstanding performance in single image super-resolution (SISR). Despite their remarkable performance, the lack of high-frequency information in the recovered images remains a core problem. Moreover, as the networks increase in depth and width, deep CNN-based SR methods are faced with the challenge of computational complexity in practice. A promising and under-explored solution is to adapt the amount of compute based on the different frequency bands of the input. To this end, we present a novel Frequency-based Enhancement Block (FEB) which explicitly enhances the information of high frequencies while forwarding low-frequencies to the output. In particular, this block efficiently decomposes features into low- and high-frequency and assigns more computation to high-frequency ones. Thus, it can help the network generate more discriminative representations by explicitly recovering finer details. Our FEB design is simple and generic and can be used as a direct replacement of commonly used SR blocks with no need to change network architectures. We experimentally show that when replacing SR blocks with FEB we consistently improve the reconstruction error, while reducing the number of parameters in the model. Moreover, we propose a lightweight SR model - Frequency-based Enhancement Network (FENet) - based on FEB that matches the performance of larger models. Extensive experiments demonstrate that our proposal performs favorably against the state-of-the-art SR algorithms in terms of visual quality, memory footprint, and inference time.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

A Closer look at embedding propagation for manifold smoothing

  • Velazquez Dorta, Diego Alejandro|||0000-0001-6782-4336
  • Rodríguez López, Pau|||0000-0002-1689-8084
  • Gonfaus, Josep M.|||0000-0003-2079-4103
  • Roca, F. Xavier|||0000-0002-7043-7334
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Supervised training of neural networks requires a large amount of manually annotated data and the resulting networks tend to be sensitive to out-of-distribution (OOD) data. Self- and semi-supervised training schemes reduce the amount of annotated data required during the training process. However, OOD generalization remains a major challenge for most methods. Strategies that promote smoother decision boundaries play an important role in out-of-distribution generalization. For example, embedding propagation (EP) for manifold smoothing has recently shown to considerably improve the OOD performance for few-shot classification. EP achieves smoother class manifolds by building a graph from sample embeddings and propagating information through the nodes in an unsupervised manner. In this work, we extend the original EP paper providing additional evidence and experiments showing that it attains smoother class embedding manifolds and improves results in settings beyond few-shot classification. Concretely, we show that EP improves the robustness of neural networks against multiple adversarial attacks as well as semi- and self-supervised learning performance.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

MCNet, meta-clustering learning network for micro-expression recognition

  • Wang, Ziqi
  • Fu, Wenwen
  • Zhang, Yue|||0000-0001-7106-363X
  • Li, Jiarui|||0000-0001-5556-3001
  • Gong, Wenjuan|||0000-0001-7805-3629
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Facial micro-expressions are categorized into various types based on different criteria, and typically each major category is further divided into multiple subcategories of expressions. For traditional micro-expression recognition problems, multiple subcategories of the same emotions are indiscriminately learned and verified, leading to potential misclassification, especially with negative emotions. To address the issue of intra-class variation in micro-expressions, we propose a meta-clustering learning network for micro-expression recognition called MCNet. This approach integrates the ideas of meta-learning and clustering, hierarchically clustering subcategories within a micro-expression class to generate multiple class centers for metric-based classification. The proposed method diverges from the common strategy of metric-based meta-learning algorithms, which typically use the mean feature of all samples within the same class as the class center. Furthermore, we incorporate transfer learning into the meta-learning process to jointly alleviate overfitting caused by the scarcity of micro-expression data. We conduct extensive comparative experiments based on the leave-one-subject-out protocol on three widely used micro-expression datasets. The experimental results demonstrate the competitive performance and strong generalization ability of the proposed MCNet approach.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Single image super-resolution based on directional variance attention network

  • Behjati, Parichehr|||0000-0003-4266-545X
  • Rodríguez López, Pau|||0000-0002-1689-8084
  • Fernández Tena, Carles|||0000-0001-6185-3427
  • Hupont, Isabelle|||0000-0002-9811-9397
  • Mehri, Armin|||0000-0003-3472-2530
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Recent advances in single image super-resolution (SISR) explore the power of deep convolutional neural networks (CNNs) to achieve better performance. However, most of the progress has been made by scaling CNN architectures, which usually raise computational demands and memory consumption. This makes modern architectures less applicable in practice. In addition, most CNN-based SR methods do not fully utilize the informative hierarchical features that are helpful for final image recovery. In order to address these issues, we propose a directional variance attention network (DiVANet), a computationally efficient yet accurate network for SISR. Specifically, we introduce a novel directional variance attention (DiVA) mechanism to capture long-range spatial dependencies and exploit inter-channel dependencies simultaneously for more discriminative representations. Furthermore, we propose a residual attention feature group (RAFG) for parallelizing attention and residual block computation. The output of each residual block is linearly fused at the RAFG output to provide access to the whole feature hierarchy. In parallel, DiVA extracts most relevant features from the network for improving the final output and preventing information loss along the successive operations inside the network. Experimental results demonstrate the superiority of DiVANet over the state of the art in several datasets, while maintaining relatively low computation and memory footprint. The code is available at https://github.com/pbehjatii/DiVANet.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Meta-MMFNet, meta-learning based multi-model fusion network for micro-expression recognition

  • Gong, Wenjuan|||0000-0001-7805-3629
  • Zhang, Yue|||0000-0001-7106-363X
  • Wang, Wei|||0000-0002-5750-6980
  • Cheng, Peng|||0000-0003-3516-7196
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Despite its wide applications in criminal investigations and clinical communications with patients suffering from autism, automatic micro-expression recognition remains a challenging problem because of the lack of training data and imbalanced classes problems. In this study, we proposed a meta-learning based multi-model fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on the metric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable for model-level fusion. The frame difference and optical flow features were fused, deep features were extracted from the fused feature, and finally in the meta-learning-based framework, weighted sum model fusion method was applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-art methods on four datasets. The code is available at https://github.com/wenjgong/meta-fusion-based-method.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

MCLEMCD, multimodal collaborative learning encoder for enhanced music classification from dances

  • Gong, Wenjuan|||0000-0001-7805-3629
  • Yu, Qingshuang
  • Sun, Haoran
  • Huang, Wendong
  • Cheng, Peng|||0000-0003-3516-7196
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Music classification is widely applied in the automatic organization of music archives and intelligent music interfaces. Music is frequently accompanied by other media, such as image sequences. Combining various types of media for various tasks is natural for humans but extremely difficult for machines. In this work, we propose a collaborative learning method to combine dancing motions and music cues for music classification and apply it to music recommendations from dancing motions. Dancing motions in the form of 3D joint positions contain cyclic motions synchronized with music beats, and a collaborative autoencoder is designed to fuse music cues into a dancing motion feature extraction module. The proposed method achieved 98.07 % on the MusicToDance data set and 65.29 % on the AIST++ data set. The code to run all experiments is available at https://github.com/wenjgong/musicmotion.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Evaluating Visual Counterfactual Explainers, Explaining visual counterfactual explainers

  • Velazquez Dorta, Diego Alejandro|||0000-0001-6782-4336
  • Rodríguez López, Pau|||0000-0002-1689-8084
  • Lacoste, Alexandre
  • Laradji, Issam H.
  • Roca, F. Xavier|||0000-0002-7043-7334
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Explainability methods have been widely used to provide insight into the decisions made by statistical models, thus facilitating their adoption in various domains within the industry. Counterfactual explanation methods aim to improve our understanding of a model by perturbing samples in a way that would alter its response in an unexpected manner. This information is helpful for users and for machine learning practitioners to understand and improve their models. Given the value provided by counterfactual explanations, there is a growing interest in the research community to investigate and propose new methods. However, we identify two issues that could hinder the progress in this field. (1) Existing metrics do not accurately reflect the value of an explainability method for the users. (2) Comparisons between methods are usually performed with datasets like CelebA, where images are annotated with attributes that do not fully describe them and with subjective attributes such as ''Attractive''. In this work, we address these problems by proposing an evaluation method with a principled metric to evaluate and compare different counterfactual explanation methods. The evaluation is based on a synthetic dataset where images are fully described by their annotated attributes. As a result, we are able to perform a fair comparison of multiple explainability methods in the recent literature, obtaining insights about their performance. We make the code and data public to the research community.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

A complete benchmark for polyp detection, segmentation and classification in colonoscopy images

  • Tudela, Yael
  • Majó, Mireia
  • de la Fuente, Neil
  • Galdran, Adrian
  • Krenzer, Adrian
  • Puppe, Frank
  • Yamlahi, Amine
  • Tran, Thuy Nuong
  • Matuszewski, Bogdan J.
  • Fitzgerald, Kerr
  • Bian, Cheng
  • Pan, Junwen
  • Liu, Shijle
  • Fernández-Esparrach, Gloria|||0000-0002-3378-3940
  • Histace, Aymeric|||0000-0002-3029-4412
  • Bernal del Nozal, Jorge|||0000-0001-8493-9514
Colorectal cancer (CRC) is one of the main causes of deaths worldwide. Early detection and diagnosis of its precursor lesion, the polyp, is key to reduce its mortality and to improve procedure efficiency. During the last two decades, several computational methods have been proposed to assist clinicians in detection, segmentation and classification tasks but the lack of a common public validation framework makes it difficult to determine which of them is ready to be deployed in the exploration room. This study presents a complete validation framework and we compare several methodologies for each of the polyp characterization tasks. Results show that the majority of the approaches are able to provide good performance for the detection and segmentation task, but that there is room for improvement regarding polyp classification. While studied show promising results in the assistance of polyp detection and segmentation tasks, further research should be done in classification task to obtain reliable results to assist the clinicians during the procedure. The presented framework provides a standarized method for evaluating and comparing different approaches, which could facilitate the identification of clinically prepared assisting methods.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Audio-visual scene recognition using attention-based graph convolutional model

  • Wang, Ziqi
  • Wu, Yikai
  • Wang, Yifan
  • Gong, Wenjuan|||0000-0001-7805-3629
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Scene recognition aims to automatically comprehend scenes, and is widely utilized in various fields such as autonomous driving, intelligent security, and robotics. Current research predominantly employs local audio feature extractors, which results in the extracted features being unable to accommodate long-range contextual characteristics. Moreover, regarding the extracted features, most studies assume that the features of each modality possess equal importance. Our work primarily introduces a long-range audio feature extractor and employs a self-attention module to re-weight different features, addressing the limitations of the aforementioned local audio features and the varying importance of different modalities. We propose a visual-audio fusion model based on a self-attention-based graph convolutional neural network (SAGCN). In this model, we introduce an attention mechanism based cross-modal learning module into a structured multi-modal fusion network, and integrate the extracted features from different modalities to achieve precise scene recognition. The proposed model achieves an accuracy of 93.1% on a standard multi-modal scene recognition dataset: TAU dataset. Compared with other standard early and late fusion methods, the prediction accuracy enhances by 1.4% and 10%, respectively. For comparison with the SOTA methods, SAGCN exceeded the TAU baseline and attentional graph convolutional network on the TAU dataset by 8.3% and 1.5%, respectively, and achieved a 95.0% accuracy on the UCF101 dataset, outperforming the evolved loss method by 1.2% and the cross-modal deep clusterin method by 0.8%. The code is available at https://github.com/submission1234/SAGCN.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Logo Detection with No Priors

  • Velazquez Dorta, Diego Alejandro|||0000-0001-6782-4336
  • Gonfaus, Josep M.|||0000-0003-2079-4103
  • Rodríguez López, Pau|||0000-0002-1689-8084
  • Roca, F. Xavier|||0000-0002-7043-7334
  • Ozawa, Seiichi|||0000-0002-0965-0064
  • Gonzàlez, Jordi|||0000-0001-8033-0306
In recent years, top referred methods on object detection like R-CNN have implemented this task as a combination of proposal region generation and supervised classification on the proposed bounding boxes. Although this pipeline has achieved state-of-the-art results in multiple datasets, it has inherent limitations that make object detection a very complex and inefficient task in computational terms. Instead of considering this standard strategy, in this paper we enhance Detection Transformers (DETR) which tackles object detection as a set-prediction problem directly in an end-to-end fully differentiable pipeline without requiring priors. In particular, we incorporate Feature Pyramids (FP) to the DETR architecture and demonstrate the effectiveness of the resulting DETR-FP approach on improving logo detection results thanks to the improved detection of small logos. So, without requiring any domain specific prior to be fed to the model, DETR-FP obtains competitive results on the OpenLogo and MS-COCO datasets offering a relative improvement of up to 30%, when compared to a Faster R-CNN baseline which strongly depends on hand-designed priors.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Masked Discriminators for Content-Consistent Unpaired Image-to-Image Translation

  • Stuhr, Bonifaz
  • Brauer, Jürgen
  • Schick, Bernhard
  • Gonzàlez, Jordi|||0000-0001-8033-0306
A common goal of unpaired image-to-image translation is to preserve content consistency between source images and translated images while mimicking the style of the target domain. Due to biases between the datasets of both domains, many methods suffer from inconsistencies caused by the translation process. Most approaches introduced to mitigate these inconsistencies do not constrain the discriminator, leading to an even more ill-posed training setup. Moreover, none of these approaches is designed for larger crop sizes. In this work, we show that masking the inputs of a global discriminator for both domains with a content-based mask is sufficient to reduce content inconsistencies significantly. However, this strategy leads to artifacts that can be traced back to the masking process. To reduce these artifacts, we introduce a local discriminator that operates on pairs of small crops selected with a similarity sampling strategy. Furthermore, we apply this sampling strategy to sample global input crops from the source and target dataset. In addition, we propose feature-attentive denormalization to selectively incorporate content-based statistics into the generator stream. In our experiments, we show that our method achieves state-of-the-art performance in photorealistic sim-to-real translation and weather translation and also performs well in day-to-night translation. Additionally, we propose the cKVD metric, which builds on the sKVD metric and enables the examination of translation quality at the class or category level.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Gated Fusion Network With Progressive Modality Enhancement for Sentiment Analysis

  • Xin, Feng
  • Gong, Wenjuan|||0000-0001-7805-3629
  • Shi, Tingbo
  • Zhong, Jing
  • Li, Kechen
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Multi-modal sentiment analysis aims to recognize the sentiment expressed by the studied subjects utilizing textual, visual, and acoustic cues. Given the vast application prospects of multi-modal sentiment analysis in domains such as human-computer interaction and intelligent customer service systems, it has gained widespread attention in recent years. The design of fusion strategy significantly impacts analysis results. Different modalities varied on the extent of contributions to the sentiment analysis process, and the textual modality tends to play a crucial role. However, most existing fusion strategies have ignored the issue of uneven distribution of sentiment information across modalities. Therefore, in this study, we introduce a gated fusion network with progressive modality enhancement. To effectively address the challenge of imbalanced sentiment distribution, we propose a progressive modality enhancement approach that facilitates the interaction of sentiment cues. Furthermore, to overcome the limitations of insufficient inter-modal interactions observed in previous works, we introduce a gated multi-modal information interaction module. This module comprehensively exploits multi-modal data, ensuring that each modality can effectively utilize information from others. Experimental results demonstrate that the proposed method achieves highly competitive performance on the CMU-MOSI, CMU-MOSEI, and CH-SIMS datasets. Furthermore, the ablation experiments confirm the dominant role of the textual modality, while indicating that the visual modality generally performs better than the acoustic modality across our experimental settings.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy

  • García-Rodríguez, Ana
  • Tudela, Yael
  • Córdova, Henry|||0000-0002-6636-6764
  • Carballal, Sabela
  • Ordás, Ingrid|||0000-0001-7632-0340
  • Moreira, Leticia|||0000-0002-4518-8591
  • Vaquero, Eva|||0000-0003-0098-5295
  • Ortiz, Oswaldo
  • Rivero, Liseth
  • Sánchez, F. Javier|||0000-0002-9364-3122
  • Cuatrecasas, Miriam|||0000-0003-3063-0110
  • Pellisé, Maria|||0000-0001-9739-4998
  • Bernal del Nozal, Jorge|||0000-0001-8493-9514
  • Fernández-Esparrach, Gloria|||0000-0002-3378-3940
Background and study aims Artificial intelligence is currently able to accurately predict the histology of colorectal polyps. However, systems developed to date use complex optical technologies and have not been tested in vivo. The objective of this study was to evaluate the efficacy of a new deep learning-based optical diagnosis system, ATENEA, in a real clinical setting using only high-definition white light endoscopy (WLE) and to compare its performance with endoscopists. Methods ATENEA was prospectively tested in real life on consecutive polyps detected in colorectal cancer screening colonoscopies at Hospital Clínic. No images were discarded, and only WLE was used. The in vivo ATENEA's prediction (adenoma vs non-adenoma) was compared with the prediction of four staff endoscopists without specific training in optical diagnosis for the study purposes. Endoscopists were blind to the ATENEA output. Histology was the gold standard. Results Ninety polyps (median size: 5 mm, range: 2-25) from 31 patients were included of which 69 (76.7 %) were adenomas. ATENEA correctly predicted the histology in 63 of 69 (91.3 %, 95 % CI: 82 %-97 %) adenomas and 12 of 21 (57.1 %, 95 % CI: 34 %-78 %) non-adenomas while endoscopists made correct predictions in 52 of 69 (75.4 %, 95 % CI: 60 %-85 %) and 20 of 21 (95.2 %, 95 % CI: 76 %-100 %), respectively. The global accuracy was 83.3 % (95 % CI: 74%-90 %) and 80 % (95 % CI: 70 %-88 %) for ATENEA and endoscopists, respectively. Conclusion ATENEA can accurately be used for in vivo characterization of colorectal polyps, enabling the endoscopist to make direct decisions. ATENEA showed a global accuracy similar to that of endoscopists despite an unsatisfactory performance for non-adenomatous lesions.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

A Spatio-Temporal Spotting Network with Sliding Windows for Micro-Expression Detection

  • Fu, Wenwen
  • An, Zhihong
  • Huang, Wendong
  • Sun, Haoran
  • Gong, Wenjuan|||0000-0001-7805-3629
  • Gonzàlez, Jordi|||0000-0001-8033-0306
Micro-expressions reveal underlying emotions and are widely applied in political psychology, lie detection, law enforcement and medical care. Micro-expression spotting aims to detect the temporal locations of facial expressions from video sequences and is a crucial task in micro-expression recognition. In this study, the problem of micro-expression spotting is formulated as micro-expression classification per frame. We propose an effective spotting model with sliding windows called the spatio-temporal spotting network. The method involves a sliding window detection mechanism, combines the spatial features from the local key frames and the global temporal features and performs micro-expression spotting. The experiments are conducted on the CAS(ME) (Formula presented.) database and the SAMM Long Videos database, and the results demonstrate that the proposed method outperforms the state-of-the-art method by (Formula presented.) for the CAS(ME) (Formula presented.) and (Formula presented.) for the SAMM Long Videos according to overall F-scores.