Pasar al contenido principal

APRENDIZAJE PROFUNDO MULTI-TAREA PARA RECONOCIMIENTO DE OBJETOS

TIN2016-79717-R

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2016
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario CENTRO DE VISION POR COMPUTADOR

Resultados relacionados

Resultados totales (Incluyendo duplicados): 12
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Mix and Match Networks, Cross-Modal Alignment for Zero-Pair Image-to-Image Translation

  • Wang, Yaxing|||0000-0002-6055-7164
  • Herranz, Luis|||0000-0002-7022-3395
  • Weijer, Joost van de|||0000-0002-9656-9706
This paper addresses the problem of inferring unseen cross-modal image-to-image translations between multiple modalities. We assume that only some of the pairwise translations have been seen (i.e. trained) and infer the remaining unseen translations (where training pairs are not available). We propose mix and match networks, an approach where multiple encoders and decoders are aligned in such a way that the desired translation can be obtained by simply cascading the source encoder and the target decoder, even when they have not interacted during the training stage (i.e. unseen). The main challenge lies in the alignment of the latent representations at the bottlenecks of encoder-decoder pairs. We propose an architecture with several tools to encourage alignment, including autoencoders and robust side information and latent consistency losses. We show the benefits of our approach in terms of effectiveness and scalability compared with other pairwise image-to-image translation approaches. We also propose zero-pair cross-modal image translation, a challenging setting where the objective is inferring semantic segmentation from depth (and vice-versa) without explicit segmentation-depth pairs, and only from two (disjoint) segmentation-RGB and depth-RGB training sets. We observe that a certain part of the shared information between unseen modalities might not be reachable, so we further propose a variant that leverages pseudo-pairs which allows us to exploit this shared information between the unseen modalities.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Domain-Adaptive Deep Network Compression

  • Masana, Marc|||0000-0003-3254-3096
  • Weijer, Joost van de|||0000-0002-9656-9706
  • Herranz, Luis|||0000-0002-7022-3395
  • Bagdanov, Andrew|||0000-0001-6408-7043
  • Alvarez, Jose M.
Deep Neural Networks trained on large datasets can be easily transferred to new domains with far fewer labeled examples by a process called fine-tuning. This has the advantage that representations learned in the large source domain can be exploited on smaller target domains. However, networks designed to be optimal for the source task are often prohibitively large for the target task. In this work we address the compression of networks after domain transfer. We focus on compression algorithms based on low-rank matrix decomposition. Existing methods base compression solely on learned network weights and ignore the statistics of network activations. We show that domain transfer leads to large shifts in network activations and that it is desirable to take this into account when compressing. We demonstrate that considering activation statistics when compressing weights leads to a rank-constrained regression problem with a closed-form solution. Because our method takes into account the target domain, it can more optimally remove the redundancy in the weights. Experiments show that our Domain Adaptive Low Rank (DALR) method significantly outperforms existing low-rank compression techniques. With our approach, the fc6 layer of VGG19 can be compressed more than 4x more than using truncated SVD alone - with only a minor or no loss in accuracy. When applied to domain-transferred networks it allows for compression down to only 5-20% of the original number of parameters with only a minor drop in performance.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Transferring GANs, Generating images from limited data

  • Wang, Yaxing|||0000-0002-6055-7164
  • Wu, Chenshen
  • Herranz, Luis|||0000-0002-7022-3395
  • Weijer, Joost van de|||0000-0002-9656-9706
  • Gonzalez-Garcia, Abel
  • Raducanu, Bogdan|||0000-0003-3648-8020
Transferring knowledge of pre-trained networks to new domains by means of fine-tuning is a widely used practice for applications based on discriminative models. To the best of our knowledge this practice has not been studied within the context of generative deep networks. Therefore, we study domain adaptation applied to image generation with generative adversarial networks. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Our results show that using knowledge from pre-trained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when target data is limited. We show that these conclusions can also be drawn for conditional GANs even when the pre-trained model was trained without conditioning. Our results also suggest that density is more important than diversity and a dataset with one or few densely sampled classes is a better source model than more diverse datasets such as ImageNet or Places.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

MineGAN, Effective Knowledge Transfer from GANs to Target Domains with Few Images

  • Wang, Yaxing|||0000-0002-6055-7164
  • Gonzalez-Garcia, Abel
  • Berga, David
  • Herranz, Luis|||0000-0002-7022-3395
  • Khan, Fahad Shahbaz|||0000-0002-4263-3143
  • Weijer, Joost van de|||0000-0002-9656-9706
One of the attractive characteristics of deep neural networks is their ability to transfer knowledge obtained in one domain to other related domains. As a result, high-quality networks can be trained in domains with relatively little training data. This property has been extensively studied for discriminative networks but has received significantly less attention for generative models. Given the often enormous effort required to train GANs, both computationally as well as in the dataset collection, the re-use of pretrained GANs is a desirable objective. We propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs. This is done using a miner network that identifies which part of the generative distribution of each pretrained GAN outputs samples closest to the target domain. Mining effectively steers GAN sampling towards suitable regions of the latent space, which facilitates the posterior finetuning and avoids pathologies of other methods such as mode collapse and lack of flexibility. We perform experiments on several complex datasets using various GAN architectures (BigGAN, Progressive GAN) and show that the proposed method, called MineGAN, effectively transfers knowledge to domains with few target images, outperforming existing methods. In addition, MineGAN can successfully transfer knowledge from multiple pretrained GANs. Our code is available at: \url{https://github.com/yaxingwang/MineGAN}.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

GTCreator, a flexible annotation tool for image-based datasets

  • Bernal del Nozal, Jorge|||0000-0001-8493-9514
  • Histace, Aymeric|||0000-0002-3029-4412
  • Masana, Marc|||0000-0003-3254-3096
  • Angermann, Quentin
  • Sánchez Montes, Cristina|||0000-0002-9692-9521
  • Rodríguez De Miguel, Cristina|||0000-0002-5576-0854
  • Hammami, Maroua
  • García-Rodríguez, Ana
  • Córdova, Henry|||0000-0002-6636-6764
  • Romain, Olivier
  • Fernández-Esparrach, Gloria|||0000-0002-3378-3940
  • Dray, Xavier
  • Sánchez, F. Javier|||0000-0002-9364-3122
Purpose:: Methodology evaluation for decision support systems for health is a time-consuming task. To assess performance of polyp detection methods in colonoscopy videos, clinicians have to deal with the annotation of thousands of images. Current existing tools could be improved in terms of flexibility and ease of use. Methods:: We introduce GTCreator, a flexible annotation tool for providing image and text annotations to image-based datasets. It keeps the main basic functionalities of other similar tools while extending other capabilities such as allowing multiple annotators to work simultaneously on the same task or enhanced dataset browsing and easy annotation transfer aiming to speed up annotation processes in large datasets. Results:: The comparison with other similar tools shows that GTCreator allows to obtain fast and precise annotation of image datasets, being the only one which offers full annotation editing and browsing capabilites. Conclusion:: Our proposed annotation tool has been proven to be efficient for large image dataset annotation, as well as showing potential of use in other stages of method evaluation such as experimental setup or results analysis.
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Variable Rate Deep Image Compression With Modulated Autoencoder

  • Yang, Fei
  • Herranz, Luis
  • Van de Weijer, Joost
  • Iglesias-Guitian, Jose A.
  • López, Antonio M.
  • Mozerov, Mikhail G.
This version of the paper has been accepted for publication. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., [Abstract]: Variable rate is a requirement for flexible and adaptable image and video compression. However, deep image compression methods (DIC) are optimized for a single fixed rate-distortion (R-D) tradeoff. While this can be addressed by training multiple models for different tradeoffs, the memory requirements increase proportionally to the number of models. Scaling the bottleneck representation of a shared autoencoder can provide variable rate compression with a single shared autoencoder. However, the R-D performance using this simple mechanism degrades in low bitrates, and also shrinks the effective range of bitrates. To address these limitations, we formulate the problem of variable R-D optimization for DIC, and propose modulated autoencoders (MAEs), where the representations of a shared autoencoder are adapted to the specific R-D tradeoff via a modulation network. Jointly training this modulated autoencoder and the modulation network provides an effective way to navigate the R-D operational curve. Our experiments show that the proposed method can achieve almost the same R-D performance of independent models with significantly fewer parameters, This work was supported in part by Audi Electronics Venture GmbH. The Computer Vision Center (CVC) was supported by the Generalitat de Catalunya CERCA Program and its ACCIO agency. The work of F. Yang was supported by the Chinese Scholarship Council under Grant 201706290127. The work of L. Herranz was supported by the Spanish project RTI2018-102285-A-I00. The work of J. van de Weijer was supported by the Spanish project TIN2016-79717-R. The work of A. M. López and J. A. I. Guitián was supported by the project TIN2017-88709-R (MINECO/AEI/FEDER, UE). The work of A. M. López was also supported by the ICREA Academia programme. The work of J. A. I. Guitián and L. Herranz was supported by the EU’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie Grant 665919
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Recognizing New Classes with Synthetic Data in the Loop, Application to Traffic Sign Recognition

  • Villalonga, Gabriel|||0000-0002-1155-9374
  • Weijer, Joost van de|||0000-0002-9656-9706
  • López Peña, Antonio M.|||0000-0002-6979-5783
On-board vision systems may need to increase the number of classes that can be recognized in a relatively short period. For instance, a traffic sign recognition system may suddenly be required to recognize new signs. Since collecting and annotating samples of such new classes may need more time than we wish, especially for uncommon signs, we propose a method to generate these samples by combining synthetic images and Generative Adversarial Network (GAN) technology. In particular, the GAN is trained on synthetic and real-world samples from known classes to perform synthetic-to-real domain adaptation, but applied to synthetic samples of the new classes. Using the Tsinghua dataset with a synthetic counterpart, SYNTHIA-TS, we have run an extensive set of experiments. The results show that the proposed method is indeed effective, provided that we use a proper Convolutional Neural Network (CNN) to perform the traffic sign recognition (classification) task as well as a proper GAN to transform the synthetic images. Here, a ResNet101-based classifier and domain adaptation based on CycleGAN performed extremely well for a ratio for new/known classes; even for more challenging ratios such as , the results are also very positive.
Dipòsit Digital de Documents de la UAB
Publicación

Metric learning for novelty and anomaly detection

  • Masana, Marc|||0000-0003-3254-3096
  • Ruiz López, Idoia|||0000-0003-0529-0871
  • Serrat, Joan|||0000-0002-4554-199X
  • Weijer, Joost van de|||0000-0002-9656-9706
  • López Peña, Antonio M.|||0000-0002-6979-5783
When neural networks process images which do not resemble the distribution seen during training, so called out-of-distribution images, they often make wrong predictions, and do so too confidently. The capability to detect out-of-distribution images is therefore crucial for many real-world applications. We divide out-of-distribution detection between novelty detection -images of classes which are not in the training set but are related to those-, and anomaly detection -images with classes which are unrelated to the training set. By related we mean they contain the same type of objects, like digits in MNIST and SVHN. Most existing work has focused on anomaly detection, and has addressed this problem considering networks trained with the cross-entropy loss. Differently from them, we propose to use metric learning which does not have the drawback of the softmax layer (inherent to cross-entropy methods), which forces the network to divide its prediction power over the learned classes. We perform extensive experiments and evaluate both novelty and anomaly detection, even in a relevant application such as traffic sign recognition, obtaining comparable or better results than previous works.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Active learning for deep detection neural networks

  • Aghdam, Hamed H.
  • Gonzalez Garcia, Abel
  • Weijer, Joost van de|||0000-0002-9656-9706
  • López Peña, Antonio M.|||0000-0002-6979-5783
The cost of drawing object bounding boxes (i.e. labeling) for millions of images is prohibitively high. For instance, labeling pedestrians in a regular urban image could take 35 seconds on average. Active learning aims to reduce the cost of labeling by selecting only those images that are informative to improve the detection network accuracy. In this paper, we propose a method to perform active learning of object detectors based on convolutional neural networks. We propose a new image-level scoring process to rank unlabeled images for their automatic selection, which clearly outperforms classical scores. The proposed method can be applied to videos and sets of still images. In the former case, temporal selection rules can complement our scoring process. As a relevant use case, we extensively study the performance of our method on the task of pedestrian detection. Overall, the experiments show that the proposed method performs better than random selection.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

Rotate your networks, better weight consolidation and less catastrophic forgetting

  • Liu, Xialei
  • Masana, Marc|||0000-0003-3254-3096
  • Herranz, Luis|||0000-0002-7022-3395
  • Weijer, Joost van de|||0000-0002-9656-9706
  • López Peña, Antonio M.|||0000-0002-6979-5783
  • Bagdanov, Andrew|||0000-0001-6408-7043
In this paper we propose an approach to avoiding catastrophic forgetting in sequential task learning scenarios. Our technique is based on a network reparameterization that approximately diagonalizes the Fisher Information Matrix of the network parameters. This reparameterization takes the form of a factorized rotation of parameter space which, when used in conjunction with Elastic Weight Consolidation (which assumes a diagonal Fisher Information Matrix), leads to significantly better performance on lifelong learning of sequential tasks. Experimental results on the MNIST, CIFAR-100, CUB-200 and Stanford-40 datasets demonstrate that we significantly improve the results of standard elastic weight consolidation, and that we obtain competitive results when compared to the state-of-the-art in lifelong learning without forgetting.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Variable rate deep image compression with modulated autoencoder

  • Yang, Fei|||0000-0003-4099-6511
  • Herranz, Luis|||0000-0002-7022-3395
  • Weijer, Joost van de|||0000-0002-9656-9706
  • Iglesias-Guitian, Jose A.|||0000-0002-0817-1010
  • López Peña, Antonio M.|||0000-0002-6979-5783
  • Mozerov, Mikhail|||0000-0002-1117-743X
Variable rate is a requirement for flexible and adaptable image and video compression. However, deep image compression methods (DIC) are optimized for a single fixed rate-distortion (R-D) tradeoff. While this can be addressed by training multiple models for different tradeoffs, the memory requirements increase proportionally to the number of models. Scaling the bottleneck representation of a shared autoencoder can provide variable rate compression with a single shared autoencoder. However, the R-D performance using this simple mechanism degrades in low bitrates, and also shrinks the effective range of bitrates. To address these limitations, we formulate the problem of variable R-D optimization for DIC, and propose modulated autoencoders (MAEs), where the representations of a shared autoencoder are adapted to the specific R-D tradeoff via a modulation network. Jointly training this modulated autoencoder and the modulation network provides an effective way to navigate the R-D operational curve. Our experiments show that the proposed method can achieve almost the same R-D performance of independent models with significantly fewer parameters.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Beyond eleven color names for image understanding

  • Yu, Lu
  • Zhang, Lichao
  • Weijer, Joost van de|||0000-0002-9656-9706
  • Khan, Fahad Shahbaz|||0000-0002-4263-3143
  • Cheng, Yongmei
  • Parraga, Carlos Alejandro|||0000-0002-3809-241X
Color description is one of the fundamental problems of image understanding. One of the popular ways to represent colors is by means of color names. Most existing work on color names focuses on only the eleven basic color terms of the English language. This could be limiting the discriminative power of these representations, and representations based on more color names are expected to perform better. However, there exists no clear strategy to choose additional color names. We collect a dataset of 28 additional color names. To ensure that the resulting color representation has high discriminative power we propose a method to order the additional color names according to their complementary nature with the basic color names. This allows us to compute color name representations with high discriminative power of arbitrary length. In the experiments we show that these new color name descriptors outperform the existing color name descriptor on the taskof visual tracking, person re-identification and image classification.