Pasar al contenido principal

APRENDIZAJE SEMI-SUPERVISADO PARA LA ANOTACION AUTOMATICA DE DATOS

PID2020-115734RB-C21

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario CENTRO DE VISION POR COMPUTADOR

Resultados relacionados

Resultados totales (Incluyendo duplicados): 7
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Closing the gap in domain adaptation for semantic segmentation, a time-aware method

  • Serrat, Joan|||0000-0002-4554-199X
  • Gomez Zurita, Jose Luis|||0000-0001-9511-1915
  • López Peña, Antonio M.|||0000-0002-6979-5783
Semantic segmentation models need a large number of images to be effectively trained but manual annotation of such images has a high cost. Active domain adaptation addresses this problem by pretraining the model with a synthetically generated dataset and then fine-tuning it with a few selected label annotations (the "budget") on real images to account for the domain shift. Previous works annotate a percentage of either individual pixels or whole target images. We argue that the first is infeasible in practice, and the second spends part of the budget on classes that the pretrained model may have already learned well. We propose a method based on the annotation of regions computed by Segment Anything, a recently introduced foundation model for class-agnostic image segmentation. The key idea is to assign a ground truth label to each of a tiny subset of regions, those for which the model is more uncertain. In order to increase the number of annotated regions we propagate the ground truth labels to most similar regions according to a hierarchical clustering algorithm that uses the features learned by the pretrained model. Our method outperforms the state-of-the-art on the GTA5 to Cityscapes benchmark by using fewer annotations, almost closing the gap between the synthetically pre-trained model and that obtained with full supervision of the real images. Furthermore, we present competitive results for budgets less than 1% of samples and also for a larger and more challenging target dataset, Mapillary Vistas.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Minimizing Human Labeling in Training Deep Models for Pedestrian Intention Prediction

  • Riaz, Muhammad Naveed|||0000-0002-0250-5913
  • Wielgosz, Maciej|||0000-0002-4401-2957
  • López Peña, Antonio M.|||0000-0002-6979-5783
Accurately predicting whether pedestrians will cross in front of an autonomous vehicle is essential for ensuring safe and comfortable maneuvers. However, developing models for this task remains challenging due to the limited availability of diverse datasets containing both crossing (C) and non-crossing (NC) scenarios. Therefore, we propose a procedure that leverages synthetic videos with C/NC labels and an untrained model whose architecture is designed for C/NC prediction to automatically produce C/NC labels for a set of real-world videos. Thus, this procedure performs a synth-to-real unsupervised domain adaptation for C/NC prediction, so we term it S2R-UDA-CP. To assess the effectiveness of S2R-UDA-CP in self-labeling, we utilize two state-of-the-art models, PedGNN and ST-CrossingPose, and we rely on the publicly-available PedSynth dataset, which consists of synthetic videos with C/NC labels. Notably, once the real-world videos are self-labeled, they can be used to train models different from those used in S2R-UDA-CP. These models are designed to operate onboard a vehicle, whereas S2R-UDA-CP is an offline procedure. To evaluate the quality of the C/NC labels generated by S2R-UDA-CP, we also employ PedGraph+ (another literature referent) as it is not used in S2R-UDA-CP. Overall, the results show that training models to predict C/NC using videos labeled by S2R-UDA-CP achieves performance even better than models trained on human-labeled data. Our study also highlights different discrepancies between automatic and human labeling. To the best of our knowledge, this is the first study to evaluate synth-to-real self-labeling for C/NC prediction.
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

All for one, and one for all: UrbanSyn Dataset, the third Musketeer of synthetic driving scenes

  • Gómez Zurita, José Luis
  • Silva, Manuel
  • Seoane, Antonio
  • Borràs, Agnés
  • Noriega, M. A.
  • Ros, German
  • Iglesias-Guitian, Jose A.
  • López, Antonio M.
This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/. This version of the article has been accepted for publication in Neurocomputing (ISSN 1872-8286)., [Abstract]: We introduce UrbanSyn, a photorealistic dataset acquired through semi-procedurally generated synthetic urban driving scenarios. Developed using high-quality geometry and materials, UrbanSyn provides pixel-level ground truth, including depth, semantic segmentation, and instance segmentation with object bounding boxes and occlusion degree. It complements GTAV and Synscapes datasets to form what we coin as the 'Three Musketeers'. We demonstrate the value of the Three Musketeers in unsupervised domain adaptation for image semantic segmentation. Results on real-world datasets, Cityscapes, Mapillary Vistas, and BDD100K, establish new benchmarks, largely attributed to UrbanSyn. We make UrbanSyn openly and freely accessible, This work has been supported by the Spanish grants Ref. PID2020-115734RB-C21 (ADA/SSL-ADA subproject) and PID2020-115734RB-C22 (ADA/PGAS-ADA subproject), both funded by MCIN/AEI/ 10.13039/501100011033. Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program, Spain. CVC’s authors acknowledge the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities. Jose A. Iglesias-Guitian acknowledges the financial support to his general research activities given by UDC-Inditex InTalent programme, the Spanish Ministry of Science and Innovation (AEI/RYC2018-025385-I), and Xunta de Galicia, Spain (ED431F 2021/11, EU-FEDER ED431G 2019/01), Xunta de Galicia; ED431F 2021/11, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Exploring the effects of synthetic data generation: a case study on autonomous driving for semantic segmentation

  • Silva, Manuel
  • Seoane, Antonio
  • Mures, Omar A.
  • López, Antonio M.
  • Iglesias-Guitian, Jose A.
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract]: Rendering 3D virtual scenarios has become a popular alternative for generating per-pixel-labeled image datasets, especially in fields like autonomous driving. The approach is valuable for training neural perception models, such as semantic segmentation models, particularly when data might be scarce, expensive, or difficult to collect. However, fundamental questions persist within the research community regarding the generation and processing of these synthetic images, particularly a better understanding of the key factors influencing the performance of deep learning models trained with such synthetic images. In response, we conducted a series of experiments to elucidate the impact that common aspects involved in the generation of rendered synthetic images may have on the performance of neural semantic segmentation tasks. Our study used a recent autonomous driving synthetic dataset as our main testbed, allowing us to investigate the effect of different approaches when modeling their geometric, material, and lighting details. We also studied the impact of rendering noise, typically produced by path-tracing algorithms, as well as the impact of using different color transformations and tonemapping algorithms., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been supported by Spanish grants Ref. PID2020-115734RB-C22 and PID2020-115734RB-C21, both funded by MCIN/AEI/10.13039/501100011033. Manuel Silva acknowledges Xunta de Galicia (Consellería de Cultura, Educacion e Universidade) for his predoctoral grant (ED481A-2023-191). Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program and the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities. J.A. Iglesias-Guitian also acknowledges the UDC-Inditex InTalent program, the Ministry of Science and Innovation (AEI/ RYC2018-025385-I) and Xunta de Galicia (ED431F 2021/11)., Xunta de Galicia; ED481A-2023-191, Xunta de Galicia; ED431F 2021/11
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

The contribution of luminance and chromatic channels to color assimilation

  • Otazu Porter, Xavier|||0000-0002-4982-791X
  • Cerdá Company, Xim|||0000-0001-7359-5453
Color induction is the phenomenon where the physical and the perceived colors of an object differ owing to the color distribution and the spatial configuration of the surrounding objects. Previous works studying this phenomenon on the lsY MacLeod-Boynton color space, show that color assimilation is present only when the magnocellular pathway (i.e., the Y axis) is activated (i.e., when there are luminance differences). Concretely, the authors showed that the effect is mainly induced by the koniocellular pathway (s axis), but not by the parvocellular pathway (l axis), suggesting that when magnocellular pathway is activated it inhibits the koniocellular pathway. In the present work, we study whether parvo-, konio-, and magnocellular pathways may influence on each other through the color induction effect. Our results show that color assimilation does not depend on a chromatic-chromatic interaction, and that chromatic assimilation is driven by the interaction between luminance and chromatic channels (mainly the magno- and the koniocellular pathways). Our results also show that chromatic induction is greatly decreased when all three visual pathways are simultaneously activated, and that chromatic pathways could influence each other through the magnocellular (luminance) pathway. In addition, we observe that chromatic channels can influence the luminance channel, hence inducing a small brightness induction. All these results show that color induction is a highly complex process where interactions between the several visual pathways are yet unknown and should be studied in greater detail.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Hierarchical novelty detection for traffic sign recognition

  • Ruiz López, Idoia|||0000-0003-0529-0871
  • Serrat, Joan|||0000-0002-4554-199X
Recent works have made significant progress in novelty detection, i.e., the problem of detecting samples of novel classes, never seen during training, while classifying those that belong to known classes. However, the only information this task provides about novel samples is that they are unknown. In this work, we leverage hierarchical taxonomies of classes to provide informative outputs for samples of novel classes. We predict their closest class in the taxonomy, i.e., its parent class. We address this problem, known as hierarchical novelty detection, by proposing a novel loss, namely Hierarchical Cosine Loss that is designed to learn class prototypes along with an embedding of discriminative features consistent with the taxonomy. We apply it to traffic sign recognition, where we predict the parent class semantics for new types of traffic signs. Our model beats state-of-the art approaches on two large scale traffic sign benchmarks, Mapillary Traffic Sign Dataset (MTSD) and Tsinghua-Tencent 100K (TT100K), and performs similarly on natural images benchmarks (AWA2, CUB). For TT100K and MTSD, our approach is able to detect novel samples at the correct nodes of the hierarchy with 81% and 36% of accuracy, respectively, at 80% known class accuracy.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Co-training for unsupervised domain adaptation of semantic segmentation models

  • Gomez Zurita, Jose Luis|||0000-0001-9511-1915
  • Villalonga, Gabriel|||0000-0002-1155-9374
  • López Peña, Antonio M.|||0000-0002-6979-5783
Semantic image segmentation is a core task for autonomous driving, which is performed by deep models. Since training these models draws to a curse of human-based image labeling, the use of synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies addressing an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new co-training procedure for synth-to-real UDA of semantic segmentation models. It performs iterations where the (unlabeled) real-world training images are labeled by intermediate deep models trained with both the (labeled) synthetic images and the real-world ones labeled in previous iterations. More specifically, a self-training stage provides two domain-adapted models and a model collaboration loop allows the mutual improvement of these two models. The final semantic segmentation labels (pseudo-labels) for the real-world images are provided by these two models. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for onboard semantic segmentation. Our procedure shows improvements ranging from approximately 13 to 31 mIoU points over baselines.