Skip to main content

Advanced Rendering for Scientific Visualization and Simulation

RYC2018-025385-I

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de Promoción del Talento y su Empleabilidad en I+D+i
Subprogram Subprograma Estatal de Incorporación
Call Ramón y Cajal
Call year 2018
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSIDADE DA CORUÑA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 8
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

A comprehensive handball dynamics dataset for game situation classification

  • Mures, Omar A.
  • Taibo, Javier
  • Padrón, Emilio J.
  • Iglesias-Guitian, Jose A.
[Abstract] This article presents a comprehensive dataset of labeled game situations obtained from multiple professional handball matches, which corresponds to the research paper entitled “PlayNet: Real-time Handball Play Classification with Kalman Embeddings and Neural Networks” [1]. The dataset encompasses approximately 11 hours of footage from five handball games played in two different arenas, resulting in around 1 million data frames. Each frame has been meticulously labeled using seven distinct game situation classes (left and right attacks, left and right transitions, left and right penalties, and timeouts). Notably, the dataset does not contain video frames, but provides a synthetic normalized representation of each frame. This representation includes information about player, referee, and ball positions, as well as player and referee velocities, for every labeled game situation. We obtained said details automatically by using an object detector to infer the positions of players, referees, and the ball in each frame. After tracking the detected agent positions across frames, the extracted coordinates underwent normalization through a “bird's eye” perspective transform, ensuring that the data remained unaffected by variations in camera configurations across different arenas. Finally, a Kalman filter was applied to improve the robustness of player positions and derive their velocities. The labeling process was performed by domain experts employing a custom system designed to annotate game situations, considering the play type and its contextual setting. In conclusion, researchers can utilize this dataset for several purposes: game analysis, automated broadcasting, or game summarization. Furthermore, this dataset can contribute to a broader understanding of the relationship between player dynamics and game situations, shedding light on the level of granularity required for accurately classifying them., This work has been developed under the European Innovation Council [Pilot No 954040], Xunta de Galicia [ED431F 2021/11, ED431G 2019/01, ED431C 2021/30], Spanish Ministry of Science and Innovation [MCIN/AEI/10.13039/501100011033, AEI/RYC2018-025385-I, PID2019- 104184RB-I00], and Jose A. Iglesias-Guitian also acknowledges the UDC-Inditex InTalent programme. Funding for open access charge: Universidade da Coruña/CISUG., Xunta de Galicia; ED431F 2021/11, Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2021/30
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Neural James-Stein Combiner for Unbiased and Biased Renderings

  • Gu, Jeongmin
  • Iglesias-Guitian, Jose A.
  • Moon, Bochang
This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in https://doi.org/10.1145/3550454.3555496.", [Abstract]: Unbiased rendering algorithms such as path tracing produce accurate images given a huge number of samples, but in practice, the techniques often leave visually distracting artifacts (i.e., noise) in their rendered images due to a limited time budget. A favored approach for mitigating the noise problem is applying learning-based denoisers to unbiased but noisy rendered images and suppressing the noise while preserving image details. However, such denoising techniques typically introduce a systematic error, i.e., the denoising bias, which does not decline as rapidly when increasing the sample size, unlike the other type of error, i.e., variance. It can technically lead to slow numerical convergence of the denoising techniques. We propose a new combination framework built upon the James-Stein (JS) estimator, which merges a pair of unbiased and biased rendering images, e.g., a path-traced image and its denoised result. Unlike existing post-correction techniques for image denoising, our framework helps an input denoiser have lower errors than its unbiased input without relying on accurate estimation of per-pixel denoising errors. We demonstrate that our framework based on the well-established JS theories allows us to improve the error reduction rates of state-of-the-art learning-based denoisers more robustly than recent post-denoisers, We appreciate the anonymous reviewers for the constructive comments. We also thank the following authors and artists for each scene: Mareck, SlykDragko, Wig42, NovaZeeke and thecali (training scenes in Fig. 9), aXel (Glass-of-water), Cem Yuksel (Curly-hair), NewSee20135 (Staircase), Ondřej Karlík (Pool), Tiziano Portenier (Bookshelf for the Mitsuba porting), and Christian Schüller (Dragon). Bochang Moon is the corresponding author of the paper. This work was supported by the National Research Foundation of Korea (NRF) funded by the Korea government (MSIT) (No. 2020R1A2C4002425) and Ministry of Culture, Sports and Tourism and Korea Creative Content Agency (No. R2021080001). Jose A. Iglesias-Guitian was supported by a 2021 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation. He also acknowledges the UDC-Inditex InTalent programme, the Spanish Ministry of Science and Innovation (AEI/PID2020-115734RB-C22 and AEI/ RYC2018-025385-I), Xunta de Galicia (ED431F 2021/11) and EUFEDER Galicia (ED431G 2019/01)., Korea. Korea government (MSIT); 2020R1A2C4002425, Korea. Ministry of Culture, Sports and Tourism; R2021080001, Xunta de Galicia; ED431F 2021/11, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

All for one, and one for all: UrbanSyn Dataset, the third Musketeer of synthetic driving scenes

  • Gómez Zurita, José Luis
  • Silva, Manuel
  • Seoane, Antonio
  • Borràs, Agnés
  • Noriega, M. A.
  • Ros, German
  • Iglesias-Guitian, Jose A.
  • López, Antonio M.
This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/. This version of the article has been accepted for publication in Neurocomputing (ISSN 1872-8286)., [Abstract]: We introduce UrbanSyn, a photorealistic dataset acquired through semi-procedurally generated synthetic urban driving scenarios. Developed using high-quality geometry and materials, UrbanSyn provides pixel-level ground truth, including depth, semantic segmentation, and instance segmentation with object bounding boxes and occlusion degree. It complements GTAV and Synscapes datasets to form what we coin as the 'Three Musketeers'. We demonstrate the value of the Three Musketeers in unsupervised domain adaptation for image semantic segmentation. Results on real-world datasets, Cityscapes, Mapillary Vistas, and BDD100K, establish new benchmarks, largely attributed to UrbanSyn. We make UrbanSyn openly and freely accessible, This work has been supported by the Spanish grants Ref. PID2020-115734RB-C21 (ADA/SSL-ADA subproject) and PID2020-115734RB-C22 (ADA/PGAS-ADA subproject), both funded by MCIN/AEI/ 10.13039/501100011033. Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program, Spain. CVC’s authors acknowledge the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities. Jose A. Iglesias-Guitian acknowledges the financial support to his general research activities given by UDC-Inditex InTalent programme, the Spanish Ministry of Science and Innovation (AEI/RYC2018-025385-I), and Xunta de Galicia, Spain (ED431F 2021/11, EU-FEDER ED431G 2019/01), Xunta de Galicia; ED431F 2021/11, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publication Contribución de congreso (ConferenceOutput)

Immersive 3D Medical Visualization in Virtual Reality using Stereoscopic Volumetric Path Tracing

  • Taibo, Javier
  • Iglesias-Guitian, Jose A.
[Abstract] Scientific visualizations using physically-based lighting models play a crucial role in enhancing both image quality and realism. In the domain of medical visualization, this trend has gained significant traction under the term cinematic rendering (CR). It enables the creation of 3D photorealistic reconstructions from medical data, offering great potential for aiding healthcare professionals in the analysis and study of volumetric datasets. However, the adoption of such advanced rendering for immersive virtual reality (VR) faces two main limitations related to their high computational demands. First, these techniques are frequently used to produce pre-recorded videos and offline content, thereby restricting interactivity to predefined volume appearance and lighting settings. Second, when deployed in head-tracked VR environments they can induce cyber-sickness symptoms due to the disturbing flicker caused by noisy Monte Carlo renderings. Consequently, the scope for meaningful interactive operations is constrained in this modality, in contrast with the versatile capabilities of classical direct volume rendering (DVR). In this work, we introduce an immersive 3D medical visualization system capable of producing photorealistic and fully interactive stereoscopic visualizations on head-mounted display (HMD) devices. Our approach extends previous linear regression denoising to enable real-time stereoscopic cinematic rendering within AR/VR settings. We demonstrate the capabilities of the resulting VR system, like its interactive rendering, appearance and transfer function editing., The anonymized medical datasets employed in this study were obtained courtesy of the OsiriX Foundation (specifically, MANIX and MACOESSIX), and were freely available through Embodi3D LLC (HAND). We thank Prof. Jérôme Schmid for the helpful and in-sightful discussions about the usability of the system by healthcare professionals. We also thank Prof. Bochang Moon for our helpful discussions on image-based denoising using weighted recursive least squares. Both authors would like to express their gratitude with Tom Boudard, Enrique García, Cristina Pelayo and Miguel Osorio who all helped us improve the VR GUI and tools of the system. We also thank Manuel Silva for his last-minute help in preparing the supplementary website, and Bea Blanco for her administrative assistance. We also thank Giovanni Fiorilli for his support in building our HTC Developer Partnership. Jose A. Iglesias-Guitian was supported by a 2021 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation. He also acknowledges the UDC-Inditex InTalent programme, the Spanish Ministry of Science and Innovation (AEI/RYC2018-025385-I) and Xunta de Galicia (ED431F 2021/11, Xunta de Galicia; ED431F 2021/11
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

PlayNet: real-time handball play classification with Kalman embeddings and neural networks

  • Mures, Omar A.
  • Taibo, Javier
  • Padrón, Emilio J.
  • Iglesias-Guitian, Jose A.
Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature, [Abstract] Real-time play recognition and classification algorithms are crucial for automating video production and live broadcasts of sporting events. However, current methods relying on human pose estimation and deep neural networks introduce high latency on commodity hardware, limiting their usability in low-cost real-time applications. We present PlayNet, a novel approach toreal-time handball play classification. Our method is based on Kalman embeddings, a new low-dimensional representation for game states that enables efficient operation on commodity hardware and customized camera layouts. Firstly, we leverage Kalman filtering to detect and track the main agents in the playing field, allowing us to represent them in a single normalized coordinate space. Secondly,weutilize a neural network trained in nonlinear dimensionality reduction through fuzzy topological data structure analysis. As a result, PlayNet achieves real-time play classification with under 55 ms of latency on commodity hardware, making it a promising addition to automated live broadcasting and game analysis pipelines., This work has been developed under the European Innovation Council Pilot No 954040. This work was supported also by ED431F 2021/11 and ED431G 2019/01 funded by Xunta de Galicia. Emilio J. Padrón’s work was also partially supported through the research projects PID2019-104184RB-I00 funded by MCIN/AEI/10.13039/501100011033, and ED431C 2021/30. Jose A. Iglesias-Guitian also acknowledges the UDC-Inditex InTalent programme and the Spanish Ministry of Science and Innovation (AEI/RYC2018-025385-I)., Xunta de Galicia; ED431F 2021/11, Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2021/30
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Exploring the effects of synthetic data generation: a case study on autonomous driving for semantic segmentation

  • Silva, Manuel
  • Seoane, Antonio
  • Mures, Omar A.
  • López, Antonio M.
  • Iglesias-Guitian, Jose A.
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract]: Rendering 3D virtual scenarios has become a popular alternative for generating per-pixel-labeled image datasets, especially in fields like autonomous driving. The approach is valuable for training neural perception models, such as semantic segmentation models, particularly when data might be scarce, expensive, or difficult to collect. However, fundamental questions persist within the research community regarding the generation and processing of these synthetic images, particularly a better understanding of the key factors influencing the performance of deep learning models trained with such synthetic images. In response, we conducted a series of experiments to elucidate the impact that common aspects involved in the generation of rendered synthetic images may have on the performance of neural semantic segmentation tasks. Our study used a recent autonomous driving synthetic dataset as our main testbed, allowing us to investigate the effect of different approaches when modeling their geometric, material, and lighting details. We also studied the impact of rendering noise, typically produced by path-tracing algorithms, as well as the impact of using different color transformations and tonemapping algorithms., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been supported by Spanish grants Ref. PID2020-115734RB-C22 and PID2020-115734RB-C21, both funded by MCIN/AEI/10.13039/501100011033. Manuel Silva acknowledges Xunta de Galicia (Consellería de Cultura, Educacion e Universidade) for his predoctoral grant (ED481A-2023-191). Antonio M. López acknowledges the financial support to his general research activities given by ICREA under the ICREA Academia Program and the support of the Generalitat de Catalunya CERCA Program and its ACCIO agency to CVC’s general activities. J.A. Iglesias-Guitian also acknowledges the UDC-Inditex InTalent program, the Ministry of Science and Innovation (AEI/ RYC2018-025385-I) and Xunta de Galicia (ED431F 2021/11)., Xunta de Galicia; ED481A-2023-191, Xunta de Galicia; ED431F 2021/11
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Real-Time Denoising of Volumetric Path Tracing for Direct Volume Rendering

  • Iglesias-Guitian, Jose A.
  • Mane, Prajita
  • Moon, Bochang
© 2020 IEEE. This version of the article has been accepted for publication, after peer review. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The Version of Record is available online at: https://doi.org/10.1109/TVCG.2020.3037680, [Abstract]: Direct volume rendering (DVR) using volumetric path tracing (VPT) is a scientific visualization technique that simulates light transport with objects’ matter using physically-based lighting models. Monte Carlo (MC) path tracing is often used with surface models, yet its application for volumetric models is difficult due to the complexity of integrating MC light-paths in volumetric media with none or smooth material boundaries. Moreover, auxiliary geometry-buffers (G-buffers) produced for volumes are typically very noisy, failing to guide image denoisers relying on that information to preserve image details. This makes existing real-time denoisers, which take noise-free G-buffers as their input, less effective when denoising VPT images. We propose the necessary modifications to an image-based denoiser previously used when rendering surface models, and demonstrate effective denoising of VPT images. In particular, our denoising exploits temporal coherence between frames, without relying on noise-free G-buffers, which has been a common assumption of existing denoisers for surface-models. Our technique preserves high-frequency details through a weighted recursive least squares that handles heterogeneous noise for volumetric models. We show for various real data sets that our method improves the visual fidelity and temporal stability of VPT during classic DVR operations such as camera movements, modifications of the light sources, and editions to the volume transfer function, The authors would like to thank reviewers for their insightful feedback. We also thank S. Oh, F. Lumbreras and J. Serrat for their help with Tensorflow, and T. Kroes for Exposure Render. The datasets are courtesy of the OsiriX Foundation (MANIX), University of Texas (CHAMELEON), University of Arizona (HELODERMA) and XYZ RGB Inc. (DRAGON). This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie g.a. No 665919. J.A. Iglesias-Guitian acknowledges the UDC-Inditex InTalent programme, the Spanish Ministry project TIN2017-88709-R, RYC2018-025385-I (MCIU/AEI/FEDER, EU), FEDER Galicia ED431G 2019/01 and the Nvidia GPU Grant Program. B. Moon was supported by The Cross-Ministry Giga KOREA Project grant funded by the Korea government (MSIT) (No. GK20P0300), Xunta de Galicia; ED431G 2019/01, Korea. Korea government (MSIT); GK20P0300
RUC. Repositorio da Universidade da Coruña
Publication Artículo científico (JournalArticle)

Should I render or should AI Generate? Crafting Synthetic Semantic Segmentation Datasets with Controlled Generation

  • Mures, Omar A.
  • Silva, Manuel
  • Lijó-Sanchez, Manuel
  • Padrón, Emilio J.
  • Iglesias-Guitian, Jose A.
Versión aceptada, publicada en modo "Early Publication" por el editor., [Abstract]: This work explores the integration of generative AI models for automatically generating synthetic image-labeled data. Our approach leverages controllable Diffusion Models to generate synthetic variations of semantically labeled images. Synthetic datasets for semantic segmentation struggle to represent real-world subtleties, such as different weather conditions or fine details, typically relying on costly simulations and rendering. However, Diffusion Models can generate diverse images using input text prompts and guidance images, like semantic masks. Our work introduces and tests a novel methodology for generating labeled synthetic images, with an initial focus on semantic segmentation, a demanding computer vision task. We showcase our approach in two distinct image segmentation domains, outperforming traditional computer graphics simulations in efficiently creating diverse datasets and training downstream models. We leverage generative models for crafting synthetically labeled images, posing the question: “Should I render or should AI generate?”. Our results endorse a paradigm shift towards controlled generation models, Xunta de Galicia: ED431F 2021/11, Xunta de Galicia: ED431G 2019/01, The authors would like to thank José R. López from Deep Design Systems S.L. for the car parts dataset (PART). This work has been supported by Xunta de Galicia: ED431F 2021/11 and ED431G 2019/01. It was also partially supported through the research projects AEI/PID2020-115734RB-C22 and PID2022-136435NB-I00, funded by MCIN/AEI/10.13039/501100011033 and by ‘ERDF A way of making Europe’, EU. Jose A. Iglesias-Guitian also acknowledges the UDC-Inditex InTalent programme and the Spanish Ministry of Science and Innovation (AEI/RYC2018-025385-I)