Pasar al contenido principal

SISTEMAS DE COMPRESION EFICIENTES PARA DATOS CIENTIFICOS DE NUEVA GENERACION

PID2024-156292OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para la Investigación y el Desarrollo Experimental
Subprograma Subprograma Estatal de Generación de Conocimiento Científico-Técnico y Desarrollo Experimental
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2024
Plan Estatal 2024-2027 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD AUTONOMA DE BARCELONA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Improving Individual and Regional Rainfall-Runoff Modeling in North American Watersheds Through Feature Selection and Hyperparameter Optimization

  • Ghanati, Bahareh
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Precise rainfall-runoff modeling (RRM) is vital for disaster management, resource conservation, and mitigation. Recent deep learning-based methods, such as long short-term memory (LSTM) networks, often struggle with major challenges, including temporal sensitivity, feature selection, generalizability, and hyperparameter tuning. The objective of this study is to develop an accurate and generalizable rainfall-runoff modeling framework that addresses the four aforementioned challenges. We propose a novel RRM framework that integrates transductive LSTM (TLSTM) to capture fine-grained temporal changes, off-policy proximal policy optimization (PPO) combined with Shapley Additive exPlanations (SHAP)-based reward functions for feature selection, an enhanced generative adversarial network (GAN) for online data augmentation, and Bayesian optimization hyperband (BOHB) for efficient hyperparameter tuning. TLSTM uses transductive learning, where samples near the test point are given extra weight, to capture fine-grained temporal shifts. Off-policy PPO contributes to this process by selecting features sensitive to temporal patterns in RRM. Our improved GAN conducts online data augmentation by excluding some gradients, increasing diversity and relevance in synthetic data. Finally, BOHB accelerates hyperparameter tuning by merging Bayesian optimization with the scaling efficiency of Hyperband. We evaluate our model using the Comprehensive Attributes and Meteorology for Large-Sample Studies (CAMELS) dataset under individual and regional scenarios. It achieves Nash-Sutcliffe efficiency (NSE) scores of 0.588 and 0.873, surpassing the baseline scores of 0.548 and 0.830, respectively. The generalizability of our approach was assessed on the hydro-climatic datasets for North America (HYSETS), also yielding improved performance. These improvements indicate more accurate capture of flow dynamics and peak events, supporting a robust and interpretable framework for RRM.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Robustness of Lossy Multispectral Compression to Simulated Instrumental Noise: A Comparative Study

  • González de Regàs, Jordi|||0009-0006-8525-5150
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Lossy compression of multispectral data has been widely adopted to tackle the downlink bottleneck in space-based Earth observation. However, compression artifacts may interact with the instrumental noise inherent to the sensors, affecting both the performance of the compressors and the effectiveness of denoising techniques. This letter evaluates the effects of lossy compression in the presence of instrumental noise and the corresponding denoising procedures on multispectral data, comparing current techniques for on-board lossy compression (CCSDS 122.0-B-1, near-lossless CCSDS 123.0-B-2, JPEG 2000, and exogenous KLT-JPEG 2000) with recent reduced-complexity models based on neural networks (mijares, chabert, oberlin and serra (MCOS) and spectral orthogonal transform encoder lossY (SORTENY)]. The rate-distortion performance of each codec is analyzed across four simulated instrumental noise conditions: thermal noise, dark signal, missing lines, and parallax-induced band misalignment. Experimental results show that all the evaluated codecs are resilient under thermal noise, dark signal, and missing line artifacts. However, for band misalignment artifacts, the compressors using a linear spectral transform, as E-KLT-JPEG 2000 and SORTENY, experienced a significant degradation.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

SGST, bit-depth recovery-based lossless compression for hyperspectral images

  • Xi, Zuoxin
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Deep learning applications to hyperspectral image (HSI) compression have attracted significant interest in recent years. Existing deep learning methods often rely on convolutional architectures, limiting their ability to capture long-range spectral dependencies. Their use of latent feature maps also requires full decoding before any downstream analysis. To address these limitations, we propose the Spectral Group Swin Transformer (SGST), a bit-depth recovery model that explicitly separates the most and least significant bits (MSB and LSB) and employs a transformer-based structure to enhance contextual modeling. Moreover, the MSB can be directly used for preliminary downstream analysis. Experiments on the HySpecNet-11k dataset show that SGST improves reconstruction quality by over 10 dB in PSNR compared to 3D-CAE and SSCNet, with lower entropy in lossy settings. While SGST's entropy rate in lossless compression is higher than the recent LineRWKV, its significantly lower computational complexity makes it better suited for onboard applications. Code is available at: https://github.com/ZuoxinXi/SGST.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Lossless Compression of Modern Astronomical Data Using a Novel Learned Predictor

  • Quintas Torra, Pau|||0009-0001-9873-279X
  • Fernández-Mellado, Xavier|||0009-0006-6806-4416
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Pinho, Armando J.|||0000-0002-9164-0016
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
The rapid growth of science-ready data volumes generated by modern ground-based astronomical observatories poses significant challenges for data storage and transmission. The de facto standard for storing these data is FITS, which incorporates a limited set of built-in compression methods. In this work, we present a novel lossless compression approach for astronomical images based on a weighted linear predictor combined with a CCSDS 123.0-B-2 mapper and a contextual binary arithmetic encoder. Three predictor configurations are evaluated, ranging from handcrafted designs to an optimized, dataset-adaptive model. Experimental results show consistent improvements over state-of-the-art lossless compressors. The optimized predictor achieves the best performance, improving compression by up to 0.471 bits per pixel (6.40%) compared to the best-performing FITS compressor, FPACK Hcompress. When applied to large-scale instruments such as the VLT, this corresponds to a reduction of approximately 0.25 TB per day (4.48%), yielding annual storage savings exceeding 90 TB. The implementation is publicly available at https://github.com/xavifeme00/Astronomy-Compressor.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Robustness of Lossy Multispectral Compression to Simulated Instrumental Noise: A Comparative Study

  • González de Regàs, Jordi
  • Mijares, Sebastiá
  • Bartrina, Joan
  • Serra-Sagristà, Joan
Lossy compression of multispectral data has been widely adopted to tackle the downlink bottleneck in space-based Earth observation. However, compression artifacts may interact with the instrumental noise inherent to the sensors, affecting both the performance of the compressors and the effectiveness of denoising techniques. This letter evaluates the effects of lossy compression in the presence of instrumental noise and the corresponding denoising procedures on multispectral data, comparing current techniques for on-board lossy compression (CCSDS 122.0-B-1, near-lossless CCSDS 123.0-B-2, JPEG 2000, and exogenous KLT-JPEG 2000) with recent reduced-complexity models based on neural networks (mijares, chabert, oberlin and serra (MCOS) and spectral orthogonal transform encoder lossY (SORTENY)]. The rate-distortion performance of each codec is analyzed across four simulated instrumental noise conditions: thermal noise, dark signal, missing lines, and parallax-induced band misalignment. Experimental results show that all the evaluated codecs are resilient under thermal noise, dark signal, and missing line artifacts. However, for band misalignment artifacts, the compressors using a linear spectral transform, as E-KLT-JPEG 2000 and SORTENY, experienced a significant degradation., This work was supported in part by Spanish Ministry of Science and Innovation (MICINN), and European Regional Development Fund (FEDER), funded by MCIN/AEI/10.13039/501100011033/FEDER, UE, under Grant PID2021-125258OB-I00 and Grant PID2024-156292OB-I00; in part by Catalan Government under Grant SGR2021-00643; in part by the Talent and Society Programme, Institute for Space Studies of Catalonia (IEEC) under Grant IEEC/60/2024; and in part by IEEC and Catalan Government through the Project “Data Compression and Machine Learning for Earth Observation Satellites,” within the framework of the New Space Strategy of Catalonia 2024 and 2025, Peer reviewed