Pasar al contenido principal

COTS Y APRENDIZAJE EN LA COMPRESION DE DATOS EN NANOSATELITES

PID2021-125258OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD AUTONOMA DE BARCELONA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 21
Encontrada(s) 1 página(s)

Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Integration of Generative-Adversarial-Network-Based Data Compaction and Spatial Attention Transductive Long Short-Term Memory for Improved Rainfall-Runoff Modeling

  • Ghanati, Bahareh
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
This work presents a novel approach to rainfall-runoff modeling. We incorporate GAN-based data compaction into a spatial-attention-enhanced transductive long short-term memory (TLSTM) network. The GAN component reduces data dimensions while retaining essential features. This compaction enables the TLSTM to capture complex temporal dependencies in rainfall-runoff patterns more effectively. When tested on the CAMELS dataset, the model significantly outperforms benchmark LSTM-based models. For 8-day runoff forecasts, our model achieves an NSE of 0.536, compared to 0.326 from the closest competitor. The integration of GAN-based feature extraction with spatial attention mechanisms improves predictive accuracy, particularly for peak-flow events. This method offers a powerful solution for addressing current challenges in water resource management and disaster planning under extreme climate conditions.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Lossy Compression of Integer Astronomical Images Preserving Photometric Properties

  • Maireles González, Òscar|||0000-0003-4754-0373
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Observatories are producing astronomical image data at quickly increasing rates. As a result, the efficiency of the compression methods employed is critical to meet the storage and distribution requirements of both observatories and scientists. This paper presents a novel lossy compression technique that is able to preserve the results of photometry analysis with high fidelity while improving upon the state of the art in terms of compression performance. The proposed compression pipeline combines a flexible bi-region quantization scheme with the lossless, dictionary-based, LPAQ9M encoder. The quantization process allows compression performance and photometric fidelity to be precisely tailored to different scientific requirements. A representative data set of 16-bit integer astronomical images produced by telescopes from all around the world has been employed to empirically assess its compression-fidelity trade-offs, and compare them to those of the de facto standard Fpack compressor. In these experiments, the widespread SExtractor software is employed as the ground truth for photometric analysis. Results indicate that after lossy compression with our proposed method, the decompressed data allows consistent detection of over 99% of all astronomical objects for all tested telescopes, maintaining the highest photometric fidelity (as compared to state of the art lossy techniques). When compared to the best configuration of Fpack (Hcompress lossy using 1 quantization parameter) at similar compression rates, our proposed method provides better photometry precision: 7.15% more objects are detected with magnitude errors below 0.01, and 9.13% more objects with magnitudes below SExtractor's estimated measurement error. Compared to the best lossless compression results, the proposed pipeline allows us to reduce the compressed data set volume by up to 38.75% and 27.94% while maintaining 90% and 95%, respectively, of the detected objects with magnitude differences lower than 0.01 mag; and up to 18.93% while maintaining 90% of the detected objects with magnitude differences lower than the photometric measure error.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Resiliency and Efficiency of the CCSDS 124.0-B-1 Telemetry Compression Standard

  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Evans, David
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
  • Blanes, Ian|||0000-0001-8939-1666
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
The Consultative Committee for Space Data Systems (CCSDS) recently published a lossless compression standard for housekeeping and telemetry. These data are critical for the safe and productive operation of virtually all remote sensing missions, including Earth observation. The new standard, CCSDS 124.0-B-1 "Robust Compression of Fixed-Length Housekeeping Data," allows lossless, real-time telemetry packet transmission with low energy and memory requirements. Furthermore, it offers flexible resiliency mechanisms to resume decompression after losing one or more data packets. This work provides an accessible introduction to CCSDS 124.0-B-1, paying special attention to its resiliency features. Two novel and complementary decoder resynchronization strategies are also presented, which maximize the amount of correctly decoded data. Comprehensive experiments with real and simulated mission data were conducted for this work to assess the standard's performance and the impact of user-defined parameters. Results indicate competitive coding performance, with average compression ratios between 4 and 16 depending on the dataset. Smooth trade-offs are also observed between compression performance and the degree of protection against data loss events, which enhances the usability of CCSDS 124.0-B-1 in multiple mission types and operational conditions.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Improving Individual and Regional Rainfall-Runoff Modeling in North American Watersheds Through Feature Selection and Hyperparameter Optimization

  • Ghanati, Bahareh
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Precise rainfall-runoff modeling (RRM) is vital for disaster management, resource conservation, and mitigation. Recent deep learning-based methods, such as long short-term memory (LSTM) networks, often struggle with major challenges, including temporal sensitivity, feature selection, generalizability, and hyperparameter tuning. The objective of this study is to develop an accurate and generalizable rainfall-runoff modeling framework that addresses the four aforementioned challenges. We propose a novel RRM framework that integrates transductive LSTM (TLSTM) to capture fine-grained temporal changes, off-policy proximal policy optimization (PPO) combined with Shapley Additive exPlanations (SHAP)-based reward functions for feature selection, an enhanced generative adversarial network (GAN) for online data augmentation, and Bayesian optimization hyperband (BOHB) for efficient hyperparameter tuning. TLSTM uses transductive learning, where samples near the test point are given extra weight, to capture fine-grained temporal shifts. Off-policy PPO contributes to this process by selecting features sensitive to temporal patterns in RRM. Our improved GAN conducts online data augmentation by excluding some gradients, increasing diversity and relevance in synthetic data. Finally, BOHB accelerates hyperparameter tuning by merging Bayesian optimization with the scaling efficiency of Hyperband. We evaluate our model using the Comprehensive Attributes and Meteorology for Large-Sample Studies (CAMELS) dataset under individual and regional scenarios. It achieves Nash-Sutcliffe efficiency (NSE) scores of 0.588 and 0.873, surpassing the baseline scores of 0.548 and 0.830, respectively. The generalizability of our approach was assessed on the hydro-climatic datasets for North America (HYSETS), also yielding improved performance. These improvements indicate more accurate capture of flow dynamics and peak events, supporting a robust and interpretable framework for RRM.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Robustness of Lossy Multispectral Compression to Simulated Instrumental Noise: A Comparative Study

  • González de Regàs, Jordi|||0009-0006-8525-5150
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Lossy compression of multispectral data has been widely adopted to tackle the downlink bottleneck in space-based Earth observation. However, compression artifacts may interact with the instrumental noise inherent to the sensors, affecting both the performance of the compressors and the effectiveness of denoising techniques. This letter evaluates the effects of lossy compression in the presence of instrumental noise and the corresponding denoising procedures on multispectral data, comparing current techniques for on-board lossy compression (CCSDS 122.0-B-1, near-lossless CCSDS 123.0-B-2, JPEG 2000, and exogenous KLT-JPEG 2000) with recent reduced-complexity models based on neural networks (mijares, chabert, oberlin and serra (MCOS) and spectral orthogonal transform encoder lossY (SORTENY)]. The rate-distortion performance of each codec is analyzed across four simulated instrumental noise conditions: thermal noise, dark signal, missing lines, and parallax-induced band misalignment. Experimental results show that all the evaluated codecs are resilient under thermal noise, dark signal, and missing line artifacts. However, for band misalignment artifacts, the compressors using a linear spectral transform, as E-KLT-JPEG 2000 and SORTENY, experienced a significant degradation.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Fixed-Quality Compression of Remote Sensing Images With Neural Networks

  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Chabert, Marie|||0000-0001-9869-4693
  • Oberlin, Thomas|||0000-0002-9680-4227
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Fixed-quality image compression is a coding paradigm where the tolerated introduced distortion is set by the user. This article proposes a novel fixed-quality compression method for remote sensing images. It is based on a neural architecture we have recently proposed for multirate satellite image compression. In this article, we show how to efficiently estimate the reconstruction quality using an appropriate statistical model. The performance of our approach is assessed and compared against recent fixed-quality coding techniques and standards in terms of accuracy and rate-distortion, as well as with recent machine learning compression methods in rate-distortion, showing competitive results. In particular, the proposed method does not introduce artifacts even when coding neighboring areas at different qualities.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Learned Spectral and Spatial Transforms for Multispectral Remote Sensing Data Compression

  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
As more and more multispectral and hyperspectral platforms are deployed for Earth observation (EO), limited downlink capacity increases the pressure for more efficient data compression algorithms. Machine learning (ML) has been successfully applied to produce highly competitive compression models though this performance has typically been at the cost of high computational complexity, a crucial limitation for on-board remote sensing data compression. To address these issues, a reduced-complexity multispectral and hyperspectral data compression architecture is proposed. Using separate spectral and spatial transforms, the complexity of the proposed models is scalable on the number of bands, regardless of the compression ratios. This proposal outperforms state-of-the-art ML compression models as well as established lossy compression methods such as JPEG 2000 prepended with a spectral Karhunen-Loève transform (KLT) on a variety of remote sensing data sources. The performance improvement is achieved with a lower complexity than said ML models. To reproduce our results, training and test data are publicly available at https://gici.uab.cat/GiciWebPage/datasets.php and source code at https://github.com/smijares/mbhs2025.
Dipòsit Digital de Documents de la UAB
Publicación

Visual image analysis of high-rate local fixed-quality compression with neural networks

  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Bruno, Mickaël
  • Latry, Christophe
  • Chabert, Marie|||0000-0001-9869-4693
  • Oberlin, Thomas|||0000-0002-9680-4227
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Fixed-quality compression is a lossy compression paradigm in which data is encoded at a user-defined tolerated distortion. When that tolerated distortion is allocated in certain regions of the image, it is referred to as local fixed-quality compression. It consists in compressing more complex regions at a higher bit rate and less complex areas at a lower bit rate, in order to reach the same level of quality with a reduced overall bit rate. Indeed, the global trend in high resolution satellite imagery is to drive the compression algorithm with a distortion target rather than with a bit rate target, which allows to allocate the bit rate resource where it is needed~\cite{camarero2015fixedquality}. The introduction of neural networks to data compression, particularly lossy image compression, has produced a breakthrough in the field. These novel methods outperform established algorithms that are the result of decades of innovation and fine-tuning. These advances have been applied to remote sensing satellite images through reduced-complexity algorithms that are compatible with on-board devices~\cite{alves2021mdpi, mijares2023multirate}. Indeed, missions such as ESA's Phi-Sat 2 will soon be pioneering the usage of neural compression on board remote sensing satellites~\cite{guerrisi2023phisat2}. Recent contributions have proposed methods for local fixed-quality compression using neural networks for remote sensing data, presenting rate-distortion and accuracy results for rates up to 1.5 bps or 2.5 bps~\cite{mijares2024fixedquality}. However in live applications target bit rates (and therefore image quality) are typically higher, so that distortion is at the level of instrumental noise. In this contribution, models for local fixed-quality compression at high qualities/bit rates are evaluated and visual image analysis is conducted on the artefacts. Furthermore, the computational capacity of the method is evaluated, showing it is compatible with new-generation on-board devices.
Dipòsit Digital de Documents de la UAB
Publicación Capítulo o parte de un libro (BookPart)

SGST, bit-depth recovery-based lossless compression for hyperspectral images

  • Xi, Zuoxin
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Deep learning applications to hyperspectral image (HSI) compression have attracted significant interest in recent years. Existing deep learning methods often rely on convolutional architectures, limiting their ability to capture long-range spectral dependencies. Their use of latent feature maps also requires full decoding before any downstream analysis. To address these limitations, we propose the Spectral Group Swin Transformer (SGST), a bit-depth recovery model that explicitly separates the most and least significant bits (MSB and LSB) and employs a transformer-based structure to enhance contextual modeling. Moreover, the MSB can be directly used for preliminary downstream analysis. Experiments on the HySpecNet-11k dataset show that SGST improves reconstruction quality by over 10 dB in PSNR compared to 3D-CAE and SSCNet, with lower entropy in lossy settings. While SGST's entropy rate in lossless compression is higher than the recent LineRWKV, its significantly lower computational complexity makes it better suited for onboard applications. Code is available at: https://github.com/ZuoxinXi/SGST.
Dipòsit Digital de Documents de la UAB
Publicación

Region of Interest Coding in Photsat Mission

  • Quintas Torra, Pau|||0009-0001-9873-279X
  • Fernández-Mellado, Xavier|||0009-0006-6806-4416
  • Maireles González, Òscar|||0000-0003-4754-0373
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
The Institute of Space Studies of Catalonia (IEEC) coordinates the PhotSat project [1], which involves the development and construction of a satellite to monitor the 10 million brightest stars in the sky for at least two years. The scientific objective of this project is to complement the Large Synoptic Survey Telescope (LSST) [2] and the GAIA mission [3] by examining the intensity of stars. In the PhotSat mission, a CubeSat is expected to be launched at the end of 2025, with the mission of scanning the entire sky using two cameras-one with an ultraviolet detector and another with a visible detector. The desired configurations of the camera field of view (FoV) and the sensor step size are 8◦ and 4◦, respectively, which will generate a total of 16.75 GB of image data that will need to be transmitted from the satellite to the Montsec ground station [4]. Since the download capacity to the Montsec station is limited to 8 GB per day, some compression is required. The compression presented in this paper consists of two stages. The first stage uses an auto- matic object detection technique based on photometry [5], which analyzes the pixel intensities of the acquired data to identify the pixels corresponding to the Regions of Interest (ROI). This pro- cess adjusts the image so that only the ROIs are preserved, eliminating irrelevant pixels. In the second stage, the modified data is losslessly compressed using a prediction-based compressor and a contextual binary arithmetic encoder [6]. To provide contextual information around the ROI, the ROI can be expanded using the parameter ROI Extended Coding Pixels (RECP), which adds a specified number of pixels around the detected objects. Additionally, both the predictor and ROI coding can be independently enabled or disabled, allowing for faster execution and, under certain circumstances, improved coding performance. Experimental results show that when our proposal is used with ROI coding disabled, all acquired data can be losslessly compressed to 5.86 GB. However, when ROI coding is enabled, compression performance improves significantly; for example, if RECP is set to 0 or 64, the acquired data can be compressed to 0.16 GB and 4.13 GB, respectively-a very significant compression.
Dipòsit Digital de Documents de la UAB
Publicación

On Board Compression For PhotSat Mission

  • Fernández-Mellado, Xavier|||0009-0006-6806-4416
  • Quintas Torra, Pau|||0009-0001-9873-279X
  • Ferrer Jorge, Gabriel
  • Maireles González, Òscar|||0000-0003-4754-0373
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Carrasco, Josep Manel
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
PhotSat is a satellite mission aimed at exploring deep space through high-resolution imaging using advanced optical sensors. Utilizing a CubeSat platform, the mission seeks to obtain precise astronomical data at a low cost, leveraging miniaturized technologies that reduce complexity while maximizing observational capabilities. The platform is equipped with a high-performance optical system capable of capturing images in two spectral bands (visual and ultraviolet), making it possible to detect transient events such as exoplanets or variable stars. The preliminary design of PhotSat integrates an innovative data compression system based on a binary context-based arithmetic coder, ensuring the preservation of quality in crucial observations while optimizing bandwidth and storage usage. A comparative experiment was conducted with five state-of-the-art codecs, and the results show that the proposed codec outperforms the others in terms of compression efficiency. PhotSat establishes a significant precedent in the field of astronomy with CubeSats, serving as a proof of concept for the feasibility of low-cost scientific missions yielding high-quality results.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Lossless Compression of Modern Astronomical Data Using a Novel Learned Predictor

  • Quintas Torra, Pau|||0009-0001-9873-279X
  • Fernández-Mellado, Xavier|||0009-0006-6806-4416
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Mijares i Verdú, Sebastià|||0000-0003-1038-6413
  • Pinho, Armando J.|||0000-0002-9164-0016
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
The rapid growth of science-ready data volumes generated by modern ground-based astronomical observatories poses significant challenges for data storage and transmission. The de facto standard for storing these data is FITS, which incorporates a limited set of built-in compression methods. In this work, we present a novel lossless compression approach for astronomical images based on a weighted linear predictor combined with a CCSDS 123.0-B-2 mapper and a contextual binary arithmetic encoder. Three predictor configurations are evaluated, ranging from handcrafted designs to an optimized, dataset-adaptive model. Experimental results show consistent improvements over state-of-the-art lossless compressors. The optimized predictor achieves the best performance, improving compression by up to 0.471 bits per pixel (6.40%) compared to the best-performing FITS compressor, FPACK Hcompress. When applied to large-scale instruments such as the VLT, this corresponds to a reduction of approximately 0.25 TB per day (4.48%), yielding annual storage savings exceeding 90 TB. The implementation is publicly available at https://github.com/xavifeme00/Astronomy-Compressor.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Robustness of Lossy Multispectral Compression to Simulated Instrumental Noise: A Comparative Study

  • González de Regàs, Jordi
  • Mijares, Sebastiá
  • Bartrina, Joan
  • Serra-Sagristà, Joan
Lossy compression of multispectral data has been widely adopted to tackle the downlink bottleneck in space-based Earth observation. However, compression artifacts may interact with the instrumental noise inherent to the sensors, affecting both the performance of the compressors and the effectiveness of denoising techniques. This letter evaluates the effects of lossy compression in the presence of instrumental noise and the corresponding denoising procedures on multispectral data, comparing current techniques for on-board lossy compression (CCSDS 122.0-B-1, near-lossless CCSDS 123.0-B-2, JPEG 2000, and exogenous KLT-JPEG 2000) with recent reduced-complexity models based on neural networks (mijares, chabert, oberlin and serra (MCOS) and spectral orthogonal transform encoder lossY (SORTENY)]. The rate-distortion performance of each codec is analyzed across four simulated instrumental noise conditions: thermal noise, dark signal, missing lines, and parallax-induced band misalignment. Experimental results show that all the evaluated codecs are resilient under thermal noise, dark signal, and missing line artifacts. However, for band misalignment artifacts, the compressors using a linear spectral transform, as E-KLT-JPEG 2000 and SORTENY, experienced a significant degradation., This work was supported in part by Spanish Ministry of Science and Innovation (MICINN), and European Regional Development Fund (FEDER), funded by MCIN/AEI/10.13039/501100011033/FEDER, UE, under Grant PID2021-125258OB-I00 and Grant PID2024-156292OB-I00; in part by Catalan Government under Grant SGR2021-00643; in part by the Talent and Society Programme, Institute for Space Studies of Catalonia (IEEC) under Grant IEEC/60/2024; and in part by IEEC and Catalan Government through the Project “Data Compression and Machine Learning for Earth Observation Satellites,” within the framework of the New Space Strategy of Catalonia 2024 and 2025, Peer reviewed
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Accelerating BPC-PaCo through visually lossless techniques

  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
  • de Cea Dominguez, Carlos|||0000-0002-0868-6479
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
Fast image codecs are a current need in applications that deal with large amounts of images. Graphics Processing Units (GPUs) are suitable processors to speed up most kinds of algorithms, especially when they allow fine-grain parallelism. Bitplane Coding with Parallel Coefficient processing (BPC-PaCo) is a recently proposed algorithm for the core stage of wavelet-based image codecs tailored for the highly parallel architectures of GPUs. This algorithm provides complexity scalability to allow faster execution at the expense of coding efficiency. Its main drawback is that the speedup and loss in image quality is controlled only roughly, resulting in visible distortion at low and medium rates. This paper addresses this issue by integrating techniques of visually lossless coding into BPC-PaCo. The resulting method minimizes the visual distortion introduced in the compressed file, obtaining higher-quality images to a human observer. Experimental results also indicate 12% speedups with respect to BPC-PaCo.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Probability models for highly parallel image coding architecture

  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
A key aspect of image coding systems is the probability model employed to code the data. The more precise the probability estimates inferred by the model, the higher the coding efficiency achieved. In general, probability models adjust the estimates after coding every new symbol. The main difficulty to apply such a strategy to a highly parallel coding engine is that many symbols are coded simultaneously, so the probability adaptation requires a different approach. The strategy employed in previous works utilizes stationary estimates collected a priori from a training set. Its main drawback is that statistics are dependent of the image type, so different images require different training sets. This work introduces two probability models for a highly parallel architecture that, similarly to conventional systems, adapt probabilities while coding data. One of the proposed models estimates probabilities through a finite state machine, while the other employs the statistics of already coded symbols via a sliding window. Experimental results indicate that the latter approach improves the performance achieved by the other models, including that of JPEG2000 and High Throughput JPEG2000, at medium and high rates with only a slight increase in computational complexity.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Reducing data dependencies in the feedback loop of the CCSDS 123.0-B-2 predictor

  • Sánchez, Antonio José|||0000-0002-2142-7885
  • Blanes, Ian|||0000-0001-8939-1666
  • Barrios, Yubal|||0000-0001-6186-9971
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
  • Sarmiento, Roberto|||0000-0002-4843-0507
On-board multi- and hyperspectral instruments acquire large volumes of data that need to be processed with the limited computational and storage resources. In this context, the CCSDS 123.0-B-2 standard emerges as an interesting option to compress multi- and hyperspectral images on-board satellites, supporting both lossless and near-lossless compression with low complexity and reduced power consumption. Nonetheless, the inclusion of a feedback loop in the CCSDS 123.0-B-2 predictor to support near-lossless compression introduces significant data dependencies that hinder real-time processing, particularly due to the presence of a quantization stage within this loop. This work provides an analysis of the aforementioned data dependencies and proposes two strategies aiming at maximizing throughput in hardware implementations and thus enabling real-time processing. In particular, through an elaborate mathematical derivation, the quantization stage is removed completely from the feedback loop. This reduces the critical path, which allows for shorter initiation intervals in a pipelined hardware implementation and higher throughput. This is achieved without any impact in the compression performance, which is identical to the one obtained by the original data flow of the predictor.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Fast and Efficient Entropy Coding Architectures for Massive Data Compression

  • Aulí-Llinàs, Francesc|||0000-0002-3208-9957
The compression of data is fundamental to alleviating the costs of transmitting and storing massive datasets employed in myriad fields of our society. Most compression systems employ an entropy coder in their coding pipeline to remove the redundancy of coded symbols. The entropy-coding stage needs to be efficient, to yield high compression ratios, and fast, to process large amounts of data rapidly. Despite their widespread use, entropy coders are commonly assessed for some particular scenario or coding system. This work provides a general framework to assess and optimize different entropy coders. First, the paper describes three main families of entropy coders, namely those based on variable-to-variable length codes (V2VLC), arithmetic coding (AC), and tabled asymmetric numeral systems (tANS). Then, a low-complexity architecture for the most representative coder(s) of each family is presented-more precisely, a general version of V2VLC, the MQ, M, and a fixed-length version of AC and two different implementations of tANS. These coders are evaluated under different coding conditions in terms of compression efficiency and computational throughput. The results obtained suggest that V2VLC and tANS achieve the highest compression ratios for most coding rates and that the AC coder that uses fixed-length codewords attains the highest throughput. The experimental evaluation discloses the advantages and shortcomings of each entropy-coding scheme, providing insights that may help to select this stage in forthcoming compression systems.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Efficient Lossless Compression of Integer Astronomical Data

  • Maireles González, Òscar|||0000-0003-4754-0373
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
Each new generation of telescope produces increasingly larger astronomical data volumes, which are expected to reach the order of exabytes in the next decade. Effective and fast data compression methods are paramount to help the scientific community contain storage costs and improve transmission times. Astronomical data differs significantly from natural and Earth-observation images, asking for specifically tailored compression approaches. This paper presents a novel lossless compression technique that employs the discrete Haar wavelet transform within the JPEG 2000 standard. Its performance is compared to that of a comprehensive selection of compressors, including fpack, the most common technique in astronomical observatories, as well as other algorithms highly competitive for other types of data. Experiments are performed on a large data set of 16 bit integer images, produced by telescopes around the world and representative of a wide variety of astronomical scenarios. The proposed technique has two modes. The first mode outperforms all the other tested techniques in terms of compression performance. It surpasses the most competitive configuration of fpack by, respectively, 5.3% (about 0.3 bits per sample), having also 4.5% lower compression and decompression times. The second mode is the fastest among all tested techniques. Its compression and decompression times are 2.5 and 3.5 times faster than the fastest configuration of fpack, while also yielding a 2.4% better compression performance (0.15 bits per sample).
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Prediction-based coding with rate control for lossless region of interest in pathology imaging

  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Sánchez Silva, Víctor Francisco|||0000-0002-7089-7031
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
  • Jamshidi, Pouya
  • Castellani, J.
Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Fast Run-Length Compression of Point Cloud Geometry

  • Tzamarias, Dion E. O.|||0000-0001-5111-0830
  • Chow, Kevin|||0000-0001-9693-9677
  • Blanes, Ian|||0000-0001-8939-1666
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
The increase in popularity of point-cloud-oriented applications has triggered the development of specialized compression algorithms. In this paper, a novel algorithm is developed for the lossless geometry compression of voxelized point clouds following an intra-frame design. The encoded voxels are arranged into runs and are encoded through a single-pass application directly on the voxel domain. This is done without representing the point cloud via an octree nor rendering the voxel space through an occupancy matrix, therefore decreasing the memory requirements of the method. Each run is compressed using a context-adaptive arithmetic encoder yielding state-of-the-art compression results, with gains of up to 15% over TMC13, MPEG's standard for point cloud geometry compression. Several proposed contributions accelerate the calculations of each run's probability limits prior to arithmetic encoding. As a result, the encoder attains a low computational complexity described by a linear relation to the number of occupied voxels leading to an average speedup of 1.8 over TMC13 in encoding speeds. Various experiments are conducted assessing the proposed algorithm's state-of-the-art performance in terms of compression ratio and encoding speeds.
Dipòsit Digital de Documents de la UAB
Publicación Artículo científico (JournalArticle)

Removing Data Dependencies in the CCSDS 123.0-B-2 Predictor Weight Updating

  • Barrios, Yubal|||0000-0001-6186-9971
  • Bartrina-Rapesta, Joan|||0000-0002-1551-3680
  • Hernández-Cabronero, Miguel|||0000-0001-9301-4337
  • Sánchez, Antonio José|||0000-0002-2142-7885
  • Blanes, Ian|||0000-0001-8939-1666
  • Serra-Sagristà, Joan|||0000-0003-4729-9292
  • Sarmiento, Roberto|||0000-0002-4843-0507
The Consultative Committee for Space Data Systems (CCSDS) first standardized near-lossless coding capabilities in the CCSDS 123.0-B-2 algorithm. However, this standard does not describe strategies to produce high-throughput hardware implementations, which are not trivial to derive from its definition. At the same time, throughput optimizations without significant compression performance penalties are paramount to enable real-time compression on-board next-generation satellites. This work demonstrates that the weight update stage of the CCSDS 123.0-B-2 predictor can be selectively bypassed to enhance throughput for both lossless and near-lossless modes with minimal impact on compression performance and still produce fully compliant bitstreams. Skipping the weight update implies that those weights must be carefully chosen outside the original CCSDS 123.0-B-2 pipeline. Two strategies are proposed to select effective weight values based on whether a priori information about the current image is exploited or not. Comprehensive experimental results are presented for both proposed strategies and for lossless and near-lossless regimes, using a representative set of hyperspectral images. The coding penalty is, on average, 1% for lossless and 8% for near-lossless, depending on the strategy used to set the initial weights. The proposed method obtains a maximum throughput of one processed sample per clock cycle when it is evaluated using high-level synthesis (HLS), consuming 4.6% of the look-up tables (LUTs) and 31.1% of the internal memory on a Xilinx Kintex UltraScale space-grade field programmable gate array (FPGA).