EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hierarchical Autoencoder-Based Lossy Compression for Large-Scale High-Resolution Scientific Data

Hieu Le, Jian Tao

Computing&AI Connect · 2024

DOI: 10.69709/caic.2024.193132 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computing&AI Connect, 2024. Verfasst von Hieu Le und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Lossy compression has become an essential technique to reduce data size in many domains. This type of compression is especially valuable for large-scale scientific data, whose size ranges up to several petabytes. Although Autoencoder-based models have been successfully leveraged to compress images and videos, such neural networks have not widely gained attention in the scientific data domain. Our work presents a neural network that not only significantly compresses large-scale scientific data, but also maintains high reconstruction quality. The proposed model is tested with scientific benchmark data available publicly and applied to a large-scale high-resolution climate modeling data set. Our model achieves a compression ratio of 140 on several benchmark data sets without compromising the reconstruction quality. 2D simulation data from the High-Resolution Community Earth System Model (CESM) Version 1.3 over 500 years are also being compressed with a compression ratio of 200 while the reconstruction error is negligible for scientific analysis.

Zitieren

APA

Le, H., & Tao, J. (2024). Hierarchical Autoencoder-Based Lossy Compression for Large-Scale High-Resolution Scientific Data. Computing&AI Connect. https://doi.org/10.69709/caic.2024.193132

Vancouver

Le H, Tao J. Hierarchical Autoencoder-Based Lossy Compression for Large-Scale High-Resolution Scientific Data. Computing&AI Connect. 2024. doi:10.69709/caic.2024.193132

BibTeX
@article{Le2024,
  author = {Le, Hieu and Tao, Jian},
  title = {Hierarchical Autoencoder-Based Lossy Compression for Large-Scale High-Resolution Scientific Data},
  journal = {Computing&AI Connect},
  year = {2024},
  doi = {10.69709/caic.2024.193132}
}
Kostenlos starten →