EUVIMEDEUVIMED entdecken
Crossref

High-Ratio Compression for Machine-Generated Data

Jiujing Zhang, Zhitao Shen, Shiyu Yang, Lingkai Meng, Chuan Xiao, Wei Jia, Yue Li, Qinhui Sun, Wenjie Zhang, Xuemin Lin

Proceedings of the ACM on Management of Data · 08.12.2023 · 1 (4) · 1-27

DOI: 10.1145/3626732 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 12.12.2023. Verfasst von Jiujing Zhang und 9 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: JSPS Kakenhi, National Key R&D Program of China, ARC Future Fellowship.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Machine-generated data is rapidly growing and poses challenges for data-intensive systems, especially as the growth of data outpaces the growth of storage space. To cope with the storage issue, compression plays a critical role in storage engines, particularly for data-intensive applications, where a high compression ratio and efficient random access are essential. However, existing compression techniques tend to focus on general-purpose and data block approaches, but overlook the inherent structure of machine-generated data and hence result in low compression ratios or limited lookup efficiency. To address these limitations, we introduce the Pattern-Based Compression (PBC) algorithm, which specifically targets patterns in machine-generated data to achieve Pareto-optimality in most cases. Unlike traditional data block-based methods, PBC compresses data on a per-record basis, facilitating rapid random access. Our experimental evaluation demonstrates that PBC, on average, achieves a compression ratio twice as high as the state-of-the-art techniques while maintaining competitive compression and decompression speeds. We also integrate PBC to a production database system and achieve improvements on both comparison ratio and throughput.

Förderung

  • JSPS Kakenhi
  • National Key R&D Program of China
  • ARC Future Fellowship
  • NSFC
  • the scholarship of China Scholarship Council
  • CCF-AFSG Research Fund
  • ARC Discovery Project
  • CREST

Zitieren

APA

Zhang, J., Shen, Z., Yang, S., Meng, L., Xiao, C., Jia, W., Li, Y., Sun, Q., Zhang, W., & Lin, X. (2023). High-Ratio Compression for Machine-Generated Data. Proceedings of the ACM on Management of Data, 1(4), 1-27. https://doi.org/10.1145/3626732

Vancouver

Zhang J, Shen Z, Yang S, Meng L, Xiao C, Jia W, et al. High-Ratio Compression for Machine-Generated Data. Proceedings of the ACM on Management of Data. 2023;1(4):1-27. doi:10.1145/3626732

BibTeX
@article{Zhang2023,
  author = {Zhang, Jiujing and Shen, Zhitao and Yang, Shiyu and Meng, Lingkai and Xiao, Chuan and Jia, Wei and Li, Yue and Sun, Qinhui and Zhang, Wenjie and Lin, Xuemin},
  title = {High-Ratio Compression for Machine-Generated Data},
  journal = {Proceedings of the ACM on Management of Data},
  year = {2023},
  volume = {1},
  number = {4},
  pages = {1--27},
  doi = {10.1145/3626732}
}
Kostenlos starten →