EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction mapping

Hui Sun iD, Yingfeng Zheng, Haonan Xie, Huidong Ma, Cheng Zhong, Meng Yan iD, Xiaoguang Liu, Gang Wang

Bioinformatics · 2024

DOI: 10.1093/bioinformatics/btae323 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, 2024. Verfasst von Hui Sun und 7 weiteren Autor:innen. Beteiligte Einrichtung: Nankai-Baidu Joint Laboratory, Parallel and Distributed Software Technology Laboratory, TMCC, SysNet, DISSec, GTIISC, College of Computer Science, Na…. Hauptthemen laut MEDLINE: Algorithms, Data Compression. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation The quality scores data (QSD) account for 70% in compressed FastQ files obtained from the short and long reads sequencing technologies. Designing effective compressors for QSD that counterbalance compression ratio, time cost, and memory consumption is essential in scenarios such as large-scale genomics data sharing and long-term data backup. This study presents a novel parallel lossless QSD-dedicated compression algorithm named PQSDC, which fulfills the above requirements well. PQSDC is based on two core components: a parallel sequences-partition model designed to reduce peak memory consumption and time cost during compression and decompression processes, as well as a parallel four-level run-length prediction mapping model to enhance compression ratio. Besides, the PQSDC algorithm is also designed to be highly concurrent using multicore CPU clusters. Results We evaluate PQSDC and four state-of-the-art compression algorithms on 27 real-world datasets, including 61.857 billion QSD characters and 632.908 million QSD sequences. (1) For short reads, compared to baselines, the maximum improvement of PQSDC reaches 7.06% in average compression ratio, and 8.01% in weighted average compression ratio. During compression and decompression, the maximum total time savings of PQSDC are 79.96% and 84.56%, respectively; the maximum average memory savings are 68.34% and 77.63%, respectively. (2) For long reads, the maximum improvement of PQSDC reaches 12.51% and 13.42% in average and weighted average compression ratio, respectively. The maximum total time savings during compression and decompression are 53.51% and 72.53%, respectively; the maximum average memory savings are 19.44% and 17.42%, respectively. (3) Furthermore, PQSDC ranks second in compression robustness among the tested algorithms, indicating that it is less affected by the probability distribution of the QSD collections. Overall, our work provides a promising solution for QSD parallel compression, which balances storage cost, time consumption, and memory occupation primely. Availability and implementation The proposed PQSDC compressor can be downloaded from https://github.com/fahaihi/PQSDC.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsData CompressionGenomicsHigh-Throughput Nucleotide SequencingSequence Analysis, DNASoftwareHumans

Autor:innen und Institutionen

  • Nankai-Baidu Joint Laboratory, Parallel and Distributed Software Technology Laboratory, TMCC, SysNet, DISSec, GTIISC, College of Computer Science, Nankai University, Tianjin 300350, China.
  • Institute of Artificial Intelligence, School of Electrical Engineering, Guangxi University, Nanning 530004, China.
  • Key Laboratory of Parallel, Distributed and Intelligent of Guangxi Universities and Colleges, School of Computer, Electronics and Information, Guangxi University, Nanning 530004, China.

ORCID-Profile: Hui Sun, Meng Yan

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Sun, H., Zheng, Y., Xie, H., Ma, H., Zhong, C., Yan, M., Liu, X., & Wang, G. (2024). PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction mapping. Bioinformatics. https://doi.org/10.1093/bioinformatics/btae323

Vancouver

Sun H, Zheng Y, Xie H, Ma H, Zhong C, Yan M, et al. PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction mapping. Bioinformatics. 2024. doi:10.1093/bioinformatics/btae323

BibTeX
@article{Sun2024,
  author = {Sun, Hui and Zheng, Yingfeng and Xie, Haonan and Ma, Huidong and Zhong, Cheng and Yan, Meng and Liu, Xiaoguang and Wang, Gang},
  title = {PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction mapping},
  journal = {Bioinformatics},
  year = {2024},
  doi = {10.1093/bioinformatics/btae323}
}
Kostenlos starten →