Improving the Restore Performance of Fine-Grained Deduplication on Docker Container Image Storage
IEEE Transactions on Parallel and Distributed Systems · 2026-06 · 37 (6) · 1247-1264
DOI: 10.1109/tpds.2026.3678624 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Parallel and Distributed Systems, 2026-06. Verfasst von Haoliang Tan und 6 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Gefördert von: Major Key Project of PCL, National Natural Science Foundation of China, Basic and Applied Basic Research Foundation of Guangdong Province.
Abstract (Originalsprache)
The rapid growth of container images leads to heavy storage pressures on image registries. Fine-grained deduplication, such as at the file and chunk levels, is a promising technique for reducing the storage space in image registries, compared to Docker’s native coarse-grained image layer deduplication. However, fine-grained deduplication often incurs significant image restoration latency due to fragmented I/Os, resulting in up to 8× restoration I/O slowdowns. Moreover, existing restore-optimized deduplication techniques are not tailored to the characteristics of the container image, resulting in low deduplication ratios of images. Consequently, restoring performance remains a critical barrier to the practical adoption of fine-grained deduplication in container image registries. To address this challenge, we propose MiDedup, a restore-friendly fine-grained deduplication approach tailored for container image registries. MiDedup is built on an underexplored observation: the redundancy across layers of images follows a non-uniform distribution. Based on this insight, MiDedup introduces three core techniques: ① Across-layer-aware reorganization, which reorganizes deduplicated data into a compact, sequential layout, significantly reduces fragmented I/Os during image restore. ② Popularity-aware rewriting, which selectively rewrites hot image layers to further improve the balance between deduplication ratio and restore performance. ③ Hybrid-granularity deduplication, which combines file-level and chunk-level deduplication to reduce metadata and fragmented data. Experiments on the real-world Docker image dataset show that MiDedup reduces the 50–80% restore I/O overhead of fine-grained deduplication, which is comparable to the restore performance of native layer-level deduplication, while achieving 1.2–7.3× higher deduplication ratios.
Förderung
- Major Key Project of PCL
- National Natural Science Foundation of China
- Basic and Applied Basic Research Foundation of Guangdong Province
- Shenzhen Science and Technology Program
Zitieren
APA
Tan, H., Ou, W., Zou, X., Qin, L., Deng, C., Gu, Z., & Xia, W. (2026). Improving the Restore Performance of Fine-Grained Deduplication on Docker Container Image Storage. IEEE Transactions on Parallel and Distributed Systems, 37(6), 1247-1264. https://doi.org/10.1109/tpds.2026.3678624
Vancouver
Tan H, Ou W, Zou X, Qin L, Deng C, Gu Z, et al. Improving the Restore Performance of Fine-Grained Deduplication on Docker Container Image Storage. IEEE Transactions on Parallel and Distributed Systems. 2026;37(6):1247-1264. doi:10.1109/tpds.2026.3678624
BibTeX
@article{Tan2026,
author = {Tan, Haoliang and Ou, Wenhao and Zou, Xiangyu and Qin, Lisha and Deng, Cai and Gu, Zhaoquan and Xia, Wen},
title = {Improving the Restore Performance of Fine-Grained Deduplication on Docker Container Image Storage},
journal = {IEEE Transactions on Parallel and Distributed Systems},
year = {2026},
volume = {37},
number = {6},
pages = {1247--1264},
doi = {10.1109/tpds.2026.3678624}
}
EUVIMED