StreamDedup : Distributed In-line Deduplication for Disaggregated Storage
ACM Transactions on Reconfigurable Technology and Systems · 30.06.2026 · 19 (2) · 1-26
DOI: 10.1145/3799896 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Reconfigurable Technology and Systems, online erschienen am 15.04.2026. Verfasst von Jiayong Li und 8 weiteren Autor:innen. Die Arbeit nennt 71 Literaturangaben.
Abstract (Originalsprache)
Efficient data reduction techniques, including deduplication and compression, are essential in storage systems, affecting performance and longevity. Existing data deduplication approaches often focus on intra-SSD deduplication, missing opportunities for cross-node deduplication, or have scalability issues when aiming for low latency and high-throughput data reduction on large-scale, distributed SSD arrays. We propose StreamDedup, a distributed stream accelerator implementing a transparent layer of deduplication as a network-attached, middle-tier service between the compute and storage tiers. StreamDedup manages all aspects of data deduplication and compression and can be seamlessly integrated into existing systems. It is RDMA-enabled and highly scalable, enhancing data processing capacities for large-scale storage systems. Our prototype, deployed on FPGAs, demonstrates that StreamDedup achieves a throughput of 12.7 GB/s on a single node, matching the network bandwidth of disaggregated storage, with a latency of less than 50 µs. Across 10 nodes, StreamDedup shows an almost linear increase in throughput with less than 60 µs of latency.
Zitieren
APA
Li, J., Dann, J., He, Z., Alonso, G., Chalamalasetti, S. R., Milojicic, D., Evans, L., Veprinsky, A., & Shi, R. (2026). StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems, 19(2), 1-26. https://doi.org/10.1145/3799896
Vancouver
Li J, Dann J, He Z, Alonso G, Chalamalasetti SR, Milojicic D, et al. StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems. 2026;19(2):1-26. doi:10.1145/3799896
BibTeX
@article{Li2026,
author = {Li, Jiayong and Dann, Jonas and He, Zhenhao and Alonso, Gustavo and Chalamalasetti, Sai Rahul and Milojicic, Dejan and Evans, Lance and Veprinsky, Alex and Shi, Runbin},
title = {StreamDedup : Distributed In-line Deduplication for Disaggregated Storage},
journal = {ACM Transactions on Reconfigurable Technology and Systems},
year = {2026},
volume = {19},
number = {2},
pages = {1--26},
doi = {10.1145/3799896}
}
EUVIMED