EUVIMEDOpdag EUVIMED
Crossref

StreamDedup : Distributed In-line Deduplication for Disaggregated Storage

Jiayong Li, Jonas Dann, Zhenhao He, Gustavo Alonso, Sai Rahul Chalamalasetti, Dejan Milojicic, Lance Evans, Alex Veprinsky, Runbin Shi

ACM Transactions on Reconfigurable Technology and Systems · 2026-06-30 · 19 (2) · 1-26

DOI: 10.1145/3799896 ↗

At a glance

Journal article in ACM Transactions on Reconfigurable Technology and Systems, published online on 2026-04-15. Written by Jiayong Li and 8 co-authors. 71 references.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Efficient data reduction techniques, including deduplication and compression, are essential in storage systems, affecting performance and longevity. Existing data deduplication approaches often focus on intra-SSD deduplication, missing opportunities for cross-node deduplication, or have scalability issues when aiming for low latency and high-throughput data reduction on large-scale, distributed SSD arrays. We propose StreamDedup, a distributed stream accelerator implementing a transparent layer of deduplication as a network-attached, middle-tier service between the compute and storage tiers. StreamDedup manages all aspects of data deduplication and compression and can be seamlessly integrated into existing systems. It is RDMA-enabled and highly scalable, enhancing data processing capacities for large-scale storage systems. Our prototype, deployed on FPGAs, demonstrates that StreamDedup achieves a throughput of 12.7 GB/s on a single node, matching the network bandwidth of disaggregated storage, with a latency of less than 50 µs. Across 10 nodes, StreamDedup shows an almost linear increase in throughput with less than 60 µs of latency.

Cite

APA

Li, J., Dann, J., He, Z., Alonso, G., Chalamalasetti, S. R., Milojicic, D., Evans, L., Veprinsky, A., & Shi, R. (2026). StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems, 19(2), 1-26. https://doi.org/10.1145/3799896

Vancouver

Li J, Dann J, He Z, Alonso G, Chalamalasetti SR, Milojicic D, et al. StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems. 2026;19(2):1-26. doi:10.1145/3799896

BibTeX
@article{Li2026,
  author = {Li, Jiayong and Dann, Jonas and He, Zhenhao and Alonso, Gustavo and Chalamalasetti, Sai Rahul and Milojicic, Dejan and Evans, Lance and Veprinsky, Alex and Shi, Runbin},
  title = {StreamDedup : Distributed In-line Deduplication for Disaggregated Storage},
  journal = {ACM Transactions on Reconfigurable Technology and Systems},
  year = {2026},
  volume = {19},
  number = {2},
  pages = {1--26},
  doi = {10.1145/3799896}
}
Start for free →