EUVIMEDEUVIMED entdecken
Crossref

StreamDedup : Distributed In-line Deduplication for Disaggregated Storage

Jiayong Li, Jonas Dann, Zhenhao He, Gustavo Alonso, Sai Rahul Chalamalasetti, Dejan Milojicic, Lance Evans, Alex Veprinsky, Runbin Shi

ACM Transactions on Reconfigurable Technology and Systems · 30.06.2026 · 19 (2) · 1-26

DOI: 10.1145/3799896 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Reconfigurable Technology and Systems, online erschienen am 15.04.2026. Verfasst von Jiayong Li und 8 weiteren Autor:innen. Die Arbeit nennt 71 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Efficient data reduction techniques, including deduplication and compression, are essential in storage systems, affecting performance and longevity. Existing data deduplication approaches often focus on intra-SSD deduplication, missing opportunities for cross-node deduplication, or have scalability issues when aiming for low latency and high-throughput data reduction on large-scale, distributed SSD arrays. We propose StreamDedup, a distributed stream accelerator implementing a transparent layer of deduplication as a network-attached, middle-tier service between the compute and storage tiers. StreamDedup manages all aspects of data deduplication and compression and can be seamlessly integrated into existing systems. It is RDMA-enabled and highly scalable, enhancing data processing capacities for large-scale storage systems. Our prototype, deployed on FPGAs, demonstrates that StreamDedup achieves a throughput of 12.7 GB/s on a single node, matching the network bandwidth of disaggregated storage, with a latency of less than 50 µs. Across 10 nodes, StreamDedup shows an almost linear increase in throughput with less than 60 µs of latency.

Zitieren

APA

Li, J., Dann, J., He, Z., Alonso, G., Chalamalasetti, S. R., Milojicic, D., Evans, L., Veprinsky, A., & Shi, R. (2026). StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems, 19(2), 1-26. https://doi.org/10.1145/3799896

Vancouver

Li J, Dann J, He Z, Alonso G, Chalamalasetti SR, Milojicic D, et al. StreamDedup : Distributed In-line Deduplication for Disaggregated Storage. ACM Transactions on Reconfigurable Technology and Systems. 2026;19(2):1-26. doi:10.1145/3799896

BibTeX
@article{Li2026,
  author = {Li, Jiayong and Dann, Jonas and He, Zhenhao and Alonso, Gustavo and Chalamalasetti, Sai Rahul and Milojicic, Dejan and Evans, Lance and Veprinsky, Alex and Shi, Runbin},
  title = {StreamDedup : Distributed In-line Deduplication for Disaggregated Storage},
  journal = {ACM Transactions on Reconfigurable Technology and Systems},
  year = {2026},
  volume = {19},
  number = {2},
  pages = {1--26},
  doi = {10.1145/3799896}
}
Kostenlos starten →