Shard: A Scalable and Resize-Optimized Hash Index on Disaggregated Memory
Proceedings of the VLDB Endowment · 01.12.2025 · 19 (4) · 684-697
DOI: 10.14778/3785297.3785309 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 18.03.2026. Verfasst von Hantian Zha und 9 weiteren Autor:innen. Die Arbeit nennt 84 Literaturangaben.
Abstract (Originalsprache)
Disaggregated memory (DM) separates memory and computing resources into distinct pools, improving resource utilization, scalability, and data sharing in data centers and cloud environments. These systems utilize RDMA-capable networks, which provide high throughput and low latency, making them well-suited for high-performance indexing in data storage systems. However, existing DM-optimized hash indexes face significant challenges in achieving the one RTT goal due to excessive remote read/write accesses, correctness issues in concurrent operations, high latency during resizing, and costly multi-node synchronization. This paper addresses these challenges by introducing a novel architecture called Shard, designed to enhance the performance of hash indexes in disaggregated memory. We leverage the structure of Iceberg Hashing to ensure that each key is mapped to fewer buckets. We propose the Ordered-CAS technique to minimize read/write accesses and ensure correctness when handling duplicate keys. To address the trade-offs between resizing and synchronization, we adopt a lazy resizing strategy and propose the RDMA-combining and adaptive frequency synchronization (AFS) techniques. We implement Shard and conduct a comprehensive evaluation on DM. The results show that Shard outperforms state-of-the-art DM-optimized hash indexes by at most 6.7X (RACE), 3.6X (SepHash), and 1.8X (Outback) in YCSB workloads, respectively.
Zitieren
APA
Zha, H., Ma, T., Lu, B., Wang, Y., He, D., Luo, Y., Zhang, D., Chai, Y., Chen, Y., & Pan, A. (2025). Shard: A Scalable and Resize-Optimized Hash Index on Disaggregated Memory. Proceedings of the VLDB Endowment, 19(4), 684-697. https://doi.org/10.14778/3785297.3785309
Vancouver
Zha H, Ma T, Lu B, Wang Y, He D, Luo Y, et al. Shard: A Scalable and Resize-Optimized Hash Index on Disaggregated Memory. Proceedings of the VLDB Endowment. 2025;19(4):684-697. doi:10.14778/3785297.3785309
BibTeX
@article{Zha2025,
author = {Zha, Hantian and Ma, Teng and Lu, Baotong and Wang, Yuansen and He, Dongbiao and Luo, Yuanhui and Zhang, Dafang and Chai, Yunpeng and Chen, Yuxing and Pan, Anqun},
title = {Shard: A Scalable and Resize-Optimized Hash Index on Disaggregated Memory},
journal = {Proceedings of the VLDB Endowment},
year = {2025},
volume = {19},
number = {4},
pages = {684--697},
doi = {10.14778/3785297.3785309}
}
EUVIMED