HAKES : Scalable Vector Database for Embedding Search Service
Proceedings of the VLDB Endowment · 2025-05 · 18 (9) · 3049-3062
DOI: 10.14778/3746405.3746427 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 03.09.2025. Verfasst von Guoyu Hu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 64 Literaturangaben.
Abstract (Originalsprache)
Modern deep learning models capture the semantics of complex data by transforming them into high-dimensional embedding vectors. Emerging applications, such as retrieval-augmented generation, use approximate nearest neighbor (ANN) search in the embedding vector space to find similar data. Existing vector databases provide indexes for efficient ANN searches, with graph-based indexes being the most popular due to their low latency and high recall in real-world high-dimensional datasets. However, these indexes are costly to build, suffer from significant contention under concurrent read-write workloads, and scale poorly to multiple servers. Our goal is to build a vector database that achieves high throughput and high recall under concurrent read-write workloads. To this end, we first propose an ANN index with an explicit two-stage design combining a fast filter stage with highly compressed vectors and a refine stage to ensure recall, and we devise a novel lightweight machine learning technique to fine-tune the index parameters. We introduce an early termination check to dynamically adapt the search process for each query. Next, we add support for writes while maintaining search performance by decoupling the management of the learned parameters. Finally, we design HAKES, a distributed vector database that serves the new index in a disaggregated architecture. We evaluate our index and system against 12 state-of-the-art indexes and three distributed vector databases, using high-dimensional embedding datasets generated by deep learning models. The experimental results show that our index outperforms index baselines in the high recall region and under concurrent read-write workloads. Furthermore, HAKES is scalable and achieves up to 16x higher throughputs than the baselines.
Zitieren
APA
Hu, G., Cai, S., Dinh, T. T. A., Xie, Z., Yue, C., Chen, G., & Ooi, B. C. (2025). HAKES : Scalable Vector Database for Embedding Search Service. Proceedings of the VLDB Endowment, 18(9), 3049-3062. https://doi.org/10.14778/3746405.3746427
Vancouver
Hu G, Cai S, Dinh TTA, Xie Z, Yue C, Chen G, et al. HAKES : Scalable Vector Database for Embedding Search Service. Proceedings of the VLDB Endowment. 2025;18(9):3049-3062. doi:10.14778/3746405.3746427
BibTeX
@article{Hu2025,
author = {Hu, Guoyu and Cai, Shaofeng and Dinh, Tien Tuan Anh and Xie, Zhongle and Yue, Cong and Chen, Gang and Ooi, Beng Chin},
title = {HAKES : Scalable Vector Database for Embedding Search Service},
journal = {Proceedings of the VLDB Endowment},
year = {2025},
volume = {18},
number = {9},
pages = {3049--3062},
doi = {10.14778/3746405.3746427}
}
EUVIMED