A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search
ACM Transactions on Storage · 31.08.2026 · 22 (3) · 1-26
DOI: 10.1145/3736589 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Storage, online erschienen am 04.08.2026. Verfasst von Haikun Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Natural Science Foundation of Hubei Province.
Abstract (Originalsprache)
Approximate nearest neighbor search (ANNS) in high-dimensional vector spaces has become increasingly crucial in database and machine learning applications. Most previous ANNS algorithms require TB-scale memory to store indices of billion-scale datasets, making their deployment extremely expensive for high-performance ANNS services. The emerging SmartSSD technology offers an opportunity to achieve scalable ANNS via near data processing (NDP). However, there remain several challenges to directly adopt existing ANNS algorithms on multiple SmartSSDs. In this article, we present SmartANNS, a SmartSSD-empowered billion-scale ANNS solution based on a hierarchical indexing methodology. We propose several novel designs to achieve near-linear scaling with multiple SmartSSDs. First, we propose a “host CPUs + SmartSSDs” cooperative architecture incorporated with hierarchical indices to significantly reduce data accesses and computations on SmartSSDs. Second, we propose dynamic task scheduling based on optimized data layout to achieve both load balancing and data reusing for multiple SmartSSDs. Third, we further propose a learning-based shard pruning algorithm to eliminate unnecessary computations on SmartSSDs. We implement SmartANNS using Samsung’s commercial SmartSSDs. Experimental results show that SmartANNS can improve query per second (QPS) by up to 10.7× compared with the state-of-the-art SmartSSD-based ANNS solution–CSDANNS. Moreover, SmartANNS can achieve near-linear performance scalability for large-scale datasets using multiple SmartSSDs.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Natural Science Foundation of Hubei Province
Zitieren
APA
Liu, H., Tian, B., Duan, Z., Liao, X., & Zhang, Y. (2026). A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search. ACM Transactions on Storage, 22(3), 1-26. https://doi.org/10.1145/3736589
Vancouver
Liu H, Tian B, Duan Z, Liao X, Zhang Y. A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search. ACM Transactions on Storage. 2026;22(3):1-26. doi:10.1145/3736589
BibTeX
@article{Liu2026,
author = {Liu, Haikun and Tian, Bing and Duan, Zhuohui and Liao, Xiaofei and Zhang, Yu},
title = {A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search},
journal = {ACM Transactions on Storage},
year = {2026},
volume = {22},
number = {3},
pages = {1--26},
doi = {10.1145/3736589}
}
EUVIMED