EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search

Haikun Liu, Bing Tian, Zhuohui Duan, Xiaofei Liao, Yu Zhang

ACM Transactions on Storage · 31.08.2026 · 22 (3) · 1-26

DOI: 10.1145/3736589 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Storage, online erschienen am 04.08.2026. Verfasst von Haikun Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Natural Science Foundation of Hubei Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Approximate nearest neighbor search (ANNS) in high-dimensional vector spaces has become increasingly crucial in database and machine learning applications. Most previous ANNS algorithms require TB-scale memory to store indices of billion-scale datasets, making their deployment extremely expensive for high-performance ANNS services. The emerging SmartSSD technology offers an opportunity to achieve scalable ANNS via near data processing (NDP). However, there remain several challenges to directly adopt existing ANNS algorithms on multiple SmartSSDs. In this article, we present SmartANNS, a SmartSSD-empowered billion-scale ANNS solution based on a hierarchical indexing methodology. We propose several novel designs to achieve near-linear scaling with multiple SmartSSDs. First, we propose a “host CPUs + SmartSSDs” cooperative architecture incorporated with hierarchical indices to significantly reduce data accesses and computations on SmartSSDs. Second, we propose dynamic task scheduling based on optimized data layout to achieve both load balancing and data reusing for multiple SmartSSDs. Third, we further propose a learning-based shard pruning algorithm to eliminate unnecessary computations on SmartSSDs. We implement SmartANNS using Samsung’s commercial SmartSSDs. Experimental results show that SmartANNS can improve query per second (QPS) by up to 10.7× compared with the state-of-the-art SmartSSD-based ANNS solution–CSDANNS. Moreover, SmartANNS can achieve near-linear performance scalability for large-scale datasets using multiple SmartSSDs.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Natural Science Foundation of Hubei Province

Zitieren

APA

Liu, H., Tian, B., Duan, Z., Liao, X., & Zhang, Y. (2026). A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search. ACM Transactions on Storage, 22(3), 1-26. https://doi.org/10.1145/3736589

Vancouver

Liu H, Tian B, Duan Z, Liao X, Zhang Y. A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search. ACM Transactions on Storage. 2026;22(3):1-26. doi:10.1145/3736589

BibTeX
@article{Liu2026,
  author = {Liu, Haikun and Tian, Bing and Duan, Zhuohui and Liao, Xiaofei and Zhang, Yu},
  title = {A SmartSSD-based Near Data Processing Architecture for Scalable Billion-point Approximate Nearest Neighbor Search},
  journal = {ACM Transactions on Storage},
  year = {2026},
  volume = {22},
  number = {3},
  pages = {1--26},
  doi = {10.1145/3736589}
}
Kostenlos starten →