EUVIMEDEUVIMED entdecken
Crossref

CoTra: Towards Efficient and Scalable Distributed Vector Search with RDMA

Xiangyu Zhi, Meng Chen, Xiao Yan, Baotong Lu, Hui Li, Qianxi Zhang, Qi Chen, James Cheng

Proceedings of the ACM on Management of Data · 02.04.2026 · 4 (1) · 1-27

DOI: 10.1145/3786634 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 07.04.2026. Verfasst von Xiangyu Zhi und 7 weiteren Autor:innen. Die Arbeit nennt 68 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Similarity-based vector search facilitates many important applications such as search and recommendation but is limited by the memory capacity and bandwidth of a single machine due to large datasets and intensive data read. In this paper, we present CoTra, a system that scales up vector search for distributed execution. We observe a tension between computation and communication efficiency, which is the main challenge for good scalability, i.e., handling the local vectors on each machine independently blows up computation as the pruning power of vector index is not fully utilized, while running a global index over all machines introduces rich data dependencies and thus extensive communication. To resolve such tension, we leverage the fact that vector search is approximate in nature and robust to asynchronous execution. In particular, we run collaborative vector search over the machines with algorithm-system co-designs including clustering-based data partitioning to reduce communication, asynchronous execution to avoid communication stall, and task push to reduce network traffic. To make collaborative search efficient, we introduce a suite of system optimizations including task scheduling, communication batching, and storage format. We evaluate CoTra on real datasets and compare with four baselines. The results show that when using 16 machines, the query throughput of CoTra scales to 9.8-13.4x over a single machine and is 2.12-3.58x of the best-performing baseline at 0.95 recall@10.

Zitieren

APA

Zhi, X., Chen, M., Yan, X., Lu, B., Li, H., Zhang, Q., Chen, Q., & Cheng, J. (2026). CoTra: Towards Efficient and Scalable Distributed Vector Search with RDMA. Proceedings of the ACM on Management of Data, 4(1), 1-27. https://doi.org/10.1145/3786634

Vancouver

Zhi X, Chen M, Yan X, Lu B, Li H, Zhang Q, et al. CoTra: Towards Efficient and Scalable Distributed Vector Search with RDMA. Proceedings of the ACM on Management of Data. 2026;4(1):1-27. doi:10.1145/3786634

BibTeX
@article{Zhi2026,
  author = {Zhi, Xiangyu and Chen, Meng and Yan, Xiao and Lu, Baotong and Li, Hui and Zhang, Qianxi and Chen, Qi and Cheng, James},
  title = {CoTra: Towards Efficient and Scalable Distributed Vector Search with RDMA},
  journal = {Proceedings of the ACM on Management of Data},
  year = {2026},
  volume = {4},
  number = {1},
  pages = {1--27},
  doi = {10.1145/3786634}
}
Kostenlos starten →