AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models
Proceedings of the ACM on Management of Data · 10.02.2025 · 3 (1) · 1-26
DOI: 10.1145/3709661 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 11.02.2025. Verfasst von Runjie Yu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Higher Education Discipline Innovation Project.
Abstract (Originalsprache)
The knowledge retrieval methods such as Approximate Nearest Neighbor Search (ANNS) significantly enhance the generation quality of Large Language Models (LLMs) by introducing external knowledge, and this method is called Retrieval-augmented generation (RAG). However, due to the rapid growth of data size, ANNS tends to store large-scale data on disk, which greatly increases the response time of RAG systems. This paper presents AquaPipe, which pipelines the execution of disk-based ANNS and the LLM prefill phase in an RAG system, effectively overlapping the latency of knowledge retrieval and model inference to enhance the overall performance, while guaranteeing data quality. First, ANNS's recall-aware prefetching strategy enables the early return of partial text with acceptable accuracy so the prefill phase can launch before getting the full results. Then, we adaptively choose the remove-after-prefill or re-prefill strategies based on the LLM cost model to effectively correct disturbed pipelines caused by wrong early returns. Finally, the pipelined prefill dynamically changes the granularity of chunk size to balance the overlap efficiency and GPU efficiency, adjusting to ANNS tasks that converge at different speeds. Our experiments have demonstrated the effectiveness of AquaPipe. It successfully masks the latency of disk-based ANNS by 56% to 99%, resulting in a 1.3× to 2.6× reduction of the response time of the RAG, while the extra recall loss caused by prefetching is limited to approximately 1%.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Higher Education Discipline Innovation Project
Zitieren
APA
Yu, R., Huang, W., Bai, S., Zhou, J., & Wu, F. (2025). AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models. Proceedings of the ACM on Management of Data, 3(1), 1-26. https://doi.org/10.1145/3709661
Vancouver
Yu R, Huang W, Bai S, Zhou J, Wu F. AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models. Proceedings of the ACM on Management of Data. 2025;3(1):1-26. doi:10.1145/3709661
BibTeX
@article{Yu2025,
author = {Yu, Runjie and Huang, Weizhou and Bai, Shuhan and Zhou, Jian and Wu, Fei},
title = {AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models},
journal = {Proceedings of the ACM on Management of Data},
year = {2025},
volume = {3},
number = {1},
pages = {1--26},
doi = {10.1145/3709661}
}
EUVIMED