EUVIMEDEUVIMED entdecken
Crossref

AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models

Runjie Yu, Weizhou Huang, Shuhan Bai, Jian Zhou, Fei Wu

Proceedings of the ACM on Management of Data · 10.02.2025 · 3 (1) · 1-26

DOI: 10.1145/3709661 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 11.02.2025. Verfasst von Runjie Yu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Higher Education Discipline Innovation Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The knowledge retrieval methods such as Approximate Nearest Neighbor Search (ANNS) significantly enhance the generation quality of Large Language Models (LLMs) by introducing external knowledge, and this method is called Retrieval-augmented generation (RAG). However, due to the rapid growth of data size, ANNS tends to store large-scale data on disk, which greatly increases the response time of RAG systems. This paper presents AquaPipe, which pipelines the execution of disk-based ANNS and the LLM prefill phase in an RAG system, effectively overlapping the latency of knowledge retrieval and model inference to enhance the overall performance, while guaranteeing data quality. First, ANNS's recall-aware prefetching strategy enables the early return of partial text with acceptable accuracy so the prefill phase can launch before getting the full results. Then, we adaptively choose the remove-after-prefill or re-prefill strategies based on the LLM cost model to effectively correct disturbed pipelines caused by wrong early returns. Finally, the pipelined prefill dynamically changes the granularity of chunk size to balance the overlap efficiency and GPU efficiency, adjusting to ANNS tasks that converge at different speeds. Our experiments have demonstrated the effectiveness of AquaPipe. It successfully masks the latency of disk-based ANNS by 56% to 99%, resulting in a 1.3× to 2.6× reduction of the response time of the RAG, while the extra recall loss caused by prefetching is limited to approximately 1%.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Higher Education Discipline Innovation Project

Zitieren

APA

Yu, R., Huang, W., Bai, S., Zhou, J., & Wu, F. (2025). AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models. Proceedings of the ACM on Management of Data, 3(1), 1-26. https://doi.org/10.1145/3709661

Vancouver

Yu R, Huang W, Bai S, Zhou J, Wu F. AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models. Proceedings of the ACM on Management of Data. 2025;3(1):1-26. doi:10.1145/3709661

BibTeX
@article{Yu2025,
  author = {Yu, Runjie and Huang, Weizhou and Bai, Shuhan and Zhou, Jian and Wu, Fei},
  title = {AquaPipe: A Quality-Aware Pipeline for Knowledge Retrieval and Large Language Models},
  journal = {Proceedings of the ACM on Management of Data},
  year = {2025},
  volume = {3},
  number = {1},
  pages = {1--26},
  doi = {10.1145/3709661}
}
Kostenlos starten →