EUVIMEDEUVIMED entdecken
Crossref

Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems

Shengming Zhao, Yuchen Shao, Yuheng Huang, Jiayang Song, Zhijie Wang, Chengcheng Wan, Lei Ma

ACM Transactions on Software Engineering and Methodology · 15.04.2026

DOI: 10.1145/3802824 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 15.04.2026. Verfasst von Shengming Zhao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 95 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities. However, practitioners face significant challenges when making RAG deployment decisions. While existing research prioritizes algorithmic innovations, a systematic gap persists in understanding fundamental engineering trade-offs that determine RAG success. We present the first comprehensive study of three universal RAG deployment decisions: whether to deploy RAG, how much information to retrieve, and how to integrate retrieved knowledge effectively. Through systematic experiments across three LLMs and six datasets spanning question answering and code generation tasks, we reveal critical insights: (1) RAG deployment must be highly selective, with variable recall thresholds and failure modes affecting up to 12.6% of samples even with perfect documents. (2) Optimal retrieval volume exhibits task-dependent behavior—QA tasks show universal patterns (5-10 documents optimal) while code generation requires scenario-specific optimization. (3) Knowledge integration effectiveness depends on task and model characteristics, with code generation benefiting significantly from prompting methods while question answering shows minimal improvement. These findings demonstrate that universal RAG strategies prove inadequate. Effective RAG systems require context-aware design decisions based on task characteristics and model capabilities. Our analysis provides evidence-based guidance for practitioners and establishes foundational insights for principled RAG deployment. Our code, data and artifacts are publicly available at https://github.com/ShengmingZ/RAG_Benchmark_Code_QA .

Zitieren

APA

Zhao, S., Shao, Y., Huang, Y., Song, J., Wang, Z., Wan, C., & Ma, L. (2026). Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3802824

Vancouver

Zhao S, Shao Y, Huang Y, Song J, Wang Z, Wan C, et al. Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems. ACM Transactions on Software Engineering and Methodology. 2026. doi:10.1145/3802824

BibTeX
@article{Zhao2026,
  author = {Zhao, Shengming and Shao, Yuchen and Huang, Yuheng and Song, Jiayang and Wang, Zhijie and Wan, Chengcheng and Ma, Lei},
  title = {Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2026},
  doi = {10.1145/3802824}
}
Kostenlos starten →