EUVIMEDEUVIMED entdecken
Crossref

An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities

Zezhou Yang, Sirong Chen, Cuiyun Gao, Zhenhao Li, Xing Hu, Kui Liu, Xin Xia

ACM Transactions on Software Engineering and Methodology · 30.09.2025 · 34 (7) · 1-28

DOI: 10.1145/3717061 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 14.08.2025. Verfasst von Zezhou Yang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 45-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Guangdong Province, Shenzhen-Hong Kong Jointly Funded Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Code generation aims to automatically generate code snippets of specific programming language according to natural language descriptions. The continuous advancements in deep learning, particularly pre-trained models, have empowered the code generation task to achieve remarkable performance. One main challenge of pre-trained models for code generation is the semantic gap between developers’ natural language requirements and source code. To address the issue, prior studies typically adopt a retrieval-augmented framework for the task, where the similar code snippets collected by a retrieval process can be leveraged to help understand the requirements and provide guidance for the generation process. In a retrieval-augmented framework, similar data can be retrieved from the database using a retrieval algorithm, and original input data can be fused with retrieved data by different fusion strategies. However, there is a lack of systematic study on the application of this framework for code generation, including the impact of the final generated results and the specific usage of the framework. In this article, we choose three popular pre-trained code models, namely CodeGen, UniXcoder, and CodeT5, to assess the impact of the quality and utilization of retrieved code on the retrieval-augmented framework. Our analysis shows that the retrieval-augmented framework is beneficial for improving the performance of the existing pre-trained models. We also provide suggestions on the utilization of the retrieval-augmented code generation framework: BM25 and Sequential Integration Fusion are recommended due to their convenience and superior performance. Sketch Filling Fusion, which extracts a sketch of relevant code, could help the model improve its performance further. Additionally, we conduct experiments to investigate the influence of the retrieval-augmented framework on large language models for code generation, showing the effectiveness of the framework, and we discuss the tradeoff between performance improvement and computational costs in each phase within the framework.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Guangdong Province
  • Shenzhen-Hong Kong Jointly Funded Project
  • Shenzhen Basic Research
  • Shenzhen International Science and Technology Cooperation
  • Major Key Project of PCL

Zitieren

APA

Yang, Z., Chen, S., Gao, C., Li, Z., Hu, X., Liu, K., & Xia, X. (2025). An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities. ACM Transactions on Software Engineering and Methodology, 34(7), 1-28. https://doi.org/10.1145/3717061

Vancouver

Yang Z, Chen S, Gao C, Li Z, Hu X, Liu K, et al. An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities. ACM Transactions on Software Engineering and Methodology. 2025;34(7):1-28. doi:10.1145/3717061

BibTeX
@article{Yang2025,
  author = {Yang, Zezhou and Chen, Sirong and Gao, Cuiyun and Li, Zhenhao and Hu, Xing and Liu, Kui and Xia, Xin},
  title = {An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  volume = {34},
  number = {7},
  pages = {1--28},
  doi = {10.1145/3717061}
}
Kostenlos starten →