EUVIMEDEUVIMED entdecken
Crossref

AceCoder : An Effective Prompting Technique Specialized in Code Generation

Jia Li, Yunfei Zhao, Yongmin Li, Ge Li, Zhi Jin

ACM Transactions on Software Engineering and Methodology · 30.11.2024 · 33 (8) · 1-26

DOI: 10.1145/3675395 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 21.11.2024. Verfasst von Jia Li und 4 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key R & D Program, Major Program (JD) of Hubei Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have shown great success in code generation. LLMs take as the input a prompt and output the code. How to make prompts (i.e., Prompting Techniques ) is a key question. Existing prompting techniques are designed for natural language generation and have low accuracy in code generation. In this article, we propose a new prompting technique named AceCoder . Our motivation is that code generation meets two unique challenges (i.e., requirement understanding and code implementation). AceCoder contains two novel mechanisms (i.e., guided code generation and example retrieval) to solve these challenges. ❶ Guided code generation asks LLMs first to analyze requirements and output an intermediate preliminary (e.g., test cases). The preliminary clarifies requirements and tells LLMs “what to write.” ❷ Example retrieval selects similar programs as examples in prompts, which provide lots of relevant content (e.g., algorithms, APIs) and teach LLMs “how to write.” We apply AceCoder to four LLMs (e.g., GPT-3.5, CodeGeeX) and evaluate it on three public benchmarks using the Pass@ \(k\) . Results show that AceCoder can significantly improve the performance of LLMs on code generation. In terms of Pass@1, AceCoder outperforms the SOTA baseline by up to 56.4% in MBPP, 70.7% in MBJP, and 88.4% in MBJSP . AceCoder is effective in LLMs with different sizes (i.e., 6B–13B) and different languages (i.e., Python, Java, and JavaScript). Human evaluation shows human developers prefer programs from AceCoder .

Förderung

  • National Natural Science Foundation of China
  • National Key R & D Program
  • Major Program (JD) of Hubei Province

Zitieren

APA

Li, J., Zhao, Y., Li, Y., Li, G., & Jin, Z. (2024). AceCoder : An Effective Prompting Technique Specialized in Code Generation. ACM Transactions on Software Engineering and Methodology, 33(8), 1-26. https://doi.org/10.1145/3675395

Vancouver

Li J, Zhao Y, Li Y, Li G, Jin Z. AceCoder : An Effective Prompting Technique Specialized in Code Generation. ACM Transactions on Software Engineering and Methodology. 2024;33(8):1-26. doi:10.1145/3675395

BibTeX
@article{Li2024,
  author = {Li, Jia and Zhao, Yunfei and Li, Yongmin and Li, Ge and Jin, Zhi},
  title = {AceCoder : An Effective Prompting Technique Specialized in Code Generation},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2024},
  volume = {33},
  number = {8},
  pages = {1--26},
  doi = {10.1145/3675395}
}
Kostenlos starten →