EUVIMEDEUVIMED entdecken
Crossref

Structured Chain-of-Thought Prompting for Code Generation

Jia Li, Ge Li, Yongmin Li, Zhi Jin

ACM Transactions on Software Engineering and Methodology · 28.02.2025 · 34 (2) · 1-23

DOI: 10.1145/3690635 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 21.01.2025. Verfasst von Jia Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 159-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key R&D Program, Major Program (JD) of Hubei Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have shown impressive abilities in code generation. Chain-of-Thought (CoT) prompting is the state-of-the-art approach to utilizing LLMs. CoT prompting asks LLMs first to generate CoTs (i.e., intermediate natural language reasoning steps) and then output the code. However, the accuracy of CoT prompting still cannot satisfy practical applications. For example, gpt-3.5-turbo with CoT prompting only achieves 53.29% Pass@1 in HumanEval. In this article, we propose Structured CoTs (SCoTs) and present a novel prompting technique for code generation named SCoT prompting. Our motivation is that human developers follow structured programming. Developers use three programming structures (i.e., sequential, branch, and loop) to design and implement structured programs. Thus, we ask LLMs to use three programming structures to generate SCoTs (structured reasoning steps) before outputting the final code. Compared to CoT prompting, SCoT prompting explicitly introduces programming structures and unlocks the structured programming thinking of LLMs. We apply SCoT prompting to two LLMs (i.e., gpt-4-turbo, gpt-3.5-turbo, and DeepSeek Coder-Instruct- \(\{\) 1.3B, 6.7B, 33B \(\}\) ) and evaluate it on three benchmarks (i.e., HumanEval, MBPP, and MBCPP). SCoT prompting outperforms CoT prompting by up to 13.79% in Pass@1. SCoT prompting is robust to examples and achieves substantial improvements. The human evaluation also shows human developers prefer programs from SCoT prompting.

Förderung

  • National Natural Science Foundation of China
  • National Key R&D Program
  • Major Program (JD) of Hubei Province

Zitieren

APA

Li, J., Li, G., Li, Y., & Jin, Z. (2025). Structured Chain-of-Thought Prompting for Code Generation. ACM Transactions on Software Engineering and Methodology, 34(2), 1-23. https://doi.org/10.1145/3690635

Vancouver

Li J, Li G, Li Y, Jin Z. Structured Chain-of-Thought Prompting for Code Generation. ACM Transactions on Software Engineering and Methodology. 2025;34(2):1-23. doi:10.1145/3690635

BibTeX
@article{Li2025,
  author = {Li, Jia and Li, Ge and Li, Yongmin and Jin, Zhi},
  title = {Structured Chain-of-Thought Prompting for Code Generation},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  volume = {34},
  number = {2},
  pages = {1--23},
  doi = {10.1145/3690635}
}
Kostenlos starten →