Comparing Large Language Models and Human Programmers for Generating Programming Code
Advanced Science · 2025-02 · 12 (8)
DOI: 10.1002/advs.202412279 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Advanced Science, online erschienen am 30.12.2024. Verfasst von Wenpin Hou und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics, Mailman School of Public Health, Columbia University, New York City, NY, 10032, USA. Hauptthemen laut MEDLINE: Programming Languages, Software. Die Arbeit wurde bisher 53-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NHGRI NIH HHS, NIGMS NIH HHS, NIH HHS.
Abstract (Originalsprache)
The performance of seven large language models (LLMs) in generating programming code using various prompt strategies, programming languages, and task difficulties is systematically evaluated. GPT‐4 substantially outperforms other LLMs, including Gemini Ultra and Claude 2. The coding performance of GPT‐4 varies considerably with different prompt strategies. In most LeetCode and GeeksforGeeks coding contests evaluated in this study, GPT‐4, employing the optimal prompt strategy, outperforms 85 percent of human participants in a competitive environment, many of whom are students and professionals with moderate programming experience. GPT‐4 demonstrates strong capabilities in translating code between different programming languages and in learning from past errors. The computational efficiency of the code generated by GPT‐4 is comparable to that of human programmers. GPT‐4 is also capable of handling broader programming tasks, including front‐end design and database operations. These results suggest that GPT‐4 has the potential to serve as a reliable assistant in programming code generation and software development. A programming assistant is designed based on an optimal prompt strategy to facilitate the practical use of LLMs for programming.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biostatistics, Mailman School of Public Health, Columbia University, New York City, NY, 10032, USA.
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC, 07024, USA.
ORCID-Profile: Zhicheng Ji
Förderung
- NHGRI NIH HHS
- NIGMS NIH HHS
- NIH HHS
- National Institutes of Health
Zitieren
APA
Hou, W., & Ji, Z. (2025). Comparing Large Language Models and Human Programmers for Generating Programming Code. Advanced Science, 12(8). https://doi.org/10.1002/advs.202412279
Vancouver
Hou W, Ji Z. Comparing Large Language Models and Human Programmers for Generating Programming Code. Advanced Science. 2025;12(8). doi:10.1002/advs.202412279
BibTeX
@article{Hou2025,
author = {Hou, Wenpin and Ji, Zhicheng},
title = {Comparing Large Language Models and Human Programmers for Generating Programming Code},
journal = {Advanced Science},
year = {2025},
volume = {12},
number = {8},
doi = {10.1002/advs.202412279}
}
EUVIMED