EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparing Large Language Models and Human Programmers for Generating Programming Code

Wenpin Hou, Zhicheng Ji iD

Advanced Science · 2025-02 · 12 (8)

DOI: 10.1002/advs.202412279 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Advanced Science, online erschienen am 30.12.2024. Verfasst von Wenpin Hou und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics, Mailman School of Public Health, Columbia University, New York City, NY, 10032, USA. Hauptthemen laut MEDLINE: Programming Languages, Software. Die Arbeit wurde bisher 53-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NHGRI NIH HHS, NIGMS NIH HHS, NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The performance of seven large language models (LLMs) in generating programming code using various prompt strategies, programming languages, and task difficulties is systematically evaluated. GPT‐4 substantially outperforms other LLMs, including Gemini Ultra and Claude 2. The coding performance of GPT‐4 varies considerably with different prompt strategies. In most LeetCode and GeeksforGeeks coding contests evaluated in this study, GPT‐4, employing the optimal prompt strategy, outperforms 85 percent of human participants in a competitive environment, many of whom are students and professionals with moderate programming experience. GPT‐4 demonstrates strong capabilities in translating code between different programming languages and in learning from past errors. The computational efficiency of the code generated by GPT‐4 is comparable to that of human programmers. GPT‐4 is also capable of handling broader programming tasks, including front‐end design and database operations. These results suggest that GPT‐4 has the potential to serve as a reliable assistant in programming code generation and software development. A programming assistant is designed based on an optimal prompt strategy to facilitate the practical use of LLMs for programming.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Programming LanguagesSoftwareHumansLarge Language Models

Stichwörter der Autor:innen

artificial intelligencecomputer programminghuman‐computer interactionlarge language models

Autor:innen und Institutionen

  • Department of Biostatistics, Mailman School of Public Health, Columbia University, New York City, NY, 10032, USA.
  • Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC, 07024, USA.

ORCID-Profile: Zhicheng Ji

Förderung

  • NHGRI NIH HHS
  • NIGMS NIH HHS
  • NIH HHS
  • National Institutes of Health

Zitieren

APA

Hou, W., & Ji, Z. (2025). Comparing Large Language Models and Human Programmers for Generating Programming Code. Advanced Science, 12(8). https://doi.org/10.1002/advs.202412279

Vancouver

Hou W, Ji Z. Comparing Large Language Models and Human Programmers for Generating Programming Code. Advanced Science. 2025;12(8). doi:10.1002/advs.202412279

BibTeX
@article{Hou2025,
  author = {Hou, Wenpin and Ji, Zhicheng},
  title = {Comparing Large Language Models and Human Programmers for Generating Programming Code},
  journal = {Advanced Science},
  year = {2025},
  volume = {12},
  number = {8},
  doi = {10.1002/advs.202412279}
}
Kostenlos starten →