EUVIMEDEUVIMED entdecken
Crossref

Evaluation of the Code Quality Generated by Generative AI

Salame Azbarka, Ali Mughrabi, Mario Sabbagh, Idan Roth

Information Systems Student Research Journal · 31.08.2025 · 2 · 1-17

DOI: 10.5617/issrj.13928 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Systems Student Research Journal, online erschienen am 31.08.2025. Verfasst von Salame Azbarka und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study aims to evaluate and differentiate the effectiveness of the leading Generative-AI tools for code generation by examining key code quality metrics. The research focuses on the evaluation of correctness, readability, and the efficiency of GenAI code. The research methodology comprised two stages: an initial benchmark of four generative AI tools, followed by further evaluation of the two top-performing models from the first stage. The four generative AI toolsChatGPT, Copilot, Claude, and Gemini- generated a code for ten coding problems, and then for the second benchmark, the best two Generative AI were further compared with an additional sample of 30 problems. The code was generated in Python, and all of the problems are from the competition coding website leetcode.com. The first benchmark results indicate that ChatGPT generated the best quality code with the highest score in most metrics, followed by Copilot, Claude, and finally, Gemini. In the second benchmark, in code correctness, ChatGPT outperformed Copilot, with 17 out of 30 code snippets being completely correct compared to Copilot's 12. In terms of code readability, both ChatGPT and Copilot scored highly, averaging 82.3% and 83.7%, respectively, while Copilot displayed a higher efficiency score. Nevertheless, the differences were statistically insignificant and, thus, may indicate that the tools are largely equivalent overall, though they may use different approaches to memory usage.

Zitieren

APA

Azbarka, S., Mughrabi, A., Sabbagh, M., & Roth, I. (2025). Evaluation of the Code Quality Generated by Generative AI. Information Systems Student Research Journal, 2, 1-17. https://doi.org/10.5617/issrj.13928

Vancouver

Azbarka S, Mughrabi A, Sabbagh M, Roth I. Evaluation of the Code Quality Generated by Generative AI. Information Systems Student Research Journal. 2025;2:1-17. doi:10.5617/issrj.13928

BibTeX
@article{Azbarka2025,
  author = {Azbarka, Salame and Mughrabi, Ali and Sabbagh, Mario and Roth, Idan},
  title = {Evaluation of the Code Quality Generated by Generative AI},
  journal = {Information Systems Student Research Journal},
  year = {2025},
  volume = {2},
  pages = {1--17},
  doi = {10.5617/issrj.13928}
}
Kostenlos starten →