EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Comparative Evaluation of Large Language Models in Computer Programming Education

Melina Najimi, Saba Yazdani, Marzieh Ahmadzadeh

Proceedings of the Canadian Engineering Education Association (CEEA) · 07.08.2026

DOI: 10.24908/pceea.2026.21576 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the Canadian Engineering Education Association (CEEA), online erschienen am 07.08.2026. Verfasst von Melina Najimi und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Delivering high-quality formative feedback in large introductory programming courses remains challenging despite the widespread use of automated grading systems. Although large language models (LLMs) can generate natural-language explanations grounded in student code and test outcomes, their diagnostic reliability and pedagogical suitability require systematic evaluation. This study presents a comparative analysis of six LLMs for generating formative feedback on introductory Java programs containing predefined defects under controlled conditions. Model outputs were evaluated across key dimensions of feedback quality and performance. Results reveal substantial cross-model variation, particularly in multi-defect scenarios. More precise fault localization was associated with higher diagnostic accuracy and completeness, but also with more frequent instructional-constraint violations. No single model consistently outperformed others across all criteria, and response latency did not reliably correspond to diagnostic quality. These findings underscore the context-dependent nature of LLM-generated formative programming feedback.

Zitieren

APA

Najimi, M., Yazdani, S., & Ahmadzadeh, M. (2026). Comparative Evaluation of Large Language Models in Computer Programming Education. Proceedings of the Canadian Engineering Education Association (CEEA). https://doi.org/10.24908/pceea.2026.21576

Vancouver

Najimi M, Yazdani S, Ahmadzadeh M. Comparative Evaluation of Large Language Models in Computer Programming Education. Proceedings of the Canadian Engineering Education Association (CEEA). 2026. doi:10.24908/pceea.2026.21576

BibTeX
@article{Najimi2026,
  author = {Najimi, Melina and Yazdani, Saba and Ahmadzadeh, Marzieh},
  title = {Comparative Evaluation of Large Language Models in Computer Programming Education},
  journal = {Proceedings of the Canadian Engineering Education Association (CEEA)},
  year = {2026},
  doi = {10.24908/pceea.2026.21576}
}
Kostenlos starten →