A High-Quality Generation Approach for Educational Programming Projects Using LLM
IEEE Transactions on Learning Technologies · 2024 · 17 · 2242-2255
DOI: 10.1109/tlt.2024.3499751 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Learning Technologies, 2024. Verfasst von Tian Song und 2 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 40 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
High-quality programming projects for education are critically required in teaching. However, it is hard to develop those projects efficiently and artificially constrained by the lecturers' experience and background. The recent popularity of large language models (LLMs) has led to a great number of applications in the field of education, but concerns persist that the output might be unreliable when dealing with intricate requirements. In this study, we design a customized role-based agent (CRBA), which can be configured for different roles specializing in specific areas of expertise, making the LLM yield content of higher specialization. An iterative architecture of multi-CRBAs is proposed to generate multistep projects, where CRBAs automatically criticize and optimize the LLM's intermediate outputs to enhance quality. We propose ten evaluation metrics across three aspects to assess project quality through expert grading. Further, we conduct an A/B test among 60 undergraduate students in a programming course and collect their feedback through a questionnaire. According to the students' rating results, the LLM-generated projects have comparable performance to man-made ones in terms of project description, learning step setting, assistance to students, and overall project quality. This study effectively integrates LLM into educational scenarios and enhances the efficiency of creating high-quality and practical programming exercises for lecturers.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Song, T., Zhang, H., & Xiao, Y. (2024). A High-Quality Generation Approach for Educational Programming Projects Using LLM. IEEE Transactions on Learning Technologies, 17, 2242-2255. https://doi.org/10.1109/tlt.2024.3499751
Vancouver
Song T, Zhang H, Xiao Y. A High-Quality Generation Approach for Educational Programming Projects Using LLM. IEEE Transactions on Learning Technologies. 2024;17:2242-2255. doi:10.1109/tlt.2024.3499751
BibTeX
@article{Song2024,
author = {Song, Tian and Zhang, Hang and Xiao, Yijia},
title = {A High-Quality Generation Approach for Educational Programming Projects Using LLM},
journal = {IEEE Transactions on Learning Technologies},
year = {2024},
volume = {17},
pages = {2242--2255},
doi = {10.1109/tlt.2024.3499751}
}
EUVIMED