Comparing Automated Scoring Using Natural Language Generation and Understanding Models in Constructed-Response Assessment for Economic Education
The SNU Journal of Education Research · 30.03.2026 · 35 (1) · 85-113
DOI: 10.54346/sjer.2026.35.1.85 ↗Kurz gefasst
Fachartikel in der Zeitschrift The SNU Journal of Education Research, online erschienen am 30.03.2026. Verfasst von Kyung Hoon Kim und 1 weiteren Autor:innen.
Abstract (Originalsprache)
This study compares automated scoring performance between natural language generation (NLG) and natural language understanding (NLU) models for constructed-response items in the economics unit of middle school social studies. Three items reflecting the 2022 National Curriculum categories—knowledge and understanding, process and skills, and values and attitudes—were administered to 300 students, yielding 900 responses. Agreement between human raters and AI scoring was examined using Quadratic Weighted Kappa and related indices. Both models showed high agreement with human ratings for knowledge-and-understanding item. For process-and-skills item, the NLU model performed better due to its classification-based scoring approach, whereas for values-and-attitudes item the NLG model showed relatively stronger performance. The NLG model tended to underscore responses, while the NLU model tended to overscore. These findings suggest that AI-based automated scoring can support teachers’ assessment, but model selection should vary according to item characteristics and evaluation purposes.
Zitieren
APA
Kim, K. H., & Park, H. J. (2026). Comparing Automated Scoring Using Natural Language Generation and Understanding Models in Constructed-Response Assessment for Economic Education. The SNU Journal of Education Research, 35(1), 85-113. https://doi.org/10.54346/sjer.2026.35.1.85
Vancouver
Kim KH, Park HJ. Comparing Automated Scoring Using Natural Language Generation and Understanding Models in Constructed-Response Assessment for Economic Education. The SNU Journal of Education Research. 2026;35(1):85-113. doi:10.54346/sjer.2026.35.1.85
BibTeX
@article{Kim2026,
author = {Kim, Kyung Hoon and Park, Hyung Joon},
title = {Comparing Automated Scoring Using Natural Language Generation and Understanding Models in Constructed-Response Assessment for Economic Education},
journal = {The SNU Journal of Education Research},
year = {2026},
volume = {35},
number = {1},
pages = {85--113},
doi = {10.54346/sjer.2026.35.1.85}
}
EUVIMED