EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models

Wesley Morris, Langdon Holmes, Joon Suh Choi, Scott Crossley

International Journal of Artificial Intelligence in Education · 2025-06 · 35 (2) · 559-586

DOI: 10.1007/s40593-024-00418-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Artificial Intelligence in Education, 2025-06. Verfasst von Wesley Morris und 3 weiteren Autor:innen. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent developments in the field of artificial intelligence allow for improved performance in the automated assessment of extended response items in mathematics, potentially allowing for the scoring of these items cheaply and at scale. This study details the grand prize-winning approach to developing large language models (LLMs) to automatically score the ten items in the National Assessment of Educational Progress (NAEP) Math Scoring Challenge. The approach uses extensive preprocessing that balanced the class labels for each item. This was done by identifying and filtering over-represented classes using a classifier trained on document-term matrices and data augmentation of under-represented classes using a generative pre-trained large language model (Grammarly’s Coedit-XL; Raheja et al., 2023). We also use input modification schemes that were hand-crafted to each item type and included information from parts of the multi-step math problem students had to solve. Finally, we finetune several pre-trained large language models on the modified input for each individual item in the NAEP automated math scoring challenge, with DeBERTa (He et al., 2021a) showing the best performance. This approach achieved human-like agreement (less than QWK 0.05 difference from human–human agreement) on nine out of the ten items in a held-out test set.

Zitieren

APA

Morris, W., Holmes, L., Choi, J. S., & Crossley, S. (2025). Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models. International Journal of Artificial Intelligence in Education, 35(2), 559-586. https://doi.org/10.1007/s40593-024-00418-w

Vancouver

Morris W, Holmes L, Choi JS, Crossley S. Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models. International Journal of Artificial Intelligence in Education. 2025;35(2):559-586. doi:10.1007/s40593-024-00418-w

BibTeX
@article{Morris2025,
  author = {Morris, Wesley and Holmes, Langdon and Choi, Joon Suh and Crossley, Scott},
  title = {Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models},
  journal = {International Journal of Artificial Intelligence in Education},
  year = {2025},
  volume = {35},
  number = {2},
  pages = {559--586},
  doi = {10.1007/s40593-024-00418-w}
}
Kostenlos starten →