Towards Scalable Annotation of Math Word Problems: Knowledge Component Tagging via LLMs and Sentence Embeddings
International Conference on Computers in Education · 01.12.2025
DOI: 10.58459/icce.2025.5919 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Conference on Computers in Education, online erschienen am 01.12.2025. Verfasst von Chor Seng Tan und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Mathematical word problems are a crucial component of mathematics education, requiring students to integrate multiple reasoning skills. Annotating these problems with knowledge components (KC) enables better personalized learning, adaptive tutoring, and AI-driven educational assessment. However, manual annotation is time-consuming and inconsistent, limiting the scalability of KC-based learning systems. In this work, we introduce a new labeled dataset of MWPs, derived from ASDiv and GSM8K, with KCs aligned to the Common Core mathematics framework. Using this dataset, we benchmark two different methods to perform automatic KC annotation without any labeled examples, namely LLM KC tagging and SBERT sentence embedding similarity scoring. Our results highlight key strengths and limitations of LLMs in this task, revealing challenges in consistency and reasoning alignment with human labels. We then show that SBERT-based similarity scoring underperforms LLM KC tagging, but can be significantly enhanced by combining the two methods, which addresses their respective limitations. This study provides critical insights into the feasibility of automated KC tagging, laying the foundation for future research in AI-assisted curriculum design and intelligent tutoring systems.
Zitieren
APA
Tan, C. S., Wei, C., & Kim, J. (2025). Towards Scalable Annotation of Math Word Problems: Knowledge Component Tagging via LLMs and Sentence Embeddings. International Conference on Computers in Education. https://doi.org/10.58459/icce.2025.5919
Vancouver
Tan CS, Wei C, Kim J. Towards Scalable Annotation of Math Word Problems: Knowledge Component Tagging via LLMs and Sentence Embeddings. International Conference on Computers in Education. 2025. doi:10.58459/icce.2025.5919
BibTeX
@article{Tan2025,
author = {Tan, Chor Seng and Wei, Chengwei and Kim, Jung-jae},
title = {Towards Scalable Annotation of Math Word Problems: Knowledge Component Tagging via LLMs and Sentence Embeddings},
journal = {International Conference on Computers in Education},
year = {2025},
doi = {10.58459/icce.2025.5919}
}
EUVIMED