Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues
Computers · 15.12.2025 · 14 (12) · 555
DOI: 10.3390/computers14120555 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computers, online erschienen am 15.12.2025. Verfasst von Kok Wai Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Educational chatbots are increasingly deployed to scaffold student learning, yet educators lack scalable ways to assess the cognitive depth of these dialogues in situ. Bloom’s taxonomy provides a principled lens for characterizing reasoning, but manual tagging of conversational turns is costly and difficult to scale for learning analytics. We present a reproducible high-confidence pseudo-labeling pipeline for multi-label Bloom classification of Socratic student–chatbot exchanges. The dataset comprises 6716 utterances collected from conversations between a Socratic chatbot and 34 undergraduate statistics students at Nanyang Technological University. From three chronologically selected workbooks with expert Bloom annotations, we trained and compared two labeling tracks: (i) a calibrated classical approach using SentenceTransformer (all-MiniLM-L6-v2) embeddings with one-vs-rest Logistic Regression, Linear SVM, XGBoost, and MLP, followed by per-class precision–recall threshold tuning; and (ii) a lightweight LLM track using GPT-4o-mini after supervised fine-tuning. Class-specific thresholds tuned on 5-fold cross-validation were then applied in a single pass to assign high-confidence pseudo-labels to the remaining unlabeled exchanges, avoiding feedback-loop confirmation bias. Fine-tuned GPT-4o-mini achieved the highest prevalence-weighted performance (micro-F1 =0.814), whereas calibrated classical models yielded stronger balance across Bloom levels (best macro-F1 =0.630 with Linear SVM; best classical micro-F1 =0.759 with Logistic Regression). Both model families reflect the corpus skew toward lower-order cognition, with LLMs excelling on common patterns and linear models better preserving rarer higher-order labels, while results should be interpreted as a proof-of-concept given limited gold labeling, the approach substantially reduces annotation burden and provides a practical pathway for Bloom-aware learning analytics and future real-time adaptive chatbot support.
Zitieren
APA
Lee, K. W., Ang, Y. S., & Lai, J. W. (2025). Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues. Computers, 14(12), 555. https://doi.org/10.3390/computers14120555
Vancouver
Lee KW, Ang YS, Lai JW. Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues. Computers. 2025;14(12):555. doi:10.3390/computers14120555
BibTeX
@article{Lee2025,
author = {Lee, Kok Wai and Ang, Yee Sin and Lai, Joel Weijia},
title = {Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues},
journal = {Computers},
year = {2025},
volume = {14},
number = {12},
pages = {555},
doi = {10.3390/computers14120555}
}
EUVIMED