EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues

Kok Wai Lee, Yee Sin Ang, Joel Weijia Lai

Computers · 2025-12-15 · 14 (12) · 555

DOI: 10.3390/computers14120555 ↗

At a glance

Journal article in Computers, published online on 2025-12-15. Written by Kok Wai Lee and 2 co-authors. Cited 2 times so far (Crossref); 31 references. The full text is freely available (open access, CC BY 4.0).

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Educational chatbots are increasingly deployed to scaffold student learning, yet educators lack scalable ways to assess the cognitive depth of these dialogues in situ. Bloom’s taxonomy provides a principled lens for characterizing reasoning, but manual tagging of conversational turns is costly and difficult to scale for learning analytics. We present a reproducible high-confidence pseudo-labeling pipeline for multi-label Bloom classification of Socratic student–chatbot exchanges. The dataset comprises 6716 utterances collected from conversations between a Socratic chatbot and 34 undergraduate statistics students at Nanyang Technological University. From three chronologically selected workbooks with expert Bloom annotations, we trained and compared two labeling tracks: (i) a calibrated classical approach using SentenceTransformer (all-MiniLM-L6-v2) embeddings with one-vs-rest Logistic Regression, Linear SVM, XGBoost, and MLP, followed by per-class precision–recall threshold tuning; and (ii) a lightweight LLM track using GPT-4o-mini after supervised fine-tuning. Class-specific thresholds tuned on 5-fold cross-validation were then applied in a single pass to assign high-confidence pseudo-labels to the remaining unlabeled exchanges, avoiding feedback-loop confirmation bias. Fine-tuned GPT-4o-mini achieved the highest prevalence-weighted performance (micro-F1 =0.814), whereas calibrated classical models yielded stronger balance across Bloom levels (best macro-F1 =0.630 with Linear SVM; best classical micro-F1 =0.759 with Logistic Regression). Both model families reflect the corpus skew toward lower-order cognition, with LLMs excelling on common patterns and linear models better preserving rarer higher-order labels, while results should be interpreted as a proof-of-concept given limited gold labeling, the approach substantially reduces annotation burden and provides a practical pathway for Bloom-aware learning analytics and future real-time adaptive chatbot support.

Cite

APA

Lee, K. W., Ang, Y. S., & Lai, J. W. (2025). Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues. Computers, 14(12), 555. https://doi.org/10.3390/computers14120555

Vancouver

Lee KW, Ang YS, Lai JW. Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues. Computers. 2025;14(12):555. doi:10.3390/computers14120555

BibTeX
@article{Lee2025,
  author = {Lee, Kok Wai and Ang, Yee Sin and Lai, Joel Weijia},
  title = {Learning Analytics with Scalable Bloom’s Taxonomy Labeling of Socratic Chatbot Dialogues},
  journal = {Computers},
  year = {2025},
  volume = {14},
  number = {12},
  pages = {555},
  doi = {10.3390/computers14120555}
}
Start for free →