EUVIMEDEUVIMED entdecken
Crossref

Automated Coding of Communications in Collaborative Problem‐Solving Tasks Using ChatGPT

Jiangang Hao, Wenju Cui, Patrick Kyllonen, Emily Kerzabi, Lei Liu, Michael Flor

Journal of Educational Measurement · 2025-12 · 62 (4) · 809-837

DOI: 10.1111/jedm.70014 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Educational Measurement, online erschienen am 23.10.2025. Verfasst von Jiangang Hao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Army Research Institute for the Behavioral and Social Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Collaborative problem solving is widely recognized as a critical 21st‐century skill. Assessing collaborative problem solving depends on coding the communication data using a construct‐relevant framework, and this process has long been a major bottleneck to scaling up such assessments. Based on five datasets and two coding frameworks, we demonstrate that ChatGPT can code communication data to a satisfactory level, though performance varies across ChatGPT models and depends on the coding framework and task characteristics. Interestingly, newer reasoning‐focused models, such as GPT‐o1‐mini and GPT‐o3‐mini, do not necessarily yield better coding results. Additionally, we show that refining prompts based on feedback from miscoded cases can improve coding accuracy in some instances, though the effectiveness of this approach is not consistent across all tasks. These findings offer practical guidance for researchers and practitioners in developing scalable, efficient methods to analyze communication data in support of 21st‐century skill assessment.

Förderung

  • Army Research Institute for the Behavioral and Social Sciences

Zitieren

APA

Hao, J., Cui, W., Kyllonen, P., Kerzabi, E., Liu, L., & Flor, M. (2025). Automated Coding of Communications in Collaborative Problem‐Solving Tasks Using ChatGPT. Journal of Educational Measurement, 62(4), 809-837. https://doi.org/10.1111/jedm.70014

Vancouver

Hao J, Cui W, Kyllonen P, Kerzabi E, Liu L, Flor M. Automated Coding of Communications in Collaborative Problem‐Solving Tasks Using ChatGPT. Journal of Educational Measurement. 2025;62(4):809-837. doi:10.1111/jedm.70014

BibTeX
@article{Hao2025,
  author = {Hao, Jiangang and Cui, Wenju and Kyllonen, Patrick and Kerzabi, Emily and Liu, Lei and Flor, Michael},
  title = {Automated Coding of Communications in Collaborative Problem‐Solving Tasks Using ChatGPT},
  journal = {Journal of Educational Measurement},
  year = {2025},
  volume = {62},
  number = {4},
  pages = {809--837},
  doi = {10.1111/jedm.70014}
}
Kostenlos starten →