AI versus Faculty: A Comparative Study of Narrative Feedback on Medical Students' Written Mental Status Exams
Medical Research Archives · 2025 · 14 (3)
DOI: 10.18103/mra.v14i3.7347 ↗Kurz gefasst
Fachartikel in der Zeitschrift Medical Research Archives, 2025. Verfasst von Elle Cleaves und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Objective
The authors evaluated narrative feedback generated by ChatGPT and faculty members for medical students' Mental Status Exam (MSE) write-ups. The study compared feedback quality and usefulness and assessed whether students and academic psychiatrists could identify the feedback source.
Methods
Medical students (N=164) wrote MSEs and received blinded feedback from either a faculty member (for low-scoring write-ups, n=43) or ChatGPT (for high-scoring write-ups, n=121). Students rated the feedback's quality and usefulness and guessed its origin. Three academic psychiatrists also conducted a blinded evaluation, rating both feedback types for the low-scoring MSEs, choosing the superior version, and guessing the source.
Results
Students rated AI feedback quality significantly higher than faculty feedback (mean=4.22 vs. 3.5). Academic psychiatrists preferred the AI-generated feedback in 93% of cases. Only 29% of students receiving AI feedback correctly identified its source. Psychiatrists correctly identified AI feedback only 23% of the time and misattributed faculty feedback as AI-generated 71% of the time.
Conclusions
AI-generated feedback was perceived as high-quality by students and preferred by expert raters. The difficulty in distinguishing AI from faculty feedback suggests generative AI can produce feedback comparable or superior to human experts, offering a scalable tool to support medical education and reduce faculty workload.
Zitieren
APA
Cleaves, E., Belmonte, D., Raiciulescu, S., & Duncan, J. (2025). AI versus Faculty: A Comparative Study of Narrative Feedback on Medical Students' Written Mental Status Exams. Medical Research Archives, 14(3). https://doi.org/10.18103/mra.v14i3.7347
Vancouver
Cleaves E, Belmonte D, Raiciulescu S, Duncan J. AI versus Faculty: A Comparative Study of Narrative Feedback on Medical Students' Written Mental Status Exams. Medical Research Archives. 2025;14(3). doi:10.18103/mra.v14i3.7347
BibTeX
@article{Cleaves2025,
author = {Cleaves, Elle and Belmonte, David and Raiciulescu, Sorana and Duncan, Joshua},
title = {AI versus Faculty: A Comparative Study of Narrative Feedback on Medical Students' Written Mental Status Exams},
journal = {Medical Research Archives},
year = {2025},
volume = {14},
number = {3},
doi = {10.18103/mra.v14i3.7347}
}
EUVIMED