Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study
Journal of Educational Evaluation for Health Professions · 2026
DOI: 10.3352/jeehp.2026.23.9 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Educational Evaluation for Health Professions, 2026. Verfasst von Piotr Szychowiak und 7 weiteren Autor:innen. Beteiligte Einrichtung: Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France. Hauptthemen laut MEDLINE: Large Language Models, Educational Measurement, Clinical Competence, Students, Medical. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 10 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Developing objective structured clinical examination (OSCE) stations is time-consuming for medical teachers. We aimed to evaluate the ability of a large language model (LLM) to generate ready-to-use OSCE stations. Five OSCE stations generated by the LLM GPT-4o were evaluated by 7 expert assessors using a 5-point Likert scale and compared with 5 teacher-written stations targeting similar learning objectives. A station was considered to be of good quality if most assessors responded “agree” or “strongly agree” to the statement “The station is good enough to be used by students.” All teacher-written stations were rated as being of good quality, compared with only one GPT-4o-generated station. The LLM produced adequate clinical scenarios when reference knowledge was provided and tasks were clearly ordered, but it failed to generate reliable assessment grids. Careful review by teachers remained essential. GPT-4o failed to consistently produce fully ready-to-use OSCE stations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France.
- Réanimation Chirurgicale, Centre Hospitalier Régional Universitaire de Tours, Orleans, France.
- Service des Maladies Infectieuses et Tropicales, Centre Hospitalier Régional Universitaire de Tours, Orleans, France.
- Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France. maxime.desgrouas@chu-orleans.fr.
Zitieren
APA
Szychowiak, P., Wong-So, J., Messet, H., Faure, M., Breteau, I., Jamard, S., Barbier, F., & Desgrouas, M. (2026). Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2026.23.9
Vancouver
Szychowiak P, Wong-So J, Messet H, Faure M, Breteau I, Jamard S, et al. Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study. Journal of Educational Evaluation for Health Professions. 2026. doi:10.3352/jeehp.2026.23.9
BibTeX
@article{Szychowiak2026,
author = {Szychowiak, Piotr and Wong-So, Jonathan and Messet, Hélène and Faure, Mélanie and Breteau, Isaure and Jamard, Simon and Barbier, François and Desgrouas, Maxime},
title = {Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study},
journal = {Journal of Educational Evaluation for Health Professions},
year = {2026},
doi = {10.3352/jeehp.2026.23.9}
}
EUVIMED