EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study

Piotr Szychowiak, Jonathan Wong-So, Hélène Messet, Mélanie Faure, Isaure Breteau, Simon Jamard, François Barbier, Maxime Desgrouas

Journal of Educational Evaluation for Health Professions · 2026

DOI: 10.3352/jeehp.2026.23.9 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Educational Evaluation for Health Professions, 2026. Verfasst von Piotr Szychowiak und 7 weiteren Autor:innen. Beteiligte Einrichtung: Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France. Hauptthemen laut MEDLINE: Large Language Models, Educational Measurement, Clinical Competence, Students, Medical. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 10 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Developing objective structured clinical examination (OSCE) stations is time-consuming for medical teachers. We aimed to evaluate the ability of a large language model (LLM) to generate ready-to-use OSCE stations. Five OSCE stations generated by the LLM GPT-4o were evaluated by 7 expert assessors using a 5-point Likert scale and compared with 5 teacher-written stations targeting similar learning objectives. A station was considered to be of good quality if most assessors responded “agree” or “strongly agree” to the statement “The station is good enough to be used by students.” All teacher-written stations were rated as being of good quality, compared with only one GPT-4o-generated station. The LLM produced adequate clinical scenarios when reference knowledge was provided and tasks were clearly ordered, but it failed to generate reliable assessment grids. Careful review by teachers remained essential. GPT-4o failed to consistently produce fully ready-to-use OSCE stations.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsEducational MeasurementClinical CompetenceStudents, MedicalEducation, Medical, UndergraduatePilot ProjectsHumans

Stichwörter der Autor:innen

Artificial intelligenceClinical competenceEducational measurementFranceMedical education

Autor:innen und Institutionen

  • Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France.
  • Réanimation Chirurgicale, Centre Hospitalier Régional Universitaire de Tours, Orleans, France.
  • Service des Maladies Infectieuses et Tropicales, Centre Hospitalier Régional Universitaire de Tours, Orleans, France.
  • Médecine Intensive Réanimation, Centre Hospitalier Universitaire d'Orléans, Orleans, France. maxime.desgrouas@chu-orleans.fr.

Zitieren

APA

Szychowiak, P., Wong-So, J., Messet, H., Faure, M., Breteau, I., Jamard, S., Barbier, F., & Desgrouas, M. (2026). Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study. Journal of Educational Evaluation for Health Professions. https://doi.org/10.3352/jeehp.2026.23.9

Vancouver

Szychowiak P, Wong-So J, Messet H, Faure M, Breteau I, Jamard S, et al. Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study. Journal of Educational Evaluation for Health Professions. 2026. doi:10.3352/jeehp.2026.23.9

BibTeX
@article{Szychowiak2026,
  author = {Szychowiak, Piotr and Wong-So, Jonathan and Messet, Hélène and Faure, Mélanie and Breteau, Isaure and Jamard, Simon and Barbier, François and Desgrouas, Maxime},
  title = {Large language model-generated versus teacher-written objective structured clinical examination stations for medical students: a blinded comparative pilot study},
  journal = {Journal of Educational Evaluation for Health Professions},
  year = {2026},
  doi = {10.3352/jeehp.2026.23.9}
}
Kostenlos starten →