Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective
New Zealand Medical Student Journal · 28.03.2026
DOI: 10.57129/001c.159636 ↗Kurz gefasst
Fachartikel in der Zeitschrift New Zealand Medical Student Journal, online erschienen am 28.03.2026. Verfasst von Zhaochu Geng und 6 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
AIM: This study investigates how varying prompt conditions influence the quality and clinical coherence of responses generated by a large language model (GPT-4o mini) in simulated psychiatric OSCE scenarios. METHODS: Four psychiatric OSCE cases were presented to GPT-4o mini under four conditions with increasing details: a standard clinical prompt, a context-enhanced prompt, and two variation prompts incorporating irrelevant or distracting information. GPT-4o mini was asked to perform key OSCE tasks, history-taking, risk assessment, explanation, and management for each case. Responses were scored using a standardised, structured rubric and analysed thematically. RESULTS: GPT-4o mini generated clinically relevant responses under standard and context-enhanced prompts. However, performance declined as irrelevant information was introduced. Quantitative scores dropped significantly across the different conditions, and qualitative analysis revealed reduced coherence, increased verbosity, and difficulty prioritising clinical content. CONCLUSIONS: LLMs like GPT-4o mini can generate useful responses when provided with clear and concise prompt instructions. However, in this study, we noted that clinical accuracy and coherence deteriorated in the presence of distracting or ambiguous input. This highlights the need for critical evaluation and unambiguous literacy when using LLMs in medical education.
Zitieren
APA
Geng, Z., Webster, C. S., Chen, Y., Ng, L., Krägeloh, C. U., Li, A., & Henning, M. A. (2026). Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective. New Zealand Medical Student Journal, 0. https://doi.org/10.57129/001c.159636
Vancouver
Geng Z, Webster CS, Chen Y, Ng L, Krägeloh CU, Li A, et al. Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective. New Zealand Medical Student Journal. 2026;0. doi:10.57129/001c.159636
BibTeX
@article{Geng2026,
author = {Geng, Zhaochu and Webster, Craig S. and Chen, Yan and Ng, Lillian and Krägeloh, Christian U. and Li, Angel and Henning, Marcus A.},
title = {Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective},
journal = {New Zealand Medical Student Journal},
year = {2026},
volume = {0},
doi = {10.57129/001c.159636}
}
EUVIMED