Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study
Journal of Educational Evaluation for Health Professions · 08.07.2024 · 21 · 17
DOI: 10.3352/jeehp.2024.21.17 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Educational Evaluation for Health Professions, online erschienen am 08.07.2024. Verfasst von Max Samuel Yudovich und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Urology, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, USA. Hauptthemen laut MEDLINE: Educational Measurement, Urology, Clinical Competence. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 12 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Purpose
This study aimed to evaluate the performance of Chat Generative Pre-Trained Transformer (ChatGPT) with respect to standardized urology multiple-choice items in the United States.
Methods
In total, 700 multiple-choice urology board exam-style items were submitted to GPT-3.5 and GPT-4, and responses were recorded. Items were categorized based on topic and question complexity (recall, interpretation, and problem-solving). The accuracy of GPT-3.5 and GPT-4 was compared across item types in February 2024.
Results
GPT-4 answered 44.4% of items correctly compared to 30.9% for GPT-3.5 (P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Urology, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, USA.
- Penn State College of Medicine, Hershey, PA, USA.
Zitieren
APA
Yudovich, M. S., Makarova, E., Hague, C. M., & Raman, J. D. (2024). Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study. Journal of Educational Evaluation for Health Professions, 21, 17. https://doi.org/10.3352/jeehp.2024.21.17
Vancouver
Yudovich MS, Makarova E, Hague CM, Raman JD. Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study. Journal of Educational Evaluation for Health Professions. 2024;21:17. doi:10.3352/jeehp.2024.21.17
BibTeX
@article{Yudovich2024,
author = {Yudovich, Max Samuel and Makarova, Elizaveta and Hague, Christian Michael and Raman, Jay Dilip},
title = {Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study},
journal = {Journal of Educational Evaluation for Health Professions},
year = {2024},
volume = {21},
pages = {17},
doi = {10.3352/jeehp.2024.21.17}
}
EUVIMED