EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study

Max Samuel Yudovich, Elizaveta Makarova, Christian Michael Hague, Jay Dilip Raman

Journal of Educational Evaluation for Health Professions · 08.07.2024 · 21 · 17

DOI: 10.3352/jeehp.2024.21.17 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Educational Evaluation for Health Professions, online erschienen am 08.07.2024. Verfasst von Max Samuel Yudovich und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Urology, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, USA. Hauptthemen laut MEDLINE: Educational Measurement, Urology, Clinical Competence. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 12 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study aimed to evaluate the performance of Chat Generative Pre-Trained Transformer (ChatGPT) with respect to standardized urology multiple-choice items in the United States.

Methods

In total, 700 multiple-choice urology board exam-style items were submitted to GPT-3.5 and GPT-4, and responses were recorded. Items were categorized based on topic and question complexity (recall, interpretation, and problem-solving). The accuracy of GPT-3.5 and GPT-4 was compared across item types in February 2024.

Results

GPT-4 answered 44.4% of items correctly compared to 30.9% for GPT-3.5 (P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Educational MeasurementUrologyClinical CompetenceHumansUnited StatesSpecialty Boards

Stichwörter der Autor:innen

Artificial intelligenceCertificationGraduate medical educationUnited StatesUrology

Autor:innen und Institutionen

  • Department of Urology, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, USA.
  • Penn State College of Medicine, Hershey, PA, USA.

Zitieren

APA

Yudovich, M. S., Makarova, E., Hague, C. M., & Raman, J. D. (2024). Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study. Journal of Educational Evaluation for Health Professions, 21, 17. https://doi.org/10.3352/jeehp.2024.21.17

Vancouver

Yudovich MS, Makarova E, Hague CM, Raman JD. Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study. Journal of Educational Evaluation for Health Professions. 2024;21:17. doi:10.3352/jeehp.2024.21.17

BibTeX
@article{Yudovich2024,
  author = {Yudovich, Max Samuel and Makarova, Elizaveta and Hague, Christian Michael and Raman, Jay Dilip},
  title = {Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study},
  journal = {Journal of Educational Evaluation for Health Professions},
  year = {2024},
  volume = {21},
  pages = {17},
  doi = {10.3352/jeehp.2024.21.17}
}
Kostenlos starten →