Can ChatGPT Generate Acceptable Case‐Based Multiple‐Choice Questions for Medical School Anatomy Exams? A Pilot Study on Item Difficulty and Discrimination
Clinical Anatomy · 2025-05 · 38 (4) · 505-510
DOI: 10.1002/ca.24271 ↗Kurz gefasst
Fachartikel in der Zeitschrift Clinical Anatomy, online erschienen am 24.03.2025. Verfasst von Yavuz Selim Kıyak und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara, Turkey. Hauptthemen laut MEDLINE: Anatomy, Educational Measurement, Education, Medical, Undergraduate. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 34 Literaturangaben.
Abstract (Originalsprache)
Developing high‐quality multiple‐choice questions (MCQs) for medical school exams is effortful and time‐consuming. In this study, we investigated the ability of ChatGPT to generate case‐based anatomy MCQs with acceptable levels of item difficulty and discrimination for medical school exams. We used ChatGPT to generate case‐based anatomy MCQs for an endocrine and urogenital system exam based on a framework for artificial intelligence (AI)‐assisted item generation. The questions were evaluated by experts, approved by the department, and administered to 502 second‐year medical students (372 Turkish‐language, 130 English‐language). The items were analyzed to determine the discrimination and difficulty indices. The item discrimination indices ranged from 0.29 to 0.54, indicating acceptable differentiation between high‐ and low‐performing students. All items in Turkish (six out of six) and five out of six in English met the higher discrimination threshold (≥ 0.30) required for large‐scale standardized tests. The item difficulty indices ranged from 0.41 to 0.89, most items falling within the moderate difficulty range (0.20–0.80). Therefore, it was concluded that ChatGPT can generate case‐based anatomy MCQs with acceptable psychometric properties, offering a promising tool for medical educators. However, human expertise remains crucial for reviewing and refining AI‐generated assessment items. Future research should explore AI‐generated MCQs across various anatomy topics and investigate different AI models for question generation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medical Education and Informatics, Faculty of Medicine, Gazi University, Ankara, Turkey.
- Department of Anatomy, Faculty of Medicine, Gazi University, Ankara, Turkey.
ORCID-Profile: Yavuz Selim Kıyak, Tuncay Veysel Peker
Zitieren
APA
Kıyak, Y. S., Soylu, A., Coşkun, Ö., Budakoğlu, I. İ., & Peker, T. V. (2025). Can ChatGPT Generate Acceptable Case‐Based Multiple‐Choice Questions for Medical School Anatomy Exams? A Pilot Study on Item Difficulty and Discrimination. Clinical Anatomy, 38(4), 505-510. https://doi.org/10.1002/ca.24271
Vancouver
Kıyak YS, Soylu A, Coşkun Ö, Budakoğlu Iİ, Peker TV. Can ChatGPT Generate Acceptable Case‐Based Multiple‐Choice Questions for Medical School Anatomy Exams? A Pilot Study on Item Difficulty and Discrimination. Clinical Anatomy. 2025;38(4):505-510. doi:10.1002/ca.24271
BibTeX
@article{Kyak2025,
author = {Kıyak, Yavuz Selim and Soylu, Ayşe and Coşkun, Özlem and Budakoğlu, Işıl İrem and Peker, Tuncay Veysel},
title = {Can ChatGPT Generate Acceptable Case‐Based Multiple‐Choice Questions for Medical School Anatomy Exams? A Pilot Study on Item Difficulty and Discrimination},
journal = {Clinical Anatomy},
year = {2025},
volume = {38},
number = {4},
pages = {505--510},
doi = {10.1002/ca.24271}
}
EUVIMED