EUVIMEDEUVIMED entdecken
Crossref

An artificial intelligence-driven platform for practice question generation

Andrew Zahn iD, Seth Overla iD, D J Lowrie iD, Christine Y Zhou iD, Sally A Santen iD, Weibing Zheng iD, Laurah Turner iD

Academic Medicine · 2026

DOI: 10.1093/acamed/wvaf074 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Academic Medicine, 2026. Verfasst von Andrew Zahn und 6 weiteren Autor:innen. Beteiligte Einrichtung: University of Cincinnati College of Medicine, Cincinnati, OH, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Educational Measurement, Licensure, Medical, Education, Medical, Undergraduate. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 8 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Problem High-stakes licensing exams such as the United States Medical Licensing Examination (USMLE) play a critical role in medical education, influencing both trainee progression and patient outcomes. Access to high-quality board preparation resources is uneven and often cost-prohibitive, disproportionately affecting students from underrepresented or financially disadvantaged backgrounds. Approach An artificial intelligence (AI)-driven system to generate USMLE-style practice questions aligned with National Board of Medical Examiners (NBME) item-writing guidelines using a Large Language Model (LLM) enhanced with retrieval augmented generation, chain-of-thought and few-shot prompting, and JavaScript Object Notation schema validation was developed and piloted at the University of Cincinnati College of Medicine between November and December 2023. Five lectures from a preclinical hematology course were selected, and 565 questions were generated for 177 first-year medical students. A human-in-the-loop process, led by a faculty course director, ensured content validity and adherence to educational standards. Validated questions were deployed via a mobile app, allowing students to practice, receive performance feedback, and access an AI tutor. Outcomes Of the 565 questions, 490 (87%) were deemed accurate and NBME-compliant. Eighty students used the question bank, completing up to 220 questions each. Although not statistically significant, increased use trended toward improved performance on related exam questions. Qualitative feedback highlighted enthusiasm for AI-assisted study tools, with calls for broader content coverage. Next Steps This pilot demonstrates that LLMs can generate high-quality, guideline-aligned practice questions. To improve scalability and reduce faculty workload, future iterations will incorporate AI-based review agents for pre-screening content. The platform is intended to be expanded to additional courses, training phases, and health professions. Ongoing refinement will focus on improving content specificity and maintaining accuracy, especially in advanced and subspecialty education.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceEducational MeasurementLicensure, MedicalEducation, Medical, UndergraduateHumansUnited StatesLarge Language ModelsIntelligent SystemsPilot Projects

Stichwörter der Autor:innen

artificial intelligenceautomated item generationdesign-based researchlarge language modelsmedical education technology

Autor:innen und Institutionen

  • University of Cincinnati College of Medicine, Cincinnati, OH, United States.
  • Office of Artificial Intelligence and Educational Informatics, Department of Biostatistics, Health Informatics, and Data Sciences, University of Cincinnati College of Medicine, Cincinnati, OH, United States.
  • Department of Medical Education, University of Cincinnati College of Medicine, OH, United States.
  • Division of Pulmonary, Critical Care, and Sleep Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States.
  • Department of Emergency Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States.
  • School of Information Technology, University of Cincinnati College of Education, Criminal Justice, and Human Services, Cincinnati, OH, United States.

ORCID-Profile: Andrew Zahn, Seth Overla, D J Lowrie, Christine Y Zhou, Sally A Santen, Weibing Zheng, Laurah Turner

Zitieren

APA

Zahn, A., Overla, S., Lowrie, D. J., Zhou, C. Y., Santen, S. A., Zheng, W., & Turner, L. (2026). An artificial intelligence-driven platform for practice question generation. Academic Medicine. https://doi.org/10.1093/acamed/wvaf074

Vancouver

Zahn A, Overla S, Lowrie DJ, Zhou CY, Santen SA, Zheng W, et al. An artificial intelligence-driven platform for practice question generation. Academic Medicine. 2026. doi:10.1093/acamed/wvaf074

BibTeX
@article{Zahn2026,
  author = {Zahn, Andrew and Overla, Seth and Lowrie, D J and Zhou, Christine Y and Santen, Sally A and Zheng, Weibing and Turner, Laurah},
  title = {An artificial intelligence-driven platform for practice question generation},
  journal = {Academic Medicine},
  year = {2026},
  doi = {10.1093/acamed/wvaf074}
}
Kostenlos starten →