Large Language Models Can Generate High-Quality Pathology Multiple-Choice Questions Comparable With Questions Written by a Human Expert
Modern Pathology · 2026-01 · 39 (1) · 100940
DOI: 10.1016/j.modpat.2025.100940 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Modern Pathology, 2026-01. Verfasst von Michael J Borowitz und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pathology, Sol Goldman Pancreatic Cancer Research Center, Johns Hopkins University School of Medicine, Baltimore, Maryland; Sidney Kimm…. Hauptthemen laut MEDLINE: Pathology, Educational Measurement, Language. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: School of Medicine, Johns Hopkins University.
Abstract (Originalsprache)
Multiple-choice questions can be effective tools to assess student and trainee performance, but the creation of these questions can be time consuming and requires expertise. To test the quality of pathology test questions created by large language models (LLMs), 100 questions on pancreas pathology were written by a human expert, and 50 questions were generated by each of 2 LLMs (ChatGPT-4.0 and Gemini 2.5 Flash). After an initial review, 16% of the multiple-choice questions generated by the 2 LLMs had to be revised through additional interactive prompting. The final set of questions was then evaluated by 190 volunteers with a variety of backgrounds and levels of expertise. We found that ChatGPT-generated-but not Gemini-generated-questions were rated as easier than human-authored questions; there were slightly more poor/unacceptable questions compared with adequate/good/excellent questions written by the LLMs than those written by the human expert (11.7% vs 10.1%; odds ratio, 1.64; 95% CI, 1.13-2.37; P = .009), but there was no difference in the proportion of questions rated good or excellent. Qualitatively, human-authored questions were thought to be most clinically realistic but felt to be more inconsistent and sometimes thought to be testing trivial points. There was no difference in the mean point biserial between human-authored and LLM-generated questions (0.31 vs 0.29; P = .56). As LLMs improve, they will form a useful tool for the efficient generation of large numbers of high-quality pathology test questions.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Pathology, Sol Goldman Pancreatic Cancer Research Center, Johns Hopkins University School of Medicine, Baltimore, Maryland; Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Med…
- Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, Maryland; Department of Oncology, Sol Goldman Pancreatic Cancer Research Center, Johns Hopkins University School of Medi…
- Department of Pathology, Sol Goldman Pancreatic Cancer Research Center, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Förderung
- School of Medicine, Johns Hopkins University
Zitieren
APA
Borowitz, M. J., Blackford, A. L., Nagelia, S., & Hruban, R. H. (2026). Large Language Models Can Generate High-Quality Pathology Multiple-Choice Questions Comparable With Questions Written by a Human Expert. Modern Pathology, 39(1), 100940. https://doi.org/10.1016/j.modpat.2025.100940
Vancouver
Borowitz MJ, Blackford AL, Nagelia S, Hruban RH. Large Language Models Can Generate High-Quality Pathology Multiple-Choice Questions Comparable With Questions Written by a Human Expert. Modern Pathology. 2026;39(1):100940. doi:10.1016/j.modpat.2025.100940
BibTeX
@article{Borowitz2026,
author = {Borowitz, Michael J and Blackford, Amanda L and Nagelia, Suman and Hruban, Ralph H},
title = {Large Language Models Can Generate High-Quality Pathology Multiple-Choice Questions Comparable With Questions Written by a Human Expert},
journal = {Modern Pathology},
year = {2026},
volume = {39},
number = {1},
pages = {100940},
doi = {10.1016/j.modpat.2025.100940}
}
EUVIMED