Benchmarking publicly accessible large language models for high-myopia multiple-choice question generation in digital ophthalmic education and public health training
Frontiers in Public Health · 05.05.2026 · 14
DOI: 10.3389/fpubh.2026.1843045 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Public Health, online erschienen am 05.05.2026. Verfasst von Ligang Jiang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Ophthalmology, Quzhou Affiliated Hospital of Wenzhou Medical University, Quzhou People's Hospital, Quzhou, Zhejiang, China. Hauptthemen laut MEDLINE: Large Language Models, Ophthalmology, Benchmarking, Myopia. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Digital tools are reshaping public health education and training, yet evidence on whether large language models (LLMs) can generate specialist ophthalmic teaching materials remains limited. High myopia (HM), a vision-threatening condition with long-term management needs and public health relevance, provides a suitable setting for evaluating this capability. This study compared five LLMs in generating HM-related multiple-choice questions (MCQs) for ophthalmic education.
Methods
Five LLMs (ChatGPT-5.4, Gemini 3, DeepSeek, Kimi K2.5, and Doubao) completed 60 predefined HM MCQ generation tasks each, yielding 300 MCQs. A standardized blueprint covered four domains: basic knowledge, clinical cases, diagnosis and treatment decision-making, and screening/follow-up management. Objective evaluation included structural completeness, format compliance, keyed-answer accuracy, output features, and response time. Two ophthalmology experts rated six domains using 5-point Likert scales, and Spearman analyses examined associations among text features, response time, and expert ratings.
Results
All models achieved 100.0% initial structural acceptability, structural completeness, and format compliance. Keyed-answer accuracy was highest for ChatGPT-5.4 and Gemini 3 (both 100.0%), followed by DeepSeek (98.3%) and Kimi K2.5 and Doubao (both 95.0%). Significant between-model differences were observed across all output features and response time (all p < 0.001). ChatGPT-5.4 generated the shortest stems, Gemini 3 the shortest explanations and fastest responses, and Kimi K2.5 and Doubao the longest explanations and total outputs. Inter-rater agreement was good (ICC range, 0.835–0.885). Significant differences were found in clarity, distractor quality, and mean subjective score (all p < 0.001), but not in content rigor, educational usefulness, cognitive-level alignment, or overall usability. DeepSeek achieved the highest median mean score, while direct usability was highest for ChatGPT-5.4 (91.7%) and Gemini 3 (90.0%). Content rigor was strongly associated with overall usability ( ρ = 0.85, p < 0.05), whereas distractor quality was negatively associated with explanation length ( ρ = −0.43, p < 0.05) and total output length ( ρ = −0.37, p < 0.05).
Conclusion
LLMs can reliably generate structurally valid HM-related MCQs under standardized Chinese prompting conditions. Their value may lie in supporting digital ophthalmic education and public health training, although expert oversight remains necessary because meaningful differences persist in factual accuracy, distractor quality, and direct usability.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Ophthalmology, Quzhou Affiliated Hospital of Wenzhou Medical University, Quzhou People's Hospital, Quzhou, Zhejiang, China.
- Quzhou College of Technology, Quzhou, Zhejiang, China.
- The Eye Hospital, School of Ophthalmology & Optometry, Wenzhou Medical University, Wenzhou, China.
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine Vision and Brain Health), Wenzhou, Zhejiang, China.
- Wenzhou Institute, University of Chinese Academy of Sciences, Wenzhou, China.
Zitieren
APA
Jiang, L., Jiang, X., Wu, W., & Jiang, F. (2026). Benchmarking publicly accessible large language models for high-myopia multiple-choice question generation in digital ophthalmic education and public health training. Frontiers in Public Health, 14. https://doi.org/10.3389/fpubh.2026.1843045
Vancouver
Jiang L, Jiang X, Wu W, Jiang F. Benchmarking publicly accessible large language models for high-myopia multiple-choice question generation in digital ophthalmic education and public health training. Frontiers in Public Health. 2026;14. doi:10.3389/fpubh.2026.1843045
BibTeX
@article{Jiang2026,
author = {Jiang, Ligang and Jiang, Xin and Wu, Wencan and Jiang, Fangzheng},
title = {Benchmarking publicly accessible large language models for high-myopia multiple-choice question generation in digital ophthalmic education and public health training},
journal = {Frontiers in Public Health},
year = {2026},
volume = {14},
doi = {10.3389/fpubh.2026.1843045}
}
EUVIMED