Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies
Ophthalmology Science · 2026-05 · 6 (5) · 101158
DOI: 10.1016/j.xops.2026.101158 ↗Kurz gefasst
Fachartikel in der Zeitschrift Ophthalmology Science, 2026-05. Verfasst von Xuanqiao Lin und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Ophthalmology, Eye, Ear, Nose, and Throat Hospital of Fudan University, Shanghai, People's Republic of China. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Fudan University, National Natural Science Foundation of China.
Abstract (Originalsprache)
Purpose
To evaluate contemporary deep-reasoning large language models (LLMs) for early assessment of ophthalmic emergencies using sequential, workflow-mimicking information levels.
Design
Cross-sectional, vignette-based, head-to-head comparative evaluation. Subjects: Thirty-four deidentified emergency ophthalmology teaching cases curated from a publicly accessible repository.
Methods
Each case was reconstructed into 3 sequential information levels (level 1 [L1]: history; level 2: basic examination; level 3 [L3]: specialist examination). Six LLMs (Doubao, DeepSeek, Kimi-2, ChatGPT-5, Gemini-3, and Grok-4), operating in deep-reasoning mode, generated outputs that were independently scored by 2 ophthalmologists. Diagnoses were graded as fully correct, partially correct, or incorrect; triage category (typical vs. atypical emergency) was rated as correct or incorrect. Ancillary test recommendations were mapped to a prespecified 10-category taxonomy and classified as undertesting, exact match, or overtesting. A four-level composite outcome integrated diagnostic correctness, triage accuracy, and testing. Main Outcome Measures: Diagnostic correctness (fully correct, partially correct, incorrect), triage-category accuracy, ancillary test recommendation patterns, and composite outcome (ideal, safe but overtesting, potentially dangerous, intermediate).
Results
Across 612 model-case-level outputs, 46.9% of diagnoses were fully correct, 24.5% partially correct, and 28.6% incorrect. Fully correct diagnoses increased from 43.1% at L1 to 53.9% at L3 (P = 0.048). Overall triage category accuracy was 85.3% (range, 76.5%-94.1% across models; P = 0.003) and did not differ across information levels (P = 0.89). Ancillary test recommendations most commonly reflected undertesting (51.0%), followed by overtesting (27.5%) and exact matches (21.6%) (P < 0.001 across models). In generalized estimating equation pairwise comparisons, ChatGPT-5 showed higher odds of a fully correct diagnosis than DeepSeek (odds ratio [OR], 3.54; 95% confidence interval [CI], 1.49-8.43) and Gemini-3 (OR, 2.24; 95% CI, 1.31-3.83), and lower odds of potentially dangerous composite outcomes than DeepSeek (OR, 0.28; 95% CI, 0.10-0.74) and Gemini-3 (OR, 0.31; 95% CI, 0.11-0.89).
Conclusions
Deep-reasoning LLMs demonstrated high triage category accuracy and moderate diagnostic performance for ophthalmic emergencies, with diagnostic correctness improving at higher information levels. However, ancillary testing patterns varied substantially, and ideal composite safety profiles were uncommon, supporting cautious, supervised deployment with explicit guardrails governing workup recommendations. Financial Disclosures: The author has no/the authors have no proprietary or commercial interest in any materials discussed in this article.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Ophthalmology, Eye, Ear, Nose, and Throat Hospital of Fudan University, Shanghai, People's Republic of China.
- Department of Ophthalmology, Jiaxing Hospital of Traditional Chinese Medicine, Jiaxing, China.
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China.
Förderung
- Fudan University
- National Natural Science Foundation of China
Zitieren
APA
Lin, X., Yang, Y., Ren, Y., Wang, L., & Cai, L. (2026). Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies. Ophthalmology Science, 6(5), 101158. https://doi.org/10.1016/j.xops.2026.101158
Vancouver
Lin X, Yang Y, Ren Y, Wang L, Cai L. Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies. Ophthalmology Science. 2026;6(5):101158. doi:10.1016/j.xops.2026.101158
BibTeX
@article{Lin2026,
author = {Lin, Xuanqiao and Yang, Yizhou and Ren, Yuecheng and Wang, Lan and Cai, Lei},
title = {Comparative Evaluation of Deep-Reasoning Large Language Models for Ophthalmic Emergencies},
journal = {Ophthalmology Science},
year = {2026},
volume = {6},
number = {5},
pages = {101158},
doi = {10.1016/j.xops.2026.101158}
}
EUVIMED