Comparative performance of large language models for patient-initiated ophthalmology consultations
Frontiers in Public Health · 2025
DOI: 10.3389/fpubh.2025.1673045 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Frontiers in Public Health, 2025. Verfasst von Mingxue Huang und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Nursing, Southwest Medical University, Luzhou, China. Hauptthemen laut MEDLINE: Ophthalmology, Language, Referral and Consultation. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) are increasingly accessed by lay users for medical advice. This study aims to conduct a comprehensive evaluation of the responses generated by five large language models.
Methods
We identified 31 ophthalmology-related questions most frequently raised by patients during routine consultations and subsequently elicited responses from five large language models: ChatGPT-4o, DeepSeek-V3, Doubao, Wenxin Yiyan 4.0 Turbo, and Qwen. A five-point likert scale was employed to assess each model across five domains: accuracy, logical consistency, coherence, safety, and content accessibility. Additionally, textual characteristics, including character, word, and sentence counts, were quantitatively analyzed.
Results
ChatGPT-4o and DeepSeek-V3 achieved the highest overall performance, with statistically superior accuracy and logical consistency ( p < 0.05). Existing safety evaluations indicate that both Doubao and Wenxin Yiyan 4.0 Turbo exhibit significant security deficiencies. Conversely, Qwen generated significantly longer outputs, as evidenced by greater character, word, and sentence counts.
Conclusion
ChatGPT-4o and DeepSeek-V3 demonstrated the highest overall performance and are best suited for laypersons seeking ophthalmic information. Doubao and Qwen, with their richer clinical terminology, better serve users with medical training, whereas Wenxin Yiyan 4.0 Turbo most effectively supports patients’ pre-procedural understanding of diagnostic procedures. Prospective randomized controlled trials are required to determine whether integrating the top-performing model into pre-consultation triage improves patient comprehension.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Nursing, Southwest Medical University, Luzhou, China.
- School of Future Technology, South China University of Technology, Guangzhou, China.
- Shenzhen Eye Hospital, Shenzhen Eye Medical Center, Southern Medical University, Shenzhen, China.
Zitieren
APA
Huang, M., Wang, X., Zhou, S., Cui, X., Zhang, Z., Xu, Y., Yang, W., & Chi, W. (2025). Comparative performance of large language models for patient-initiated ophthalmology consultations. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2025.1673045
Vancouver
Huang M, Wang X, Zhou S, Cui X, Zhang Z, Xu Y, et al. Comparative performance of large language models for patient-initiated ophthalmology consultations. Frontiers in Public Health. 2025. doi:10.3389/fpubh.2025.1673045
BibTeX
@article{Huang2025,
author = {Huang, Mingxue and Wang, Xiaoyan and Zhou, Shiqi and Cui, Xinyu and Zhang, Zilin and Xu, Yanwu and Yang, Weihua and Chi, Wei},
title = {Comparative performance of large language models for patient-initiated ophthalmology consultations},
journal = {Frontiers in Public Health},
year = {2025},
doi = {10.3389/fpubh.2025.1673045}
}
EUVIMED