EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Comparative performance of large language models for patient-initiated ophthalmology consultations

Mingxue Huang, Xiaoyan Wang, Shiqi Zhou, Xinyu Cui, Zilin Zhang, Yanwu Xu, Weihua Yang, Wei Chi

Frontiers in Public Health · 2025

DOI: 10.3389/fpubh.2025.1673045 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Frontiers in Public Health, 2025. Verfasst von Mingxue Huang und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Nursing, Southwest Medical University, Luzhou, China. Hauptthemen laut MEDLINE: Ophthalmology, Language, Referral and Consultation. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly accessed by lay users for medical advice. This study aims to conduct a comprehensive evaluation of the responses generated by five large language models.

Methods

We identified 31 ophthalmology-related questions most frequently raised by patients during routine consultations and subsequently elicited responses from five large language models: ChatGPT-4o, DeepSeek-V3, Doubao, Wenxin Yiyan 4.0 Turbo, and Qwen. A five-point likert scale was employed to assess each model across five domains: accuracy, logical consistency, coherence, safety, and content accessibility. Additionally, textual characteristics, including character, word, and sentence counts, were quantitatively analyzed.

Results

ChatGPT-4o and DeepSeek-V3 achieved the highest overall performance, with statistically superior accuracy and logical consistency ( p < 0.05). Existing safety evaluations indicate that both Doubao and Wenxin Yiyan 4.0 Turbo exhibit significant security deficiencies. Conversely, Qwen generated significantly longer outputs, as evidenced by greater character, word, and sentence counts.

Conclusion

ChatGPT-4o and DeepSeek-V3 demonstrated the highest overall performance and are best suited for laypersons seeking ophthalmic information. Doubao and Qwen, with their richer clinical terminology, better serve users with medical training, whereas Wenxin Yiyan 4.0 Turbo most effectively supports patients’ pre-procedural understanding of diagnostic procedures. Prospective randomized controlled trials are required to determine whether integrating the top-performing model into pre-consultation triage improves patient comprehension.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

OphthalmologyLanguageReferral and ConsultationHumansLarge Language Models

Stichwörter der Autor:innen

consultationhealthcarelarge language modelophthalmologypatient education

Autor:innen und Institutionen

  • School of Nursing, Southwest Medical University, Luzhou, China.
  • School of Future Technology, South China University of Technology, Guangzhou, China.
  • Shenzhen Eye Hospital, Shenzhen Eye Medical Center, Southern Medical University, Shenzhen, China.

Zitieren

APA

Huang, M., Wang, X., Zhou, S., Cui, X., Zhang, Z., Xu, Y., Yang, W., & Chi, W. (2025). Comparative performance of large language models for patient-initiated ophthalmology consultations. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2025.1673045

Vancouver

Huang M, Wang X, Zhou S, Cui X, Zhang Z, Xu Y, et al. Comparative performance of large language models for patient-initiated ophthalmology consultations. Frontiers in Public Health. 2025. doi:10.3389/fpubh.2025.1673045

BibTeX
@article{Huang2025,
  author = {Huang, Mingxue and Wang, Xiaoyan and Zhou, Shiqi and Cui, Xinyu and Zhang, Zilin and Xu, Yanwu and Yang, Weihua and Chi, Wei},
  title = {Comparative performance of large language models for patient-initiated ophthalmology consultations},
  journal = {Frontiers in Public Health},
  year = {2025},
  doi = {10.3389/fpubh.2025.1673045}
}
Kostenlos starten →