EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Digital guides in eye care: Comparing AI model accuracy and reliability

Hakan Veli Savaş iD, Osman Altay iD

DIGITAL HEALTH · 2026-02 · 12

DOI: 10.1177/20552076261433657 ↗

Kurz gefasst

Fachartikel in der Zeitschrift DIGITAL HEALTH, online erschienen am 08.04.2026. Verfasst von Hakan Veli Savaş und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Ophthalmology, Karakoçan State Hospital, Karakoçan, Elazığ, Turkey. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

The aim of this study was to comparatively evaluate four large language models (LLMs) used for patient education in ophthalmology in terms of accuracy, reliability, and patient safety across different ophthalmic subspecialties.

Methods

In this cross-sectional evaluation, a total of 50 frequently asked patient questions covering five ophthalmic subspecialties (strabismus/pediatric ophthalmology, oculoplastics, cataract and refractive surgery, retina, and dry eye) were included. All questions were submitted in a text-only format to ChatGPT o3 Mini High, Gemini 2.0 Pro, Claude-Sonnet 3.7, and LLaMA 3.1 405B. The generated responses were independently evaluated by five blinded ophthalmologists using a 10-point scale assessing accuracy, currency, informativeness/clarity, and patient safety. Potentially unsafe content was identified and categorized using a predefined structured error taxonomy.

Results

Marked differences in performance were observed among the models. Mean scores were 3.44 for Gemini, 2.99 for ChatGPT, 2.48 for Claude, and 1.09 for LLaMA. Gemini demonstrated higher performance across most subspecialties, whereas in the retina subspecialty, ChatGPT and Claude generated comparatively stronger responses. Of the 200 evaluated responses, 19 (9.5%) contained potentially unsafe content, with the lowest proportion observed for Gemini and the highest for LLaMA.

Conclusions

LLMs can generate useful responses for patient education in ophthalmology, but performance varies by model and subspecialty. Within this 50-question, text-only expert-rating framework, Gemini 2.0 Pro and ChatGPT o3 Mini High provided relatively higher accuracy and reliability in most areas, whereas LLaMA 3.1 405B lagged. Larger and clinically integrated evaluations, including direct assessment of patient understanding and behavior, are needed to define their safe use in practice.

Schlagwörter

Stichwörter der Autor:innen

Ophthalmologyexpert evaluationlarge language modelspatient educationpatient safety

Autor:innen und Institutionen

  • Department of Ophthalmology, Karakoçan State Hospital, Karakoçan, Elazığ, Turkey.
  • Department of Software Engineering, Manisa Celal Bayar University, Manisa, Turkey.

ORCID-Profile: Hakan Veli Savaş, Osman Altay

Zitieren

APA

Savaş, H. V., & Altay, O. (2026). Digital guides in eye care: Comparing AI model accuracy and reliability. DIGITAL HEALTH, 12. https://doi.org/10.1177/20552076261433657

Vancouver

Savaş HV, Altay O. Digital guides in eye care: Comparing AI model accuracy and reliability. DIGITAL HEALTH. 2026;12. doi:10.1177/20552076261433657

BibTeX
@article{Sava2026,
  author = {Savaş, Hakan Veli and Altay, Osman},
  title = {Digital guides in eye care: Comparing AI model accuracy and reliability},
  journal = {DIGITAL HEALTH},
  year = {2026},
  volume = {12},
  doi = {10.1177/20552076261433657}
}
Kostenlos starten →