EUVIMEDEUVIMED entdecken
Querschnittstudie

Appropriateness and Readability of Large Language Model Chatbot Responses to Frequently Asked Questions About Dry Eye Disease: Cross-Sectional Study

Bedia Kesimal, Sücattin İlker Kocamış

Akdeniz Medical Journal · 20.08.2026 · 12 (1)

DOI: 10.53394/akd.1859263 ↗

Kurz gefasst

Querschnittstudie in der Zeitschrift Akdeniz Medical Journal, online erschienen am 20.08.2026. Verfasst von Bedia Kesimal und 1 weiteren Autor:innen. Die Arbeit nennt 18 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Large language model chatbots are increasingly consulted for medical information. This study evaluated the accuracy and readability of chatbot responses to common patient questions on dry eye disease.

Methods

This cross-sectional study analysed responses from four chatbots (ChatGPT 3.5, ChatGPT 4.0, Google Gemini, and Microsoft Copilot) to fifty standardised questions about dry eye disease. Two ophthalmologists independently rated accuracy on a five-point Likert scale, with inter-rater agreement measured by Cohen’s kappa. Readability was assessed using Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman–Liau Index, Simple Measure of Gobbledygook, Flesch Reading Ease, and Reach score. Statistical tests included repeated-measures analysis of variance or Friedman tests with Bonferroni correction.

Results

Agreement between raters was excellent (kappa = 0.88). Google Gemini showed the highest accuracy (4.84 ± 0.37), followed by Microsoft Copilot (4.76 ± 0.48), ChatGPT 4.0 (4.42 ± 0.50), and ChatGPT 3.5 (4.32 ± 0.59; p < 0.001). Gemini and Copilot significantly outperformed both ChatGPT versions. Readability differed significantly (p < 0.001). ChatGPT 4.0 produced the simplest texts, with the lowest grade levels, highest Flesch Reading Ease and broadest Reach. Gemini and ChatGPT 3.5 generated more complex responses, while Copilot showed intermediate values.

Conclusions

Chatbots demonstrated complementary strengths. Gemini and Copilot were most accurate, whereas ChatGPT 4.0 was most readable and accessible. Chatbots may aid patient education in dry eye disease, but professional oversight remains necessary.

Zitieren

APA

Kesimal, B., & Kocamış, S. İ. (2026). Appropriateness and Readability of Large Language Model Chatbot Responses to Frequently Asked Questions About Dry Eye Disease: Cross-Sectional Study. Akdeniz Medical Journal, 12(1). https://doi.org/10.53394/akd.1859263

Vancouver

Kesimal B, Kocamış Sİ. Appropriateness and Readability of Large Language Model Chatbot Responses to Frequently Asked Questions About Dry Eye Disease: Cross-Sectional Study. Akdeniz Medical Journal. 2026;12(1). doi:10.53394/akd.1859263

BibTeX
@article{Kesimal2026,
  author = {Kesimal, Bedia and Kocamış, Sücattin İlker},
  title = {Appropriateness and Readability of Large Language Model Chatbot Responses to Frequently Asked Questions About Dry Eye Disease: Cross-Sectional Study},
  journal = {Akdeniz Medical Journal},
  year = {2026},
  volume = {12},
  number = {1},
  doi = {10.53394/akd.1859263}
}
Kostenlos starten →