EUVIMEDEUVIMED entdecken
Crossref

Eyes on the Text: Assessing Readability of Artificial Intelligence and Ophthalmologist Responses to Patient Surgery Queries

Sai S Kurapati, Derek J Barnett, Antonio Yaghy, Cameron J Sabet, David N Younessi, Dang Nguyen, John C Lin, Ingrid U Scott

Ophthalmologica · 10.03.2025 · 248 (3) · 149-159

DOI: 10.1159/000544917 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Ophthalmologica, online erschienen am 10.03.2025. Verfasst von Sai S Kurapati und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Ophthalmology, Penn State College of Medicine, Hershey, Pennsylvania, USA, skurapati1@pennstatehealth.psu.edu. Hauptthemen laut MEDLINE: Artificial Intelligence, Ophthalmologists, Comprehension, Ophthalmologic Surgical Procedures. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 34 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Generative artificial intelligence (AI) technologies like GPT-4 can instantaneously provide health information to patients; however, the readability of these outputs compared to ophthalmologist-written responses is unknown. This study aimed to evaluate the readability of GPT-4-generated and ophthalmologist-written responses to patient queries about ophthalmic surgery.

Methods

This retrospective cross-sectional study used 200 randomly selected patient questions about ophthalmic surgery extracted from the American Academy of Ophthalmology’s EyeSmart platform. The questions were inputted into GPT-4, and the generated responses were recorded. Ophthalmologist-written replies to the same questions were compiled for comparison. Readability of GPT-4 and ophthalmologist responses was assessed using six validated metrics: Flesch Kincaid Reading Ease (FK-RE), Flesch Kincaid Grade Level (FK-GL), Gunning Fog Score (GFS), SMOG Index (SI), Coleman Liau Index (CLI), and Automated Readability Index (ARI). Descriptive statistics, one-way ANOVA, Shapiro-Wilk, and Levene’s tests (α = 0.05) were used to compare readability between the two groups.

Results

GPT-4 used a higher percentage of complex words (24.42%) compared to ophthalmologists (17.76%), although mean (standard deviation) word count per sentence was similar (18.43 [2.95] and 18.01 [6.09]). Across all metrics (FK-RE; FK-GL; GFS; SI; CLI; and ARI), GPT-4 responses were at a higher grade level (34.39 [8.51]; 13.19 [2.63]; 16.37 [2.04]; 12.18 [1.43]; 15.72 [1.40]; 12.99 [1.86]) than ophthalmologists’ responses (50.61 [15.53]; 10.71 [2.99]; 14.13 [3.55]; 10.07 [2.46]; 12.64 [2.93]; 10.40 [3.61]), with both sources necessitating a 12th-grade education for comprehension. ANOVA tests showed significance (p < 0.05) for all comparisons except word count (p = 0.438).

Conclusion

The National Institutes of Health advises health information to be written at a 6th- to 7th-grade level. Both GPT-4- and ophthalmologist-written answers exceeded this recommendation, with GPT-4 showing a greater gap. Information accessibility is vital when designing patient resources, particularly with the rise of AI as an educational tool.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceOphthalmologistsComprehensionOphthalmologic Surgical ProceduresReadingHumansCross-Sectional StudiesRetrospective StudiesMaleFemaleSurveys and QuestionnairesOphthalmology

Stichwörter der Autor:innen

Artificial intelligenceHealth disparitiesHealth literacyOphthalmic surgery

Autor:innen und Institutionen

  • Department of Ophthalmology, Penn State College of Medicine, Hershey, Pennsylvania, USA, skurapati1@pennstatehealth.psu.edu.
  • Department of Ophthalmology, Penn State College of Medicine, Hershey, Pennsylvania, USA.
  • New England Eye Center, Boston, Massachusetts, USA.
  • Department of Medicine, Georgetown University School of Medicine, Washington, DC, USA.
  • Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA.
  • Corrigan Minehan Heart Center, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts, USA.

Zitieren

APA

Kurapati, S. S., Barnett, D. J., Yaghy, A., Sabet, C. J., Younessi, D. N., Nguyen, D., Lin, J. C., & Scott, I. U. (2025). Eyes on the Text: Assessing Readability of Artificial Intelligence and Ophthalmologist Responses to Patient Surgery Queries. Ophthalmologica, 248(3), 149-159. https://doi.org/10.1159/000544917

Vancouver

Kurapati SS, Barnett DJ, Yaghy A, Sabet CJ, Younessi DN, Nguyen D, et al. Eyes on the Text: Assessing Readability of Artificial Intelligence and Ophthalmologist Responses to Patient Surgery Queries. Ophthalmologica. 2025;248(3):149-159. doi:10.1159/000544917

BibTeX
@article{Kurapati2025,
  author = {Kurapati, Sai S and Barnett, Derek J and Yaghy, Antonio and Sabet, Cameron J and Younessi, David N and Nguyen, Dang and Lin, John C and Scott, Ingrid U},
  title = {Eyes on the Text: Assessing Readability of Artificial Intelligence and Ophthalmologist Responses to Patient Surgery Queries},
  journal = {Ophthalmologica},
  year = {2025},
  volume = {248},
  number = {3},
  pages = {149--159},
  doi = {10.1159/000544917}
}
Kostenlos starten →