Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study
Journal of Medical Internet Research · 2025-06-04 · 27 · e69955
DOI: 10.2196/69955 ↗At a glance
Querschnittstudie in Journal of Medical Internet Research, published online on 2025-06-04. Written by John Will and 5 co-authors. Institution: Medical Center Information Technology Department of Health Informatics, New York University Langone Health, New York, NY, United States. Major MEDLINE topics: Patient Education as Topic, Internet, Comprehension, Language. Cited 139 times so far (Crossref); 43 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Online accessible patient education materials (PEMs) are essential for patient empowerment. However, studies have shown that these materials often exceed the recommended sixth-grade reading level, making them difficult for many patients to understand. Large language models (LLMs) have the potential to simplify PEMs into more readable educational content.
Objective
We sought to evaluate whether 3 LLMs (ChatGPT [OpenAI], Gemini [Google], and Claude [Anthropic PBC]) can optimize the readability of PEMs to the recommended reading level without compromising accuracy.
Methods
This cross-sectional study used 60 randomly selected PEMs available online from 3 websites. We prompted LLMs to simplify the reading level of online PEMs. The primary outcome was the readability of the original online PEMs compared with the LLM-simplified versions. Readability scores were calculated using 4 validated indices Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook Index. Accuracy and understandability were also assessed as balancing measures, with understandability measured using the Patient Education Materials Assessment Tool-Understandability (PEMAT-U).
Results
The original readability scores for the American Heart Association (AHA), American Cancer Society (ACS), and American Stroke Association (ASA) websites were above the recommended sixth-grade level, with mean grade level scores of 10.7,10.0, and 9.6, respectively. After optimization by the LLMs, readability scores significantly improved across all 3 websites when compared with the original text. Compared with the original website, Wilcoxon signed rank test showed ChatGPT improved the readability to 7.6 from 10.1 (P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Medical Center Information Technology Department of Health Informatics, New York University Langone Health, New York, NY, United States.
- Division of Hospital Medicine, Department of Medicine, New York University Langone Health, New York, NY, United States.
- Ronald O. Perelman Department of Emergency Medicine, New York University Grossman School of Medicine, New York, NY, United States.
- Department of Medicine, New York University Long Island School of Medicine, Mineola, NY, United States.
ORCID profiles: John Will, Mahin Gupta, Jonah Zaretsky, Aliesha Dowlath, Paul Testa, Jonah Feldman
Cite
APA
Will, J., Gupta, M., Zaretsky, J., Dowlath, A., Testa, P., & Feldman, J. (2025). Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study. Journal of Medical Internet Research, 27, e69955. https://doi.org/10.2196/69955
Vancouver
Will J, Gupta M, Zaretsky J, Dowlath A, Testa P, Feldman J. Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study. Journal of Medical Internet Research. 2025;27:e69955. doi:10.2196/69955
BibTeX
@article{Will2025,
author = {Will, John and Gupta, Mahin and Zaretsky, Jonah and Dowlath, Aliesha and Testa, Paul and Feldman, Jonah},
title = {Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e69955},
doi = {10.2196/69955}
}
EUVIMED