EUVIMEDOpdag EUVIMED
QuerschnittstudieOpen access

Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study

John Will iD, Mahin Gupta iD, Jonah Zaretsky iD, Aliesha Dowlath iD, Paul Testa iD, Jonah Feldman iD

Journal of Medical Internet Research · 2025-06-04 · 27 · e69955

DOI: 10.2196/69955 ↗

At a glance

Querschnittstudie in Journal of Medical Internet Research, published online on 2025-06-04. Written by John Will and 5 co-authors. Institution: Medical Center Information Technology Department of Health Informatics, New York University Langone Health, New York, NY, United States. Major MEDLINE topics: Patient Education as Topic, Internet, Comprehension, Language. Cited 139 times so far (Crossref); 43 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Online accessible patient education materials (PEMs) are essential for patient empowerment. However, studies have shown that these materials often exceed the recommended sixth-grade reading level, making them difficult for many patients to understand. Large language models (LLMs) have the potential to simplify PEMs into more readable educational content.

Objective

We sought to evaluate whether 3 LLMs (ChatGPT [OpenAI], Gemini [Google], and Claude [Anthropic PBC]) can optimize the readability of PEMs to the recommended reading level without compromising accuracy.

Methods

This cross-sectional study used 60 randomly selected PEMs available online from 3 websites. We prompted LLMs to simplify the reading level of online PEMs. The primary outcome was the readability of the original online PEMs compared with the LLM-simplified versions. Readability scores were calculated using 4 validated indices Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, and Simple Measure of Gobbledygook Index. Accuracy and understandability were also assessed as balancing measures, with understandability measured using the Patient Education Materials Assessment Tool-Understandability (PEMAT-U).

Results

The original readability scores for the American Heart Association (AHA), American Cancer Society (ACS), and American Stroke Association (ASA) websites were above the recommended sixth-grade level, with mean grade level scores of 10.7,10.0, and 9.6, respectively. After optimization by the LLMs, readability scores significantly improved across all 3 websites when compared with the original text. Compared with the original website, Wilcoxon signed rank test showed ChatGPT improved the readability to 7.6 from 10.1 (P

Subjects

MeSH terms (MEDLINE)

Patient Education as TopicInternetComprehensionLanguageCross-Sectional StudiesHumansReadingHealth LiteracyLarge Language Models

Author keywords

artificial intelligencehealth educationhealth literacypatient educationreadability

Authors and institutions

  • Medical Center Information Technology Department of Health Informatics, New York University Langone Health, New York, NY, United States.
  • Division of Hospital Medicine, Department of Medicine, New York University Langone Health, New York, NY, United States.
  • Ronald O. Perelman Department of Emergency Medicine, New York University Grossman School of Medicine, New York, NY, United States.
  • Department of Medicine, New York University Long Island School of Medicine, Mineola, NY, United States.

ORCID profiles: John Will, Mahin Gupta, Jonah Zaretsky, Aliesha Dowlath, Paul Testa, Jonah Feldman

Cite

APA

Will, J., Gupta, M., Zaretsky, J., Dowlath, A., Testa, P., & Feldman, J. (2025). Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study. Journal of Medical Internet Research, 27, e69955. https://doi.org/10.2196/69955

Vancouver

Will J, Gupta M, Zaretsky J, Dowlath A, Testa P, Feldman J. Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study. Journal of Medical Internet Research. 2025;27:e69955. doi:10.2196/69955

BibTeX
@article{Will2025,
  author = {Will, John and Gupta, Mahin and Zaretsky, Jonah and Dowlath, Aliesha and Testa, Paul and Feldman, Jonah},
  title = {Enhancing the Readability of Online Patient Education Materials Using Large Language Models: Cross-Sectional Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e69955},
  doi = {10.2196/69955}
}
Start for free →