EUVIMEDSkopri EUVIMED
Open access

Readability of AI-Generated Patient Visit Summaries in Orthopedic Surgery: Retrospective Analysis

Edward Lee Major Iii, Vivek P Shah, Amber N Carroll, James R Goetz, Chaitu Malempati, Sameer Badarudeen

Journal of Medical Internet Research · 2026-08-20 · 28 · e87283-e87283

DOI: 10.2196/87283 ↗

At a glance

Journal article in Journal of Medical Internet Research, published online on 2026-08-20. Written by Edward Lee Major Iii and 5 co-authors. Institution: Department of Orthopaedic Surgery and Sports Medicine, College of Medicine, University of Kentucky, 740 S Limestone St, Lexington, KY, United States,…. Major MEDLINE topics: Comprehension, Health Literacy, Orthopedic Procedures, Artificial Intelligence. 33 references. The full text is freely available in PubMed Central.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Patient visit summaries (PVS) are patient-facing documents intended to reinforce communication and promote patient education after clinical encounters. Despite national recommendations that patient education materials be written at or below a sixth-grade reading level, most orthopedic materials substantially exceed this threshold. Current visit summaries are also time-consuming to generate, lack personalization, and often fail to meet patient literacy needs. AI-based scribes can generate personalized PVS in real time directly from patient-provider conversations, offering a potential solution.

Objective

This study aimed to evaluate the readability of AI-generated PVS produced by a commercial AI scribe platform in an orthopedic surgery setting and determine their alignment with established literacy standards for patient-facing materials.

Methods

A total of 1007 consecutive AI-generated PVS from an academic orthopedic surgery outpatient clinic between December 2023 and May 2024 were reviewed. Following standardized preprocessing, including restoration of original section headings and removal of diagnosis label headers, summaries identified as incomplete were excluded (n=25), yielding a final study cohort of 982 summaries. Readability was assessed using 5 validated indices: the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score (FRES), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Index. The proportions of PVS meeting the sixth- and eighth-grade benchmarks were calculated. Spearman’s rank correlation (ρ) assessed associations between word count and readability metrics. The Kendall Coefficient of Concordance ( W ) was used to evaluate agreement among indices after reverse-coding FRES for directional alignment. Statistical significance was set at P

Subjects

MeSH terms (MEDLINE)

ComprehensionHealth LiteracyOrthopedic ProceduresArtificial IntelligenceOrthopedicsHumansRetrospective StudiesPatient Education as Topic

Author keywords

AIartificial intelligencehealth literacypatient education materialspatient visit summaryreadability

Authors and institutions

  • Department of Orthopaedic Surgery and Sports Medicine, College of Medicine, University of Kentucky, 740 S Limestone St, Lexington, KY, United States, 1 323 873 4940.
  • Department of Orthopaedic Surgery, McLaren Regional Medical Center, Flint, MI, United States.
  • Department of Orthopaedic Surgery, Indiana University, Indianapolis, IN, United States.

Cite

APA

Major Iii, E. L., Shah, V. P., Carroll, A. N., Goetz, J. R., Malempati, C., & Badarudeen, S. (2026). Readability of AI-Generated Patient Visit Summaries in Orthopedic Surgery: Retrospective Analysis. Journal of Medical Internet Research, 28, e87283-e87283. https://doi.org/10.2196/87283

Vancouver

Major Iii EL, Shah VP, Carroll AN, Goetz JR, Malempati C, Badarudeen S. Readability of AI-Generated Patient Visit Summaries in Orthopedic Surgery: Retrospective Analysis. Journal of Medical Internet Research. 2026;28:e87283-e87283. doi:10.2196/87283

BibTeX
@article{MajorIii2026,
  author = {Major Iii, Edward Lee and Shah, Vivek P and Carroll, Amber N and Goetz, James R and Malempati, Chaitu and Badarudeen, Sameer},
  title = {Readability of AI-Generated Patient Visit Summaries in Orthopedic Surgery: Retrospective Analysis},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e87283--e87283},
  doi = {10.2196/87283}
}
Start for free →