A Blinded Analysis of Quality and Fidelity in Orthopaedic Patient Education Materials Simplified by ChatGPT and Humans
Journal of Bone and Joint Surgery · 2026
DOI: 10.2106/jbjs.25.00982 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Bone and Joint Surgery, 2026. Verfasst von Joseph E Nassar und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedics, Warren Alpert Medical School of Brown University, East Providence, Rhode Island. Hauptthemen laut MEDLINE: Patient Education as Topic, Comprehension, Orthopedics, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 40 Literaturangaben.
Themen: Selbstmanagement bei chronischen Erkrankungen, Künstliche Intelligenz in der Pflege
Abstract (Originalsprache)
Background
Orthopaedic patient education materials (PEMs) within Epic’s Elsevier library often exceed the recommended sixth-grade reading level, with a mean grade of 8.6 in English and 5.8 in Spanish, risking poor patient comprehension and adherence. The present study evaluated whether artificial intelligence (AI)-based text simplification can improve readability while preserving clinical accuracy. The objectives were to use previously established readability data for English and Spanish PEMs as baselines, to assess the impact of human-based and ChatGPT-based simplification on reading grade level, and to compare the fidelity of simplified texts against standard materials.
Methods
In March 2025, 806 orthopaedic PEM documents were simplified using standardized ChatGPT prompts. Readability was reassessed using validated English and Spanish formulas, and fidelity was evaluated in the 86 PEMs that also had human easy-to-read versions. Two blinded clinicians compared human and ChatGPT-4o outputs with the originals to identify hallucinations, omissions, and inconsistencies according to severity. Following the release of ChatGPT-5, an unblinded post hoc analysis was performed using identical criteria.
Results
ChatGPT-4o-simplified PEMs showed mean reading grade levels of 6.1 in English and 3.5 in Spanish. Compared with human simplifications, ChatGPT-4o showed fewer English omissions, similar Spanish omissions, fewer inconsistencies in both languages, and comparable English hallucinations, but higher Spanish hallucinations. Compared with ChatGPT-4o, ChatGPT-5 preserved English performance and improved Spanish fidelity, reducing hallucinations to human-comparable rates.
Conclusions
AI-driven simplification can produce orthopaedic PEMs that are easier to read while maintaining acceptable fidelity. The improvements observed with ChatGPT-5 highlight its potential for clinician-supervised use in generating accessible and reliable PEMs. Clinical Relevance: This study is clinically relevant because orthopaedic PEMs are routinely delivered through the Epic electronic health record and directly affect patient understanding, consent, and adherence in both English and Spanish. By evaluating the readability and fidelity of AI-simplified materials across languages, this study informs safe, scalable strategies to improve patient communication in everyday orthopaedic practice.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Orthopedics, Warren Alpert Medical School of Brown University, East Providence, Rhode Island.
ORCID-Profile: Joseph E Nassar, Maxwell Sahhar, Lama A Ammar, Joseph Carroll, Anne-Emilie Rouffiac, Marco Kaper, Alex Hernandez-Manriquez, Manjot Singh, Edward Akelman, Alan H Daniels
Zitieren
APA
Nassar, J. E., Sahhar, M., Ammar, L. A., Carroll, J., Rouffiac, A. E., Kaper, M., Hernandez-Manriquez, A., Singh, M., Akelman, E., & Daniels, A. H. (2026). A Blinded Analysis of Quality and Fidelity in Orthopaedic Patient Education Materials Simplified by ChatGPT and Humans. Journal of Bone and Joint Surgery. https://doi.org/10.2106/jbjs.25.00982
Vancouver
Nassar JE, Sahhar M, Ammar LA, Carroll J, Rouffiac AE, Kaper M, et al. A Blinded Analysis of Quality and Fidelity in Orthopaedic Patient Education Materials Simplified by ChatGPT and Humans. Journal of Bone and Joint Surgery. 2026. doi:10.2106/jbjs.25.00982
BibTeX
@article{Nassar2026,
author = {Nassar, Joseph E and Sahhar, Maxwell and Ammar, Lama A and Carroll, Joseph and Rouffiac, Anne-Emilie and Kaper, Marco and Hernandez-Manriquez, Alex and Singh, Manjot and Akelman, Edward and Daniels, Alan H},
title = {A Blinded Analysis of Quality and Fidelity in Orthopaedic Patient Education Materials Simplified by ChatGPT and Humans},
journal = {Journal of Bone and Joint Surgery},
year = {2026},
doi = {10.2106/jbjs.25.00982}
}
EUVIMED