Quality of AI‐Generated Patient Education for Pre‐ and Post‐Operative Tracheostomy Care
Otolaryngology–Head and Neck Surgery · 2026-08-04
DOI: 10.1002/ohn.70359 ↗At a glance
Journal article in Otolaryngology–Head and Neck Surgery, published online on 2026-08-04. Written by Keer Zhang and 8 co-authors. Institution: Department of Head and Neck Surgery, David Geffen School of Medicine at University of California, Los Angeles, California, USA. Major MEDLINE topics: Tracheostomy, Patient Education as Topic, Postoperative Care. 37 references.
Abstract (original language)
Objective
To evaluate the accuracy, completeness, clarity, source transparency, and readability of leading AI chatbot responses to patient questions about tracheostomy and to determine whether AI tools can reliably support patient education where high‐quality guidance is critical for safety. Study Design Cross‐sectional content analysis.
Setting
Virtual study environment using publicly accessible AI platforms, with expert evaluation conducted via Qualtrics‐based distribution.
Methods
Twelve frequently asked questions about tracheostomy care were identified using search‐listening tools and clinician input, then submitted to 5 AI chatbots — ChatGPT4, Google Gemini 2.0, Microsoft Copilot, DeepSeek V3, and Grok 3 — and to a senior laryngologist. Three blinded laryngologists independently evaluated each response using the Quality Analysis of Medical Artificial Intelligence instrument. Readability was assessed using nine metrics.
Results
Gemini 2.0 achieved significantly higher completeness scores than physician responses ( P < .001), with DeepSeek and Grok 3 ( P < .05) also outperforming ( P < .05). Accuracy did not differ significantly between AI‐ and expert‐generated responses. On average, the AI models outperformed physician in clarity, completeness, and usefulness based on QAMAI scoring ( P < .05). All AI and expert responses exceeded the NIH‐recommended 6th‐grade reading level, ranging from 10th−13th grade ( P < .001). Inter‐rater reliability was 78%.
Conclusion
AI chatbots can generate accurate and comprehensive responses to common tracheostomy care questions, demonstrating potential to support patient education. However, they continue to lack guaranteed, verifiable sourcing, and this study did not assess actual patient comprehension of the AI‐generated responses. Future efforts should focus on adapting AI‐generated education materials to meet health literacy standards and evaluating their direct impact on patient understanding and outcomes.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Head and Neck Surgery, David Geffen School of Medicine at University of California, Los Angeles, California, USA.
ORCID profiles: Keer Zhang, Lauran K Evans, Desiree Delavary, Christian Wooten, Travis L Shiba, Joshua D Feintuch, Dinesh K Chhetri
Cite
APA
Zhang, K., Evans, L. K., Delavary, D., Wooten, C., Kim, M., Shiba, T. L., Kadin, N., Feintuch, J. D., & Chhetri, D. K. (2026). Quality of AI‐Generated Patient Education for Pre‐ and Post‐Operative Tracheostomy Care. Otolaryngology–Head and Neck Surgery. https://doi.org/10.1002/ohn.70359
Vancouver
Zhang K, Evans LK, Delavary D, Wooten C, Kim M, Shiba TL, et al. Quality of AI‐Generated Patient Education for Pre‐ and Post‐Operative Tracheostomy Care. Otolaryngology–Head and Neck Surgery. 2026. doi:10.1002/ohn.70359
BibTeX
@article{Zhang2026,
author = {Zhang, Keer and Evans, Lauran K and Delavary, Desiree and Wooten, Christian and Kim, Minjae and Shiba, Travis L and Kadin, Natalie and Feintuch, Joshua D and Chhetri, Dinesh K},
title = {Quality of AI‐Generated Patient Education for Pre‐ and Post‐Operative Tracheostomy Care},
journal = {Otolaryngology–Head and Neck Surgery},
year = {2026},
doi = {10.1002/ohn.70359}
}
EUVIMED