Quality assessment of artificial intelligence‐generated versus human‐written hospital summaries evaluating detail, usefulness, and continuity of care
Journal of Hospital Medicine · 2026-04 · 21 (4) · 375-379
DOI: 10.1002/jhm.70163 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Hospital Medicine, online erschienen am 30.09.2025. Verfasst von Douglas Challener und 4 weiteren Autor:innen. Beteiligte Einrichtung: Division of Public Health, Infectious Diseases, and Occupational Medicine-Mayo Clinic, Rochester, Minnesota, USA. Hauptthemen laut MEDLINE: Artificial Intelligence, Continuity of Patient Care, Patient Discharge Summaries, Quality Assurance, Health Care. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 14 Literaturangaben.
Abstract (Originalsprache)
Background
Hospital discharge summaries are critical for ensuring continuity of care, but their quality often varies. Large language models (LLMs) have the potential to standardize and enhance the efficiency of this documentation process.
Objectives
To evaluate the quality of hospital discharge summaries created by an LLM‐based hospital course drafting tool created by Epic Systems compared with human‐written summaries.
Methods
Retrospective study at a single tertiary‐care institution in 2024. The cohort included 100 adult hospitalizations lasting >72 h across medical and surgical dismissing services. No interventions were performed. Summaries (LLM‐generated vs. human‐written) were independently reviewed using a standardized rubric covering nine domains (e.g., comprehensiveness, clarity, relevance). Scores were normalized and compared. Readability was assessed using Flesch Reading Ease.
Results
LLM‐generated summaries outperformed human‐written summaries across all criteria ( p < .05), with the greatest difference observed in comprehensiveness (LLM median 0.62 vs. human −0.23). Human‐written summaries from surgical services scored lower than those from medical services, but LLM performance was consistent across both. Human summaries had higher Flesch Reading Ease scores (33.11 vs. 26.2; p < .05), reflecting simpler language.
Conclusions
LLM‐generated summaries demonstrated superior quality, consistency, and clinical utility compared with human‐written summaries, highlighting their potential to improve documentation efficiency and standardization.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Division of Public Health, Infectious Diseases, and Occupational Medicine-Mayo Clinic, Rochester, Minnesota, USA.
- Division of Hospital Internal Medicine-Mayo Clinic, Rochester, Minnesota, USA.
- Division of Pulmonary and Critical Care Medicine-Mayo Clinic, Rochester, Minnesota, USA.
- Department of Emergency Medicine-Mayo Clinic, Rochester, Minnesota, USA.
ORCID-Profile: Douglas Challener
Zitieren
APA
Challener, D., Ayanian, S., Ryu, A., O'Horo, J., & Heaton, H. (2026). Quality assessment of artificial intelligence‐generated versus human‐written hospital summaries evaluating detail, usefulness, and continuity of care. Journal of Hospital Medicine, 21(4), 375-379. https://doi.org/10.1002/jhm.70163
Vancouver
Challener D, Ayanian S, Ryu A, O'Horo J, Heaton H. Quality assessment of artificial intelligence‐generated versus human‐written hospital summaries evaluating detail, usefulness, and continuity of care. Journal of Hospital Medicine. 2026;21(4):375-379. doi:10.1002/jhm.70163
BibTeX
@article{Challener2026,
author = {Challener, Douglas and Ayanian, Shant and Ryu, Alexander and O'Horo, John and Heaton, Heather},
title = {Quality assessment of artificial intelligence‐generated versus human‐written hospital summaries evaluating detail, usefulness, and continuity of care},
journal = {Journal of Hospital Medicine},
year = {2026},
volume = {21},
number = {4},
pages = {375--379},
doi = {10.1002/jhm.70163}
}
EUVIMED