Usage of and Satisfaction with Artificial Intelligence-Generated Draft Replies to Patient Portal Messages
Applied Clinical Informatics · 01.05.2026 · 17 (03) · 510-517
DOI: 10.1055/a-2893-7363 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Clinical Informatics, erschienen am 01.05.2026. Verfasst von Aidan P Crowley und 6 weiteren Autor:innen. Beteiligte Einrichtung: Leonard Davis Institute of Health Economics, University of Pennsylvania, Pennsylvania, United States, Philadelphia. Hauptthemen laut MEDLINE: Patient Portals, Artificial Intelligence, Patient Satisfaction. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Objectives
This study aimed to describe the usage and perceptions of physicians and nurses in primary and specialty care on the acceptability, usability, and quality of artificial intelligence (AI)-generated draft replies to patient portal messages.
Methods
We conducted a pilot study of 80 users across seven clinics within a large academic health system in Pennsylvania. Epic's Augmented Response Technology (ART) used a large language model to present clinicians with draft replies to patient messages. To measure usage, we calculated the frequency with which replies were initiated with ART-generated drafts, as well as the overlap between drafts and replies sent to patients. To measure acceptability, usability, and quality, we developed and disseminated a 15-item survey.
Results
Voluntary pilot participants included 80 users (36 physicians, 25 nurses [registered nurses/licensed practice nurses], 10 nurse practitioners, 9 medical assistants). The mean rate of starting with an ART draft was 20.2%, and 40% of replies exhibited little to no editing of the draft. Of the 52 survey respondents, 66% (n = 33) agreed that AI-generated drafts were useful, and 46% (n = 23) agreed that ART improved the quality of replies to patient messages. Across survey questions, a higher percentage of nurses reported favorable opinions of the technology than did physicians. In free-response questions, the most common themes were that the message content was helpful (28.3%, n = 13) and that it reduced perceived effort or time (23.9%, n = 11), but that drafts occasionally contained incorrect or inappropriate content (34.9%, n = 15) or too often suggested an appointment (27.9%, n = 12).
Conclusion
In this voluntary pilot study, AI-generated draft replies to patient portal messages were generally rated as acceptable and useful, with nurses and primary care clinicians more often reporting acceptance of the technology than physicians or specialty care clinicians. Scaling efforts should continue to monitor acceptability and effects on cognitive burden, in addition to safety.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Pennsylvania, United States, Philadelphia.
- The Wharton School of the University of Pennsylvania, Department of Health Care Management and Economics, Pennsylvania, United States, Philadelphia.
- Perelman School of Medicine at the University of Pennsylvania, Pennsylvania, United States, Philadelphia.
- Penn Medicine, University of Pennsylvania Health System, Pennsylvania, United States, Philadelphia.
- University of Pennsylvania Health System, Department of EHR Transformation, Pennsylvania, United States, Philadelphia.
- University of Pennsylvania Health System, Department of Emergency Medicine, Pennsylvania, United States, Philadelphia.
ORCID-Profile: Aidan P Crowley
Zitieren
APA
Crowley, A. P., Hanish, A., Lubken, J., Becker, M., Rosen, C., Moon, J., & Regli, S. H. (2026). Usage of and Satisfaction with Artificial Intelligence-Generated Draft Replies to Patient Portal Messages. Applied Clinical Informatics, 17(03), 510-517. https://doi.org/10.1055/a-2893-7363
Vancouver
Crowley AP, Hanish A, Lubken J, Becker M, Rosen C, Moon J, et al. Usage of and Satisfaction with Artificial Intelligence-Generated Draft Replies to Patient Portal Messages. Applied Clinical Informatics. 2026;17(03):510-517. doi:10.1055/a-2893-7363
BibTeX
@article{Crowley2026,
author = {Crowley, Aidan P and Hanish, Asaf and Lubken, Jason and Becker, Michael and Rosen, Carole and Moon, Jeffrey and Regli, Susan H},
title = {Usage of and Satisfaction with Artificial Intelligence-Generated Draft Replies to Patient Portal Messages},
journal = {Applied Clinical Informatics},
year = {2026},
volume = {17},
number = {03},
pages = {510--517},
doi = {10.1055/a-2893-7363}
}
EUVIMED