EUVIMEDEUVIMED entdecken
Open Access

Artificial Intelligence–Generated Draft Replies to Patient Inbox Messages

Patricia Garcia, Stephen P Ma, Shreya Shah, Margaret Smith, Yejin Jeong, Anna Devon-Sand, Ming Tai-Seale, Kevin Takazawa, Danyelle Clutter, Kyle Vogt, Carlene Lugtu, Matthew Rojo und 4 weitere

JAMA Network Open · 20.03.2024 · 7 (3) · e243201

DOI: 10.1001/jamanetworkopen.2024.3201 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMA Network Open, online erschienen am 20.03.2024. Verfasst von Patricia Garcia und 15 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Stanford University School of Medicine, Stanford, California. Hauptthemen laut MEDLINE: Artificial Intelligence, Academic Medical Centers. Die Arbeit wurde bisher 295-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Importance The emergence and promise of generative artificial intelligence (AI) represent a turning point for health care. Rigorous evaluation of generative AI deployment in clinical practice is needed to inform strategic decision-making.

Objective

To evaluate the implementation of a large language model used to draft responses to patient messages in the electronic inbox. Design, Setting, and Participants A 5-week, prospective, single-group quality improvement study was conducted from July 10 through August 13, 2023, at a single academic medical center (Stanford Health Care). All attending physicians, advanced practice practitioners, clinic nurses, and clinical pharmacists from the Divisions of Primary Care and Gastroenterology and Hepatology were enrolled in the pilot.

Intervention

Draft replies to patient portal messages generated by a Health Insurance Portability and Accountability Act–compliant electronic health record–integrated large language model. Main Outcomes and Measures The primary outcome was AI-generated draft reply utilization as a percentage of total patient message replies. Secondary outcomes included changes in time measures and clinician experience as assessed by survey.

Results

A total of 197 clinicians were enrolled in the pilot; 35 clinicians who were prepilot beta users, out of office, or not tied to a specific ambulatory clinic were excluded, leaving 162 clinicians included in the analysis. The survey analysis cohort consisted of 73 participants (45.1%) who completed both the presurvey and postsurvey. In gastroenterology and hepatology, there were 58 physicians and APPs and 10 nurses. In primary care, there were 83 physicians and APPs, 4 nurses, and 8 clinical pharmacists. The mean AI-generated draft response utilization rate across clinicians was 20%. There was no change in reply action time, write time, or read time between the prepilot and pilot periods. There were statistically significant reductions in the 4-item physician task load score derivative (mean [SD], 61.31 [17.23] presurvey vs 47.26 [17.11] postsurvey; paired difference, −13.87; 95% CI, −17.38 to −9.50; P < .001) and work exhaustion scores (mean [SD], 1.95 [0.79] presurvey vs 1.62 [0.68] postsurvey; paired difference, −0.33; 95% CI, −0.50 to −0.17; P < .001). Conclusions and Relevance In this quality improvement study of an early implementation of generative AI, there was notable adoption, usability, and improvement in assessments of burden and burnout. There was no improvement in time. Further code-to-bedside testing is needed to guide future development and organizational strategy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceAcademic Medical CentersUnited StatesHumansProspective StudiesAmbulatory Care FacilitiesBurnout, Psychological

Autor:innen und Institutionen

  • Department of Medicine, Stanford University School of Medicine, Stanford, California.
  • Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Stanford University School of Medicine, Stanford, California.
  • Department of Family Medicine, University of California San Diego School of Medicine, La Jolla.
  • Technology and Digital Solutions, Stanford Medicine, Stanford, California.
  • Nursing Informatics & Innovation, Stanford Healthcare, Stanford, California.
  • WellMD Center, Stanford University School of Medicine, Stanford, California.

Zitieren

APA

Garcia, P., Ma, S. P., Shah, S., Smith, M., Jeong, Y., Devon-Sand, A., Tai-Seale, M., Takazawa, K., Clutter, D., Vogt, K., Lugtu, C., Rojo, M., Lin, S., Shanafelt, T., Pfeffer, M. A., & Sharp, C. (2024). Artificial Intelligence–Generated Draft Replies to Patient Inbox Messages. JAMA Network Open, 7(3), e243201. https://doi.org/10.1001/jamanetworkopen.2024.3201

Vancouver

Garcia P, Ma SP, Shah S, Smith M, Jeong Y, Devon-Sand A, et al. Artificial Intelligence–Generated Draft Replies to Patient Inbox Messages. JAMA Network Open. 2024;7(3):e243201. doi:10.1001/jamanetworkopen.2024.3201

BibTeX
@article{Garcia2024,
  author = {Garcia, Patricia and Ma, Stephen P and Shah, Shreya and Smith, Margaret and Jeong, Yejin and Devon-Sand, Anna and Tai-Seale, Ming and Takazawa, Kevin and Clutter, Danyelle and Vogt, Kyle and Lugtu, Carlene and Rojo, Matthew and Lin, Steven and Shanafelt, Tait and Pfeffer, Michael A and Sharp, Christopher},
  title = {Artificial Intelligence–Generated Draft Replies to Patient Inbox Messages},
  journal = {JAMA Network Open},
  year = {2024},
  volume = {7},
  number = {3},
  pages = {e243201},
  doi = {10.1001/jamanetworkopen.2024.3201}
}
Kostenlos starten →