Automating Responses to Patient Portal Messages Using Generative AI
Applied Clinical Informatics · 2025-05 · 16 (03) · 718-731
DOI: 10.1055/a-2565-9155 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Clinical Informatics, online erschienen am 30.07.2025. Verfasst von Amarpreet Kaur und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania. Hauptthemen laut MEDLINE: Patient Portals, Artificial Intelligence, Automation. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, NIH HHS, National Institutes of Health.
Abstract (Originalsprache)
Patient portals bridge patient and provider communications but exacerbate physician and nursing burnout. Large language models (LLMs) can generate message responses that are viewed favorably by health care professionals/providers (HCPs); however, these studies have not included diverse message types or new prompt-engineering strategies. Our goal is to investigate and compare the quality and precision of GPT-generated message responses versus real doctor responses across the spectrum of message types within a patient portal. We used prompt engineering techniques to craft synthetic provider responses tailored to adult primary care patients. We enrolled a sample of primary care providers in a cross-sectional study to compare authentic with synthetic patient portal message responses generated by GPT-3.5-turbo, July 2023 version (GPT). The survey assessed each response's empathy, relevance, medical accuracy, and readability on a scale from 0 to 5. Respondents were asked to identify responses that were GPT-generated versus provider-generated. Mean scores for all metrics were computed for subsequent analysis. A total of 49 HCPs participated in the survey (59% completion rate), comprising 16 physicians and 32 advanced practice providers (APPs). In comparison to responses generated by real doctors, GPT-generated responses scored statistically significantly higher than doctors in two of the four parameters: empathy (p < 0.05) and readability (p < 0.05). However, no statistically significant difference was observed for relevance and accuracy (p > 0.05). Although readability scores were significantly different, the absolute difference was small, and the clinical significance of this finding remains uncertain. Our findings affirm the potential of GPT-generated message responses to achieve comparable levels of empathy, relevance, and readability to those found in typical responses crafted by HCPs. Additional studies should be done within provider workflows and with careful evaluation of patient attitudes and concerns related to the ethics as well as the quality of generated responses in all settings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.
- School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, Pennsylvania.
- Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee.
Förderung
- NLM NIH HHS
- NIH HHS
- National Institutes of Health
Zitieren
APA
Kaur, A., Budko, A., Liu, K., Eaton, E., Steitz, B. D., & Johnson, K. B. (2025). Automating Responses to Patient Portal Messages Using Generative AI. Applied Clinical Informatics, 16(03), 718-731. https://doi.org/10.1055/a-2565-9155
Vancouver
Kaur A, Budko A, Liu K, Eaton E, Steitz BD, Johnson KB. Automating Responses to Patient Portal Messages Using Generative AI. Applied Clinical Informatics. 2025;16(03):718-731. doi:10.1055/a-2565-9155
BibTeX
@article{Kaur2025,
author = {Kaur, Amarpreet and Budko, Alexander and Liu, Katrina and Eaton, Eric and Steitz, Bryan D and Johnson, Kevin B},
title = {Automating Responses to Patient Portal Messages Using Generative AI},
journal = {Applied Clinical Informatics},
year = {2025},
volume = {16},
number = {03},
pages = {718--731},
doi = {10.1055/a-2565-9155}
}
EUVIMED