EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Development and evaluation of an artificial intelligence-based workflow for the prioritization of patient portal messages

Jie Yang, Jonathan So iD, Hao Zhang iD, Simon Jones, Denise M Connolly, Claudia Golding, Esmelin Griffes, Adam C Szerencsy, Tzer Jason Wu, Yindalon Aphinyanaphongs, Vincent J Major iD

JAMIA Open · 01.07.2024 · 7 (3)

DOI: 10.1093/jamiaopen/ooae078 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 16.08.2024. Verfasst von Jie Yang und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Informatics, NYU Langone Health, New York, NY 10016, United States. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 8 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NCATS NIH HHS, National Center for Advancing Translational Sciences, National Institutes of Health.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Accelerating demand for patient messaging has impacted the practice of many providers. Messages are not recommended for urgent medical issues, but some do require rapid attention. This presents an opportunity for artificial intelligence (AI) methods to prioritize review of messages. Our study aimed to highlight some patient portal messages for prioritized review using a custom AI system integrated into the electronic health record (EHR). Materials and Methods We developed a Bidirectional Encoder Representations from Transformers (BERT)-based large language model using 40 132 patient-sent messages to identify patterns involving high acuity topics that warrant an immediate callback. The model was then implemented into 2 shared pools of patient messages managed by dozens of registered nurses. A primary outcome, such as the time before messages were read, was evaluated with a difference-in-difference methodology.

Results

Model validation on an expert-reviewed dataset (n = 7260) yielded very promising performance (C-statistic = 97%, average-precision = 72%). A binarized output (precision = 67%, sensitivity = 63%) was integrated into the EHR for 2 years. In a pre-post analysis (n = 396 466), an improvement exceeding the trend was observed in the time high-scoring messages sit unread (21 minutes, 63 vs 42 for messages sent outside business hours).

Discussion

Our work shows great promise in improving care when AI is aligned with human workflow. Future work involves audience expansion, aiding users with suggested actions, and drafting responses.

Conclusion

Many patients utilize patient portal messages, and while most messages are routine, a small fraction describe alarming symptoms. Our AI-based workflow shortens the turnaround time to get a trained clinician to review these messages to provide safer, higher-quality care.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligencenatural language processingpatient portalssupervised machine-learningtriage

Autor:innen und Institutionen

  • Department of Health Informatics, NYU Langone Health, New York, NY 10016, United States.
  • Department of Population Health, NYU Grossman School of Medicine, New York, NY 10016, United States.
  • Clinical Access Center, NYU Langone Health, New York, NY 10016, United States.
  • Department of Medicine, NYU Grossman School of Medicine, New York, NY 10016, United States.

ORCID-Profile: Jonathan So, Hao Zhang, Vincent J Major

Förderung

  • NCATS NIH HHS
  • National Center for Advancing Translational Sciences
  • National Institutes of Health

Zitieren

APA

Yang, J., So, J., Zhang, H., Jones, S., Connolly, D. M., Golding, C., Griffes, E., Szerencsy, A. C., Wu, T. J., Aphinyanaphongs, Y., & Major, V. J. (2024). Development and evaluation of an artificial intelligence-based workflow for the prioritization of patient portal messages. JAMIA Open, 7(3). https://doi.org/10.1093/jamiaopen/ooae078

Vancouver

Yang J, So J, Zhang H, Jones S, Connolly DM, Golding C, et al. Development and evaluation of an artificial intelligence-based workflow for the prioritization of patient portal messages. JAMIA Open. 2024;7(3). doi:10.1093/jamiaopen/ooae078

BibTeX
@article{Yang2024,
  author = {Yang, Jie and So, Jonathan and Zhang, Hao and Jones, Simon and Connolly, Denise M and Golding, Claudia and Griffes, Esmelin and Szerencsy, Adam C and Wu, Tzer Jason and Aphinyanaphongs, Yindalon and Major, Vincent J},
  title = {Development and evaluation of an artificial intelligence-based workflow for the prioritization of patient portal messages},
  journal = {JAMIA Open},
  year = {2024},
  volume = {7},
  number = {3},
  doi = {10.1093/jamiaopen/ooae078}
}
Kostenlos starten →