Using large language model to guide patients to create efficient and comprehensive clinical care message
Journal of the American Medical Informatics Association · 2024
DOI: 10.1093/jamia/ocae142 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Siru Liu und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 45-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, NIA NIH HHS, NIH HHS.
Abstract (Originalsprache)
Objective
This study aims to investigate the feasibility of using Large Language Models (LLMs) to engage with patients at the time they are drafting a question to their healthcare providers, and generate pertinent follow-up questions that the patient can answer before sending their message, with the goal of ensuring that their healthcare provider receives all the information they need to safely and accurately answer the patient’s question, eliminating back-and-forth messaging, and the associated delays and frustrations.
Methods
We collected a dataset of patient messages sent between January 1, 2022 to March 7, 2023 at Vanderbilt University Medical Center. Two internal medicine physicians identified 7 common scenarios. We used 3 LLMs to generate follow-up questions: (1) Comprehensive LLM Artificial Intelligence Responder (CLAIR): a locally fine-tuned LLM, (2) GPT4 with a simple prompt, and (3) GPT4 with a complex prompt. Five physicians rated them with the actual follow-ups written by healthcare providers on clarity, completeness, conciseness, and utility.
Results
For five scenarios, our CLAIR model had the best performance. The GPT4 model received higher scores for utility and completeness but lower scores for clarity and conciseness. CLAIR generated follow-up questions with similar clarity and conciseness as the actual follow-ups written by healthcare providers, with higher utility than healthcare providers and GPT4, and lower completeness than GPT4, but better than healthcare providers.
Conclusion
LLMs can generate follow-up patient messages designed to clarify a medical question that compares favorably to those generated by healthcare providers.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
- Department of Computer Science, Vanderbilt University, Nashville, TN 37212, United States.
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
- Department of Pediatric Endocrinology, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
ORCID-Profile: Allison B Mccoy, Adam Wright
Förderung
- NLM NIH HHS
- NIA NIH HHS
- NIH HHS
- NIH
Zitieren
APA
Liu, S., Wright, A. P., Mccoy, A. B., Huang, S. S., Genkins, J. Z., Peterson, J. F., Kumah-Crystal, Y. A., Martinez, W., Carew, B., Mize, D., Steitz, B., & Wright, A. (2024). Using large language model to guide patients to create efficient and comprehensive clinical care message. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae142
Vancouver
Liu S, Wright AP, Mccoy AB, Huang SS, Genkins JZ, Peterson JF, et al. Using large language model to guide patients to create efficient and comprehensive clinical care message. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae142
BibTeX
@article{Liu2024,
author = {Liu, Siru and Wright, Aileen P and Mccoy, Allison B and Huang, Sean S and Genkins, Julian Z and Peterson, Josh F and Kumah-Crystal, Yaa A and Martinez, William and Carew, Babatunde and Mize, Dara and Steitz, Bryan and Wright, Adam},
title = {Using large language model to guide patients to create efficient and comprehensive clinical care message},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
doi = {10.1093/jamia/ocae142}
}
EUVIMED