Clinical risk prediction using language models: benefits and considerations
Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1856-1864
DOI: 10.1093/jamia/ocae030 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 27.02.2024. Verfasst von Angeela Acharya und 7 weiteren Autor:innen. Beteiligte Einrichtung: George Mason University, Fairfax, VA, United States. Hauptthemen laut MEDLINE: Electronic Health Records, Machine Learning. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NSF, Office of Research Computing, George Mason University.
Abstract (Originalsprache)
Objective
The use of electronic health records (EHRs) for clinical risk prediction is on the rise. However, in many practical settings, the limited availability of task-specific EHR data can restrict the application of standard machine learning pipelines. In this study, we investigate the potential of leveraging language models (LMs) as a means to incorporate supplementary domain knowledge for improving the performance of various EHR-based risk prediction tasks.
Methods
We propose two novel LM-based methods, namely “LLaMA2-EHR” and “Sent-e-Med.” Our focus is on utilizing the textual descriptions within structured EHRs to make risk predictions about future diagnoses. We conduct a comprehensive comparison with previous approaches across various data types and sizes.
Results
Experiments across 6 different methods and 3 separate risk prediction tasks reveal that employing LMs to represent structured EHRs, such as diagnostic histories, results in significant performance improvements when evaluated using standard metrics such as area under the receiver operating characteristic (ROC) curve and precision-recall (PR) curve. Additionally, they offer benefits such as few-shot learning, the ability to handle previously unseen medical concepts, and adaptability to various medical vocabularies. However, it is noteworthy that outcomes may exhibit sensitivity to a specific prompt.
Conclusion
LMs encompass extensive embedded knowledge, making them valuable for the analysis of EHRs in the context of risk prediction. Nevertheless, it is important to exercise caution in their application, as ongoing safety concerns related to LMs persist and require continuous consideration.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- George Mason University, Fairfax, VA, United States.
- Staten Island Performing Provider System, Staten Island, NY, United States.
Förderung
- NSF
- Office of Research Computing
- George Mason University
Zitieren
APA
Acharya, A., Shrestha, S., Chen, A., Conte, J., Avramovic, S., Sikdar, S., Anastasopoulos, A., & Das, S. (2024). Clinical risk prediction using language models: benefits and considerations. Journal of the American Medical Informatics Association, 31(9), 1856-1864. https://doi.org/10.1093/jamia/ocae030
Vancouver
Acharya A, Shrestha S, Chen A, Conte J, Avramovic S, Sikdar S, et al. Clinical risk prediction using language models: benefits and considerations. Journal of the American Medical Informatics Association. 2024;31(9):1856-1864. doi:10.1093/jamia/ocae030
BibTeX
@article{Acharya2024,
author = {Acharya, Angeela and Shrestha, Sulabh and Chen, Anyi and Conte, Joseph and Avramovic, Sanja and Sikdar, Siddhartha and Anastasopoulos, Antonios and Das, Sanmay},
title = {Clinical risk prediction using language models: benefits and considerations},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
volume = {31},
number = {9},
pages = {1856--1864},
doi = {10.1093/jamia/ocae030}
}
EUVIMED