Selective prediction for extracting unstructured clinical data
Journal of the American Medical Informatics Association · 22.12.2023 · 31 (1) · 188-197
DOI: 10.1093/jamia/ocad182 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 28.09.2023. Verfasst von Akshay Swaminathan und 19 weiteren Autor:innen. Beteiligte Einrichtung: Stanford University School of Medicine, Stanford, CA, United States. Hauptthemen laut MEDLINE: Support Vector Machine, Adenocarcinoma. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NIAID NIH HHS, National Institute of Drug Abuse Clinical Trials Network, Tuolc Inc, Roche Inc.
Abstract (Originalsprache)
Objective
While there are currently approaches to handle unstructured clinical data, such as manual abstraction and structured proxy variables, these methods may be time-consuming, not scalable, and imprecise. This article aims to determine whether selective prediction, which gives a model the option to abstain from generating a prediction, can improve the accuracy and efficiency of unstructured clinical data abstraction. Materials and Methods We trained selective classifiers (logistic regression, random forest, support vector machine) to extract 5 variables from clinical notes: depression (n = 1563), glioblastoma (GBM, n = 659), rectal adenocarcinoma (DRA, n = 601), and abdominoperineal resection (APR, n = 601) and low anterior resection (LAR, n = 601) of adenocarcinoma. We varied the cost of false positives (FP), false negatives (FN), and abstained notes and measured total misclassification cost.
Results
The depression selective classifiers abstained on anywhere from 0% to 97% of notes, and the change in total misclassification cost ranged from −58% to 9%. Selective classifiers abstained on 5%–43% of notes across the GBM and colorectal cancer models. The GBM selective classifier abstained on 43% of notes, which led to improvements in sensitivity (0.94 to 0.96), specificity (0.79 to 0.96), PPV (0.89 to 0.98), and NPV (0.88 to 0.91) when compared to a non-selective classifier and when compared to structured proxy variables.
Discussion
We showed that selective classifiers outperformed both non-selective classifiers and structured proxy variables for extracting data from unstructured clinical notes.
Conclusion
Selective prediction should be considered when abstaining is preferable to making an incorrect prediction.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Stanford University School of Medicine, Stanford, CA, United States.
- Cerebral Inc. Claymont, DE, United States.
- Department of Biology, Stanford University, Stanford, CA, United States.
- Department of Bioengineering, Stanford University, Stanford, CA, United States.
- Department of Computer Science, Stanford University, Stanford, CA, United States.
- Department of Management Science and Engineering, Stanford University, Stanford, CA, United States.
ORCID-Profile: Akshay Swaminathan, William Wang, Olivier Gevaert
Förderung
- NIAID NIH HHS
- National Institute of Drug Abuse Clinical Trials Network, Tuolc Inc, Roche Inc
Zitieren
APA
Swaminathan, A., Lopez, I., Wang, W., Srivastava, U., Tran, E., Bhargava-Shah, A., Wu, J. Y., Ren, A. L., Caoili, K., Bui, B., Alkhani, L., Lee, S., Mohit, N., Seo, N., Macedo, N., Cheng, W., Liu, C., Thomas, R., Chen, J. H., & Gevaert, O. (2023). Selective prediction for extracting unstructured clinical data. Journal of the American Medical Informatics Association, 31(1), 188-197. https://doi.org/10.1093/jamia/ocad182
Vancouver
Swaminathan A, Lopez I, Wang W, Srivastava U, Tran E, Bhargava-Shah A, et al. Selective prediction for extracting unstructured clinical data. Journal of the American Medical Informatics Association. 2023;31(1):188-197. doi:10.1093/jamia/ocad182
BibTeX
@article{Swaminathan2023,
author = {Swaminathan, Akshay and Lopez, Ivan and Wang, William and Srivastava, Ujwal and Tran, Edward and Bhargava-Shah, Aarohi and Wu, Janet Y and Ren, Alexander L and Caoili, Kaitlin and Bui, Brandon and Alkhani, Layth and Lee, Susan and Mohit, Nathan and Seo, Noel and Macedo, Nicholas and Cheng, Winson and Liu, Charles and Thomas, Reena and Chen, Jonathan H and Gevaert, Olivier},
title = {Selective prediction for extracting unstructured clinical data},
journal = {Journal of the American Medical Informatics Association},
year = {2023},
volume = {31},
number = {1},
pages = {188--197},
doi = {10.1093/jamia/ocad182}
}
EUVIMED