Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation
Journal of Medical Internet Research · 2023
DOI: 10.2196/43153 ↗At a glance
Journal article in Journal of Medical Internet Research, 2023. Written by Zhi-Geng Jin and 5 co-authors. Institution: Department of Pulmonary Vascular and Thrombotic Disease, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China. Major MEDLINE topics: Venous Thromboembolism, Venous Thrombosis, Decision Support Systems, Clinical, Pulmonary Embolism. Cited 14 times so far (Crossref); 32 references. The full text is freely available in PubMed Central.
Topics: Thromboseprophylaxe
Abstract (original language)
Background
It remains unknown whether capturing data from electronic health records (EHRs) using natural language processing (NLP) can improve venous thromboembolism (VTE) detection in different clinical settings.
Objective
The aim of this study was to validate the NLP algorithm in a clinical decision support system for VTE risk assessment and integrated care (DeVTEcare) to identify VTEs from EHRs.
Methods
All inpatients aged ≥18 years in the Sixth Medical Center of the Chinese People's Liberation Army General Hospital from January 1 to December 31, 2021, were included as the validation cohort. The sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR–, respectively), area under the receiver operating characteristic curve (AUC), and F1-scores along with their 95% CIs were used to analyze the performance of the NLP tool, with manual review of medical records as the reference standard for detecting deep vein thrombosis (DVT) and pulmonary embolism (PE). The primary end point was the performance of the NLP approach embedded into the EHR for VTE identification. The secondary end points were the performances to identify VTE among different hospital departments with different VTE risks. Subgroup analyses were performed among age, sex, and the study season.
Results
Among 30,152 patients (median age 56 [IQR 41-67] years; 14,247/30,152, 47.3% females), the prevalence of VTE, PE, and DVT was 2.1% (626/30,152), 0.6% (177/30,152), and 1.8% (532/30,152), respectively. The sensitivity, specificity, LR+, LR–, AUC, and F1-score of NLP-facilitated VTE detection were 89.9% (95% CI 87.3%-92.2%), 99.8% (95% CI 99.8%-99.9%), 483 (95% CI 370-629), 0.10 (95% CI 0.08-0.13), 0.95 (95% CI 0.94-0.96), and 0.90 (95% CI 0.90-0.91), respectively. Among departments of surgery, internal medicine, and intensive care units, the highest specificity (100% vs 99.7% vs 98.8%, respectively), LR+ (3202 vs 321 vs 77, respectively), and F1-score (0.95 vs 0.89 vs 0.92, respectively) were in the surgery department (all P89% AUC and F1-score. The NLP algorithm performed better among patients aged ≤65 years than among those aged >65 years (F1-score 0.93 vs 0.89, respectively; P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Pulmonary Vascular and Thrombotic Disease, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China.
- Chinese People's Liberation Army Medical School, Beijing, China.
- Quality Management Division, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China.
- Institute for Hospital Management Research, Chinese People's Liberation Army General Hospital, Beijing, China.
ORCID profiles: Zhi-Geng Jin, Hui Zhang, Mei-Hui Tai, Ying Yang, Yuan Yao, Yu-Tao Guo
Cite
APA
Jin, Z. G., Zhang, H., Tai, M. H., Yang, Y., Yao, Y., & Guo, Y. T. (2023). Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation. Journal of Medical Internet Research. https://doi.org/10.2196/43153
Vancouver
Jin ZG, Zhang H, Tai MH, Yang Y, Yao Y, Guo YT. Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation. Journal of Medical Internet Research. 2023. doi:10.2196/43153
BibTeX
@article{Jin2023,
author = {Jin, Zhi-Geng and Zhang, Hui and Tai, Mei-Hui and Yang, Ying and Yao, Yuan and Guo, Yu-Tao},
title = {Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation},
journal = {Journal of Medical Internet Research},
year = {2023},
doi = {10.2196/43153}
}
EUVIMED