EUVIMEDSkopri EUVIMED
Open access

Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation

Zhi-Geng Jin iD, Hui Zhang iD, Mei-Hui Tai iD, Ying Yang iD, Yuan Yao iD, Yu-Tao Guo iD

Journal of Medical Internet Research · 2023

DOI: 10.2196/43153 ↗

At a glance

Journal article in Journal of Medical Internet Research, 2023. Written by Zhi-Geng Jin and 5 co-authors. Institution: Department of Pulmonary Vascular and Thrombotic Disease, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China. Major MEDLINE topics: Venous Thromboembolism, Venous Thrombosis, Decision Support Systems, Clinical, Pulmonary Embolism. Cited 14 times so far (Crossref); 32 references. The full text is freely available in PubMed Central.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

It remains unknown whether capturing data from electronic health records (EHRs) using natural language processing (NLP) can improve venous thromboembolism (VTE) detection in different clinical settings.

Objective

The aim of this study was to validate the NLP algorithm in a clinical decision support system for VTE risk assessment and integrated care (DeVTEcare) to identify VTEs from EHRs.

Methods

All inpatients aged ≥18 years in the Sixth Medical Center of the Chinese People's Liberation Army General Hospital from January 1 to December 31, 2021, were included as the validation cohort. The sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR–, respectively), area under the receiver operating characteristic curve (AUC), and F1-scores along with their 95% CIs were used to analyze the performance of the NLP tool, with manual review of medical records as the reference standard for detecting deep vein thrombosis (DVT) and pulmonary embolism (PE). The primary end point was the performance of the NLP approach embedded into the EHR for VTE identification. The secondary end points were the performances to identify VTE among different hospital departments with different VTE risks. Subgroup analyses were performed among age, sex, and the study season.

Results

Among 30,152 patients (median age 56 [IQR 41-67] years; 14,247/30,152, 47.3% females), the prevalence of VTE, PE, and DVT was 2.1% (626/30,152), 0.6% (177/30,152), and 1.8% (532/30,152), respectively. The sensitivity, specificity, LR+, LR–, AUC, and F1-score of NLP-facilitated VTE detection were 89.9% (95% CI 87.3%-92.2%), 99.8% (95% CI 99.8%-99.9%), 483 (95% CI 370-629), 0.10 (95% CI 0.08-0.13), 0.95 (95% CI 0.94-0.96), and 0.90 (95% CI 0.90-0.91), respectively. Among departments of surgery, internal medicine, and intensive care units, the highest specificity (100% vs 99.7% vs 98.8%, respectively), LR+ (3202 vs 321 vs 77, respectively), and F1-score (0.95 vs 0.89 vs 0.92, respectively) were in the surgery department (all P89% AUC and F1-score. The NLP algorithm performed better among patients aged ≤65 years than among those aged >65 years (F1-score 0.93 vs 0.89, respectively; P

Subjects

MeSH terms (MEDLINE)

Venous ThromboembolismVenous ThrombosisDecision Support Systems, ClinicalPulmonary EmbolismFemaleHumansAdolescentAdultMiddle AgedMaleNatural Language ProcessingRisk AssessmentElectronic Health RecordsAlgorithms

Author keywords

deep vein thrombosiselectronic health recordnatural language processingpulmonary embolismvenous thromboembolism

Authors and institutions

  • Department of Pulmonary Vascular and Thrombotic Disease, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China.
  • Chinese People's Liberation Army Medical School, Beijing, China.
  • Quality Management Division, Sixth Medical Center of Chinese People's Liberation Army General Hospital, Beijing, China.
  • Institute for Hospital Management Research, Chinese People's Liberation Army General Hospital, Beijing, China.

ORCID profiles: Zhi-Geng Jin, Hui Zhang, Mei-Hui Tai, Ying Yang, Yuan Yao, Yu-Tao Guo

Cite

APA

Jin, Z. G., Zhang, H., Tai, M. H., Yang, Y., Yao, Y., & Guo, Y. T. (2023). Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation. Journal of Medical Internet Research. https://doi.org/10.2196/43153

Vancouver

Jin ZG, Zhang H, Tai MH, Yang Y, Yao Y, Guo YT. Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation. Journal of Medical Internet Research. 2023. doi:10.2196/43153

BibTeX
@article{Jin2023,
  author = {Jin, Zhi-Geng and Zhang, Hui and Tai, Mei-Hui and Yang, Ying and Yao, Yuan and Guo, Yu-Tao},
  title = {Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation},
  journal = {Journal of Medical Internet Research},
  year = {2023},
  doi = {10.2196/43153}
}
Start for free →