An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study
JMIR Medical Informatics · 2025
DOI: 10.2196/63720 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift JMIR Medical Informatics, 2025. Verfasst von John Novoa-Laurentiev und 15 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Brigham & Women's Hospital, 75 Francis Street, Boston, MA, 02115, United States, 1 8572824088. Hauptthemen laut MEDLINE: Venous Thromboembolism, Primary Health Care, Natural Language Processing, Electronic Health Records. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCATS NIH HHS.
Themen: Thromboseprophylaxe
Abstract (Originalsprache)
Background
Diagnosis of venous thromboembolism (VTE) is often delayed, and facilitating earlier diagnosis may improve associated morbidity and mortality. Clinical notes contain information not found elsewhere in the medical record that could facilitate timely VTE diagnosis and accurate quality measurement. However, extracting relevant information from unstructured clinical notes is complex. Today, there are relatively few electronic clinical quality measures (eCQMs) in our national payment program and none that use natural language processing (NLP) techniques for data extraction. NLP holds great promise for making quality measurement more accurate and more efficient. Given the potential of NLP-based applications to facilitate more accurate VTE detection, primary care is one clinical setting in urgent need of this type of tool.
Objective
This study aimed to develop a tool that extracts VTE symptoms from clinical notes for use within an eCQM to quantify the rate of delayed diagnosis of VTE in primary care settings.
Methods
We iteratively developed an NLP-based data extraction tool, venous thromboembolism symptom extractor (VTExt), on an internal dataset using a rule-based approach to extract VTE symptoms from primary care clinical note text. The VTE symptoms lexicon was derived and optimized with physician guidance and externally validated using datasets from 2 independent health care organizations. We performed 26 rounds of performance evaluation of notes sampled from the case cohort (17,585 patient progress note sentences from 279 patient notes), and 5 rounds of evaluation of the control cohort (2838 patient progress note sentences from 50 patient notes). VTExt’s performance was evaluated using evaluation metrics, including area under the curve, positive predictive value, negative predictive value, sensitivity, and specificity.
Results
VTExt achieved near-perfect performance in extracting VTE symptoms from primary care notes sampled from records of patients diagnosed with or without VTE. In external validation, VTExt achieved promising performance in 2 additional geographically distant organizations using different electronic health record systems. When compared against a deep learning model and 4 machine learning models, VTExt exhibited similar or even improved performance across all metrics.
Conclusions
This study demonstrates a data-driven NLP-based approach to clinical note information extraction that can be generalized to different electronic health record systems across different institutions. Due to the robust performance of this tool, VTExt is the first NLP application to be used in a nationally endorsed eCQM.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medicine, Brigham & Women's Hospital, 75 Francis Street, Boston, MA, 02115, United States, 1 8572824088.
- Department of Medicine, Harvard Medical School, Boston, MA, United States.
- Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, United States.
- Department of Biomedical Informatics, University of Kentucky, Lexington, KY, United States.
- Department of Information Services, Penn State Health, Hershey, PA, United States.
- Department of Medicine, Penn State College of Medicine, Hershey, PA, United States.
ORCID-Profile: John Novoa-Laurentiev, Mica Bowen, Avery Pullman, Wenyu Song, Ania Syrowatka, Jin Chen, Frank Chang, Krissy Gray, Purushottam Panta, Luwei Liu, Khalid Nawab, Shadi Hijjawi, Richard Schreiber, Li Zhou, Patricia C Dykes
Förderung
- NCATS NIH HHS
Zitieren
APA
Novoa-Laurentiev, J., Bowen, M., Pullman, A., Song, W., Syrowatka, A., Chen, J., Sainlaire, M., Chang, F., Gray, K., Panta, P., Liu, L., Nawab, K., Hijjawi, S., Schreiber, R., Zhou, L., & Dykes, P. C. (2025). An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study. JMIR Medical Informatics. https://doi.org/10.2196/63720
Vancouver
Novoa-Laurentiev J, Bowen M, Pullman A, Song W, Syrowatka A, Chen J, et al. An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study. JMIR Medical Informatics. 2025. doi:10.2196/63720
BibTeX
@article{NovoaLaurentiev2025,
author = {Novoa-Laurentiev, John and Bowen, Mica and Pullman, Avery and Song, Wenyu and Syrowatka, Ania and Chen, Jin and Sainlaire, Michael and Chang, Frank and Gray, Krissy and Panta, Purushottam and Liu, Luwei and Nawab, Khalid and Hijjawi, Shadi and Schreiber, Richard and Zhou, Li and Dykes, Patricia C},
title = {An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study},
journal = {JMIR Medical Informatics},
year = {2025},
doi = {10.2196/63720}
}
EUVIMED