EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study

John Novoa-Laurentiev iD, Mica Bowen iD, Avery Pullman iD, Wenyu Song iD, Ania Syrowatka iD, Jin Chen iD, Michael Sainlaire, Frank Chang iD, Krissy Gray iD, Purushottam Panta iD, Luwei Liu iD, Khalid Nawab iD und 4 weitere

JMIR Medical Informatics · 2025

DOI: 10.2196/63720 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift JMIR Medical Informatics, 2025. Verfasst von John Novoa-Laurentiev und 15 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Brigham & Women's Hospital, 75 Francis Street, Boston, MA, 02115, United States, 1 8572824088. Hauptthemen laut MEDLINE: Venous Thromboembolism, Primary Health Care, Natural Language Processing, Electronic Health Records. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCATS NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Diagnosis of venous thromboembolism (VTE) is often delayed, and facilitating earlier diagnosis may improve associated morbidity and mortality. Clinical notes contain information not found elsewhere in the medical record that could facilitate timely VTE diagnosis and accurate quality measurement. However, extracting relevant information from unstructured clinical notes is complex. Today, there are relatively few electronic clinical quality measures (eCQMs) in our national payment program and none that use natural language processing (NLP) techniques for data extraction. NLP holds great promise for making quality measurement more accurate and more efficient. Given the potential of NLP-based applications to facilitate more accurate VTE detection, primary care is one clinical setting in urgent need of this type of tool.

Objective

This study aimed to develop a tool that extracts VTE symptoms from clinical notes for use within an eCQM to quantify the rate of delayed diagnosis of VTE in primary care settings.

Methods

We iteratively developed an NLP-based data extraction tool, venous thromboembolism symptom extractor (VTExt), on an internal dataset using a rule-based approach to extract VTE symptoms from primary care clinical note text. The VTE symptoms lexicon was derived and optimized with physician guidance and externally validated using datasets from 2 independent health care organizations. We performed 26 rounds of performance evaluation of notes sampled from the case cohort (17,585 patient progress note sentences from 279 patient notes), and 5 rounds of evaluation of the control cohort (2838 patient progress note sentences from 50 patient notes). VTExt’s performance was evaluated using evaluation metrics, including area under the curve, positive predictive value, negative predictive value, sensitivity, and specificity.

Results

VTExt achieved near-perfect performance in extracting VTE symptoms from primary care notes sampled from records of patients diagnosed with or without VTE. In external validation, VTExt achieved promising performance in 2 additional geographically distant organizations using different electronic health record systems. When compared against a deep learning model and 4 machine learning models, VTExt exhibited similar or even improved performance across all metrics.

Conclusions

This study demonstrates a data-driven NLP-based approach to clinical note information extraction that can be generalized to different electronic health record systems across different institutions. Due to the robust performance of this tool, VTExt is the first NLP application to be used in a nationally endorsed eCQM.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Venous ThromboembolismPrimary Health CareNatural Language ProcessingElectronic Health RecordsAlgorithmsQuality Indicators, Health CareHumansFemaleMaleMiddle Aged

Stichwörter der Autor:innen

AINLPVTEalgorithm developmentalgorithm validationartificial intelligenceclinical noteclinical qualitydiagnosiselectronic clinical quality measureextractionextraction toolnatural language processingprimary carethromboembolism

Autor:innen und Institutionen

  • Department of Medicine, Brigham & Women's Hospital, 75 Francis Street, Boston, MA, 02115, United States, 1 8572824088.
  • Department of Medicine, Harvard Medical School, Boston, MA, United States.
  • Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, United States.
  • Department of Biomedical Informatics, University of Kentucky, Lexington, KY, United States.
  • Department of Information Services, Penn State Health, Hershey, PA, United States.
  • Department of Medicine, Penn State College of Medicine, Hershey, PA, United States.

ORCID-Profile: John Novoa-Laurentiev, Mica Bowen, Avery Pullman, Wenyu Song, Ania Syrowatka, Jin Chen, Frank Chang, Krissy Gray, Purushottam Panta, Luwei Liu, Khalid Nawab, Shadi Hijjawi, Richard Schreiber, Li Zhou, Patricia C Dykes

Förderung

  • NCATS NIH HHS

Zitieren

APA

Novoa-Laurentiev, J., Bowen, M., Pullman, A., Song, W., Syrowatka, A., Chen, J., Sainlaire, M., Chang, F., Gray, K., Panta, P., Liu, L., Nawab, K., Hijjawi, S., Schreiber, R., Zhou, L., & Dykes, P. C. (2025). An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study. JMIR Medical Informatics. https://doi.org/10.2196/63720

Vancouver

Novoa-Laurentiev J, Bowen M, Pullman A, Song W, Syrowatka A, Chen J, et al. An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study. JMIR Medical Informatics. 2025. doi:10.2196/63720

BibTeX
@article{NovoaLaurentiev2025,
  author = {Novoa-Laurentiev, John and Bowen, Mica and Pullman, Avery and Song, Wenyu and Syrowatka, Ania and Chen, Jin and Sainlaire, Michael and Chang, Frank and Gray, Krissy and Panta, Purushottam and Liu, Luwei and Nawab, Khalid and Hijjawi, Shadi and Schreiber, Richard and Zhou, Li and Dykes, Patricia C},
  title = {An Extraction Tool for Venous Thromboembolism Symptom Identification in Primary Care Notes to Facilitate Electronic Clinical Quality Measure Reporting: Algorithm Development and Validation Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/63720}
}
Kostenlos starten →