Natural language processing improves reliable identification of COVID-19 compared to diagnostic codes alone
American Journal of Epidemiology · 04.11.2025 · 194 (11) · 3348-3354
DOI: 10.1093/aje/kwaf162 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Epidemiology, online erschienen am 30.07.2025. Verfasst von Nathaniel Hendrix und 5 weiteren Autor:innen. Beteiligte Einrichtung: Center for Professionalism and Value in Health Care, American Board of Family Medicine, Washington, DC 20036, United States. Hauptthemen laut MEDLINE: Natural Language Processing, COVID-19. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: FDA HHS, US Food & Drug Administration.
Abstract (Originalsprache)
Observational COVID-19 studies often rely on diagnostic codes, but their accuracy and potential for differential misclassification across patient subgroups are unclear. In this proof of concept study, we examined age, race, and ethnicity as predictors of differential misclassification by comparing the classification accuracy of diagnostic codes to classifiers based on natural language processing (NLP) of clinical notes. We assessed differential misclassification in two primary care-based samples from the American Family Cohort: first, a cohort of 5000 patients with COVID-19 status assessed by physicians based on notes; and second, 21 659 patients (of 1 560 564) who received COVID-specific antivirals. Using annotated note data, we trained and tested three NLP classifiers (tree-based, recurrent neural network, and transformer-based). Approximately 63% of likely COVID-19 patients in the two samples had a documented ICD-10 code for COVID-19. Sensitivity was highest among younger patients (68.6% for <18 years versus 60.6% for those 75+), and for Hispanic patients (68.0% vs 58.5% for Black/African American patients). The tree-based classifier had the highest area under the ROC curve (0.92), although it was less accurate among older patients. NLP performance drastically worsened predicting data collected post-training. While NLP may improve cohort identification, frequent retraining is likely needed to capture changing documentation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Professionalism and Value in Health Care, American Board of Family Medicine, Washington, DC 20036, United States.
- Department of Epidemiology and Population Health, Stanford School of Medicine, Palo Alto, CA 94304, United States.
- Robert Graham Center, American Academy of Family Physicians, Washington, DC 20036, United States.
ORCID-Profile: Nathaniel Hendrix, Rishi V Parikh, Madeline Taskier, Grace Walter, Robert L Phillips, David H Rehkopf
Förderung
- FDA HHS
- US Food & Drug Administration
Zitieren
APA
Hendrix, N., Parikh, R. V., Taskier, M., Walter, G., Phillips, R. L., & Rehkopf, D. H. (2025). Natural language processing improves reliable identification of COVID-19 compared to diagnostic codes alone. American Journal of Epidemiology, 194(11), 3348-3354. https://doi.org/10.1093/aje/kwaf162
Vancouver
Hendrix N, Parikh RV, Taskier M, Walter G, Phillips RL, Rehkopf DH. Natural language processing improves reliable identification of COVID-19 compared to diagnostic codes alone. American Journal of Epidemiology. 2025;194(11):3348-3354. doi:10.1093/aje/kwaf162
BibTeX
@article{Hendrix2025,
author = {Hendrix, Nathaniel and Parikh, Rishi V and Taskier, Madeline and Walter, Grace and Phillips, Robert L and Rehkopf, David H},
title = {Natural language processing improves reliable identification of COVID-19 compared to diagnostic codes alone},
journal = {American Journal of Epidemiology},
year = {2025},
volume = {194},
number = {11},
pages = {3348--3354},
doi = {10.1093/aje/kwaf162}
}
EUVIMED