Sociodemographic biases in a commercial AI model for intracranial hemorrhage detection
Emergency Radiology · 22.07.2024 · 31 (5) · 713-723
DOI: 10.1007/s10140-024-02270-w ↗Kurz gefasst
Fachartikel in der Zeitschrift Emergency Radiology, online erschienen am 22.07.2024. Verfasst von Annie Trang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA. Hauptthemen laut MEDLINE: Intracranial Hemorrhages, Tomography, X-Ray Computed, Sensitivity and Specificity. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 40 Literaturangaben. Gefördert von: School of Medicine, University of Maryland.
Abstract (Originalsprache)
PURPOSE: To evaluate whether a commercial AI tool for intracranial hemorrhage (ICH) detection on head CT exhibited sociodemographic biases. METHODS: Our retrospective study reviewed 9736 consecutive, adult non-contrast head CT scans performed between November 2021 and February 2022 in a single healthcare system. Each CT scan was evaluated by a commercial ICH AI tool and a board-certified neuroradiologist; ground truth was defined as final radiologist determination of ICH presence/absence. After evaluating the AI tool's aggregate diagnostic performance, sub-analyses based on sociodemographic groups (age, sex, race, ethnicity, insurance status, and Area of Deprivation Index [ADI] scores) assessed for biases. χ2 test or Fisher's exact tests evaluated for statistical significance with p ≤ 0.05. RESULTS: Our patient population was 50% female (mean age 60 ± 19 years). The AI tool had an aggregate accuracy of 93% [9060/9736], sensitivity of 85% [1140/1338], specificity of 94% [7920/ 8398], positive predictive value (PPV) of 71% [1140/1618] and negative predictive value (NPV) of 98% [7920/8118]. Sociodemographic biases were identified, including lower PPV for patients who were females (67.3% [62,441/656] vs. 72.7% [699/962], p = 0.02), Black (66.7% [454/681] vs. 73.2% [686/937], p = 0.005), non-Hispanic/non-Latino (69.7% [1038/1490] vs. 95.4% [417/437]), p = 0.009), and who had Medicaid/Medicare (69.9% [754/1078]) or Private (66.5% [228/343]) primary insurance (p = 0.003). Lower sensitivity was seen for patients in the third quartile of national (78.8% [241/306], p = 0.001) and state ADI scores (79.0% [22/287], p = 0.001). CONCLUSIONS: In our healthcare system, a commercial AI tool had lower performance for ICH detection than previously reported and demonstrated several sociodemographic biases.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA.
- Department of Diagnostic Imaging, St. Jude Children's Research Hospital, 262 Danny Thomas Place, MS 220, Room I3109, Memphis, TN, 38105-3678, USA. paulyimd@gmail.com.
Förderung
- School of Medicine, University of Maryland
Zitieren
APA
Trang, A., Putman, K., Savani, D., Chatterjee, D., Zhao, J., Kamel, P., Jeudy, J. J., Parekh, V. S., & Yi, P. H. (2024). Sociodemographic biases in a commercial AI model for intracranial hemorrhage detection. Emergency Radiology, 31(5), 713-723. https://doi.org/10.1007/s10140-024-02270-w
Vancouver
Trang A, Putman K, Savani D, Chatterjee D, Zhao J, Kamel P, et al. Sociodemographic biases in a commercial AI model for intracranial hemorrhage detection. Emergency Radiology. 2024;31(5):713-723. doi:10.1007/s10140-024-02270-w
BibTeX
@article{Trang2024,
author = {Trang, Annie and Putman, Kristin and Savani, Dharmam and Chatterjee, Devina and Zhao, Jerry and Kamel, Peter and Jeudy, Jean J and Parekh, Vishwa S and Yi, Paul H},
title = {Sociodemographic biases in a commercial AI model for intracranial hemorrhage detection},
journal = {Emergency Radiology},
year = {2024},
volume = {31},
number = {5},
pages = {713--723},
doi = {10.1007/s10140-024-02270-w}
}
EUVIMED