Frequency and characteristics of errors by artificial intelligence (AI) in reading screening mammography: a systematic review
Breast Cancer Research and Treatment · 2024
DOI: 10.1007/s10549-024-07353-3 ↗Kurz gefasst
Systematische Übersicht in der Zeitschrift Breast Cancer Research and Treatment, 2024. Verfasst von Aileen Zeng und 4 weiteren Autor:innen. Beteiligte Einrichtung: The Daffodil Centre, The University of Sydney, a Joint Venture with Cancer Council New South Wales, Sydney, NSW, Australia. Hauptthemen laut MEDLINE: Mammography, Artificial Intelligence, Breast Neoplasms. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The Daffodil Centre, National Breast Cancer Foundation, National Health and Medical Research Council.
Abstract (Originalsprache)
Purpose
Artificial intelligence (AI) for reading breast screening mammograms could potentially replace (some) human-reading and improve screening effectiveness. This systematic review aims to identify and quantify the types of AI errors to better understand the consequences of implementing this technology.
Methods
Electronic databases were searched for external validation studies of the accuracy of AI algorithms in real-world screening mammograms. Descriptive synthesis was performed on error types and frequency. False negative proportions (FNP) and false positive proportions (FPP) were pooled within AI positivity thresholds using random-effects meta-analysis.
Results
Seven retrospective studies (447,676 examinations; published 2019–2022) met inclusion criteria. Five studies reported AI error as false negatives or false positives. Pooled FPP decreased incrementally with increasing positivity threshold (71.83% [95% CI 69.67, 73.90] at Transpara 3 to 10.77% [95% CI 8.34, 13.79] at Transpara 9). Pooled FNP increased incrementally from 0.02% [95% CI 0.01, 0.03] (Transpara 3) to 0.12% [95% CI 0.06, 0.26] (Transpara 9), consistent with a trade-off with FPP. Heterogeneity within thresholds reflected algorithm version and completeness of the reference standard. Other forms of AI error were reported rarely (location error and technical error in one study each).
Conclusion
AI errors are largely interpreted in the framework of test accuracy. FP and FN errors show expected variability not only by positivity threshold, but also by algorithm version and study quality. Reporting of other forms of AI errors is sparse, despite their potential implications for adoption of the technology. Considering broader types of AI error would add nuance to reporting that can inform inferences about AI’s utility.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- The Daffodil Centre, The University of Sydney, a Joint Venture with Cancer Council New South Wales, Sydney, NSW, Australia.
- School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia.
- Westmead Applied Research Centre and Faculty of Medicine and Health, The University of Sydney, Westmead, NSW, Australia.
- Wiser Healthcare, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia.
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia.
- The Daffodil Centre, The University of Sydney, a Joint Venture with Cancer Council New South Wales, Sydney, NSW, Australia. luke.marinovich@sydney.edu.au.
Förderung
- The Daffodil Centre
- National Breast Cancer Foundation
- National Health and Medical Research Council
- University of Sydney
Zitieren
APA
Zeng, A., Houssami, N., Noguchi, N., Nickel, B., & Marinovich, M. L. (2024). Frequency and characteristics of errors by artificial intelligence (AI) in reading screening mammography: a systematic review. Breast Cancer Research and Treatment. https://doi.org/10.1007/s10549-024-07353-3
Vancouver
Zeng A, Houssami N, Noguchi N, Nickel B, Marinovich ML. Frequency and characteristics of errors by artificial intelligence (AI) in reading screening mammography: a systematic review. Breast Cancer Research and Treatment. 2024. doi:10.1007/s10549-024-07353-3
BibTeX
@article{Zeng2024,
author = {Zeng, Aileen and Houssami, Nehmat and Noguchi, Naomi and Nickel, Brooke and Marinovich, M Luke},
title = {Frequency and characteristics of errors by artificial intelligence (AI) in reading screening mammography: a systematic review},
journal = {Breast Cancer Research and Treatment},
year = {2024},
doi = {10.1007/s10549-024-07353-3}
}
EUVIMED