AI-Based CXR First Reading: Current Limitations to Ensure Practical Value
Diagnostics · 16.04.2023 · 13 (8) · 1430
DOI: 10.3390/diagnostics13081430 ↗Kurz gefasst
Fachartikel in der Zeitschrift Diagnostics, online erschienen am 16.04.2023. Verfasst von Yuriy Vasilev und 5 weiteren Autor:innen. Beteiligte Einrichtung: State Budget-Funded Health Care Institution of the City of Moscow "Research and Practical Clinical Center for Diagnostics and Telemedicine Technologi…. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: "Moscow Center for Innovative Technologies in Healthcare", Moscow center for healthcare innovations.
Abstract (Originalsprache)
We performed a multicenter external evaluation of the practical and clinical efficacy of a commercial AI algorithm for chest X-ray (CXR) analysis (Lunit INSIGHT CXR). A retrospective evaluation was performed with a multi-reader study. For a prospective evaluation, the AI model was run on CXR studies; the results were compared to the reports of 226 radiologists. In the multi-reader study, the area under the curve (AUC), sensitivity, and specificity of the AI were 0.94 (CI95%: 0.87–1.0), 0.9 (CI95%: 0.79–1.0), and 0.89 (CI95%: 0.79–0.98); the AUC, sensitivity, and specificity of the radiologists were 0.97 (CI95%: 0.94–1.0), 0.9 (CI95%: 0.79–1.0), and 0.95 (CI95%: 0.89–1.0). In most regions of the ROC curve, the AI performed a little worse or at the same level as an average human reader. The McNemar test showed no statistically significant differences between AI and radiologists. In the prospective study with 4752 cases, the AUC, sensitivity, and specificity of the AI were 0.84 (CI95%: 0.82–0.86), 0.77 (CI95%: 0.73–0.80), and 0.81 (CI95%: 0.80–0.82). Lower accuracy values obtained during the prospective validation were mainly associated with false-positive findings considered by experts to be clinically insignificant and the false-negative omission of human-reported “opacity”, “nodule”, and calcification. In a large-scale prospective validation of the commercial AI algorithm in clinical practice, lower sensitivity and specificity values were obtained compared to the prior retrospective evaluation of the data of the same population.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Budget-Funded Health Care Institution of the City of Moscow "Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department", Petrovka Street, 24, Buildin…
- Department of Information and Internet Technologies, I.M. Sechenov First Moscow State Medical University of the Ministry of Health of the Russian Federation (Sechenov University), Trubetskaya Street, 8, Building 2, 1199…
ORCID-Profile: Ivan Blokhin, Kirill Arzamasov
Förderung
- "Moscow Center for Innovative Technologies in Healthcare"
- Moscow center for healthcare innovations
Zitieren
APA
Vasilev, Y., Vladzymyrskyy, A., Omelyanskaya, O., Blokhin, I., Kirpichev, Y., & Arzamasov, K. (2023). AI-Based CXR First Reading: Current Limitations to Ensure Practical Value. Diagnostics, 13(8), 1430. https://doi.org/10.3390/diagnostics13081430
Vancouver
Vasilev Y, Vladzymyrskyy A, Omelyanskaya O, Blokhin I, Kirpichev Y, Arzamasov K. AI-Based CXR First Reading: Current Limitations to Ensure Practical Value. Diagnostics. 2023;13(8):1430. doi:10.3390/diagnostics13081430
BibTeX
@article{Vasilev2023,
author = {Vasilev, Yuriy and Vladzymyrskyy, Anton and Omelyanskaya, Olga and Blokhin, Ivan and Kirpichev, Yury and Arzamasov, Kirill},
title = {AI-Based CXR First Reading: Current Limitations to Ensure Practical Value},
journal = {Diagnostics},
year = {2023},
volume = {13},
number = {8},
pages = {1430},
doi = {10.3390/diagnostics13081430}
}
EUVIMED