EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

AI-Based CXR First Reading: Current Limitations to Ensure Practical Value

Yuriy Vasilev, Anton Vladzymyrskyy, Olga Omelyanskaya, Ivan Blokhin iD, Yury Kirpichev, Kirill Arzamasov iD

Diagnostics · 16.04.2023 · 13 (8) · 1430

DOI: 10.3390/diagnostics13081430 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 16.04.2023. Verfasst von Yuriy Vasilev und 5 weiteren Autor:innen. Beteiligte Einrichtung: State Budget-Funded Health Care Institution of the City of Moscow "Research and Practical Clinical Center for Diagnostics and Telemedicine Technologi…. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: "Moscow Center for Innovative Technologies in Healthcare", Moscow center for healthcare innovations.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We performed a multicenter external evaluation of the practical and clinical efficacy of a commercial AI algorithm for chest X-ray (CXR) analysis (Lunit INSIGHT CXR). A retrospective evaluation was performed with a multi-reader study. For a prospective evaluation, the AI model was run on CXR studies; the results were compared to the reports of 226 radiologists. In the multi-reader study, the area under the curve (AUC), sensitivity, and specificity of the AI were 0.94 (CI95%: 0.87–1.0), 0.9 (CI95%: 0.79–1.0), and 0.89 (CI95%: 0.79–0.98); the AUC, sensitivity, and specificity of the radiologists were 0.97 (CI95%: 0.94–1.0), 0.9 (CI95%: 0.79–1.0), and 0.95 (CI95%: 0.89–1.0). In most regions of the ROC curve, the AI performed a little worse or at the same level as an average human reader. The McNemar test showed no statistically significant differences between AI and radiologists. In the prospective study with 4752 cases, the AUC, sensitivity, and specificity of the AI were 0.84 (CI95%: 0.82–0.86), 0.77 (CI95%: 0.73–0.80), and 0.81 (CI95%: 0.80–0.82). Lower accuracy values obtained during the prospective validation were mainly associated with false-positive findings considered by experts to be clinically insignificant and the false-negative omission of human-reported “opacity”, “nodule”, and calcification. In a large-scale prospective validation of the commercial AI algorithm in clinical practice, lower sensitivity and specificity values were obtained compared to the prior retrospective evaluation of the data of the same population.

Schlagwörter

Stichwörter der Autor:innen

AI for chest X-ray first readingexternal validationlocal test setprospective validation

Autor:innen und Institutionen

  • State Budget-Funded Health Care Institution of the City of Moscow "Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department", Petrovka Street, 24, Buildin…
  • Department of Information and Internet Technologies, I.M. Sechenov First Moscow State Medical University of the Ministry of Health of the Russian Federation (Sechenov University), Trubetskaya Street, 8, Building 2, 1199…

ORCID-Profile: Ivan Blokhin, Kirill Arzamasov

Förderung

  • "Moscow Center for Innovative Technologies in Healthcare"
  • Moscow center for healthcare innovations

Zitieren

APA

Vasilev, Y., Vladzymyrskyy, A., Omelyanskaya, O., Blokhin, I., Kirpichev, Y., & Arzamasov, K. (2023). AI-Based CXR First Reading: Current Limitations to Ensure Practical Value. Diagnostics, 13(8), 1430. https://doi.org/10.3390/diagnostics13081430

Vancouver

Vasilev Y, Vladzymyrskyy A, Omelyanskaya O, Blokhin I, Kirpichev Y, Arzamasov K. AI-Based CXR First Reading: Current Limitations to Ensure Practical Value. Diagnostics. 2023;13(8):1430. doi:10.3390/diagnostics13081430

BibTeX
@article{Vasilev2023,
  author = {Vasilev, Yuriy and Vladzymyrskyy, Anton and Omelyanskaya, Olga and Blokhin, Ivan and Kirpichev, Yury and Arzamasov, Kirill},
  title = {AI-Based CXR First Reading: Current Limitations to Ensure Practical Value},
  journal = {Diagnostics},
  year = {2023},
  volume = {13},
  number = {8},
  pages = {1430},
  doi = {10.3390/diagnostics13081430}
}
Kostenlos starten →