EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A real-world evaluation of the diagnostic accuracy of radiologists using positive predictive values verified from deep learning and natural language processing chest algorithms deployed retrospectively

Bahadar S Bhatia iD, John F Morlese, Sarah Yusuf, Yiting Xie, Bob Schallhorn, David Gruen

BJR|Open · 2023

DOI: 10.1093/bjro/tzad009 ↗

Kurz gefasst

Fachartikel in der Zeitschrift BJR|Open, 2023. Verfasst von Bahadar S Bhatia und 5 weiteren Autor:innen. Beteiligte Einrichtung: Directorate of Diagnostic Radiology, Sandwell & West Birmingham NHS Trust, Lyndon, West Bromwich B71 4HJ, United Kingdom. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: IBM Watson Health Imaging.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

This diagnostic study assessed the accuracy of radiologists retrospectively, using the deep learning and natural language processing chest algorithms implemented in Clinical Review version 3.2 for: pneumothorax, rib fractures in digital chest X-ray radiographs (CXR); aortic aneurysm, pulmonary nodules, emphysema, and pulmonary embolism in CT images.

Methods

The study design was double-blind (artificial intelligence [AI] algorithms and humans), retrospective, non-interventional, and at a single NHS Trust. Adult patients (≥18 years old) scheduled for CXR and CT were invited to enroll as participants through an opt-out process. Reports and images were de-identified, processed retrospectively, and AI-flagged discrepant findings were assigned to two lead radiologists, each blinded to patient identifiers and original radiologist. The radiologist’s findings for each clinical condition were tallied as a verified discrepancy (true positive) or not (false positive).

Results

The missed findings were: 0.02% rib fractures, 0.51% aortic aneurysm, 0.32% pulmonary nodules, 0.92% emphysema, and 0.28% pulmonary embolism. The positive predictive values (PPVs) were: pneumothorax (0%), rib fractures (5.6%), aortic dilatation (43.2%), pulmonary emphysema (46.0%), pulmonary embolus (11.5%), and pulmonary nodules (9.2%). The PPV for pneumothorax was nil owing to lack of available studies that were analysed for outpatient activity.

Conclusions

The number of missed findings was far less than generally predicted. The chest algorithms deployed retrospectively were a useful quality tool and AI augmented the radiologists’ workflow. Advances in knowledge The diagnostic accuracy of our radiologists generated missed findings of 0.02% for rib fractures CXR, 0.51% for aortic dilatation, 0.32% for pulmonary nodule, 0.92% for pulmonary emphysema, and 0.28% for pulmonary embolism for CT studies, all retrospectively evaluated with AI used as a quality tool to flag potential missed findings. It is important to account for prevalence of these chest conditions in clinical context and use appropriate clinical thresholds for decision-making, not relying solely on AI.

Schlagwörter

Stichwörter der Autor:innen

aortic dilatationdeep learningnatural language processingpneumothoraxpulmonary embolismpulmonary emphysemapulmonary nodulerib fractures

Autor:innen und Institutionen

  • Directorate of Diagnostic Radiology, Sandwell & West Birmingham NHS Trust, Lyndon, West Bromwich B71 4HJ, United Kingdom.
  • Space Research Centre, Physics & Astronomy, University of Leicester, 92 Corporation Road, Leicester LE4 5SP, United Kingdom.
  • Merge, Merative (Formerly, IBM Watson Health Imaging), Ann Arbor, Michigan, MI 48108, United States.
  • Jefferson Radiology and Radiology Partners, 111 Founders Plaza, East Hartford, Connecticut CT 06108, United States.

ORCID-Profile: Bahadar S Bhatia

Förderung

  • IBM Watson Health Imaging

Zitieren

APA

Bhatia, B. S., Morlese, J. F., Yusuf, S., Xie, Y., Schallhorn, B., & Gruen, D. (2023). A real-world evaluation of the diagnostic accuracy of radiologists using positive predictive values verified from deep learning and natural language processing chest algorithms deployed retrospectively. BJR|Open. https://doi.org/10.1093/bjro/tzad009

Vancouver

Bhatia BS, Morlese JF, Yusuf S, Xie Y, Schallhorn B, Gruen D. A real-world evaluation of the diagnostic accuracy of radiologists using positive predictive values verified from deep learning and natural language processing chest algorithms deployed retrospectively. BJR|Open. 2023. doi:10.1093/bjro/tzad009

BibTeX
@article{Bhatia2023,
  author = {Bhatia, Bahadar S and Morlese, John F and Yusuf, Sarah and Xie, Yiting and Schallhorn, Bob and Gruen, David},
  title = {A real-world evaluation of the diagnostic accuracy of radiologists using positive predictive values verified from deep learning and natural language processing chest algorithms deployed retrospectively},
  journal = {BJR|Open},
  year = {2023},
  doi = {10.1093/bjro/tzad009}
}
Kostenlos starten →