Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification
Diagnostics · 22.05.2024 · 14 (11) · 1081
DOI: 10.3390/diagnostics14111081 ↗Kurz gefasst
Fachartikel in der Zeitschrift Diagnostics, online erschienen am 22.05.2024. Verfasst von Derek Wu und 23 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Western University, London, ON N6A 5C1, Canada. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Deep learning (DL) models for medical image classification frequently struggle to generalize to data from outside institutions. Additional clinical data are also rarely collected to comprehensively assess and understand model performance amongst subgroups. Following the development of a single-center model to identify the lung sliding artifact on lung ultrasound (LUS), we pursued a validation strategy using external LUS data. As annotated LUS data are relatively scarce—compared to other medical imaging data—we adopted a novel technique to optimize the use of limited external data to improve model generalizability. Externally acquired LUS data from three tertiary care centers, totaling 641 clips from 238 patients, were used to assess the baseline generalizability of our lung sliding model. We then employed our novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method to fine-tune the baseline model and determine the minimum amount of data required to achieve predefined performance goals. A subgroup analysis was also performed and Grad-CAM++ explanations were examined. The final model was fine-tuned on one-third of the external dataset to achieve 0.917 sensitivity, 0.817 specificity, and 0.920 area under the receiver operator characteristic curve (AUC) on the external validation dataset, exceeding our predefined performance goals. Subgroup analyses identified LUS characteristics that most greatly challenged the model’s performance. Grad-CAM++ saliency maps highlighted clinically relevant regions on M-mode images. We report a multicenter study that exploits limited available external data to improve the generalizability and performance of our lung sliding model while identifying poorly performing subgroups to inform future iterative improvements. This approach may contribute to efficiencies for DL researchers working with smaller quantities of external validation data.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medicine, Western University, London, ON N6A 5C1, Canada.
- Faculty of Mathematics, University of Waterloo, Waterloo, ON N2L 3G1, Canada.
- Faculty of Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada.
- Division of Critical Care Medicine, Western University, London, ON N6A 5C1, Canada.
- Schulich School of Medicine and Dentistry, Western University, London, ON N6A 5C1, Canada.
- Independent Researcher, London, ON N6A 1L8, Canada.
ORCID-Profile: Derek Wu, Blake VanBerlo, Amir Roshankar, Hoseok Lee, Faraz Ali, Jared Tschirhart, Ashritha Durvasula, Hans Clausdorff
Zitieren
APA
Wu, D., Smith, D., VanBerlo, B., Roshankar, A., Lee, H., Li, B., Ali, F., Rahman, M., Basmaji, J., Tschirhart, J., Ford, A., VanBerlo, B., Durvasula, A., Vannelli, C., Dave, C., Deglint, J., Ho, J., Chaudhary, R., Clausdorff, H., …, & Arntfield, R. (2024). Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification. Diagnostics, 14(11), 1081. https://doi.org/10.3390/diagnostics14111081
Vancouver
Wu D, Smith D, VanBerlo B, Roshankar A, Lee H, Li B, et al. Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification. Diagnostics. 2024;14(11):1081. doi:10.3390/diagnostics14111081
BibTeX
@article{Wu2024,
author = {Wu, Derek and Smith, Delaney and VanBerlo, Blake and Roshankar, Amir and Lee, Hoseok and Li, Brian and Ali, Faraz and Rahman, Marwan and Basmaji, John and Tschirhart, Jared and Ford, Alex and VanBerlo, Bennett and Durvasula, Ashritha and Vannelli, Claire and Dave, Chintan and Deglint, Jason and Ho, Jordan and Chaudhary, Rushil and Clausdorff, Hans and Prager, Ross and Millington, Scott and Shah, Samveg and Buchanan, Brian and Arntfield, Robert},
title = {Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification},
journal = {Diagnostics},
year = {2024},
volume = {14},
number = {11},
pages = {1081},
doi = {10.3390/diagnostics14111081}
}
EUVIMED