Performance changes due to differences among annotating radiologists for training data in computerized lesion detection
International Journal of Computer Assisted Radiology and Surgery · 2024
DOI: 10.1007/s11548-024-03136-9 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2024. Verfasst von Yukihiro Nomura und 17 weiteren Autor:innen. Beteiligte Einrichtung: Center for Frontier Medical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba, 263-8522, Japan. ynomura@chiba-u.jp. Hauptthemen laut MEDLINE: Intracranial Aneurysm, Radiologists, Tomography, X-Ray Computed, Software. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Agency for Medical Research and Development, The University of Tokyo.
Abstract (Originalsprache)
Purpose
The quality and bias of annotations by annotators (e.g., radiologists) affect the performance changes in computer-aided detection (CAD) software using machine learning. We hypothesized that the difference in the years of experience in image interpretation among radiologists contributes to annotation variability. In this study, we focused on how the performance of CAD software changes with retraining by incorporating cases annotated by radiologists with varying experience.
Methods
We used two types of CAD software for lung nodule detection in chest computed tomography images and cerebral aneurysm detection in magnetic resonance angiography images. Twelve radiologists with different years of experience independently annotated the lesions, and the performance changes were investigated by repeating the retraining of the CAD software twice, with the addition of cases annotated by each radiologist. Additionally, we investigated the effects of retraining using integrated annotations from multiple radiologists.
Results
The performance of the CAD software after retraining differed among annotating radiologists. In some cases, the performance was degraded compared to that of the initial software. Retraining using integrated annotations showed different performance trends depending on the target CAD software, notably in cerebral aneurysm detection, where the performance decreased compared to using annotations from a single radiologist.
Conclusions
Although the performance of the CAD software after retraining varied among the annotating radiologists, no direct correlation with their experience was found. The performance trends differed according to the type of CAD software used when integrated annotations from multiple radiologists were used.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Frontier Medical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba, 263-8522, Japan. ynomura@chiba-u.jp.
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan. ynomura@chiba-u.jp.
- Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan.
- Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan.
- Department of Radiology, Tokyo Metropolitan Bokutoh Hospital, Tokyo, Japan.
Förderung
- Japan Agency for Medical Research and Development
- The University of Tokyo
Zitieren
APA
Nomura, Y., Hanaoka, S., Hayashi, N., Yoshikawa, T., Koshino, S., Sato, C., Tatsuta, M., Tanaka, Y., Kano, S., Nakaya, M., Inui, S., Kusakabe, M., Nakao, T., Miki, S., Watadani, T., Nakaoka, R., Shimizu, A., & Abe, O. (2024). Performance changes due to differences among annotating radiologists for training data in computerized lesion detection. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-024-03136-9
Vancouver
Nomura Y, Hanaoka S, Hayashi N, Yoshikawa T, Koshino S, Sato C, et al. Performance changes due to differences among annotating radiologists for training data in computerized lesion detection. International Journal of Computer Assisted Radiology and Surgery. 2024. doi:10.1007/s11548-024-03136-9
BibTeX
@article{Nomura2024,
author = {Nomura, Yukihiro and Hanaoka, Shouhei and Hayashi, Naoto and Yoshikawa, Takeharu and Koshino, Saori and Sato, Chiaki and Tatsuta, Momoko and Tanaka, Yuya and Kano, Shintaro and Nakaya, Moto and Inui, Shohei and Kusakabe, Masashi and Nakao, Takahiro and Miki, Soichiro and Watadani, Takeyuki and Nakaoka, Ryusuke and Shimizu, Akinobu and Abe, Osamu},
title = {Performance changes due to differences among annotating radiologists for training data in computerized lesion detection},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2024},
doi = {10.1007/s11548-024-03136-9}
}
EUVIMED