Comparison of active learning algorithms in classifying head computed tomography reports using bidirectional encoder representations from transformers
International Journal of Computer Assisted Radiology and Surgery · 2025
DOI: 10.1007/s11548-024-03316-7 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2025. Verfasst von Tomohiro Wataya und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. wataya-tomo@radiol.med.osaka-u.…. Hauptthemen laut MEDLINE: Tomography, X-Ray Computed, Algorithms, Head, Natural Language Processing. Die Arbeit nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Kaken Pharmaceutical, Osaka University.
Abstract (Originalsprache)
Purpose
Systems equipped with natural language (NLP) processing can reduce missed radiological findings by physicians, but the annotation costs are burden in the development. This study aimed to compare the effects of active learning (AL) algorithms in NLP for estimating the significance of head computed tomography (CT) reports using bidirectional encoder representations from transformers (BERT).
Methods
A total of 3728 head CT reports annotated with five categories of importance were used and UTH-BERT was adopted as the pre-trained BERT model. We assumed that 64% (2385 reports) of the data were initially in the unlabeled data pool (UDP), while the labeled data set (LD) used to train the model was empty. Twenty-five reports were repeatedly selected from the UDP and added to the LD, based on seven metrices: random sampling (RS: control), four uncertainty sampling (US) methods (least confidence (LC), margin sampling (MS), ratio of confidence (RC), and entropy sampling (ES)), and two distance-based sampling (DS) methods (cosine distance (CD) and Euclidian distance (ED)). The transition of accuracy of the model was evaluated using the test dataset.
Results
The accuracy of the models with US was significantly higher than RS when reports in LD were < 1800, whereas DS methods were significantly lower than RS. Among the US methods, MS and RC were even better than the others. With the US methods, the required labeled data decreased by 15.4–40.5%, and most efficient in RC. In addition, in the US methods, data for minor categories tended to be added to LD earlier than RS and DS.
Conclusions
In the classification task for the importance of head CT reports, US methods, especially RC and MS can lead to the effective fine-tuning of BERT models and reduce the imbalance of categories. AL can contribute to other studies on larger datasets by providing effective annotation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. wataya-tomo@radiol.med.osaka-u.ac.jp.
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. wataya-tomo@radiol.med.osaka-u.ac.jp.
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan.
- Department of Diagnostic Imaging, Osaka General Medical Center, 3-1-56. Mandai Higashi, Sumiyoshi, Osaka, 558-8558, Japan.
- Department of Diagnostic Radiology, Sakai City Medical Center, 1-1-1, Ebaracho, Sakai, Osaka, 593-8304, Japan.
- Division of Health Science, Osaka University Graduate School of Medicine, 1-7, Yamadaoka, Suita, Osaka, 565-0871, Japan.
Förderung
- Kaken Pharmaceutical
- Osaka University
Zitieren
APA
Wataya, T., Miura, A., Sakisuka, T., Fujiwara, M., Tanaka, H., Hiraoka, Y., Sato, J., Tomiyama, M., Nishigaki, D., Kita, K., Suzuki, Y., Kido, S., & Tomiyama, N. (2025). Comparison of active learning algorithms in classifying head computed tomography reports using bidirectional encoder representations from transformers. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-024-03316-7
Vancouver
Wataya T, Miura A, Sakisuka T, Fujiwara M, Tanaka H, Hiraoka Y, et al. Comparison of active learning algorithms in classifying head computed tomography reports using bidirectional encoder representations from transformers. International Journal of Computer Assisted Radiology and Surgery. 2025. doi:10.1007/s11548-024-03316-7
BibTeX
@article{Wataya2025,
author = {Wataya, Tomohiro and Miura, Azusa and Sakisuka, Takahisa and Fujiwara, Masahiro and Tanaka, Hisashi and Hiraoka, Yu and Sato, Junya and Tomiyama, Miyuki and Nishigaki, Daiki and Kita, Kosuke and Suzuki, Yuki and Kido, Shoji and Tomiyama, Noriyuki},
title = {Comparison of active learning algorithms in classifying head computed tomography reports using bidirectional encoder representations from transformers},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2025},
doi = {10.1007/s11548-024-03316-7}
}
EUVIMED