Evaluation of a BERT Natural Language Processing Model for Automating CT and MRI Triage and Protocol Selection
Canadian Association of Radiologists Journal · 2025
DOI: 10.1177/08465371241255895 ↗Kurz gefasst
Studienprotokoll in der Zeitschrift Canadian Association of Radiologists Journal, 2025. Verfasst von Jason Yao und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, McMaster University, Hamilton, ON, Canada. Hauptthemen laut MEDLINE: Triage, Natural Language Processing, Magnetic Resonance Imaging, Tomography, X-Ray Computed. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Purpose
To evaluate the accuracy of a Bidirectional Encoder Representations for Transformers (BERT) Natural Language Processing (NLP) model for automating triage and protocol selection of cross-sectional image requisitions.
Methods
A retrospective study was completed using 222 392 CT and MRI studies from a single Canadian university hospital database (January 2018-September 2022). Three hundred unique protocols (116 CT and 184 MRI) were included. A BERT model was trained, validated, and tested using an 80%-10%-10% stratified split. Naive Bayes (NB) and Support Vector Machine (SVM) machine learning models were used as comparators. Models were assessed using F1 score, precision, recall, and area under the receiver operating characteristic curve (AUROC). The BERT model was also assessed for multi-class protocol suggestion and subgroups based on referral location, modality, and imaging section.
Results
BERT was superior to SVM for protocol selection (F1 score: BERT-0.901 vs SVM-0.881). However, was not significantly different from SVM for triage prediction (F1 score: BERT-0.844 vs SVM-0.845). Both models outperformed NB for protocol and triage. BERT had superior performance on minority classes compared to SVM and NB. For multiclass prediction, BERT accuracy was up to 0.991 for top-5 protocol suggestion, and 0.981 for top-2 triage suggestion. Emergency department patients had the highest F1 scores for both protocol (0.957) and triage (0.986), compared to inpatients and outpatients.
Conclusion
The BERT NLP model demonstrated strong performance in automating the triage and protocol selection of radiology studies, showing potential to enhance radiologist workflows. These findings suggest the feasibility of using advanced NLP models to streamline radiology operations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, McMaster University, Hamilton, ON, Canada.
- St Joseph's Healthcare Hamilton, Hamilton, ON, Canada.
ORCID-Profile: Jason Yao
Zitieren
APA
Yao, J., Alabousi, A., & Mironov, O. (2025). Evaluation of a BERT Natural Language Processing Model for Automating CT and MRI Triage and Protocol Selection. Canadian Association of Radiologists Journal. https://doi.org/10.1177/08465371241255895
Vancouver
Yao J, Alabousi A, Mironov O. Evaluation of a BERT Natural Language Processing Model for Automating CT and MRI Triage and Protocol Selection. Canadian Association of Radiologists Journal. 2025. doi:10.1177/08465371241255895
BibTeX
@article{Yao2025,
author = {Yao, Jason and Alabousi, Abdullah and Mironov, Oleg},
title = {Evaluation of a BERT Natural Language Processing Model for Automating CT and MRI Triage and Protocol Selection},
journal = {Canadian Association of Radiologists Journal},
year = {2025},
doi = {10.1177/08465371241255895}
}
EUVIMED