EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automated classification of brain MRI reports using fine-tuned large language models

Jun Kanzawa, Koichiro Yasaka, Nana Fujita, Shin Fujiwara, Osamu Abe

Neuroradiology · 2024-12 · 66 (12) · 2177-2183

DOI: 10.1007/s00234-024-03427-7 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neuroradiology, online erschienen am 12.07.2024. Verfasst von Jun Kanzawa und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan. Hauptthemen laut MEDLINE: Magnetic Resonance Imaging, Brain Neoplasms, Sensitivity and Specificity. Die Arbeit wurde bisher 44-mal zitiert (Crossref) und nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The University of Tokyo.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study aimed to investigate the efficacy of fine-tuned large language models (LLM) in classifying brain MRI reports into pretreatment, posttreatment, and nontumor cases.

Methods

This retrospective study included 759, 284, and 164 brain MRI reports for training, validation, and test dataset. Radiologists stratified the reports into three groups: nontumor (group 1), posttreatment tumor (group 2), and pretreatment tumor (group 3) cases. A pretrained Bidirectional Encoder Representations from Transformers Japanese model was fine-tuned using the training dataset and evaluated on the validation dataset. The model which demonstrated the highest accuracy on the validation dataset was selected as the final model. Two additional radiologists were involved in classifying reports in the test datasets for the three groups. The model’s performance on test dataset was compared to that of two radiologists.

Results

The fine-tuned LLM attained an overall accuracy of 0.970 (95% CI: 0.930–0.990). The model’s sensitivity for group 1/2/3 was 1.000/0.864/0.978. The model’s specificity for group1/2/3 was 0.991/0.993/0.958. No statistically significant differences were found in terms of accuracy, sensitivity, and specificity between the LLM and human readers ( p ≥ 0.371). The LLM completed the classification task approximately 20–26-fold faster than the radiologists. The area under the receiver operating characteristic curve for discriminating groups 2 and 3 from group 1 was 0.994 (95% CI: 0.982–1.000) and for discriminating group 3 from groups 1 and 2 was 0.992 (95% CI: 0.982–1.000).

Conclusion

Fine-tuned LLM demonstrated a comparable performance with radiologists in classifying brain MRI reports, while requiring substantially less time.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Magnetic Resonance ImagingBrain NeoplasmsSensitivity and SpecificityHumansRetrospective StudiesFemaleMale

Stichwörter der Autor:innen

Brain tumorLarge language modelMagnetic resonance imagingNatural language processing

Autor:innen und Institutionen

  • Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan.
  • Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan. koyasaka@gmail.com.

Förderung

  • The University of Tokyo

Zitieren

APA

Kanzawa, J., Yasaka, K., Fujita, N., Fujiwara, S., & Abe, O. (2024). Automated classification of brain MRI reports using fine-tuned large language models. Neuroradiology, 66(12), 2177-2183. https://doi.org/10.1007/s00234-024-03427-7

Vancouver

Kanzawa J, Yasaka K, Fujita N, Fujiwara S, Abe O. Automated classification of brain MRI reports using fine-tuned large language models. Neuroradiology. 2024;66(12):2177-2183. doi:10.1007/s00234-024-03427-7

BibTeX
@article{Kanzawa2024,
  author = {Kanzawa, Jun and Yasaka, Koichiro and Fujita, Nana and Fujiwara, Shin and Abe, Osamu},
  title = {Automated classification of brain MRI reports using fine-tuned large language models},
  journal = {Neuroradiology},
  year = {2024},
  volume = {66},
  number = {12},
  pages = {2177--2183},
  doi = {10.1007/s00234-024-03427-7}
}
Kostenlos starten →