EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access · CC BY 4.0

The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports

Noriko Kanemaru, Koichiro Yasaka, Nana Fujita, Jun Kanzawa, Osamu Abe

Journal of Imaging Informatics in Medicine · 26.08.2024 · 38 (2) · 865-872

DOI: 10.1007/s10278-024-01242-3 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 26.08.2024. Verfasst von Noriko Kanemaru und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8655, Japan. Hauptthemen laut MEDLINE: Early Detection of Cancer, Large Language Models, Neoplasm Metastasis, Bone Neoplasms. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The University of Tokyo.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Early detection of patients with impending bone metastasis is crucial for prognosis improvement. This study aimed to investigate the feasibility of a fine-tuned, locally run large language model (LLM) in extracting patients with bone metastasis in unstructured Japanese radiology report and to compare its performance with manual annotation. This retrospective study included patients with “metastasis” in radiological reports (April 2018–January 2019, August–May 2022, and April–December 2023 for training, validation, and test datasets of 9559, 1498, and 7399 patients, respectively). Radiologists reviewed the clinical indication and diagnosis sections of the radiological report (used as input data) and classified them into groups 0 (no bone metastasis), 1 (progressive bone metastasis), and 2 (stable or decreased bone metastasis). The data for group 0 was under-sampled in training and test datasets due to group imbalance. The best-performing model from the validation set was subsequently tested using the testing dataset. Two additional radiologists (readers 1 and 2) were involved in classifying radiological reports within the test dataset for testing purposes. The fine-tuned LLM, reader 1, and reader 2 demonstrated an accuracy of 0.979, 0.996, and 0.993, sensitivity for groups 0/1/2 of 0.988/0.947/0.943, 1.000/1.000/0.966, and 1.000/0.982/0.954, and time required for classification (s) of 105, 2312, and 3094 in under-sampled test dataset ( n = 711), respectively. Fine-tuned LLM extracted patients with bone metastasis, demonstrating satisfactory performance that was comparable to or slightly lower than manual annotation by radiologists in a noticeably shorter time.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Early Detection of CancerLarge Language ModelsNeoplasm MetastasisBone NeoplasmsJapanRadiation OncologistsDatasets as TopicBone and BonesHumansTomography, X-Ray ComputedImage Processing, Computer-Assisted

Stichwörter der Autor:innen

Bone metastasisDeep learningLarge language model

Autor:innen und Institutionen

  • Department of Radiology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8655, Japan.
  • Department of Radiology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8655, Japan. koyasaka@gmail.com.

Förderung

  • The University of Tokyo

Zitieren

APA

Kanemaru, N., Yasaka, K., Fujita, N., Kanzawa, J., & Abe, O. (2024). The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports. Journal of Imaging Informatics in Medicine, 38(2), 865-872. https://doi.org/10.1007/s10278-024-01242-3

Vancouver

Kanemaru N, Yasaka K, Fujita N, Kanzawa J, Abe O. The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports. Journal of Imaging Informatics in Medicine. 2024;38(2):865-872. doi:10.1007/s10278-024-01242-3

BibTeX
@article{Kanemaru2024,
  author = {Kanemaru, Noriko and Yasaka, Koichiro and Fujita, Nana and Kanzawa, Jun and Abe, Osamu},
  title = {The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports},
  journal = {Journal of Imaging Informatics in Medicine},
  year = {2024},
  volume = {38},
  number = {2},
  pages = {865--872},
  doi = {10.1007/s10278-024-01242-3}
}
Kostenlos starten →