EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting

Ryan Shea Ying Cong Tan iD, Qian Lin, Guat Hwa Low, Ruixi Lin, Tzer Chew Goh, Christopher Chu En Chang, Fung Fung Lee, Wei Yin Chan, Wei Chong Tan, Han Jieh Tey, Fun Loon Leong, Hong Qi Tan und 12 weitere

Journal of the American Medical Informatics Association · 25.09.2023 · 30 (10) · 1657-1664

DOI: 10.1093/jamia/ocad133 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 14.07.2023. Verfasst von Ryan Shea Ying Cong Tan und 23 weiteren Autor:innen. Beteiligte Einrichtung: Division of Medical Oncology, National Cancer Centre Singapore, Singapore. Hauptthemen laut MEDLINE: Neoplasms, Radiology. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: A*STAR, Singapore Health Services under the Singhealth Duke-NUS Oncology ACP Programme.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To assess large language models on their ability to accurately infer cancer disease response from free-text radiology reports. Materials and Methods We assembled 10 602 computed tomography reports from cancer patients seen at a single institution. All reports were classified into: no evidence of disease, partial response, stable disease, or progressive disease. We applied transformer models, a bidirectional long short-term memory model, a convolutional neural network model, and conventional machine learning methods to this task. Data augmentation using sentence permutation with consistency loss as well as prompt-based fine-tuning were used on the best-performing models. Models were validated on a hold-out test set and an external validation set based on Response Evaluation Criteria in Solid Tumors (RECIST) classifications.

Results

The best-performing model was the GatorTron transformer which achieved an accuracy of 0.8916 on the test set and 0.8919 on the RECIST validation set. Data augmentation further improved the accuracy to 0.8976. Prompt-based fine-tuning did not further improve accuracy but was able to reduce the number of training reports to 500 while still achieving good performance.

Discussion

These models could be used by researchers to derive progression-free survival in large datasets. It may also serve as a decision support tool by providing clinicians an automated second opinion of disease response.

Conclusions

Large clinical language models demonstrate potential to infer cancer disease response from radiology reports at scale. Data augmentation techniques are useful to further improve performance. Prompt-based fine-tuning can significantly reduce the size of the training dataset.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

NeoplasmsRadiologyHumansMachine LearningNeural Networks, ComputerResearch ReportNatural Language Processing

Stichwörter der Autor:innen

cancerhealthcarelarge language modelsnatural language processingradiology

Autor:innen und Institutionen

  • Division of Medical Oncology, National Cancer Centre Singapore, Singapore.
  • Duke-NUS Medical School, Singapore.
  • Department of Computer Science, National University of Singapore, Singapore.
  • Institute of Systems Science, National University of Singapore, Singapore.
  • Division of Radiation Oncology, National Cancer Centre Singapore, Singapore.
  • Department of Diagnostic Radiology, Singapore General Hospital, Singapore.

ORCID-Profile: Ryan Shea Ying Cong Tan, Matthew Chin Heng Chua

Förderung

  • A*STAR
  • Singapore Health Services under the Singhealth Duke-NUS Oncology ACP Programme

Zitieren

APA

Tan, R. S. Y. C., Lin, Q., Low, G. H., Lin, R., Goh, T. C., Chang, C. C. E., Lee, F. F., Chan, W. Y., Tan, W. C., Tey, H. J., Leong, F. L., Tan, H. Q., Nei, W. L., Chay, W. Y., Tai, D. W. M., Lai, G. G. Y., Cheng, L. T., Wong, F. Y., Chua, M. C. H., …, & Ng, H. T. (2023). Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting. Journal of the American Medical Informatics Association, 30(10), 1657-1664. https://doi.org/10.1093/jamia/ocad133

Vancouver

Tan RSYC, Lin Q, Low GH, Lin R, Goh TC, Chang CCE, et al. Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting. Journal of the American Medical Informatics Association. 2023;30(10):1657-1664. doi:10.1093/jamia/ocad133

BibTeX
@article{Tan2023,
  author = {Tan, Ryan Shea Ying Cong and Lin, Qian and Low, Guat Hwa and Lin, Ruixi and Goh, Tzer Chew and Chang, Christopher Chu En and Lee, Fung Fung and Chan, Wei Yin and Tan, Wei Chong and Tey, Han Jieh and Leong, Fun Loon and Tan, Hong Qi and Nei, Wen Long and Chay, Wen Yee and Tai, David Wai Meng and Lai, Gillianne Geet Yi and Cheng, Lionel Tim-Ee and Wong, Fuh Yong and Chua, Matthew Chin Heng and Chua, Melvin Lee Kiang and Tan, Daniel Shao Weng and Thng, Choon Hua and Tan, Iain Bee Huat and Ng, Hwee Tou},
  title = {Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting},
  journal = {Journal of the American Medical Informatics Association},
  year = {2023},
  volume = {30},
  number = {10},
  pages = {1657--1664},
  doi = {10.1093/jamia/ocad133}
}
Kostenlos starten →