EUVIMEDEUVIMED entdecken
Crossref

Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials

Fabio Dennstädt, Paul Windisch, Irina Filchenko, Johannes Zink, Paul Martin Putora, Ahmed Shaheen, Roberto Gaio, Nikola Cihoric, Marie Wosny, Stefanie Aeppli, Max Schmerder, Mohamed Shelan und 1 weitere

Oncology · 13.06.2025 · 104 (4) · 410-420

DOI: 10.1159/000546946 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Oncology, online erschienen am 13.06.2025. Verfasst von Fabio Dennstädt und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland. Hauptthemen laut MEDLINE: Clinical Trials as Topic, Neoplasms, Information Storage and Retrieval, Medical Oncology. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 19 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

The automated classification of clinical trials and key categories within the medical literature is increasingly relevant, particularly in oncology, as the volume of publications and trial reports continues to expand. Large language models (LLMs) may provide new opportunities for automating diverse classification tasks. They could be used for general-purpose text classification, retrieving information about oncological trials.

Methods

A general text classification framework with adaptable prompt, model and categories for the classification was developed. The framework was tested with four datasets comprising nine binary classification questions related to oncological trials. Evaluation was conducted using a locally hosted Mixtral-8x7B-Instruct v0.1-GPTQ model and three cloud-based LLMs: Mixtral-8x7B-Instruct v0.1, Llama3.1-70B-Instruct, and Qwen-2.5–72B.

Results

The system consistently produced valid responses with the local Mixtral-8x7B-Instruct model and the Llama3.1-70B-Instruct model. It achieved a response validity rate of 99.70% and 99.88% for the cloud-based Mixtral and Qwen models, respectively. Across all models, the framework achieved an overall accuracy of >94%, precision of >92%, recall of >90%, and an F1-score of >92%. Question-specific accuracy ranged from 86.33% to 99.83% for the local Mixtral model, 85.49%–99.83% for the cloud-based Mixtral model, 90.50%–99.83% for the Llama3.1 model, and 77.13%–99.83% for the Qwen model.

Conclusion

The LLM-based classification framework exhibits robust accuracy and adaptability across various oncological trial classification tasks. While there remain some challenges such as strong prompt dependence and high computational and hardware demands, LLMs will play a crucial role in automating the classification of oncological trials and literature as the technology continues to advance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Clinical Trials as TopicNeoplasmsInformation Storage and RetrievalMedical OncologyHumansLanguageLarge Language Models

Stichwörter der Autor:innen

ClassificationKnowledge synthesisLarge language modelNatural language processingOncological trial

Autor:innen und Institutionen

  • Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland.
  • School of Medicine, University of St. Gallen, St. Gallen, Switzerland.
  • Department of Radiation Oncology, Cantonal Hospital Winterthur, Winterthur, Switzerland.
  • Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland.
  • TUM School of Computation, Information and Technology (CIT), Technical University of Munich, Heilbronn, Germany.
  • Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland, llm@data00.com.

Zitieren

APA

Dennstädt, F., Windisch, P., Filchenko, I., Zink, J., Putora, P. M., Shaheen, A., Gaio, R., Cihoric, N., Wosny, M., Aeppli, S., Schmerder, M., Shelan, M., & Hastings, J. (2025). Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials. Oncology, 104(4), 410-420. https://doi.org/10.1159/000546946

Vancouver

Dennstädt F, Windisch P, Filchenko I, Zink J, Putora PM, Shaheen A, et al. Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials. Oncology. 2025;104(4):410-420. doi:10.1159/000546946

BibTeX
@article{Dennstdt2025,
  author = {Dennstädt, Fabio and Windisch, Paul and Filchenko, Irina and Zink, Johannes and Putora, Paul Martin and Shaheen, Ahmed and Gaio, Roberto and Cihoric, Nikola and Wosny, Marie and Aeppli, Stefanie and Schmerder, Max and Shelan, Mohamed and Hastings, Janna},
  title = {Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials},
  journal = {Oncology},
  year = {2025},
  volume = {104},
  number = {4},
  pages = {410--420},
  doi = {10.1159/000546946}
}
Kostenlos starten →