EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Consensus Finding Among LLMs to Retrieve Information About Oncological Trials

Fabio Dennstädt, Paul Windisch, Irina Filchenko, Johannes Zink, Paul Martin Putora, Ahmed Shaheen, Roberto Gaio, Nikola Cihoric, Marie Wosny, Stefanie Aeppli, Max Schmerder, Mohamed Shelan und 1 weitere

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti250837 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Fabio Dennstädt und 12 weiteren Autor:innen. Beteiligte Einrichtung: Inselspital, Bern University Hospital and University of Bern, Switzerland. Hauptthemen laut MEDLINE: Clinical Trials as Topic, Neoplasms, Medical Oncology, Natural Language Processing. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Automated classification of medical literature is increasingly vital, especially in oncology. As shown in previous work, LLMs can be used as part of a flexible framework to accurately classify biomedical literature and trials. In the present study, we aimed to explore to what extent a consensus-based approach could improve classification performance.

Methods

The three LLMs Mixtral-8x7B, Meta-Llama-3.1-70B, and Qwen2.5-72B were used to classify oncological trials across four data sets with nine questions. Metrics (accuracy, precision, recall, F1-score) were assessed for individual models and consensus results.

Results

Consensus was achieved in 93.93% of cases, improving accuracy (98.34%), precision (97.01%), recall (98.11%), and F1-score (97.55%) over individual models.

Conclusions

The consensus-based LLM framework delivers high accuracy and adaptability for classifying oncological trials, with potential applications in biomedical research and trial management.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Clinical Trials as TopicNeoplasmsMedical OncologyNatural Language ProcessingInformation Storage and RetrievalHumansConsensus

Stichwörter der Autor:innen

knowledge synthesislarge language modelsnatural language processingoncologytext classification

Autor:innen und Institutionen

  • Inselspital, Bern University Hospital and University of Bern, Switzerland.
  • School of Medicine, University of St. Gallen, St. Gallen, Switzerland.
  • Cantonal Hospital Winterthur, Winterthur, Switzerland.
  • Technical University of Munich, Heilbronn, Germany.
  • Cantonal Hospital of St. Gallen, St. Gallen, Switzerland.
  • Alexandria University, Faculty of Medicine, Alexandria, Egypt.

Zitieren

APA

Dennstädt, F., Windisch, P., Filchenko, I., Zink, J., Putora, P. M., Shaheen, A., Gaio, R., Cihoric, N., Wosny, M., Aeppli, S., Schmerder, M., Shelan, M., & Hastings, J. (2025). Consensus Finding Among LLMs to Retrieve Information About Oncological Trials. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250837

Vancouver

Dennstädt F, Windisch P, Filchenko I, Zink J, Putora PM, Shaheen A, et al. Consensus Finding Among LLMs to Retrieve Information About Oncological Trials. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250837

BibTeX
@article{Dennstdt2025,
  author = {Dennstädt, Fabio and Windisch, Paul and Filchenko, Irina and Zink, Johannes and Putora, Paul Martin and Shaheen, Ahmed and Gaio, Roberto and Cihoric, Nikola and Wosny, Marie and Aeppli, Stefanie and Schmerder, Max and Shelan, Mohamed and Hastings, Janna},
  title = {Consensus Finding Among LLMs to Retrieve Information About Oncological Trials},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti250837}
}
Kostenlos starten →