Consensus Finding Among LLMs to Retrieve Information About Oncological Trials
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti250837 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Fabio Dennstädt und 12 weiteren Autor:innen. Beteiligte Einrichtung: Inselspital, Bern University Hospital and University of Bern, Switzerland. Hauptthemen laut MEDLINE: Clinical Trials as Topic, Neoplasms, Medical Oncology, Natural Language Processing. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Background
Automated classification of medical literature is increasingly vital, especially in oncology. As shown in previous work, LLMs can be used as part of a flexible framework to accurately classify biomedical literature and trials. In the present study, we aimed to explore to what extent a consensus-based approach could improve classification performance.
Methods
The three LLMs Mixtral-8x7B, Meta-Llama-3.1-70B, and Qwen2.5-72B were used to classify oncological trials across four data sets with nine questions. Metrics (accuracy, precision, recall, F1-score) were assessed for individual models and consensus results.
Results
Consensus was achieved in 93.93% of cases, improving accuracy (98.34%), precision (97.01%), recall (98.11%), and F1-score (97.55%) over individual models.
Conclusions
The consensus-based LLM framework delivers high accuracy and adaptability for classifying oncological trials, with potential applications in biomedical research and trial management.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Inselspital, Bern University Hospital and University of Bern, Switzerland.
- School of Medicine, University of St. Gallen, St. Gallen, Switzerland.
- Cantonal Hospital Winterthur, Winterthur, Switzerland.
- Technical University of Munich, Heilbronn, Germany.
- Cantonal Hospital of St. Gallen, St. Gallen, Switzerland.
- Alexandria University, Faculty of Medicine, Alexandria, Egypt.
Zitieren
APA
Dennstädt, F., Windisch, P., Filchenko, I., Zink, J., Putora, P. M., Shaheen, A., Gaio, R., Cihoric, N., Wosny, M., Aeppli, S., Schmerder, M., Shelan, M., & Hastings, J. (2025). Consensus Finding Among LLMs to Retrieve Information About Oncological Trials. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250837
Vancouver
Dennstädt F, Windisch P, Filchenko I, Zink J, Putora PM, Shaheen A, et al. Consensus Finding Among LLMs to Retrieve Information About Oncological Trials. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250837
BibTeX
@article{Dennstdt2025,
author = {Dennstädt, Fabio and Windisch, Paul and Filchenko, Irina and Zink, Johannes and Putora, Paul Martin and Shaheen, Ahmed and Gaio, Roberto and Cihoric, Nikola and Wosny, Marie and Aeppli, Stefanie and Schmerder, Max and Shelan, Mohamed and Hastings, Janna},
title = {Consensus Finding Among LLMs to Retrieve Information About Oncological Trials},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti250837}
}
EUVIMED