A Pipeline for the Automatic Identification of Randomized Controlled Oncology Trials and Assignment of Tumor Entities Using Natural Language Processing
Oncology · 2025
DOI: 10.1159/000546970 ↗Kurz gefasst
Fachartikel in der Zeitschrift Oncology, 2025. Verfasst von Paul Windisch und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Cantonal Hospital Winterthur, Winterthur, Switzerland, paul.windisch@ksw.ch. Hauptthemen laut MEDLINE: Randomized Controlled Trials as Topic, Natural Language Processing, Neoplasms, Medical Oncology.
Abstract (Originalsprache)
Background
Most tools trying to automatically extract information from medical publications are domain agnostic and process publications from any field. However, only retrieving trials from dedicated fields could have advantages for further processing of the data.
Methods
We trained a small transformer model to classify trials into randomized controlled trials (RCTs) vs. non-RCTs and oncology publications vs. non-oncology publications. In addition, we used two large language models (GPT-4o and GPT-4o mini) for the same task. We assessed the performance of the three models and then developed a simple set of rules to extract the tumor entity from the retrieved oncology RCTs.
Results
On the unseen test set consisting of 100 publications, the small transformer achieved an F1-score of 0.96 (95% CI: 0.92 - 1.00) with a precision of 1.00 and a recall of 0.92 for predicting whether a publication was an RCT. For predicting whether a publication covered an oncology topic, the F1-score was 0.84 (0.77 - 0.91) with a precision of 0.75 and a recall of 0.95. GPT-4o achieved an F1-score of 0.94 (95% CI: 0.90 - 0.99) with a precision of 0.89 and a recall of 1.00 for predicting whether a publication was an RCT. For predicting whether a publication covered an oncology topic the F1-score was 0.91 (0.85 - 0.97) with a precision of 0.91 and a recall of 0.91. The rule-based system was able to correctly assign every oncology RCT in the test set to a tumor entity.
Conclusion
In conclusion, classifying publications depending on whether they were randomized controlled oncology trials or not was feasible and enabled further processing using more specialized tools such as rule-based systems and potentially dedicated machine learning models.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, Cantonal Hospital Winterthur, Winterthur, Switzerland, paul.windisch@ksw.ch.
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland, paul.windisch@ksw.ch.
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
- Department of Radiation Oncology, Cantonal Hospital Winterthur, Winterthur, Switzerland.
Zitieren
APA
Windisch, P., Dennstädt, F., Koechli, C., Förster, R., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2025). A Pipeline for the Automatic Identification of Randomized Controlled Oncology Trials and Assignment of Tumor Entities Using Natural Language Processing. Oncology. https://doi.org/10.1159/000546970
Vancouver
Windisch P, Dennstädt F, Koechli C, Förster R, Schröder C, Aebersold DM, et al. A Pipeline for the Automatic Identification of Randomized Controlled Oncology Trials and Assignment of Tumor Entities Using Natural Language Processing. Oncology. 2025. doi:10.1159/000546970
BibTeX
@article{Windisch2025,
author = {Windisch, Paul and Dennstädt, Fabio and Koechli, Carole and Förster, Robert and Schröder, Christina and Aebersold, Daniel M and Zwahlen, Daniel R},
title = {A Pipeline for the Automatic Identification of Randomized Controlled Oncology Trials and Assignment of Tumor Entities Using Natural Language Processing},
journal = {Oncology},
year = {2025},
doi = {10.1159/000546970}
}
EUVIMED