EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Large Language Models for Supporting Clear Writing and Detecting Spin in Randomized Controlled Trials in Oncology: Comparative Analysis of GPT Models and Prompts

Carole Koechli iD, Fabio Dennstädt iD, Christina Schröder iD, Daniel M Aebersold iD, Robert Förster iD, Daniel R Zwahlen iD, Paul Windisch iD

JMIR Cancer · 2026

DOI: 10.2196/78221 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Cancer, 2026. Verfasst von Carole Koechli und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Kantonsspital Winterthur, Brauerstrasse 15, Winterthur, Switzerland, 41 52 266 26 53. Hauptthemen laut MEDLINE: Randomized Controlled Trials as Topic, Medical Oncology, Writing. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Randomized controlled trials (RCTs) are the gold standard for evaluating interventions in oncology, but reporting can be subject to “spin”—presenting results in ways that mislead readers about true efficacy.

Objective

This study aimed to investigate whether large language models (LLMs) could provide a standardized approach to detect spin, particularly in the conclusions, where it most commonly occurs.

Methods

We randomly sampled 250 two-arm, single–primary end point oncology RCTs from 7 major medical journals published between 2005 and 2023. Two authors independently annotated trials as positive or negative based on whether they met their primary end point. Three commercial LLMs (GPT-3.5 Turbo, GPT-4o, and GPT-o1) were tasked with classifying trials as positive or negative when provided with (1) conclusions only; (2) methods and conclusions; (3) methods, results, and conclusions; or (4) title and full abstract. LLM performance was evaluated against human annotations. Afterward, trials incorrectly classified as positive when the model was provided only with the conclusions but correctly classified as negative when provided with the whole abstract were analyzed for patterns that may indicate the presence of spin. Model performance was assessed using accuracy, precision, recall, and F 1 -score calculated from confusion matrices.

Results

Of the 250 trials, 146 (58.4%) were positive, and 104 (41.6%) were negative. The GPT-o1 model demonstrated the highest performance across all conditions, with F 1 -scores of 0.932 (conclusions only; 95% CI 0.90-0.96), 0.96 (methods and conclusions; 95% CI 0.93-0.98), 0.98 (methods, results, and conclusions; 95% CI 0.96-0.99), and 0.97 (title and abstract; 95% CI 0.95-0.99). Analysis of trials incorrectly classified as positive when the model was provided only with the conclusions revealed shared patterns, including absence of primary end point results, emphasis on subgroup improvements, or unclear distinction between primary and secondary end points. These patterns were almost never found in trials correctly classified as negative.

Conclusions

LLMs can effectively detect potential spin in oncology RCT reporting by identifying discrepancies between how trials are presented in the conclusions vs the full abstracts. This approach could serve as a supplementary tool for improving transparency in scientific reporting, although further development is needed to address more complex trial designs beyond those examined in this feasibility study.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Randomized Controlled Trials as TopicMedical OncologyWritingLarge Language ModelsHumansGenerative Artificial Intelligence

Stichwörter der Autor:innen

data mininglarge language modelsnatural language processingrandomized controlled trialsspin

Autor:innen und Institutionen

  • Department of Radiation Oncology, Kantonsspital Winterthur, Brauerstrasse 15, Winterthur, Switzerland, 41 52 266 26 53.
  • Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.

ORCID-Profile: Carole Koechli, Fabio Dennstädt, Christina Schröder, Daniel M Aebersold, Robert Förster, Daniel R Zwahlen, Paul Windisch

Zitieren

APA

Koechli, C., Dennstädt, F., Schröder, C., Aebersold, D. M., Förster, R., Zwahlen, D. R., & Windisch, P. (2026). Large Language Models for Supporting Clear Writing and Detecting Spin in Randomized Controlled Trials in Oncology: Comparative Analysis of GPT Models and Prompts. JMIR Cancer. https://doi.org/10.2196/78221

Vancouver

Koechli C, Dennstädt F, Schröder C, Aebersold DM, Förster R, Zwahlen DR, et al. Large Language Models for Supporting Clear Writing and Detecting Spin in Randomized Controlled Trials in Oncology: Comparative Analysis of GPT Models and Prompts. JMIR Cancer. 2026. doi:10.2196/78221

BibTeX
@article{Koechli2026,
  author = {Koechli, Carole and Dennstädt, Fabio and Schröder, Christina and Aebersold, Daniel M and Förster, Robert and Zwahlen, Daniel R and Windisch, Paul},
  title = {Large Language Models for Supporting Clear Writing and Detecting Spin in Randomized Controlled Trials in Oncology: Comparative Analysis of GPT Models and Prompts},
  journal = {JMIR Cancer},
  year = {2026},
  doi = {10.2196/78221}
}
Kostenlos starten →