EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking GPT-5 in radiation oncology: measurable gains, but persistent need for expert oversight

Udo S Dinc, Jibak Sarkar, Philipp Schubert, Sabine Semrau, Thomas Weissmann, Andre Karius, Johann Brand, Bernd-Niklas Axer, Ahmed Gomaa, Pluvio Stephan, Ishita Sheth, Sogand Beirami und 7 weitere

Frontiers in Oncology · 11.12.2025 · 15

DOI: 10.3389/fonc.2025.1695468 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Oncology, online erschienen am 11.12.2025. Verfasst von Udo S Dinc und 18 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Large language models (LLM) have shown great potential in clinical decision support and medical education. GPT-5 is a novel LLM system that has been specifically marketed towards oncology use. This study comprehensively benchmarks GPT-5 for the field of radiation oncology.

Methods

Performance was assessed using two complementary benchmarks: (i) the American College of Radiology Radiation Oncology In-Training Examination (TXIT, 2021), comprising 300 multiple-choice items, and (ii) a curated set of 60 authentic radiation oncologic vignettes representing diverse disease sites and treatment indications. For the vignette evaluation, GPT-5 was instructed to generate structured therapeutic plans and concise two-line summaries. Four board-certified radiation oncologists independently rated outputs for correctness, comprehensiveness, and hallucinations. Inter-rater reliability was quantified using Fleiss’ κ . GPT-5–14 results were compared to published GPT-3.5 and GPT-4 baselines.

Results

On the TXIT benchmark, GPT-5 achieved a mean accuracy of 92.8%, outperforming GPT-4 (78.8%) and GPT-3.5 (62.1%). Domain-specific gains were most pronounced in dose specification and diagnosis. In the vignette evaluation, GPT-5’s treatment recommendations were rated highly for correctness (mean 3.24/4, 95% CI: 3.11–3.38) and comprehensiveness (3.59/4, 95% CI: 3.49–3.69). Hallucinations were rare, flagged in 10.0% of all individual reviewer assessments (24 of 240), and no patient case reached majority consensus for their presence. Inter-rater agreement was low (Fleiss’ κ 0.083 for correctness), reflecting inherent variability in clinical judgment. Errors clustered in complex scenarios requiring precise trial knowledge or detailed clinical adaptation.

Discussion

GPT-5 clearly outperformed prior model variants on the radiation oncology multiple-choice benchmark. Although GPT-5 exhibited favorable performance in generating real-world radiation oncology treatment recommendations, correctness ratings indicate room for further improvement. While hallucinations were infrequent, the presence of substantive errors underscores that GPT-5-generated recommendations require rigorous expert oversight before clinical implementation. In addition, considerable inter-rater variability highlights the challenge of achieving consistent expert evaluation.

Schlagwörter

Stichwörter der Autor:innen

GPt-5artificial intelligencehallucinationlarge language modelsoncologic decision supportradiation oncologyreal-world evaluationtreatment recommendation

Autor:innen und Institutionen

  • Department of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
  • Comprehensive Cancer Center Erlangen-Europäische Metropolregion Nürnberg (European Metropolitan Region Nuremberg) (CCC ER-EMN), Erlangen, Germany.
  • Bavarian Cancer Research Center (BZKF), Munich, Germany.
  • Department of Radiation Oncology, University Hospital, Ludwig Maximilian University of Munich, Munich, Germany.

Zitieren

APA

Dinc, U. S., Sarkar, J., Schubert, P., Semrau, S., Weissmann, T., Karius, A., Brand, J., Axer, B. N., Gomaa, A., Stephan, P., Sheth, I., Beirami, S., Schwarz, A., Gaipl, U., Frey, B., Bert, C., Corradini, S., Fietkau, R., & Putz, F. (2025). Benchmarking GPT-5 in radiation oncology: measurable gains, but persistent need for expert oversight. Frontiers in Oncology, 15. https://doi.org/10.3389/fonc.2025.1695468

Vancouver

Dinc US, Sarkar J, Schubert P, Semrau S, Weissmann T, Karius A, et al. Benchmarking GPT-5 in radiation oncology: measurable gains, but persistent need for expert oversight. Frontiers in Oncology. 2025;15. doi:10.3389/fonc.2025.1695468

BibTeX
@article{Dinc2025,
  author = {Dinc, Udo S and Sarkar, Jibak and Schubert, Philipp and Semrau, Sabine and Weissmann, Thomas and Karius, Andre and Brand, Johann and Axer, Bernd-Niklas and Gomaa, Ahmed and Stephan, Pluvio and Sheth, Ishita and Beirami, Sogand and Schwarz, Annette and Gaipl, Udo and Frey, Benjamin and Bert, Christoph and Corradini, Stefanie and Fietkau, Rainer and Putz, Florian},
  title = {Benchmarking GPT-5 in radiation oncology: measurable gains, but persistent need for expert oversight},
  journal = {Frontiers in Oncology},
  year = {2025},
  volume = {15},
  doi = {10.3389/fonc.2025.1695468}
}
Kostenlos starten →