Evaluating the capability of large language models in radiotherapy through professional certification examinations in Japan
Journal of Radiation Research · 2026
DOI: 10.1093/jrr/rraf083 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Radiation Research, 2026. Verfasst von Noriyuki Kadoya und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Tohoku University Graduate School of Medicine, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi 980-8574, Japan. Hauptthemen laut MEDLINE: Large Language Models, Certification, Radiotherapy. Die Arbeit nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large language models (LLMs), such as ChatGPT and Grok, have rapidly advanced in natural language understanding and are increasingly being applied to specialized fields, including medicine. In this study, we evaluated the domain-specific knowledge of LLMs in radiotherapy by assessing their performance on three certification examinations in Japan: the Japanese Medical Physicist Examination, the Japanese Board Examination for Radiologists and the Japanese Board Examination for Radiation Oncologists. We assessed five LLMs—ChatGPT-5, ChatGPT-5 Pro, Grok 4, Grok 4 heavy and Gemini 2.5 Pro—by inputting all multiple-choice questions from these exams into each model and recording their responses. The AI-generated answers were compared with reference answers determined by experienced medical physicists and radiation oncologists. The results demonstrated average accuracies of 84.7 ± 2.0% (ChatGPT-5), 94.7 ± 2.1% (ChatGPT-5 Pro), 78.4 ± 1.2% (Grok 4), 81.6 ± 2.2% (Grok 4 heavy) and 88.9 ± 1.2% (Gemini 2.5 Pro). All models achieved over 75% accuracy, with ChatGPT-5 Pro consistently outperforming others, attaining an average accuracy exceeding 90% across all examinations. These findings highlight the strong potential of advanced LLMs, particularly ChatGPT-5 Pro, for future integration into radiotherapy-related applications such as automated contouring and treatment planning support.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, Tohoku University Graduate School of Medicine, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi 980-8574, Japan.
- Course of Radiological Technology, Health Sciences, Tohoku University Graduate School of Medicine, 2-1 Seiryo-machi, Aoba-ku, Sendai, 980-8575, Japan.
ORCID-Profile: Noriyuki Kadoya, Yoshiyuki Katsuta, Takaya Yamamoto, Ken Takeda, Keiichi Jingu
Zitieren
APA
Kadoya, N., Takahashi, Y., Koga, S., Tanno, H., Arai, K., Tanaka, S., Katsuta, Y., Harada, H., Omata, S., Yamamoto, T., Umezawa, R., Takeda, K., & Jingu, K. (2026). Evaluating the capability of large language models in radiotherapy through professional certification examinations in Japan. Journal of Radiation Research. https://doi.org/10.1093/jrr/rraf083
Vancouver
Kadoya N, Takahashi Y, Koga S, Tanno H, Arai K, Tanaka S, et al. Evaluating the capability of large language models in radiotherapy through professional certification examinations in Japan. Journal of Radiation Research. 2026. doi:10.1093/jrr/rraf083
BibTeX
@article{Kadoya2026,
author = {Kadoya, Noriyuki and Takahashi, Yoshiyuki and Koga, Seiya and Tanno, Hikaru and Arai, Kazuhiro and Tanaka, Shohei and Katsuta, Yoshiyuki and Harada, Hinako and Omata, So and Yamamoto, Takaya and Umezawa, Rei and Takeda, Ken and Jingu, Keiichi},
title = {Evaluating the capability of large language models in radiotherapy through professional certification examinations in Japan},
journal = {Journal of Radiation Research},
year = {2026},
doi = {10.1093/jrr/rraf083}
}
EUVIMED