Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination
Frontiers in Oncology · 17.06.2026 · 16
DOI: 10.3389/fonc.2026.1808714 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Oncology, online erschienen am 17.06.2026. Verfasst von Yuchen Zhou und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijin…. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
The rapid evolution of large language models has demonstrated human-level proficiency in general English-language medical examinations, yet their applicability in highly specialized, non-English clinical domains remains insufficiently explored.
Methods
To bridge this gap, this study benchmarked leading domestic (Qwen 3 Max, DeepSeek V3.2) and international (GPT-5, Gemini 2.5 Flash) models against a board-certified Chinese radiation oncologist, using the National Intermediate Professional Title Examination for Radiation Oncology in China. Statistical comparisons were conducted using mixed-effects logistic regression and McNemar's test for paired data.
Results
Domestic models demonstrated strong performance, with Qwen achieving an accuracy of 86.30%, which was higher than that of the single physician reference participant (adjusted P = 0.020). However, because only one human participant was included, this comparison should be interpreted as an illustrative reference rather than evidence of general superiority over radiation oncologists. In contrast, international models exhibited a marked performance decline in this specific context. Detailed cognitive stratification revealed a distinct reasoning-recall dissociation in international models, which maintained relatively robust clinical reasoning capabilities but failed significantly in localized knowledge retrieval. Notably, translating the examination into English did not bridge this performance gap for international models, while revealing a significant language penalty for certain domestic architectures (e.g., DeepSeek, P = 0.013). Granular error analysis confirmed that the majority of failures in international models like GPT-5 stemmed from discrepancies between Western and Chinese clinical guidelines rather than intrinsic reasoning deficits or hallucinations.
Discussion
These findings suggest that, within this examination-based Chinese radiation oncology benchmark, alignment with regional clinical standards may be a major contributor to model performance. However, differences in model architecture, training data, post-training, interface configuration, and potential test-set contamination may also contribute.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiation Oncology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
- Tsinghua Medicine, School of Medicine, Tsinghua University, Beijing, China.
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center For Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.
Zitieren
APA
Zhou, Y., Lin, S., Wang, X., & Hu, K. (2026). Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination. Frontiers in Oncology, 16. https://doi.org/10.3389/fonc.2026.1808714
Vancouver
Zhou Y, Lin S, Wang X, Hu K. Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination. Frontiers in Oncology. 2026;16. doi:10.3389/fonc.2026.1808714
BibTeX
@article{Zhou2026,
author = {Zhou, Yuchen and Lin, Shuyu and Wang, Xinhai and Hu, Ke},
title = {Knowledge localization is associated with higher performance of domestic large language models in a Chinese radiation oncology examination},
journal = {Frontiers in Oncology},
year = {2026},
volume = {16},
doi = {10.3389/fonc.2026.1808714}
}
EUVIMED