Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study
Journal of Medical Internet Research · 2026
DOI: 10.2196/91733 ↗At a glance
Vergleichsstudie in Journal of Medical Internet Research, 2026. Written by Ivan Viculin and 13 co-authors. Institution: Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia. Major MEDLINE topics: Lung Neoplasms, Patient Care Team, Decision Making, Clinical Decision-Making. Cited 2 times so far (Crossref); 18 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Large language models (LLMs) and large reasoning models (LRMs) have shown excellent performance on medical benchmarks, although evaluations concerning real-world medical workflows are still lacking. Lung cancer care is particularly dependent on the multidisciplinary team (MDT) integration of radiology, pathology, staging, and treatment planning, making it a high-bar setting for evaluating LRMs.
Objective
This study aimed to compare the quality of recommendations generated by 2 LRMs (GPT-5-Thinking and Deepseek-v3-r1) with each other and with human MDT decisions in real-world lung cancer cases, as well as to assess whether MDT awareness of AI comparison influences the quality of MDT decisions.
Methods
This was a single-center real-world comparative study of 100 consecutive lung cancer MDT cases (50 retrograde and 50 anterograde) from the University Hospital of Split, Croatia. For each case, deidentified structured reports (containing all necessary patient or case data, while excluding MDT conclusions) were submitted once to GPT-5-Thinking and Deepseek-v3-r1 to generate recommendations for radiologic diagnostics, pathologic diagnostics, oncologic therapy, and overall usefulness. Two independent lung oncologists graded MDT decisions and model outputs on 1-5 Likert scales. An average recommendation score (avg_rec) was calculated as the mean of radiology, pathology, and therapy scores. Analyses used Wilcoxon tests for paired model comparisons, Mann-Whitney tests for between-phase comparisons, and Spearman correlations (2-sided α=.05).
Results
Ratings showed ceiling effects. In the retrograde phase (N=50), the mean (95% CI) GPT-5-Thinking scores were higher than Deepseek-v3-r1 scores for radiologic diagnostics (4.89, 4.78-4.99 vs 4.76, 4.62-4.89; P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia.
- Laboratory for AI in Biomedicine, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
- Department of Oncology, University Clinical Hospital Mostar, Mostar, Federation of B&H, Bosnia and Herzegovina.
- Department of Oncology, University Hospital Centre Osijek, Osijek, County of Osijek-Baranja, Croatia.
- Department of Oncology, University Hospital of Split, Spinčićeva 1, Split, Split-Dalmatia, Croatia.
- Laboratory for AI in Biomedicine, Department of Pathophysiology, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
Cite
APA
Viculin, I., Vrdoljak, J., Tomić, K., Canjko, I., Čerina Pavlinović, D., Ćurin, M., Bošković, L., Družianić, Z., Vuković, B., Dunkic, N., Popović, V., Mladinov, S., Božić, J., & Vrdoljak, E. (2026). Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. https://doi.org/10.2196/91733
Vancouver
Viculin I, Vrdoljak J, Tomić K, Canjko I, Čerina Pavlinović D, Ćurin M, et al. Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. 2026. doi:10.2196/91733
BibTeX
@article{Viculin2026,
author = {Viculin, Ivan and Vrdoljak, Josip and Tomić, Krešimir and Canjko, Ivana and Čerina Pavlinović, Dora and Ćurin, Mateo and Bošković, Lidija and Družianić, Zvonimir and Vuković, Bartul and Dunkic, Niko and Popović, Vide and Mladinov, Suzana and Božić, Joško and Vrdoljak, Eduard},
title = {Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study},
journal = {Journal of Medical Internet Research},
year = {2026},
doi = {10.2196/91733}
}
EUVIMED