Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study
Journal of Medical Internet Research · 2026
DOI: 10.2196/91733 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Medical Internet Research, 2026. Verfasst von Ivan Viculin und 13 weiteren Autor:innen. Beteiligte Einrichtung: Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia. Hauptthemen laut MEDLINE: Lung Neoplasms, Patient Care Team, Decision Making, Clinical Decision-Making. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Large language models (LLMs) and large reasoning models (LRMs) have shown excellent performance on medical benchmarks, although evaluations concerning real-world medical workflows are still lacking. Lung cancer care is particularly dependent on the multidisciplinary team (MDT) integration of radiology, pathology, staging, and treatment planning, making it a high-bar setting for evaluating LRMs.
Objective
This study aimed to compare the quality of recommendations generated by 2 LRMs (GPT-5-Thinking and Deepseek-v3-r1) with each other and with human MDT decisions in real-world lung cancer cases, as well as to assess whether MDT awareness of AI comparison influences the quality of MDT decisions.
Methods
This was a single-center real-world comparative study of 100 consecutive lung cancer MDT cases (50 retrograde and 50 anterograde) from the University Hospital of Split, Croatia. For each case, deidentified structured reports (containing all necessary patient or case data, while excluding MDT conclusions) were submitted once to GPT-5-Thinking and Deepseek-v3-r1 to generate recommendations for radiologic diagnostics, pathologic diagnostics, oncologic therapy, and overall usefulness. Two independent lung oncologists graded MDT decisions and model outputs on 1-5 Likert scales. An average recommendation score (avg_rec) was calculated as the mean of radiology, pathology, and therapy scores. Analyses used Wilcoxon tests for paired model comparisons, Mann-Whitney tests for between-phase comparisons, and Spearman correlations (2-sided α=.05).
Results
Ratings showed ceiling effects. In the retrograde phase (N=50), the mean (95% CI) GPT-5-Thinking scores were higher than Deepseek-v3-r1 scores for radiologic diagnostics (4.89, 4.78-4.99 vs 4.76, 4.62-4.89; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia.
- Laboratory for AI in Biomedicine, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
- Department of Oncology, University Clinical Hospital Mostar, Mostar, Federation of B&H, Bosnia and Herzegovina.
- Department of Oncology, University Hospital Centre Osijek, Osijek, County of Osijek-Baranja, Croatia.
- Department of Oncology, University Hospital of Split, Spinčićeva 1, Split, Split-Dalmatia, Croatia.
- Laboratory for AI in Biomedicine, Department of Pathophysiology, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
Zitieren
APA
Viculin, I., Vrdoljak, J., Tomić, K., Canjko, I., Čerina Pavlinović, D., Ćurin, M., Bošković, L., Družianić, Z., Vuković, B., Dunkic, N., Popović, V., Mladinov, S., Božić, J., & Vrdoljak, E. (2026). Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. https://doi.org/10.2196/91733
Vancouver
Viculin I, Vrdoljak J, Tomić K, Canjko I, Čerina Pavlinović D, Ćurin M, et al. Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. 2026. doi:10.2196/91733
BibTeX
@article{Viculin2026,
author = {Viculin, Ivan and Vrdoljak, Josip and Tomić, Krešimir and Canjko, Ivana and Čerina Pavlinović, Dora and Ćurin, Mateo and Bošković, Lidija and Družianić, Zvonimir and Vuković, Bartul and Dunkic, Niko and Popović, Vide and Mladinov, Suzana and Božić, Joško and Vrdoljak, Eduard},
title = {Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study},
journal = {Journal of Medical Internet Research},
year = {2026},
doi = {10.2196/91733}
}
EUVIMED