EUVIMEDOpdag EUVIMED
VergleichsstudieOpen access

Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study

Ivan Viculin, Josip Vrdoljak, Krešimir Tomić, Ivana Canjko, Dora Čerina Pavlinović, Mateo Ćurin, Lidija Bošković, Zvonimir Družianić, Bartul Vuković, Niko Dunkic, Vide Popović, Suzana Mladinov and 2 more

Journal of Medical Internet Research · 2026

DOI: 10.2196/91733 ↗

At a glance

Vergleichsstudie in Journal of Medical Internet Research, 2026. Written by Ivan Viculin and 13 co-authors. Institution: Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia. Major MEDLINE topics: Lung Neoplasms, Patient Care Team, Decision Making, Clinical Decision-Making. Cited 2 times so far (Crossref); 18 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Large language models (LLMs) and large reasoning models (LRMs) have shown excellent performance on medical benchmarks, although evaluations concerning real-world medical workflows are still lacking. Lung cancer care is particularly dependent on the multidisciplinary team (MDT) integration of radiology, pathology, staging, and treatment planning, making it a high-bar setting for evaluating LRMs.

Objective

This study aimed to compare the quality of recommendations generated by 2 LRMs (GPT-5-Thinking and Deepseek-v3-r1) with each other and with human MDT decisions in real-world lung cancer cases, as well as to assess whether MDT awareness of AI comparison influences the quality of MDT decisions.

Methods

This was a single-center real-world comparative study of 100 consecutive lung cancer MDT cases (50 retrograde and 50 anterograde) from the University Hospital of Split, Croatia. For each case, deidentified structured reports (containing all necessary patient or case data, while excluding MDT conclusions) were submitted once to GPT-5-Thinking and Deepseek-v3-r1 to generate recommendations for radiologic diagnostics, pathologic diagnostics, oncologic therapy, and overall usefulness. Two independent lung oncologists graded MDT decisions and model outputs on 1-5 Likert scales. An average recommendation score (avg_rec) was calculated as the mean of radiology, pathology, and therapy scores. Analyses used Wilcoxon tests for paired model comparisons, Mann-Whitney tests for between-phase comparisons, and Spearman correlations (2-sided α=.05).

Results

Ratings showed ceiling effects. In the retrograde phase (N=50), the mean (95% CI) GPT-5-Thinking scores were higher than Deepseek-v3-r1 scores for radiologic diagnostics (4.89, 4.78-4.99 vs 4.76, 4.62-4.89; P

Subjects

MeSH terms (MEDLINE)

Lung NeoplasmsPatient Care TeamDecision MakingClinical Decision-MakingHumansLarge Language ModelsFemaleMale

Author keywords

artificial intelligenceclinical decision-makinglarge language modelslarge reasoning modelslung cancer

Authors and institutions

  • Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia.
  • Laboratory for AI in Biomedicine, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
  • Department of Oncology, University Clinical Hospital Mostar, Mostar, Federation of B&H, Bosnia and Herzegovina.
  • Department of Oncology, University Hospital Centre Osijek, Osijek, County of Osijek-Baranja, Croatia.
  • Department of Oncology, University Hospital of Split, Spinčićeva 1, Split, Split-Dalmatia, Croatia.
  • Laboratory for AI in Biomedicine, Department of Pathophysiology, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.

Cite

APA

Viculin, I., Vrdoljak, J., Tomić, K., Canjko, I., Čerina Pavlinović, D., Ćurin, M., Bošković, L., Družianić, Z., Vuković, B., Dunkic, N., Popović, V., Mladinov, S., Božić, J., & Vrdoljak, E. (2026). Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. https://doi.org/10.2196/91733

Vancouver

Viculin I, Vrdoljak J, Tomić K, Canjko I, Čerina Pavlinović D, Ćurin M, et al. Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. 2026. doi:10.2196/91733

BibTeX
@article{Viculin2026,
  author = {Viculin, Ivan and Vrdoljak, Josip and Tomić, Krešimir and Canjko, Ivana and Čerina Pavlinović, Dora and Ćurin, Mateo and Bošković, Lidija and Družianić, Zvonimir and Vuković, Bartul and Dunkic, Niko and Popović, Vide and Mladinov, Suzana and Božić, Joško and Vrdoljak, Eduard},
  title = {Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  doi = {10.2196/91733}
}
Start for free →