EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study

Ivan Viculin, Josip Vrdoljak, Krešimir Tomić, Ivana Canjko, Dora Čerina Pavlinović, Mateo Ćurin, Lidija Bošković, Zvonimir Družianić, Bartul Vuković, Niko Dunkic, Vide Popović, Suzana Mladinov und 2 weitere

Journal of Medical Internet Research · 2026

DOI: 10.2196/91733 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Medical Internet Research, 2026. Verfasst von Ivan Viculin und 13 weiteren Autor:innen. Beteiligte Einrichtung: Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia. Hauptthemen laut MEDLINE: Lung Neoplasms, Patient Care Team, Decision Making, Clinical Decision-Making. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) and large reasoning models (LRMs) have shown excellent performance on medical benchmarks, although evaluations concerning real-world medical workflows are still lacking. Lung cancer care is particularly dependent on the multidisciplinary team (MDT) integration of radiology, pathology, staging, and treatment planning, making it a high-bar setting for evaluating LRMs.

Objective

This study aimed to compare the quality of recommendations generated by 2 LRMs (GPT-5-Thinking and Deepseek-v3-r1) with each other and with human MDT decisions in real-world lung cancer cases, as well as to assess whether MDT awareness of AI comparison influences the quality of MDT decisions.

Methods

This was a single-center real-world comparative study of 100 consecutive lung cancer MDT cases (50 retrograde and 50 anterograde) from the University Hospital of Split, Croatia. For each case, deidentified structured reports (containing all necessary patient or case data, while excluding MDT conclusions) were submitted once to GPT-5-Thinking and Deepseek-v3-r1 to generate recommendations for radiologic diagnostics, pathologic diagnostics, oncologic therapy, and overall usefulness. Two independent lung oncologists graded MDT decisions and model outputs on 1-5 Likert scales. An average recommendation score (avg_rec) was calculated as the mean of radiology, pathology, and therapy scores. Analyses used Wilcoxon tests for paired model comparisons, Mann-Whitney tests for between-phase comparisons, and Spearman correlations (2-sided α=.05).

Results

Ratings showed ceiling effects. In the retrograde phase (N=50), the mean (95% CI) GPT-5-Thinking scores were higher than Deepseek-v3-r1 scores for radiologic diagnostics (4.89, 4.78-4.99 vs 4.76, 4.62-4.89; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Lung NeoplasmsPatient Care TeamDecision MakingClinical Decision-MakingHumansLarge Language ModelsFemaleMale

Stichwörter der Autor:innen

artificial intelligenceclinical decision-makinglarge language modelslarge reasoning modelslung cancer

Autor:innen und Institutionen

  • Department for Pulmonary Disease, University Hospital of Split, Split, Split-Dalmatia, Croatia.
  • Laboratory for AI in Biomedicine, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.
  • Department of Oncology, University Clinical Hospital Mostar, Mostar, Federation of B&H, Bosnia and Herzegovina.
  • Department of Oncology, University Hospital Centre Osijek, Osijek, County of Osijek-Baranja, Croatia.
  • Department of Oncology, University Hospital of Split, Spinčićeva 1, Split, Split-Dalmatia, Croatia.
  • Laboratory for AI in Biomedicine, Department of Pathophysiology, School of Medicine, University of Split, Split, Split-Dalmatia, Croatia.

Zitieren

APA

Viculin, I., Vrdoljak, J., Tomić, K., Canjko, I., Čerina Pavlinović, D., Ćurin, M., Bošković, L., Družianić, Z., Vuković, B., Dunkic, N., Popović, V., Mladinov, S., Božić, J., & Vrdoljak, E. (2026). Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. https://doi.org/10.2196/91733

Vancouver

Viculin I, Vrdoljak J, Tomić K, Canjko I, Čerina Pavlinović D, Ćurin M, et al. Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study. Journal of Medical Internet Research. 2026. doi:10.2196/91733

BibTeX
@article{Viculin2026,
  author = {Viculin, Ivan and Vrdoljak, Josip and Tomić, Krešimir and Canjko, Ivana and Čerina Pavlinović, Dora and Ćurin, Mateo and Bošković, Lidija and Družianić, Zvonimir and Vuković, Bartul and Dunkic, Niko and Popović, Vide and Mladinov, Suzana and Božić, Joško and Vrdoljak, Eduard},
  title = {Evaluating Large Reasoning Models Versus Human Multidisciplinary Teams in Lung Cancer Decision-Making: Real-World Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  doi = {10.2196/91733}
}
Kostenlos starten →