EUVIMEDEUVIMED entdecken
VergleichsstudieLeitlinieOpen Access · CC BY 4.0

Performance of large language models in urological decision support: a guideline-based comparative evaluation in urolithiasis

Adem Tunçekin, Yasin Aktaş

Urolithiasis · 2026

DOI: 10.1007/s00240-026-02013-1 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Urolithiasis, 2026. Verfasst von Adem Tunçekin und 1 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Medicine, Department of Urology, Uşak University, Uşak, Turkey. dr_adem65@hotmail.com. Hauptthemen laut MEDLINE: Large Language Models, Urolithiasis, Decision Support Systems, Clinical, Urology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Usak University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are increasingly investigated for their potential role in guideline-based clinical information support. However, their consistency with subspecialty guidelines, particularly in urolithiasis, remains underexplored. This study aimed to evaluate the performance of four large language models (LLMs); GPT-4, GPT-4-turbo, Claude, and Gemini in generating guideline-concordant responses to clinical questions related to urolithiasis. A total of 105 clinical questions were independently developed by the authors based on urolithiasis management principles. Each LLM generated responses in two separate sessions. Two experienced urologists evaluated the outputs for accuracy and concordance with guideline recommendations. Inter-rater agreement analysis demonstrated fair agreement between evaluators. Differences across models and guideline categories were assessed using appropriate statistical tests. All four LLMs demonstrated high guideline adherence, with mean total scores ranging from 86.5 ± 5.2 (Gemini) to 92.8 ± 3.1 (Claude). Claude achieved the highest correlation with expert ratings ( r = 0.94, p < 0.01). There were no statistically significant differences across models or among the nine clinical categories ( p > 0.05). Session-to-session repeatability was also high for all models, with intra-model correlation coefficients exceeding 0.90. LLMs, particularly Claude, can provide reliable, guideline-consistent answers to urolithiasis-related clinical queries. Their consistent performance across themes suggests utility as adjunctive informational tools for guideline-based urological education and support, although further validation in real-world clinical settings remains necessary.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsUrolithiasisDecision Support Systems, ClinicalUrologyHumansPractice Guidelines as TopicGuideline Adherence

Stichwörter der Autor:innen

Artificial intelligenceClinical decision supportGuideline adherenceLarge language modelsNatural language processingUrolithiasis

Autor:innen und Institutionen

  • Faculty of Medicine, Department of Urology, Uşak University, Uşak, Turkey. dr_adem65@hotmail.com.
  • Faculty of Medicine, Department of Urology, Uşak University, Uşak, Turkey.

Förderung

  • Usak University

Zitieren

APA

Tunçekin, A., & Aktaş, Y. (2026). Performance of large language models in urological decision support: a guideline-based comparative evaluation in urolithiasis. Urolithiasis. https://doi.org/10.1007/s00240-026-02013-1

Vancouver

Tunçekin A, Aktaş Y. Performance of large language models in urological decision support: a guideline-based comparative evaluation in urolithiasis. Urolithiasis. 2026. doi:10.1007/s00240-026-02013-1

BibTeX
@article{Tunekin2026,
  author = {Tunçekin, Adem and Aktaş, Yasin},
  title = {Performance of large language models in urological decision support: a guideline-based comparative evaluation in urolithiasis},
  journal = {Urolithiasis},
  year = {2026},
  doi = {10.1007/s00240-026-02013-1}
}
Kostenlos starten →