EUVIMEDEUVIMED entdecken
Leitlinie

Accuracy, Completeness, and Clarity of an AI-Based Chatbot for the EAU Neuro-Urology Guidelines

Sabrina De Cillis, Riccardo Lombardo, Daniele Amparore, Matteo Romagnoli, Marta Santioni, Manfredi Bruno Sequi, Federico Piramide, Alberto Piana, Alberto Quara', Matteo Manfredi, Eleonora Rosato, Mauro Ragonese und 16 weitere

European Urology Focus · 2026-08

DOI: 10.1016/j.euf.2026.06.025 ↗

Kurz gefasst

Leitlinie in der Zeitschrift European Urology Focus, 2026-08. Verfasst von Sabrina De Cillis und 27 weiteren Autor:innen. Beteiligte Einrichtung: Division of Urology, San Luigi Gonzaga Hospital, Department of Oncology, University of Turin, Orbassano, Italy. Die Arbeit nennt 20 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study aimed to externally validate the performance of the European Association of Urology (EAU) Guidelines Bot in neuro-urology by assessing the accuracy, completeness, and clarity of chatbot-generated answers to guideline-based questions and to compare its performance with that of a general-purpose large language model (ChatGPT 5.5). A cross-sectional validation study was conducted using 47 questions derived from the EAU Neuro-Urology Guidelines. Each question was linked to a specific recommendation and classified by recommendation strength (strong vs weak). Questions were independently submitted to both the EAU Guidelines Bot and ChatGPT 5.5 without additional prompting. Two expert urologists independently evaluated each response for accuracy, completeness, and clarity using a five-point Likert scale; discrepancies were resolved by a third reviewer. Overall, 45 questions (95.7%) were linked to strong recommendations and two (4.3%) to weak recommendations. The EAU Guidelines Bot and ChatGPT 5.5 achieved identical mean accuracy scores (4.96 ± 0.20), with all responses rated as highly accurate (Likert 4-5). ChatGPT 5.5 indicated significantly higher completeness scores than did the EAU Guidelines Bot (4.74 ± 0.44 vs 4.57 ± 0.54; p = 0.011), whereas clarity scores were not significantly different (4.83 ± 0.38 vs 4.77 ± 0.43; p = 0.083). High-quality completeness was observed in 46/47 EAU Guidelines Bot responses (97.9%) and 47/47 ChatGPT responses (100%). Score discrepancies between systems were identified in ten of 47 questions (21.3%) and were limited to completeness and clarity domains. Performance remained uniformly high across recommendation grades, with no meaningful differences observed. The EAU Guidelines Bot showed excellent accuracy, completeness, and clarity when applied to neuro-urology guideline-based questions. Its performance was comparable to that of ChatGPT 5.5, with both systems providing highly accurate guideline-concordant responses. Although ChatGPT 5.5 generated more comprehensive answers, the EAU Guidelines Bot maintained closer adherence to the original guideline recommendations. Although not a substitute for clinical judgment, the tool appears to be a reliable adjunct for rapid access to evidence-based neuro-urological guidance.

Schlagwörter

Stichwörter der Autor:innen

Artificial intelligenceClinical guidelinesDecision support systemsNeurogenic lower urinary tract dysfunction

Autor:innen und Institutionen

  • Division of Urology, San Luigi Gonzaga Hospital, Department of Oncology, University of Turin, Orbassano, Italy.
  • Department of Urology, Sapienza University of Rome, Rome, Italy. Electronic address: rlombardo@me.com.
  • Department of Oncology, University of Turin, Orbassano, Italy; Division of Urology, Department of Surgery, Candiolo Cancer Institute FPO-IRCCS, Candiolo, Italy.
  • Department of Urology, Sapienza University of Rome, Rome, Italy.
  • Department of Urology, University of Tor Vergata, Rome, Italy.
  • Department of Urology, Gemelli University, Rome, Italy.

Zitieren

APA

De Cillis, S., Lombardo, R., Amparore, D., Romagnoli, M., Santioni, M., Sequi, M. B., Piramide, F., Piana, A., Quara', A., Manfredi, M., Rosato, E., Ragonese, M., Santarelli, V., Spatafora, P., Gallo, G., Pagliarulo, V., Tema, G., Franco, A., Pastore, A. L., …, & Next-Gen Research Group (2026). Accuracy, Completeness, and Clarity of an AI-Based Chatbot for the EAU Neuro-Urology Guidelines. European Urology Focus. https://doi.org/10.1016/j.euf.2026.06.025

Vancouver

De Cillis S, Lombardo R, Amparore D, Romagnoli M, Santioni M, Sequi MB, et al. Accuracy, Completeness, and Clarity of an AI-Based Chatbot for the EAU Neuro-Urology Guidelines. European Urology Focus. 2026. doi:10.1016/j.euf.2026.06.025

BibTeX
@article{DeCillis2026,
  author = {De Cillis, Sabrina and Lombardo, Riccardo and Amparore, Daniele and Romagnoli, Matteo and Santioni, Marta and Sequi, Manfredi Bruno and Piramide, Federico and Piana, Alberto and Quara', Alberto and Manfredi, Matteo and Rosato, Eleonora and Ragonese, Mauro and Santarelli, Valerio and Spatafora, Pietro and Gallo, Giacomo and Pagliarulo, Vincenzo and Tema, Giorgia and Franco, Antonio and Pastore, Antonio Luigi and Rocco, Bernardo and Gacci, Mauro and Fiori, Cristian and Finazzi, Enrico and Sciarra, Alessandro and Tubaro, Andrea and Porpiglia, Francesco and De Nunzio, Cosimo and Next-Gen Research Group, Next-Gen Research Group},
  title = {Accuracy, Completeness, and Clarity of an AI-Based Chatbot for the EAU Neuro-Urology Guidelines},
  journal = {European Urology Focus},
  year = {2026},
  doi = {10.1016/j.euf.2026.06.025}
}
Kostenlos starten →