Artificial Intelligence Chatbots as Patient Information Sources in Penile Cancer: A Multi-Platform Evaluation
Société Internationale d’Urologie Journal · 06.08.2026 · 7 (4) · 47
DOI: 10.3390/siuj7040047 ↗Kurz gefasst
Fachartikel in der Zeitschrift Société Internationale d’Urologie Journal, online erschienen am 06.08.2026. Verfasst von Kunind Oberoi und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background/Objectives: Penile cancer carries a disproportionate burden of stigma and delayed presentation, with affected men increasingly turning to artificial intelligence (AI) chatbots as an anonymous information source. This study evaluated the quality, readability, understandability, actionability and clinical accuracy of AI chatbot responses to standardised penile cancer patient queries across six publicly available platforms.
Methods
Fourteen standardised questions were submitted to ChatGPT-4o, Gemini, Perplexity, Microsoft Copilot, Claude, and DeepSeek, generating 84 responses. The responses were evaluated using DISCERN (information quality, scored 16–80), Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P) (understandability and actionability, scored 0–100%), and Flesch–Kincaid grade level (reading complexity, recommended threshold ≤Grade 8). Between-platform and between-domain comparisons used the Kruskal–Wallis test with Dunn’s post hoc analysis. Clinical accuracy was assessed against the 2026 European Association of Urology-American Society of Clinical Oncology (EAU-ASCO) Collaborative Guidelines on Penile Cancer using a three-point ordinal scale across 10 guideline-scorable questions (maximum 20 points per platform); the four questions not addressed by the guidelines were scored separately for factual accuracy against authoritative external evidence.
Results
No significant between-platform differences were identified for DISCERN (p = 0.796) or PEMAT-P understandability (p = 0.147). All platforms exceeded the 70% understandability adequacy threshold. Perplexity and Copilot demonstrated significantly higher actionability than all other platforms (100% vs. 75%, p < 0.001). No platforms achieved the recommended Grade 8 reading threshold, with Claude generating significantly more complex responses than ChatGPT-4o and Perplexity (grade 11.4 vs. 8.65 and 8.60, p < 0.05). Significant variation was identified across clinical domains for DISCERN (p < 0.001), Flesch–Kincaid grade level (p < 0.001), and word count (p < 0.001), with treatment-related questions achieving the highest information quality but also generating the most complex responses. Clinical accuracy scores ranged from 13/20 (ChatGPT-4o) to 16/20 (Copilot). Two critical errors were identified: Claude and DeepSeek both recommended bleomycin-containing chemotherapy regimens, directly contradicting the EAU-ASCO Strong recommendation against bleomycin due to pulmonary toxicity risk. Responses to the four survivorship and quality-of-life questions were factually accurate against external evidence in 23 of 24 cases.
Conclusions
AI chatbot responses to penile cancer patient queries are broadly understandable but consistently fail to meet recommended readability thresholds and provide limited actionable guidance. Two platforms recommended bleomycin-containing regimens against an EAU-ASCO Strong recommendation; no platforms achieved full guideline concordance. Urologists should counsel patients on the limitations of AI chatbots as a health information source.
Zitieren
APA
Oberoi, K., Hassan, S., Woon, D., & Sethi, K. (2026). Artificial Intelligence Chatbots as Patient Information Sources in Penile Cancer: A Multi-Platform Evaluation. Société Internationale d’Urologie Journal, 7(4), 47. https://doi.org/10.3390/siuj7040047
Vancouver
Oberoi K, Hassan S, Woon D, Sethi K. Artificial Intelligence Chatbots as Patient Information Sources in Penile Cancer: A Multi-Platform Evaluation. Société Internationale d’Urologie Journal. 2026;7(4):47. doi:10.3390/siuj7040047
BibTeX
@article{Oberoi2026,
author = {Oberoi, Kunind and Hassan, Sadia and Woon, Dixon and Sethi, Kapil},
title = {Artificial Intelligence Chatbots as Patient Information Sources in Penile Cancer: A Multi-Platform Evaluation},
journal = {Société Internationale d’Urologie Journal},
year = {2026},
volume = {7},
number = {4},
pages = {47},
doi = {10.3390/siuj7040047}
}
EUVIMED