Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation
Genauigkeit, Verständlichkeit und Qualität von KI-Chatbot-Antworten für Patientenaufklärung zu intermittierendem Fasten bei chronischer Nierenerkrankung: prospektive strukturierte Bewertung
Frontiers in Nephrology · 29.05.2026 · 6
DOI: 10.3389/fneph.2026.1857793 ↗Auf einen Blick
ChatGPT-5 und Claude 4.5 Sonnet beantworteten 25 Fragen zu intermittierendem Fasten bei CKD klinisch insgesamt korrekt und vorsichtig. ChatGPT-5 war besser an die Gesundheitskompetenz Erwachsener angepasst.
Fragestellung
Wie genau, konsistent, verständlich und qualitativ sind Antworten von ChatGPT-5 und Claude 4.5 Sonnet zu intermittierendem Fasten bei CKD?
Methode
Prospektive strukturierte Evaluation mit 25 patientenzentrierten Fragen in 7 klinischen Domänen; jede Frage wurde beiden Modellen zweimal unter Standardbedingungen gestellt.
Ergebnisse
Beide Modelle zeigten in den ersten Antworten identische Genauigkeit und nach erneutem Prompting nach 24 Stunden eine Verbesserung. Die interne Konsistenz lag bei 90 %; DISCERN war vergleichbar.
Schlussfolgerung der Autor:innen
Beide Chatbots lieferten klinisch korrekte und vorsichtige Antworten zu intermittierendem Fasten bei CKD. Die Autor:innen raten dennoch davon ab, sie als primäre Quelle für Beratung zu nutzen.
Kernaussagen
- 25 Fragen zu intermittierendem Fasten bei CKD
- Interne Konsistenz beider Modelle: 90 %
- ChatGPT-5 besser an die Gesundheitskompetenz Erwachsener angepasst
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Nephrology, online erschienen am 29.05.2026. Verfasst von Joel Shah und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Internal Medicine, Texas Tech University Health Sciences Center, El Paso, TX, United States. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Intermittent fasting (IF) represents a dietary strategy with known metabolic benefits, however its safety and applicability in patients with chronic kidney disease (CKD) remains poorly understood. AI chatbots’ reliability for clinical and dietary guidance in clinically complex areas remains underexplored. No study has examined AI performance on IF in patients with CKD. This study evaluated the accuracy, consistency, readability, and quality of responses generated by ChatGPT-5 and Claude 4.5 Sonnet regarding IF in CKD.
Methods
Twenty-five patient-centered questions spanning seven clinical domains were developed by two board-certified nephrologists and submitted independently to ChatGPT-5 and Claude 4.5 Sonnet twice per model. Prompts were submitted under standard and unmodified user conditions without customized system instructions to represent a zero-shot evaluation that is illustrative of typical patient use. Responses were evaluated for accuracy, consistency (comparatively between responses), and appropriateness by the evaluators. Response readability and quality were assessed using the Flesch-Kincaid Grade Level (FKGL) formula and the DISCERN instrument respectively.
Results
Both ChatGPT-5 and Claude 4.5 Sonnet demonstrated identical accuracy scores during the first question prompts with notable improvement upon re-prompting after 24-hours. Both models demonstrated an identical internal consistency at 90%. ChatGPT-5 produced responses at the middle school level, better aligned with the US adult health literacy benchmark, while Claude 4.5 Sonnet generated responses at the high school level. DISCERN scoring was comparable between chatbots indicative of good response quality. Both chatbots had similar performance regarding individual subdomains.
Conclusions
Both chatbots illustrated clinically accurate and cautious responses to questions regarding IF in CKD. ChatGPT-5 responses were better suited to the average health literacy of the general United States adult population. Despite our findings, we do not recommend either chatbot be used as the primary source for clinical counseling regarding IF in CKD. It is important for clinicians to help to guide patients toward evidence-based clinical information.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Internal Medicine, Texas Tech University Health Sciences Center, El Paso, TX, United States.
- Department of Education and Internal Medicine, Texas Tech University Health Sciences Center, El Paso, TX, United States.
- Department of Nephrology, Carl T Hayden VA Medical Center, University of Arizona, Phoenix, AZ, United States.
- Department of Nephrology and Hypertension, Mayo Clinic, Rochester, MN, United States.
Zitieren
APA
Shah, J., Wells-Gatnik, W., Palmer, B., Acharya, P., & Cheungpasitporn, W. (2026). Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation. Frontiers in Nephrology, 6. https://doi.org/10.3389/fneph.2026.1857793
Vancouver
Shah J, Wells-Gatnik W, Palmer B, Acharya P, Cheungpasitporn W. Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation. Frontiers in Nephrology. 2026;6. doi:10.3389/fneph.2026.1857793
BibTeX
@article{Shah2026,
author = {Shah, Joel and Wells-Gatnik, William and Palmer, Biff and Acharya, Prakrati and Cheungpasitporn, Wisit},
title = {Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation},
journal = {Frontiers in Nephrology},
year = {2026},
volume = {6},
doi = {10.3389/fneph.2026.1857793}
}
EUVIMED