Evaluating ChatGPT’s Accuracy in Responding to Patient Education Questions on Acute Kidney Injury and Continuous Renal Replacement Therapy
Blood Purification · 26.04.2024 · 53 (9) · 725-731
DOI: 10.1159/000539065 ↗Kurz gefasst
Fachartikel in der Zeitschrift Blood Purification, online erschienen am 26.04.2024. Verfasst von Mohammad Salman Sheikh und 6 weiteren Autor:innen. Beteiligte Einrichtung: Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA. Hauptthemen laut MEDLINE: Acute Kidney Injury, Patient Education as Topic, Continuous Renal Replacement Therapy. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 24 Literaturangaben.
Themen: Selbstmanagement bei chronischen Erkrankungen, Künstliche Intelligenz in der Pflege
Abstract (Originalsprache)
Introduction
Acute kidney injury (AKI) and continuous renal replacement therapy (CRRT) are critical areas in nephrology. The effectiveness of ChatGPT in simpler, patient education-oriented questions has not been thoroughly assessed. This study evaluates the proficiency of ChatGPT 4.0 in responding to such questions, subjected to various linguistic alterations.
Methods
Eighty-nine questions were sourced from the Mayo Clinic Handbook for educating patients on AKI and CRRT. These questions were categorized as original, paraphrased with different interrogative adverbs, paraphrased resulting in incomplete sentences, and paraphrased containing misspelled words. Two nephrologists verified the questions for medical accuracy. A χ2 test was conducted to ascertain notable discrepancies in ChatGPT 4.0’s performance across these formats.
Results
ChatGPT provided notable accuracy in handling a variety of question formats for patient education in AKI and CRRT. Across all question types, ChatGPT demonstrated an accuracy of 97% for both original and adverb-altered questions and 98% for questions with incomplete sentences or misspellings. Specifically for AKI-related questions, the accuracy was consistently maintained at 97% for all versions. In the subset of CRRT-related questions, the tool achieved a 96% accuracy for original and adverb-altered questions, and this increased to 98% for questions with incomplete sentences or misspellings. The statistical analysis revealed no significant difference in performance across these varied question types (p value: 1.00 for AKI and 1.00 for CRRT), and there was no notable disparity between the artificial intelligence (AI)’s responses to AKI and CRRT questions (p value: 0.71).
Conclusion
ChatGPT 4.0 demonstrates consistent and high accuracy in interpreting and responding to queries related to AKI and CRRT, irrespective of linguistic modifications. These findings suggest that ChatGPT 4.0 has the potential to be a reliable support tool in the delivery of patient education, by accurately providing information across a range of question formats. Further research is needed to explore the direct impact of AI-generated responses on patient understanding and education outcomes.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA.
- Chakri Naruebodindra Medical Institute, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Salaya, Thailand.
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA.
Zitieren
APA
Sheikh, M. S., Thongprayoon, C., Suppadungsuk, S., Miao, J., Qureshi, F., Kashani, K., & Cheungpasitporn, W. (2024). Evaluating ChatGPT’s Accuracy in Responding to Patient Education Questions on Acute Kidney Injury and Continuous Renal Replacement Therapy. Blood Purification, 53(9), 725-731. https://doi.org/10.1159/000539065
Vancouver
Sheikh MS, Thongprayoon C, Suppadungsuk S, Miao J, Qureshi F, Kashani K, et al. Evaluating ChatGPT’s Accuracy in Responding to Patient Education Questions on Acute Kidney Injury and Continuous Renal Replacement Therapy. Blood Purification. 2024;53(9):725-731. doi:10.1159/000539065
BibTeX
@article{Sheikh2024,
author = {Sheikh, Mohammad Salman and Thongprayoon, Charat and Suppadungsuk, Supawadee and Miao, Jing and Qureshi, Fawad and Kashani, Kianoush and Cheungpasitporn, Wisit},
title = {Evaluating ChatGPT’s Accuracy in Responding to Patient Education Questions on Acute Kidney Injury and Continuous Renal Replacement Therapy},
journal = {Blood Purification},
year = {2024},
volume = {53},
number = {9},
pages = {725--731},
doi = {10.1159/000539065}
}
EUVIMED