Evaluating AI Chatbot Responses to Postkidney Transplant Inquiries
Transplantation Proceedings · 2025-03 · 57 (2) · 394-405
DOI: 10.1016/j.transproceed.2024.12.028 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transplantation Proceedings, 2025-03. Verfasst von Yihua Zhan und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Urology, Sun Yat-sen Memorial Hospital, Guangzhou, China. Hauptthemen laut MEDLINE: Patient Education as Topic, Internet, Kidney Transplantation. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 10 Literaturangaben.
Abstract (Originalsprache)
This study evaluated the capability of three AI chatbots-ChatGPT 4.0, Claude 3.0, and Gemini Pro, as well as Google-in responding to common postkidney transplantation inquiries. We compiled a list of frequently asked postkidney transplant questions using Google and Bing. Response quality was rated on a 5-point Likert scale, while understandability and actionability were measured with the Patient Education Materials Assessment Tool (PEMAT). Readability was assessed using the Flesch Reading Ease and Flesch-Kincaid Grade Level metrics, with statistical analysis conducted via non-parametric tests, specifically the Kruskal-Wallis test, using SPSS. We gathered 127 questions, which were addressed by the chatbots and Google. The responses were of high quality (median Likert score: 4 [4,5]), good understandability (median PEMAT understandability score: 72.7% [62.5,77.8]), but poor actionability (median PEMAT operability score: 20% [0%-20%]). The readability was challenging (median Flesch Reading Ease score: 22.1 [8.7,34.8]), with a Flesch-Kincaid Grade Level akin to undergraduate-level text (median score: 14.7 [12.3,16.7]). Among the chatbots, Claude 3.0 provided the most reliable responses, though they required a higher reading level. ChatGPT 4.0 offered the most comprehensible responses. Moreover, Google did not outperform the chatbots in any of the scoring metrics.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Urology, Sun Yat-sen Memorial Hospital, Guangzhou, China.
- South China Normal University, Guangzhou, China.
- The Education University of Hong Kong, Hong Kong, China.
- Department of Urology, Sun Yat-sen Memorial Hospital, Guangzhou, China. Electronic address: yuhao26@mail.sysu.edu.cn.
Zitieren
APA
Zhan, Y., Chen, X., Ye, F., Wu, Z., Usman, M., Yuan, Z., Wu, H., Huang, J., & Yu, H. (2025). Evaluating AI Chatbot Responses to Postkidney Transplant Inquiries. Transplantation Proceedings, 57(2), 394-405. https://doi.org/10.1016/j.transproceed.2024.12.028
Vancouver
Zhan Y, Chen X, Ye F, Wu Z, Usman M, Yuan Z, et al. Evaluating AI Chatbot Responses to Postkidney Transplant Inquiries. Transplantation Proceedings. 2025;57(2):394-405. doi:10.1016/j.transproceed.2024.12.028
BibTeX
@article{Zhan2025,
author = {Zhan, Yihua and Chen, Xutao and Ye, Feihong and Wu, Zhikai and Usman, Muhammad and Yuan, Zhihan and Wu, Han and Huang, Jian and Yu, Hao},
title = {Evaluating AI Chatbot Responses to Postkidney Transplant Inquiries},
journal = {Transplantation Proceedings},
year = {2025},
volume = {57},
number = {2},
pages = {394--405},
doi = {10.1016/j.transproceed.2024.12.028}
}
EUVIMED