Accuracy, Clarity, and Comprehensiveness of ChatGPT Outputs for Commonly Asked Questions About Living Kidney Donation
Clinical Transplantation · 2025
DOI: 10.1111/ctr.70303 ↗At a glance
Journal article in Clinical Transplantation, 2025. Written by Ria Singla and 10 co-authors. Institution: Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. Major MEDLINE topics: Living Donors, Kidney Transplantation, Tissue and Organ Procurement, Nephrectomy. Cited 2 times so far (Crossref); 19 references. The full text is freely available (open access, CC BY-NC-ND 4.0).
Abstract (original language)
Introduction
The effectiveness of ChatGPT responses to common living kidney donation (LKD) queries remains unclear.
Methods
We surveyed nephrologists and living kidney donors/candidates to evaluate ChatGPT‐3.5's accuracy, comprehensiveness, and clarity in answering common donation questions in English and French. Ratings used a 5‐point Likert scale, with percentage agreement and modified Fleiss’ Kappa measuring inter‐rater consistency.
Results
The evaluation of ChatGPT‐3.5's responses varied between nephrologists and kidney donors/candidates. Nephrologists showed moderate percentage agreement for English responses (50%–59%) and poor agreement for French responses (9%–45%). Kidney donors/candidates exhibited high agreement for English (90%–100%) but low for French (0%–77%). Inter‐rater agreement among nephrologists was moderate for both English (Kappa 0.74, 95% CI: 0.67, 0.79, p < 0.0001) and French (Kappa 0.70, 95% CI: 0.64, 0.77, p < 0.0001). In contrast, inter‐rater agreement was poor among donors/candidates for both English (Kappa −0.10, 95% CI: −0.14, −0.07, p = 0.99) and French (Kappa −0.03, 95% CI: −0.07, 0, p = 0.81).
Conclusion
ChatGPT 3.5's responses to common LKD queries demonstrated limited agreement among nephrologists and kidney donors/donor candidates, highlighting its lack of reliability as a supplement to existing educational materials for living kidney donor programs in English and French.
Subjects
MeSH terms (MEDLINE)
Authors and institutions
- Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
- Division of Nephrology, Department of Medicine, The Ottawa Hospital and Kidney Research Centre, Ottawa, Canada.
ORCID profiles: Ann Bugeja
Cite
APA
Singla, R., Lodhi, S., Kibret, T., Jegatheswaran, J., Glavinovic, T., Massicotte-Azarniouch, D., Karpinski, J., Powell, R., Burns, K., Sood, M. M., & Bugeja, A. (2025). Accuracy, Clarity, and Comprehensiveness of ChatGPT Outputs for Commonly Asked Questions About Living Kidney Donation. Clinical Transplantation. https://doi.org/10.1111/ctr.70303
Vancouver
Singla R, Lodhi S, Kibret T, Jegatheswaran J, Glavinovic T, Massicotte-Azarniouch D, et al. Accuracy, Clarity, and Comprehensiveness of ChatGPT Outputs for Commonly Asked Questions About Living Kidney Donation. Clinical Transplantation. 2025. doi:10.1111/ctr.70303
BibTeX
@article{Singla2025,
author = {Singla, Ria and Lodhi, Sumiya and Kibret, Taddele and Jegatheswaran, Januvi and Glavinovic, Tamara and Massicotte-Azarniouch, David and Karpinski, Jolanta and Powell, Rinu and Burns, Kevin and Sood, Manish M and Bugeja, Ann},
title = {Accuracy, Clarity, and Comprehensiveness of ChatGPT Outputs for Commonly Asked Questions About Living Kidney Donation},
journal = {Clinical Transplantation},
year = {2025},
doi = {10.1111/ctr.70303}
}
EUVIMED