Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors
Physiotherapy · 2026-03 · 130 · 101838
DOI: 10.1016/j.physio.2025.101838 ↗Kurz gefasst
Fachartikel in der Zeitschrift Physiotherapy, 2026-03. Verfasst von C Bernal-Utrera und 5 weiteren Autor:innen. Beteiligte Einrichtung: Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain. Hauptthemen laut MEDLINE: Breast Neoplasms, Cancer Survivors, Exercise Therapy, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
OBJECTIVE: To assess the accuracy and reliability of an artificial intelligence (AI) chatbot (ChatGPT) in providing answers about exercise recommendations for breast cancer survivors. DESIGN: Cross-sectional study. METHODS: We extracted recommendations from recent systematic reviews of clinical practice guidelines (CPGs) on therapeutic physical exercise in breast cancer survivors. Clinical questions were developed and queried to ChatGPT-4. We evaluated the performance of ChatGPT-4 as a counseling tool by assessing the accuracy of responses (percentage of agreement with CPGs recommendations, weighted Cohen's kappa and the percentage of text wording similarity) and the intra-rater and inter-rater reliability in grading ChatGPT answers (kappa value). RESULTS: We tested 15 clinical questions. The accuracy of the AI chatbot´s responses was considered unacceptable, with only 7% of responses being reasonably accurate (1/14), 64% of responses being partially accurate (9/14) and 29% being completely incorrect (4/14). It was observed a low Kappa coefficient (k = 0.244, CI: 0.089 to 0.577) and the similarity of responses was also considered unacceptable, with 27.2% of overlapping text wording. Intra and inter-rater reliability showed moderate to good values in all cases. CONCLUSIONS: ChatGPT does not appear to be an accurate counselling tool for answering questions about exercise recommendations for breast cancer survivors. Compared to CPG´s recommendations, the accuracy of ChatGPT responses was considered poor, with moderate to good reliability. It is important to make patients know that they should not only base their decisions on information coming from ChatGPT. CONTRIBUTION OF THE PAPER.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain.
- Programa Doctoral en Ciencias de la Salud, Universidad de Sevilla, Sevilla, Spain.
- Escuela Universitaria de Osuna (Centre Attached to the University of Seville), Sevilla, Spain.
- Emergency Department, Juan Ramón Jiménez University Hospital, Huelva, Spain.
- Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain; Instituto de Biomedicina de Sevilla-IBiS (Hospitales Universitarios Virgen del Rocío y Macarena/CSI…
- Faculty of Health, UNIE University, Madrid, Spain.
Zitieren
APA
Bernal-Utrera, C., Bravo-Vázquez, A., Montero-Bancalero, F. J., Suárez-Vega, A., Casuso-Holgado, M. J., & Anarte-Lazo, E. (2026). Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors. Physiotherapy, 130, 101838. https://doi.org/10.1016/j.physio.2025.101838
Vancouver
Bernal-Utrera C, Bravo-Vázquez A, Montero-Bancalero FJ, Suárez-Vega A, Casuso-Holgado MJ, Anarte-Lazo E. Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors. Physiotherapy. 2026;130:101838. doi:10.1016/j.physio.2025.101838
BibTeX
@article{BernalUtrera2026,
author = {Bernal-Utrera, C and Bravo-Vázquez, A and Montero-Bancalero, F J and Suárez-Vega, A and Casuso-Holgado, M J and Anarte-Lazo, E},
title = {Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors},
journal = {Physiotherapy},
year = {2026},
volume = {130},
pages = {101838},
doi = {10.1016/j.physio.2025.101838}
}
EUVIMED