EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors

C Bernal-Utrera, A Bravo-Vázquez, F J Montero-Bancalero, A Suárez-Vega, M J Casuso-Holgado, E Anarte-Lazo

Physiotherapy · 2026-03 · 130 · 101838

DOI: 10.1016/j.physio.2025.101838 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Physiotherapy, 2026-03. Verfasst von C Bernal-Utrera und 5 weiteren Autor:innen. Beteiligte Einrichtung: Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain. Hauptthemen laut MEDLINE: Breast Neoplasms, Cancer Survivors, Exercise Therapy, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

OBJECTIVE: To assess the accuracy and reliability of an artificial intelligence (AI) chatbot (ChatGPT) in providing answers about exercise recommendations for breast cancer survivors. DESIGN: Cross-sectional study. METHODS: We extracted recommendations from recent systematic reviews of clinical practice guidelines (CPGs) on therapeutic physical exercise in breast cancer survivors. Clinical questions were developed and queried to ChatGPT-4. We evaluated the performance of ChatGPT-4 as a counseling tool by assessing the accuracy of responses (percentage of agreement with CPGs recommendations, weighted Cohen's kappa and the percentage of text wording similarity) and the intra-rater and inter-rater reliability in grading ChatGPT answers (kappa value). RESULTS: We tested 15 clinical questions. The accuracy of the AI chatbot´s responses was considered unacceptable, with only 7% of responses being reasonably accurate (1/14), 64% of responses being partially accurate (9/14) and 29% being completely incorrect (4/14). It was observed a low Kappa coefficient (k = 0.244, CI: 0.089 to 0.577) and the similarity of responses was also considered unacceptable, with 27.2% of overlapping text wording. Intra and inter-rater reliability showed moderate to good values in all cases. CONCLUSIONS: ChatGPT does not appear to be an accurate counselling tool for answering questions about exercise recommendations for breast cancer survivors. Compared to CPG´s recommendations, the accuracy of ChatGPT responses was considered poor, with moderate to good reliability. It is important to make patients know that they should not only base their decisions on information coming from ChatGPT. CONTRIBUTION OF THE PAPER.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Breast NeoplasmsCancer SurvivorsExercise TherapyArtificial IntelligenceHumansFemaleCross-Sectional StudiesReproducibility of ResultsPractice Guidelines as TopicMiddle AgedSurveys and QuestionnairesGenerative Artificial Intelligence

Stichwörter der Autor:innen

Artificial intelligenceBreast cancerChatGPTTherapeutic exercise

Autor:innen und Institutionen

  • Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain.
  • Programa Doctoral en Ciencias de la Salud, Universidad de Sevilla, Sevilla, Spain.
  • Escuela Universitaria de Osuna (Centre Attached to the University of Seville), Sevilla, Spain.
  • Emergency Department, Juan Ramón Jiménez University Hospital, Huelva, Spain.
  • Departamento de Fisioterapia, Facultad de Enfermería, Fisioterapia y Podología, Universidad de Sevilla, Sevilla, Spain; Instituto de Biomedicina de Sevilla-IBiS (Hospitales Universitarios Virgen del Rocío y Macarena/CSI…
  • Faculty of Health, UNIE University, Madrid, Spain.

Zitieren

APA

Bernal-Utrera, C., Bravo-Vázquez, A., Montero-Bancalero, F. J., Suárez-Vega, A., Casuso-Holgado, M. J., & Anarte-Lazo, E. (2026). Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors. Physiotherapy, 130, 101838. https://doi.org/10.1016/j.physio.2025.101838

Vancouver

Bernal-Utrera C, Bravo-Vázquez A, Montero-Bancalero FJ, Suárez-Vega A, Casuso-Holgado MJ, Anarte-Lazo E. Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors. Physiotherapy. 2026;130:101838. doi:10.1016/j.physio.2025.101838

BibTeX
@article{BernalUtrera2026,
  author = {Bernal-Utrera, C and Bravo-Vázquez, A and Montero-Bancalero, F J and Suárez-Vega, A and Casuso-Holgado, M J and Anarte-Lazo, E},
  title = {Evaluation of ChatGPT accuracy and reliability in answering questions about exercise recommendations for breast cancer survivors},
  journal = {Physiotherapy},
  year = {2026},
  volume = {130},
  pages = {101838},
  doi = {10.1016/j.physio.2025.101838}
}
Kostenlos starten →