Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources
ANZ Journal of Surgery · 2025
DOI: 10.1111/ans.70053 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift ANZ Journal of Surgery, 2025. Verfasst von Harry Collin und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia. Hauptthemen laut MEDLINE: Aortic Aneurysm, Abdominal, Patient Education as Topic, Artificial Intelligence. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Themen: Selbstmanagement bei chronischen Erkrankungen, Künstliche Intelligenz in der Pflege
Abstract (Originalsprache)
Backgrounds Abdominal aortic aneurysms (AAA) carry significant risks, yet patient understanding is often limited, with online resources typically low quality. ChatGPT, an artificial intelligence (AI) chatbot, presents a new frontier in patient education, but concerns remain about misinformation. This study evaluates the quality of ChatGPT‐generated patient information on AAA.
Methods
Eight patient questions on AAA were sourced from a reputable online resource for patient information funded by the Australian Government's Healthdirect Australia (HDA) website and input into ChatGPT's free (ChatGPT‐4o mini) and paid (ChatGPT‐4) models. A vascular surgeon evaluated response appropriateness. Readability was assessed using the Flesch–Kincaid test. The Patient Education Materials Assessment Tool (PEMAT) measured understandability and actionability, with responses scoring ≥75% for both considered high‐quality.
Results
All responses were deemed clinically appropriate. Mean response length was longer for ChatGPT than HDA. Readability was at a college level for ChatGPT, while HDA was at a 10th to 12th‐grade level. One response was high‐quality (generated by paid ChatGPT) with a PEMAT actionability score of ≥75%. Actionability scores were otherwise low across all sources with ChatGPT responses more likely to contain identifiable actions, although these were often not clearly presented. ChatGPT responses were marginally more understandable than HDA.
Conclusions
ChatGPT‐generated information on AAA was appropriate and understandable, outperforming HDA in both aspects. However, AI responses are at a more advanced reading level and lack actionable instructions. AI chatbots show promise as supplemental tools for AAA patient education, but further refinement is needed to enhance their effectiveness in supporting informed decision‐making.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia.
- Department of Urology, Mater Hospital, Brisbane, Queensland, Australia.
- Department of Vascular and Endovascular Surgery, Mater Hospital, Brisbane, Queensland, Australia.
- Department of Vascular and Endovascular Surgery, Northern Hospital, Melbourne, Victoria, Australia.
- Department of Vascular and Endovascular Surgery, Princess Alexandra Hospital, Brisbane, Queensland, Australia.
- Department of Vascular and Endovascular Surgery, Waikato Hospital, Hamilton, New Zealand.
ORCID-Profile: Harry Collin, Chelsea Tong, Philip Allan
Zitieren
APA
Collin, H., Tong, C., Srinivas, A., Pegler, A., Allan, P., & Hagley, D. (2025). Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources. ANZ Journal of Surgery. https://doi.org/10.1111/ans.70053
Vancouver
Collin H, Tong C, Srinivas A, Pegler A, Allan P, Hagley D. Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources. ANZ Journal of Surgery. 2025. doi:10.1111/ans.70053
BibTeX
@article{Collin2025,
author = {Collin, Harry and Tong, Chelsea and Srinivas, Abhishekh and Pegler, Angus and Allan, Philip and Hagley, Daniel},
title = {Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources},
journal = {ANZ Journal of Surgery},
year = {2025},
doi = {10.1111/ans.70053}
}
EUVIMED