EUVIMEDObjevte EUVIMED
VergleichsstudieOpen access · CC BY-NC 4.0

Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources

Harry Collin iD, Chelsea Tong iD, Abhishekh Srinivas, Angus Pegler, Philip Allan iD, Daniel Hagley

ANZ Journal of Surgery · 2025

DOI: 10.1111/ans.70053 ↗

At a glance

Vergleichsstudie in ANZ Journal of Surgery, 2025. Written by Harry Collin and 5 co-authors. Institution: Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia. Major MEDLINE topics: Aortic Aneurysm, Abdominal, Patient Education as Topic, Artificial Intelligence. Cited 10 times so far (Crossref); 34 references. The full text is freely available (open access, CC BY-NC 4.0).

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Backgrounds Abdominal aortic aneurysms (AAA) carry significant risks, yet patient understanding is often limited, with online resources typically low quality. ChatGPT, an artificial intelligence (AI) chatbot, presents a new frontier in patient education, but concerns remain about misinformation. This study evaluates the quality of ChatGPT‐generated patient information on AAA.

Methods

Eight patient questions on AAA were sourced from a reputable online resource for patient information funded by the Australian Government's Healthdirect Australia (HDA) website and input into ChatGPT's free (ChatGPT‐4o mini) and paid (ChatGPT‐4) models. A vascular surgeon evaluated response appropriateness. Readability was assessed using the Flesch–Kincaid test. The Patient Education Materials Assessment Tool (PEMAT) measured understandability and actionability, with responses scoring ≥75% for both considered high‐quality.

Results

All responses were deemed clinically appropriate. Mean response length was longer for ChatGPT than HDA. Readability was at a college level for ChatGPT, while HDA was at a 10th to 12th‐grade level. One response was high‐quality (generated by paid ChatGPT) with a PEMAT actionability score of ≥75%. Actionability scores were otherwise low across all sources with ChatGPT responses more likely to contain identifiable actions, although these were often not clearly presented. ChatGPT responses were marginally more understandable than HDA.

Conclusions

ChatGPT‐generated information on AAA was appropriate and understandable, outperforming HDA in both aspects. However, AI responses are at a more advanced reading level and lack actionable instructions. AI chatbots show promise as supplemental tools for AAA patient education, but further refinement is needed to enhance their effectiveness in supporting informed decision‐making.

Subjects

MeSH terms (MEDLINE)

Aortic Aneurysm, AbdominalPatient Education as TopicArtificial IntelligenceHumansAustraliaComprehensionMaleFemaleInternetGenerative Artificial Intelligence

Author keywords

AAAAIChatGPTaortic abdominal aneurysmartificial intelligencepatient informationvascular surgery

Authors and institutions

  • Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia.
  • Department of Urology, Mater Hospital, Brisbane, Queensland, Australia.
  • Department of Vascular and Endovascular Surgery, Mater Hospital, Brisbane, Queensland, Australia.
  • Department of Vascular and Endovascular Surgery, Northern Hospital, Melbourne, Victoria, Australia.
  • Department of Vascular and Endovascular Surgery, Princess Alexandra Hospital, Brisbane, Queensland, Australia.
  • Department of Vascular and Endovascular Surgery, Waikato Hospital, Hamilton, New Zealand.

ORCID profiles: Harry Collin, Chelsea Tong, Philip Allan

Cite

APA

Collin, H., Tong, C., Srinivas, A., Pegler, A., Allan, P., & Hagley, D. (2025). Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources. ANZ Journal of Surgery. https://doi.org/10.1111/ans.70053

Vancouver

Collin H, Tong C, Srinivas A, Pegler A, Allan P, Hagley D. Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources. ANZ Journal of Surgery. 2025. doi:10.1111/ans.70053

BibTeX
@article{Collin2025,
  author = {Collin, Harry and Tong, Chelsea and Srinivas, Abhishekh and Pegler, Angus and Allan, Philip and Hagley, Daniel},
  title = {Evaluating the role of AI chatbots in patient education for abdominal aortic aneurysms: a comparison of ChatGPT and conventional resources},
  journal = {ANZ Journal of Surgery},
  year = {2025},
  doi = {10.1111/ans.70053}
}
Start for free →