EUVIMEDEUVIMED entdecken
Crossref

Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients

Ethan Chervonski iD, Keerthi B Harish, Caron B Rockman, Mikel Sadek, Katherine A Teter iD, Glenn R Jacobowitz, Todd L Berland, Joann Lohr, Colleen Moore, Thomas S Maldonado

Vascular · 2025

DOI: 10.1177/17085381241240550 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Vascular, 2025. Verfasst von Ethan Chervonski und 9 weiteren Autor:innen. Beteiligte Einrichtung: New York University Grossman School of Medicine, New York, NY, USA. Hauptthemen laut MEDLINE: Artificial Intelligence, Vascular Surgical Procedures, Patient Education as Topic, Health Knowledge, Attitudes, Practice. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 32 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Generative artificial intelligence (AI) has emerged as a promising tool to engage with patients. The objective of this study was to assess the quality of AI responses to common patient questions regarding vascular surgery disease processes.

Methods

OpenAI’s ChatGPT-3.5 and Google Bard were queried with 24 mock patient questions spanning seven vascular surgery disease domains. Six experienced vascular surgery faculty at a tertiary academic center independently graded AI responses on their accuracy (rated 1–4 from completely inaccurate to completely accurate), completeness (rated 1–4 from totally incomplete to totally complete), and appropriateness (binary). Responses were also evaluated with three readability scales.

Results

ChatGPT responses were rated, on average, more accurate than Bard responses (3.08 ± 0.33 vs 2.82 ± 0.40, p < .01). ChatGPT responses were scored, on average, more complete than Bard responses (2.98 ± 0.34 vs 2.62 ± 0.36, p < .01). Most ChatGPT responses (75.0%, n = 18) and almost half of Bard responses (45.8%, n = 11) were unanimously deemed appropriate. Almost one-third of Bard responses (29.2%, n = 7) were deemed inappropriate by at least two reviewers (29.2%), and two Bard responses (8.4%) were considered inappropriate by the majority. The mean Flesch Reading Ease, Flesch–Kincaid Grade Level, and Gunning Fog Index of ChatGPT responses were 29.4 ± 10.8, 14.5 ± 2.2, and 17.7 ± 3.1, respectively, indicating that responses were readable with a post-secondary education. Bard’s mean readability scores were 58.9 ± 10.5, 8.2 ± 1.7, and 11.0 ± 2.0, respectively, indicating that responses were readable with a high-school education ( p < .0001 for three metrics). ChatGPT’s mean response length (332 ± 79 words) was higher than Bard’s mean response length (183 ± 53 words, p < .001). There was no difference in the accuracy, completeness, readability, or response length of ChatGPT or Bard between disease domains ( p > .05 for all analyses).

Conclusions

AI offers a novel means of educating patients that avoids the inundation of information from “Dr Google” and the time barriers of physician-patient encounters. ChatGPT provides largely valid, though imperfect, responses to myriad patient questions at the expense of readability. While Bard responses are more readable and concise, their quality is poorer. Further research is warranted to better understand failure points for large language models in vascular surgery patient education.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceVascular Surgical ProceduresPatient Education as TopicHealth Knowledge, Attitudes, PracticeSocial MediaHumansComprehensionSurveys and QuestionnairesHealth LiteracyGenerative Artificial Intelligence

Stichwörter der Autor:innen

ChatGPTVascular surgeryartificial intelligencegoogle bardpatient educationreadability

Autor:innen und Institutionen

  • New York University Grossman School of Medicine, New York, NY, USA.
  • Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA.
  • Dorn Veterans Affairs Medical Center, Columbia, SC, USA.
  • InVein Clinic, Cape Girardeau, MO, USA.

ORCID-Profile: Ethan Chervonski, Katherine A Teter

Zitieren

APA

Chervonski, E., Harish, K. B., Rockman, C. B., Sadek, M., Teter, K. A., Jacobowitz, G. R., Berland, T. L., Lohr, J., Moore, C., & Maldonado, T. S. (2025). Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients. Vascular. https://doi.org/10.1177/17085381241240550

Vancouver

Chervonski E, Harish KB, Rockman CB, Sadek M, Teter KA, Jacobowitz GR, et al. Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients. Vascular. 2025. doi:10.1177/17085381241240550

BibTeX
@article{Chervonski2025,
  author = {Chervonski, Ethan and Harish, Keerthi B and Rockman, Caron B and Sadek, Mikel and Teter, Katherine A and Jacobowitz, Glenn R and Berland, Todd L and Lohr, Joann and Moore, Colleen and Maldonado, Thomas S},
  title = {Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients},
  journal = {Vascular},
  year = {2025},
  doi = {10.1177/17085381241240550}
}
Kostenlos starten →