EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma

Kameel Khabaz, Nicole J Newman-Hung, Jennifer R Kallini, Joseph Kendal, Alexander B Christ, Nicholas M Bernthal, Lauren E Wessel iD

Journal of Surgical Oncology · 2025-03 · 131 (4) · 719-724

DOI: 10.1002/jso.27966 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Surgical Oncology, online erschienen am 29.10.2024. Verfasst von Kameel Khabaz und 6 weiteren Autor:innen. Beteiligte Einrichtung: David Geffen School of Medicine at UCLA, Los Angeles, California, USA. Hauptthemen laut MEDLINE: Artificial Intelligence, Bone Neoplasms, Patient Education as Topic, Osteosarcoma. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NIAMS NIH HHS, The authors received no specific funding for this work..

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background and Objectives The potential impacts of artificial intelligence (AI) chatbots on care for patients with bone sarcoma is poorly understood. Elucidating potential risks and benefits would allow surgeons to define appropriate roles for these tools in clinical care.

Methods

Eleven questions on bone sarcoma diagnosis, treatment, and recovery were inputted into three AI chatbots. Answers were assessed on a 5‐point Likert scale for five clinical accuracy metrics: relevance to the question, balance and lack of bias, basis on established data, factual accuracy, and completeness in scope. Responses were quantitatively assessed for empathy and readability. The Patient Education Materials Assessment Tool (PEMAT) was assessed for understandability and actionability.

Results

Chatbots scored highly on relevance (4.24) and balance/lack of bias (4.09) but lower on basing responses on established data (3.77), completeness (3.68), and factual accuracy (3.66). Responses generally scored well on understandability (84.30%), while actionability scores were low for questions on treatment (64.58%) and recovery (60.64%). GPT‐4 exhibited the highest empathy (4.12). Readability scores averaged between 10.28 for diagnosis questions to 11.65 for recovery questions.

Conclusions

While AI chatbots are promising tools, current limitations in factual accuracy and completeness, as well as concerns of inaccessibility to populations with lower health literacy, may significantly limit their clinical utility.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceBone NeoplasmsPatient Education as TopicOsteosarcomaSarcomaHumansComprehensionSurveys and QuestionnairesGenerative Artificial Intelligence

Stichwörter der Autor:innen

Ewing sarcomaartificial intelligencebone sarcomaschatbotschondrosarcomaosteosarcoma

Autor:innen und Institutionen

  • David Geffen School of Medicine at UCLA, Los Angeles, California, USA.
  • Department of Orthopaedic Surgery, University of California, Los Angeles, California, USA.
  • Department of Surgery, University of Calgary, Calgary, Alberta, Canada.

ORCID-Profile: Lauren E Wessel

Förderung

  • NIAMS NIH HHS
  • The authors received no specific funding for this work.

Zitieren

APA

Khabaz, K., Newman-Hung, N. J., Kallini, J. R., Kendal, J., Christ, A. B., Bernthal, N. M., & Wessel, L. E. (2025). Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma. Journal of Surgical Oncology, 131(4), 719-724. https://doi.org/10.1002/jso.27966

Vancouver

Khabaz K, Newman-Hung NJ, Kallini JR, Kendal J, Christ AB, Bernthal NM, et al. Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma. Journal of Surgical Oncology. 2025;131(4):719-724. doi:10.1002/jso.27966

BibTeX
@article{Khabaz2025,
  author = {Khabaz, Kameel and Newman-Hung, Nicole J and Kallini, Jennifer R and Kendal, Joseph and Christ, Alexander B and Bernthal, Nicholas M and Wessel, Lauren E},
  title = {Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma},
  journal = {Journal of Surgical Oncology},
  year = {2025},
  volume = {131},
  number = {4},
  pages = {719--724},
  doi = {10.1002/jso.27966}
}
Kostenlos starten →