EUVIMEDEUVIMED entdecken
Vergleichsstudie

Quality and clinical implications of AI chatbot responses to common patient questions about total knee arthroplasty: a multidimensional comparative study

Mehmet Ekici, Eyyüp Serdar Yalvaç, Kürşat Tuğrul Okur, Ekin Barış Demir, Muhammed Ali Demir, Çağrı Üner, Mehmet Aydemir

The Knee · 2026

DOI: 10.1016/j.knee.2026.104648 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift The Knee, 2026. Verfasst von Mehmet Ekici und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedics and Traumatology, Yozgat City Hospital, Yozgat, Türkiye. Electronic address: dr.mehmetekici.ortopedi@gmail.com. Hauptthemen laut MEDLINE: Arthroplasty, Replacement, Knee, Patient Education as Topic, Artificial Intelligence. Die Arbeit nennt 19 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

INTRODUCTION: Patients considering total knee arthroplasty (TKA) frequently seek online guidance on indications, risks, recovery, rehabilitation, and precautions. While authoritative sources exist, generative AI chatbots are increasingly used for rapid answers, yet their quality, usability, and transparency may vary. METHODS: In this cross-sectional comparative study, five common TKA patient FAQs were selected using clinic FAQs, AAOS OrthoInfo, search trends, and Delphi panel validation. Nine chatbots were accessed on 21 March 2026 using free public interfaces, in incognito sessions with cleared cookies. Queries were zero-shot, using only each question text. Each chatbot's five answers were pooled into a single "5-answer packet." Seven orthopaedic specialists, blinded to model identity, scored each packet using QUEST, DISCERN, PEMAT-Understandability/Actionability (PEMAT-U/A), JAMA Benchmarks, Trust (1-5), and Global Quality Score (GQS, 1-5). Inter-rater reliability was assessed via ICC (two-way random effects, absolute agreement, average measures). Model differences were tested using one-way ANOVA with Tukey HSD, reporting η2 effect sizes (α = 0.05). RESULTS: ICC-average ranged 0.765-0.941. All scales differed significantly by model (ANOVA: QUEST F = 5.931; DISCERN F = 4.182; PEMAT-U F = 6.394; PEMAT-A F = 4.948; JAMA F = 17.083; Trust F = 4.031; GQS F = 4.477; all p ≤ 0.001; η2 0.374-0.717). DeepSeek had the highest DISCERN mean (70.57 ± 2.64), ChatGPT-4o the highest PEMAT-U/A (86.62 ± 4.45; 83.38 ± 9.81), and Microsoft Copilot the highest JAMA mean (2.00 ± 0.00). CONCLUSION: AI chatbot performance for TKA FAQs is domain-dependent across quality, usability, and transparency. These findings have direct implications for orthopedic practice, particularly in preoperative patient counseling and shared decision-making in TKA.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Arthroplasty, Replacement, KneePatient Education as TopicArtificial IntelligenceHumansCross-Sectional StudiesReproducibility of ResultsGenerative Artificial IntelligenceSurveys and Questionnaires

Stichwörter der Autor:innen

DISCERNLarge language modelsPEMATPatient educationQUESTTotal knee arthroplasty

Autor:innen und Institutionen

  • Department of Orthopedics and Traumatology, Yozgat City Hospital, Yozgat, Türkiye. Electronic address: dr.mehmetekici.ortopedi@gmail.com.
  • Department of Orthopedics and Traumatology, Yozgat City Hospital, Yozgat, Türkiye.
  • Department of Orthopedics and Traumatology, Kanuni Training and Research Hospital, Trabzon, Türkiye.
  • Department of Orthopedics and Traumatology, Etlik City Hospital, Ankara, Türkiye.
  • Department of Orthopedics and Traumatology, Kayseri State Hospital, Kayseri, Türkiye.
  • Department of Orthopedics and Traumatology, Yozgat City Hospital, Yozgat, Türkiye; Department of Orthopedics and Traumatology, Faculty of Medicine, Kırıkkale University, Kırıkkale, Türkiye.

Zitieren

APA

Ekici, M., Yalvaç, E. S., Okur, K. T., Demir, E. B., Demir, M. A., Üner, Ç., & Aydemir, M. (2026). Quality and clinical implications of AI chatbot responses to common patient questions about total knee arthroplasty: a multidimensional comparative study. The Knee. https://doi.org/10.1016/j.knee.2026.104648

Vancouver

Ekici M, Yalvaç ES, Okur KT, Demir EB, Demir MA, Üner Ç, et al. Quality and clinical implications of AI chatbot responses to common patient questions about total knee arthroplasty: a multidimensional comparative study. The Knee. 2026. doi:10.1016/j.knee.2026.104648

BibTeX
@article{Ekici2026,
  author = {Ekici, Mehmet and Yalvaç, Eyyüp Serdar and Okur, Kürşat Tuğrul and Demir, Ekin Barış and Demir, Muhammed Ali and Üner, Çağrı and Aydemir, Mehmet},
  title = {Quality and clinical implications of AI chatbot responses to common patient questions about total knee arthroplasty: a multidimensional comparative study},
  journal = {The Knee},
  year = {2026},
  doi = {10.1016/j.knee.2026.104648}
}
Kostenlos starten →