EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY-NC 4.0

Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation

Kamil Balaban iD, Mehmet Batu Ertan iD, Mahmut Kalem

Journal of Orthopaedic Surgery · 2026-09 · 34 (3)

DOI: 10.1177/10225536261504909 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Orthopaedic Surgery, online erschienen am 05.11.2026. Verfasst von Kamil Balaban und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedics and Traumatology, Ministry of Health Finike State Hospital, Antalya, Turkey. Hauptthemen laut MEDLINE: Comprehension, Amputation, Surgical, Patient Education as Topic, Lower Extremity. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To compare the clinical accuracy, information quality and readability of chatbot responses to common patient questions about lower extremity amputation.

Methods

Twenty frequently asked patient questions were submitted once to each of ChatGPT, Gemini and DeepSeek in separate sessions. Two reviewers, blinded to chatbot identity, independently assessed responses using a four-category clinical accuracy scale and the DISCERN instrument. Readability was quantified as the mean grade level across the Flesch-Kincaid Grade Level, Gunning Fog, Coleman-Liau, and Simple Measure of Gobbledygook indices. Comparisons used Friedman tests and Holm-adjusted Wilcoxon signed-rank tests.

Results

Sixty responses were analysed. Mean (±standard deviation) clinical accuracy scores were 2.20 ± 0.57, 1.55 ± 0.65 and 1.20 ± 0.38 for ChatGPT, Gemini and DeepSeek, respectively (lower scores indicate greater accuracy). Gemini and DeepSeek received lower scores than ChatGPT (adjusted p=0.012 and p=0.002, respectively), whereas the Gemini–DeepSeek difference was not significant (p=0.090). Mean DISCERN scores were 50.05 ± 5.78, 53.33 ± 5.43 and 58.23 ± 3.39, respectively, with significant differences in all pairwise comparisons. Mean reading grades were 12.93 ± 1.28, 13.17 ± 1.36 and 11.74 ± 1.15, respectively; DeepSeek had lower grades than both comparators (both adjusted p

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ComprehensionAmputation, SurgicalPatient Education as TopicLower ExtremityHumans

Stichwörter der Autor:innen

AmputationConsumer Health InformationLarge Language ModelsLower ExtremityPatient EducationSurgical

Autor:innen und Institutionen

  • Department of Orthopedics and Traumatology, Ministry of Health Finike State Hospital, Antalya, Turkey.
  • Department of Orthopedics and Traumatology, Ankara University School of Medicine, Ankara, Turkey.

ORCID-Profile: Kamil Balaban, Mehmet Batu Ertan

Zitieren

APA

Balaban, K., Ertan, M. B., & Kalem, M. (2026). Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation. Journal of Orthopaedic Surgery, 34(3). https://doi.org/10.1177/10225536261504909

Vancouver

Balaban K, Ertan MB, Kalem M. Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation. Journal of Orthopaedic Surgery. 2026;34(3). doi:10.1177/10225536261504909

BibTeX
@article{Balaban2026,
  author = {Balaban, Kamil and Ertan, Mehmet Batu and Kalem, Mahmut},
  title = {Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation},
  journal = {Journal of Orthopaedic Surgery},
  year = {2026},
  volume = {34},
  number = {3},
  doi = {10.1177/10225536261504909}
}
Kostenlos starten →