Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation
Journal of Orthopaedic Surgery · 2026-09 · 34 (3)
DOI: 10.1177/10225536261504909 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Orthopaedic Surgery, online erschienen am 05.11.2026. Verfasst von Kamil Balaban und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedics and Traumatology, Ministry of Health Finike State Hospital, Antalya, Turkey. Hauptthemen laut MEDLINE: Comprehension, Amputation, Surgical, Patient Education as Topic, Lower Extremity. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Objective
To compare the clinical accuracy, information quality and readability of chatbot responses to common patient questions about lower extremity amputation.
Methods
Twenty frequently asked patient questions were submitted once to each of ChatGPT, Gemini and DeepSeek in separate sessions. Two reviewers, blinded to chatbot identity, independently assessed responses using a four-category clinical accuracy scale and the DISCERN instrument. Readability was quantified as the mean grade level across the Flesch-Kincaid Grade Level, Gunning Fog, Coleman-Liau, and Simple Measure of Gobbledygook indices. Comparisons used Friedman tests and Holm-adjusted Wilcoxon signed-rank tests.
Results
Sixty responses were analysed. Mean (±standard deviation) clinical accuracy scores were 2.20 ± 0.57, 1.55 ± 0.65 and 1.20 ± 0.38 for ChatGPT, Gemini and DeepSeek, respectively (lower scores indicate greater accuracy). Gemini and DeepSeek received lower scores than ChatGPT (adjusted p=0.012 and p=0.002, respectively), whereas the Gemini–DeepSeek difference was not significant (p=0.090). Mean DISCERN scores were 50.05 ± 5.78, 53.33 ± 5.43 and 58.23 ± 3.39, respectively, with significant differences in all pairwise comparisons. Mean reading grades were 12.93 ± 1.28, 13.17 ± 1.36 and 11.74 ± 1.15, respectively; DeepSeek had lower grades than both comparators (both adjusted p
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Orthopedics and Traumatology, Ministry of Health Finike State Hospital, Antalya, Turkey.
- Department of Orthopedics and Traumatology, Ankara University School of Medicine, Ankara, Turkey.
ORCID-Profile: Kamil Balaban, Mehmet Batu Ertan
Zitieren
APA
Balaban, K., Ertan, M. B., & Kalem, M. (2026). Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation. Journal of Orthopaedic Surgery, 34(3). https://doi.org/10.1177/10225536261504909
Vancouver
Balaban K, Ertan MB, Kalem M. Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation. Journal of Orthopaedic Surgery. 2026;34(3). doi:10.1177/10225536261504909
BibTeX
@article{Balaban2026,
author = {Balaban, Kamil and Ertan, Mehmet Batu and Kalem, Mahmut},
title = {Clinical accuracy, information quality and readability of chatbot responses to patient questions about lower extremity amputation},
journal = {Journal of Orthopaedic Surgery},
year = {2026},
volume = {34},
number = {3},
doi = {10.1177/10225536261504909}
}
EUVIMED