EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

How accurately do large language models answer patient questions on anterior cruciate ligament tears? A comparative study

Salim Youssef, Anton Brehmer, Peter Melcher, Ralf Henkelmann, Pierre Hepp, Jan Theopold, Maria Elze, Yasmin Youssef, Georg Osterhoff

The Knee · 2026

DOI: 10.1016/j.knee.2025.11.002 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift The Knee, 2026. Verfasst von Salim Youssef und 8 weiteren Autor:innen. Beteiligte Einrichtung: University of Leipzig, Department of Orthopedics, Trauma and Plastic Surgery, Leipzig, Germany. Electronic address: salim.youssef@medizin.uni-leipzig…. Hauptthemen laut MEDLINE: Anterior Cruciate Ligament Injuries, Language, Patient Education as Topic. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Large language models (LLMs) are increasingly used in the medical sector, raising questions about their reliability for patient education. With more LLMs becoming publicly available, it remains unclear whether meaningful performance differences exist between them. This is particularly relevant for anterior cruciate ligament (ACL) injuries, which mainly affect young, active individuals, those most likely to seek health advice from AI. This study aimed to evaluate and directly compare the accuracy of five leading LLMs in answering common patient questions about ACL tears. METHODS: Fourteen commonly asked patient questions were identified in a systematic online search. Each question was submitted to five LLMs: ChatGPT-4, Gemini 2.0, Llama 3.1, DeepSeek-V3, and Grok3. Responses were assessed for accuracy by orthopedic consultants using a five-point Likert scale. Word count was recorded as a proxy for readability. Statistical analysis included ANOVA by Tukey's HSD post hoc test. RESULTS: All models achieved mean accuracy scores ≥3 (mostly accurate). DeepSeek (3.61) and Grok (3.59) demonstrated significantly higher mean accuracies than Llama (3.25; P < 0.05). ChatGPT and Gemini achieved mean scores of 3.48 and 3.52, respectively. Models generating longer responses, such as Grok and DeepSeek, tended to offer greater accuracy, whereas Llama produced the shortest and least accurate answers. CONCLUSIONS: All tested LLMs show promise for patient education regarding ACL injuries, but notable performance differences exist. Model choice is therefore critical. While all responses were evaluated by clinical experts, the lack of guideline-based validation highlights the need for further studies assessing both accuracy and patient comprehension.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Anterior Cruciate Ligament InjuriesLanguagePatient Education as TopicHumansSurveys and QuestionnairesReproducibility of ResultsLarge Language Models

Stichwörter der Autor:innen

ACL injuryArtificial intelligenceLarge language modelsOrthopedicsPatient education

Autor:innen und Institutionen

  • University of Leipzig, Department of Orthopedics, Trauma and Plastic Surgery, Leipzig, Germany. Electronic address: salim.youssef@medizin.uni-leipzig.de.
  • University of Leipzig, Department of Orthopedics, Trauma and Plastic Surgery, Leipzig, Germany.
  • Helios Klinik Leisnig, Department of Orthopedics and Trauma Surgery, Leisnig, Germany.
  • BG Klinkum Unfallkrankenhaus Berlin, Department of Trauma Surgery and Orthopedics, Berlin, Germany.

Zitieren

APA

Youssef, S., Brehmer, A., Melcher, P., Henkelmann, R., Hepp, P., Theopold, J., Elze, M., Youssef, Y., & Osterhoff, G. (2026). How accurately do large language models answer patient questions on anterior cruciate ligament tears? A comparative study. The Knee. https://doi.org/10.1016/j.knee.2025.11.002

Vancouver

Youssef S, Brehmer A, Melcher P, Henkelmann R, Hepp P, Theopold J, et al. How accurately do large language models answer patient questions on anterior cruciate ligament tears? A comparative study. The Knee. 2026. doi:10.1016/j.knee.2025.11.002

BibTeX
@article{Youssef2026,
  author = {Youssef, Salim and Brehmer, Anton and Melcher, Peter and Henkelmann, Ralf and Hepp, Pierre and Theopold, Jan and Elze, Maria and Youssef, Yasmin and Osterhoff, Georg},
  title = {How accurately do large language models answer patient questions on anterior cruciate ligament tears? A comparative study},
  journal = {The Knee},
  year = {2026},
  doi = {10.1016/j.knee.2025.11.002}
}
Kostenlos starten →