Systematic Review on Large Language Models in Orthopaedic Surgery
Journal of Clinical Medicine · 20.08.2025 · 14 (16) · 5876
DOI: 10.3390/jcm14165876 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift Journal of Clinical Medicine, online erschienen am 20.08.2025. Verfasst von Kevin Mo und 8 weiteren Autor:innen. Beteiligte Einrichtung: Orthopaedic Surgery, Valley Hospital Medical Center, 620 Shadow Ln, Las Vegas, NV 89106, USA. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background/Objectives: Since ChatGPT was released in 2022, many Large Language Models (LLM) have been developed, showing potential to expand the field of orthopaedic surgery. This is the first systematic review looking at the current state of research of LLMs in orthopaedic surgery. The aim of this study is to identify which LLMs are researched, assess their functionalities, and evaluate their quality of results.
Methods
The systematic review was conducted using PubMed, Embase, and Cochrane Library databases in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
Results
A total of 60 studies were included in the final review, all of which included ChatGPT versions 3.0 or 4.0. There were five studies that included Bard and one article each for Perplexity AI and Bing. Most studies assessed orthopaedic assessment questions (23 studies) and their ability to correctly answer free ended questions (31 studies). Outcome measures used to assess the accuracy of LLMs in most of the included studies were the percentage of correct answers on multiple-choice questions or expert-graded consensus to open-ended responses. The accuracy of ChatGPT 4.0 in orthopaedic assessment questions ranged from 47.2 to 73.6% without images, and 35.7–65.85% with images. The accuracy of ChatGPT 3.5 was 29.4–55.8% without images and 22.4–46.34% with images. The accuracy of Bard ranged from 49.8 to 58%. Orthopaedic residents consistently scored better than LLMs in the range of 74.2–75.3%.
Conclusions
ChatGPT 4 showed significant improvement over ChatGPT 3.5 in answering orthopaedic assessment questions. When comparing performances of orthopaedic residents to LLMs, orthopaedic residents scored higher overall. There remains significant opportunity for development of LLM performance on orthopaedic assessments as well as image-based analysis and clinical documentation.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Orthopaedic Surgery, Valley Hospital Medical Center, 620 Shadow Ln, Las Vegas, NV 89106, USA.
- Touro University Nevada College of Osteopathic Medicine, 874 American Pacific Dr, Henderson, NV 89104, USA.
- Desert Orthopaedic Center, 2800 E. Desert Inn Rd, Las Vegas, NV 89121, USA.
ORCID-Profile: Evan Dunn, Gio Girgis, John Walsh
Zitieren
APA
Mo, K., Lin, R., Dunn, E., Girgis, G., Fang, W., Walsh, J., Banyai-Flores, N., Watson, T., & Lee, D. (2025). Systematic Review on Large Language Models in Orthopaedic Surgery. Journal of Clinical Medicine, 14(16), 5876. https://doi.org/10.3390/jcm14165876
Vancouver
Mo K, Lin R, Dunn E, Girgis G, Fang W, Walsh J, et al. Systematic Review on Large Language Models in Orthopaedic Surgery. Journal of Clinical Medicine. 2025;14(16):5876. doi:10.3390/jcm14165876
BibTeX
@article{Mo2025,
author = {Mo, Kevin and Lin, Rowen and Dunn, Evan and Girgis, Gio and Fang, William and Walsh, John and Banyai-Flores, Nicole and Watson, Troy and Lee, Daniel},
title = {Systematic Review on Large Language Models in Orthopaedic Surgery},
journal = {Journal of Clinical Medicine},
year = {2025},
volume = {14},
number = {16},
pages = {5876},
doi = {10.3390/jcm14165876}
}
EUVIMED