EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitSystematische ÜbersichtOpen Access · CC BY 4.0

Systematic Review on Large Language Models in Orthopaedic Surgery

Kevin Mo, Rowen Lin, Evan Dunn iD, Gio Girgis iD, William Fang, John Walsh iD, Nicole Banyai-Flores, Troy Watson, Daniel Lee

Journal of Clinical Medicine · 20.08.2025 · 14 (16) · 5876

DOI: 10.3390/jcm14165876 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift Journal of Clinical Medicine, online erschienen am 20.08.2025. Verfasst von Kevin Mo und 8 weiteren Autor:innen. Beteiligte Einrichtung: Orthopaedic Surgery, Valley Hospital Medical Center, 620 Shadow Ln, Las Vegas, NV 89106, USA. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Since ChatGPT was released in 2022, many Large Language Models (LLM) have been developed, showing potential to expand the field of orthopaedic surgery. This is the first systematic review looking at the current state of research of LLMs in orthopaedic surgery. The aim of this study is to identify which LLMs are researched, assess their functionalities, and evaluate their quality of results.

Methods

The systematic review was conducted using PubMed, Embase, and Cochrane Library databases in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Results

A total of 60 studies were included in the final review, all of which included ChatGPT versions 3.0 or 4.0. There were five studies that included Bard and one article each for Perplexity AI and Bing. Most studies assessed orthopaedic assessment questions (23 studies) and their ability to correctly answer free ended questions (31 studies). Outcome measures used to assess the accuracy of LLMs in most of the included studies were the percentage of correct answers on multiple-choice questions or expert-graded consensus to open-ended responses. The accuracy of ChatGPT 4.0 in orthopaedic assessment questions ranged from 47.2 to 73.6% without images, and 35.7–65.85% with images. The accuracy of ChatGPT 3.5 was 29.4–55.8% without images and 22.4–46.34% with images. The accuracy of Bard ranged from 49.8 to 58%. Orthopaedic residents consistently scored better than LLMs in the range of 74.2–75.3%.

Conclusions

ChatGPT 4 showed significant improvement over ChatGPT 3.5 in answering orthopaedic assessment questions. When comparing performances of orthopaedic residents to LLMs, orthopaedic residents scored higher overall. There remains significant opportunity for development of LLM performance on orthopaedic assessments as well as image-based analysis and clinical documentation.

Schlagwörter

Stichwörter der Autor:innen

ChatGPTlarge language modelsorthopaedic surgerysystematic review

Autor:innen und Institutionen

  • Orthopaedic Surgery, Valley Hospital Medical Center, 620 Shadow Ln, Las Vegas, NV 89106, USA.
  • Touro University Nevada College of Osteopathic Medicine, 874 American Pacific Dr, Henderson, NV 89104, USA.
  • Desert Orthopaedic Center, 2800 E. Desert Inn Rd, Las Vegas, NV 89121, USA.

ORCID-Profile: Evan Dunn, Gio Girgis, John Walsh

Zitieren

APA

Mo, K., Lin, R., Dunn, E., Girgis, G., Fang, W., Walsh, J., Banyai-Flores, N., Watson, T., & Lee, D. (2025). Systematic Review on Large Language Models in Orthopaedic Surgery. Journal of Clinical Medicine, 14(16), 5876. https://doi.org/10.3390/jcm14165876

Vancouver

Mo K, Lin R, Dunn E, Girgis G, Fang W, Walsh J, et al. Systematic Review on Large Language Models in Orthopaedic Surgery. Journal of Clinical Medicine. 2025;14(16):5876. doi:10.3390/jcm14165876

BibTeX
@article{Mo2025,
  author = {Mo, Kevin and Lin, Rowen and Dunn, Evan and Girgis, Gio and Fang, William and Walsh, John and Banyai-Flores, Nicole and Watson, Troy and Lee, Daniel},
  title = {Systematic Review on Large Language Models in Orthopaedic Surgery},
  journal = {Journal of Clinical Medicine},
  year = {2025},
  volume = {14},
  number = {16},
  pages = {5876},
  doi = {10.3390/jcm14165876}
}
Kostenlos starten →