EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comparative evaluation of LLMs in orthopedic surgery

Gnaneswar Chundi, Abhiram Dawar, Syed Sarwar, Sanjiv Prasad, Michael Vosbikian, Irfan Ahmed

Journal of Orthopaedic Reports · 2026-04 · 5 (2) · 100728

DOI: 10.1016/j.jorep.2025.100728 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Orthopaedic Reports, 2026-04. Verfasst von Gnaneswar Chundi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

AIMS/OBJECTIVES This study aimed to evaluate the performance of leading large language models (LLMs) in orthopedic surgery, with a focus on diagnostic accuracy, radiographic interpretation, subspecialty-specific performance, consistency, and gender bias. Unlike previous investigations that focused solely on ChatGPT, we assessed Claude-3-Sonnet, GPT-4o, Gemini-1.5, and Meta’s LLaMA Vision-Instruct models against Orthopaedic In-Training Examination (OITE) questions. METHODS We tested each LLM on 2,906 multiple-choice questions from the OITE question bank. Questions were categorized by subspecialty and by presence of images. Models provided answer choices, confidence scores (1–4), and justifications. Each question was administered three times to evaluate consistency. Statistical analyses included Z-tests, t-tests, ANOVA, chi-square tests, and logistic regression. Rasch transformation enabled comparison to PGY-1 and PGY-5 resident performance. Gender bias was evaluated based on performance differences across gender-specific cases. RESULTS GPT-4o achieved the highest accuracy (72.8%), outperforming all other models. Performance improved with larger model sizes across all vendors. All models showed diminished accuracy on image-based questions (mean 10.9% lower, p

Zitieren

APA

Chundi, G., Dawar, A., Sarwar, S., Prasad, S., Vosbikian, M., & Ahmed, I. (2026). Comparative evaluation of LLMs in orthopedic surgery. Journal of Orthopaedic Reports, 5(2), 100728. https://doi.org/10.1016/j.jorep.2025.100728

Vancouver

Chundi G, Dawar A, Sarwar S, Prasad S, Vosbikian M, Ahmed I. Comparative evaluation of LLMs in orthopedic surgery. Journal of Orthopaedic Reports. 2026;5(2):100728. doi:10.1016/j.jorep.2025.100728

BibTeX
@article{Chundi2026,
  author = {Chundi, Gnaneswar and Dawar, Abhiram and Sarwar, Syed and Prasad, Sanjiv and Vosbikian, Michael and Ahmed, Irfan},
  title = {Comparative evaluation of LLMs in orthopedic surgery},
  journal = {Journal of Orthopaedic Reports},
  year = {2026},
  volume = {5},
  number = {2},
  pages = {100728},
  doi = {10.1016/j.jorep.2025.100728}
}
Kostenlos starten →