EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection

Woojin Shin iD, Jaeyoung Hong, Sunwoo Lee iD, Seongchan Park, Hyoungtae Kim iD, Suenghwan Jo iD

Journal of Clinical Medicine · 08.04.2026 · 15 (8) · 2830

DOI: 10.3390/jcm15082830 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Clinical Medicine, online erschienen am 08.04.2026. Verfasst von Woojin Shin und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Medicine, Chosun University, Gwangju 61452, Republic of Korea. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Chosun University Hospital.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Periprosthetic joint infection (PJI) remains a devastating complication following arthroplasty. Systematic reviews of PJI provide essential evidence to inform clinical practice; however, the screening process remains labor-intensive. Recent advancements in large language models (LLMs) offer potential for automating literature screening, though evaluation of current generation models is needed.

Methods

This validation study evaluated GPT-5, GPT-5 Pro, and Gemini 2.5 Pro in replicating the title/abstract and full-text screening stages of a published systematic review on intraosseous versus intravenous antibiotic prophylaxis in total joint arthroplasty. Title/abstract screening was performed on 165 articles, followed by a full-text eligibility assessment of 26 articles. Accuracy, sensitivity, specificity, and Cohen’s kappa (κ) were calculated against human screening decisions as the gold standard.

Results

In title/abstract screening, GPT-5 Pro achieved the highest accuracy (92.1–92.7%) and specificity (98.6–99.3%), while GPT-5 demonstrated the highest sensitivity (84.6–96.1%). In full-text screening, Gemini 2.5 Pro showed the most consistent performance across repeated evaluations (κ = 0.839 in both trials), whereas GPT-5 Pro exhibited marked intra-model variability (κ = 0.399 to 0.920).

Conclusions

Current-generation LLMs achieve near-human accuracy in systematic review screening for PJI research, though substantial intra-model variability underscores the continued need for human oversight in systematic review workflows.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligence assisted screeninglarge language modelperiprosthetic joint infectionsystematic reviewtotal joint arthroplasty

Autor:innen und Institutionen

  • College of Medicine, Chosun University, Gwangju 61452, Republic of Korea.
  • Department of Orthopedic Surgery, Chosun University Hospital, Gwangju 61453, Republic of Korea.

ORCID-Profile: Woojin Shin, Sunwoo Lee, Hyoungtae Kim, Suenghwan Jo

Förderung

  • National Research Foundation of Korea
  • Chosun University Hospital

Zitieren

APA

Shin, W., Hong, J., Lee, S., Park, S., Kim, H., & Jo, S. (2026). Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection. Journal of Clinical Medicine, 15(8), 2830. https://doi.org/10.3390/jcm15082830

Vancouver

Shin W, Hong J, Lee S, Park S, Kim H, Jo S. Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection. Journal of Clinical Medicine. 2026;15(8):2830. doi:10.3390/jcm15082830

BibTeX
@article{Shin2026,
  author = {Shin, Woojin and Hong, Jaeyoung and Lee, Sunwoo and Park, Seongchan and Kim, Hyoungtae and Jo, Suenghwan},
  title = {Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection},
  journal = {Journal of Clinical Medicine},
  year = {2026},
  volume = {15},
  number = {8},
  pages = {2830},
  doi = {10.3390/jcm15082830}
}
Kostenlos starten →