Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection
Journal of Clinical Medicine · 08.04.2026 · 15 (8) · 2830
DOI: 10.3390/jcm15082830 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Clinical Medicine, online erschienen am 08.04.2026. Verfasst von Woojin Shin und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Medicine, Chosun University, Gwangju 61452, Republic of Korea. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Chosun University Hospital.
Abstract (Originalsprache)
Background
Periprosthetic joint infection (PJI) remains a devastating complication following arthroplasty. Systematic reviews of PJI provide essential evidence to inform clinical practice; however, the screening process remains labor-intensive. Recent advancements in large language models (LLMs) offer potential for automating literature screening, though evaluation of current generation models is needed.
Methods
This validation study evaluated GPT-5, GPT-5 Pro, and Gemini 2.5 Pro in replicating the title/abstract and full-text screening stages of a published systematic review on intraosseous versus intravenous antibiotic prophylaxis in total joint arthroplasty. Title/abstract screening was performed on 165 articles, followed by a full-text eligibility assessment of 26 articles. Accuracy, sensitivity, specificity, and Cohen’s kappa (κ) were calculated against human screening decisions as the gold standard.
Results
In title/abstract screening, GPT-5 Pro achieved the highest accuracy (92.1–92.7%) and specificity (98.6–99.3%), while GPT-5 demonstrated the highest sensitivity (84.6–96.1%). In full-text screening, Gemini 2.5 Pro showed the most consistent performance across repeated evaluations (κ = 0.839 in both trials), whereas GPT-5 Pro exhibited marked intra-model variability (κ = 0.399 to 0.920).
Conclusions
Current-generation LLMs achieve near-human accuracy in systematic review screening for PJI research, though substantial intra-model variability underscores the continued need for human oversight in systematic review workflows.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Medicine, Chosun University, Gwangju 61452, Republic of Korea.
- Department of Orthopedic Surgery, Chosun University Hospital, Gwangju 61453, Republic of Korea.
ORCID-Profile: Woojin Shin, Sunwoo Lee, Hyoungtae Kim, Suenghwan Jo
Förderung
- National Research Foundation of Korea
- Chosun University Hospital
Zitieren
APA
Shin, W., Hong, J., Lee, S., Park, S., Kim, H., & Jo, S. (2026). Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection. Journal of Clinical Medicine, 15(8), 2830. https://doi.org/10.3390/jcm15082830
Vancouver
Shin W, Hong J, Lee S, Park S, Kim H, Jo S. Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection. Journal of Clinical Medicine. 2026;15(8):2830. doi:10.3390/jcm15082830
BibTeX
@article{Shin2026,
author = {Shin, Woojin and Hong, Jaeyoung and Lee, Sunwoo and Park, Seongchan and Kim, Hyoungtae and Jo, Suenghwan},
title = {Efficacy of Large Language Models for Screening of Systematic Reviews on Periprosthetic Joint Infection},
journal = {Journal of Clinical Medicine},
year = {2026},
volume = {15},
number = {8},
pages = {2830},
doi = {10.3390/jcm15082830}
}
EUVIMED