High identification and positive‐negative discrimination but limited detailed grading accuracy of ChatGPT‐4o in knee osteoarthritis radiographs
Knee Surgery, Sports Traumatology, Arthroscopy · 2025-05 · 33 (5) · 1911-1919
DOI: 10.1002/ksa.12639 ↗Kurz gefasst
Fachartikel in der Zeitschrift Knee Surgery, Sports Traumatology, Arthroscopy, online erschienen am 07.03.2025. Verfasst von Jiesheng Zhu und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Orthopedics, The Second Affiliated Hospital of Wenzhou Medical University, Yuying Children's Hospital, Wenzhou, China. Hauptthemen laut MEDLINE: Osteoarthritis, Knee, Knee Joint. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 32 Literaturangaben.
Abstract (Originalsprache)
Purpose
To explore the potential of ChatGPT‐4o in analysing radiographic images of knee osteoarthritis (OA) and to assess its grading accuracy, feature identification and reliability, thereby helping surgeons to improve diagnostic accuracy and efficiency.
Methods
A total of 117 anterior‒posterior knee radiographs from patients (23.1% men, 76.9% women, mean age 69.7 ± 7.99 years) were analysed. Two senior orthopaedic surgeons and ChatGPT‐4o independently graded images with the Kellgren–Lawrence (K–L), Ahlbäck and International Knee Documentation Committee (IKDC) systems. A consensus reference standard was established by a third radiologist. ChatGPT‐4o's performance metrics (accuracy, precision, recall and F1 score) were calculated, and its reliability was assessed via two evaluations separated by a 2‐week interval, with intraclass correlation coefficients (ICCs) determined.
Results
ChatGPT‐4o achieved a 100% identification rate for knee radiographs and demonstrated strong binary classification performance (precision: 0.95, recall: 0.83, F score: 0.88). However, its detailed grading accuracy (35%) was substantially lower than that of surgeons (89.6%). Severe underestimation of OA severity occurred in 49.3% of the cases. Interrater reliability for surgeons was excellent (ICC: 0.78–0.91), whereas ChatGPT‐4o showed poor initial consistency (ICC: 0.16–0.28), improving marginally in the second evaluation (ICC: 0.22–0.39).
Conclusion
ChatGPT‐4o has the potential to rapidly identify and binary classify knee OA on radiographs. However, its detailed grading accuracy remains suboptimal, with a notable tendency to underestimate severe cases. This limits its current clinical utility for precise staging. Future research should focus on optimising its grading performance and improving accuracy to enhance diagnostic reliability. level of Evidence Level III, retrospective comparative study.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Orthopedics, The Second Affiliated Hospital of Wenzhou Medical University, Yuying Children's Hospital, Wenzhou, China.
- Department of Radiology, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, China.
Zitieren
APA
Zhu, J., Jiang, Y., Chen, D., Lu, Y., Huang, Y., Lin, Y., & Fan, P. (2025). High identification and positive‐negative discrimination but limited detailed grading accuracy of ChatGPT‐4o in knee osteoarthritis radiographs. Knee Surgery, Sports Traumatology, Arthroscopy, 33(5), 1911-1919. https://doi.org/10.1002/ksa.12639
Vancouver
Zhu J, Jiang Y, Chen D, Lu Y, Huang Y, Lin Y, et al. High identification and positive‐negative discrimination but limited detailed grading accuracy of ChatGPT‐4o in knee osteoarthritis radiographs. Knee Surgery, Sports Traumatology, Arthroscopy. 2025;33(5):1911-1919. doi:10.1002/ksa.12639
BibTeX
@article{Zhu2025,
author = {Zhu, Jiesheng and Jiang, Yilun and Chen, Daosen and Lu, Yi and Huang, Yijiang and Lin, Yimu and Fan, Pei},
title = {High identification and positive‐negative discrimination but limited detailed grading accuracy of ChatGPT‐4o in knee osteoarthritis radiographs},
journal = {Knee Surgery, Sports Traumatology, Arthroscopy},
year = {2025},
volume = {33},
number = {5},
pages = {1911--1919},
doi = {10.1002/ksa.12639}
}
EUVIMED