EUVIMEDObjevte EUVIMED
Beobachtungsstudie

Performance of a Large Language Model for Predicting Clinical and Radiographic Outcomes in Knee Osteoarthritis: Single-Center Prospective Observational Study

Leistung eines Large Language Model zur Vorhersage klinischer und radiografischer Outcomes bei Kniearthrose: monozentrische prospektive Beobachtungsstudie

Zhiqiang Li iD, Yuhan Zhang iD, Mingwei Li iD

JMIR Medical Informatics · 2026-11-12 · 14 · e107092

DOI: 10.2196/107092 ↗

Auf einen Blick

Eine monozentrische Studie prüfte DeepSeek bei Kniearthrose. Das Modell war für die Basiserfassung gut, die 6-Monats-Prognosen und die patientenbezogene Stabilität waren jedoch begrenzt.

Fragestellung

Wie gut bewertet DeepSeek den Ausgangsbefund bei Kniearthrose, wie zuverlässig sagt es 6-Monats-Outcomes voraus und wie reproduzierbar sind die Ergebnisse?

Methode

Monozentrische, prospektive Beobachtungskohorte mit verblindeter retrospektiver Bewertung des LLM; 128 Patient:innen, 111 in der Primäranalyse. DeepSeek-V4-Flash wurde dreimal mit Ärzt:innen und einem Persistenzmodell verglichen.

Ergebnisse

Basale Diagnoseübereinstimmung mit der Ärztin/dem Arzt 105/111 (94,6 %). 6-Monats-Vorhersagen waren schlechter als die der Ärztin/des Arztes (WOMAC MAE 5,92-6,58 vs. 3,86; KL 66,7 %-68,5 % vs. 86,5 %) und nicht besser als das Persistenzmodell (MAE 5,43).

Schlussfolgerung der Autor:innen

Die geprüfte DeepSeek-Konfiguration kann die strukturierte Basiserfassung bei KOA unterstützen. Die prognostische Genauigkeit über 6 Monate und die Stabilität auf Patientenebene bleiben begrenzt; ein Zusatznutzen gegenüber dem Persistenzmodell ist nötig.

Kernaussagen

  • Basale Diagnoseübereinstimmung: 105/111 (94,6 %)
  • 6-Monats-Prognosen unterlegen dem ärztlichen Vergleich
  • Hohe ICC, aber geringe exakte Übereinstimmung zwischen Runs

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Beobachtungsstudie in JMIR Medical Informatics, published online on 2026-11-12. Written by Zhiqiang Li and 2 co-authors. Institution: Department of Rheumatology and Immunology, Fuxing Hospital, Capital Medical University, Beijing, China. Major MEDLINE topics: Osteoarthritis, Knee, Large Language Models, Radiography. 27 references.

Auf einen BlickAbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Large language models (LLMs) can extract information from unstructured clinical text, but it remains unclear whether their ability to assess current status translates into reliable predictions of longitudinal outcomes.

Objective

This study aimed to evaluate DeepSeek’s ability to assess baseline clinical and radiographic status in knee osteoarthritis (KOA), its performance in predicting 6-month outcomes, and its interrun reproducibility.

Methods

This study used a single-center, prospective observational cohort design to conduct a blinded, retrospective evaluation of the LLM. A total of 128 patients with primary KOA were enrolled, of whom 111 (86.7%) completed the 6-month follow-up and were included in the primary analysis. DeepSeek-V4-Flash and a rheumatologist independently assessed KOA diagnoses, baseline Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scores, and Kellgren-Lawrence (KL) grading based on the same standardized interview records and imaging reports and predicted 6-month outcomes. The LLM was run independently 3 times. The actual outcomes determined by another senior physician served as the reference, and the results were compared with the physician’s predictions and the persistence model.

Results

The LLM showed high baseline diagnostic agreement with the physician (105/111, 94.6%) and good agreement on the WOMAC and KL, but 6-month predictions (WOMAC mean absolute error=5.92-6.58; KL exact agreement=74/111, 66.7%-76/111, 68.5%) were worse than those of the physician comparator (mean absolute error=3.86; 96/111, 86.5%) and not superior to those of the persistence model (mean absolute error=5.43). Despite high interrun intraclass correlation coefficients (0.982 and 0.965), exact agreement between runs was poor (35/111, 31.5% and 7/111, 6.3%).

Conclusions

The evaluated DeepSeek configuration may support structured baseline KOA assessment. However, its 6-month predictive accuracy and patient-level stability remain limited. Prognostic applications require evidence of added value over a persistence model.

Subjects

MeSH terms (MEDLINE)

Osteoarthritis, KneeLarge Language ModelsRadiographyHumansProspective StudiesFemaleMaleMiddle AgedAgedReproducibility of Results

Author keywords

DeepSeekKellgren-Lawrence classificationWOMACWestern Ontario and McMaster Universities Osteoarthritis Indexknee osteoarthritislarge language modeloutcome prediction

Authors and institutions

  • Department of Rheumatology and Immunology, Fuxing Hospital, Capital Medical University, Beijing, China.

ORCID profiles: Zhiqiang Li, Yuhan Zhang, Mingwei Li

Cite

APA

Li, Z., Zhang, Y., & Li, M. (2026). Performance of a Large Language Model for Predicting Clinical and Radiographic Outcomes in Knee Osteoarthritis: Single-Center Prospective Observational Study. JMIR Medical Informatics, 14, e107092. https://doi.org/10.2196/107092

Vancouver

Li Z, Zhang Y, Li M. Performance of a Large Language Model for Predicting Clinical and Radiographic Outcomes in Knee Osteoarthritis: Single-Center Prospective Observational Study. JMIR Medical Informatics. 2026;14:e107092. doi:10.2196/107092

BibTeX
@article{Li2026,
  author = {Li, Zhiqiang and Zhang, Yuhan and Li, Mingwei},
  title = {Performance of a Large Language Model for Predicting Clinical and Radiographic Outcomes in Knee Osteoarthritis: Single-Center Prospective Observational Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e107092},
  doi = {10.2196/107092}
}
Start for free →