EUVIMEDEUVIMED entdecken
KohortenstudieOpen Access

Evaluation of Machine Learning Model Performance in Diabetic Foot Ulcer: Retrospective Cohort Study

Bewertung der Leistung eines Machine-Learning-Modells bei diabetischem Fußulkus: retrospektive Kohortenstudie

Veerle Y van Velze iD, Hendrico L Burger iD, Tim J van der Steenhoven iD, Hani Al-Ers iD, Lauren N Goncalves iD, Daniël Eefting iD, Willem-Jan J de Jong iD, Harm J Smeets iD, Janna C Specken Welleweerd iD, Joost R van der Vorst iD, Sandy Uchtmann iD, Robert Rissmann iD und 3 weitere

JMIR Medical Informatics · 10.10.2025 · 13 · e71994

DOI: 10.2196/71994 ↗

Auf einen Blick

In 700 Patient:innen mit diabetischem Fußulkus schnitt ein SVM-Modell bei der Vorhersage der vollständigen Wundheilung am besten ab. Auch RF und XGBoost zeigten gute Leistung und Kalibrierung.

Fragestellung

Wie gut sagen verschiedene Machine-Learning-Modelle die vollständige Wundheilung bei diabetischem Fußulkus vorher?

Methode

Retrospektive Kohortenstudie mit Vergleich von 7 ML-Modellen auf einem DFU-Datensatz; Training, Kreuzvalidierung und Optimierung per Random Sampling; Bewertung mit Kalibrierungskurven, Brier Score und DCA.

Ergebnisse

Explorativer Datensatz mit 700 Patient:innen und 199 Variablen. SVM: Accuracy 0,853 (95 %-KI 0,810–0,896), AUC 0,922 (95 %-KI 0,889–0,955). RF: Accuracy 0,838; AUC 0,917. XGBoost: Accuracy 0,815; AUC 0,889.

Schlussfolgerung der Autor:innen

Nach Ansicht der Autor:innen sind fehlende Werte, Merkmalsauswahl und Klassenungleichgewicht zentrale Schritte bei ML-Anwendungen. In diesem DFU-Datensatz erzielte SVM die beste Leistung, gefolgt von RF und XGBoost.

Kernaussagen

  • SVM erreichte die höchste Vorhersageleistung für vollständige Wundheilung.
  • RF und XGBoost lagen hinter SVM, zeigten aber ebenfalls gute Werte.
  • Kalibrierung und klinischer Nutzen wurden mit Brier Score und DCA geprüft.

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Kohortenstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 10.10.2025. Verfasst von Veerle Y van Velze und 14 weiteren Autor:innen. Beteiligte Einrichtung: Department of Surgery, Haaglanden Medical Center, The Hague, The Netherlands. Hauptthemen laut MEDLINE: Diabetic Foot, Machine Learning. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Machine learning (ML) has shown great potential in recognizing complex disease patterns and supporting clinical decision-making. Diabetic foot ulcers (DFUs) represent a significant multifactorial medical problem with high incidence and severe outcomes, providing an ideal example for a comprehensive framework that encompasses all essential steps for implementing ML in a clinically relevant fashion.

Objective

This paper aims to provide a framework for the proper use of ML algorithms to predict clinical outcomes of multifactorial diseases and their treatments.

Methods

The comparison of ML models was performed on a DFU dataset. The selection of patient characteristics associated with wound healing was based on outcomes of statistical tests, that is, ANOVA and chi-square test, and validated on expert recommendations. Imputation and balancing of patient records were performed with MIDAS (Multiple Imputation with Denoising Autoencoders) Touch and adaptive synthetic sampling, respectively. Logistic regression, support vector machine (SVM), k-nearest neighbors, random forest (RF), extreme gradient boosting (XGBoost), Bayesian additive regression trees, and artificial neural network were trained, cross-validated, and optimized using random sampling on the patient dataset. To evaluate model calibration and clinical utility, calibration curves, Brier scores, and decision curve analysis (DCA) were performed.

Results

The exploratory dataset consisted of 700 patient records with 199 variables. After dataset cleaning, the variables used for model training included age, smoking status, toe systolic pressure, blood pressure, oxygen saturation, hemoglobin, hemoglobin A1c, estimated glomerular filtration rate, wound location, diabetes type, Texas wound classification, neuropathy, and wound area measurement. The SVM obtained a stable accuracy of 0.853 (95% CI 0.810-0.896) with an area under the receiver operating characteristic curve of 0.922 (95% CI 0.889-0.955). The RF and XGBoost acquired an accuracy of 0.838 (95% CI 0.793-0.883) and 0.815 (95% CI 0.768-0.862), respectively, with areas under the receiver operating characteristic curve of 0.917 (95% CI 0.883-0.951) for RF and 0.889 (95% CI 0.849-0.929) for XGBoost. SVM, RF, and XGBoost were well-calibrated, with average Brier scores around 0.127 (SD 0.013). DCA showed that the SVM provided the highest net clinical benefit across relevant risk thresholds.

Conclusions

Handling missing values, feature selection, and addressing class imbalance are critical components of the key steps in developing ML applications for clinical research. Seven models were selected for comparing their predictive power regarding complete wound healing, and each model representing a different branch in ML. In this initial DFU dataset used as an example, the SVM achieved the best performance in predicting clinical outcomes, followed by RF and XGBoost. The model’s calibration and clinical utility were determined through calibration curves, Brier scores, and DCA, demonstrating its potential relevance in clinical decision-making.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Diabetic FootMachine LearningHumansRetrospective StudiesMaleFemaleMiddle AgedAgedAlgorithmsBayes TheoremWound Healing

Stichwörter der Autor:innen

Bayesian additive regression treesartificial neural networkcomplete wound healingdiabetic foot ulcerextreme gradient boostingk-nearest neighborlogistic regressionmachine learningrandom forestsupport vector machine

Autor:innen und Institutionen

  • Department of Surgery, Haaglanden Medical Center, The Hague, The Netherlands.
  • Centre for Human Drug Research, Leiden, The Netherlands.
  • The Hague University of Applied Sciences, The Hague, Zoetermeer, The Netherlands.
  • University Vascular Centre West, Leiden, The Hague, Delft, The Netherlands.
  • Department of Vascular Surgery, Leiden University Medical Center, Leiden, The Netherlands.
  • Department of Dermatology, Leiden University Medical Center, Leiden, The Netherlands.

ORCID-Profile: Veerle Y van Velze, Hendrico L Burger, Tim J van der Steenhoven, Hani Al-Ers, Lauren N Goncalves, Daniël Eefting, Willem-Jan J de Jong, Harm J Smeets, Janna C Specken Welleweerd, Joost R van der Vorst, Sandy Uchtmann, Robert Rissmann, Jaap F Hamming, Lampros Stergioulas, Koen Ea van der Bogt

Zitieren

APA

van Velze, V. Y., Burger, H. L., van der Steenhoven, T. J., Al-Ers, H., Goncalves, L. N., Eefting, D., de Jong, W. J., Smeets, H. J., Welleweerd, J. C. S., van der Vorst, J. R., Uchtmann, S., Rissmann, R., Hamming, J. F., Stergioulas, L., & van der Bogt, K. E. (2025). Evaluation of Machine Learning Model Performance in Diabetic Foot Ulcer: Retrospective Cohort Study. JMIR Medical Informatics, 13, e71994. https://doi.org/10.2196/71994

Vancouver

van Velze VY, Burger HL, van der Steenhoven TJ, Al-Ers H, Goncalves LN, Eefting D, et al. Evaluation of Machine Learning Model Performance in Diabetic Foot Ulcer: Retrospective Cohort Study. JMIR Medical Informatics. 2025;13:e71994. doi:10.2196/71994

BibTeX
@article{vanVelze2025,
  author = {van Velze, Veerle Y and Burger, Hendrico L and van der Steenhoven, Tim J and Al-Ers, Hani and Goncalves, Lauren N and Eefting, Daniël and de Jong, Willem-Jan J and Smeets, Harm J and Welleweerd, Janna C Specken and van der Vorst, Joost R and Uchtmann, Sandy and Rissmann, Robert and Hamming, Jaap F and Stergioulas, Lampros and van der Bogt, Koen Ea},
  title = {Evaluation of Machine Learning Model Performance in Diabetic Foot Ulcer: Retrospective Cohort Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  volume = {13},
  pages = {e71994},
  doi = {10.2196/71994}
}
Kostenlos starten →