EUVIMEDEUVIMED entdecken
Crossref

Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models

Remy J H Martens iD, William P T M van Doorn, Mathie P G Leers iD, Steven J R Meex, Floris Helmich

The Journal of Applied Laboratory Medicine · 2025

DOI: 10.1093/jalm/jfae115 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of Applied Laboratory Medicine, 2025. Verfasst von Remy J H Martens und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Chemistry and Hematology, Zuyderland Medical Center, Sittard-Geleen, the Netherlands. Hauptthemen laut MEDLINE: Machine Learning, Renal Insufficiency, Chronic. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 35 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Interest in prediction models, including machine learning (ML) models, based on laboratory data has increased tremendously. Uncertainty in laboratory measurements and predictions based on such data are inherently intertwined. This study developed a framework for assessing the impact of biological and analytical variation on the prediction uncertainty of categorical prediction models.

Methods

Practical application was demonstrated for the prediction of renal function loss (Chronic Kidney Disease Epidemiology Collaboration [CKD-EPI] equation) and 31-day mortality (advanced ML model) in 6360 emergency department patients. Model outcome was calculated in 100 000 simulations of variation in laboratory parameters. Subsequently, the percentage of discordant predictions was calculated with the original prediction as reference. Simulations were repeated assuming increasing levels of analytical variation.

Results

For the ML model, area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity were 0.90, 0.44, and 0.96, respectively. At base analytical variation, the median [2.5th–97.5th percentiles] percentage of discordant predictions was 0% [0%–28.8%]. In addition, 7.2% of patients had >5% discordant predictions. At 6× base analytical variation, the median [2.5th–97.5th percentiles] percentage of discordant predictions was 0% [0%–38.8%]. In addition, 11.7% of patients had >5% discordant predictions. However, the impact of analytical variation was limited compared with biological variation. AUROC, sensitivity, and specificity were not affected by variation in laboratory parameters.

Conclusions

The impact of biological and analytical variation on the prediction uncertainty of categorical prediction models, including ML models, can be estimated by the occurrence of discordant predictions in a simulation model. Nevertheless, discordant predictions at the individual level do not necessarily affect model performance at the population level.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningRenal Insufficiency, ChronicHumansUncertaintyROC CurveMaleFemaleEmergency Service, HospitalModels, StatisticalArea Under CurveKidney Function TestsSensitivity and Specificity

Autor:innen und Institutionen

  • Department of Clinical Chemistry and Hematology, Zuyderland Medical Center, Sittard-Geleen, the Netherlands.
  • Central Diagnostic Laboratory, Maastricht University Medical Centre+, Maastricht, the Netherlands.
  • Faculty of Science, Environmental Sciences, Open Universiteit, Heerlen, the Netherlands.

ORCID-Profile: Remy J H Martens, Mathie P G Leers

Zitieren

APA

Martens, R. J. H., van Doorn, W. P. T. M., Leers, M. P. G., Meex, S. J. R., & Helmich, F. (2025). Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models. The Journal of Applied Laboratory Medicine. https://doi.org/10.1093/jalm/jfae115

Vancouver

Martens RJH, van Doorn WPTM, Leers MPG, Meex SJR, Helmich F. Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models. The Journal of Applied Laboratory Medicine. 2025. doi:10.1093/jalm/jfae115

BibTeX
@article{Martens2025,
  author = {Martens, Remy J H and van Doorn, William P T M and Leers, Mathie P G and Meex, Steven J R and Helmich, Floris},
  title = {Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models},
  journal = {The Journal of Applied Laboratory Medicine},
  year = {2025},
  doi = {10.1093/jalm/jfae115}
}
Kostenlos starten →