Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models
The Journal of Applied Laboratory Medicine · 2025
DOI: 10.1093/jalm/jfae115 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Journal of Applied Laboratory Medicine, 2025. Verfasst von Remy J H Martens und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Chemistry and Hematology, Zuyderland Medical Center, Sittard-Geleen, the Netherlands. Hauptthemen laut MEDLINE: Machine Learning, Renal Insufficiency, Chronic. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 35 Literaturangaben.
Abstract (Originalsprache)
Background
Interest in prediction models, including machine learning (ML) models, based on laboratory data has increased tremendously. Uncertainty in laboratory measurements and predictions based on such data are inherently intertwined. This study developed a framework for assessing the impact of biological and analytical variation on the prediction uncertainty of categorical prediction models.
Methods
Practical application was demonstrated for the prediction of renal function loss (Chronic Kidney Disease Epidemiology Collaboration [CKD-EPI] equation) and 31-day mortality (advanced ML model) in 6360 emergency department patients. Model outcome was calculated in 100 000 simulations of variation in laboratory parameters. Subsequently, the percentage of discordant predictions was calculated with the original prediction as reference. Simulations were repeated assuming increasing levels of analytical variation.
Results
For the ML model, area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity were 0.90, 0.44, and 0.96, respectively. At base analytical variation, the median [2.5th–97.5th percentiles] percentage of discordant predictions was 0% [0%–28.8%]. In addition, 7.2% of patients had >5% discordant predictions. At 6× base analytical variation, the median [2.5th–97.5th percentiles] percentage of discordant predictions was 0% [0%–38.8%]. In addition, 11.7% of patients had >5% discordant predictions. However, the impact of analytical variation was limited compared with biological variation. AUROC, sensitivity, and specificity were not affected by variation in laboratory parameters.
Conclusions
The impact of biological and analytical variation on the prediction uncertainty of categorical prediction models, including ML models, can be estimated by the occurrence of discordant predictions in a simulation model. Nevertheless, discordant predictions at the individual level do not necessarily affect model performance at the population level.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Clinical Chemistry and Hematology, Zuyderland Medical Center, Sittard-Geleen, the Netherlands.
- Central Diagnostic Laboratory, Maastricht University Medical Centre+, Maastricht, the Netherlands.
- Faculty of Science, Environmental Sciences, Open Universiteit, Heerlen, the Netherlands.
ORCID-Profile: Remy J H Martens, Mathie P G Leers
Zitieren
APA
Martens, R. J. H., van Doorn, W. P. T. M., Leers, M. P. G., Meex, S. J. R., & Helmich, F. (2025). Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models. The Journal of Applied Laboratory Medicine. https://doi.org/10.1093/jalm/jfae115
Vancouver
Martens RJH, van Doorn WPTM, Leers MPG, Meex SJR, Helmich F. Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models. The Journal of Applied Laboratory Medicine. 2025. doi:10.1093/jalm/jfae115
BibTeX
@article{Martens2025,
author = {Martens, Remy J H and van Doorn, William P T M and Leers, Mathie P G and Meex, Steven J R and Helmich, Floris},
title = {Unraveling Uncertainty: The Impact of Biological and Analytical Variation on the Prediction Uncertainty of Categorical Prediction Models},
journal = {The Journal of Applied Laboratory Medicine},
year = {2025},
doi = {10.1093/jalm/jfae115}
}
EUVIMED