Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti250968 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Peijin Han und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of Michigan, Ann Arbor, MI, USA. Hauptthemen laut MEDLINE: Algorithms, Dementia. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
The performance of prediction algorithms is typically measured using four metrics: sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). These metrics are usually calculated on samples drawn from patient populations. However, the performance metrics computed over a deliberately biased sample would not directly extend to its source population. Further, it is often necessary to infer the metric values for a population different from where the sample was drawn. In this paper, we illustrate methods to solve both challenges. Specifically, given the underlying patient distribution, we show corrections to the formula for these metrics based on two common inverse probability weighting methods: standard cell weighting and logistic regression weighting. We conduct simulation experiments to identify patients living with dementia and compare these methods in performance corrections with different sample sizes for different prevalence settings. We empirically show that weighting methods can correct the estimated values for algorithms’ performance. Standard cell weighting is preferred over logistic regression weighting when the sample size is small and only the strata information is available in the populations of interest.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University of Michigan, Ann Arbor, MI, USA.
Zitieren
APA
Han, P., Zhang, G., & Vydiswaran, V. G. V. (2025). Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250968
Vancouver
Han P, Zhang G, Vydiswaran VGV. Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250968
BibTeX
@article{Han2025,
author = {Han, Peijin and Zhang, Guanghao and Vydiswaran, V G Vinod},
title = {Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti250968}
}
EUVIMED