EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations

Peijin Han, Guanghao Zhang, V G Vinod Vydiswaran

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti250968 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Peijin Han und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of Michigan, Ann Arbor, MI, USA. Hauptthemen laut MEDLINE: Algorithms, Dementia. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The performance of prediction algorithms is typically measured using four metrics: sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). These metrics are usually calculated on samples drawn from patient populations. However, the performance metrics computed over a deliberately biased sample would not directly extend to its source population. Further, it is often necessary to infer the metric values for a population different from where the sample was drawn. In this paper, we illustrate methods to solve both challenges. Specifically, given the underlying patient distribution, we show corrections to the formula for these metrics based on two common inverse probability weighting methods: standard cell weighting and logistic regression weighting. We conduct simulation experiments to identify patients living with dementia and compare these methods in performance corrections with different sample sizes for different prevalence settings. We empirically show that weighting methods can correct the estimated values for algorithms’ performance. Standard cell weighting is preferred over logistic regression weighting when the sample size is small and only the strata information is available in the populations of interest.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsDementiaHumansLogistic ModelsBiasSample SizeSensitivity and SpecificityComputer Simulation

Stichwörter der Autor:innen

performance metrics correctionsample and populationsweighting performance metrics

Autor:innen und Institutionen

  • University of Michigan, Ann Arbor, MI, USA.

Zitieren

APA

Han, P., Zhang, G., & Vydiswaran, V. G. V. (2025). Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250968

Vancouver

Han P, Zhang G, Vydiswaran VGV. Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250968

BibTeX
@article{Han2025,
  author = {Han, Peijin and Zhang, Guanghao and Vydiswaran, V G Vinod},
  title = {Correcting Performance Metrics Bias During Generalization from Biased Samples to Populations},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti250968}
}
Kostenlos starten →