EUVIMEDEUVIMED entdecken
Open Access

Assessing Covariate Balance With Small Sample Sizes

George Hripcsak iD, Linying Zhang, Yong Chen iD, Kelly Li, Marc A Suchard iD, Patrick B Ryan, Martijn J Schuemie iD

Statistics in Medicine · 2025

DOI: 10.1002/sim.70212 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistics in Medicine, 2025. Verfasst von George Hripcsak und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA. Hauptthemen laut MEDLINE: Propensity Score. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, NLM NIH HHS, NHLBI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Propensity score adjustment addresses confounding by balancing covariates in subject treatment groups through matching, stratification, or weighting. Diagnostics test the success of adjustment. For example, if the standardized mean difference (SMD) for a relevant covariate exceeds a threshold like 0.1, the covariate is considered imbalanced and the study may be biased. Unfortunately, for studies with small or moderate numbers of subjects, the probability of identifying a study as biased because of chance imbalance can be grossly larger than a given nominal level like 0.05, yet that chance imbalance may not cause significant bias. In this paper, we illustrate that chance imbalance is operative in real‐world settings even for moderate sample sizes of 2000. We identify a previously unrecognized challenge that as meta‐analyses increase the precision of an effect estimate, the diagnostics must also undergo meta‐analysis for a corresponding increase in precision. We propose an alternative diagnostic that checks whether the SMD statistically significantly exceeds the threshold. Through simulation and real‐world data, we find that this diagnostic achieves a better trade‐off of type 1 error rate and power than standard nominal threshold tests and not testing for sample sizes from 250 to 4000 and for 20 to 100 000 covariates. We confirm that in network studies, meta‐analysis of effect estimates must be accompanied by meta‐analysis of the diagnostics or else systematic confounding may overwhelm the estimated effect. Our procedure supports the review of large numbers of covariates, enabling more rigorous diagnostics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Propensity ScoreHumansSample SizeComputer SimulationBiasMeta-Analysis as TopicData Interpretation, StatisticalModels, Statistical

Stichwörter der Autor:innen

confoundingcovariate balancemeta‐analysisobservational researchpropensity score

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA.
  • Observational Health Data Science and Informatics, New York, New York, USA.
  • VA Informatics and Computing Infrastructure, US Department of Veterans Affairs, Salt Lake City, Utah, USA.
  • Institute for Informatics, Data Science and Biostatistics, Washington University in St. Louis, St. Louis, Missouri, USA.
  • The Center for Health AI and Synthesis of Evidence (CHASE), University of Pennsylvania, Philadelphia, Pennsylvania, USA.
  • Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania, USA.

ORCID-Profile: George Hripcsak, Yong Chen, Marc A Suchard, Martijn J Schuemie

Förderung

  • NIH HHS
  • NLM NIH HHS
  • NHLBI NIH HHS
  • National Institutes of Health

Zitieren

APA

Hripcsak, G., Zhang, L., Chen, Y., Li, K., Suchard, M. A., Ryan, P. B., & Schuemie, M. J. (2025). Assessing Covariate Balance With Small Sample Sizes. Statistics in Medicine. https://doi.org/10.1002/sim.70212

Vancouver

Hripcsak G, Zhang L, Chen Y, Li K, Suchard MA, Ryan PB, et al. Assessing Covariate Balance With Small Sample Sizes. Statistics in Medicine. 2025. doi:10.1002/sim.70212

BibTeX
@article{Hripcsak2025,
  author = {Hripcsak, George and Zhang, Linying and Chen, Yong and Li, Kelly and Suchard, Marc A and Ryan, Patrick B and Schuemie, Martijn J},
  title = {Assessing Covariate Balance With Small Sample Sizes},
  journal = {Statistics in Medicine},
  year = {2025},
  doi = {10.1002/sim.70212}
}
Kostenlos starten →