A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference
Statistics in Medicine · 2026-08 · 45 (18-19)
DOI: 10.1002/sim.70676 ↗At a glance
Journal article in Statistics in Medicine, published online on 2026-07-22. Written by Pamela A Shaw and 7 co-authors. Institution: Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA. Major MEDLINE topics: Models, Statistical, Causality, Pharmacoepidemiology. 28 references. The full text is freely available in PubMed Central. Funded by: NIAID NIH HHS, U.S. Food and Drug Administration, National Institute of Allergy and Infectious Diseases.
Abstract (original language)
Plasmode simulation has become an important tool for evaluating operating characteristics of different statistical methods in complex settings, such as pharmacoepidemiological studies of treatment effectiveness using electronic health records (EHR) data. These studies provide insight into how estimator performance is impacted by challenges including rare events, small sample size and so forth that can indicate which among a set of methods performs best in a real‐world dataset. Plasmode simulation combines data resampled from a real‐world dataset with data from a probabilistic model to generate a known truth for an estimand in realistic data. There are different potential plasmode strategies currently in use. We compare two popular plasmode simulation frameworks for the evaluation of methods estimating a point treatment estimand. We provide numerical evidence and a theoretical result that shows one of these frameworks can cause certain estimators to incorrectly appear overly biased. Detailed simulation studies using both model‐generated and real‐world EHR data demonstrate these pitfalls remain at large sample sizes and when analyzing data from a randomized controlled trial. We conclude with guidance for the choice of a plasmode framework that maintains good theoretical properties to allow a fair evaluation of statistical methods while also maintaining the desired similarity to real data.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA.
- Department of Biostatistics, University of Washington, Seattle, Washington, USA.
- TL Revolution, LLC, Cambridge, Massachusetts, USA.
- Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA.
- Department of Biostatistics, School of Public Health, University of California at Berkeley, Berkeley, California, USA.
ORCID profiles: Pamela A Shaw, Brian D Williamson, Susan M Shortreed
Funding
- NIAID NIH HHS
- U.S. Food and Drug Administration
- National Institute of Allergy and Infectious Diseases
Cite
APA
Shaw, P. A., Gruber, S., Williamson, B. D., Desai, R., Shortreed, S. M., Krakauer, C., Nelson, J. C., & van der Laan, M. J. (2026). A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference. Statistics in Medicine, 45(18-19). https://doi.org/10.1002/sim.70676
Vancouver
Shaw PA, Gruber S, Williamson BD, Desai R, Shortreed SM, Krakauer C, et al. A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference. Statistics in Medicine. 2026;45(18-19). doi:10.1002/sim.70676
BibTeX
@article{Shaw2026,
author = {Shaw, Pamela A and Gruber, Susan and Williamson, Brian D and Desai, Rishi and Shortreed, Susan M and Krakauer, Chloe and Nelson, Jennifer C and van der Laan, Mark J},
title = {A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference},
journal = {Statistics in Medicine},
year = {2026},
volume = {45},
number = {18-19},
doi = {10.1002/sim.70676}
}
EUVIMED