EUVIMEDDescoperă EUVIMED
Open access

A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference

Pamela A Shaw iD, Susan Gruber, Brian D Williamson iD, Rishi Desai, Susan M Shortreed iD, Chloe Krakauer, Jennifer C Nelson, Mark J van der Laan

Statistics in Medicine · 2026-08 · 45 (18-19)

DOI: 10.1002/sim.70676 ↗

At a glance

Journal article in Statistics in Medicine, published online on 2026-07-22. Written by Pamela A Shaw and 7 co-authors. Institution: Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA. Major MEDLINE topics: Models, Statistical, Causality, Pharmacoepidemiology. 28 references. The full text is freely available in PubMed Central. Funded by: NIAID NIH HHS, U.S. Food and Drug Administration, National Institute of Allergy and Infectious Diseases.

RezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

Plasmode simulation has become an important tool for evaluating operating characteristics of different statistical methods in complex settings, such as pharmacoepidemiological studies of treatment effectiveness using electronic health records (EHR) data. These studies provide insight into how estimator performance is impacted by challenges including rare events, small sample size and so forth that can indicate which among a set of methods performs best in a real‐world dataset. Plasmode simulation combines data resampled from a real‐world dataset with data from a probabilistic model to generate a known truth for an estimand in realistic data. There are different potential plasmode strategies currently in use. We compare two popular plasmode simulation frameworks for the evaluation of methods estimating a point treatment estimand. We provide numerical evidence and a theoretical result that shows one of these frameworks can cause certain estimators to incorrectly appear overly biased. Detailed simulation studies using both model‐generated and real‐world EHR data demonstrate these pitfalls remain at large sample sizes and when analyzing data from a randomized controlled trial. We conclude with guidance for the choice of a plasmode framework that maintains good theoretical properties to allow a fair evaluation of statistical methods while also maintaining the desired similarity to real data.

Subjects

MeSH terms (MEDLINE)

Models, StatisticalCausalityPharmacoepidemiologyComputer SimulationHumansElectronic Health RecordsRandomized Controlled Trials as TopicData Interpretation, StatisticalBiasSample Size

Author keywords

data‐generating processoutcome‐generating modelparametric simulationsresamplingsimulation studiesstatistical plasmodes

Authors and institutions

  • Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA.
  • Department of Biostatistics, University of Washington, Seattle, Washington, USA.
  • TL Revolution, LLC, Cambridge, Massachusetts, USA.
  • Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA.
  • Department of Biostatistics, School of Public Health, University of California at Berkeley, Berkeley, California, USA.

ORCID profiles: Pamela A Shaw, Brian D Williamson, Susan M Shortreed

Funding

  • NIAID NIH HHS
  • U.S. Food and Drug Administration
  • National Institute of Allergy and Infectious Diseases

Cite

APA

Shaw, P. A., Gruber, S., Williamson, B. D., Desai, R., Shortreed, S. M., Krakauer, C., Nelson, J. C., & van der Laan, M. J. (2026). A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference. Statistics in Medicine, 45(18-19). https://doi.org/10.1002/sim.70676

Vancouver

Shaw PA, Gruber S, Williamson BD, Desai R, Shortreed SM, Krakauer C, et al. A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference. Statistics in Medicine. 2026;45(18-19). doi:10.1002/sim.70676

BibTeX
@article{Shaw2026,
  author = {Shaw, Pamela A and Gruber, Susan and Williamson, Brian D and Desai, Rishi and Shortreed, Susan M and Krakauer, Chloe and Nelson, Jennifer C and van der Laan, Mark J},
  title = {A Cautionary Note for Plasmode Simulation Studies in the Setting of Causal Inference},
  journal = {Statistics in Medicine},
  year = {2026},
  volume = {45},
  number = {18-19},
  doi = {10.1002/sim.70676}
}
Start for free →