Assessing bias and precision in state policy evaluations: a comparative analysis of time-varying estimators using policy simulations
American Journal of Epidemiology · 08.07.2026 · 195 (7) · 2010-2019
DOI: 10.1093/aje/kwag041 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift American Journal of Epidemiology, online erschienen am 28.02.2026. Verfasst von Max Griswold und 8 weiteren Autor:innen. Beteiligte Einrichtung: Economics, Statistics & Sociology, RAND, Santa Monica, CA, United States. Hauptthemen laut MEDLINE: Health Policy, Opiate Overdose. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NIDA NIH HHS, NIH HHS, National Institutes of Health.
Abstract (Originalsprache)
Using state-level opioid overdose mortality data (1999-2016), we evaluated the performance of panel data estimators for capturing time-varying impacts of state-level policies. Most health policy evaluations assume static treatment effects, yet many interventions exhibit dynamic impacts, raising methodological questions about optimal estimation strategies. We simulated four time-varying treatment scenarios reflecting common policy dynamics (gradual increase, gradual decline, temporary effects, and inconsistent trajectories) and compared seven methods: two-way fixed effects event study, debiased autoregressive model, augmented synthetic control, difference in differences with staggered adoption, event study with heterogeneous treatment, two-stage differences in differences, and differences-in-differences imputation. Performance was assessed using bias, standard errors, coverage probability, and root mean squared error. Estimator performance varied substantially across scenarios. Augmented synthetic controls showed lower bias but higher variance when policy effectiveness diminished over time. Difference-in-difference approaches provided reasonable coverage in some scenarios but struggled with non-monotonic effects, while autoregressive methods exhibited lower variability but underestimated uncertainty. Overall, no single estimator performed best across settings. For epidemiological policy evaluations, particularly time-sensitive interventions like opioid-related policies, researchers should weigh bias-variance tradeoffs and align methodological choices with expected effect trajectories. Careful selection of analytic approaches is critical to avoid misattribution of policy effects and ensure valid conclusions about population health outcomes.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Economics, Statistics & Sociology, RAND, Santa Monica, CA, United States.
- Economics, Statistics & Sociology, RAND, Arlington, VA, United States.
- Economics, Statistics & Sociology, RAND, Pittsburgh, PA, United States.
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, QC, Canada.
- Department of Psychiatry, Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ, United States.
- Engineering & Applied Sciences, RAND, Santa Monica, CA, United States.
ORCID-Profile: Max Griswold, Pedro Nascimento de Lima
Förderung
- NIDA NIH HHS
- NIH HHS
- National Institutes of Health
Zitieren
APA
Griswold, M., Griffin, B. A., Rubinstein, M., Liu, M., Schuler, M., Stone, E., de Lima, P. N., Stein, B. D., & Stuart, E. A. (2026). Assessing bias and precision in state policy evaluations: a comparative analysis of time-varying estimators using policy simulations. American Journal of Epidemiology, 195(7), 2010-2019. https://doi.org/10.1093/aje/kwag041
Vancouver
Griswold M, Griffin BA, Rubinstein M, Liu M, Schuler M, Stone E, et al. Assessing bias and precision in state policy evaluations: a comparative analysis of time-varying estimators using policy simulations. American Journal of Epidemiology. 2026;195(7):2010-2019. doi:10.1093/aje/kwag041
BibTeX
@article{Griswold2026,
author = {Griswold, Max and Griffin, Beth Ann and Rubinstein, Max and Liu, Mincen and Schuler, Megan and Stone, Elizabeth and de Lima, Pedro Nascimento and Stein, Bradley D and Stuart, Elizabeth A},
title = {Assessing bias and precision in state policy evaluations: a comparative analysis of time-varying estimators using policy simulations},
journal = {American Journal of Epidemiology},
year = {2026},
volume = {195},
number = {7},
pages = {2010--2019},
doi = {10.1093/aje/kwag041}
}
EUVIMED