EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Risk of Bias in Experiments, Quasi-Experiments and Natural Experiments Across Disciplines: Discussion Paper and Assessment Framework

Hugh Sharma Waddington iD, David B Wilson, Terri Pigott, Ariel M Aloe, Peter Tugwell, Vivian Welch, Gavin Stewart

Campbell Systematic Reviews: Better Evidence for a Better World · 2026-06 · 22 (2)

DOI: 10.1177/18911803261435894 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Campbell Systematic Reviews: Better Evidence for a Better World, online erschienen am 18.04.2026. Verfasst von Hugh Sharma Waddington und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Population Health, Planetary Health Group, London School of Hygiene and Tropical Medicine, London, UK. Die Arbeit nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome Trust.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The evidence we provide to support decision-making should be rigorously appraised so that the findings are shown to be valuable. We discuss the risk of bias in impact evaluations on social and natural science topics – that is, studies using a variety of experimental, quasi-experimental and natural experimental approaches to quantify the causal effect of an intervention, program or policy on an outcome of interest. Existing tools to facilitate evaluation of the risk of bias are usually conceptualized to assess either randomized controlled trials (RCTs) or non-randomized studies of interventions, often called quasi-experimental designs (QEDs) or natural experimental evaluations, but not both. The tools do not adequately reflect how common sources of bias might be addressed in impact evaluations in the social and natural sciences, or the bias sources particular to certain types of design, such as participant reactivity to researcher observation in a trial, or when modelling incorporates known selection mechanisms other than randomization, such as subversion of the assignment rule in a discontinuity design. We present a heuristic to assist reviewers in assessing the confidence in causal inferences. Our approach emphasizes four common sources of bias across RCTs, QEDs and natural experiments – the equivalence of groups, the fidelity of study conditions, the adequacy of measurement, and the reporting of analyses – and we provide signaling questions to evaluate these sources of bias for particular types of study. The approach should be adapted to suit interventions and review topics of interest.

Schlagwörter

Stichwörter der Autor:innen

meta-analysisnatural experimentquasi-experimentrandomized controlled trialrisk of biassystematic review

Autor:innen und Institutionen

  • Department of Population Health, Planetary Health Group, London School of Hygiene and Tropical Medicine, London, UK.
  • Criminology, Law and Society Department, George Mason University, Fairfax, VA, USA.
  • School of Public Health and the College of Education & Human Development, Georgia State University, Atlanta, GA, USA.
  • College of Education, University of Iowa, Iowa City, IA, USA.
  • Department of Epidemiology & Community Medicine, Faculty of Medicine, Bruyère Research Institute, University of Ottawa, Ottawa, ON, Canada.
  • The Campbell Collaboration, Philadelphia, PA, USA.

ORCID-Profile: Hugh Sharma Waddington

Förderung

  • Wellcome Trust

Zitieren

APA

Sharma Waddington, H., Wilson, D. B., Pigott, T., Aloe, A. M., Tugwell, P., Welch, V., & Stewart, G. (2026). Risk of Bias in Experiments, Quasi-Experiments and Natural Experiments Across Disciplines: Discussion Paper and Assessment Framework. Campbell Systematic Reviews: Better Evidence for a Better World, 22(2). https://doi.org/10.1177/18911803261435894

Vancouver

Sharma Waddington H, Wilson DB, Pigott T, Aloe AM, Tugwell P, Welch V, et al. Risk of Bias in Experiments, Quasi-Experiments and Natural Experiments Across Disciplines: Discussion Paper and Assessment Framework. Campbell Systematic Reviews: Better Evidence for a Better World. 2026;22(2). doi:10.1177/18911803261435894

BibTeX
@article{SharmaWaddington2026,
  author = {Sharma Waddington, Hugh and Wilson, David B and Pigott, Terri and Aloe, Ariel M and Tugwell, Peter and Welch, Vivian and Stewart, Gavin},
  title = {Risk of Bias in Experiments, Quasi-Experiments and Natural Experiments Across Disciplines: Discussion Paper and Assessment Framework},
  journal = {Campbell Systematic Reviews: Better Evidence for a Better World},
  year = {2026},
  volume = {22},
  number = {2},
  doi = {10.1177/18911803261435894}
}
Kostenlos starten →