Questioning the Credibility of Educational Meta-Analyses: Insights From Multimethod Reanalyses
Review of Educational Research · 10.11.2026
DOI: 10.3102/00346543261492727 ↗Kurz gefasst
Metaanalyse in der Zeitschrift Review of Educational Research, online erschienen am 10.11.2026. Verfasst von Luke Rowe. Die Arbeit nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
This study investigates the credibility of meta-analyses published in the Review of Educational Research from 2012 to 2022 by surveying effect-size inflation, statistical power, heterogeneity, and analytic reproducibility using classical and robust Bayesian methods. A corpus of 362 separate meta-analytic effects derived from 76 published articles was included. Data from a subset of 53 articles with 189 effects were reanalyzed; 94% of experimental effects and 79% of observational reported correlations exceeded the magnitude of those reproduced using robust Bayesian estimates. The difference in the magnitude of the mean reported effect sizes and correlations was substantially larger than those reproduced using robust Bayesian methods (ranging from 29% to 267%). Median retrospective power was extremely low in experimental research, with only 4.21% of meta-analyses exceeding Cohen’s .80 threshold using robust Bayesian estimates; observed power based on 10,672 primary-study effects indicated that only 29.0% met this benchmark. Observational studies demonstrated comparatively higher power, with 53.3% surpassing Cohen’s .80 threshold using robust Bayesian estimates, and 68% of the 8,189 correlations sufficiently powered to detect their reported correlation strength. Median heterogeneity remained high across designs, with I 2 values exceeding 80% in experimental and 90% in observational meta-analyses. Just over half (55.3%) of meta-analyses initially shared sufficient data, increasing to ~70% following email request. Only ~5% shared sufficient code for partial analytic reproduction. These findings identify potential concerns regarding effect-size inflation, statistical power, heterogeneity, and data and code availability that may constrain reproducibility in educational meta-analyses.
Zitieren
APA
Rowe, L. (2026). Questioning the Credibility of Educational Meta-Analyses: Insights From Multimethod Reanalyses. Review of Educational Research. https://doi.org/10.3102/00346543261492727
Vancouver
Rowe L. Questioning the Credibility of Educational Meta-Analyses: Insights From Multimethod Reanalyses. Review of Educational Research. 2026. doi:10.3102/00346543261492727
BibTeX
@article{Rowe2026,
author = {Rowe, Luke},
title = {Questioning the Credibility of Educational Meta-Analyses: Insights From Multimethod Reanalyses},
journal = {Review of Educational Research},
year = {2026},
doi = {10.3102/00346543261492727}
}
EUVIMED