Meta-analyses in psychology often overestimate evidence for and size of effects
Royal Society Open Science · 2023-07 · 10 (7)
DOI: 10.1098/rsos.230224 ↗Kurz gefasst
Metaanalyse in der Zeitschrift Royal Society Open Science, online erschienen am 05.07.2023. Verfasst von František Bartoš und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NWO.
Abstract (Originalsprache)
Adjusting for publication bias is essential when drawing meta-analytic inferences. However, most methods that adjust for publication bias do not perform well across a range of research conditions, such as the degree of heterogeneity in effect sizes across studies. Sladekova et al . 2022 (Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods. Psychol. Methods ) tried to circumvent this complication by selecting the methods that are most appropriate for a given set of conditions, and concluded that publication bias on average causes only minimal over-estimation of effect sizes in psychology. However, this approach suffers from a ‘Catch-22’ problem—to know the underlying research conditions, one needs to have adjusted for publication bias correctly, but to correctly adjust for publication bias, one needs to know the underlying research conditions. To alleviate this problem, we conduct an alternative analysis, robust Bayesian meta-analysis (RoBMA), which is not based on model-selection but on model-averaging . In RoBMA, models that predict the observed results better are given correspondingly larger weights. A RoBMA reanalysis of Sladekova et al .’s dataset reveals that more than 60% of meta-analyses in psychology notably overestimate the evidence for the presence of the meta-analytic effect and more than 50% overestimate its magnitude.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands.
- Department of Experimental Psychology, University College London, London, UK.
- Deakin Laboratory for the Meta-Analysis of Research (DeLMAR), Deakin University, Burwood, Australia.
- Department of Economics, School of Business, Deakin University, Burwood, Australia.
- School of Psychology, University of Sussex, Brighton, UK.
ORCID-Profile: František Bartoš, Maximilian Maier, David R Shanks, Martina Sladekova, Eric-Jan Wagenmakers
Förderung
- NWO
Zitieren
APA
Bartoš, F., Maier, M., Shanks, D. R., Stanley, T. D., Sladekova, M., & Wagenmakers, E. J. (2023). Meta-analyses in psychology often overestimate evidence for and size of effects. Royal Society Open Science, 10(7). https://doi.org/10.1098/rsos.230224
Vancouver
Bartoš F, Maier M, Shanks DR, Stanley TD, Sladekova M, Wagenmakers EJ. Meta-analyses in psychology often overestimate evidence for and size of effects. Royal Society Open Science. 2023;10(7). doi:10.1098/rsos.230224
BibTeX
@article{Barto2023,
author = {Bartoš, František and Maier, Maximilian and Shanks, David R and Stanley, T D and Sladekova, Martina and Wagenmakers, Eric-Jan},
title = {Meta-analyses in psychology often overestimate evidence for and size of effects},
journal = {Royal Society Open Science},
year = {2023},
volume = {10},
number = {7},
doi = {10.1098/rsos.230224}
}
EUVIMED