EUVIMEDEUVIMED entdecken
MetaanalyseOpen Access

Meta-analyses in psychology often overestimate evidence for and size of effects

František Bartoš iD, Maximilian Maier iD, David R Shanks iD, T D Stanley, Martina Sladekova iD, Eric-Jan Wagenmakers iD

Royal Society Open Science · 2023-07 · 10 (7)

DOI: 10.1098/rsos.230224 ↗

Kurz gefasst

Metaanalyse in der Zeitschrift Royal Society Open Science, online erschienen am 05.07.2023. Verfasst von František Bartoš und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NWO.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Adjusting for publication bias is essential when drawing meta-analytic inferences. However, most methods that adjust for publication bias do not perform well across a range of research conditions, such as the degree of heterogeneity in effect sizes across studies. Sladekova et al . 2022 (Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods. Psychol. Methods ) tried to circumvent this complication by selecting the methods that are most appropriate for a given set of conditions, and concluded that publication bias on average causes only minimal over-estimation of effect sizes in psychology. However, this approach suffers from a ‘Catch-22’ problem—to know the underlying research conditions, one needs to have adjusted for publication bias correctly, but to correctly adjust for publication bias, one needs to know the underlying research conditions. To alleviate this problem, we conduct an alternative analysis, robust Bayesian meta-analysis (RoBMA), which is not based on model-selection but on model-averaging . In RoBMA, models that predict the observed results better are given correspondingly larger weights. A RoBMA reanalysis of Sladekova et al .’s dataset reveals that more than 60% of meta-analyses in psychology notably overestimate the evidence for the presence of the meta-analytic effect and more than 50% overestimate its magnitude.

Schlagwörter

Stichwörter der Autor:innen

Bayesian inferenceRoBMAmeta-analysismodel-averagingmodel-selectionpublication bias

Autor:innen und Institutionen

  • Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands.
  • Department of Experimental Psychology, University College London, London, UK.
  • Deakin Laboratory for the Meta-Analysis of Research (DeLMAR), Deakin University, Burwood, Australia.
  • Department of Economics, School of Business, Deakin University, Burwood, Australia.
  • School of Psychology, University of Sussex, Brighton, UK.

ORCID-Profile: František Bartoš, Maximilian Maier, David R Shanks, Martina Sladekova, Eric-Jan Wagenmakers

Förderung

  • NWO

Zitieren

APA

Bartoš, F., Maier, M., Shanks, D. R., Stanley, T. D., Sladekova, M., & Wagenmakers, E. J. (2023). Meta-analyses in psychology often overestimate evidence for and size of effects. Royal Society Open Science, 10(7). https://doi.org/10.1098/rsos.230224

Vancouver

Bartoš F, Maier M, Shanks DR, Stanley TD, Sladekova M, Wagenmakers EJ. Meta-analyses in psychology often overestimate evidence for and size of effects. Royal Society Open Science. 2023;10(7). doi:10.1098/rsos.230224

BibTeX
@article{Barto2023,
  author = {Bartoš, František and Maier, Maximilian and Shanks, David R and Stanley, T D and Sladekova, Martina and Wagenmakers, Eric-Jan},
  title = {Meta-analyses in psychology often overestimate evidence for and size of effects},
  journal = {Royal Society Open Science},
  year = {2023},
  volume = {10},
  number = {7},
  doi = {10.1098/rsos.230224}
}
Kostenlos starten →