Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0306911 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Xingyu Li und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Education, Stanford University, Stanford, California, United States of America. Hauptthemen laut MEDLINE: Psychology. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large sample size (N) is seen as a key criterion in judging the replicability of psychological research, a phenomenon we refer to as the N-Heuristic. This heuristic has led to the incentivization of fast, online, non-behavioral studies—to the potential detriment of psychological science. While large N should in principle increase statistical power and thus the replicability of effects, in practice it may not. Large-N studies may have other attributes that undercut their power or validity. Consolidating data from all systematic, large-scale attempts at replication (N = 307 original-replication study pairs), we find that the original study’s sample size did not predict its likelihood of being replicated ( r s = -0.02, p = 0.741), even with study design and research area controlled. By contrast, effect size emerged as a substantial predictor ( r s = 0.21, p < 0.001), which held regardless of the study’s sample size. N may be a poor predictor of replicability because studies with larger N investigated smaller effects ( r s = -0.49, p < 0.001). Contrary to these results, a survey of 215 professional psychologists, presenting them with a comprehensive list of methodological criteria, found sample size to be rated as the most important criterion in judging a study’s replicability. Our findings strike a cautionary note with respect to the prioritization of large N in judging the replicability of psychological science.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Graduate School of Education, Stanford University, Stanford, California, United States of America.
- School of Humanities and Social Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China.
- Teachers College, Columbia University, New York, New York, United States of America.
- Department of Psychology, Stanford University, Stanford, California, United States of America.
ORCID-Profile: Xingyu Li
Zitieren
APA
Li, X., Liu, J., Gao, W., & Cohen, G. L. (2024). Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science. PLOS ONE. https://doi.org/10.1371/journal.pone.0306911
Vancouver
Li X, Liu J, Gao W, Cohen GL. Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science. PLOS ONE. 2024. doi:10.1371/journal.pone.0306911
BibTeX
@article{Li2024,
author = {Li, Xingyu and Liu, Jiting and Gao, Weijia and Cohen, Geoffrey L},
title = {Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0306911}
}
EUVIMED