EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science

Xingyu Li iD, Jiting Liu, Weijia Gao, Geoffrey L Cohen

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0306911 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Xingyu Li und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Education, Stanford University, Stanford, California, United States of America. Hauptthemen laut MEDLINE: Psychology. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large sample size (N) is seen as a key criterion in judging the replicability of psychological research, a phenomenon we refer to as the N-Heuristic. This heuristic has led to the incentivization of fast, online, non-behavioral studies—to the potential detriment of psychological science. While large N should in principle increase statistical power and thus the replicability of effects, in practice it may not. Large-N studies may have other attributes that undercut their power or validity. Consolidating data from all systematic, large-scale attempts at replication (N = 307 original-replication study pairs), we find that the original study’s sample size did not predict its likelihood of being replicated ( r s = -0.02, p = 0.741), even with study design and research area controlled. By contrast, effect size emerged as a substantial predictor ( r s = 0.21, p < 0.001), which held regardless of the study’s sample size. N may be a poor predictor of replicability because studies with larger N investigated smaller effects ( r s = -0.49, p < 0.001). Contrary to these results, a survey of 215 professional psychologists, presenting them with a comprehensive list of methodological criteria, found sample size to be rated as the most important criterion in judging a study’s replicability. Our findings strike a cautionary note with respect to the prioritization of large N in judging the replicability of psychological science.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PsychologySample SizeHumansReproducibility of ResultsHeuristics

Autor:innen und Institutionen

  • Graduate School of Education, Stanford University, Stanford, California, United States of America.
  • School of Humanities and Social Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China.
  • Teachers College, Columbia University, New York, New York, United States of America.
  • Department of Psychology, Stanford University, Stanford, California, United States of America.

ORCID-Profile: Xingyu Li

Zitieren

APA

Li, X., Liu, J., Gao, W., & Cohen, G. L. (2024). Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science. PLOS ONE. https://doi.org/10.1371/journal.pone.0306911

Vancouver

Li X, Liu J, Gao W, Cohen GL. Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science. PLOS ONE. 2024. doi:10.1371/journal.pone.0306911

BibTeX
@article{Li2024,
  author = {Li, Xingyu and Liu, Jiting and Gao, Weijia and Cohen, Geoffrey L},
  title = {Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0306911}
}
Kostenlos starten →