Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study
American Journal of Epidemiology · 2025
DOI: 10.1093/aje/kwaf184 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Epidemiology, 2025. Verfasst von Sarah F Ackley und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Epidemiology, Brown University, Providence, RI, United States. Hauptthemen laut MEDLINE: Periodicals as Topic, Epidemiology. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NIH NIA.
Abstract (Originalsprache)
Epidemiologists have advocated for reporting confidence intervals and deemphasizing P values to address long-standing concerns about null-hypothesis statistical-significance testing, P hacking, and reproducibility. It is unknown if efforts to reduce reliance on P values have altered the distribution of P values. For 21 332 abstracts published 2000 to 2024 in 4 major epidemiology journals, two-sided P values (N = 25 288) were calculated from estimates and confidence intervals scraped using ChatGPT’s 4o-mini model. We evaluated trends over time to determine whether the empirical distribution of P values changed. We fitted to expected P-value distributions and simulated these distributions with and without assuming changes in statistical power over time. Average P values decreased from 2000 to 2024; the fraction of P values just below the .05 threshold also decreased. Fits to models indicate that statistical power increased. Increasing power would reduce average P value while also decreasing P values near the .05 threshold—precisely the trends observed in epidemiology journals. Although the frequency of P values near the .05 threshold has declined modestly, this likely reflects increases in statistical power rather than decreases in P hacking.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Epidemiology, Brown University, Providence, RI, United States.
- Department of Epidemiology, Boston University, Boston, MA, United States.
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, CA, United States.
ORCID-Profile: Sarah F Ackley, Erin Ferguson, M Maria Glymour
Förderung
- NIA NIH HHS
- NIH NIA
Zitieren
APA
Ackley, S. F., Andrews, R. M., Seaman, C., Flanders, M., Chen, R., Wang, J., Lopes, G., Sims, K. D., Buto, P., Ferguson, E., Allen, I. E., & Glymour, M. M. (2025). Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study. American Journal of Epidemiology. https://doi.org/10.1093/aje/kwaf184
Vancouver
Ackley SF, Andrews RM, Seaman C, Flanders M, Chen R, Wang J, et al. Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study. American Journal of Epidemiology. 2025. doi:10.1093/aje/kwaf184
BibTeX
@article{Ackley2025,
author = {Ackley, Sarah F and Andrews, Ryan M and Seaman, Christopher and Flanders, Michael and Chen, Ruijia and Wang, Jingxuan and Lopes, Grissel and Sims, Kendra D and Buto, Peter and Ferguson, Erin and Allen, Isabel Elaine and Glymour, M Maria},
title = {Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study},
journal = {American Journal of Epidemiology},
year = {2025},
doi = {10.1093/aje/kwaf184}
}
EUVIMED