EUVIMEDEUVIMED entdecken
Open Access

Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study

Sarah F Ackley iD, Ryan M Andrews, Christopher Seaman, Michael Flanders, Ruijia Chen, Jingxuan Wang, Grissel Lopes, Kendra D Sims, Peter Buto, Erin Ferguson iD, Isabel Elaine Allen, M Maria Glymour iD

American Journal of Epidemiology · 2025

DOI: 10.1093/aje/kwaf184 ↗

Kurz gefasst

Fachartikel in der Zeitschrift American Journal of Epidemiology, 2025. Verfasst von Sarah F Ackley und 11 weiteren Autor:innen. Beteiligte Einrichtung: Department of Epidemiology, Brown University, Providence, RI, United States. Hauptthemen laut MEDLINE: Periodicals as Topic, Epidemiology. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NIH NIA.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Epidemiologists have advocated for reporting confidence intervals and deemphasizing P values to address long-standing concerns about null-hypothesis statistical-significance testing, P hacking, and reproducibility. It is unknown if efforts to reduce reliance on P values have altered the distribution of P values. For 21 332 abstracts published 2000 to 2024 in 4 major epidemiology journals, two-sided P values (N = 25 288) were calculated from estimates and confidence intervals scraped using ChatGPT’s 4o-mini model. We evaluated trends over time to determine whether the empirical distribution of P values changed. We fitted to expected P-value distributions and simulated these distributions with and without assuming changes in statistical power over time. Average P values decreased from 2000 to 2024; the fraction of P values just below the .05 threshold also decreased. Fits to models indicate that statistical power increased. Increasing power would reduce average P value while also decreasing P values near the .05 threshold—precisely the trends observed in epidemiology journals. Although the frequency of P values near the .05 threshold has declined modestly, this likely reflects increases in statistical power rather than decreases in P hacking.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Periodicals as TopicEpidemiologyHumansData Interpretation, StatisticalModels, StatisticalComputer SimulationConfidence IntervalsReproducibility of Results

Stichwörter der Autor:innen

P hackingP-curve analysisP-valuepublication biasreproducibility

Autor:innen und Institutionen

  • Department of Epidemiology, Brown University, Providence, RI, United States.
  • Department of Epidemiology, Boston University, Boston, MA, United States.
  • Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, CA, United States.

ORCID-Profile: Sarah F Ackley, Erin Ferguson, M Maria Glymour

Förderung

  • NIA NIH HHS
  • NIH NIA

Zitieren

APA

Ackley, S. F., Andrews, R. M., Seaman, C., Flanders, M., Chen, R., Wang, J., Lopes, G., Sims, K. D., Buto, P., Ferguson, E., Allen, I. E., & Glymour, M. M. (2025). Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study. American Journal of Epidemiology. https://doi.org/10.1093/aje/kwaf184

Vancouver

Ackley SF, Andrews RM, Seaman C, Flanders M, Chen R, Wang J, et al. Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study. American Journal of Epidemiology. 2025. doi:10.1093/aje/kwaf184

BibTeX
@article{Ackley2025,
  author = {Ackley, Sarah F and Andrews, Ryan M and Seaman, Christopher and Flanders, Michael and Chen, Ruijia and Wang, Jingxuan and Lopes, Grissel and Sims, Kendra D and Buto, Peter and Ferguson, Erin and Allen, Isabel Elaine and Glymour, M Maria},
  title = {Trends in the distribution of P values in epidemiology journals: a statistical, P -curve, and simulation study},
  journal = {American Journal of Epidemiology},
  year = {2025},
  doi = {10.1093/aje/kwaf184}
}
Kostenlos starten →