On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials
PLOS One · 2025
DOI: 10.1371/journal.pone.0325920 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Farrokh Habibzadeh. Beteiligte Einrichtung: Independent Research Consultant, Shiraz, Iran. Hauptthemen laut MEDLINE: Randomized Controlled Trials as Topic. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Reproducibility crisis is among major concerns of many scientists worldwide. Some researchers believe that the crisis is mostly attributed to the conventional p significance threshold value arbitrarily chosen to be 0.05 and propose to lower the cut-off to 0.005. Reducing the cut-off, although decreases the false-positive rate, is associated with an increase in false-negative rate. Recently, a flexible p significance threshold that minimizes the weighted sum of errors in statistical inference tests of hypothesis was proposed.
Methods
The current in silico study was conducted to compare the error rates under different conditions assumed for the p significance threshold—0.05, 0.005, and a flexible threshold. Using a Monte Carlo simulation, the false-positive rate (when the null hypothesis was true) and false-negative rate (when the alternative hypothesis was true) were calculated in a hypothetical randomized clinical trial.
Results
Increasing the study sample size was associated with a reduction in the false-negative rate, however, the false-positive rate occurred at a fixed value regardless of the sample size when fixed significance thresholds were used; the rate decreased, however, when the flexible threshold was employed. While employing the flexible threshold abolished the reproducibility crisis to a large extent, the method uncovered an inherent conflict in the frequentist statistical inference framework. Calculation of the flexible p significance threshold is only possible a posteriori , after the results are obtained. The threshold would thus be different even for replicas, which is in contradiction to the common sense.
Conclusions
It seems that relying on frequentist statistical inference and the p value is no longer a viable approach. Emphasis should be shifted toward alternative approaches for data analysis, Bayesian statistical methods, for example.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Independent Research Consultant, Shiraz, Iran.
ORCID-Profile: Farrokh Habibzadeh
Zitieren
APA
Habibzadeh, F. (2025). On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials. PLOS One. https://doi.org/10.1371/journal.pone.0325920
Vancouver
Habibzadeh F. On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials. PLOS One. 2025. doi:10.1371/journal.pone.0325920
BibTeX
@article{Habibzadeh2025,
author = {Habibzadeh, Farrokh},
title = {On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials},
journal = {PLOS One},
year = {2025},
doi = {10.1371/journal.pone.0325920}
}
EUVIMED