EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials

Farrokh Habibzadeh iD

PLOS One · 2025

DOI: 10.1371/journal.pone.0325920 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Farrokh Habibzadeh. Beteiligte Einrichtung: Independent Research Consultant, Shiraz, Iran. Hauptthemen laut MEDLINE: Randomized Controlled Trials as Topic. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Reproducibility crisis is among major concerns of many scientists worldwide. Some researchers believe that the crisis is mostly attributed to the conventional p significance threshold value arbitrarily chosen to be 0.05 and propose to lower the cut-off to 0.005. Reducing the cut-off, although decreases the false-positive rate, is associated with an increase in false-negative rate. Recently, a flexible p significance threshold that minimizes the weighted sum of errors in statistical inference tests of hypothesis was proposed.

Methods

The current in silico study was conducted to compare the error rates under different conditions assumed for the p significance threshold—0.05, 0.005, and a flexible threshold. Using a Monte Carlo simulation, the false-positive rate (when the null hypothesis was true) and false-negative rate (when the alternative hypothesis was true) were calculated in a hypothetical randomized clinical trial.

Results

Increasing the study sample size was associated with a reduction in the false-negative rate, however, the false-positive rate occurred at a fixed value regardless of the sample size when fixed significance thresholds were used; the rate decreased, however, when the flexible threshold was employed. While employing the flexible threshold abolished the reproducibility crisis to a large extent, the method uncovered an inherent conflict in the frequentist statistical inference framework. Calculation of the flexible p significance threshold is only possible a posteriori , after the results are obtained. The threshold would thus be different even for replicas, which is in contradiction to the common sense.

Conclusions

It seems that relying on frequentist statistical inference and the p value is no longer a viable approach. Emphasis should be shifted toward alternative approaches for data analysis, Bayesian statistical methods, for example.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Randomized Controlled Trials as TopicReproducibility of ResultsHumansMonte Carlo MethodSample SizeComputer SimulationFalse Positive Reactions

Autor:innen und Institutionen

  • Independent Research Consultant, Shiraz, Iran.

ORCID-Profile: Farrokh Habibzadeh

Zitieren

APA

Habibzadeh, F. (2025). On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials. PLOS One. https://doi.org/10.1371/journal.pone.0325920

Vancouver

Habibzadeh F. On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials. PLOS One. 2025. doi:10.1371/journal.pone.0325920

BibTeX
@article{Habibzadeh2025,
  author = {Habibzadeh, Farrokh},
  title = {On the effect of flexible adjustment of the p value significance threshold on the reproducibility of randomized clinical trials},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0325920}
}
Kostenlos starten →