Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia
British Journal of Anaesthesia · 2024
DOI: 10.1016/j.bja.2023.10.036 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift British Journal of Anaesthesia, 2024. Verfasst von David Sidebotham und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auc…. Hauptthemen laut MEDLINE: Anesthesia, Anesthesiology. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
BACKGROUND: The American Statistical Association has highlighted problems with null hypothesis significance testing and outlined alternative approaches that may 'supplement or even replace P-values'. One alternative is to report the false positive risk (FPR), which quantifies the chance the null hypothesis is true when the result is statistically significant. METHODS: We reviewed single-centre, randomised trials in 10 anaesthesia journals over 6 yr where differences in a primary binary outcome were statistically significant. We calculated a Bayes factor by two methods (Gunel, Kass). From the Bayes factor we calculated the FPR for different prior beliefs for a real treatment effect. Prior beliefs were quantified by assigning pretest probabilities to the null and alternative hypotheses. RESULTS: For equal pretest probabilities of 0.5, the median (inter-quartile range [IQR]) FPR was 6% (1-22%) by the Gunel method and 6% (1-19%) by the Kass method. One in five trials had an FPR ≥20%. For trials reporting P-values 0.01-0.05, the median (IQR) FPR was 25% (16-30%) by the Gunel method and 20% (16-25%) by the Kass method. More than 90% of trials reporting P-values 0.01-0.05 required a pretest probability >0.5 to achieve an FPR of 5%. The median (IQR) difference in the FPR calculated by the two methods was 0% (0-2%). CONCLUSIONS: Our findings suggest that a substantial proportion of single-centre trials in anaesthesia reporting statistically significant differences provide limited evidence of real treatment effects, or, alternatively, required an implausibly high prior belief in a real treatment effect. CLINICAL TRIAL REGISTRATION: PROSPERO (CRD42023350783).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand; Department of Anaesthesiol…
- Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand.
- Department of Anaesthesiology, Faculty of Health Sciences, University of Auckland, New Zealand; Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
- Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand.
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Jacksonville, FL, USA.
Zitieren
APA
Sidebotham, D., Dominick, F., Deng, C., Barlow, J., & Jones, P. M. (2024). Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. British Journal of Anaesthesia. https://doi.org/10.1016/j.bja.2023.10.036
Vancouver
Sidebotham D, Dominick F, Deng C, Barlow J, Jones PM. Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. British Journal of Anaesthesia. 2024. doi:10.1016/j.bja.2023.10.036
BibTeX
@article{Sidebotham2024,
author = {Sidebotham, David and Dominick, Felicity and Deng, Carolyn and Barlow, Jake and Jones, Philip M},
title = {Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia},
journal = {British Journal of Anaesthesia},
year = {2024},
doi = {10.1016/j.bja.2023.10.036}
}
EUVIMED