EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitOpen Access · CC BY 4.0

Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia

David Sidebotham, Felicity Dominick, Carolyn Deng, Jake Barlow, Philip M Jones

British Journal of Anaesthesia · 2024

DOI: 10.1016/j.bja.2023.10.036 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift British Journal of Anaesthesia, 2024. Verfasst von David Sidebotham und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auc…. Hauptthemen laut MEDLINE: Anesthesia, Anesthesiology. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: The American Statistical Association has highlighted problems with null hypothesis significance testing and outlined alternative approaches that may 'supplement or even replace P-values'. One alternative is to report the false positive risk (FPR), which quantifies the chance the null hypothesis is true when the result is statistically significant. METHODS: We reviewed single-centre, randomised trials in 10 anaesthesia journals over 6 yr where differences in a primary binary outcome were statistically significant. We calculated a Bayes factor by two methods (Gunel, Kass). From the Bayes factor we calculated the FPR for different prior beliefs for a real treatment effect. Prior beliefs were quantified by assigning pretest probabilities to the null and alternative hypotheses. RESULTS: For equal pretest probabilities of 0.5, the median (inter-quartile range [IQR]) FPR was 6% (1-22%) by the Gunel method and 6% (1-19%) by the Kass method. One in five trials had an FPR ≥20%. For trials reporting P-values 0.01-0.05, the median (IQR) FPR was 25% (16-30%) by the Gunel method and 20% (16-25%) by the Kass method. More than 90% of trials reporting P-values 0.01-0.05 required a pretest probability >0.5 to achieve an FPR of 5%. The median (IQR) difference in the FPR calculated by the two methods was 0% (0-2%). CONCLUSIONS: Our findings suggest that a substantial proportion of single-centre trials in anaesthesia reporting statistically significant differences provide limited evidence of real treatment effects, or, alternatively, required an implausibly high prior belief in a real treatment effect. CLINICAL TRIAL REGISTRATION: PROSPERO (CRD42023350783).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AnesthesiaAnesthesiologyHumansBayes TheoremData Interpretation, StatisticalResearch Design

Stichwörter der Autor:innen

Bayes factorBayes' theoremanaesthesiafalse positive riskresearch designsample sizesignificance testing

Autor:innen und Institutionen

  • Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand; Department of Anaesthesiol…
  • Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand.
  • Department of Anaesthesiology, Faculty of Health Sciences, University of Auckland, New Zealand; Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
  • Department of Cardiothoracic and ORL Anaesthesia, Auckland City Hospital, Auckland, New Zealand; Cardiothoracic and Vascular Intensive Care Unit, Auckland City Hospital, Auckland, New Zealand.
  • Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Jacksonville, FL, USA.

Zitieren

APA

Sidebotham, D., Dominick, F., Deng, C., Barlow, J., & Jones, P. M. (2024). Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. British Journal of Anaesthesia. https://doi.org/10.1016/j.bja.2023.10.036

Vancouver

Sidebotham D, Dominick F, Deng C, Barlow J, Jones PM. Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia. British Journal of Anaesthesia. 2024. doi:10.1016/j.bja.2023.10.036

BibTeX
@article{Sidebotham2024,
  author = {Sidebotham, David and Dominick, Felicity and Deng, Carolyn and Barlow, Jake and Jones, Philip M},
  title = {Statistically significant differences versus convincing evidence of real treatment effects: an analysis of the false positive risk for single-centre trials in anaesthesia},
  journal = {British Journal of Anaesthesia},
  year = {2024},
  doi = {10.1016/j.bja.2023.10.036}
}
Kostenlos starten →