Estimating the extent of selective reporting: An application to economics
Research Synthesis Methods · 2024
DOI: 10.1002/jrsm.1711 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Synthesis Methods, 2024. Verfasst von Stephan B Bruns und 4 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Environmental Sciences, Hasselt University, Hasselt, Belgium. Hauptthemen laut MEDLINE: Research Design. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: German Research Foundation, Special Research Fund (BOF) UHasselt, Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
Using a sample of 70,399 published p ‐values from 192 meta‐analyses, we empirically estimate the counterfactual distribution of p ‐values in the absence of any biases. Comparing observed p ‐values with counterfactually expected p ‐values allows us to estimate how many p ‐values are published as being statistically significant when they should have been published as non‐significant. We estimate the extent of selectively reported p ‐values to range between 57.7% and 71.9% of the significant p ‐values. The counterfactual p ‐value distribution also allows us to assess shifts of p ‐values along the entire distribution of published p ‐values, revealing that particularly very small p ‐values ( p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centre for Environmental Sciences, Hasselt University, Hasselt, Belgium.
- Department of Economics, University of Göttingen, Göttingen, Germany.
- Meta-Research Innovation Center at Stanford (METRICS), Stanford, California, USA.
- Department of Economics, Deakin University, Melbourne, Victoria, Australia.
- Department of Medicine, Stanford University, Stanford, California, USA.
- Department of Epidemiology and Population Health, Stanford University, Stanford, California, USA.
ORCID-Profile: Stephan B Bruns, T D Stanley
Förderung
- German Research Foundation
- Special Research Fund (BOF) UHasselt
- Deutsche Forschungsgemeinschaft
- Bijzonder Onderzoeksfonds UGent
Zitieren
APA
Bruns, S. B., Deressa, T. K., Stanley, T. D., Doucouliagos, C., & Ioannidis, J. P. A. (2024). Estimating the extent of selective reporting: An application to economics. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1711
Vancouver
Bruns SB, Deressa TK, Stanley TD, Doucouliagos C, Ioannidis JPA. Estimating the extent of selective reporting: An application to economics. Research Synthesis Methods. 2024. doi:10.1002/jrsm.1711
BibTeX
@article{Bruns2024,
author = {Bruns, Stephan B and Deressa, Teshome K and Stanley, T D and Doucouliagos, Chris and Ioannidis, John P A},
title = {Estimating the extent of selective reporting: An application to economics},
journal = {Research Synthesis Methods},
year = {2024},
doi = {10.1002/jrsm.1711}
}
EUVIMED