EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Estimating the extent of selective reporting: An application to economics

Stephan B Bruns iD, Teshome K Deressa, T D Stanley iD, Chris Doucouliagos, John P A Ioannidis

Research Synthesis Methods · 2024

DOI: 10.1002/jrsm.1711 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Research Synthesis Methods, 2024. Verfasst von Stephan B Bruns und 4 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Environmental Sciences, Hasselt University, Hasselt, Belgium. Hauptthemen laut MEDLINE: Research Design. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: German Research Foundation, Special Research Fund (BOF) UHasselt, Deutsche Forschungsgemeinschaft.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Using a sample of 70,399 published p ‐values from 192 meta‐analyses, we empirically estimate the counterfactual distribution of p ‐values in the absence of any biases. Comparing observed p ‐values with counterfactually expected p ‐values allows us to estimate how many p ‐values are published as being statistically significant when they should have been published as non‐significant. We estimate the extent of selectively reported p ‐values to range between 57.7% and 71.9% of the significant p ‐values. The counterfactual p ‐value distribution also allows us to assess shifts of p ‐values along the entire distribution of published p ‐values, revealing that particularly very small p ‐values ( p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Research DesignHumansMeta-Analysis as TopicPublication BiasEconomicsModels, StatisticalData Interpretation, StatisticalAlgorithmsReproducibility of ResultsBias

Stichwörter der Autor:innen

meta‐researchpublication biasp‐hackingselective reporting

Autor:innen und Institutionen

  • Centre for Environmental Sciences, Hasselt University, Hasselt, Belgium.
  • Department of Economics, University of Göttingen, Göttingen, Germany.
  • Meta-Research Innovation Center at Stanford (METRICS), Stanford, California, USA.
  • Department of Economics, Deakin University, Melbourne, Victoria, Australia.
  • Department of Medicine, Stanford University, Stanford, California, USA.
  • Department of Epidemiology and Population Health, Stanford University, Stanford, California, USA.

ORCID-Profile: Stephan B Bruns, T D Stanley

Förderung

  • German Research Foundation
  • Special Research Fund (BOF) UHasselt
  • Deutsche Forschungsgemeinschaft
  • Bijzonder Onderzoeksfonds UGent

Zitieren

APA

Bruns, S. B., Deressa, T. K., Stanley, T. D., Doucouliagos, C., & Ioannidis, J. P. A. (2024). Estimating the extent of selective reporting: An application to economics. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1711

Vancouver

Bruns SB, Deressa TK, Stanley TD, Doucouliagos C, Ioannidis JPA. Estimating the extent of selective reporting: An application to economics. Research Synthesis Methods. 2024. doi:10.1002/jrsm.1711

BibTeX
@article{Bruns2024,
  author = {Bruns, Stephan B and Deressa, Teshome K and Stanley, T D and Doucouliagos, Chris and Ioannidis, John P A},
  title = {Estimating the extent of selective reporting: An application to economics},
  journal = {Research Synthesis Methods},
  year = {2024},
  doi = {10.1002/jrsm.1711}
}
Kostenlos starten →