EUVIMEDEUVIMED entdecken
Crossref

p -Hacking and Publication Bias in Design-Based Causal Studies in Information Systems

Mike Nguyen, Minh Nguyen

Information Systems Research · 17.08.2026

DOI: 10.1287/isre.2024.1365 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Systems Research, online erschienen am 17.08.2026. Verfasst von Mike Nguyen und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Design-based causal methods such as difference in differences, instrumental variables, randomized controlled trials, regression discontinuity, and synthetic control have become the foundation of evidence-based decision making in information systems (IS). Their value, however, depends on the credibility of the published evidence. Analyzing 7,516 hypothesis tests from 558 articles published in seven leading IS journals between 2000 and 2023, we identify systematic clustering of statistical results immediately above the conventional 5% significance threshold, particularly for difference in differences, instrumental variables, and randomized controlled trials. These patterns suggest that publication incentives and researcher degrees of freedom may distort the evidence base used by managers and policymakers. We translate these findings into practical recommendations for researchers, reviewers, and editors. Specifically, we advocate greater transparency in research design and reporting, stronger requirements for data and code availability, broader acceptance of well-executed null findings, routine sensitivity analyses, and method-specific reporting standards for causal studies. We also introduce an IS-calibrated local false discovery rate that helps readers assess the credibility of reported causal estimates. By strengthening evidence quality rather than discouraging causal research, these recommendations aim to improve the reliability, reproducibility, and policy relevance of empirical findings that increasingly guide organizational investment, digital transformation, platform governance, and public policy decisions.

Zitieren

APA

Nguyen, M., & Nguyen, M. (2026). p -Hacking and Publication Bias in Design-Based Causal Studies in Information Systems. Information Systems Research. https://doi.org/10.1287/isre.2024.1365

Vancouver

Nguyen M, Nguyen M. p -Hacking and Publication Bias in Design-Based Causal Studies in Information Systems. Information Systems Research. 2026. doi:10.1287/isre.2024.1365

BibTeX
@article{Nguyen2026,
  author = {Nguyen, Mike and Nguyen, Minh},
  title = {p -Hacking and Publication Bias in Design-Based Causal Studies in Information Systems},
  journal = {Information Systems Research},
  year = {2026},
  doi = {10.1287/isre.2024.1365}
}
Kostenlos starten →