Reducing Interference Bias in Online Marketplace Experiments Using Cluster Randomization: Evidence from a Pricing Meta-experiment on Airbnb
Management Science · 05.04.2024
DOI: 10.1287/mnsc.2020.01157 ↗Kurz gefasst
Fachartikel in der Zeitschrift Management Science, online erschienen am 05.04.2024. Verfasst von David Holtz und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 33 Literaturangaben.
Abstract (Originalsprache)
Online marketplace designers frequently run randomized experiments to measure the impact of proposed product changes. However, given that marketplaces are inherently connected, total average treatment effect (TATE) estimates obtained through individual-level randomized experiments may be biased because of violations of the stable unit treatment value assumption, a phenomenon we refer to as “interference bias.” Cluster randomization (i.e., the practice of randomizing treatment assignment at the level of “clusters” of similar individuals) is an established experiment design technique for countering interference bias in social networks, but it is unclear ex ante if it will be effective in marketplace settings. In this paper, we use a meta-experiment or “experiment over experiments” conducted on Airbnb to both provide empirical evidence of interference bias in online marketplace settings and assess the viability of cluster randomization as a tool for reducing interference bias in marketplace TATE estimates. Results from our meta-experiment indicate that at least 20% of the TATE estimate produced by an individual-level randomized evaluation of the platform fee increase we study is attributable to interference bias and eliminated through the use of cluster randomization. We also find suggestive, nonstatistically significant evidence that interference bias in seller-side experiments is more severe in demand-constrained geographies and that the efficacy of cluster randomization at reducing interference bias increases with cluster quality. This paper was accepted by Chris Forman, information systems. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2020.01157 .
Zitieren
APA
Holtz, D., Lobel, F., Lobel, R., Liskovich, I., & Aral, S. (2024). Reducing Interference Bias in Online Marketplace Experiments Using Cluster Randomization: Evidence from a Pricing Meta-experiment on Airbnb. Management Science. https://doi.org/10.1287/mnsc.2020.01157
Vancouver
Holtz D, Lobel F, Lobel R, Liskovich I, Aral S. Reducing Interference Bias in Online Marketplace Experiments Using Cluster Randomization: Evidence from a Pricing Meta-experiment on Airbnb. Management Science. 2024. doi:10.1287/mnsc.2020.01157
BibTeX
@article{Holtz2024,
author = {Holtz, David and Lobel, Felipe and Lobel, Ruben and Liskovich, Inessa and Aral, Sinan},
title = {Reducing Interference Bias in Online Marketplace Experiments Using Cluster Randomization: Evidence from a Pricing Meta-experiment on Airbnb},
journal = {Management Science},
year = {2024},
doi = {10.1287/mnsc.2020.01157}
}
EUVIMED