Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning
International Journal of Information and Communication Technology Trends · 19.10.2026 · 6 (2) · 368-381
DOI: 10.66525/ijictt144 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Information and Communication Technology Trends, online erschienen am 19.10.2026. Verfasst von Gabriel Hayes und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Marketing teams own large logs of past campaigns but can rarely experiment with new contact policies online. Uplift models estimate the incremental effect of a single contact, yet repeated contacts interact through fatigue, carryover and unsubscription, so one-shot uplift targeting is myopic. We study offline causal reinforcement learning for this setting and propose Uplift-Decomposed Q-learning (UDQ). Under sequential ignorability, the advantage of contacting a customer equals the immediate causal uplift plus a discounted long-term uplift; UDQ estimates the first term with an honest causal forest, learns only the second by temporal-difference updates with shrinkage, and acts on a lower confidence bound built from the forest's standard errors. On the real Hillstrom randomized e-mail experiment (64,000 customers), one-shot targeting policies earn no more than blanket mailing, which motivates the dynamic view. On a 12-period benchmark calibrated on the same experiment, UDQ reaches a net value of 0.180 per customer under uniform logging, 13% above myopic causal-forest targeting, 26% above conservative Q-learning, 75% above a reimplemented causal-forest-plus-DQN framework trained in a learned simulator, and 90% of an oracle agent. Under rule-based logging it ties the best baselines while sending 27% fewer e-mails. Ablations show that the gains come from anchoring the Q-function on the causal-forest uplift and from uncertainty-based pessimism, whereas the learned long-term term must be strongly shrunk and adds little at this data scale.
Zitieren
APA
Hayes, G., & Tan, G. (2026). Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning. International Journal of Information and Communication Technology Trends, 6(2), 368-381. https://doi.org/10.66525/ijictt144
Vancouver
Hayes G, Tan G. Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning. International Journal of Information and Communication Technology Trends. 2026;6(2):368-381. doi:10.66525/ijictt144
BibTeX
@article{Hayes2026,
author = {Hayes, Gabriel and Tan, Grace},
title = {Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning},
journal = {International Journal of Information and Communication Technology Trends},
year = {2026},
volume = {6},
number = {2},
pages = {368--381},
doi = {10.66525/ijictt144}
}
EUVIMED