EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning

Gabriel Hayes, Grace Tan

International Journal of Information and Communication Technology Trends · 19.10.2026 · 6 (2) · 368-381

DOI: 10.66525/ijictt144 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Information and Communication Technology Trends, online erschienen am 19.10.2026. Verfasst von Gabriel Hayes und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Marketing teams own large logs of past campaigns but can rarely experiment with new contact policies online. Uplift models estimate the incremental effect of a single contact, yet repeated contacts interact through fatigue, carryover and unsubscription, so one-shot uplift targeting is myopic. We study offline causal reinforcement learning for this setting and propose Uplift-Decomposed Q-learning (UDQ). Under sequential ignorability, the advantage of contacting a customer equals the immediate causal uplift plus a discounted long-term uplift; UDQ estimates the first term with an honest causal forest, learns only the second by temporal-difference updates with shrinkage, and acts on a lower confidence bound built from the forest's standard errors. On the real Hillstrom randomized e-mail experiment (64,000 customers), one-shot targeting policies earn no more than blanket mailing, which motivates the dynamic view. On a 12-period benchmark calibrated on the same experiment, UDQ reaches a net value of 0.180 per customer under uniform logging, 13% above myopic causal-forest targeting, 26% above conservative Q-learning, 75% above a reimplemented causal-forest-plus-DQN framework trained in a learned simulator, and 90% of an oracle agent. Under rule-based logging it ties the best baselines while sending 27% fewer e-mails. Ablations show that the gains come from anchoring the Q-function on the causal-forest uplift and from uncertainty-based pessimism, whereas the learned long-term term must be strongly shrunk and adds little at this data scale.

Zitieren

APA

Hayes, G., & Tan, G. (2026). Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning. International Journal of Information and Communication Technology Trends, 6(2), 368-381. https://doi.org/10.66525/ijictt144

Vancouver

Hayes G, Tan G. Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning. International Journal of Information and Communication Technology Trends. 2026;6(2):368-381. doi:10.66525/ijictt144

BibTeX
@article{Hayes2026,
  author = {Hayes, Gabriel and Tan, Grace},
  title = {Offline Causal Reinforcement Learning for Dynamic Marketing Uplift Optimization with Uplift-Decomposed Q-Learning},
  journal = {International Journal of Information and Communication Technology Trends},
  year = {2026},
  volume = {6},
  number = {2},
  pages = {368--381},
  doi = {10.66525/ijictt144}
}
Kostenlos starten →