Differentially Private Reward Functions in Markov Decision Processes: Policy Synthesis and Tradeoffs
IEEE Open Journal of Control Systems · 2026 · 5 · 243-257
DOI: 10.1109/ojcsys.2026.3687085 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Open Journal of Control Systems, 2026. Verfasst von Alexander Benvenuti und 6 weiteren Autor:innen. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSF, Graduate Research Fellowship, AFRL.
Abstract (Originalsprache)
Policy synthesis in Markov decision processes uses a known reward function to compute a policy that maximizes it. However, onlookers may infer reward functions by observing agents, which can reveal sensitive information. Therefore, in this paper we introduce and compare two methods for privatizing reward functions in policy synthesis for multi-agent Markov decision processes, which generalize Markov decision processes. Reward functions are privatized using differential privacy, a statistical framework for protecting sensitive data. The methods we develop perturb (i) each agent's individual reward function or (ii) the joint reward function shared by all agents. We prove that both of these methods are differentially private and show approach (i) provides better performance. We then develop guidelines for selecting reward functions to achieve high performance while remaining private, and we quantify the increase in computation required to compute policies with privatized rewards. Numerical simulations are performed on several examples, and these examples show that even relatively strong differential privacy ($\epsilon =1.3$) induces as little as a 5% decrease in performance and 0.016% increase in computation time.
Förderung
- NSF
- Graduate Research Fellowship
- AFRL
- ONR
- AFOSR
Zitieren
APA
Benvenuti, A., Hawkins, C., Fallin, B., Chen, B., Bialy, B., Dennis, M., & Hale, M. T. (2026). Differentially Private Reward Functions in Markov Decision Processes: Policy Synthesis and Tradeoffs. IEEE Open Journal of Control Systems, 5, 243-257. https://doi.org/10.1109/ojcsys.2026.3687085
Vancouver
Benvenuti A, Hawkins C, Fallin B, Chen B, Bialy B, Dennis M, et al. Differentially Private Reward Functions in Markov Decision Processes: Policy Synthesis and Tradeoffs. IEEE Open Journal of Control Systems. 2026;5:243-257. doi:10.1109/ojcsys.2026.3687085
BibTeX
@article{Benvenuti2026,
author = {Benvenuti, Alexander and Hawkins, Calvin and Fallin, Brandon and Chen, Bo and Bialy, Brendan and Dennis, Miriam and Hale, Matthew T.},
title = {Differentially Private Reward Functions in Markov Decision Processes: Policy Synthesis and Tradeoffs},
journal = {IEEE Open Journal of Control Systems},
year = {2026},
volume = {5},
pages = {243--257},
doi = {10.1109/ojcsys.2026.3687085}
}
EUVIMED