Causal Temporal Reasoning for Markov Decision Processes
Research Directions: Cyber-Physical Systems · 2025
DOI: 10.1017/cbp.2025.2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Directions: Cyber-Physical Systems, 2025. Verfasst von Milad Kazemi und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
We present PCFTL (Probabilistic CounterFactual Temporal Logic) , a new probabilistic temporal logic for the verification of Markov Decision Processes (MDP). PCFTL introduces operators for causal inference, allowing us to express interventional and counterfactual queries. Given a path formula φ, an interventional property is concerned with the satisfaction probability of φ if we apply a particular change I to the MDP (e.g., switching to a different policy); a counterfactual formula allows us to compute, given an observed MDP path τ, what the outcome of φ would have been had we applied I in the past and under the same random factors that led to observing τ. Our approach represents a departure from existing probabilistic temporal logics that do not support such counterfactual reasoning. From a syntactic viewpoint, we introduce a counterfactual operator that subsumes both interventional and counterfactual probabilities as well as the traditional probabilistic operator. This makes our logic strictly more expressive than PCTL⋆. The semantics of PCFTL rely on a structural causal model translation of the MDP, which provides a representation amenable to counterfactual inference. We evaluate PCFTL in the context of safe reinforcement learning using a benchmark of grid-world models.
Zitieren
APA
Kazemi, M., Lally, J., & Paoletti, N. (2025). Causal Temporal Reasoning for Markov Decision Processes. Research Directions: Cyber-Physical Systems. https://doi.org/10.1017/cbp.2025.2
Vancouver
Kazemi M, Lally J, Paoletti N. Causal Temporal Reasoning for Markov Decision Processes. Research Directions: Cyber-Physical Systems. 2025. doi:10.1017/cbp.2025.2
BibTeX
@article{Kazemi2025,
author = {Kazemi, Milad and Lally, Jessica and Paoletti, Nicola},
title = {Causal Temporal Reasoning for Markov Decision Processes},
journal = {Research Directions: Cyber-Physical Systems},
year = {2025},
doi = {10.1017/cbp.2025.2}
}
EUVIMED