Safe Reinforcement Learning under Regularized Probabilistic Counterexample Guidance
ACM Transactions on Autonomous and Adaptive Systems · 22.07.2026
DOI: 10.1145/3831680 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Autonomous and Adaptive Systems, online erschienen am 22.07.2026. Verfasst von Xiaotong Ji und 1 weiteren Autor:innen. Die Arbeit nennt 81 Literaturangaben.
Abstract (Originalsprache)
Safe exploration in reinforcement learning remains a critical challenge for safety-critical autonomous systems, where the typical trial-and-error learning process can lead to hazardous outcomes. While several existing approaches incorporate kinematic models or external knowledge to limit the exploration of unsafe behaviors, their effectiveness is significantly weakened in the presence of incomplete or sparse knowledge. This paper introduces a counterexample-guided reinforcement learning method that navigates safe exploration in autonomous systems without prior knowledge, even when safety and optimality conflict. Our method geometrically abstracts discrete and continuous state-space systems into compact, PAC-learnable models that capture safety-relevant information. We then generate probabilistic counterexamples of the safety requirement to regulate online exploration toward minimizing safety violations, relying on minimal offline counterexample-guided simulations. We further propose a novel belief-based regularization method to address the distributional shift between online and offline learning and to balance optimization and safety, ensuring conservative behavior with theoretical guarantees. Our evaluations demonstrate the effectiveness of the method in significantly reducing safety violations without compromising cumulative rewards when benchmarked against other Q-learning or actor-critic methods with unconstrained or safety-constrained exploration.
Zitieren
APA
Ji, X., & Filieri, A. (2026). Safe Reinforcement Learning under Regularized Probabilistic Counterexample Guidance. ACM Transactions on Autonomous and Adaptive Systems. https://doi.org/10.1145/3831680
Vancouver
Ji X, Filieri A. Safe Reinforcement Learning under Regularized Probabilistic Counterexample Guidance. ACM Transactions on Autonomous and Adaptive Systems. 2026. doi:10.1145/3831680
BibTeX
@article{Ji2026,
author = {Ji, Xiaotong and Filieri, Antonio},
title = {Safe Reinforcement Learning under Regularized Probabilistic Counterexample Guidance},
journal = {ACM Transactions on Autonomous and Adaptive Systems},
year = {2026},
doi = {10.1145/3831680}
}
EUVIMED