EUVIMEDEUVIMED entdecken
Crossref

Conformalized proximal policy optimization: statistical uncertainty quantification for principled exploration in reinforcement learning

Bailing Zhang, Genlang Chen, Weiguo Feng

Robot Learning · 28.04.2026 · 1-16

DOI: 10.55092/rl20260011 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Robot Learning, online erschienen am 28.04.2026. Verfasst von Bailing Zhang und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

: Reinforcement learning (RL) excels in diverse domains, yet its deployment in safety-critical applications is hindered by the lack of principled uncertainty quantification. Existing methods either lack formal statistical guarantees or incur significant computational costs. We propose Conformalized Proximal Policy Optimization (CP-PPO), integrating conformal prediction into the PPO framework to provide finite-sample, distribution-free prediction intervals. CP-PPO employs a sliding window conformal calibration to maintain approximate statistical validity despite non-stationary policy optimization, and leverages value function uncertainty to drive adaptive entropy regularization for principled exploration. We evaluate CP-PPO on five diverse Gymnasium environments spanning discrete and continuous control. CP-PPO achieves a 63% performance improvement in Pendulum-v1 while maintaining empirical coverage that precisely matches the theoretical target (90.0% for α = 0 . 1) across all environments. Comprehensive ablation studies demonstrate that both conformal calibration and adaptive entropy contribute to performance gains, and that coverage guarantees hold reliably across a wide range of hyperparameters ( α ∈ [ 0 . 01 , 0 . 3 ] , N cal ∈ [ 100 , 2000 ] ). CP-PPO establishes conformal prediction as a promising framework for online RL uncertainty quantification, offering empirical validation of finite-sample coverage properties in non-stationary sequential decision-making.

Zitieren

APA

Zhang, B., Chen, G., & Feng, W. (2026). Conformalized proximal policy optimization: statistical uncertainty quantification for principled exploration in reinforcement learning. Robot Learning, 1-16. https://doi.org/10.55092/rl20260011

Vancouver

Zhang B, Chen G, Feng W. Conformalized proximal policy optimization: statistical uncertainty quantification for principled exploration in reinforcement learning. Robot Learning. 2026:1-16. doi:10.55092/rl20260011

BibTeX
@article{Zhang2026,
  author = {Zhang, Bailing and Chen, Genlang and Feng, Weiguo},
  title = {Conformalized proximal policy optimization: statistical uncertainty quantification for principled exploration in reinforcement learning},
  journal = {Robot Learning},
  year = {2026},
  pages = {1--16},
  doi = {10.55092/rl20260011}
}
Kostenlos starten →