Robust Reward Machine Learning and Execution in Noisy Environments
Journal of Artificial Intelligence Research · 22.10.2026 · 87
DOI: 10.1613/jair.1.22539 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence Research, online erschienen am 22.10.2026. Verfasst von Roko Parac und 5 weiteren Autor:innen.
Abstract (Originalsprache)
Background
Reward machines (RMs) represent non-Markovian reward functions as finite-state machines over high-level propositional events. Traditionally, these events are generated by a deterministic labelling function that maps environment states to proposition sets. However, real-world labelling functions are inherently noisy.
Objectives
Design and implement a method that learns RMs in the presence of noisy labelling functions and exploits them for policy learning in both discrete and continuous environments.
Methods
We propose Prob-IRM, a method that learns RMs from noisy proposition histories using inductive logic programming (ILP) and exploits them for policy learning. Noisy proposition histories are represented as training examples for ILP, which learns an RM that maximises coverage of low-uncertainty histories. RM learning is interleaved with policy optimisation. Prob-IRM uses a belief over RM states to determine when to relearn the RM and to provide additional guiding rewards.
Results
Prob-IRM is evaluated on environments with discrete and continuous state spaces under varying levels of labelling noise. Prob-IRM performs comparably to baselines using handcrafted RMs and outperforms approaches that use recurrent neural networks to encode history, despite operating with noisy labels.
Conclusions
Prob-IRM represents a step towards deploying reward machines in noisy, real-world environments.
Zitieren
APA
Parac, R., Nodari, L., Furelos-Blanco, D., Ardon, L., Cerutti, F., & Russo, A. (2026). Robust Reward Machine Learning and Execution in Noisy Environments. Journal of Artificial Intelligence Research, 87. https://doi.org/10.1613/jair.1.22539
Vancouver
Parac R, Nodari L, Furelos-Blanco D, Ardon L, Cerutti F, Russo A. Robust Reward Machine Learning and Execution in Noisy Environments. Journal of Artificial Intelligence Research. 2026;87. doi:10.1613/jair.1.22539
BibTeX
@article{Parac2026,
author = {Parac, Roko and Nodari, Lorenzo and Furelos-Blanco, Daniel and Ardon, Leo and Cerutti, Federico and Russo, Alessandra},
title = {Robust Reward Machine Learning and Execution in Noisy Environments},
journal = {Journal of Artificial Intelligence Research},
year = {2026},
volume = {87},
doi = {10.1613/jair.1.22539}
}
EUVIMED