EUVIMEDEUVIMED entdecken
Crossref

Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning

Pierriccardo Olivieri, Fausto Lasca, Alessandro Gianola, Matteo Papini

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24613-24621

DOI: 10.1609/aaai.v40i29.39645 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Pierriccardo Olivieri und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this work, we propose a novel framework for the logical specification of non-Markovian rewards in Markov Decision Processes (MDPs) with large state spaces. Our approach leverages Linear Temporal Logic Modulo Theories over finite traces (LTLfMT), a more expressive extension of classical temporal logic in which predicates are first-order formulas of arbitrary first-order theories rather than simple Boolean variables. This enhanced expressiveness enables the specification of complex tasks over unstructured and heterogeneous data domains, promoting a unified and reusable framework that eliminates the need for manual predicate encoding. However, the increased expressive power of LTLfMT introduces additional theoretical and computational challenges compared to standard LTLf specifications. We address these challenges from a theoretical standpoint, identifying a fragment of LTLfMT that is tractable but sufficiently expressive for reward specification in an infinite-state-space context. From a practical perspective, we introduce a method based on reward machines and Hindsight Experience Replay (HER) to translate first-order logic specifications and address reward sparsity. We evaluate this approach to a continuous-control setting using Non-Linear Arithmetic Theory, showing that it enables natural specification of complex tasks. Experimental results show how a tailored implementation of HER is fundamental in solving tasks with complex goals.

Zitieren

APA

Olivieri, P., Lasca, F., Gianola, A., & Papini, M. (2026). Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24613-24621. https://doi.org/10.1609/aaai.v40i29.39645

Vancouver

Olivieri P, Lasca F, Gianola A, Papini M. Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24613-24621. doi:10.1609/aaai.v40i29.39645

BibTeX
@article{Olivieri2026,
  author = {Olivieri, Pierriccardo and Lasca, Fausto and Gianola, Alessandro and Papini, Matteo},
  title = {Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {29},
  pages = {24613--24621},
  doi = {10.1609/aaai.v40i29.39645}
}
Kostenlos starten →