RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30682-30689
DOI: 10.1609/aaai.v40i36.40324 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yue Fang und 5 weiteren Autor:innen.
Abstract (Originalsprache)
Signal Temporal Logic (STL) is a powerful formal language for specifying real-time specifications of Cyber-Physical Systems (CPS). Transforming specifications written in natural language into STL formulas automatically has attracted increasing attention. Existing rule-based methods depend heavily on rigid pattern matching and domain-specific knowledge, limiting their generalizability and scalability. Recently, Supervised Fine-Tuning (SFT) of large language models (LLMs) has been successfully applied to transform natural language into STL. However, the lack of fine-grained supervision on atomic proposition correctness, semantic fidelity, and formula readability often leads SFT-based methods to produce formulas misaligned with the intended meaning. To address these issues, we propose RESTL, a reinforcement learning (RL)-based framework for the transformation from natural language to STL. RESTL introduces multiple independently trained reward models that provide fine-grained, multi-faceted feedback from four perspectives, i.e., atomic proposition consistency, semantic alignment, formula succinctness, and symbol matching. These reward models are trained with a curriculum learning strategy to improve their feedback accuracy, and their outputs are aggregated into a unified signal that guides the optimization of the STL generator via Proximal Policy Optimization (PPO). Experimental results demonstrate that RESTL significantly outperforms state-of-the-art methods in both automatic metrics and human evaluations.
Zitieren
APA
Fang, Y., Jin, Z., An, J., Chen, H., Chen, X., & Zhan, N. (2026). RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30682-30689. https://doi.org/10.1609/aaai.v40i36.40324
Vancouver
Fang Y, Jin Z, An J, Chen H, Chen X, Zhan N. RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30682-30689. doi:10.1609/aaai.v40i36.40324
BibTeX
@article{Fang2026,
author = {Fang, Yue and Jin, Zhi and An, Jie and Chen, Hongshen and Chen, Xiaohong and Zhan, Naijun},
title = {RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {36},
pages = {30682--30689},
doi = {10.1609/aaai.v40i36.40324}
}
EUVIMED