EUVIMEDEUVIMED entdecken
Crossref

Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning

Hao Zhang, Hao Wang, Zhen Kan

IEEE Robotics and Automation Letters · 2023-08 · 8 (8) · 4831-4838

DOI: 10.1109/lra.2023.3290511 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Robotics and Automation Letters, 2023-08. Verfasst von Hao Zhang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 31 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automaton based approaches have enabled robots to perform various complex tasks. However, most existing automaton based algorithms highly rely on the manually customized representation of states for the considered task, limiting its applicability in deep reinforcement learning algorithms. To address this issue, by incorporating Transformer into reinforcement learning, we develop a Double-Transformer-guided Temporal Logic framework (T2TL) that exploits the structural feature of Transformer twice, i.e., first encoding the LTL instruction via the Transformer module for efficient understanding of task instructions during the training and then encoding the context variable via the Transformer again for improved task performance. Particularly, the LTL instruction is specified by co-safe LTL. As a semantics-preserving rewriting operation, LTL progression is exploited to decompose the complex task into learnable sub-goals, which not only converts non-Markovian reward decision processes to Markovian ones, but also improves the sampling efficiency by simultaneous learning of multiple sub-tasks. An environment-agnostic LTL pre-training scheme is further incorporated to facilitate the learning of the Transformer module resulting in an improved representation of LTL. The simulation results demonstrate the effectiveness of the T2TL framework.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Zhang, H., Wang, H., & Kan, Z. (2023). Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning. IEEE Robotics and Automation Letters, 8(8), 4831-4838. https://doi.org/10.1109/lra.2023.3290511

Vancouver

Zhang H, Wang H, Kan Z. Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning. IEEE Robotics and Automation Letters. 2023;8(8):4831-4838. doi:10.1109/lra.2023.3290511

BibTeX
@article{Zhang2023,
  author = {Zhang, Hao and Wang, Hao and Kan, Zhen},
  title = {Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning},
  journal = {IEEE Robotics and Automation Letters},
  year = {2023},
  volume = {8},
  number = {8},
  pages = {4831--4838},
  doi = {10.1109/lra.2023.3290511}
}
Kostenlos starten →