Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning
CAAI Transactions on Intelligence Technology · 2026-04 · 11 (2) · 332-348
DOI: 10.1049/cit2.70109 ↗Kurz gefasst
Fachartikel in der Zeitschrift CAAI Transactions on Intelligence Technology, online erschienen am 13.02.2026. Verfasst von Anjie Zhu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Intrinsic motivation serves as the predominant paradigm of exploration in reinforcement learning. In pursuit of an informative and robust state representation, the behavioural metric groups behaviourally equivalent states together, which share the same single‐step reward and transition distribution. However, due to the presence of uninformative rewards and the dynamic nature of procedurally generated environments, these behavioural metric‐based approaches could limit the effectiveness of the learnt state representations, potentially leading to a representation collapse and an ineffective exploration. Therefore, a more comprehensive and generalisable behavioural metric is needed to overcome the above issues. In this work, we approach the exploration problem from a novel perspective, extending beyond the conventional single‐step assessments to encompass a long‐term consideration of the whole trajectory. Specifically, we propose a novel t rajectory‐level b ehavioural m etric (TBM) that exploits temporal dependencies of the trajectory and captures the underlying sequential information of behaviour patterns. To achieve an effective trajectory representation for exploration, we develop a p ivotal s tate i dentifier (PSI) and a t rajectory r eturn e stimator (TRE) to distinguish the diverse contributions of individual states in the trajectory. Moreover, an auxiliary representation regulariser is developed to promote the diversity and informativeness of the trajectory representation, mitigating the risk of representation mode collapse. Extensive experiments and empirical analysis conducted on procedurally generated environments showcase the superior performance of our proposed framework.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhu, A., Yang, Y., Zhao, G., & Shao, J. (2026). Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning. CAAI Transactions on Intelligence Technology, 11(2), 332-348. https://doi.org/10.1049/cit2.70109
Vancouver
Zhu A, Yang Y, Zhao G, Shao J. Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning. CAAI Transactions on Intelligence Technology. 2026;11(2):332-348. doi:10.1049/cit2.70109
BibTeX
@article{Zhu2026,
author = {Zhu, Anjie and Yang, Yongjun and Zhao, Guangyi and Shao, Jie},
title = {Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning},
journal = {CAAI Transactions on Intelligence Technology},
year = {2026},
volume = {11},
number = {2},
pages = {332--348},
doi = {10.1049/cit2.70109}
}
EUVIMED