EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning

Anjie Zhu, Yongjun Yang, Guangyi Zhao, Jie Shao

CAAI Transactions on Intelligence Technology · 2026-04 · 11 (2) · 332-348

DOI: 10.1049/cit2.70109 ↗

Kurz gefasst

Fachartikel in der Zeitschrift CAAI Transactions on Intelligence Technology, online erschienen am 13.02.2026. Verfasst von Anjie Zhu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Intrinsic motivation serves as the predominant paradigm of exploration in reinforcement learning. In pursuit of an informative and robust state representation, the behavioural metric groups behaviourally equivalent states together, which share the same single‐step reward and transition distribution. However, due to the presence of uninformative rewards and the dynamic nature of procedurally generated environments, these behavioural metric‐based approaches could limit the effectiveness of the learnt state representations, potentially leading to a representation collapse and an ineffective exploration. Therefore, a more comprehensive and generalisable behavioural metric is needed to overcome the above issues. In this work, we approach the exploration problem from a novel perspective, extending beyond the conventional single‐step assessments to encompass a long‐term consideration of the whole trajectory. Specifically, we propose a novel t rajectory‐level b ehavioural m etric (TBM) that exploits temporal dependencies of the trajectory and captures the underlying sequential information of behaviour patterns. To achieve an effective trajectory representation for exploration, we develop a p ivotal s tate i dentifier (PSI) and a t rajectory r eturn e stimator (TRE) to distinguish the diverse contributions of individual states in the trajectory. Moreover, an auxiliary representation regulariser is developed to promote the diversity and informativeness of the trajectory representation, mitigating the risk of representation mode collapse. Extensive experiments and empirical analysis conducted on procedurally generated environments showcase the superior performance of our proposed framework.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Zhu, A., Yang, Y., Zhao, G., & Shao, J. (2026). Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning. CAAI Transactions on Intelligence Technology, 11(2), 332-348. https://doi.org/10.1049/cit2.70109

Vancouver

Zhu A, Yang Y, Zhao G, Shao J. Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning. CAAI Transactions on Intelligence Technology. 2026;11(2):332-348. doi:10.1049/cit2.70109

BibTeX
@article{Zhu2026,
  author = {Zhu, Anjie and Yang, Yongjun and Zhao, Guangyi and Shao, Jie},
  title = {Temporal Dependency‐Aware Trajectory‐Level Behavioural Metric for Exploration in Reinforcement Learning},
  journal = {CAAI Transactions on Intelligence Technology},
  year = {2026},
  volume = {11},
  number = {2},
  pages = {332--348},
  doi = {10.1049/cit2.70109}
}
Kostenlos starten →