TNCOA: Efficient Exploration via Observation‐Action Constraint on Trajectory‐Based Intrinsic Reward
CAAI Transactions on Intelligence Technology · 2026-04 · 11 (2) · 411-427
DOI: 10.1049/cit2.70100 ↗Kurz gefasst
Fachartikel in der Zeitschrift CAAI Transactions on Intelligence Technology, online erschienen am 28.01.2026. Verfasst von Jingxiang Ma und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Efficient exploration is critical in handling sparse rewards and partial observability in deep reinforcement learning. However, most existing intrinsic reward methods based on novelty rely on single‐step observations or Euclidean distances. These approaches struggle to capture trajectory‐level novelty and often perform poorly in partially observable settings. Moreover, they typically ignore the role of actions in driving observation changes, as not all actions lead to meaningful state transitions. To overcome these limitations, we propose a trajectory‐level novelty measure that estimates the novelty of a state by comparing current observations with past ones along the trajectory. To focus on meaningful exploration, we incorporate the mutual information between actions and trajectory novelty to filter out random fluctuations and retain only novelty caused by the agent's actions. Additionally, we introduce a first‐visit constraint on observation–action pairs, rewarding only interactions that result in state transitions to enhance exploration efficiency. We conducted experiments in the MiniGrid‐ObstructedMaze environment characterised by complex object interactions and sparse rewards. Results demonstrate that our method achieves state‐of‐the‐art performance in convergence speed and average returns. Furthermore, it shows strong generalisation on high‐dimensional Atari benchmarks and demonstrates robust performance in more challenging MiniGrid variants. Implementation code is available at: https://github.com/MurrayMa0816/TNCOA .
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Ma, J., Ma, H., & Zhang, Y. (2026). TNCOA: Efficient Exploration via Observation‐Action Constraint on Trajectory‐Based Intrinsic Reward. CAAI Transactions on Intelligence Technology, 11(2), 411-427. https://doi.org/10.1049/cit2.70100
Vancouver
Ma J, Ma H, Zhang Y. TNCOA: Efficient Exploration via Observation‐Action Constraint on Trajectory‐Based Intrinsic Reward. CAAI Transactions on Intelligence Technology. 2026;11(2):411-427. doi:10.1049/cit2.70100
BibTeX
@article{Ma2026,
author = {Ma, Jingxiang and Ma, Hongbin and Zhang, Youzhi},
title = {TNCOA: Efficient Exploration via Observation‐Action Constraint on Trajectory‐Based Intrinsic Reward},
journal = {CAAI Transactions on Intelligence Technology},
year = {2026},
volume = {11},
number = {2},
pages = {411--427},
doi = {10.1049/cit2.70100}
}
EUVIMED