ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation
Frontiers in Neurorobotics · 12.06.2026 · 20
DOI: 10.3389/fnbot.2026.1860170 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Neurorobotics, online erschienen am 12.06.2026. Verfasst von Xukun Liu und 6 weiteren Autor:innen. Beteiligte Einrichtung: Northwest Institute of Mechanical and Electrical Engineering, Xianyang, Shaanxi, China. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Learning robust and temporally consistent manipulation policies from long-horizon visual observations remains a fundamental challenge in imitation learning. While recent Transformer-based approaches reduce compounding errors via temporally extended action chunks, most methods rely on deterministic or unimodal action representations, limiting their ability to capture the inherent multimodality of expert demonstrations.
Methods
In this work, we propose ST-HONet, a Spatio-Temporal Hierarchical Network that integrates three adaptive optimization mechanisms operating jointly across spatial and temporal dimensions. ST-HONet formulates policy learning as conditional generation of extended action segments using a Transformer-based conditional variational autoencoder to explicitly model multimodal expert behaviors through a structured latent representation. To ensure stable optimization over long temporal horizons, we introduce a unified spatio-temporal training framework combining adaptive data augmentation, progressive latent regularization, and multi-stage optimization strategies.
Results
We evaluate ST-HONet on RoboTwin 2.0, a large-scale benchmark for long-horizon bimanual manipulation under domain randomization with automatically generated expert demonstrations. Across representative manipulation tasks, ST-HONet achieves consistently higher task success rates compared to baseline models, while incurring only minimal additional computational overhead.
Discussion
These results demonstrate that explicitly modeling multimodality via a structured latent space, combined with joint spatio-temporal training mechanisms, significantly improves policy robustness and temporal consistency in long-horizon visuomotor imitation learning. The minimal computational overhead further supports the practical deployability of ST-HONet in real-world systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Northwest Institute of Mechanical and Electrical Engineering, Xianyang, Shaanxi, China.
Zitieren
APA
Liu, X., Xie, F., Xu, K., Liu, Z., Wei, S., Li, G., & Sun, X. (2026). ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation. Frontiers in Neurorobotics, 20. https://doi.org/10.3389/fnbot.2026.1860170
Vancouver
Liu X, Xie F, Xu K, Liu Z, Wei S, Li G, et al. ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation. Frontiers in Neurorobotics. 2026;20. doi:10.3389/fnbot.2026.1860170
BibTeX
@article{Liu2026,
author = {Liu, Xukun and Xie, Fengjuan and Xu, Kai and Liu, Zhenyu and Wei, Shenggang and Li, Guangning and Sun, Xu},
title = {ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation},
journal = {Frontiers in Neurorobotics},
year = {2026},
volume = {20},
doi = {10.3389/fnbot.2026.1860170}
}
EUVIMED