EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation

Xukun Liu, Fengjuan Xie, Kai Xu, Zhenyu Liu, Shenggang Wei, Guangning Li, Xu Sun

Frontiers in Neurorobotics · 12.06.2026 · 20

DOI: 10.3389/fnbot.2026.1860170 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Neurorobotics, online erschienen am 12.06.2026. Verfasst von Xukun Liu und 6 weiteren Autor:innen. Beteiligte Einrichtung: Northwest Institute of Mechanical and Electrical Engineering, Xianyang, Shaanxi, China. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Learning robust and temporally consistent manipulation policies from long-horizon visual observations remains a fundamental challenge in imitation learning. While recent Transformer-based approaches reduce compounding errors via temporally extended action chunks, most methods rely on deterministic or unimodal action representations, limiting their ability to capture the inherent multimodality of expert demonstrations.

Methods

In this work, we propose ST-HONet, a Spatio-Temporal Hierarchical Network that integrates three adaptive optimization mechanisms operating jointly across spatial and temporal dimensions. ST-HONet formulates policy learning as conditional generation of extended action segments using a Transformer-based conditional variational autoencoder to explicitly model multimodal expert behaviors through a structured latent representation. To ensure stable optimization over long temporal horizons, we introduce a unified spatio-temporal training framework combining adaptive data augmentation, progressive latent regularization, and multi-stage optimization strategies.

Results

We evaluate ST-HONet on RoboTwin 2.0, a large-scale benchmark for long-horizon bimanual manipulation under domain randomization with automatically generated expert demonstrations. Across representative manipulation tasks, ST-HONet achieves consistently higher task success rates compared to baseline models, while incurring only minimal additional computational overhead.

Discussion

These results demonstrate that explicitly modeling multimodality via a structured latent space, combined with joint spatio-temporal training mechanisms, significantly improves policy robustness and temporal consistency in long-horizon visuomotor imitation learning. The minimal computational overhead further supports the practical deployability of ST-HONet in real-world systems.

Schlagwörter

Stichwörter der Autor:innen

bimanual manipulationlong-horizon visuomotor imitation learningrobot policy learningspatio-temporally hierarchical optimizationvariational action transformer

Autor:innen und Institutionen

  • Northwest Institute of Mechanical and Electrical Engineering, Xianyang, Shaanxi, China.

Zitieren

APA

Liu, X., Xie, F., Xu, K., Liu, Z., Wei, S., Li, G., & Sun, X. (2026). ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation. Frontiers in Neurorobotics, 20. https://doi.org/10.3389/fnbot.2026.1860170

Vancouver

Liu X, Xie F, Xu K, Liu Z, Wei S, Li G, et al. ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation. Frontiers in Neurorobotics. 2026;20. doi:10.3389/fnbot.2026.1860170

BibTeX
@article{Liu2026,
  author = {Liu, Xukun and Xie, Fengjuan and Xu, Kai and Liu, Zhenyu and Wei, Shenggang and Li, Guangning and Sun, Xu},
  title = {ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation},
  journal = {Frontiers in Neurorobotics},
  year = {2026},
  volume = {20},
  doi = {10.3389/fnbot.2026.1860170}
}
Kostenlos starten →