EUVIMEDEUVIMED entdecken
Crossref

Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning

Xiuxiu Qi, Yu Yang, Jiannong Cao, Luyao Bai, Chongshan Fan, Chengtai Cao, Hongpeng Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24900-24908

DOI: 10.1609/aaai.v40i29.39677 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiuxiu Qi und 6 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Language-Conditioned Manipulation (LCM) facilitates human-robot interaction via Behavioral Cloning (BC), which learns control policies from human demonstrations and serves as a cornerstone of embodied AI. Overcoming compounding errors in sequential action decisions remains a central challenge to improving BC performance. Existing approaches mitigate compounding errors through data augmentation, expressive representation, or temporal abstraction. However, they suffer from physical discontinuities and semantic-physical misalignment, leading to inaccurate action cloning and intermittent execution. In this paper, we present Continuous vision-language-action Co-Learning with Semantic-Physical Alignment (CCoL), a novel BC framework that ensures temporally consistent execution and fine-grained semantic grounding. It generates robust and smooth action execution trajectories through continuous co-learning across vision, language, and proprioceptive inputs (i.e., robot internal states). Meanwhile, we anchor language semantics to visuomotor representations by a bidirectional cross-attention to learn contextual information for action generation, successfully overcoming the problem of semantic-physical misalignment. Extensive experiments show that CCoL achieves an average 8.0% relative improvement across three simulation suites, with up to 19.2% relative gain in human-demonstrated bimanual insertion tasks. Real-world tests on a 7-DoF robot further confirm CCoL’s generalization under unseen and noisy object states.

Zitieren

APA

Qi, X., Yang, Y., Cao, J., Bai, L., Fan, C., Cao, C., & Wang, H. (2026). Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24900-24908. https://doi.org/10.1609/aaai.v40i29.39677

Vancouver

Qi X, Yang Y, Cao J, Bai L, Fan C, Cao C, et al. Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24900-24908. doi:10.1609/aaai.v40i29.39677

BibTeX
@article{Qi2026,
  author = {Qi, Xiuxiu and Yang, Yu and Cao, Jiannong and Bai, Luyao and Fan, Chongshan and Cao, Chengtai and Wang, Hongpeng},
  title = {Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {29},
  pages = {24900--24908},
  doi = {10.1609/aaai.v40i29.39677}
}
Kostenlos starten →