Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24900-24908
DOI: 10.1609/aaai.v40i29.39677 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiuxiu Qi und 6 weiteren Autor:innen.
Abstract (Originalsprache)
Language-Conditioned Manipulation (LCM) facilitates human-robot interaction via Behavioral Cloning (BC), which learns control policies from human demonstrations and serves as a cornerstone of embodied AI. Overcoming compounding errors in sequential action decisions remains a central challenge to improving BC performance. Existing approaches mitigate compounding errors through data augmentation, expressive representation, or temporal abstraction. However, they suffer from physical discontinuities and semantic-physical misalignment, leading to inaccurate action cloning and intermittent execution. In this paper, we present Continuous vision-language-action Co-Learning with Semantic-Physical Alignment (CCoL), a novel BC framework that ensures temporally consistent execution and fine-grained semantic grounding. It generates robust and smooth action execution trajectories through continuous co-learning across vision, language, and proprioceptive inputs (i.e., robot internal states). Meanwhile, we anchor language semantics to visuomotor representations by a bidirectional cross-attention to learn contextual information for action generation, successfully overcoming the problem of semantic-physical misalignment. Extensive experiments show that CCoL achieves an average 8.0% relative improvement across three simulation suites, with up to 19.2% relative gain in human-demonstrated bimanual insertion tasks. Real-world tests on a 7-DoF robot further confirm CCoL’s generalization under unseen and noisy object states.
Zitieren
APA
Qi, X., Yang, Y., Cao, J., Bai, L., Fan, C., Cao, C., & Wang, H. (2026). Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24900-24908. https://doi.org/10.1609/aaai.v40i29.39677
Vancouver
Qi X, Yang Y, Cao J, Bai L, Fan C, Cao C, et al. Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24900-24908. doi:10.1609/aaai.v40i29.39677
BibTeX
@article{Qi2026,
author = {Qi, Xiuxiu and Yang, Yu and Cao, Jiannong and Bai, Luyao and Fan, Chongshan and Cao, Chengtai and Wang, Hongpeng},
title = {Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {29},
pages = {24900--24908},
doi = {10.1609/aaai.v40i29.39677}
}
EUVIMED