EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-08 · 48 (8) · 9782-9796
DOI: 10.1109/tpami.2026.3682514 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-08. Verfasst von Bingqian Lin und 12 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 73 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shanghai Commitee of Science and Technology, China, Scientific Research Innovation Capability Support Project for Young Faculty.
Abstract (Originalsprache)
Recent studies have revealed the potential of training open-source Large Language Models (LLMs) to unleash LLMs' reasoning ability for enhancing vision-language navigation (VLN) performance, and simultaneously mitigate the domain gap between LLMs' training corpus and the VLN task. However, these approaches predominantly adopt straightforward input-output mapping paradigms, causing the mapping learning difficult and the navigational decisions unexplainable. Chain-of-Thought (CoT) training is a promising way to improve both navigational decision accuracy and interpretability, while the complexity of the navigation task makes the perfect CoT labels unavailable and may lead to overfitting through pure CoT supervised fine-tuning. To address these issues, we propose EvolveNav, a novel sElf-improving embodied reasoning paradigm that realizes adaptable and generalizable navigational reasoning for boosting LLM-based vision-language Navigation. Specifically, EvolveNav involves a two-stage training process: (1) Formalized CoT Supervised Fine-Tuning, where we train the model with curated formalized CoT labels to first activate the model's navigational reasoning capabilities, and simultaneously increase the reasoning speed; (2) Self-Reflective Post-Training, where the model is iteratively trained with its own reasoning outputs as self-enriched CoT labels to enhance the supervision diversity. A self-reflective auxiliary task is also designed to encourage the model to learn correct reasoning patterns by contrasting with wrong ones. Experimental results under both task-specific and cross-task training paradigms demonstrate the consistent superiority of EvolveNav over previous LLM-based VLN approaches on various popular benchmarks, including R2R, REVERIE, CVDN, and SOON. EvolveNav open avenues for exploring effective self-improving reasoning paradigms, enabling building agents capable of self-evolving for promoting LLM-based embodied AI research.
Förderung
- National Natural Science Foundation of China
- Shanghai Commitee of Science and Technology, China
- Scientific Research Innovation Capability Support Project for Young Faculty
- National Postdoctoral Program for Innovative Talents
- China Postdoctoral Science Foundation
- General Embodied AI Center of Sun Yat-sen University
- Shanghai Artificial Intelligence Laboratory through XPLORER PRIZE Grants
Zitieren
APA
Lin, B., Nie, Y., Zai, K. L., Wei, Z., Han, M., Xu, R., Niu, M., Han, J., Zhang, H., Lin, L., Chen, B., Lu, C., & Liang, X. (2026). EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(8), 9782-9796. https://doi.org/10.1109/tpami.2026.3682514
Vancouver
Lin B, Nie Y, Zai KL, Wei Z, Han M, Xu R, et al. EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(8):9782-9796. doi:10.1109/tpami.2026.3682514
BibTeX
@article{Lin2026,
author = {Lin, Bingqian and Nie, Yunshuang and Zai, Khun Loun and Wei, Ziming and Han, Mingfei and Xu, Rongtao and Niu, Minzhe and Han, Jianhua and Zhang, Hanwang and Lin, Liang and Chen, Bokui and Lu, Cewu and Liang, Xiaodan},
title = {EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {8},
pages = {9782--9796},
doi = {10.1109/tpami.2026.3682514}
}
EUVIMED