EUVIMEDEUVIMED entdecken
Crossref

STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models

Yantao Lu, Shiqi Sun, Ning Liu, Bo Jiang, Ying Zhang, Jinchao Chen, Chenglie Du

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24097-24105

DOI: 10.1609/aaai.v40i29.39588 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yantao Lu und 6 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision-and-Language Navigation (VLN) plays a critical role in tasks of embodied AI, particularly in unseen environments following natural language instructions. Recent advancements leverage large language models (LLMs) to improve the accuracy and generalizability of VLN systems by encoding image sequences as dense token representations. However, this tokenization approach incurs substantial computational overhead due to two key inefficiencies: 1) ego-centric camera views often include navigation-irrelevant re- gions (e.g., sky or distant backgrounds), and 2) high-frame-rate image sequences introduce temporal redundancy. To address these challenges, we propose Spatial-Temporal Efficient Visual Token Pruning (STEP-Nav), a unified frame- work that simultaneously prunes redundant visual tokens and fine-tunes VLN models to preserve navigation performance. In particular, STEP-Nav incorporates a distance- and content-aware token evaluation mechanism to remove irrelevant tokens at the spatial level, along with temporal level similarity-based filtering to reduce redundancy across sequential frames. To ensure pruning does not harm task performance, we introduce a distortion-aware fine-tuning strategy that aligns pruned-token representations with their full-token counterparts while maintaining navigation accuracy. Experiments on the R2R and RxR benchmarks using Navid-CE and NavGPT-2 as base models demonstrate that STEP-Nav preserves over 95% of the performance while reducing 66.7% of tokens, outperforming existing token pruning baselines.

Zitieren

APA

Lu, Y., Sun, S., Liu, N., Jiang, B., Zhang, Y., Chen, J., & Du, C. (2026). STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24097-24105. https://doi.org/10.1609/aaai.v40i29.39588

Vancouver

Lu Y, Sun S, Liu N, Jiang B, Zhang Y, Chen J, et al. STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24097-24105. doi:10.1609/aaai.v40i29.39588

BibTeX
@article{Lu2026,
  author = {Lu, Yantao and Sun, Shiqi and Liu, Ning and Jiang, Bo and Zhang, Ying and Chen, Jinchao and Du, Chenglie},
  title = {STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {29},
  pages = {24097--24105},
  doi = {10.1609/aaai.v40i29.39588}
}
Kostenlos starten →