EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ESceme: Vision-and-Language Navigation with Episodic Scene Memory

Qi Zheng, Daqing Liu, Chaoyue Wang, Jing Zhang, Dadong Wang, Dacheng Tao

International Journal of Computer Vision · 2025-01 · 133 (1) · 254-274

DOI: 10.1007/s11263-024-02159-8 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 26.07.2024. Verfasst von Qi Zheng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of Sydney.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision-and-language navigation (VLN) simulates a visual agent that follows natural-language navigation instructions in real-world scenes. Existing approaches have made enormous progress in navigation in new environments, such as beam search, pre-exploration, and dynamic or hierarchical history encoding. To balance generalization and efficiency, we resort to memorizing visited scenarios apart from the ongoing route while navigating. In this work, we introduce a mechanism of Episodic Scene memory (ESceme) for VLN that wakes an agent’s memories of past visits when it enters the current scene. The episodic scene memory allows the agent to envision a bigger picture of the next prediction. This way, the agent learns to utilize dynamically updated information instead of merely adapting to the current observations. We provide a simple yet effective implementation of ESceme by enhancing the accessible views at each location and progressively completing the memory while navigating. We verify the superiority of ESceme on short-horizon (R2R), long-horizon (R4R), and vision-and-dialog (CVDN) VLN tasks. Our ESceme also wins first place on the CVDN leaderboard. Code is available: https://github.com/qizhust/esceme .

Förderung

  • University of Sydney

Zitieren

APA

Zheng, Q., Liu, D., Wang, C., Zhang, J., Wang, D., & Tao, D. (2025). ESceme: Vision-and-Language Navigation with Episodic Scene Memory. International Journal of Computer Vision, 133(1), 254-274. https://doi.org/10.1007/s11263-024-02159-8

Vancouver

Zheng Q, Liu D, Wang C, Zhang J, Wang D, Tao D. ESceme: Vision-and-Language Navigation with Episodic Scene Memory. International Journal of Computer Vision. 2025;133(1):254-274. doi:10.1007/s11263-024-02159-8

BibTeX
@article{Zheng2025,
  author = {Zheng, Qi and Liu, Daqing and Wang, Chaoyue and Zhang, Jing and Wang, Dadong and Tao, Dacheng},
  title = {ESceme: Vision-and-Language Navigation with Episodic Scene Memory},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {1},
  pages = {254--274},
  doi = {10.1007/s11263-024-02159-8}
}
Kostenlos starten →