EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

VLM-Nav: Mapless UAV navigation using monocular vision driven by vision-language models

Gobinda Chandra Sarker, Akm Azad, Sejuti Rahman, Md Mehedi Hasan iD

PLOS One · 03.04.2026 · 21 (4) · e0345778

DOI: 10.1371/journal.pone.0345778 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 03.04.2026. Verfasst von Gobinda Chandra Sarker und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, University at Albany, Albany, New York, United States of America. Hauptthemen laut MEDLINE: Unmanned Aerial Devices, Vision, Monocular. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deanship of Scientific Research, Imam Mohammed Ibn Saud Islamic University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous vehicles, such as Unmanned Aerial Vehicles (UAVs), have the potential to completely reshape various industries such as parcel delivery, agriculture, surveillance, monitoring, and search-and-rescue missions. Consequently, the demand for safe, cost-effective, and intelligent navigation systems is crucial to ensure reliable performance in complex and dynamic environments. In this study, we propose a novel vision-based UAV navigation method that integrates depthmap estimation with a Vision-Language Model (VLM) for efficient obstacle avoidance and path planning. The system processes RGB images captured by the UAV, transforming them into depth maps using DepthAnything-V2, a powerful zero-shot depth estimator. These depth maps are then analyzed by the VLM, which detects nearby obstacles and plans avoidance maneuvers. We have explored the Gemini-flash and GPT-4o models as VLM in our study. A fully connected network integrates the VLM output with the UAV’s relative heading angle to predict the optimal course of action, enabling the UAV to dynamically navigate complex environments toward its target. The system’s effectiveness is validated through simulations in AirSim using Blocks and the Downtown West environment. The UAV consistently reaches its destination, avoiding obstacles and achieving a near-perfect task completion rate of 0.98. By eliminating the need for costly sensors such as LiDAR and operating without pre-existing maps, our solution provides a cost-efficient, generalizable approach to real-time UAV navigation, especially in unfamiliar or dynamic settings, and highlights emerging trends in autonomous systems research that utilize VLMs.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Unmanned Aerial DevicesVision, MonocularAlgorithmsLarge Language Models

Autor:innen und Institutionen

  • Department of Electrical and Computer Engineering, University at Albany, Albany, New York, United States of America.
  • Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh.
  • Department of Mathematics and Statistics, Faculty of Science, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia.
  • Department of Computer Science, New Uzbekistan University, Tashkent, Uzbekistan.

ORCID-Profile: Md Mehedi Hasan

Förderung

  • Deanship of Scientific Research, Imam Mohammed Ibn Saud Islamic University

Zitieren

APA

Sarker, G. C., Azad, A., Rahman, S., & Hasan, M. M. (2026). VLM-Nav: Mapless UAV navigation using monocular vision driven by vision-language models. PLOS One, 21(4), e0345778. https://doi.org/10.1371/journal.pone.0345778

Vancouver

Sarker GC, Azad A, Rahman S, Hasan MM. VLM-Nav: Mapless UAV navigation using monocular vision driven by vision-language models. PLOS One. 2026;21(4):e0345778. doi:10.1371/journal.pone.0345778

BibTeX
@article{Sarker2026,
  author = {Sarker, Gobinda Chandra and Azad, Akm and Rahman, Sejuti and Hasan, Md Mehedi},
  title = {VLM-Nav: Mapless UAV navigation using monocular vision driven by vision-language models},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {4},
  pages = {e0345778},
  doi = {10.1371/journal.pone.0345778}
}
Kostenlos starten →