EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios

Mohamed Amine Ferrag, Abderrahmane Lakas, Merouane Debbah

IEEE Open Journal of Vehicular Technology · 2026 · 7 · 2542-2558

DOI: 10.1109/ojvt.2026.3721777 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Open Journal of Vehicular Technology, 2026. Verfasst von Mohamed Amine Ferrag und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: United Arab Emirates University Research Start-up Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous aerial systems increasingly rely on large language models (LLMs) for mission planning, perception, and decision-making; yet, the lack of standardized, physically grounded benchmarks limits systematic evaluation of their reasoning capabilities. To address this gap, we introduce UAVBench, an open benchmark dataset comprising 50,000 validated UAV flight scenarios generated through taxonomy-guided LLM prompting and multi-stage safety validation. Each scenario is encoded in a structured JSON schema encompassing mission objectives, vehicle configuration, environmental conditions, and quantitative risk labels, providing a unified representation of UAV operations across diverse domains. Building on this foundation, we present UAVBench_MCQ, a reasoning-oriented extension containing 50,000 multiple-choice questions spanning ten cognitive and ethical reasoning styles—from aerodynamics and navigation to multi-agent coordination and hybrid integrated reasoning. This framework enables interpretable, machine-checkable assessment of UAV-specific cognition under realistic operational contexts. We evaluate 32 state-of-the-art LLMs, including GPT-5, ChatGPT 4o, Gemini 2.5 Flash, DeepSeek V3, Qwen3 235B, and ERNIE 4.5 300B, and find strong performance in perception and policy reasoning but persistent challenges in ethics-aware and resource-constrained decision-making. UAVBench establishes a reproducible, physically grounded foundation for benchmarking agentic AI in autonomous aerial systems and advancing next-generation UAV reasoning intelligence.

Förderung

  • United Arab Emirates University Research Start-up Program

Zitieren

APA

Ferrag, M. A., Lakas, A., & Debbah, M. (2026). UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios. IEEE Open Journal of Vehicular Technology, 7, 2542-2558. https://doi.org/10.1109/ojvt.2026.3721777

Vancouver

Ferrag MA, Lakas A, Debbah M. UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios. IEEE Open Journal of Vehicular Technology. 2026;7:2542-2558. doi:10.1109/ojvt.2026.3721777

BibTeX
@article{Ferrag2026,
  author = {Ferrag, Mohamed Amine and Lakas, Abderrahmane and Debbah, Merouane},
  title = {UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios},
  journal = {IEEE Open Journal of Vehicular Technology},
  year = {2026},
  volume = {7},
  pages = {2542--2558},
  doi = {10.1109/ojvt.2026.3721777}
}
Kostenlos starten →