EUVIMEDEUVIMED entdecken
Crossref

A Novel MLLM-Based Approach for Autonomous Driving in Different Weather Conditions

Sonda Fourati, Wael Jaafar, Noura Baccar

Computing&AI Connect · 2025

DOI: 10.69709/caic.2025.177363 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computing&AI Connect, 2025. Verfasst von Sonda Fourati und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous driving (AD) technology promises to revolutionize daily transportation by making it safer, more efficient, and more comfortable. Its role in reducing traffic accidents and improving mobility is vital to the future of intelligent transportation systems. AD systems (ADS) are expected to function reliably across diverse and challenging environments. However, existing solutions often struggle under harsh weather conditions such as foggy, rainy, or stormy circumstances, and mostly rely on unimodal inputs, thus limiting their adaptability and performance. Meanwhile, multimodal large language models (MLLMs) have shown remarkable capabilities in perception, reasoning, and decision-making, yet their application in AD, particularly under extreme environmental conditions, remains largely unexplored. Consequently, this paper proposes MLLM-AD-4o, a novel AD agent that leverages prompt engineering to integrate camera and LiDAR inputs for enhanced perception and control. MLLM-AD-4o dynamically adapts to available sensor modalities and is built upon GPT-4o to ensure contextual reasoning and decision-making. To support realistic evaluation, the agent was developed using the LimSim++ framework, which integrates the SUMO and CARLA driving simulators. Experiments are conducted under harsh conditions, including bad weather, poor visibility, and complex traffic scenarios. The MLLM-AD-4o agent’s robustness and performance are assessed for decision-making, perception, and control. The obtained results demonstrate the agent’s ability to maintain high levels of safety and efficiency, even in extreme conditions, using different perception components (e.g., cameras only, cameras with LiDAR, etc.). Finally, this work provides valuable insights into integrating MLLMs with AD frameworks, paving the way for fully safe ADS.

Zitieren

APA

Fourati, S., Jaafar, W., & Baccar, N. (2025). A Novel MLLM-Based Approach for Autonomous Driving in Different Weather Conditions. Computing&AI Connect. https://doi.org/10.69709/caic.2025.177363

Vancouver

Fourati S, Jaafar W, Baccar N. A Novel MLLM-Based Approach for Autonomous Driving in Different Weather Conditions. Computing&AI Connect. 2025. doi:10.69709/caic.2025.177363

BibTeX
@article{Fourati2025,
  author = {Fourati, Sonda and Jaafar, Wael and Baccar, Noura},
  title = {A Novel MLLM-Based Approach for Autonomous Driving in Different Weather Conditions},
  journal = {Computing&AI Connect},
  year = {2025},
  doi = {10.69709/caic.2025.177363}
}
Kostenlos starten →