EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

From Detection to Action: A Multimodal AI Framework for Traffic Incident Response

Afaq Ahmed, Muhammad Farhan, Hassan Eesaar, Kil To Chong, Hilal Tayara

Drones · 09.12.2024 · 8 (12) · 741

DOI: 10.3390/drones8120741 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Drones, online erschienen am 09.12.2024. Verfasst von Afaq Ahmed und 4 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Education (MCE).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the rising incidence of traffic accidents and growing environmental concerns, the demand for advanced systems to ensure traffic and environmental safety has become increasingly urgent. This paper introduces an automated highway safety management framework that integrates computer vision and natural language processing for real-time monitoring, analysis, and reporting of traffic incidents. The system not only identifies accidents but also aids in coordinating emergency responses, such as dispatching ambulances, fire services, and police, while simultaneously managing traffic flow. The approach begins with the creation of a diverse highway accident dataset, combining public datasets with drone and CCTV footage. YOLOv11s is retrained on this dataset to enable real-time detection of critical traffic elements and anomalies, such as collisions and fires. A vision–language model (VLM), Moondream2, is employed to generate detailed scene descriptions, which are further refined by a large language model (LLM), GPT 4-Turbo, to produce concise incident reports and actionable suggestions. These reports are automatically sent to relevant authorities, ensuring prompt and effective response. The system’s effectiveness is validated through the analysis of diverse accident videos and zero-shot simulation testing within the Webots environment. The results highlight the potential of combining drone and CCTV imagery with AI-driven methodologies to improve traffic management and enhance public safety. Future work will include refining detection models, expanding dataset diversity, and deploying the framework in real-world scenarios using live drone and CCTV feeds. This study lays the groundwork for scalable and reliable solutions to address critical traffic safety challenges.

Förderung

  • Ministry of Education (MCE)

Zitieren

APA

Ahmed, A., Farhan, M., Eesaar, H., Chong, K. T., & Tayara, H. (2024). From Detection to Action: A Multimodal AI Framework for Traffic Incident Response. Drones, 8(12), 741. https://doi.org/10.3390/drones8120741

Vancouver

Ahmed A, Farhan M, Eesaar H, Chong KT, Tayara H. From Detection to Action: A Multimodal AI Framework for Traffic Incident Response. Drones. 2024;8(12):741. doi:10.3390/drones8120741

BibTeX
@article{Ahmed2024,
  author = {Ahmed, Afaq and Farhan, Muhammad and Eesaar, Hassan and Chong, Kil To and Tayara, Hilal},
  title = {From Detection to Action: A Multimodal AI Framework for Traffic Incident Response},
  journal = {Drones},
  year = {2024},
  volume = {8},
  number = {12},
  pages = {741},
  doi = {10.3390/drones8120741}
}
Kostenlos starten →