From Detection to Action: A Multimodal AI Framework for Traffic Incident Response
Drones · 09.12.2024 · 8 (12) · 741
DOI: 10.3390/drones8120741 ↗Kurz gefasst
Fachartikel in der Zeitschrift Drones, online erschienen am 09.12.2024. Verfasst von Afaq Ahmed und 4 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Education (MCE).
Abstract (Originalsprache)
With the rising incidence of traffic accidents and growing environmental concerns, the demand for advanced systems to ensure traffic and environmental safety has become increasingly urgent. This paper introduces an automated highway safety management framework that integrates computer vision and natural language processing for real-time monitoring, analysis, and reporting of traffic incidents. The system not only identifies accidents but also aids in coordinating emergency responses, such as dispatching ambulances, fire services, and police, while simultaneously managing traffic flow. The approach begins with the creation of a diverse highway accident dataset, combining public datasets with drone and CCTV footage. YOLOv11s is retrained on this dataset to enable real-time detection of critical traffic elements and anomalies, such as collisions and fires. A vision–language model (VLM), Moondream2, is employed to generate detailed scene descriptions, which are further refined by a large language model (LLM), GPT 4-Turbo, to produce concise incident reports and actionable suggestions. These reports are automatically sent to relevant authorities, ensuring prompt and effective response. The system’s effectiveness is validated through the analysis of diverse accident videos and zero-shot simulation testing within the Webots environment. The results highlight the potential of combining drone and CCTV imagery with AI-driven methodologies to improve traffic management and enhance public safety. Future work will include refining detection models, expanding dataset diversity, and deploying the framework in real-world scenarios using live drone and CCTV feeds. This study lays the groundwork for scalable and reliable solutions to address critical traffic safety challenges.
Förderung
- Ministry of Education (MCE)
Zitieren
APA
Ahmed, A., Farhan, M., Eesaar, H., Chong, K. T., & Tayara, H. (2024). From Detection to Action: A Multimodal AI Framework for Traffic Incident Response. Drones, 8(12), 741. https://doi.org/10.3390/drones8120741
Vancouver
Ahmed A, Farhan M, Eesaar H, Chong KT, Tayara H. From Detection to Action: A Multimodal AI Framework for Traffic Incident Response. Drones. 2024;8(12):741. doi:10.3390/drones8120741
BibTeX
@article{Ahmed2024,
author = {Ahmed, Afaq and Farhan, Muhammad and Eesaar, Hassan and Chong, Kil To and Tayara, Hilal},
title = {From Detection to Action: A Multimodal AI Framework for Traffic Incident Response},
journal = {Drones},
year = {2024},
volume = {8},
number = {12},
pages = {741},
doi = {10.3390/drones8120741}
}
EUVIMED