EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Transformer-Based Multimodal Object Detection System for Real-World Applications

Sunnia Ikram, Imran Sarwar Bajwa, Amna Ikram, M. Abdullah-Al-Wadud, Haleema Pk

IEEE Access · 2025 · 13 · 29162-29176

DOI: 10.1109/access.2025.3539569 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Sunnia Ikram und 4 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Saud University, Riyadh, Saudi Arabia,.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Obstacle detection is a critical task for visually impaired individuals to ensure safe navigation and hazard avoidance. This study presents FusionSight, an innovative multimodal fusion model that integrates radar and image data to address challenges in real-time object classification for dynamic environments. The system leverages an Arduino Uni microcontroller for data acquisition and transmission, enabling seamless communication between radar and image datasets and the cloud environment. For image data, the Vision Transformer (ViT) was employed to extract high-level features, capturing fine details and long-range dependencies essential for accurate object recognition. Concurrently. Radar data was processed using a Convolutional Neural Network (CNN) to extract spatial and temporal features such as distance, speed and velocity critically for understanding object dynamics. To unify these diverse modalities, a Feature Fusion Multimodal Transformer (FFMA) was utilized, facilitating the integration of complementary features into a comprehensive representation. This fusion mechanism enables the model to effectively handle challenges such as occlusion, overlapping objects and varying lighting conditions. The unified features were classified into four categories “close, far, moving and fast-moving” using a Feed-Forward Neural Network (FFN). The classification results were then converted into actionable audible feedback, providing real-time navigation assistance to visually impaired users. The FusionSight model set a benchmark in multimodal data fusion, achieving an impressive classification accuracy of 99% when using static dataset and 98% accuracy with real-time dataset. This study demonstrates the practical implementation navigation for visually impaired individuals and other use cases involving dynamic and complex environments.

Förderung

  • King Saud University, Riyadh, Saudi Arabia,

Zitieren

APA

Ikram, S., Bajwa, I. S., Ikram, A., Abdullah-Al-Wadud, M., & Pk, H. (2025). A Transformer-Based Multimodal Object Detection System for Real-World Applications. IEEE Access, 13, 29162-29176. https://doi.org/10.1109/access.2025.3539569

Vancouver

Ikram S, Bajwa IS, Ikram A, Abdullah-Al-Wadud M, Pk H. A Transformer-Based Multimodal Object Detection System for Real-World Applications. IEEE Access. 2025;13:29162-29176. doi:10.1109/access.2025.3539569

BibTeX
@article{Ikram2025,
  author = {Ikram, Sunnia and Bajwa, Imran Sarwar and Ikram, Amna and Abdullah-Al-Wadud, M. and Pk, Haleema},
  title = {A Transformer-Based Multimodal Object Detection System for Real-World Applications},
  journal = {IEEE Access},
  year = {2025},
  volume = {13},
  pages = {29162--29176},
  doi = {10.1109/access.2025.3539569}
}
Kostenlos starten →