EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multimodal Self-Supervised Learning Framework for Scene Understanding in Autonomous Driving Systems

Madhuvarsu Subhash Chandra Manvith, K. Namitha, Dhanush M

IEEE Open Journal of the Computer Society · 2026 · 7 · 1333-1343

DOI: 10.1109/ojcs.2026.3707444 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Open Journal of the Computer Society, 2026. Verfasst von Madhuvarsu Subhash Chandra Manvith und 2 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate scene understanding is a key requirement for autonomous driving systems, especially when operating under varying and partially observable scenes. However, it is also known that most of the perception systems are heavily dependent on large-scale annotated datasets, making them vulnerable to sensor degradation, illumination, and weather. In this regard, we propose a unified multimodal framework combining self-supervised representation learning and supervised proposal initialization for robust scene understanding using RGB, thermal, LiDAR, and radar sensors. The proposed framework is based on modality-specific self-supervised encoders, a transformer-based multimodal fusion component, structured scene graphs, and graph neural networks for relational understanding. The experiments are performed on nuScenes, nuImages, and FLIR datasets, demonstrating the effectiveness of the proposed framework in improving robustness under varying illumination, occlusions, and sensor degradation, and also outperforming other baselines for object detection, semantic segmentation, and trajectory prediction tasks. The experiments demonstrate that multimodal self-supervised learning, when coupled with relational understanding, is a reliable and scalable framework for autonomous driving perception systems.

Zitieren

APA

Manvith, M. S. C., Namitha, K., & M, D. (2026). A Multimodal Self-Supervised Learning Framework for Scene Understanding in Autonomous Driving Systems. IEEE Open Journal of the Computer Society, 7, 1333-1343. https://doi.org/10.1109/ojcs.2026.3707444

Vancouver

Manvith MSC, Namitha K, M D. A Multimodal Self-Supervised Learning Framework for Scene Understanding in Autonomous Driving Systems. IEEE Open Journal of the Computer Society. 2026;7:1333-1343. doi:10.1109/ojcs.2026.3707444

BibTeX
@article{Manvith2026,
  author = {Manvith, Madhuvarsu Subhash Chandra and Namitha, K. and M, Dhanush},
  title = {A Multimodal Self-Supervised Learning Framework for Scene Understanding in Autonomous Driving Systems},
  journal = {IEEE Open Journal of the Computer Society},
  year = {2026},
  volume = {7},
  pages = {1333--1343},
  doi = {10.1109/ojcs.2026.3707444}
}
Kostenlos starten →