EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Deep Learning‐Based Panoptic Segmentation of Camera, Radar, and LiDAR Data From Autonomous Vehicles for Obstacle Avoidance

Solomon Dargie Ageze, Riessom Weldegiorgis, Temesgen Tadesse Feisa, Samuel Gizachew Alamirew

Journal of Sensors · 2026-01 · 2026 (1)

DOI: 10.1155/js/5528121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Sensors, online erschienen am 10.11.2026. Verfasst von Solomon Dargie Ageze und 3 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous vehicles (AVs) are becoming an integral part of modern transportation systems. Depending on the level of autonomy, control is shared between the driver and the vehicle, with lower levels of autonomy allowing the driver to retain greater control and decision‐making authority. However, existing perception systems still lack a thorough understanding of the environment, particularly in effectively distinguishing between “thing” and “stuff” classes. Hence, to improve scene understanding, this paper proposes a multimodal deep learning (DL) framework using Detectron2 and Mask2Former for panoptic segmentation in AVs. The framework integrates camera, light detection and ranging (LiDAR), and radar data and performs downstream vehicle control using a model predictive controller (MPC) implemented in Simulink. The proposed framework was validated on the nuScenes dataset, where it achieved remarkable segmentation performance with an accuracy of 99.2%, mean intersection over union (mIoU) of 91.5%, panoptic quality (PQ) of 68.1%, segmentation quality (SQ) of 85.2%, and recognition quality (RQ) of 75.9%. For LiDAR point cloud and radar data, the Mask2Former model was employed, optimized for 3D point cloud segmentation and Doppler velocity estimation, and it attained robust results on the panoptic nuScenes dataset with an mIoU of 78.8%, PQ of 76.8%, and RQ of 89.5%. Experimental analysis confirmed that Detectron2 effectively segments objects, while Mask2Former provides precise segmentation capabilities. The trained model was further integrated into a Simulink‐based driving scenario to enable near‐real‐time obstacle avoidance by adjusting the steering angle. Moreover, comparative analysis demonstrates that the proposed method outperforms existing segmentation approaches. Hence, these findings emphasize the significance of multimodal sensor fusion in enhancing environmental perception and represent a substantial advancement in autonomous driving applications.

Zitieren

APA

Ageze, S. D., Weldegiorgis, R., Feisa, T. T., & Alamirew, S. G. (2026). Deep Learning‐Based Panoptic Segmentation of Camera, Radar, and LiDAR Data From Autonomous Vehicles for Obstacle Avoidance. Journal of Sensors, 2026(1). https://doi.org/10.1155/js/5528121

Vancouver

Ageze SD, Weldegiorgis R, Feisa TT, Alamirew SG. Deep Learning‐Based Panoptic Segmentation of Camera, Radar, and LiDAR Data From Autonomous Vehicles for Obstacle Avoidance. Journal of Sensors. 2026;2026(1). doi:10.1155/js/5528121

BibTeX
@article{Ageze2026,
  author = {Ageze, Solomon Dargie and Weldegiorgis, Riessom and Feisa, Temesgen Tadesse and Alamirew, Samuel Gizachew},
  title = {Deep Learning‐Based Panoptic Segmentation of Camera, Radar, and LiDAR Data From Autonomous Vehicles for Obstacle Avoidance},
  journal = {Journal of Sensors},
  year = {2026},
  volume = {2026},
  number = {1},
  doi = {10.1155/js/5528121}
}
Kostenlos starten →