EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection

Xinyu Gao, Anhong Wang, Hao Jing, Shiao Xu, Tammam Tillo

IET Image Processing · 2026-01 · 20 (1)

DOI: 10.1049/ipr2.70408 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Image Processing, online erschienen am 03.07.2026. Verfasst von Xinyu Gao und 4 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As autonomous driving progresses toward levels 4/5, researchers increasingly adopt multi‐sensor fusion to break the accuracy bottleneck in 3D object detection. Yet current multi‐modal frameworks are limited in aligning heterogeneous features and enabling joint cross‐modal inference, hindering the full exploitation of inter‐modal complementarity. We propose a novel framework based on multi‐modal, multi‐view serial fusion for 3D detection (MMSF‐DET). In the perspective view (PV), which is closer to the image domain, we introduce a perspective‐view multi‐scale bidirectional fusion module (PV‐MBF) built on a sectorized point‐cloud grid to realize initial cross‐modal fusion. In the birdn the bird. (BEV), we design a BEV‐MBF that leverages bilinear interpolation to perform deep, MBF of cross‐modal features. To overcome the shallow interactions of conventional parallel fusion and strengthen spatial awareness, we devise a serial fusion strategy that operates across the PV and BEV branches. Additionally, we employ a collaborative training scheme with auxiliary tasks—3D semantic segmentation, 2D object detection, and 2D semantic segmentation—and cross‐modal multi‐task joint optimization to further enhance performance. Experiments on KITTI and the Waymo open dataset (WOD) demonstrate superiority over existing multi‐modal detectors, validating the effectiveness and advancement of the proposed fusion strategy and multi‐task learning framework.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Gao, X., Wang, A., Jing, H., Xu, S., & Tillo, T. (2026). MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing, 20(1). https://doi.org/10.1049/ipr2.70408

Vancouver

Gao X, Wang A, Jing H, Xu S, Tillo T. MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing. 2026;20(1). doi:10.1049/ipr2.70408

BibTeX
@article{Gao2026,
  author = {Gao, Xinyu and Wang, Anhong and Jing, Hao and Xu, Shiao and Tillo, Tammam},
  title = {MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection},
  journal = {IET Image Processing},
  year = {2026},
  volume = {20},
  number = {1},
  doi = {10.1049/ipr2.70408}
}
Kostenlos starten →