EUVIMEDObjevte EUVIMED
Open access · CC BY-NC 4.0

MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection

Xinyu Gao, Anhong Wang, Hao Jing, Shiao Xu, Tammam Tillo

IET Image Processing · 2026-01 · 20 (1)

DOI: 10.1049/ipr2.70408 ↗

At a glance

Journal article in IET Image Processing, published online on 2026-07-03. Written by Xinyu Gao and 4 co-authors. 49 references. The full text is freely available (open access, CC BY-NC 4.0). Funded by: National Natural Science Foundation of China.

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

As autonomous driving progresses toward levels 4/5, researchers increasingly adopt multi‐sensor fusion to break the accuracy bottleneck in 3D object detection. Yet current multi‐modal frameworks are limited in aligning heterogeneous features and enabling joint cross‐modal inference, hindering the full exploitation of inter‐modal complementarity. We propose a novel framework based on multi‐modal, multi‐view serial fusion for 3D detection (MMSF‐DET). In the perspective view (PV), which is closer to the image domain, we introduce a perspective‐view multi‐scale bidirectional fusion module (PV‐MBF) built on a sectorized point‐cloud grid to realize initial cross‐modal fusion. In the birdn the bird. (BEV), we design a BEV‐MBF that leverages bilinear interpolation to perform deep, MBF of cross‐modal features. To overcome the shallow interactions of conventional parallel fusion and strengthen spatial awareness, we devise a serial fusion strategy that operates across the PV and BEV branches. Additionally, we employ a collaborative training scheme with auxiliary tasks—3D semantic segmentation, 2D object detection, and 2D semantic segmentation—and cross‐modal multi‐task joint optimization to further enhance performance. Experiments on KITTI and the Waymo open dataset (WOD) demonstrate superiority over existing multi‐modal detectors, validating the effectiveness and advancement of the proposed fusion strategy and multi‐task learning framework.

Funding

  • National Natural Science Foundation of China

Cite

APA

Gao, X., Wang, A., Jing, H., Xu, S., & Tillo, T. (2026). MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing, 20(1). https://doi.org/10.1049/ipr2.70408

Vancouver

Gao X, Wang A, Jing H, Xu S, Tillo T. MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing. 2026;20(1). doi:10.1049/ipr2.70408

BibTeX
@article{Gao2026,
  author = {Gao, Xinyu and Wang, Anhong and Jing, Hao and Xu, Shiao and Tillo, Tammam},
  title = {MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection},
  journal = {IET Image Processing},
  year = {2026},
  volume = {20},
  number = {1},
  doi = {10.1049/ipr2.70408}
}
Start for free →