MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection
IET Image Processing · 2026-01 · 20 (1)
DOI: 10.1049/ipr2.70408 ↗Kurz gefasst
Fachartikel in der Zeitschrift IET Image Processing, online erschienen am 03.07.2026. Verfasst von Xinyu Gao und 4 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
As autonomous driving progresses toward levels 4/5, researchers increasingly adopt multi‐sensor fusion to break the accuracy bottleneck in 3D object detection. Yet current multi‐modal frameworks are limited in aligning heterogeneous features and enabling joint cross‐modal inference, hindering the full exploitation of inter‐modal complementarity. We propose a novel framework based on multi‐modal, multi‐view serial fusion for 3D detection (MMSF‐DET). In the perspective view (PV), which is closer to the image domain, we introduce a perspective‐view multi‐scale bidirectional fusion module (PV‐MBF) built on a sectorized point‐cloud grid to realize initial cross‐modal fusion. In the birdn the bird. (BEV), we design a BEV‐MBF that leverages bilinear interpolation to perform deep, MBF of cross‐modal features. To overcome the shallow interactions of conventional parallel fusion and strengthen spatial awareness, we devise a serial fusion strategy that operates across the PV and BEV branches. Additionally, we employ a collaborative training scheme with auxiliary tasks—3D semantic segmentation, 2D object detection, and 2D semantic segmentation—and cross‐modal multi‐task joint optimization to further enhance performance. Experiments on KITTI and the Waymo open dataset (WOD) demonstrate superiority over existing multi‐modal detectors, validating the effectiveness and advancement of the proposed fusion strategy and multi‐task learning framework.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Gao, X., Wang, A., Jing, H., Xu, S., & Tillo, T. (2026). MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing, 20(1). https://doi.org/10.1049/ipr2.70408
Vancouver
Gao X, Wang A, Jing H, Xu S, Tillo T. MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection. IET Image Processing. 2026;20(1). doi:10.1049/ipr2.70408
BibTeX
@article{Gao2026,
author = {Gao, Xinyu and Wang, Anhong and Jing, Hao and Xu, Shiao and Tillo, Tammam},
title = {MMSF‐DET : Multi‐Modal Multi‐View Serial Fusion for 3D Object Detection},
journal = {IET Image Processing},
year = {2026},
volume = {20},
number = {1},
doi = {10.1049/ipr2.70408}
}
EUVIMED