EUVIMEDEUVIMED entdecken
Crossref

DeepInteraction++: Multi-Modality Interaction for Autonomous Driving

Zeyu Yang, Nan Song, Wei Li, Xiatian Zhu, Li Zhang, Philip H S Torr

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-08 · 47 (8) · 6749-6763

DOI: 10.1109/tpami.2025.3565194 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-08. Verfasst von Zeyu Yang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 75 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing top-performance autonomous driving systems typically rely on the multi-modal fusion strategy for reliable scene understanding. This design is however fundamentally restricted due to overlooking the modality-specific strengths and finally hampering the model performance. To address this limitation, in this work, we introduce a novel modality interaction strategy that allows individual per-modality representations to be learned and maintained throughout, enabling their unique characteristics to be exploited during the whole perception pipeline. To demonstrate the effectiveness of the proposed strategy, we design DeepInteraction++, a multi-modal interaction framework characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder. Specifically, the encoder is implemented as a dual-stream Transformer with specialized attention operation for information exchange and integration between separate modality-specific representations. Our multi-modal representational learning incorporates both object-centric, precise sampling-based feature alignment and global dense information spreading, essential for the more challenging planning task. The decoder is designed to iteratively refine the predictions by alternately aggregating information from separate representations in a unified modality-agnostic manner, realizing multi-modal predictive interaction. Extensive experiments demonstrate the superior performance of the proposed framework on both 3D object detection and end-to-end autonomous driving tasks.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yang, Z., Song, N., Li, W., Zhu, X., Zhang, L., & Torr, P. H. S. (2025). DeepInteraction++: Multi-Modality Interaction for Autonomous Driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(8), 6749-6763. https://doi.org/10.1109/tpami.2025.3565194

Vancouver

Yang Z, Song N, Li W, Zhu X, Zhang L, Torr PHS. DeepInteraction++: Multi-Modality Interaction for Autonomous Driving. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(8):6749-6763. doi:10.1109/tpami.2025.3565194

BibTeX
@article{Yang2025,
  author = {Yang, Zeyu and Song, Nan and Li, Wei and Zhu, Xiatian and Zhang, Li and Torr, Philip H S},
  title = {DeepInteraction++: Multi-Modality Interaction for Autonomous Driving},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {8},
  pages = {6749--6763},
  doi = {10.1109/tpami.2025.3565194}
}
Kostenlos starten →