EUVIMEDEUVIMED entdecken
Crossref

t-READi: Transformer-Powered Robust and Efficient Multimodal Inference for Autonomous Driving

Pengfei Hu, Yuhang Qian, Tianyue Zheng, Ang Li, Zhe Chen, Yue Gao, Xiuzhen Cheng, Jun Luo

IEEE Transactions on Mobile Computing · 2025-01 · 24 (1) · 135-149

DOI: 10.1109/tmc.2024.3462437 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2025-01. Verfasst von Pengfei Hu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, Shandong Science Fund for Excellent Young Scholars.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Given the wide adoption of multimodal sensors (e.g., camera, lidar, radar) by autonomous vehicles (AVs), deep analytics to fuse their outputs for a robust perception become imperative. However, existing fusion methods often make two assumptions rarely holding in practice: i) similar data distributions for all inputs and ii) constant availability for all sensors. Because, for example, lidars have various resolutions and failures of radars may occur, such variability often results in significant performance degradation in fusion. To this end, we present t-READi, an adaptive inference system that accommodates the variability of multimodal sensory data and thus enables robust and efficient perception. t-READi identifies variation-sensitive yet structure-specific model parameters; it then adapts only these parameters while keeping the rest intact. t-READi also leverages a cross-modality contrastive learning method to compensate for the loss from missing modalities. Both functions are implemented to maintain compatibility with existing multimodal deep fusion methods. The extensive experiments evidently demonstrate that compared with the status quo approaches, t-READi not only improves the average inference accuracy by more than 6% but also reduces the inference latency by almost 15× with the cost of only 5% extra memory overhead in the worst case under realistic data and modal variations.

Förderung

  • National Key R&D Program of China
  • National Natural Science Foundation of China
  • Shandong Science Fund for Excellent Young Scholars
  • Shandong Science Fund
  • Southern University of Science and Technology
  • Shenzhen Higher Education Institutions Stable Support Program

Zitieren

APA

Hu, P., Qian, Y., Zheng, T., Li, A., Chen, Z., Gao, Y., Cheng, X., & Luo, J. (2025). t-READi: Transformer-Powered Robust and Efficient Multimodal Inference for Autonomous Driving. IEEE Transactions on Mobile Computing, 24(1), 135-149. https://doi.org/10.1109/tmc.2024.3462437

Vancouver

Hu P, Qian Y, Zheng T, Li A, Chen Z, Gao Y, et al. t-READi: Transformer-Powered Robust and Efficient Multimodal Inference for Autonomous Driving. IEEE Transactions on Mobile Computing. 2025;24(1):135-149. doi:10.1109/tmc.2024.3462437

BibTeX
@article{Hu2025,
  author = {Hu, Pengfei and Qian, Yuhang and Zheng, Tianyue and Li, Ang and Chen, Zhe and Gao, Yue and Cheng, Xiuzhen and Luo, Jun},
  title = {t-READi: Transformer-Powered Robust and Efficient Multimodal Inference for Autonomous Driving},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2025},
  volume = {24},
  number = {1},
  pages = {135--149},
  doi = {10.1109/tmc.2024.3462437}
}
Kostenlos starten →