EUVIMEDEUVIMED entdecken
Crossref

RayD3D: Distilling Depth Knowledge Along the Ray for Robust Multi-View 3D Object Detection

Rui Ding, Zhaonian Kuang, Zongwei Zhou, Meng Yang, Xinhu Zheng, Gang Hua

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (5) · 3605-3613

DOI: 10.1609/aaai.v40i5.37359 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Rui Ding und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-view 3D detection with bird’s eye view (BEV) is crucial for autonomous driving and robotics, but its robustness in real-world is limited as it struggles to predict accurate depth values. A mainstream solution, cross-modal distillation, transfers depth information from LiDAR to camera models but also unintentionally transfers depth-irrelevant information (e.g. LiDAR density). To mitigate this issue, we propose RayD3D, which transfers crucial depth knowledge along the ray: a line projecting from the camera to true location of an object. It is based on the fundamental imaging principle that predicted location of this object can only vary along this ray, which is finally determined by predicted depth value. Therefore, distilling along the ray enables more effective depth information transfer. More specifically, we design two ray-based distillation modules. Ray-based Contrastive Distillation (RCD) incorporates contrastive learning into distillation by sampling along the ray to learn how LiDAR accurately locates objects. Ray-based Weighted Distillation (RWD) adaptively adjusts distillation weight based on the ray to minimize the interference of depth-irrelevant information in LiDAR. For validation, we widely apply RayD3D into three representative types of BEV-based models, including BEVDet, BEVDepth4D, and BEVFormer. Our method is trained on clean NuScenes, and tested on both clean NuScenes and RoboBEV with a variety types of data corruptions. Our method significantly improves the robustness of all the three base models in all scenarios without increasing inference costs, and achieves the best when compared to recently released multi-view and distillation models.

Zitieren

APA

Ding, R., Kuang, Z., Zhou, Z., Yang, M., Zheng, X., & Hua, G. (2026). RayD3D: Distilling Depth Knowledge Along the Ray for Robust Multi-View 3D Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 40(5), 3605-3613. https://doi.org/10.1609/aaai.v40i5.37359

Vancouver

Ding R, Kuang Z, Zhou Z, Yang M, Zheng X, Hua G. RayD3D: Distilling Depth Knowledge Along the Ray for Robust Multi-View 3D Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(5):3605-3613. doi:10.1609/aaai.v40i5.37359

BibTeX
@article{Ding2026,
  author = {Ding, Rui and Kuang, Zhaonian and Zhou, Zongwei and Yang, Meng and Zheng, Xinhu and Hua, Gang},
  title = {RayD3D: Distilling Depth Knowledge Along the Ray for Robust Multi-View 3D Object Detection},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {5},
  pages = {3605--3613},
  doi = {10.1609/aaai.v40i5.37359}
}
Kostenlos starten →