EUVIMEDEUVIMED entdecken
Crossref

DFormer++: Improving RGBD Representation Learning for Semantic Segmentation

Bo-Wen Yin, Jiao-Long Cao, Dan Xu, Ming-Ming Cheng, Qibin Hou

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-07 · 48 (7) · 7195-7208

DOI: 10.1109/tpami.2026.3658114 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-07. Verfasst von Bo-Wen Yin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: National Key Research and Development Project of China, National Natural Science Foundation of China, Science and Technology Support Program of Tianjin, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We explore the potential of pretrain-and-finetune manner on the RGB-D semantic segmentation to solve the common mismatch problem in this field. Specifically, we present DFormer++, a novel RGB-D pretrain-and-finetune framework to learn transferable representations for RGB-D semantic segmentation. This paper has two vital innovations. 1) Framework perspective: Different from the existing methods that finetune RGB pretrained backbone to the RGB-D scenes, we pretrain the backbone using image-depth pairs from ImageNet-1 K, and hence the model is endowed with the capacity to encode RGB-D representations; 2) Architecture perspective: Our model comprises a sequence of RGB-D attention blocks, which are tailored for encoding both RGB and depth information through a novel attention mechanism. Our DFormer++ avoids the mismatched encoding of the 3D geometry relationships in depth maps by RGB pretrained backbones, which widely lies in previous works but has not been resolved. Meanwhile, the tailored architecture greatly reduces redundant parameters for encoding RGB-D data and achieves efficient and accurate perception. Experimental results show that our DFormer++ achieves new cutting-edge performance on three popular RGB-D semantic segmentation benchmarks.

Förderung

  • National Key Research and Development Project of China
  • National Natural Science Foundation of China
  • Science and Technology Support Program of Tianjin, China
  • Fundamental Research Funds for the Central Universities
  • Nankai University

Zitieren

APA

Yin, B. W., Cao, J. L., Xu, D., Cheng, M. M., & Hou, Q. (2026). DFormer++: Improving RGBD Representation Learning for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(7), 7195-7208. https://doi.org/10.1109/tpami.2026.3658114

Vancouver

Yin BW, Cao JL, Xu D, Cheng MM, Hou Q. DFormer++: Improving RGBD Representation Learning for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(7):7195-7208. doi:10.1109/tpami.2026.3658114

BibTeX
@article{Yin2026,
  author = {Yin, Bo-Wen and Cao, Jiao-Long and Xu, Dan and Cheng, Ming-Ming and Hou, Qibin},
  title = {DFormer++: Improving RGBD Representation Learning for Semantic Segmentation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {7},
  pages = {7195--7208},
  doi = {10.1109/tpami.2026.3658114}
}
Kostenlos starten →