DFormer++: Improving RGBD Representation Learning for Semantic Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-07 · 48 (7) · 7195-7208
DOI: 10.1109/tpami.2026.3658114 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-07. Verfasst von Bo-Wen Yin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: National Key Research and Development Project of China, National Natural Science Foundation of China, Science and Technology Support Program of Tianjin, China.
Abstract (Originalsprache)
We explore the potential of pretrain-and-finetune manner on the RGB-D semantic segmentation to solve the common mismatch problem in this field. Specifically, we present DFormer++, a novel RGB-D pretrain-and-finetune framework to learn transferable representations for RGB-D semantic segmentation. This paper has two vital innovations. 1) Framework perspective: Different from the existing methods that finetune RGB pretrained backbone to the RGB-D scenes, we pretrain the backbone using image-depth pairs from ImageNet-1 K, and hence the model is endowed with the capacity to encode RGB-D representations; 2) Architecture perspective: Our model comprises a sequence of RGB-D attention blocks, which are tailored for encoding both RGB and depth information through a novel attention mechanism. Our DFormer++ avoids the mismatched encoding of the 3D geometry relationships in depth maps by RGB pretrained backbones, which widely lies in previous works but has not been resolved. Meanwhile, the tailored architecture greatly reduces redundant parameters for encoding RGB-D data and achieves efficient and accurate perception. Experimental results show that our DFormer++ achieves new cutting-edge performance on three popular RGB-D semantic segmentation benchmarks.
Förderung
- National Key Research and Development Project of China
- National Natural Science Foundation of China
- Science and Technology Support Program of Tianjin, China
- Fundamental Research Funds for the Central Universities
- Nankai University
Zitieren
APA
Yin, B. W., Cao, J. L., Xu, D., Cheng, M. M., & Hou, Q. (2026). DFormer++: Improving RGBD Representation Learning for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(7), 7195-7208. https://doi.org/10.1109/tpami.2026.3658114
Vancouver
Yin BW, Cao JL, Xu D, Cheng MM, Hou Q. DFormer++: Improving RGBD Representation Learning for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(7):7195-7208. doi:10.1109/tpami.2026.3658114
BibTeX
@article{Yin2026,
author = {Yin, Bo-Wen and Cao, Jiao-Long and Xu, Dan and Cheng, Ming-Ming and Hou, Qibin},
title = {DFormer++: Improving RGBD Representation Learning for Semantic Segmentation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {7},
pages = {7195--7208},
doi = {10.1109/tpami.2026.3658114}
}
EUVIMED