Pixel Difference Convolutional Network for RGB-D Semantic Segmentation
IEEE Transactions on Circuits and Systems for Video Technology · 2024-03 · 34 (3) · 1481-1492
DOI: 10.1109/tcsvt.2023.3296162 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-03. Verfasst von Jun Yang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 60-mal zitiert (Crossref) und nennt 72 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, China Scholarship Council.
Abstract (Originalsprache)
RGB-D semantic segmentation can be advanced with convolutional neural networks due to the availability of Depth data. Although objects cannot be easily discriminated by just the 2D appearance, with the local pixel difference and geometric patterns in Depth, they can be well separated in some cases. Considering the fixed grid kernel structure, CNNs are limited to lack the ability to capture detailed, fine-grained information and thus cannot achieve accurate pixel-level semantic segmentation. To solve this problem in the CNN structure, we propose a Pixel Difference Convolutional Network (PDCNet) to capture detailed intrinsic patterns by aggregating both intensity and gradient information in the local range for Depth data and global range for RGB data, respectively. Precisely, PDCNet consists of a Depth branch and an RGB branch. For the Depth branch, we propose a Pixel Difference Convolution (PDC) to consider local and detailed geometric information in Depth data via aggregating both intensity and gradient information. For the RGB branch, we contribute a lightweight Cascade Large Kernel (CLK) to extend PDC, namely CPDC, to enjoy global contexts for RGB data and further boost performance. Consequently, the local and global pixel differences from both modal data are seamlessly incorporated into PDCNet during the information propagation process. Experiments on three challenging benchmark datasets, $i.e.$ , NYUDv2 (78.4 Pixel Acc., 53.5 mIoU), SUN RGB-D (83.3 Pixel Acc., 49.6 mIoU) and SID Dataset (83.1 Pixel Acc., 61.4 mIoU) reveal that our PDCNet achieves state-of-the-art performance for the semantic segmentation task.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- China Scholarship Council
Zitieren
APA
Yang, J., Bai, L., Sun, Y., Tian, C., Mao, M., & Wang, G. (2024). Pixel Difference Convolutional Network for RGB-D Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 34(3), 1481-1492. https://doi.org/10.1109/tcsvt.2023.3296162
Vancouver
Yang J, Bai L, Sun Y, Tian C, Mao M, Wang G. Pixel Difference Convolutional Network for RGB-D Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(3):1481-1492. doi:10.1109/tcsvt.2023.3296162
BibTeX
@article{Yang2024,
author = {Yang, Jun and Bai, Lizhi and Sun, Yaoru and Tian, Chunqi and Mao, Maoyu and Wang, Guorun},
title = {Pixel Difference Convolutional Network for RGB-D Semantic Segmentation},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {3},
pages = {1481--1492},
doi = {10.1109/tcsvt.2023.3296162}
}
EUVIMED