A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation
IEEE Transactions on Circuits and Systems for Video Technology · 2023-06 · 33 (6) · 2892-2905
DOI: 10.1109/tcsvt.2022.3229359 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-06. Verfasst von Shenlu Zhao und 1 weiteren Autor:innen. Die Arbeit wurde bisher 90-mal zitiert (Crossref) und nennt 57 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shaanxi Innovation Team Project.
Abstract (Originalsprache)
Similar to other multi-modal pixel-level prediction tasks, existing RGB-T semantic segmentation methods usually employ a two-stream structure to extract RGB and thermal infrared (TIR) features, respectively, and adopt the same fusion strategies to integrate different levels of unimodal features. This will result in inadequate extraction of unimodal features and exploitation of cross-modal information from the paired RGB and TIR images. Alternatively, in this paper, we present a novel RGB-T semantic segmentation model, i.e., FDCNet, where a feature divide-and-conquer strategy performs unimodal feature extraction and cross-modal feature fusion in one go. Concretely, we first employ a two-stream structure to extract unimodal low-level features, followed by a Siamese structure to extract unimodal high-level features from the paired RGB and TIR images. This concise but efficient structure enables to take into account both the modality discrepancies of low-level features and the underlying semantic consistency of high-level features across the paired RGB and TIR images. Furthermore, considering the characteristics of different layers of features, a Cross-modal Spatial Activation (CSA) module and a Cross-modal Channel Activation (CCA) module are presented for the fusion of low-level RGB and TIR features and for the fusion of high-level RGB and TIR features, respectively, thus facilitating the capture of cross-modal information. On top of that, with an embedded Cross-scale Interaction Context (CIC) module for mining multi-scale contextual information, our proposed model (i.e., FDCNet) for RGB-T semantic segmentation achieves new state-of-the-art experimental results on MFNet dataset and PST900 dataset.
Förderung
- National Natural Science Foundation of China
- Shaanxi Innovation Team Project
Zitieren
APA
Zhao, S., & Zhang, Q. (2023). A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 33(6), 2892-2905. https://doi.org/10.1109/tcsvt.2022.3229359
Vancouver
Zhao S, Zhang Q. A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(6):2892-2905. doi:10.1109/tcsvt.2022.3229359
BibTeX
@article{Zhao2023,
author = {Zhao, Shenlu and Zhang, Qiang},
title = {A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {6},
pages = {2892--2905},
doi = {10.1109/tcsvt.2022.3229359}
}
EUVIMED