Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection
IEEE Transactions on Circuits and Systems for Video Technology · 2023-04 · 33 (4) · 1787-1801
DOI: 10.1109/tcsvt.2022.3215979 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-04. Verfasst von Gang Chen und 6 weiteren Autor:innen. Die Arbeit wurde bisher 102-mal zitiert (Crossref) und nennt 88 Literaturangaben. Gefördert von: Natural Science Foundation of China, Zhejiang Natural Science Foundation of China.
Abstract (Originalsprache)
The ability of capturing the complementary information of multi-modality data is critical to the development of multi-modality salient object detection (SOD). Most of existing studies attempt to integrate multi-modality information through various fusion strategies. However, most of these methods ignore the inherent differences in multi-modality data, resulting in poor performance when dealing with some challenging scenarios. In this paper, we propose a novel Modality-Induced Transfer-Fusion Network (MITF-Net) for RGB-D and RGB-T SOD by fully exploring the complementarity in multi-modality data. Specifically, we first deploy a modality transfer fusion (MTF) module to bridge the semantic gap between single and multi-modality data, and then mine the cross-modality complementarity based on point-to-point structural similarity information. Then, we design a cycle-separated attention (CSA) module to optimize the cross-layer information recurrently, and measure the effectiveness of cross-layer features through point-wise convolution-based multi-scale channel attention. Furthermore, we refine the boundaries in the decoding stage to obtain high-quality saliency maps with sharp boundaries. Extensive experiments on 13 RGB-D and RGB-T SOD datasets show that the proposed MITF-Net achieves a competitive and excellent performance.
Förderung
- Natural Science Foundation of China
- Zhejiang Natural Science Foundation of China
Zitieren
APA
Chen, G., Shao, F., Chai, X., Chen, H., Jiang, Q., Meng, X., & Ho, Y. (2023). Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection. IEEE Transactions on Circuits and Systems for Video Technology, 33(4), 1787-1801. https://doi.org/10.1109/tcsvt.2022.3215979
Vancouver
Chen G, Shao F, Chai X, Chen H, Jiang Q, Meng X, et al. Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(4):1787-1801. doi:10.1109/tcsvt.2022.3215979
BibTeX
@article{Chen2023,
author = {Chen, Gang and Shao, Feng and Chai, Xiongli and Chen, Hangwei and Jiang, Qiuping and Meng, Xiangchao and Ho, Yo-Sung},
title = {Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {4},
pages = {1787--1801},
doi = {10.1109/tcsvt.2022.3215979}
}
EUVIMED