JSNet++: Dynamic Filters and Pointwise Correlation for 3D Point Cloud Instance and Semantic Segmentation
IEEE Transactions on Circuits and Systems for Video Technology · 2023-04 · 33 (4) · 1854-1867
DOI: 10.1109/tcsvt.2022.3218076 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-04. Verfasst von Lin Zhao und 1 weiteren Autor:innen. Die Arbeit wurde bisher 55-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
In this paper, we propose a novel joint instance and semantic segmentation approach, called JSNet++, to address the instance and semantic segmentation tasks of 3D point clouds simultaneously. We first introduce a basic joint segmentation framework (JSNet). It fuses features from different layers of the backbone network to obtain more discriminative features and makes the two tasks take advantage of each other with a joint instance and semantic segmentation (JISS) module. Specifically, the JISS transforms semantic features into instance embedding space, and then the transformed features are fused with instance features to facilitate instance segmentation. Meanwhile, the JISS module also makes semantic segmentation benefit from instance segmentation by aggregating instance features to semantic feature space. To further reduce the memory consumption of JSNet, we design a dynamic filters for convolution (DFConv) on point clouds. Specifically, we exploit the geometry and density information to generate the dynamic filters, which are used to perform depthwise convolution with the input features. Afterwards, we unify the spatial correlation and channel correlation into a module to fully explore the pointwise correlation in point clouds, and we develop an improved JISS module (JISS*) by using the pointwise correlation module to further improve the accuracy of segmentation. Finally, based on the JSNet, DFConv and JISS*, we propose a new joint segmentation network, termed JSNet++. Experimental results on the benchmarks S3DIS and ScanNet v2 datasets demonstrate the effectiveness of our approach, and our method achieves significant performance improvements over baseline on both instance and semantic segmentation.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhao, L., & Tao, W. (2023). JSNet++: Dynamic Filters and Pointwise Correlation for 3D Point Cloud Instance and Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 33(4), 1854-1867. https://doi.org/10.1109/tcsvt.2022.3218076
Vancouver
Zhao L, Tao W. JSNet++: Dynamic Filters and Pointwise Correlation for 3D Point Cloud Instance and Semantic Segmentation. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(4):1854-1867. doi:10.1109/tcsvt.2022.3218076
BibTeX
@article{Zhao2023,
author = {Zhao, Lin and Tao, Wenbing},
title = {JSNet++: Dynamic Filters and Pointwise Correlation for 3D Point Cloud Instance and Semantic Segmentation},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {4},
pages = {1854--1867},
doi = {10.1109/tcsvt.2022.3218076}
}
EUVIMED