SipMaskv2: Enhanced Fast Image and Video Instance Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.03.2023 · 45 (3) · 3798-3812
DOI: 10.1109/tpami.2022.3180564 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.03.2023. Verfasst von Jiale Cao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Tianjin Municipal Science and Technology Program.
Abstract (Originalsprache)
We propose a fast single-stage method for both image and video instance segmentation, called SipMask, that preserves the instance spatial information by performing multiple sub-region mask predictions. The main module in our method is a light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for the sub-regions within a bounding-box, enabling a better delineation of spatially adjacent instances. To better correlate mask prediction with object detection, we further propose a mask alignment weighting loss and a feature alignment scheme. In addition, we identify two issues that impede the performance of single-stage instance segmentation and introduce two modules, including a sample selection scheme and an instance refinement module, to address these two issues. Experiments are performed on both image instance segmentation dataset MS COCO and video instance segmentation dataset YouTube-VIS. On MS COCO test-dev set, our method achieves a state-of-the-art performance. In terms of real-time capabilities, it outperforms YOLACT by a gain of 3.0% (mask AP) under the similar settings, while operating at a comparable speed. On YouTube-VIS validation set, our method also achieves promising results. The source code is available at https://github.com/JialeCao001/SipMask.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Tianjin Municipal Science and Technology Program
- Natural Science Foundation of Tianjin Municipality
- CAAI-Huawei MindSpore Open Fund
Zitieren
APA
Cao, J., Pang, Y., Anwer, R. M., Cholakkal, H., Khan, F. S., & Shao, L. (2023). SipMaskv2: Enhanced Fast Image and Video Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3), 3798-3812. https://doi.org/10.1109/tpami.2022.3180564
Vancouver
Cao J, Pang Y, Anwer RM, Cholakkal H, Khan FS, Shao L. SipMaskv2: Enhanced Fast Image and Video Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(3):3798-3812. doi:10.1109/tpami.2022.3180564
BibTeX
@article{Cao2023,
author = {Cao, Jiale and Pang, Yanwei and Anwer, Rao Muhammad and Cholakkal, Hisham and Khan, Fahad Shahbaz and Shao, Ling},
title = {SipMaskv2: Enhanced Fast Image and Video Instance Segmentation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {3},
pages = {3798--3812},
doi = {10.1109/tpami.2022.3180564}
}
EUVIMED