EUVIMEDEUVIMED entdecken
Crossref

Scalable Video Object Segmentation With Identification Mechanism

Zongxin Yang, Jiaxu Miao, Yunchao Wei, Wenguan Wang, Xiaohan Wang, Yi Yang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-09 · 46 (9) · 6247-6262

DOI: 10.1109/tpami.2024.3383592 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-09. Verfasst von Zongxin Yang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 107 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper delves into the challenges of achieving scalable and effective multi-object modeling for semi-supervised Video Object Segmentation (VOS). Previous VOS methods decode features with a single positive object, limiting the learning of multi-object representation as they must match and segment each target separately under multi-object scenarios. Additionally, earlier techniques catered to specific application objectives and lacked the flexibility to fulfill different speed-accuracy requirements. To address these problems, we present two innovative approaches, Associating Objects with Transformers (AOT) and Associating Objects with Scalable Transformers (AOST). In pursuing effective multi-object modeling, AOT introduces the IDentification (ID) mechanism to allocate each object a unique identity. This approach enables the network to model the associations among all objects simultaneously, thus facilitating the tracking and segmentation of objects in a single network pass. To address the challenge of inflexible deployment, AOST further integrates scalable long short-term transformers that incorporate scalable supervision and layer-wise ID-based attention. This enables online architecture scalability in VOS for the first time and overcomes ID embeddings' representation limitations. Given the absence of a benchmark for VOS involving densely multi-object annotations, we propose a challenging Video Object Segmentation in the Wild (VOSW) benchmark to validate our approaches. We evaluated various AOT and AOST variants using extensive experiments across VOSW and five commonly used VOS benchmarks, including YouTube-VOS 2018 & 2019 Val, DAVIS-2017 Val & Test, and DAVIS-2016. Our approaches surpass the state-of-the-art competitors and display exceptional efficiency and scalability consistently across all six benchmarks. Moreover, we notably achieved the 1st position in the 3 rd Large-scale Video Object Segmentation Challenge.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Yang, Z., Miao, J., Wei, Y., Wang, W., Wang, X., & Yang, Y. (2024). Scalable Video Object Segmentation With Identification Mechanism. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 6247-6262. https://doi.org/10.1109/tpami.2024.3383592

Vancouver

Yang Z, Miao J, Wei Y, Wang W, Wang X, Yang Y. Scalable Video Object Segmentation With Identification Mechanism. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(9):6247-6262. doi:10.1109/tpami.2024.3383592

BibTeX
@article{Yang2024,
  author = {Yang, Zongxin and Miao, Jiaxu and Wei, Yunchao and Wang, Wenguan and Wang, Xiaohan and Yang, Yi},
  title = {Scalable Video Object Segmentation With Identification Mechanism},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {9},
  pages = {6247--6262},
  doi = {10.1109/tpami.2024.3383592}
}
Kostenlos starten →