EUVIMEDEUVIMED entdecken
Crossref

OpenVIS: Open-vocabulary Video Instance Segmentation

Pinxue Guo, Hao Huang, Peiyang He, Xuefeng Liu, Tianjun Xiao, Wenqiang Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (3) · 3275-3283

DOI: 10.1609/aaai.v39i3.32338 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Pinxue Guo und 5 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Open-vocabulary Video Instance Segmentation (OpenVIS) can simultaneously detect, segment, and track arbitrary object categories in a video, without being constrained to categories seen during training. In this work, we propose InstFormer, a carefully designed framework for the OpenVIS task that achieves powerful open-vocabulary capabilities through lightweight fine-tuning with limited-category data. InstFormer begins with the open-world mask proposal network, encouraged to propose all potential instance class-agnostic masks by the contrastive instance margin loss. Next, we introduce InstCLIP, adapted from pre-trained CLIP with Instance Guidance Attention, which encodes open-vocabulary instance tokens efficiently. These instance tokens not only enable open-vocabulary classification but also offer strong universal tracking capabilities. Furthermore, to prevent the tracking module from being constrained by the training data with limited categories, we propose the universal rollout association, which transforms the tracking problem into predicting the next frame’s instance tracking token. The experimental results demonstrate the proposed InstFormer achieve state-of-the-art capabilities on a comprehensive OpenVIS evaluation benchmark, while also achieves competitive performance in fully supervised VIS task.

Zitieren

APA

Guo, P., Huang, H., He, P., Liu, X., Xiao, T., & Zhang, W. (2025). OpenVIS: Open-vocabulary Video Instance Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 3275-3283. https://doi.org/10.1609/aaai.v39i3.32338

Vancouver

Guo P, Huang H, He P, Liu X, Xiao T, Zhang W. OpenVIS: Open-vocabulary Video Instance Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(3):3275-3283. doi:10.1609/aaai.v39i3.32338

BibTeX
@article{Guo2025,
  author = {Guo, Pinxue and Huang, Hao and He, Peiyang and Liu, Xuefeng and Xiao, Tianjun and Zhang, Wenqiang},
  title = {OpenVIS: Open-vocabulary Video Instance Segmentation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {3},
  pages = {3275--3283},
  doi = {10.1609/aaai.v39i3.32338}
}
Kostenlos starten →