ORL-VIS: Occlusion-Robust Long Video Instance Segmentation via Temporal Decoupling and Hierarchical Query Propagation
IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16587-16601
DOI: 10.1109/tcsvt.2026.3719009 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Yuran Chen und 8 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Long video instance segmentation (VIS) is essential for instance-level video understanding, but frequent occlusions make it difficult to maintain accurate masks and consistent identities over extended temporal spans. We argue that this difficulty mainly arises from three aspects. First, there exists a pronounced temporal gap between short-clip training and long-video inference. Constrained by computational cost, existing VIS models are typically trained on short clips, which biases their temporal modeling toward local continuity and short-term variations. Such a training paradigm is insufficient for modeling long-term occlusion, where objects may disappear and later reappear with large appearance or spatial changes, making temporal association unreliable during long-video inference. Second, this difficulty is further aggravated by unified temporal modeling. Different temporal patterns, such as stable tracking, rapid motion, and long-term occlusion, are often entangled within the same representation space, which may confuse the temporal cues required for reliable long-range association. Third, when an instance reappears after occlusion, its identity association across frames becomes difficult to maintain, leading to identity drift and incorrect re-association. To address these challenges and strengthen long-range temporal association, we propose Occlusion-Robust Long Video Instance Segmentation (ORL-VIS), a framework that improves long-video VIS from both data and model perspectives. At the data level, a cross-dataset multi-instance online augmentation strategy (CD-MIA) simulates long-term occlusion and reappearance patterns within short training clips, thereby reducing the training–inference temporal discrepancy. At the model level, a Mixture of Experts (MoE) architecture decouples heterogeneous temporal patterns by routing distinct temporal dynamics to specialized experts for pattern-specific modeling, while a Hierarchical Query Propagation (HQP) mechanism propagates complementary geometric and semantic identity cues to strengthen cross-frame identity alignment for reappearing instances. Extensive experiments on YouTube-VIS 2019, YouTube-VIS 2021, YouTube-VIS 2022, OVIS, and two self-built datasets demonstrate that ORL-VIS achieves state-of-the-art performance and exhibits strong robustness in challenging long-video scenarios with frequent occlusions.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Chen, Y., Cai, X., Liu, L., Gong, Z., Ge, Y., Sun, Q., Qin, A. K., Ge, H., & Li, C. (2026). ORL-VIS: Occlusion-Robust Long Video Instance Segmentation via Temporal Decoupling and Hierarchical Query Propagation. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16587-16601. https://doi.org/10.1109/tcsvt.2026.3719009
Vancouver
Chen Y, Cai X, Liu L, Gong Z, Ge Y, Sun Q, et al. ORL-VIS: Occlusion-Robust Long Video Instance Segmentation via Temporal Decoupling and Hierarchical Query Propagation. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16587-16601. doi:10.1109/tcsvt.2026.3719009
BibTeX
@article{Chen2026,
author = {Chen, Yuran and Cai, Xinye and Liu, Lei and Gong, Zhonglin and Ge, Yichen and Sun, Qing and Qin, A. K. and Ge, Hongwei and Li, Changhe},
title = {ORL-VIS: Occlusion-Robust Long Video Instance Segmentation via Temporal Decoupling and Hierarchical Query Propagation},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {11},
pages = {16587--16601},
doi = {10.1109/tcsvt.2026.3719009}
}
EUVIMED