Learning High-Quality Dynamic Memory for Video Object Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-05 · 47 (5) · 3452-3468
DOI: 10.1109/tpami.2025.3532306 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-05. Verfasst von Yong Liu und 9 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 81 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames with masks as memory helps segment target objects in videos. However, they mainly focus on better matching between the current frame and memory frames without paying attention to the quality of the memory. Consequently, frames with poor segmentation masks may be memorized, leading to error accumulation problems. Besides, the linear increase of memory frames with the growth of frame numbers limits the ability of the models to handle long videos. To this end, we propose a Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation quality of each frame, allowing the memory bank to selectively store accurately segmented frames and prevent error accumulation. Then, we combine the segmentation quality with temporal consistency to dynamically update the memory bank and make the models can handle videos of arbitrary length. The above operation ensures the reliability of memory frames and improves the quality of memory at the frame level. Moreover, we observe that the memory features extracted from reliable frames still contain noise and have limited representation capabilities. To address this problem, we propose to perform memory enhancement and anchoring on the basis of QDMN to improve the quality of memory from the feature level, resulting in a more robust and effective network QDMN++. Our method achieves state-of-the-art performance on all popular benchmarks. Moreover, extensive experiments demonstrate that the proposed memory screening mechanism can be applied to any memory-based methods as generic plugins.
Förderung
- National Natural Science Foundation of China
- Shenzhen Science and Technology Program
Zitieren
APA
Liu, Y., Yu, R., Yin, F., Zhao, X., Zhao, W., Xia, W., Wang, J., Wang, Y., Tang, Y., & Yang, Y. (2025). Learning High-Quality Dynamic Memory for Video Object Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5), 3452-3468. https://doi.org/10.1109/tpami.2025.3532306
Vancouver
Liu Y, Yu R, Yin F, Zhao X, Zhao W, Xia W, et al. Learning High-Quality Dynamic Memory for Video Object Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(5):3452-3468. doi:10.1109/tpami.2025.3532306
BibTeX
@article{Liu2025,
author = {Liu, Yong and Yu, Ran and Yin, Fei and Zhao, Xinyuan and Zhao, Wei and Xia, Weihao and Wang, Jiahao and Wang, Yitong and Tang, Yansong and Yang, Yujiu},
title = {Learning High-Quality Dynamic Memory for Video Object Segmentation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {5},
pages = {3452--3468},
doi = {10.1109/tpami.2025.3532306}
}
EUVIMED