EUVIMEDEUVIMED entdecken
Crossref

Motion-Aware Memory Network for Fast Video Salient Object Detection

Xing Zhao, Haoran Liang, Peipei Li, Guodao Sun, Dongdong Zhao, Ronghua Liang, Xiaofei He

IEEE Transactions on Image Processing · 2024 · 33 · 709-721

DOI: 10.1109/tip.2023.3348659 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2024. Verfasst von Xing Zhao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 45-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Previous methods based on 3DCNN, convLSTM, or optical flow have achieved great success in video salient object detection (VSOD). However, these methods still suffer from high computational costs or poor quality of the generated saliency maps. To address this, we design a space-time memory (STM)-based network that employs a standard encoder-decoder architecture. During the encoding stage, we extract high-level temporal features from the current frame and its adjacent frames, which is more efficient and practical than methods reliant on optical flow. During the decoding stage, we introduce an effective fusion strategy for both spatial and temporal branches. The semantic information of the high-level features is used to improve the object details in the low-level features. Subsequently, spatiotemporal features are methodically derived step by step to reconstruct the saliency maps. Moreover, inspired by the boundary supervision prevalent in image salient object detection (ISOD), we design a motion-aware loss that predicts object boundary motion, and simultaneously perform multitask learning for VSOD and object motion prediction. This can further enhance the model's capability to accurately extract spatiotemporal features while maintaining object integrity. Extensive experiments on several datasets demonstrate the effectiveness of our method and can achieve state-of-the-art metrics on some datasets. Our proposed model does not require optical flow or additional preprocessing, and can reach an impressive inference speed of nearly 100 FPS.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Zhejiang Provincial Natural Science Foundation of China

Zitieren

APA

Zhao, X., Liang, H., Li, P., Sun, G., Zhao, D., Liang, R., & He, X. (2024). Motion-Aware Memory Network for Fast Video Salient Object Detection. IEEE Transactions on Image Processing, 33, 709-721. https://doi.org/10.1109/tip.2023.3348659

Vancouver

Zhao X, Liang H, Li P, Sun G, Zhao D, Liang R, et al. Motion-Aware Memory Network for Fast Video Salient Object Detection. IEEE Transactions on Image Processing. 2024;33:709-721. doi:10.1109/tip.2023.3348659

BibTeX
@article{Zhao2024,
  author = {Zhao, Xing and Liang, Haoran and Li, Peipei and Sun, Guodao and Zhao, Dongdong and Liang, Ronghua and He, Xiaofei},
  title = {Motion-Aware Memory Network for Fast Video Salient Object Detection},
  journal = {IEEE Transactions on Image Processing},
  year = {2024},
  volume = {33},
  pages = {709--721},
  doi = {10.1109/tip.2023.3348659}
}
Kostenlos starten →