EUVIMEDEUVIMED entdecken
Crossref

Exploring and Exploiting High-Order Spatial–Temporal Dynamics for Long-Term Frame Prediction

Kuai Dai, Xutao Li, Yunming Ye, Yaowei Wang, Shanshan Feng, Di Xian

IEEE Transactions on Circuits and Systems for Video Technology · 2024-03 · 34 (3) · 1841-1856

DOI: 10.1109/tcsvt.2023.3298978 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-03. Verfasst von Kuai Dai und 5 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Shenzhen Science and Technology Program, FengYun Application Pioneering Project, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Long-term spatial-temporal frame prediction focuses on predicting future image frames precisely, which has numerous applications in real-world scenarios. Existing deep learning prediction models mainly rely on advanced neural network architectures to model complicated spatial-temporal features, which make few efforts to explore high-order correlations to better capture long-term dynamics. Their prediction on long-term frames suffers from inaccurate visual and motion detail issue. In this article, we propose a high-order prediction model for long-term frame prediction, which improves the appearance and motion details by designing special high-order correlation modules in two aspects. First, to enhance the appearance details of predicted frames, we propose a high-order appearance encoder module, where high-order appearance features can be effectively captured with a carefully designed Non-local ConvLSTM. Second, to guarantee the motion accuracy of predicted sequences, we carefully design a high-order motion encoder module, which can accurately capture and preserve the high-order motion patterns with adaptive motion extractors and progressive memory banks, respectively. Comprehensive experiments are conducted on six challenging datasets from real-world scenarios, which demonstrate the effectiveness and superiority of our proposed method over state-of-the-art methods.

Förderung

  • Shenzhen Science and Technology Program
  • FengYun Application Pioneering Project
  • National Natural Science Foundation of China

Zitieren

APA

Dai, K., Li, X., Ye, Y., Wang, Y., Feng, S., & Xian, D. (2024). Exploring and Exploiting High-Order Spatial–Temporal Dynamics for Long-Term Frame Prediction. IEEE Transactions on Circuits and Systems for Video Technology, 34(3), 1841-1856. https://doi.org/10.1109/tcsvt.2023.3298978

Vancouver

Dai K, Li X, Ye Y, Wang Y, Feng S, Xian D. Exploring and Exploiting High-Order Spatial–Temporal Dynamics for Long-Term Frame Prediction. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(3):1841-1856. doi:10.1109/tcsvt.2023.3298978

BibTeX
@article{Dai2024,
  author = {Dai, Kuai and Li, Xutao and Ye, Yunming and Wang, Yaowei and Feng, Shanshan and Xian, Di},
  title = {Exploring and Exploiting High-Order Spatial–Temporal Dynamics for Long-Term Frame Prediction},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2024},
  volume = {34},
  number = {3},
  pages = {1841--1856},
  doi = {10.1109/tcsvt.2023.3298978}
}
Kostenlos starten →