EUVIMEDEUVIMED entdecken
Crossref

STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond

Zheng Chang, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Wen Gao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-09 · 47 (9) · 7916-7929

DOI: 10.1109/tpami.2025.3572735 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-09. Verfasst von Zheng Chang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: Funds for International Cooperation and Exchange of the National Natural Science Foundation of China, Key Program of the National Natural Science Foundation of China, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video prediction aims to predict future frames by modeling the complex spatiotemporal dynamics in videos. However, most existing methods only model the temporal information and the spatial information for videos in an independent manner but have not fully explored the correlations between both terms. In this paper, we propose a SpatioTemporal-Aware Unit (STAU) for video prediction and beyond by exploring the significant spatiotemporal correlations in videos. On the one hand, the motion-aware attention weights are learned from the spatial states to help aggregate the temporal states in the temporal domain. On the other hand, the appearance-aware attention weights are learned from the temporal states to help aggregate the spatial states in the spatial domain. In this way, the temporal information and the spatial information can be greatly aware of each other in both domains, during which, the spatiotemporal receptive field can also be greatly broadened for more reliable spatiotemporal modeling. Experiments are not only conducted on video prediction tasks (deterministic and stochastic), but also another task beyond video prediction, the early action recognition task. Experimental results show that the proposed STAU can achieve satisfactory performance on all tasks compared with other methods.

Förderung

  • Funds for International Cooperation and Exchange of the National Natural Science Foundation of China
  • Key Program of the National Natural Science Foundation of China
  • National Natural Science Foundation of China
  • PCL-CMCC Foundation for science and innovation

Zitieren

APA

Chang, Z., Zhang, X., Wang, S., Ma, S., & Gao, W. (2025). STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(9), 7916-7929. https://doi.org/10.1109/tpami.2025.3572735

Vancouver

Chang Z, Zhang X, Wang S, Ma S, Gao W. STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(9):7916-7929. doi:10.1109/tpami.2025.3572735

BibTeX
@article{Chang2025,
  author = {Chang, Zheng and Zhang, Xinfeng and Wang, Shanshe and Ma, Siwei and Gao, Wen},
  title = {STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {9},
  pages = {7916--7929},
  doi = {10.1109/tpami.2025.3572735}
}
Kostenlos starten →