EUVIMEDEUVIMED entdecken
Crossref

Bidirectional Transformer GAN for Long-term Human Motion Prediction

Mengyi Zhao, Hao Tang, Pan Xie, Shuling Dai, Nicu Sebe, Wei Wang

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.09.2023 · 19 (5) · 1-19

DOI: 10.1145/3579359 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 15.04.2023. Verfasst von Mengyi Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: National Key Research & Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The mainstream motion prediction methods usually focus on short-term prediction, and their predicted long-term motions often fall into an average pose, i.e., the freezing forecasting problem [ 27 ]. To mitigate this problem, we propose a novel Bidirectional Transformer-based Generative Adversarial Network (BiTGAN) for long-term human motion prediction. The bidirectional setup leads to consistent and smooth generation in both forward and backward directions. Besides, to make full use of the history motions, we split them into two parts. The first part is fed to the Transformer encoder in our BiTGAN while the second part is used as the decoder input. This strategy can alleviate the exposure problem [ 37 ]. Additionally, to better maintain both the local (i.e., frame-level pose) and global (i.e., video-level semantic) similarities between the predicted motion sequence and the real one, the soft dynamic time warping (Soft-DTW) loss is introduced into the generator. Finally, we utilize a dual-discriminator to distinguish the predicted sequence at both frame and sequence levels. Extensive experiments on the public Human3.6M dataset demonstrate that our proposed BiTGAN achieves state-of-the-art performance on long-term (4 s ) human motion prediction, and reduces the average error of all actions by 4%.

Förderung

  • National Key Research & Development Program of China

Zitieren

APA

Zhao, M., Tang, H., Xie, P., Dai, S., Sebe, N., & Wang, W. (2023). Bidirectional Transformer GAN for Long-term Human Motion Prediction. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(5), 1-19. https://doi.org/10.1145/3579359

Vancouver

Zhao M, Tang H, Xie P, Dai S, Sebe N, Wang W. Bidirectional Transformer GAN for Long-term Human Motion Prediction. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(5):1-19. doi:10.1145/3579359

BibTeX
@article{Zhao2023,
  author = {Zhao, Mengyi and Tang, Hao and Xie, Pan and Dai, Shuling and Sebe, Nicu and Wang, Wei},
  title = {Bidirectional Transformer GAN for Long-term Human Motion Prediction},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2023},
  volume = {19},
  number = {5},
  pages = {1--19},
  doi = {10.1145/3579359}
}
Kostenlos starten →