Bidirectional Transformer GAN for Long-term Human Motion Prediction
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.09.2023 · 19 (5) · 1-19
DOI: 10.1145/3579359 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 15.04.2023. Verfasst von Mengyi Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 49 Literaturangaben. Gefördert von: National Key Research & Development Program of China.
Abstract (Originalsprache)
The mainstream motion prediction methods usually focus on short-term prediction, and their predicted long-term motions often fall into an average pose, i.e., the freezing forecasting problem [ 27 ]. To mitigate this problem, we propose a novel Bidirectional Transformer-based Generative Adversarial Network (BiTGAN) for long-term human motion prediction. The bidirectional setup leads to consistent and smooth generation in both forward and backward directions. Besides, to make full use of the history motions, we split them into two parts. The first part is fed to the Transformer encoder in our BiTGAN while the second part is used as the decoder input. This strategy can alleviate the exposure problem [ 37 ]. Additionally, to better maintain both the local (i.e., frame-level pose) and global (i.e., video-level semantic) similarities between the predicted motion sequence and the real one, the soft dynamic time warping (Soft-DTW) loss is introduced into the generator. Finally, we utilize a dual-discriminator to distinguish the predicted sequence at both frame and sequence levels. Extensive experiments on the public Human3.6M dataset demonstrate that our proposed BiTGAN achieves state-of-the-art performance on long-term (4 s ) human motion prediction, and reduces the average error of all actions by 4%.
Förderung
- National Key Research & Development Program of China
Zitieren
APA
Zhao, M., Tang, H., Xie, P., Dai, S., Sebe, N., & Wang, W. (2023). Bidirectional Transformer GAN for Long-term Human Motion Prediction. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(5), 1-19. https://doi.org/10.1145/3579359
Vancouver
Zhao M, Tang H, Xie P, Dai S, Sebe N, Wang W. Bidirectional Transformer GAN for Long-term Human Motion Prediction. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(5):1-19. doi:10.1145/3579359
BibTeX
@article{Zhao2023,
author = {Zhao, Mengyi and Tang, Hao and Xie, Pan and Dai, Shuling and Sebe, Nicu and Wang, Wei},
title = {Bidirectional Transformer GAN for Long-term Human Motion Prediction},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {5},
pages = {1--19},
doi = {10.1145/3579359}
}
EUVIMED