EUVIMEDEUVIMED entdecken
Crossref

Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach

Hua Yu, Xuanzhe Fan, Yaqing Hou, Wenbin Pei, Hongwei Ge, Xin Yang, Dongsheng Zhou, Qiang Zhang, Mengjie Zhang

IEEE Transactions on Circuits and Systems for Video Technology · 2023-10 · 33 (10) · 5707-5720

DOI: 10.1109/tcsvt.2023.3255186 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-10. Verfasst von Hua Yu und 8 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China (NSFC)-Liaoning Province United Foundation, NSFC.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human motion prediction intends to predict how humans move given a historical sequence of 3D human motions. Recent transformer-based methods have attracted increasing attentions and demonstrated their promising performance in 3D human motion prediction. However, existing methods generally decompose the input of human motion information into spatial and temporal branches in a separate way and seldom consider their inherent coherence between the two branches, hence often failing to register the dynamic spatio-temporal information during the training process. Motivated by these issues, we propose a spatio-temporal cross-transformer network (STCT) for 3D human motion predictions. Specifically, we investigate various types of interaction methods (i.e., Concatenation Interaction, Msg token interaction, and Cross-transformer) to capture the coherence of the spatial and temporal branches. According to the obtained results, the proposed cross-transformer interaction method shows its superiority over other methods. Meanwhile, considering that most existing works treat the human body as a set of 3D human joint positions, the predicted human joints are proportionally less appropriate to the realistic human body due to unreasonable bone length and non-plausible poses as time progresses. We further resort to the bone constraints of human mesh to produce more realistic human motions. By fitting a parametric body model (i.e., SMPL-X model) to the predicted human joints, a reconstruction loss function is proposed to remedy the unreasonable bone length and pose errors. Comprehensive experiments on AMASS and Human3.6M datasets have demonstrated that our method achieves superior performance over compared methods.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China (NSFC)-Liaoning Province United Foundation
  • NSFC
  • Fundamental Research Funds for the Central Universities
  • Liaoning Revitalization Talents Program

Zitieren

APA

Yu, H., Fan, X., Hou, Y., Pei, W., Ge, H., Yang, X., Zhou, D., Zhang, Q., & Zhang, M. (2023). Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach. IEEE Transactions on Circuits and Systems for Video Technology, 33(10), 5707-5720. https://doi.org/10.1109/tcsvt.2023.3255186

Vancouver

Yu H, Fan X, Hou Y, Pei W, Ge H, Yang X, et al. Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(10):5707-5720. doi:10.1109/tcsvt.2023.3255186

BibTeX
@article{Yu2023,
  author = {Yu, Hua and Fan, Xuanzhe and Hou, Yaqing and Pei, Wenbin and Ge, Hongwei and Yang, Xin and Zhou, Dongsheng and Zhang, Qiang and Zhang, Mengjie},
  title = {Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2023},
  volume = {33},
  number = {10},
  pages = {5707--5720},
  doi = {10.1109/tcsvt.2023.3255186}
}
Kostenlos starten →