EUVIMEDEUVIMED entdecken
Crossref

Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation

Hui Shuai, Lele Wu, Qingshan Liu

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.04.2023 · 45 (4) · 4122-4135

DOI: 10.1109/tpami.2022.3188716 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.04.2023. Verfasst von Hui Shuai und 2 weiteren Autor:innen. Die Arbeit wurde bisher 87-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Science and Technology Program of Jiangsu Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article proposes a unified framework dubbed Multi-view and Temporal Fusing Transformer (MTF-Transformer) to adaptively handle varying view numbers and video length without camera calibration in 3D Human Pose Estimation (HPE). It consists of Feature Extractor, Multi-view Fusing Transformer (MFT), and Temporal Fusing Transformer (TFT). Feature Extractor estimates 2D pose from each image and fuses the prediction according to the confidence. It provides pose-focused feature embedding and makes subsequent modules computationally lightweight. MFT fuses the features of a varying number of views with a novel Relative-Attention block. It adaptively measures the implicit relative relationship between each pair of views and reconstructs more informative features. TFT aggregates the features of the whole sequence and predicts 3D pose via a transformer. It adaptively deals with the video of arbitrary length and fully unitizes the temporal information. The migration of transformers enables our model to learn spatial geometry better and preserve robustness for varying application scenarios. We report quantitative and qualitative results on the Human3.6M, TotalCapture, and KTH Multiview Football II. Compared with state-of-the-art methods with camera parameters, MTF-Transformer obtains competitive results and generalizes well to dynamic capture with an arbitrary number of unseen views.

Förderung

  • National Natural Science Foundation of China
  • Science and Technology Program of Jiangsu Province

Zitieren

APA

Shuai, H., Wu, L., & Liu, Q. (2023). Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4), 4122-4135. https://doi.org/10.1109/tpami.2022.3188716

Vancouver

Shuai H, Wu L, Liu Q. Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(4):4122-4135. doi:10.1109/tpami.2022.3188716

BibTeX
@article{Shuai2023,
  author = {Shuai, Hui and Wu, Lele and Liu, Qingshan},
  title = {Adaptive Multi-View and Temporal Fusing Transformer for 3D Human Pose Estimation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {4},
  pages = {4122--4135},
  doi = {10.1109/tpami.2022.3188716}
}
Kostenlos starten →