Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026
DOI: 10.1109/tpami.2026.3701408 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. Verfasst von Kaleab A Kinfu und 1 weiteren Autor:innen. Hauptthemen laut MEDLINE: Image Processing, Computer-Assisted, Posture. Die Arbeit nennt 66 Literaturangaben. Gefördert von: NIH, NSF, IARPA.
Abstract (Originalsprache)
Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have driven significant progress in 2D human pose estimation. However, achieving a good balance between accuracy, efficiency, and robustness remains challenging. For instance, CNNs are computationally efficient but struggle with long-range dependencies, while ViTs excel in capturing such dependencies but suffer from quadratic computational complexity with respect to the number of patch tokens. This paper proposes two ViT-based models for accurate, efficient, and robust 2D pose estimation. The first, EViTPose, improves computational efficiency with minimal accuracy loss by utilizing learnable joint tokens to select and process a subset of patches most relevant to the body joints, thereby enabling control over the accuracy efficiency trade-off by varying the number of processed patches. The second, UniTransPose, while not providing the same direct control over this trade-off, efficiently handles multiple scales by combining (1) an efficient multi-scale transformer encoder using both local and global attention and (2) an efficient sub-pixel CNN decoder for improved speed and accuracy. Furthermore, by incorporating joint annotation schemas from different benchmarks into a unified skeletal representation, we train robust models that learn from multiple datasets simultaneously and perform well across a range of scenarios, including variations in pose, lighting, and occlusion. Experiments on six benchmarks demonstrate that the proposed methods substantially outperform state-ofthe-art methods while improving computational efficiency. EViTPose significantly reduces computational cost (reduces GFLOPs by 30%44%) with minimal drop in accuracy (0% to 3.5%), and UniTransPose achieves accuracy improvements ranging from 0.9% to 43.8% across these benchmarks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- NIH
- NSF
- IARPA
Zitieren
APA
Kinfu, K. A., & Vidal, R. (2026). Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2026.3701408
Vancouver
Kinfu KA, Vidal R. Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026. doi:10.1109/tpami.2026.3701408
BibTeX
@article{Kinfu2026,
author = {Kinfu, Kaleab A and Vidal, Rene},
title = {Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
doi = {10.1109/tpami.2026.3701408}
}
EUVIMED