EUVIMEDEUVIMED entdecken
Crossref

Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation

Kaleab A Kinfu, Rene Vidal

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026

DOI: 10.1109/tpami.2026.3701408 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. Verfasst von Kaleab A Kinfu und 1 weiteren Autor:innen. Hauptthemen laut MEDLINE: Image Processing, Computer-Assisted, Posture. Die Arbeit nennt 66 Literaturangaben. Gefördert von: NIH, NSF, IARPA.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have driven significant progress in 2D human pose estimation. However, achieving a good balance between accuracy, efficiency, and robustness remains challenging. For instance, CNNs are computationally efficient but struggle with long-range dependencies, while ViTs excel in capturing such dependencies but suffer from quadratic computational complexity with respect to the number of patch tokens. This paper proposes two ViT-based models for accurate, efficient, and robust 2D pose estimation. The first, EViTPose, improves computational efficiency with minimal accuracy loss by utilizing learnable joint tokens to select and process a subset of patches most relevant to the body joints, thereby enabling control over the accuracy efficiency trade-off by varying the number of processed patches. The second, UniTransPose, while not providing the same direct control over this trade-off, efficiently handles multiple scales by combining (1) an efficient multi-scale transformer encoder using both local and global attention and (2) an efficient sub-pixel CNN decoder for improved speed and accuracy. Furthermore, by incorporating joint annotation schemas from different benchmarks into a unified skeletal representation, we train robust models that learn from multiple datasets simultaneously and perform well across a range of scenarios, including variations in pose, lighting, and occlusion. Experiments on six benchmarks demonstrate that the proposed methods substantially outperform state-ofthe-art methods while improving computational efficiency. EViTPose significantly reduces computational cost (reduces GFLOPs by 30%44%) with minimal drop in accuracy (0% to 3.5%), and UniTransPose achieves accuracy improvements ranging from 0.9% to 43.8% across these benchmarks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Image Processing, Computer-AssistedPostureHumansConvolutional Neural NetworksAlgorithms

Förderung

  • NIH
  • NSF
  • IARPA

Zitieren

APA

Kinfu, K. A., & Vidal, R. (2026). Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2026.3701408

Vancouver

Kinfu KA, Vidal R. Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026. doi:10.1109/tpami.2026.3701408

BibTeX
@article{Kinfu2026,
  author = {Kinfu, Kaleab A and Vidal, Rene},
  title = {Transformers With Joint Tokens and Local-Global Attention for Efficient Human Pose Estimation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  doi = {10.1109/tpami.2026.3701408}
}
Kostenlos starten →