MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba
Sensors · 2024
DOI: 10.3390/s24248158 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Jianqiang Zhang und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China. Hauptthemen laut MEDLINE: Posture, Algorithms, Neural Networks, Computer. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, National Natural Science Foundation of China and Shanxi Provincial 474 Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of 477 Sciences.
Abstract (Originalsprache)
Human pose estimation is an important research direction in the field of computer vision, which aims to accurately identify the position and posture of keypoints of the human body through images or videos. However, multi-person pose estimation yields false detection or missed detection in dense crowds, and it is still difficult to detect small targets. In this paper, we propose a Mamba-based human pose estimation. First, we design a GMamba structure to be used as a backbone network to extract human keypoints. A gating mechanism is introduced into the linear layer of Mamba, which allows the model to dynamically adjust the weights according to the different input images to locate the human keypoints more precisely. Secondly, GMamba as the backbone network can effectively solve the long-sequence problem. The direct use of convolutional downsampling reduces selectivity for different stages of information flow. We used slice downsampling (SD) to reduce the resolution of the feature map to half the original size, and then fused local features from four different locations. The fusion of multi-channel information helped the model obtain rich pose information. Finally, we introduced an adaptive threshold focus loss (ATFL) to dynamically adjust the weights of different keypoints. We assigned higher weights to error-prone keypoints to strengthen the model’s attention to these points. Thus, we effectively improved the accuracy of keypoint identification in cases of occlusion, complex background, etc., and significantly improved the overall performance of attitude estimation and anti-interference ability. Experimental results showed that the AP and AP50 of the proposed algorithm on the COCO 2017 validation set were 72.2 and 92.6. Compared with the typical algorithm, it was improved by 1.1% on AP50. The proposed method can effectively detect the keypoints of the human body, and provides stronger robustness and accuracy for the estimation of human posture in complex scenes.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China.
- College of Modern Urban Construction Industry, Tianjin Chengjian University, Tianjin 300384, China.
Förderung
- National Natural Science Foundation of China
- National Natural Science Foundation of China and Shanxi Provincial 474 Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon
- Shanghai Institute of Microsystem and Information Technology, Chinese Academy of 477 Sciences
- National Natural Science Foundation of China and Shanxi Provincial Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon
- Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences
- Guangdong Province
- School-Level Scientific Research Project of Guangdong Institute of Petrochemical Technology
Zitieren
APA
Zhang, J., Hou, J., He, Q., Yuan, Z., & Xue, H. (2024). MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba. Sensors. https://doi.org/10.3390/s24248158
Vancouver
Zhang J, Hou J, He Q, Yuan Z, Xue H. MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba. Sensors. 2024. doi:10.3390/s24248158
BibTeX
@article{Zhang2024,
author = {Zhang, Jianqiang and Hou, Jing and He, Qiusheng and Yuan, Zhengwei and Xue, Hao},
title = {MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24248158}
}
EUVIMED