EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba

Jianqiang Zhang, Jing Hou, Qiusheng He, Zhengwei Yuan, Hao Xue

Sensors · 2024

DOI: 10.3390/s24248158 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Jianqiang Zhang und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China. Hauptthemen laut MEDLINE: Posture, Algorithms, Neural Networks, Computer. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, National Natural Science Foundation of China and Shanxi Provincial 474 Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of 477 Sciences.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human pose estimation is an important research direction in the field of computer vision, which aims to accurately identify the position and posture of keypoints of the human body through images or videos. However, multi-person pose estimation yields false detection or missed detection in dense crowds, and it is still difficult to detect small targets. In this paper, we propose a Mamba-based human pose estimation. First, we design a GMamba structure to be used as a backbone network to extract human keypoints. A gating mechanism is introduced into the linear layer of Mamba, which allows the model to dynamically adjust the weights according to the different input images to locate the human keypoints more precisely. Secondly, GMamba as the backbone network can effectively solve the long-sequence problem. The direct use of convolutional downsampling reduces selectivity for different stages of information flow. We used slice downsampling (SD) to reduce the resolution of the feature map to half the original size, and then fused local features from four different locations. The fusion of multi-channel information helped the model obtain rich pose information. Finally, we introduced an adaptive threshold focus loss (ATFL) to dynamically adjust the weights of different keypoints. We assigned higher weights to error-prone keypoints to strengthen the model’s attention to these points. Thus, we effectively improved the accuracy of keypoint identification in cases of occlusion, complex background, etc., and significantly improved the overall performance of attitude estimation and anti-interference ability. Experimental results showed that the AP and AP50 of the proposed algorithm on the COCO 2017 validation set were 72.2 and 92.6. Compared with the typical algorithm, it was improved by 1.1% on AP50. The proposed method can effectively detect the keypoints of the human body, and provides stronger robustness and accuracy for the estimation of human posture in complex scenes.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PostureAlgorithmsNeural Networks, ComputerHumansImage Processing, Computer-Assisted

Stichwörter der Autor:innen

Mambadownsamplingfeature fusionloss functionpose estimation

Autor:innen und Institutionen

  • School of Electronic Information Engineering, Taiyuan University of Science and Technology, Taiyuan 030024, China.
  • College of Modern Urban Construction Industry, Tianjin Chengjian University, Tianjin 300384, China.

Förderung

  • National Natural Science Foundation of China
  • National Natural Science Foundation of China and Shanxi Provincial 474 Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon
  • Shanghai Institute of Microsystem and Information Technology, Chinese Academy of 477 Sciences
  • National Natural Science Foundation of China and Shanxi Provincial Peoples Government Jointly Funded Project of China for Coal Base and Low Carbon
  • Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences
  • Guangdong Province
  • School-Level Scientific Research Project of Guangdong Institute of Petrochemical Technology

Zitieren

APA

Zhang, J., Hou, J., He, Q., Yuan, Z., & Xue, H. (2024). MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba. Sensors. https://doi.org/10.3390/s24248158

Vancouver

Zhang J, Hou J, He Q, Yuan Z, Xue H. MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba. Sensors. 2024. doi:10.3390/s24248158

BibTeX
@article{Zhang2024,
  author = {Zhang, Jianqiang and Hou, Jing and He, Qiusheng and Yuan, Zhengwei and Xue, Hao},
  title = {MambaPose: A Human Pose Estimation Based on Gated Feedforward Network and Mamba},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24248158}
}
Kostenlos starten →