VT-3DCapsNet: Visual tempos 3D-Capsule network for video-based facial expression recognition
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0307446 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Zhuan Li und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Information Engineering, Shanghai Maritime University, Shanghai, China. Hauptthemen laut MEDLINE: Facial Expression, Neural Networks, Computer. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Technologies Research and Development Program of China, National Social Science Foundation of China.
Abstract (Originalsprache)
Facial expression recognition(FER) is a hot topic in computer vision, especially as deep learning based methods are gaining traction in this field. However, traditional convolutional neural networks (CNN) ignore the relative position relationship of key facial features (mouth, eyebrows, eyes, etc.) due to changes of facial expressions in real-world environments such as rotation, displacement or partial occlusion. In addition, most of the works in the literature do not take visual tempos into account when recognizing facial expressions that possess higher similarities. To address these issues, we propose a visual tempos 3D-CapsNet framework(VT-3DCapsNet). First, we propose 3D-CapsNet model for emotion recognition, in which we introduced improved 3D-ResNet architecture that integrated with AU-perceived attention module to enhance the ability of feature representation of capsule network, through expressing deeper hierarchical spatiotemporal features and extracting latent information (position, size, orientation) in key facial areas. Furthermore, we propose the temporal pyramid network(TPN)-based expression recognition module(TPN-ERM), which can learn high-level facial motion features from video frames to model differences in visual tempos, further improving the recognition accuracy of 3D-CapsNet. Extensive experiments are conducted on extended Kohn-Kanada (CK+) database and Acted Facial Expression in Wild (AFEW) database. The results demonstrate competitive performance of our approach compared with other state-of-the-art methods.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Information Engineering, Shanghai Maritime University, Shanghai, China.
- National Engineering Research Center of Ship Transportation Control Systems, Shanghai Ship and Shipping Research Institute, Shanghai, China.
ORCID-Profile: Jin Liu
Förderung
- National Key Technologies Research and Development Program of China
- National Social Science Foundation of China
Zitieren
APA
Li, Z., Liu, J., Wang, H., Zhang, X., Wu, Z., & Han, B. (2024). VT-3DCapsNet: Visual tempos 3D-Capsule network for video-based facial expression recognition. PLOS ONE. https://doi.org/10.1371/journal.pone.0307446
Vancouver
Li Z, Liu J, Wang H, Zhang X, Wu Z, Han B. VT-3DCapsNet: Visual tempos 3D-Capsule network for video-based facial expression recognition. PLOS ONE. 2024. doi:10.1371/journal.pone.0307446
BibTeX
@article{Li2024,
author = {Li, Zhuan and Liu, Jin and Wang, Hengyang and Zhang, Xiliang and Wu, Zhongdai and Han, Bing},
title = {VT-3DCapsNet: Visual tempos 3D-Capsule network for video-based facial expression recognition},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0307446}
}
EUVIMED