EUVIMEDEUVIMED entdecken
Crossref

Contrastive Learning of View-invariant Representations for Facial Expressions Recognition

Shuvendu Roy, Ali Etemad

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.04.2024 · 20 (4) · 1-22

DOI: 10.1145/3632960 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.12.2023. Verfasst von Shuvendu Roy und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 84 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Although there has been much progress in the area of facial expression recognition (FER), most existing methods suffer when presented with images that have been captured from viewing angles that are non-frontal and substantially different from those used in the training process. In this article, we propose ViewFX, a novel view-invariant FER framework based on contrastive learning, capable of accurately classifying facial expressions regardless of the input viewing angles during inference. ViewFX learns view-invariant features of expression using a proposed self-supervised contrastive loss, which brings together different views of the same subject with a particular expression in the embedding space. We also introduce a supervised contrastive loss to push the learned view-invariant features of each expression away from other expressions. Since facial expressions are often distinguished with very subtle differences in the learned feature space, we incorporate the Barlow twins loss to reduce the redundancy and correlations of the representations in the learned representations. The proposed method is a substantial extension of our previously proposed CL-MEx, which only had a self-supervised loss. We test the proposed framework on two public multi-view facial expression recognition datasets, KDEF and DDCF. The experiments demonstrate that our approach outperforms previous works in the area and sets a new state-of-the-art for both datasets while showing considerably less sensitivity to challenging angles and the number of output labels used for training. We also perform detailed sensitivity and ablation experiments to evaluate the impact of different components of our model as well as its sensitivity to different parameters.

Zitieren

APA

Roy, S., & Etemad, A. (2024). Contrastive Learning of View-invariant Representations for Facial Expressions Recognition. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(4), 1-22. https://doi.org/10.1145/3632960

Vancouver

Roy S, Etemad A. Contrastive Learning of View-invariant Representations for Facial Expressions Recognition. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(4):1-22. doi:10.1145/3632960

BibTeX
@article{Roy2024,
  author = {Roy, Shuvendu and Etemad, Ali},
  title = {Contrastive Learning of View-invariant Representations for Facial Expressions Recognition},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2024},
  volume = {20},
  number = {4},
  pages = {1--22},
  doi = {10.1145/3632960}
}
Kostenlos starten →