EUVIMEDEUVIMED entdecken
Crossref

A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation

Ioanna Ntinou, Enrique Sanchez, Adrian Bulat, Michel Valstar, Georgios Tzimiropoulos

IEEE Transactions on Affective Computing · 01.01.2023 · 14 (1) · 436-450

DOI: 10.1109/taffc.2021.3061605 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, erschienen am 01.01.2023. Verfasst von Ioanna Ntinou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: Horizon Centre for Doctoral Training, University of Nottingham, School of Electronic Engineering and Computer Science.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Action Units (AUs) are geometrically-based atomic facial muscle movements known to produce appearance changes at specific facial locations. Motivated by this observation we propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. To this end, we propose a simple yet efficient approach based on Heatmap Regression that merges both problems into a single task. A Heatmap models whether an AU occurs or not at a given spatial location. To accommodate the joint modelling of AUs intensity, we propose variable size heatmaps, with their amplitude and size varying according to the labelled intensity. Using Heatmap Regression, we can inherit from the progress recently witnessed in facial landmark localisation. Building upon the similarities between both problems, we devise a transfer learning approach where we exploit the knowledge of a network trained on large-scale facial landmark datasets. In particular, we explore different alternatives for transfer learning through a) fine-tuning, b) adaptation layers, c) attention maps, and d) reparametrisation. Our approach effectively inherits the rich facial features produced by a strong face alignment network, with minimal extra computational cost. We empirically validate that our system sets a new state-of-the-art on three popular datasets, namely BP4D, DISFA, and FERA2017.

Förderung

  • Horizon Centre for Doctoral Training
  • University of Nottingham
  • School of Electronic Engineering and Computer Science
  • Queen Mary University of London
  • NIHR Nottingham Biomedical Research Centre

Zitieren

APA

Ntinou, I., Sanchez, E., Bulat, A., Valstar, M., & Tzimiropoulos, G. (2023). A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation. IEEE Transactions on Affective Computing, 14(1), 436-450. https://doi.org/10.1109/taffc.2021.3061605

Vancouver

Ntinou I, Sanchez E, Bulat A, Valstar M, Tzimiropoulos G. A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation. IEEE Transactions on Affective Computing. 2023;14(1):436-450. doi:10.1109/taffc.2021.3061605

BibTeX
@article{Ntinou2023,
  author = {Ntinou, Ioanna and Sanchez, Enrique and Bulat, Adrian and Valstar, Michel and Tzimiropoulos, Georgios},
  title = {A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation},
  journal = {IEEE Transactions on Affective Computing},
  year = {2023},
  volume = {14},
  number = {1},
  pages = {436--450},
  doi = {10.1109/taffc.2021.3061605}
}
Kostenlos starten →