A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation
IEEE Transactions on Affective Computing · 01.01.2023 · 14 (1) · 436-450
DOI: 10.1109/taffc.2021.3061605 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, erschienen am 01.01.2023. Verfasst von Ioanna Ntinou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: Horizon Centre for Doctoral Training, University of Nottingham, School of Electronic Engineering and Computer Science.
Abstract (Originalsprache)
Action Units (AUs) are geometrically-based atomic facial muscle movements known to produce appearance changes at specific facial locations. Motivated by this observation we propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. To this end, we propose a simple yet efficient approach based on Heatmap Regression that merges both problems into a single task. A Heatmap models whether an AU occurs or not at a given spatial location. To accommodate the joint modelling of AUs intensity, we propose variable size heatmaps, with their amplitude and size varying according to the labelled intensity. Using Heatmap Regression, we can inherit from the progress recently witnessed in facial landmark localisation. Building upon the similarities between both problems, we devise a transfer learning approach where we exploit the knowledge of a network trained on large-scale facial landmark datasets. In particular, we explore different alternatives for transfer learning through a) fine-tuning, b) adaptation layers, c) attention maps, and d) reparametrisation. Our approach effectively inherits the rich facial features produced by a strong face alignment network, with minimal extra computational cost. We empirically validate that our system sets a new state-of-the-art on three popular datasets, namely BP4D, DISFA, and FERA2017.
Förderung
- Horizon Centre for Doctoral Training
- University of Nottingham
- School of Electronic Engineering and Computer Science
- Queen Mary University of London
- NIHR Nottingham Biomedical Research Centre
Zitieren
APA
Ntinou, I., Sanchez, E., Bulat, A., Valstar, M., & Tzimiropoulos, G. (2023). A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation. IEEE Transactions on Affective Computing, 14(1), 436-450. https://doi.org/10.1109/taffc.2021.3061605
Vancouver
Ntinou I, Sanchez E, Bulat A, Valstar M, Tzimiropoulos G. A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation. IEEE Transactions on Affective Computing. 2023;14(1):436-450. doi:10.1109/taffc.2021.3061605
BibTeX
@article{Ntinou2023,
author = {Ntinou, Ioanna and Sanchez, Enrique and Bulat, Adrian and Valstar, Michel and Tzimiropoulos, Georgios},
title = {A Transfer Learning Approach to Heatmap Regression for Action Unit Intensity Estimation},
journal = {IEEE Transactions on Affective Computing},
year = {2023},
volume = {14},
number = {1},
pages = {436--450},
doi = {10.1109/taffc.2021.3061605}
}
EUVIMED