EUVIMEDEUVIMED entdecken
Crossref

Reversible Graph Neural Network-Based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation

Tong Xu, Micol Spitale, Hao Tang, Lu Liu, Hatice Gunes, Siyang Song

IEEE Transactions on Affective Computing · 2026-07 · 17 (3) · 3563-3578

DOI: 10.1109/taffc.2026.3682774 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2026-07. Verfasst von Tong Xu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 61 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generating facial reactions in a human-human dyadic interaction is complex and highly dependent on the context since more than one facial reactions can be appropriate for the speaker's behaviour. This has challenged existing machine learning (ML) methods, whose training strategies enforce models to reproduce a specific (not multiple) facial reaction from each input speaker behaviour. This paper proposes the first multiple appropriate facial reaction generation (MAFRG) framework which re-formulates the one-to-many mapping facial reaction generation problem as a one-to-one mapping problem. This means that we approach this problem by considering generating a distribution of listeners' appropriate facial reactions instead of multiple different appropriate facial reactions, i.e., ‘many’ appropriate facial reaction labels are summarised as ‘one’ distribution label during training. Our model consists of a perceptual processor, a cognitive processor, and a motor processor. The motor processor is implemented with a novel Reversible Multi-dimensional Edge Graph Neural Network (REGNN). This allows us to obtain a distribution of appropriate real facial reactions during the training process, enabling the cognitive processor to be trained to predict the appropriate facial reaction distribution. At the inference stage, the REGNN decodes an appropriate facial reaction by using the predicted distribution as input. Experimental results demonstrate that our approach outperforms existing models in generating more appropriate, realistic, and synchronized facial reactions. The improved performance is largely attributed to the proposed appropriate facial reaction distribution learning strategy and the use of a REGNN. The code will be made publicly available at https://github.com/TongXu-05/REGNN-Multiple-Appropriate-Facial-Reaction-Generation.

Zitieren

APA

Xu, T., Spitale, M., Tang, H., Liu, L., Gunes, H., & Song, S. (2026). Reversible Graph Neural Network-Based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation. IEEE Transactions on Affective Computing, 17(3), 3563-3578. https://doi.org/10.1109/taffc.2026.3682774

Vancouver

Xu T, Spitale M, Tang H, Liu L, Gunes H, Song S. Reversible Graph Neural Network-Based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation. IEEE Transactions on Affective Computing. 2026;17(3):3563-3578. doi:10.1109/taffc.2026.3682774

BibTeX
@article{Xu2026,
  author = {Xu, Tong and Spitale, Micol and Tang, Hao and Liu, Lu and Gunes, Hatice and Song, Siyang},
  title = {Reversible Graph Neural Network-Based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation},
  journal = {IEEE Transactions on Affective Computing},
  year = {2026},
  volume = {17},
  number = {3},
  pages = {3563--3578},
  doi = {10.1109/taffc.2026.3682774}
}
Kostenlos starten →