EUVIMEDEUVIMED entdecken
Crossref

WiFE: WiFi and Vision Based Unobtrusive Emotion Recognition via Gesture and Facial Expression

Yu Gu, Xiang Zhang, Huan Yan, Jinyang Huang, Zhi Liu, Mianxiong Dong, Fuji Ren

IEEE Transactions on Affective Computing · 2023-10 · 14 (4) · 2567-2581

DOI: 10.1109/taffc.2023.3285777 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2023-10. Verfasst von Yu Gu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 75 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Emotion plays a critical role in making the computer more human-like. As the first and most essential step, emotion recognition emerges recently as a hot but relatively nascent topic, i.e., current research mainly focuses on single modality (e.g., facial expression) while human emotion expressions are multi-modal in nature. To this end, we propose an unobtrusive emotion recognition system leveraging two emotion-rich and tightly-coupled modalities, i.e., gesture and facial expression. The system design faces two major challenges, namely, how to capture the emotional expression in both modalities without disturbing the subject and how to leverage the relationship between modalities for recognizing the emotion. For the former, we explore WiFi and vision for unobtrusive and contactless gesture and facial expression sensing, respectively. For the latter, we propose a novel deep learning framework named Multi-Source Learning (MSL) to efficiently exploit both self-correlation in the modality and cross-correlation between modalities for fine-grained emotion recognition. To evaluate the proposed method, we prototype the system on low-cost commodity WiFi and vision devices, build a first-of-its-kind WiFi-Vision emotion dataset, and conduct extensive experiments. Empirical results not only verify the effectiveness of WiFE in emotion recognition, but also confirm the superiority of multi-modality over single-modality.

Zitieren

APA

Gu, Y., Zhang, X., Yan, H., Huang, J., Liu, Z., Dong, M., & Ren, F. (2023). WiFE: WiFi and Vision Based Unobtrusive Emotion Recognition via Gesture and Facial Expression. IEEE Transactions on Affective Computing, 14(4), 2567-2581. https://doi.org/10.1109/taffc.2023.3285777

Vancouver

Gu Y, Zhang X, Yan H, Huang J, Liu Z, Dong M, et al. WiFE: WiFi and Vision Based Unobtrusive Emotion Recognition via Gesture and Facial Expression. IEEE Transactions on Affective Computing. 2023;14(4):2567-2581. doi:10.1109/taffc.2023.3285777

BibTeX
@article{Gu2023,
  author = {Gu, Yu and Zhang, Xiang and Yan, Huan and Huang, Jinyang and Liu, Zhi and Dong, Mianxiong and Ren, Fuji},
  title = {WiFE: WiFi and Vision Based Unobtrusive Emotion Recognition via Gesture and Facial Expression},
  journal = {IEEE Transactions on Affective Computing},
  year = {2023},
  volume = {14},
  number = {4},
  pages = {2567--2581},
  doi = {10.1109/taffc.2023.3285777}
}
Kostenlos starten →