EUVIMEDEUVIMED entdecken
Crossref

Motion Semantic Enhancement and Autonomous Information Mining for Static-Dynamic Visual Emotion Recognition in Human–Robot Interaction

Cheng-Shan Jiang, Zhen-Tao Liu, Edwardo F. Fukushima, Jinhua She

IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-15

DOI: 10.1109/tim.2025.3578178 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Cheng-Shan Jiang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hubei Provincial Natural Science Foundation of China, Wuhan Applied Basic Research Programs Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The advancement of static-dynamic visual emotion recognition plays a pivotal role in the evolution of intelligent and empathetic machines. Currently, the progress of static visual emotion recognition (SVER) faces challenges, primarily the need to balance model efficiency with robust recognition performance. Moreover, obstacles such as noisy data and limited labeled datasets restrict models from effectively learning the appearance features and dynamic dependencies intrinsic to dynamic visual emotion recognition (DVER). In the realm of SVER, facial pixel and semantic representation are derived using a lightweight surface and landmark features embedding network, followed by neuron-energy-based feature fusion-filtering to enhance the fusion of semantic representations positively. For DVER, a Global Anchor-dependent Noisy Emotional Data Filtering (GADNEF) method is employed for noisy label learning, facilitating clip-wise filtering of ambiguous data via iterative computations of frame-wise attention statistics across batches. Furthermore, a self-supervised learning paradigm based on a unified appearance and motion-guided masked autoencoder is implemented, enabling large-scale knowledge transfer tailored for video-based facial expression analysis. Our approach has achieved accuracies of 97.07%, 98.90%, 61.39%, and 92.28% on SVER datasets including KDEF, RaFD, SFEW, and RAF-DB, respectively, while the weighted average recall (WAR) and unweighted average recall (UAR) on DVER datasets such as DFEW and MAFW were 75.12%, 64.37%, 55.26%, and 42.53%, respectively. A preliminary application experiment has also been conducted to validate the practical applicability of our method within human-robot interaction (HRI) scenarios.

Förderung

  • National Natural Science Foundation of China
  • Hubei Provincial Natural Science Foundation of China
  • Wuhan Applied Basic Research Programs Project
  • Wuhan Applied Foundational Frontier Project
  • Fundamental Research Funds for National Universities, China University of Geosciences, Wuhan
  • 111 Project, China

Zitieren

APA

Jiang, C., Liu, Z., Fukushima, E. F., & She, J. (2025). Motion Semantic Enhancement and Autonomous Information Mining for Static-Dynamic Visual Emotion Recognition in Human–Robot Interaction. IEEE Transactions on Instrumentation and Measurement, 74, 1-15. https://doi.org/10.1109/tim.2025.3578178

Vancouver

Jiang C, Liu Z, Fukushima EF, She J. Motion Semantic Enhancement and Autonomous Information Mining for Static-Dynamic Visual Emotion Recognition in Human–Robot Interaction. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-15. doi:10.1109/tim.2025.3578178

BibTeX
@article{Jiang2025,
  author = {Jiang, Cheng-Shan and Liu, Zhen-Tao and Fukushima, Edwardo F. and She, Jinhua},
  title = {Motion Semantic Enhancement and Autonomous Information Mining for Static-Dynamic Visual Emotion Recognition in Human–Robot Interaction},
  journal = {IEEE Transactions on Instrumentation and Measurement},
  year = {2025},
  volume = {74},
  pages = {1--15},
  doi = {10.1109/tim.2025.3578178}
}
Kostenlos starten →