EUVIMEDEUVIMED entdecken
Crossref

An Open-Source Benchmark of Deep Learning Models for Audio-Visual Apparent and Self-Reported Personality Recognition

Rongfan Liao, Siyang Song, Hatice Gunes

IEEE Transactions on Affective Computing · 2024-07 · 15 (3) · 1590-1607

DOI: 10.1109/taffc.2024.3363710 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2024-07. Verfasst von Rongfan Liao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 42-mal zitiert (Crossref) und nennt 86 Literaturangaben. Gefördert von: Affect AI Co., Ltd, EPSRC/UKRI, European Union's Horizon 2020 research and innovation programme project WorkingAge.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. We conduct a comprehensive investigation into all the benchmarked models to demonstrate their capabilities in modelling personality traits on two publicly available datasets, audio-visual apparent personality (ChaLearn First Impression) and self-reported personality (UDIVA) datasets. The experimental results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. We make all the code and settings of this personality computing benchmark publicly available athttps://github.com/liaorongfan/DeepPersonality.

Förderung

  • Affect AI Co., Ltd
  • EPSRC/UKRI
  • European Union's Horizon 2020 research and innovation programme project WorkingAge

Zitieren

APA

Liao, R., Song, S., & Gunes, H. (2024). An Open-Source Benchmark of Deep Learning Models for Audio-Visual Apparent and Self-Reported Personality Recognition. IEEE Transactions on Affective Computing, 15(3), 1590-1607. https://doi.org/10.1109/taffc.2024.3363710

Vancouver

Liao R, Song S, Gunes H. An Open-Source Benchmark of Deep Learning Models for Audio-Visual Apparent and Self-Reported Personality Recognition. IEEE Transactions on Affective Computing. 2024;15(3):1590-1607. doi:10.1109/taffc.2024.3363710

BibTeX
@article{Liao2024,
  author = {Liao, Rongfan and Song, Siyang and Gunes, Hatice},
  title = {An Open-Source Benchmark of Deep Learning Models for Audio-Visual Apparent and Self-Reported Personality Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2024},
  volume = {15},
  number = {3},
  pages = {1590--1607},
  doi = {10.1109/taffc.2024.3363710}
}
Kostenlos starten →