EUVIMEDEUVIMED entdecken
Crossref

Spatio-Temporal Disentanglement and Constrained Self-Attention for Multi-Modal Deception Detection

Zhiwen Shao, Hang Yang, Hancheng Zhu, Rui Yao, Lixin Zou, Mengtian Li, Bin Sheng

ACM Transactions on Multimedia Computing, Communications, and Applications · 28.02.2026 · 22 (2) · 1-20

DOI: 10.1145/3783994 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 09.02.2026. Verfasst von Zhiwen Shao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Science and Technology Program of Xuzhou, National Natural Science Foundation of China, Opening Fund of the State Key Laboratory of Industrial Control Technology, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal deception detection is a challenging yet important task, having pivotal applications in many fields such as business credibility assessment and multimedia anti-frauds. Previous methods either rely solely on spatial features or overemphasize only temporal information within or across modalities, which may overlook potential critical clues. Motivated by these observations, we propose a Spatio-Temporal Representation Disentanglement (STRD) framework for multi-modal deception detection, which uses a dual-encoder structure to learn spatial and temporal representations for each modality. Specifically, we introduce a pre-trained foundation model to act as the spatial encoder and design a lightweight network as the temporal encoder, extracting spatial semantics and capturing dynamic temporal patterns. Then, we propose a Constrained Self-Attention Block (CSAB), in which self-attention distribution of each head is regarded as spatial distribution and is constrained to attend a certain facial local region. Furthermore, we present a Cross-Modal Correlation Fusion Block (CCFB) to achieve temporal synchronization across modalities by measuring the correlations between visual and audio features. Extensive experiments show that our STRD outperforms the state-of-the-art methods on challenging DOLOS, BOL, BgOL, and RLtrial benchmarks. Particularly, STRD improves by 2.12% and 1.88% over the previous best results in terms of ACC on the DOLOS and BOL datasets, respectively. Additionally, STRD outperforms previous methods in cross-dataset testing, highlighting its superior generalization ability.

Förderung

  • Science and Technology Program of Xuzhou
  • National Natural Science Foundation of China
  • Opening Fund of the State Key Laboratory of Industrial Control Technology, China
  • Natural Science Foundation of Shanghai

Zitieren

APA

Shao, Z., Yang, H., Zhu, H., Yao, R., Zou, L., Li, M., & Sheng, B. (2026). Spatio-Temporal Disentanglement and Constrained Self-Attention for Multi-Modal Deception Detection. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(2), 1-20. https://doi.org/10.1145/3783994

Vancouver

Shao Z, Yang H, Zhu H, Yao R, Zou L, Li M, et al. Spatio-Temporal Disentanglement and Constrained Self-Attention for Multi-Modal Deception Detection. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(2):1-20. doi:10.1145/3783994

BibTeX
@article{Shao2026,
  author = {Shao, Zhiwen and Yang, Hang and Zhu, Hancheng and Yao, Rui and Zou, Lixin and Li, Mengtian and Sheng, Bin},
  title = {Spatio-Temporal Disentanglement and Constrained Self-Attention for Multi-Modal Deception Detection},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {2},
  pages = {1--20},
  doi = {10.1145/3783994}
}
Kostenlos starten →