EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Self-Supervised Contrastive Learning with Mamba-2 State Space Models for Audio-Visual Deepfake Detection

Zhen Yin, Robiatul A'dawiah Jamaluddin

Journal of AI Analytics and Applications · 30.06.2026 · 4 (2) · 1-12

DOI: 10.63646/jaiaa.2026.040201 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of AI Analytics and Applications, online erschienen am 30.06.2026. Verfasst von Zhen Yin und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the rapid proliferation of generative adversarial networks (GANs) and diffusion models, deepfake videos have achieved a level of fidelity that challenges the limits of human visual perception, posing severe threats to information security and public trust. Existing detection methodologies mostly rely on Convolutional Neural Networks (CNNs) combined with Recurrent Neural Networks (RNNs) or Transformers to extract spatial-temporal features. However, Transformers scale quadratically with long sequences. Additionally, traditional fusion methods miss subtle audio-visual inconsistencies. This paper proposes a novel, lightweight multi-modal detection framework that integrates Mamba-2 State Space Models (SSM) with Self-supervised Contrastive Learning (SSCL). Specifically, we leverage the linear complexity and the selective scan mechanism of Mamba-2 to efficiently model long-range temporal dependencies. Furthermore, a dual-branch contrastive loss is introduced to explicitly align the physical synchronization patterns between lip movements and speech phonemes. This forces the model to learn invariant forensic features rather than overfitting to specific visual artifacts. Experiments demonstrate that our proposed method achieves superior performance: an AUC of 0.976 on FaceForensics++ and 0.962 on FakeAVCeleb.

Zitieren

APA

Yin, Z., & Jamaluddin, R. A. (2026). Self-Supervised Contrastive Learning with Mamba-2 State Space Models for Audio-Visual Deepfake Detection. Journal of AI Analytics and Applications, 4(2), 1-12. https://doi.org/10.63646/jaiaa.2026.040201

Vancouver

Yin Z, Jamaluddin RA. Self-Supervised Contrastive Learning with Mamba-2 State Space Models for Audio-Visual Deepfake Detection. Journal of AI Analytics and Applications. 2026;4(2):1-12. doi:10.63646/jaiaa.2026.040201

BibTeX
@article{Yin2026,
  author = {Yin, Zhen and Jamaluddin, Robiatul A'dawiah},
  title = {Self-Supervised Contrastive Learning with Mamba-2 State Space Models for Audio-Visual Deepfake Detection},
  journal = {Journal of AI Analytics and Applications},
  year = {2026},
  volume = {4},
  number = {2},
  pages = {1--12},
  doi = {10.63646/jaiaa.2026.040201}
}
Kostenlos starten →