EUVIMEDEUVIMED entdecken
Crossref

Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection

Yibo Zhang, Weiguo Lin, Junfeng Xu

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2024 · 20 (5) · 1-23

DOI: 10.1145/3625100 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 22.01.2024. Verfasst von Yibo Zhang und 2 weiteren Autor:innen. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 55 Literaturangaben. Gefördert von: National Key Research and Development Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the continuous advancement of deepfake technology, there has been a surge in the creation of realistic fake videos. Unfortunately, the malicious utilization of deepfake poses a significant threat to societal morality and political security. Therefore, numerous researchers have proposed various deepfake detection methods. However, traditional deepfake approaches tend to focus on specific forgery features, such as artifacts or inconsistent actions, which can be vulnerable to specialized countermeasures. Recent studies show an intrinsic correlation between facial and audio cues, which can be exploited for deepfake detection. To address these challenges and enhance the robustness and generalization of deepfake detection algorithms, we propose a novel joint audio-visual deepfake detection model named AVA-CL, which is capable of detecting deepfakes in both audio and visual domains. Furthermore, exploiting the inherent correlation and consistency between audio and visual enhances the effectiveness of deepfake detection significantly. Through extensive experiments, we demonstrate that our proposed AVA-CL model outperforms many state-of-the-art (SOTA) methods with superior robustness and generalization capabilities. This research presents a promising approach for deepfake detection and reducing the harm caused by malicious use.

Förderung

  • National Key Research and Development Program

Zitieren

APA

Zhang, Y., Lin, W., & Xu, J. (2024). Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(5), 1-23. https://doi.org/10.1145/3625100

Vancouver

Zhang Y, Lin W, Xu J. Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(5):1-23. doi:10.1145/3625100

BibTeX
@article{Zhang2024,
  author = {Zhang, Yibo and Lin, Weiguo and Xu, Junfeng},
  title = {Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2024},
  volume = {20},
  number = {5},
  pages = {1--23},
  doi = {10.1145/3625100}
}
Kostenlos starten →