Swin-Fake: A Consistency Learning Transformer-Based Deepfake Video Detector
Electronics · 01.08.2024 · 13 (15) · 3045
DOI: 10.3390/electronics13153045 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 01.08.2024. Verfasst von Liang Yu Gong und 2 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Deepfake has become an emerging technology affecting cyber-security with its illegal applications in recent years. Most deepfake detectors utilize CNN-based models such as the Xception Network to distinguish real or fake media; however, their performance on cross-datasets is not ideal because they suffer from over-fitting in the current stage. Therefore, this paper proposed a spatial consistency learning method to relieve this issue in three aspects. Firstly, we increased the selections of data augmentation methods to 5, which is more than our previous study’s data augmentation methods. Specifically, we captured several equal video frames of one video and randomly selected five different data augmentations to obtain different data views to enrich the input variety. Secondly, we chose Swin Transformer as the feature extractor instead of a CNN-based backbone, which means that our approach did not utilize it for downstream tasks, and could encode these data using an end-to-end Swin Transformer, aiming to learn the correlation between different image patches. Finally, this was combined with consistency learning in our study, and consistency learning was able to determine more data relationships than supervised classification. We explored the consistency of video frames’ features by calculating their cosine distance and applied traditional cross-entropy loss to regulate this classification loss. Extensive in-dataset and cross-dataset experiments demonstrated that Swin-Fake could produce relatively good results on some open-source deepfake datasets, including FaceForensics++, DFDC, Celeb-DF and FaceShifter. By comparing our model with several benchmark models, our approach shows relatively strong robustness in detecting deepfake media.
Zitieren
APA
Gong, L. Y., Li, X. J., & Chong, P. H. J. (2024). Swin-Fake: A Consistency Learning Transformer-Based Deepfake Video Detector. Electronics, 13(15), 3045. https://doi.org/10.3390/electronics13153045
Vancouver
Gong LY, Li XJ, Chong PHJ. Swin-Fake: A Consistency Learning Transformer-Based Deepfake Video Detector. Electronics. 2024;13(15):3045. doi:10.3390/electronics13153045
BibTeX
@article{Gong2024,
author = {Gong, Liang Yu and Li, Xue Jun and Chong, Peter Han Joo},
title = {Swin-Fake: A Consistency Learning Transformer-Based Deepfake Video Detector},
journal = {Electronics},
year = {2024},
volume = {13},
number = {15},
pages = {3045},
doi = {10.3390/electronics13153045}
}
EUVIMED