Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (3) · 3040-3048
DOI: 10.1609/aaai.v39i3.32312 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Xinghe Fu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 41-mal zitiert (Crossref).
Abstract (Originalsprache)
The generalization problem is broadly recognized as a critical challenge in detecting deepfakes. Most previous work believes that the generalization gap is caused by the differences among various forgery methods. However, our investigation reveals that the generalization issue can still occur when forgery-irrelevant factors shift. In this work, we identify two biases that detectors may also be prone to overfitting: position bias and content bias, as depicted in Fig. 1. For the position bias, we observe that detectors are prone to “lazily” depending on the specific positions within an image (e.g., central regions even no forgery). As for content bias, we argue that detectors may potentially and mistakenly utilize forgery-unrelated information for detection (e.g., background, and hair). To intervene in these biases, we propose two branches for shuffling and mixing with tokens in the latent space of transformers. For the shuffling branch, we rearrange the tokens and corresponding position embedding for each image while maintaining the local correlation. For the mixing branch, we randomly select and mix the tokens in the latent space between two images with the same label within the mini-batch to recombine the content information. During the learning process, we align the outputs of detectors from different branches in both feature space and logit space. Contrastive losses for features and divergence losses for logits are applied to obtain unbiased feature representation and classifiers. We demonstrate and verify the effectiveness of our method through extensive experiments on widely used evaluation datasets.
Zitieren
APA
Fu, X., Yan, Z., Yao, T., Chen, S., & Li, X. (2025). Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 3040-3048. https://doi.org/10.1609/aaai.v39i3.32312
Vancouver
Fu X, Yan Z, Yao T, Chen S, Li X. Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(3):3040-3048. doi:10.1609/aaai.v39i3.32312
BibTeX
@article{Fu2025,
author = {Fu, Xinghe and Yan, Zhiyuan and Yao, Taiping and Chen, Shen and Li, Xi},
title = {Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {3},
pages = {3040--3048},
doi = {10.1609/aaai.v39i3.32312}
}
EUVIMED