Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach
Sensors · 16.07.2026 · 26 (14) · 4512
DOI: 10.3390/s26144512 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 16.07.2026. Verfasst von Lu Zhang und 3 weiteren Autor:innen. Beteiligte Einrichtung: College of Information and Cyber Security, People's Public Security University of China, Beijing 100038, China. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities, Shandong Provincial Public Security Science and Technology Innovation Program, China People's Public Security University.
Abstract (Originalsprache)
Deepfakes generated by advanced AI models pose growing challenges to digital media authenticity. Large Vision–Language Models (VLMs) have recently been explored for image forensics due to their multimodal representation ability. However, many existing VLM-based deepfake detection methods keep the visual encoder fixed to preserve pre-trained knowledge, which may limit the model’s sensitivity to low-level inconsistency artifacts that are important for deepfake detection. To address this issue, we propose IncoTune, an inconsistency-aware tuning framework that introduces trainable vision-side adaptation into the visual encoder and applies Directional Low-Rank Adaptation (DoRA) to selected linear projection layers in both the visual encoder and the language model. We further report an empirical observation in the ablation study: under the evaluated adapter configuration, replacing LoRA with DoRA in the frozen-vision setting does not improve the average AUC, whereas DoRA provides additional gains when combined with vision-side adaptation. Experimental results on FaceForensics++, DFD, Celeb-DF, DFDC, and DFDCP show that IncoTune improves cross-dataset frame-level detection performance over the frozen-vision baseline and achieves competitive performance among representative frame-level methods, while updating only 27.0M adapter parameters during training. Robustness experiments further evaluate the model behavior under common image degradations.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Information and Cyber Security, People's Public Security University of China, Beijing 100038, China.
- Department of Investigation, Shandong Police College, Jinan 250200, China.
- School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China.
- Laboratory for Digital and Intelligent Prevention and Control of Societal Security Risks, People's Public Security University of China, Beijing 100038, China.
Förderung
- Fundamental Research Funds for the Central Universities
- Shandong Provincial Public Security Science and Technology Innovation Program
- China People's Public Security University
- China People’s Public Security University
Zitieren
APA
Zhang, L., Peng, S., Xu, M., & Lu, T. (2026). Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach. Sensors, 26(14), 4512. https://doi.org/10.3390/s26144512
Vancouver
Zhang L, Peng S, Xu M, Lu T. Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach. Sensors. 2026;26(14):4512. doi:10.3390/s26144512
BibTeX
@article{Zhang2026,
author = {Zhang, Lu and Peng, Shufan and Xu, Mingle and Lu, Tianliang},
title = {Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach},
journal = {Sensors},
year = {2026},
volume = {26},
number = {14},
pages = {4512},
doi = {10.3390/s26144512}
}
EUVIMED