EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach

Lu Zhang, Shufan Peng, Mingle Xu, Tianliang Lu

Sensors · 16.07.2026 · 26 (14) · 4512

DOI: 10.3390/s26144512 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 16.07.2026. Verfasst von Lu Zhang und 3 weiteren Autor:innen. Beteiligte Einrichtung: College of Information and Cyber Security, People's Public Security University of China, Beijing 100038, China. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities, Shandong Provincial Public Security Science and Technology Innovation Program, China People's Public Security University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deepfakes generated by advanced AI models pose growing challenges to digital media authenticity. Large Vision–Language Models (VLMs) have recently been explored for image forensics due to their multimodal representation ability. However, many existing VLM-based deepfake detection methods keep the visual encoder fixed to preserve pre-trained knowledge, which may limit the model’s sensitivity to low-level inconsistency artifacts that are important for deepfake detection. To address this issue, we propose IncoTune, an inconsistency-aware tuning framework that introduces trainable vision-side adaptation into the visual encoder and applies Directional Low-Rank Adaptation (DoRA) to selected linear projection layers in both the visual encoder and the language model. We further report an empirical observation in the ablation study: under the evaluated adapter configuration, replacing LoRA with DoRA in the frozen-vision setting does not improve the average AUC, whereas DoRA provides additional gains when combined with vision-side adaptation. Experimental results on FaceForensics++, DFD, Celeb-DF, DFDC, and DFDCP show that IncoTune improves cross-dataset frame-level detection performance over the frozen-vision baseline and achieves competitive performance among representative frame-level methods, while updating only 27.0M adapter parameters during training. Robustness experiments further evaluate the model behavior under common image degradations.

Schlagwörter

Stichwörter der Autor:innen

IncoTunedeepfake detectioninconsistencyvision–language models

Autor:innen und Institutionen

  • College of Information and Cyber Security, People's Public Security University of China, Beijing 100038, China.
  • Department of Investigation, Shandong Police College, Jinan 250200, China.
  • School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, China.
  • Laboratory for Digital and Intelligent Prevention and Control of Societal Security Risks, People's Public Security University of China, Beijing 100038, China.

Förderung

  • Fundamental Research Funds for the Central Universities
  • Shandong Provincial Public Security Science and Technology Innovation Program
  • China People's Public Security University
  • China People’s Public Security University

Zitieren

APA

Zhang, L., Peng, S., Xu, M., & Lu, T. (2026). Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach. Sensors, 26(14), 4512. https://doi.org/10.3390/s26144512

Vancouver

Zhang L, Peng S, Xu M, Lu T. Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach. Sensors. 2026;26(14):4512. doi:10.3390/s26144512

BibTeX
@article{Zhang2026,
  author = {Zhang, Lu and Peng, Shufan and Xu, Mingle and Lu, Tianliang},
  title = {Towards Generalizable Deepfake Detection: An Inconsistency-Aware Vision–Language Model Tuning Approach},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {14},
  pages = {4512},
  doi = {10.3390/s26144512}
}
Kostenlos starten →