EUVIMEDEUVIMED entdecken
Crossref

Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video Detection

Dengyong Zhang, Wenjie Zhu, Xin Liao, Feifan Qi, Gaobo Yang, Xiangling Ding

ACM Transactions on Multimedia Computing, Communications, and Applications · 28.02.2025 · 21 (2) · 1-24

DOI: 10.1145/3664654 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 24.12.2024. Verfasst von Dengyong Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Scientific Research Fund of the Hunan Provincial Education Department of China, Hunan Provincial Funds for Distinguished Young Scholars.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the rise of the metaverse, the rapid advancement of Deepfakes technology has become closely intertwined. Within the metaverse, individuals exist in digital form and engage in interactions, transactions, and communications through virtual avatars. However, the development of Deepfakes technology has led to the proliferation of forged information disseminated under the guise of users’ virtual identities, posing significant security risks to the metaverse. Hence, there is an urgent need to research and develop more robust methods for detecting deep forgeries to address these challenges. This article explores deepfake video detection by leveraging the spatiotemporal inconsistencies generated by deepfake generation techniques, thereby proposing the interactive spatiotemporal inconsistency learning and interactive fusion (ST-ILIF) detection method, which consists of phase-aware and sequence streams. The spatial inconsistencies exhibited in frames of deepfake videos are primarily attributed to variations in the structural information contained within the phase component of the Fourier domain. To mitigate the issue of overfitting the content information, a phase-aware stream is introduced to learn the spatial inconsistencies from the phase-based reconstructed frames. Additionally, considering that deepfake videos are generated frame by frame and lack temporal consistency between frames, a sequence stream is proposed to extract temporal inconsistency features from the spatiotemporal difference information between consecutive frames. Finally, through feature interaction and fusion of the two streams, the representation ability of intermediate and classification features is further enhanced. The proposed method, which was evaluated on four mainstream datasets, outperformed most existing methods, and extensive experimental results demonstrated its effectiveness in identifying deepfake videos. Our source code is available at https://github.com/qff98/Deepfake-Video-Detection .

Förderung

  • National Natural Science Foundation of China
  • Scientific Research Fund of the Hunan Provincial Education Department of China
  • Hunan Provincial Funds for Distinguished Young Scholars

Zitieren

APA

Zhang, D., Zhu, W., Liao, X., Qi, F., Yang, G., & Ding, X. (2025). Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video Detection. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(2), 1-24. https://doi.org/10.1145/3664654

Vancouver

Zhang D, Zhu W, Liao X, Qi F, Yang G, Ding X. Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video Detection. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(2):1-24. doi:10.1145/3664654

BibTeX
@article{Zhang2025,
  author = {Zhang, Dengyong and Zhu, Wenjie and Liao, Xin and Qi, Feifan and Yang, Gaobo and Ding, Xiangling},
  title = {Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video Detection},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2025},
  volume = {21},
  number = {2},
  pages = {1--24},
  doi = {10.1145/3664654}
}
Kostenlos starten →