Deepfake detection using convolutional vision transformers and convolutional neural networks
Neural Computing and Applications · 2024-11 · 36 (31) · 19759-19775
DOI: 10.1007/s00521-024-10181-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computing and Applications, online erschienen am 08.08.2024. Verfasst von Ahmed Hatem Soudy und 7 weiteren Autor:innen. Die Arbeit wurde bisher 137-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Helwan University.
Abstract (Originalsprache)
Deepfake technology has rapidly advanced in recent years, creating highly realistic fake videos that can be difficult to distinguish from real ones. The rise of social media platforms and online forums has exacerbated the challenges of detecting misinformation and malicious content. This study leverages many papers on artificial intelligence techniques to address deepfake detection. This research proposes a deep learning (DL)-based method for detecting deepfakes. The system comprises three components: preprocessing, detection, and prediction. Preprocessing includes frame extraction, face detection, alignment, and feature cropping. Convolutional neural networks (CNNs) are employed in the eye and nose feature detection phase. A CNN combined with a vision transformer is also used for face detection. The prediction component employs a majority voting approach, merging results from the three models applied to different features, leading to three individual predictions. The model is trained on various face images using FaceForensics++ and DFDC datasets. Multiple performance metrics, including accuracy, precision, F1, and recall, are used to assess the proposed model’s performance. The experimental results indicate the potential and strengths of the proposed CNN that achieved enhanced performance with an accuracy of 97%, while the CViT-based model achieved 85% using the FaceForences++ dataset and demonstrated significant improvements in deepfake detection compared to recent studies, affirming the potential of the suggested framework for detecting deepfakes on social media. This study contributes to a broader understanding of CNN-based DL methods for deepfake detection.
Förderung
- Helwan University
Zitieren
APA
Soudy, A. H., Sayed, O., Tag-Elser, H., Ragab, R., Mohsen, S., Mostafa, T., Abohany, A. A., & Slim, S. O. (2024). Deepfake detection using convolutional vision transformers and convolutional neural networks. Neural Computing and Applications, 36(31), 19759-19775. https://doi.org/10.1007/s00521-024-10181-7
Vancouver
Soudy AH, Sayed O, Tag-Elser H, Ragab R, Mohsen S, Mostafa T, et al. Deepfake detection using convolutional vision transformers and convolutional neural networks. Neural Computing and Applications. 2024;36(31):19759-19775. doi:10.1007/s00521-024-10181-7
BibTeX
@article{Soudy2024,
author = {Soudy, Ahmed Hatem and Sayed, Omnia and Tag-Elser, Hala and Ragab, Rewaa and Mohsen, Sohaila and Mostafa, Tarek and Abohany, Amr A. and Slim, Salwa O.},
title = {Deepfake detection using convolutional vision transformers and convolutional neural networks},
journal = {Neural Computing and Applications},
year = {2024},
volume = {36},
number = {31},
pages = {19759--19775},
doi = {10.1007/s00521-024-10181-7}
}
EUVIMED