EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

On the Generalization of Deep Learning Models in Video Deepfake Detection

Davide Alessandro Coccomini iD, Roberto Caldelli iD, Fabrizio Falchi iD, Claudio Gennaro iD

Journal of Imaging · 29.04.2023 · 9 (5) · 89

DOI: 10.3390/jimaging9050089 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 29.04.2023. Verfasst von Davide Alessandro Coccomini und 3 weiteren Autor:innen. Beteiligte Einrichtung: Istituto di Scienza e Tecnologie dell'Informazione, 56124 Pisa, Italy. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: EU—NGEU, project AI4Media.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The increasing use of deep learning techniques to manipulate images and videos, commonly referred to as “deepfakes”, is making it more challenging to differentiate between real and fake content, while various deepfake detection systems have been developed, they often struggle to detect deepfakes in real-world situations. In particular, these methods are often unable to effectively distinguish images or videos when these are modified using novel techniques which have not been used in the training set. In this study, we carry out an analysis of different deep learning architectures in an attempt to understand which is more capable of better generalizing the concept of deepfake. According to our results, it appears that Convolutional Neural Networks (CNNs) seem to be more capable of storing specific anomalies and thus excel in cases of datasets with a limited number of elements and manipulation methodologies. The Vision Transformer, conversely, is more effective when trained with more varied datasets, achieving more outstanding generalization capabilities than the other methods analysed. Finally, the Swin Transformer appears to be a good alternative for using an attention-based method in a more limited data regime and performs very well in cross-dataset scenarios. All the analysed architectures seem to have a different way to look at deepfakes, but since in a real-world environment the generalization capability is essential, based on the experiments carried out, the attention-based architectures seem to provide superior performances.

Schlagwörter

Stichwörter der Autor:innen

computer visiondeep learningdeepfake detectiongeneralization

Autor:innen und Institutionen

  • Istituto di Scienza e Tecnologie dell'Informazione, 56124 Pisa, Italy.
  • National Inter-University Consortium for Telecommunications (CNIT), 50134 Florence, Italy.
  • Faculty of Economics, Universitas Mercatorum, 00186 Rome, Italy.

ORCID-Profile: Davide Alessandro Coccomini, Roberto Caldelli, Fabrizio Falchi, Claudio Gennaro

Förderung

  • EU—NGEU
  • project AI4Media

Zitieren

APA

Coccomini, D. A., Caldelli, R., Falchi, F., & Gennaro, C. (2023). On the Generalization of Deep Learning Models in Video Deepfake Detection. Journal of Imaging, 9(5), 89. https://doi.org/10.3390/jimaging9050089

Vancouver

Coccomini DA, Caldelli R, Falchi F, Gennaro C. On the Generalization of Deep Learning Models in Video Deepfake Detection. Journal of Imaging. 2023;9(5):89. doi:10.3390/jimaging9050089

BibTeX
@article{Coccomini2023,
  author = {Coccomini, Davide Alessandro and Caldelli, Roberto and Falchi, Fabrizio and Gennaro, Claudio},
  title = {On the Generalization of Deep Learning Models in Video Deepfake Detection},
  journal = {Journal of Imaging},
  year = {2023},
  volume = {9},
  number = {5},
  pages = {89},
  doi = {10.3390/jimaging9050089}
}
Kostenlos starten →