EUVIMEDEUVIMED entdecken
Crossref

Multimodal Neurosymbolic Approach for Explainable Deepfake Detection

Ijaz Ul Haq, Khalid Mahmood Malik, Khan Muhammad

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2024 · 20 (11) · 1-16

DOI: 10.1145/3624748 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 12.09.2024. Verfasst von Ijaz Ul Haq und 2 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: National Science Foundation, Michigan Transnational Research and Commercialization (MTRAC), Advanced Computing Technologies.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deepfake detection has become increasingly important in recent years owing to the widespread availability of deepfake generation technologies. Existing deepfake detection methods present two primary limitations; i.e., they are trained on a specific type of deepfake dataset, which renders them vulnerable to unseen deepfakes, and they regard deepfakes as a “black box” with limited explainability, making it difficult for non-AI experts to understand and trust the decisions. Hence, this article proposes a novel neurosymbolic deepfake detection framework that exploits the fact that human emotions cannot be imitated easily owing to their complex nature. We argue that deep fakes typically exhibit inter- or intra-modality inconsistencies in the emotional expressions of the person being manipulated. Thus, the proposed framework performs inter- and intra-modality reasoning on emotions extracted from audio and visual modalities using a psychological and arousal-valence model for deepfake detection. In addition to fake detection, the proposed framework provides textual explanations for its decisions. The results obtained using the Presidential Deepfakes Dataset and World Leaders Dataset of real and manipulated videos demonstrate the effectiveness of our approach in detecting deepfakes and highlight the potential of a neurosymbolic approach for expandability.

Förderung

  • National Science Foundation
  • Michigan Transnational Research and Commercialization (MTRAC), Advanced Computing Technologies

Zitieren

APA

Haq, I. U., Malik, K. M., & Muhammad, K. (2024). Multimodal Neurosymbolic Approach for Explainable Deepfake Detection. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(11), 1-16. https://doi.org/10.1145/3624748

Vancouver

Haq IU, Malik KM, Muhammad K. Multimodal Neurosymbolic Approach for Explainable Deepfake Detection. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(11):1-16. doi:10.1145/3624748

BibTeX
@article{Haq2024,
  author = {Haq, Ijaz Ul and Malik, Khalid Mahmood and Muhammad, Khan},
  title = {Multimodal Neurosymbolic Approach for Explainable Deepfake Detection},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2024},
  volume = {20},
  number = {11},
  pages = {1--16},
  doi = {10.1145/3624748}
}
Kostenlos starten →