EUVIMEDEUVIMED entdecken
Crossref

Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning

Hui Miao, Yuanfang Guo, Zeming Liu, Yunhong Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (1) · 612-621

DOI: 10.1609/aaai.v39i1.32042 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Hui Miao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the malicious use and dissemination of multi-modal deepfake videos, researchers start to investigate multi-modal deepfake detection. Unfortunately, most of the existing methods tune all the parameters of the deep network with limited speech video datasets and are trained under coarse-grained consistency supervision, which hinders their generalization ability in practical scenarios. To solve these problems, in this paper, we propose the first multi-task audio-visual prompt learning method for multi-modal deepfake video detection, by exploiting multiple foundation models. Specifically, we construct a two-stream multi-task learning architecture and propose sequential visual prompts and short-time audio prompts to extract multi-modal features, which are aligned at the frame level and utilized in subsequent fine-grained feature matching and fusion. Due to the natural alignment of visual content and audio signal in real data, we propose a frame-level cross-modal feature matching loss function to learn the fine-grained audio-visual consistency. Comprehensive experiments demonstrate the effectiveness and superior generalization ability of our method against the state-of-the-art methods.

Zitieren

APA

Miao, H., Guo, Y., Liu, Z., & Wang, Y. (2025). Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 39(1), 612-621. https://doi.org/10.1609/aaai.v39i1.32042

Vancouver

Miao H, Guo Y, Liu Z, Wang Y. Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(1):612-621. doi:10.1609/aaai.v39i1.32042

BibTeX
@article{Miao2025,
  author = {Miao, Hui and Guo, Yuanfang and Liu, Zeming and Wang, Yunhong},
  title = {Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {1},
  pages = {612--621},
  doi = {10.1609/aaai.v39i1.32042}
}
Kostenlos starten →