EUVIMEDEUVIMED entdecken
Crossref

Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-modal Manipulation

Huan Liu, Zichang Tan, Qiang Chen, Yunchao Wei, Yao Zhao, Jingdong Wang

International Journal of Computer Vision · 2025-03 · 133 (3) · 1392-1409

DOI: 10.1007/s11263-024-02245-x ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 07.10.2024. Verfasst von Huan Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 69 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Detecting and grounding multi-modal media manipulation (DGM4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {DGM}^4$$\end{document}) has become increasingly crucial due to the widespread dissemination of face forgery and text misinformation. In this paper, we present the Unified Frequency-Assisted transFormer framework, named UFAFormer, to address the DGM4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {DGM}^4$$\end{document} problem. Unlike previous state-of-the-art methods that solely focus on the image (RGB) domain to describe visual forgery features, we additionally introduce the frequency domain as a complementary viewpoint. By leveraging the discrete wavelet transform, we decompose images into several frequency sub-bands, capturing rich face forgery artifacts. Then, our proposed frequency encoder, incorporating intra-band and inter-band self-attentions, explicitly aggregates forgery features within and across diverse sub-bands. Moreover, to address the semantic conflicts between image and frequency domains, the forgery-aware mutual module is developed to further enable the effective interaction of disparate image and frequency features, resulting in aligned and comprehensive visual forgery representations. Finally, based on visual and textual forgery features, we propose a unified decoder that comprises two symmetric cross-modal interaction modules responsible for gathering modality-specific forgery information, along with a fusing interaction module for aggregation of both modalities. The proposed unified decoder formulates our UFAFormer as a unified framework, ultimately simplifying the overall architecture and facilitating the optimization process. Experimental results on the DGM4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {DGM}^4$$\end{document} dataset, containing several perturbations, demonstrate the superior performance of our framework compared to previous methods, setting a new benchmark in the field.

Zitieren

APA

Liu, H., Tan, Z., Chen, Q., Wei, Y., Zhao, Y., & Wang, J. (2025). Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-modal Manipulation. International Journal of Computer Vision, 133(3), 1392-1409. https://doi.org/10.1007/s11263-024-02245-x

Vancouver

Liu H, Tan Z, Chen Q, Wei Y, Zhao Y, Wang J. Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-modal Manipulation. International Journal of Computer Vision. 2025;133(3):1392-1409. doi:10.1007/s11263-024-02245-x

BibTeX
@article{Liu2025,
  author = {Liu, Huan and Tan, Zichang and Chen, Qiang and Wei, Yunchao and Zhao, Yao and Wang, Jingdong},
  title = {Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-modal Manipulation},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {3},
  pages = {1392--1409},
  doi = {10.1007/s11263-024-02245-x}
}
Kostenlos starten →