MSFT-Net: Mixture Semantic-Agnostic Manipulation Trace Enhanced Architecture for Robust Image Manipulation Localization
IEEE Transactions on Circuits and Systems for Video Technology · 2026-08 · 36 (8) · 11547-11560
DOI: 10.1109/tcsvt.2026.3679352 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-08. Verfasst von Jiayi Liu und 6 weiteren Autor:innen. Die Arbeit nennt 43 Literaturangaben. Gefördert von: New Generation Artificial Intelligence-National Science and Technology Major Project, National Natural Science Foundation of China, Hubei Provincial Key Research and Development Program, China.
Abstract (Originalsprache)
Since the proliferation of image manipulation methods, effective image manipulation localization (IML) in scenarios with post-processing operations gradually becomes a core challenge. For a long time, IML either relies on strongly semantically related features, resulting in semantic relevance bias in the localization results, or only uses a single semantic-agnostic space feature, which is unable to maintain effective localization capabilities after image post-processing operations. Inspired by this, we propose a novel mixture semantic-agnostic manipulation trace robust localization network (MSFT-Net), which specifically utilizes mixture semantic-agnostic information to achieve effective and robust IML. The MSFT-Net introduces two new modules, the mixture shared manipulation trace enhancement module (MISE) and the Multiscale Feature Association Module (FAM). MISE dynamically links multiple semantic-agnostic feature extractors using a sparsity-enhanced mixture of shared experts, enabling the extraction of diverse manipulation features for accurate localization. Furthermore, keeping the high resolution of the localization features is very important in the mask prediction stage. Therefore, FAM outputs high-resolution fused manipulation features by using the correlation of features at the same level and the spatial context information from different levels. This further improves the effectiveness of IML in post-processing scenarios. Comprehensive experiments on five datasets demonstrate that our model significantly improves both in localization accuracy (average F1 score and IoU increasing by over 9.9% and 4.0%) and robustness. The codes will be made available.
Förderung
- New Generation Artificial Intelligence-National Science and Technology Major Project
- National Natural Science Foundation of China
- Hubei Provincial Key Research and Development Program, China
- Science and Technology Planning Project of Guangdong Province
- Project of Guangdong Key Laboratory of Industrial Control System Security
Zitieren
APA
Liu, J., Ye, D., Zhang, Y., Deng, J., Liu, Z., Shang, Y., & Tian, Z. (2026). MSFT-Net: Mixture Semantic-Agnostic Manipulation Trace Enhanced Architecture for Robust Image Manipulation Localization. IEEE Transactions on Circuits and Systems for Video Technology, 36(8), 11547-11560. https://doi.org/10.1109/tcsvt.2026.3679352
Vancouver
Liu J, Ye D, Zhang Y, Deng J, Liu Z, Shang Y, et al. MSFT-Net: Mixture Semantic-Agnostic Manipulation Trace Enhanced Architecture for Robust Image Manipulation Localization. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(8):11547-11560. doi:10.1109/tcsvt.2026.3679352
BibTeX
@article{Liu2026,
author = {Liu, Jiayi and Ye, Dengpan and Zhang, Yunming and Deng, Jiacheng and Liu, Ziyi and Shang, Yueyun and Tian, Zhihong},
title = {MSFT-Net: Mixture Semantic-Agnostic Manipulation Trace Enhanced Architecture for Robust Image Manipulation Localization},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {8},
pages = {11547--11560},
doi = {10.1109/tcsvt.2026.3679352}
}
EUVIMED