An MLLM-Assisted Denoising Multiscale Visual Fusion Framework for MNER-MI
IEEE Transactions on Computational Social Systems · 2026-08 · 13 (4) · 4886-4902
DOI: 10.1109/tcss.2026.3689752 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Computational Social Systems, 2026-08. Verfasst von Xing Li und 3 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, State Key Laboratory of Radio Frequency Heterogeneous Integration.
Abstract (Originalsprache)
Multimodal named entity recognition (MNER) has become a prominent research area for information extraction from social media content. Recent statistics indicate a growing proportion of social media posts containing multiple images. However, most existing methods focus primarily on single-image scenarios. To bridge this gap, we propose a multimodal large language model-assisted denoising multiscale visual fusion framework (LD-MVF) for multiimage MNER (MNER-MI). Our framework follows a coordinated pipeline of filtering, enhancement, and fusion to systematically address the unique challenges of multiimage scenarios. Specifically, to reduce noise from irrelevant images, we introduce an image–text similarity discriminator trained with human feedback, which provides dynamic weights for visual feature aggregation. To enrich the often sparse textual context, we leverage a multimodal large language model (MLLM) to directly generate grounded auxiliary knowledge from the most relevant images, avoiding information loss caused by textual descriptions. Furthermore, we design an adaptive hierarchical fusion module that extracts multiscale visual features from multiple images and injects them as visual prefixes to enhance textual representations through a dynamic gating mechanism. These three components work cooperatively: the discriminator filters out noise to provide cleaner visual input, the MLLM enhances textual semantics with visually-grounded knowledge, and the fusion module integrates multiscale features for fine-grained cross-modal alignment. Extensive experiments demonstrate that LD-MVF achieves state-of-the-art performance on both the MNER-MI and MNER-MI-Plus datasets.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- State Key Laboratory of Radio Frequency Heterogeneous Integration
- Scientific Foundation for Youth Scholars of Shenzhen University
Zitieren
APA
Li, X., Yu, H., Jin, X., & Zhang, D. (2026). An MLLM-Assisted Denoising Multiscale Visual Fusion Framework for MNER-MI. IEEE Transactions on Computational Social Systems, 13(4), 4886-4902. https://doi.org/10.1109/tcss.2026.3689752
Vancouver
Li X, Yu H, Jin X, Zhang D. An MLLM-Assisted Denoising Multiscale Visual Fusion Framework for MNER-MI. IEEE Transactions on Computational Social Systems. 2026;13(4):4886-4902. doi:10.1109/tcss.2026.3689752
BibTeX
@article{Li2026,
author = {Li, Xing and Yu, Huilong and Jin, Xiongnan and Zhang, Ding},
title = {An MLLM-Assisted Denoising Multiscale Visual Fusion Framework for MNER-MI},
journal = {IEEE Transactions on Computational Social Systems},
year = {2026},
volume = {13},
number = {4},
pages = {4886--4902},
doi = {10.1109/tcss.2026.3689752}
}
EUVIMED