EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

VSGN: Visual–Semantic Guided Interaction Network for Multimodal Named Entity Recognition

Jianjun Yao, Zhikun Zhou, Ruisheng Li, Jiaming Zhang, Zhiwei Qi

Symmetry · 29.04.2026 · 18 (5) · 769

DOI: 10.3390/sym18050769 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Symmetry, online erschienen am 29.04.2026. Verfasst von Jianjun Yao und 4 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Lanzhou Youth Science and Technology Talent Innovation Project, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Named Entity Recognition (MNER) aims to integrate textual and visual information to identify entities with specific semantic categories. However, existing methods often suffer from insufficient intra-modal semantic modeling, coarse cross-modal alignment, and vulnerability to noisy or ambiguous expressions in social media. To address these challenges, we propose a Visual–Semantic Guided Interaction Network (VSGN), which improves multimodal representation learning from both semantic and structural perspectives. Specifically, we first design an adaptive visual–semantic fusion module that incorporates visual descriptions as semantic guidance, enabling more informative cross-modal interactions. To further enhance feature quality, we introduce a deviation-aware channel-wise inhibitory routing (CIR) mechanism, which jointly models channel importance and distributional deviation to suppress noisy or redundant visual signals. In addition, we propose a visual–semantic guided graph structure learning module (VSG), which explicitly captures structural dependencies across modalities. By enforcing distribution-level alignment between textual and visual graph representations, the model achieves structure-aware cross-modal interaction and reduces modality inconsistency. Extensive experiments on the Twitter-2015 and Twitter-2017 datasets demonstrate the effectiveness of the proposed method, achieving F1 scores of 76.72% and 87.86%, respectively. The results show that jointly modeling semantic enhancement and structural alignment leads to more robust and discriminative multimodal representations.

Förderung

  • Lanzhou Youth Science and Technology Talent Innovation Project
  • National Natural Science Foundation of China

Zitieren

APA

Yao, J., Zhou, Z., Li, R., Zhang, J., & Qi, Z. (2026). VSGN: Visual–Semantic Guided Interaction Network for Multimodal Named Entity Recognition. Symmetry, 18(5), 769. https://doi.org/10.3390/sym18050769

Vancouver

Yao J, Zhou Z, Li R, Zhang J, Qi Z. VSGN: Visual–Semantic Guided Interaction Network for Multimodal Named Entity Recognition. Symmetry. 2026;18(5):769. doi:10.3390/sym18050769

BibTeX
@article{Yao2026,
  author = {Yao, Jianjun and Zhou, Zhikun and Li, Ruisheng and Zhang, Jiaming and Qi, Zhiwei},
  title = {VSGN: Visual–Semantic Guided Interaction Network for Multimodal Named Entity Recognition},
  journal = {Symmetry},
  year = {2026},
  volume = {18},
  number = {5},
  pages = {769},
  doi = {10.3390/sym18050769}
}
Kostenlos starten →