Prompt-Guided Structured Multimodal NER with SVG and ChatGPT
Electronics · 18.03.2026 · 15 (6) · 1276
DOI: 10.3390/electronics15061276 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 18.03.2026. Verfasst von Yuzhou Ma und 3 weiteren Autor:innen. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Anhui Provincial Department of Education, China, Outstanding Scientific Research and Innovation Team of Anhui Police College.
Abstract (Originalsprache)
Multimodal named entity recognition (MNER) leverages both textual and visual information to improve entity recognition, particularly in unstructured scenarios such as social media. While existing approaches predominantly rely on raster images (e.g., JPEG, PNG), scalable vector graphics (SVG) offer unique advantages in resolution independence and structured semantic representation—an underexplored potential in multimodal learning. To fill this gap, we propose MNER-SVG, the first framework that incorporates SVG as a visual modality and enhances it with ChatGPT-generated auxiliary knowledge. Specifically, we introduce a Multimodal Similar Instance Perception Module that retrieves semantically relevant examples and prompts ChatGPT to generate contextual explanations. We further construct a Full-Text Graph and a Multimodal Interaction Graph, which are processed via Graph Attention Networks (GATs) to achieve fine-grained cross-modal alignment and feature fusion. Finally, a Conditional Random Field (CRF) layer is employed for structured decoding. To support evaluation, we present SvgNER, the first MNER dataset annotated with SVG-specific visual content. Extensive experiments demonstrate that MNER-SVG achieves state-of-the-art performance with an F1 score of 82.23%, significantly outperforming both text-only and existing multimodal baselines. This work validates the feasibility and potential of integrating vector graphics and large language model-generated knowledge into multimodal NER, opening a new research direction for structured visual semantics in fine-grained multimodal understanding.
Förderung
- Anhui Provincial Department of Education, China
- Outstanding Scientific Research and Innovation Team of Anhui Police College
Zitieren
APA
Ma, Y., Qian, H., Xia, S., & Li, W. (2026). Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics, 15(6), 1276. https://doi.org/10.3390/electronics15061276
Vancouver
Ma Y, Qian H, Xia S, Li W. Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics. 2026;15(6):1276. doi:10.3390/electronics15061276
BibTeX
@article{Ma2026,
author = {Ma, Yuzhou and Qian, Haolong and Xia, Shujun and Li, Wei},
title = {Prompt-Guided Structured Multimodal NER with SVG and ChatGPT},
journal = {Electronics},
year = {2026},
volume = {15},
number = {6},
pages = {1276},
doi = {10.3390/electronics15061276}
}
EUVIMED