EUVIMEDScopri EUVIMED
Open access · CC BY 4.0

Prompt-Guided Structured Multimodal NER with SVG and ChatGPT

Yuzhou Ma, Haolong Qian, Shujun Xia, Wei Li

Electronics · 2026-03-18 · 15 (6) · 1276

DOI: 10.3390/electronics15061276 ↗

At a glance

Journal article in Electronics, published online on 2026-03-18. Written by Yuzhou Ma and 3 co-authors. 32 references. The full text is freely available (open access, CC BY 4.0). Funded by: Anhui Provincial Department of Education, China, Outstanding Scientific Research and Innovation Team of Anhui Police College.

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Multimodal named entity recognition (MNER) leverages both textual and visual information to improve entity recognition, particularly in unstructured scenarios such as social media. While existing approaches predominantly rely on raster images (e.g., JPEG, PNG), scalable vector graphics (SVG) offer unique advantages in resolution independence and structured semantic representation—an underexplored potential in multimodal learning. To fill this gap, we propose MNER-SVG, the first framework that incorporates SVG as a visual modality and enhances it with ChatGPT-generated auxiliary knowledge. Specifically, we introduce a Multimodal Similar Instance Perception Module that retrieves semantically relevant examples and prompts ChatGPT to generate contextual explanations. We further construct a Full-Text Graph and a Multimodal Interaction Graph, which are processed via Graph Attention Networks (GATs) to achieve fine-grained cross-modal alignment and feature fusion. Finally, a Conditional Random Field (CRF) layer is employed for structured decoding. To support evaluation, we present SvgNER, the first MNER dataset annotated with SVG-specific visual content. Extensive experiments demonstrate that MNER-SVG achieves state-of-the-art performance with an F1 score of 82.23%, significantly outperforming both text-only and existing multimodal baselines. This work validates the feasibility and potential of integrating vector graphics and large language model-generated knowledge into multimodal NER, opening a new research direction for structured visual semantics in fine-grained multimodal understanding.

Funding

  • Anhui Provincial Department of Education, China
  • Outstanding Scientific Research and Innovation Team of Anhui Police College

Cite

APA

Ma, Y., Qian, H., Xia, S., & Li, W. (2026). Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics, 15(6), 1276. https://doi.org/10.3390/electronics15061276

Vancouver

Ma Y, Qian H, Xia S, Li W. Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics. 2026;15(6):1276. doi:10.3390/electronics15061276

BibTeX
@article{Ma2026,
  author = {Ma, Yuzhou and Qian, Haolong and Xia, Shujun and Li, Wei},
  title = {Prompt-Guided Structured Multimodal NER with SVG and ChatGPT},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {6},
  pages = {1276},
  doi = {10.3390/electronics15061276}
}
Start for free →