EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prompt-Guided Structured Multimodal NER with SVG and ChatGPT

Yuzhou Ma, Haolong Qian, Shujun Xia, Wei Li

Electronics · 18.03.2026 · 15 (6) · 1276

DOI: 10.3390/electronics15061276 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 18.03.2026. Verfasst von Yuzhou Ma und 3 weiteren Autor:innen. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Anhui Provincial Department of Education, China, Outstanding Scientific Research and Innovation Team of Anhui Police College.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal named entity recognition (MNER) leverages both textual and visual information to improve entity recognition, particularly in unstructured scenarios such as social media. While existing approaches predominantly rely on raster images (e.g., JPEG, PNG), scalable vector graphics (SVG) offer unique advantages in resolution independence and structured semantic representation—an underexplored potential in multimodal learning. To fill this gap, we propose MNER-SVG, the first framework that incorporates SVG as a visual modality and enhances it with ChatGPT-generated auxiliary knowledge. Specifically, we introduce a Multimodal Similar Instance Perception Module that retrieves semantically relevant examples and prompts ChatGPT to generate contextual explanations. We further construct a Full-Text Graph and a Multimodal Interaction Graph, which are processed via Graph Attention Networks (GATs) to achieve fine-grained cross-modal alignment and feature fusion. Finally, a Conditional Random Field (CRF) layer is employed for structured decoding. To support evaluation, we present SvgNER, the first MNER dataset annotated with SVG-specific visual content. Extensive experiments demonstrate that MNER-SVG achieves state-of-the-art performance with an F1 score of 82.23%, significantly outperforming both text-only and existing multimodal baselines. This work validates the feasibility and potential of integrating vector graphics and large language model-generated knowledge into multimodal NER, opening a new research direction for structured visual semantics in fine-grained multimodal understanding.

Förderung

  • Anhui Provincial Department of Education, China
  • Outstanding Scientific Research and Innovation Team of Anhui Police College

Zitieren

APA

Ma, Y., Qian, H., Xia, S., & Li, W. (2026). Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics, 15(6), 1276. https://doi.org/10.3390/electronics15061276

Vancouver

Ma Y, Qian H, Xia S, Li W. Prompt-Guided Structured Multimodal NER with SVG and ChatGPT. Electronics. 2026;15(6):1276. doi:10.3390/electronics15061276

BibTeX
@article{Ma2026,
  author = {Ma, Yuzhou and Qian, Haolong and Xia, Shujun and Li, Wei},
  title = {Prompt-Guided Structured Multimodal NER with SVG and ChatGPT},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {6},
  pages = {1276},
  doi = {10.3390/electronics15061276}
}
Kostenlos starten →