Entity-level cross-modal fusion for multimodal chinese agricultural diseases and pests named entity recognition
Smart Agricultural Technology · 2025-12 · 12 · 101188
DOI: 10.1016/j.atech.2025.101188 ↗Kurz gefasst
Fachartikel in der Zeitschrift Smart Agricultural Technology, 2025-12. Verfasst von Jingzhong Huang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Shandong Province Natural Science Foundation, National Natural Science Foundation of China.
Abstract (Originalsprache)
Named Entity Recognition (NER), as one of the popular directions in natural language processing, plays a critical role in fields such as information extraction and agricultural knowledge graph construction. However, traditional pure text-based methods often face limitations in agricultural entity recognition, including information singularity, contextual constraints, and insufficient feature fusion capabilities. Inspired by multimodal learning, this paper proposes an agricultural NER model based on entity-level cross-modal alignment, combining dual-stream entity-level feature encoding and dynamic gated attention mechanisms. First, we present a Dual-Stream Entity-Level Feature Encoding architecture. The text stream employs a Boundary-Middle (B-M) classification strategy to achieve fine-grained semantic unit segmentation, effectively addressing long-entity boundary ambiguity and parallel computing challenges. The visual stream utilizes saliency region detection to enhance multi-scale visual entity feature extraction capabilities. Second, we propose a Dynamic Cross-modal Gated Attention (DCGA) mechanism that adaptively adjusts visual feature contributions through gating weights, combined with cross-modal contrastive learning to strengthen entity-level semantic associations between text and images. To validate the model's effectiveness, we constructed a multimodal NER dataset containing 12,074 sample pairs across 10 entity categories, covering 10 crops, 82 typical diseases/pests, and related agrochemical data. The proposed method achieves a macro-average F1 score of 90.73% across 10 agricultural entity types, outperforming single-modal baselines by 5.96%, mainstream multimodal NER models by +3.06%, zero-shot GPT models by +11.41%, and fine-tuned multimodal large models by +2.1%. Comprehensive experimental results indicated that our multimodal collaborative learning framework could effectively enhance agricultural entity recognition accuracy, providing reliable technical support for downstream applications such as agricultural knowledge graph construction and intelligent question answering.
Förderung
- Shandong Province Natural Science Foundation
- National Natural Science Foundation of China
Zitieren
APA
Huang, J., Hao, X., Wang, Y., Song, R., Mu, Z., Chu, W., Papadakis, G., Niu, S., & Guo, X. (2025). Entity-level cross-modal fusion for multimodal chinese agricultural diseases and pests named entity recognition. Smart Agricultural Technology, 12, 101188. https://doi.org/10.1016/j.atech.2025.101188
Vancouver
Huang J, Hao X, Wang Y, Song R, Mu Z, Chu W, et al. Entity-level cross-modal fusion for multimodal chinese agricultural diseases and pests named entity recognition. Smart Agricultural Technology. 2025;12:101188. doi:10.1016/j.atech.2025.101188
BibTeX
@article{Huang2025,
author = {Huang, Jingzhong and Hao, Xia and Wang, Yu and Song, Ruizhi and Mu, Zenan and Chu, Wen and Papadakis, Georgios and Niu, Sijie and Guo, Xuchao},
title = {Entity-level cross-modal fusion for multimodal chinese agricultural diseases and pests named entity recognition},
journal = {Smart Agricultural Technology},
year = {2025},
volume = {12},
pages = {101188},
doi = {10.1016/j.atech.2025.101188}
}
EUVIMED