EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition

Wenjie Liu iD, Guoqing Wu, Han Wang iD, Fuji Ren iD

Sensors · 30.06.2025 · 25 (13) · 4096

DOI: 10.3390/s25134096 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 30.06.2025. Verfasst von Wenjie Liu und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China. Hauptthemen laut MEDLINE: Crops, Agricultural, Plant Diseases. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of the Jiangsu Higher Education Institutions of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Crop diseases pose a significant threat to agricultural productivity and global food security. Timely and accurate disease identification is crucial for improving crop yield and quality. While most existing deep learning-based methods focus primarily on image datasets for disease recognition, they often overlook the complementary role of textual features in enhancing visual understanding. To address this problem, we proposed a cross-modal data fusion via a vision-language model for crop disease recognition. Our approach leverages the Zhipu.ai multi-model to generate comprehensive textual descriptions of crop leaf diseases, including global description, local lesion description, and color-texture description. These descriptions are encoded into feature vectors, while an image encoder extracts image features. A cross-attention mechanism then iteratively fuses multimodal features across multiple layers, and a classification prediction module generates classification probabilities. Extensive experiments on the Soybean Disease, AI Challenge 2018, and PlantVillage datasets demonstrate that our method outperforms state-of-the-art image-only approaches with higher accuracy and fewer parameters. Specifically, with only 1.14M model parameters, our model achieves a 98.74%, 87.64% and 99.08% recognition accuracy on the three datasets, respectively. The results highlight the effectiveness of cross-modal learning in leveraging both visual and textual cues for precise and efficient disease recognition, offering a scalable solution for crop disease recognition.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Crops, AgriculturalPlant DiseasesDeep LearningImage Processing, Computer-AssistedGlycine maxAlgorithmsPlant Leaves

Stichwörter der Autor:innen

crop disease recognitioncross-model data fusionimage classificationvision-language model

Autor:innen und Institutionen

  • School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China.
  • School of Mechanical Engineering, Nantong Institute of Technology, Nantong 226002, China.
  • School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.

ORCID-Profile: Wenjie Liu, Han Wang, Fuji Ren

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of the Jiangsu Higher Education Institutions of China

Zitieren

APA

Liu, W., Wu, G., Wang, H., & Ren, F. (2025). Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition. Sensors, 25(13), 4096. https://doi.org/10.3390/s25134096

Vancouver

Liu W, Wu G, Wang H, Ren F. Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition. Sensors. 2025;25(13):4096. doi:10.3390/s25134096

BibTeX
@article{Liu2025,
  author = {Liu, Wenjie and Wu, Guoqing and Wang, Han and Ren, Fuji},
  title = {Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {13},
  pages = {4096},
  doi = {10.3390/s25134096}
}
Kostenlos starten →