DIC-Transformer: interpretation of plant disease classification results using image caption generation technology
Frontiers in Plant Science · 25.01.2024 · 14
DOI: 10.3389/fpls.2023.1273029 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Plant Science, online erschienen am 25.01.2024. Verfasst von Qingtian Zeng und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Science and Technology Development Fund.
Abstract (Originalsprache)
Disease image classification systems play a crucial role in identifying disease categories in the field of agricultural diseases. However, current plant disease image classification methods can only predict the disease category and do not offer explanations for the characteristics of the predicted disease images. Due to the current situation, this paper employed image description generation technology to produce distinct descriptions for different plant disease categories. A two-stage model called DIC-Transformer, which encompasses three tasks (detection, interpretation, and classification), was proposed. In the first stage, Faster R-CNN was utilized to detect the diseased area and generate the feature vector of the diseased image, with the Swin Transformer as the backbone. In the second stage, the model utilized the Transformer to generate image captions. It then generated the image feature vector, which is weighted by text features, to improve the performance of image classification in the subsequent classification decoder. Additionally, a dataset containing text and visualizations for agricultural diseases (ADCG-18) was compiled. The dataset contains images of 18 diseases and descriptive information about their characteristics. Then, using the ADCG-18, the DIC-Transformer was compared to 11 existing classical caption generation methods and 10 image classification models. The evaluation indicators for captions include Bleu1–4, CiderD, and Rouge. The values of BLEU-1, CIDEr-D, and ROUGE were 0.756, 450.51, and 0.721. The results of DIC-Transformer were 0.01, 29.55, and 0.014 higher than those of the highest-performing comparison model, Fc. The classification evaluation metrics include accuracy, recall, and F1 score, with accuracy at 0.854, recall at 0.854, and F1 score at 0.853. The results of DIC-Transformer were 0.024, 0.078, and 0.075 higher than those of the highest-performing comparison model, MobileNetV2. The results indicate that the DIC-Transformer outperforms other comparison models in classification and caption generation.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Science and Technology Development Fund
- Taishan Scholar Foundation of Shandong Province
- Shandong University of Science and Technology
Zitieren
APA
Zeng, Q., Sun, J., & Wang, S. (2024). DIC-Transformer: interpretation of plant disease classification results using image caption generation technology. Frontiers in Plant Science, 14. https://doi.org/10.3389/fpls.2023.1273029
Vancouver
Zeng Q, Sun J, Wang S. DIC-Transformer: interpretation of plant disease classification results using image caption generation technology. Frontiers in Plant Science. 2024;14. doi:10.3389/fpls.2023.1273029
BibTeX
@article{Zeng2024,
author = {Zeng, Qingtian and Sun, Jian and Wang, Shansong},
title = {DIC-Transformer: interpretation of plant disease classification results using image caption generation technology},
journal = {Frontiers in Plant Science},
year = {2024},
volume = {14},
doi = {10.3389/fpls.2023.1273029}
}
EUVIMED