EUVIMEDEUVIMED entdecken
Crossref

IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection

Zewen Li, Zitong Yu, Qilang Ye, Weicheng Xie, Wei Zhuo, Linlin Shen

IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-12

DOI: 10.1109/tim.2025.3635334 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Zewen Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Open Fund of National Engineering Laboratory for Big Data System Computing Technology, Basic and Applied Basic Research Foundation of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The robust causal capability of multimodal large language models (MLLMs) holds the potential of detecting defective objects in industrial anomaly detection (IAD). However, most traditional IAD methods lack the ability to provide multiturn human–machine dialogs and detailed descriptions, such as the color of objects, the shape of an anomaly, or specific types of anomalies. At the same time, methods based on large pretrained models have not fully stimulated the ability of large models in anomaly detection tasks. In this article, we explore the combination of rich text semantics with both image-level and pixel-level information from images and propose IAD-GPT, a novel paradigm based on MLLMs for IAD. We employ abnormal prompt generator (APG) to generate detailed anomaly prompts for specific objects. These specific prompts from the large language model (LLM) are used to activate the detection and segmentation functions of the pretrained visual-language model (i.e., CLIP). To enhance the visual grounding ability of MLLMs, we propose text-guided enhancer (TGE), wherein image features interact with normal and abnormal text prompts to dynamically select enhancement pathways, which enables language models to focus on the specific aspects of visual data, enhancing their ability to accurately interpret and respond to anomalies within images. Moreover, we design a multimask fusion (MMF) module to incorporate mask as expert knowledge, which enhances the LLM’s perception of pixel-level anomalies. Extensive experiments on MVTec-AD and VisA datasets demonstrate our state-of-the-art performance on self-supervised and few-shot anomaly detection and segmentation tasks, such as MVTec-AD and VisA datasets. The codes are available at https://github.com/LiZeWen1225/IAD-GPT

Förderung

  • National Natural Science Foundation of China
  • Open Fund of National Engineering Laboratory for Big Data System Computing Technology
  • Basic and Applied Basic Research Foundation of Guangdong Province
  • Guangdong–Macau Science and Technology Innovation Joint Foundation
  • Guangdong Provincial Key Laboratory
  • Shenzhen Science and Technology Program

Zitieren

APA

Li, Z., Yu, Z., Ye, Q., Xie, W., Zhuo, W., & Shen, L. (2025). IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection. IEEE Transactions on Instrumentation and Measurement, 74, 1-12. https://doi.org/10.1109/tim.2025.3635334

Vancouver

Li Z, Yu Z, Ye Q, Xie W, Zhuo W, Shen L. IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-12. doi:10.1109/tim.2025.3635334

BibTeX
@article{Li2025,
  author = {Li, Zewen and Yu, Zitong and Ye, Qilang and Xie, Weicheng and Zhuo, Wei and Shen, Linlin},
  title = {IAD-GPT: Advancing Visual Knowledge in Multimodal Large Language Model for Industrial Anomaly Detection},
  journal = {IEEE Transactions on Instrumentation and Measurement},
  year = {2025},
  volume = {74},
  pages = {1--12},
  doi = {10.1109/tim.2025.3635334}
}
Kostenlos starten →