Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation
Sensors · 11.04.2025 · 25 (8) · 2417
DOI: 10.3390/s25082417 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 11.04.2025. Verfasst von Yifan Jiang und 2 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Plan of China, National Natural Science Foundation of China, Zhejiang Province Science and Technology Plan Project.
Abstract (Originalsprache)
In industrial scenarios, image segmentation is essential for accurately identifying defect regions. Recently, the emergence of foundation models driven by powerful computational resources and large-scale training data has brought about a paradigm shift in deep learning-based image segmentation. The Segment Anything Model (SAM) has shown exceptional performance across various downstream tasks, owing to its vast semantic knowledge and strong generalization capabilities. However, the feature distribution discrepancy, reliance on manually labeled prompts, and limited category information of SAM reduce its scalability in industrial settings. To address these issues, we propose PA-SAM, an industrial defect segmentation framework based on SAM. Firstly, to bridge the gap between SAM’s pre-training data and distinct characteristics of industrial defects, we introduce a parameter-efficient fine-tuning (PEFT) technique incorporating lightweight Multi-Scale Partial Convolution Aggregation (MSPCA) into Low-Rank Adaptation (LoRA), named MSPCA-LoRA, which effectively enhances the image encoder’s sensitivity to prior knowledge biases, while maintaining PEFT efficiency. Furthermore, we present the Image-to-Prompt Embedding Generator (IPEG), which utilizes image embeddings to autonomously create high-quality prompt embeddings for directing mask segmentation, eliminating the limitations of manually provided prompts. Finally, we apply effective refinements to SAM’s mask decoder, transforming SAM into an end-to-end semantic segmentation framework. On two real-world defect segmentation datasets, PA-SAM achieves mean Intersections over Union of 73.87% and 68.30%, as well as mean Dice coefficients of 84.90% and 80.22%, outperforming other state-of-the-art algorithms, further demonstrating its robust generalization and application potential.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
ORCID-Profile: Yifan Jiang, Jinshui Chen, Jiangang Lu
Förderung
- National Key Research and Development Plan of China
- National Natural Science Foundation of China
- Zhejiang Province Science and Technology Plan Project
Zitieren
APA
Jiang, Y., Chen, J., & Lu, J. (2025). Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation. Sensors, 25(8), 2417. https://doi.org/10.3390/s25082417
Vancouver
Jiang Y, Chen J, Lu J. Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation. Sensors. 2025;25(8):2417. doi:10.3390/s25082417
BibTeX
@article{Jiang2025,
author = {Jiang, Yifan and Chen, Jinshui and Lu, Jiangang},
title = {Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation},
journal = {Sensors},
year = {2025},
volume = {25},
number = {8},
pages = {2417},
doi = {10.3390/s25082417}
}
EUVIMED