EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation

Yifan Jiang iD, Jinshui Chen iD, Jiangang Lu iD

Sensors · 11.04.2025 · 25 (8) · 2417

DOI: 10.3390/s25082417 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 11.04.2025. Verfasst von Yifan Jiang und 2 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Plan of China, National Natural Science Foundation of China, Zhejiang Province Science and Technology Plan Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In industrial scenarios, image segmentation is essential for accurately identifying defect regions. Recently, the emergence of foundation models driven by powerful computational resources and large-scale training data has brought about a paradigm shift in deep learning-based image segmentation. The Segment Anything Model (SAM) has shown exceptional performance across various downstream tasks, owing to its vast semantic knowledge and strong generalization capabilities. However, the feature distribution discrepancy, reliance on manually labeled prompts, and limited category information of SAM reduce its scalability in industrial settings. To address these issues, we propose PA-SAM, an industrial defect segmentation framework based on SAM. Firstly, to bridge the gap between SAM’s pre-training data and distinct characteristics of industrial defects, we introduce a parameter-efficient fine-tuning (PEFT) technique incorporating lightweight Multi-Scale Partial Convolution Aggregation (MSPCA) into Low-Rank Adaptation (LoRA), named MSPCA-LoRA, which effectively enhances the image encoder’s sensitivity to prior knowledge biases, while maintaining PEFT efficiency. Furthermore, we present the Image-to-Prompt Embedding Generator (IPEG), which utilizes image embeddings to autonomously create high-quality prompt embeddings for directing mask segmentation, eliminating the limitations of manually provided prompts. Finally, we apply effective refinements to SAM’s mask decoder, transforming SAM into an end-to-end semantic segmentation framework. On two real-world defect segmentation datasets, PA-SAM achieves mean Intersections over Union of 73.87% and 68.30%, as well as mean Dice coefficients of 84.90% and 80.22%, outperforming other state-of-the-art algorithms, further demonstrating its robust generalization and application potential.

Schlagwörter

Stichwörter der Autor:innen

automated prompterdefect segmentationlow-rank adaptationparameter-efficient fine-tuningpartial convolutionsegment anything modelvision foundation model

Autor:innen und Institutionen

  • State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.

ORCID-Profile: Yifan Jiang, Jinshui Chen, Jiangang Lu

Förderung

  • National Key Research and Development Plan of China
  • National Natural Science Foundation of China
  • Zhejiang Province Science and Technology Plan Project

Zitieren

APA

Jiang, Y., Chen, J., & Lu, J. (2025). Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation. Sensors, 25(8), 2417. https://doi.org/10.3390/s25082417

Vancouver

Jiang Y, Chen J, Lu J. Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation. Sensors. 2025;25(8):2417. doi:10.3390/s25082417

BibTeX
@article{Jiang2025,
  author = {Jiang, Yifan and Chen, Jinshui and Lu, Jiangang},
  title = {Leveraging Vision Foundation Model via PConv-Based Fine-Tuning with Automated Prompter for Defect Segmentation},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {8},
  pages = {2417},
  doi = {10.3390/s25082417}
}
Kostenlos starten →