A Hybrid YOLO and Segment Anything Model Pipeline for Multi-Damage Segmentation in UAV Inspection Imagery
Sensors · 25.10.2025 · 25 (21) · 6568
DOI: 10.3390/s25216568 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 25.10.2025. Verfasst von Rafael Cabral und 3 weiteren Autor:innen. Beteiligte Einrichtung: CONSTRUCT-iRail, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The automated inspection of civil infrastructure with Unmanned Aerial Vehicles (UAVs) is hampered by the challenge of accurately segmenting multi-damage in high-resolution imagery. While foundational models like the Segment Anything Model (SAM) offer data-efficient segmentation, their effectiveness is constrained by prompting strategies, especially for geometrically complex defects. This paper presents a comprehensive comparative analysis of deep learning strategies to identify an optimal deep learning pipeline for segmenting cracks, efflorescences, and exposed rebars. It systematically evaluates three distinct end-to-end segmentation frameworks: the native output of a YOLO11 model; the Segment Anything Model (SAM), prompted by bounding boxes; and SAM, guided by a point-prompting mechanism derived from the detector’s probability map. Based on these findings, a final, optimized hybrid pipeline is proposed: for linear cracks, the native segmentation output of the SAHI-trained YOLO model is used, while for efflorescence and exposed rebar, the model’s bounding boxes are used to prompt SAM for a refined segmentation. This class-specific strategy yielded a final mean Average Precision (mAP50) of 0.593, with class-specific Intersection over Union (IoU) scores of 0.495 (cracks), 0.331 (efflorescence), and 0.205 (exposed rebar). The results establish that the future of automated inspection lies in intelligent frameworks that leverage the respective strengths of specialized detectors and powerful foundation models in a context-aware manner.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- CONSTRUCT-iRail, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal.
- iBuilt, School of Engineering, Polytechnic of Porto, 4249-015 Porto, Portugal.
ORCID-Profile: Rafael Cabral, Ricardo Santos, José A F O Correia, Diogo Ribeiro
Zitieren
APA
Cabral, R., Santos, R., Correia, J. A. F. O., & Ribeiro, D. (2025). A Hybrid YOLO and Segment Anything Model Pipeline for Multi-Damage Segmentation in UAV Inspection Imagery. Sensors, 25(21), 6568. https://doi.org/10.3390/s25216568
Vancouver
Cabral R, Santos R, Correia JAFO, Ribeiro D. A Hybrid YOLO and Segment Anything Model Pipeline for Multi-Damage Segmentation in UAV Inspection Imagery. Sensors. 2025;25(21):6568. doi:10.3390/s25216568
BibTeX
@article{Cabral2025,
author = {Cabral, Rafael and Santos, Ricardo and Correia, José A F O and Ribeiro, Diogo},
title = {A Hybrid YOLO and Segment Anything Model Pipeline for Multi-Damage Segmentation in UAV Inspection Imagery},
journal = {Sensors},
year = {2025},
volume = {25},
number = {21},
pages = {6568},
doi = {10.3390/s25216568}
}
EUVIMED