A Camera-Based Multimodal Defect Sensing Framework for Substation Equipment Monitoring via Cross-Modal Feature Mapping
Sensors · 21.06.2026 · 26 (12) · 3935
DOI: 10.3390/s26123935 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 21.06.2026. Verfasst von Ziquan Liu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Electric Power Research Institute, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 211103, China. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid Nanjing Power Supply Company (China).
Abstract (Originalsprache)
To address the limitations of vision-only defect detection, image–semantic misalignment, and spatial-logic conflicts in complex substation inspection scenarios, this paper proposes a camera-sensor-based multimodal defect sensing framework with cross-modal feature mapping for substation equipment monitoring. The proposed framework integrates field inspection images acquired by camera sensors, defect textual descriptions, and equipment topology knowledge and establishes a unified domain-adaptive pre-training–bidirectional cross-modal mapping–hierarchical reasoning workflow. First, a Contrastive Language–Image Pre-training (CLIP)-based domain-adaptive pre-training strategy is developed to enhance the representation of equipment categories, defect attributes, and inspection-scene semantics. Second, a bidirectional cross-modal feature mapping network is constructed to model fine-grained interactions between candidate visual regions and textual semantics, where uncertainty-aware fusion and prototype constraints are introduced to improve semantic alignment and defect discrimination. Third, a hierarchical neuro-symbolic reasoning module incorporates equipment topology and spatial rules for posterior verification, logical consistency checking, and false-positive suppression. Experiments on a substation inspection image dataset demonstrate that the proposed method achieves 90.8% mAP@0.5, 68.7% mAP@0.5:0.95, and 89.4% F1-score, outperforming mainstream and recent detection models.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Electric Power Research Institute, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 211103, China.
- Nanjing Power Supply Company, State Grid Jiangsu Electric Power Co., Ltd., Nanjing 210019, China.
Förderung
- State Grid Nanjing Power Supply Company (China)
Zitieren
APA
Liu, Z., Xue, H., Hu, C., Wei, C., & Zhang, C. (2026). A Camera-Based Multimodal Defect Sensing Framework for Substation Equipment Monitoring via Cross-Modal Feature Mapping. Sensors, 26(12), 3935. https://doi.org/10.3390/s26123935
Vancouver
Liu Z, Xue H, Hu C, Wei C, Zhang C. A Camera-Based Multimodal Defect Sensing Framework for Substation Equipment Monitoring via Cross-Modal Feature Mapping. Sensors. 2026;26(12):3935. doi:10.3390/s26123935
BibTeX
@article{Liu2026,
author = {Liu, Ziquan and Xue, Hai and Hu, Chengbo and Wei, Chao and Zhang, Can},
title = {A Camera-Based Multimodal Defect Sensing Framework for Substation Equipment Monitoring via Cross-Modal Feature Mapping},
journal = {Sensors},
year = {2026},
volume = {26},
number = {12},
pages = {3935},
doi = {10.3390/s26123935}
}
EUVIMED