Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition
IEEE Sensors Journal · 01.04.2025 · 25 (7) · 11545-11562
DOI: 10.1109/jsen.2025.3540562 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 01.04.2025. Verfasst von Weijie Ding und 9 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 57 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Geological Survey Project.
Abstract (Originalsprache)
Detailed and accurate identification of disaster geomorphology is important in the governance, development, and emergency decision-making of shallow water areas in estuaries. However, traditional disaster geomorphology monitoring processes are constrained by the low efficiency of multisource heterogeneous data fusion and the poor stability of recognition models. In this study, we seamlessly acquired shallow water bathymetry and topographic data using unmanned aerial vehicle (UAV) light detection and ranging (LiDAR) and unmanned surface vehicle (USV) multibeam sonar, and proposed an attention-aware multimodal multiscale cross-fusion geomorphology recognition network called AMMNet. First, we adopted a three-stream encoder–decoder structure and designed an attention mechanism-based multimodal fusion module (AMFM) for riverbank morphology and texture to better utilize the fused enhanced features at multiple extraction stages. Second, we added an atrous spatial pyramid pooling (ASPP) module between the encoder and decoder to aggregate feature information of disaster geomorphology at different scales. Finally, an integrated approach combining gradient-weighted class activation mapping (Grad-CAM) and Shapley additive explanations (SHAPs) was employed to analyze the decision-making process of the model and disaster mechanisms. To evaluate the performance of the AMMNet model, it was compared comprehensively with other typical recognition models using an UAV–USV dataset containing 16 types of morphological and textural features. The results of this study demonstrate the superiority of the AMMNet model in recognizing hazard features compared with other models. Specifically, the intersection over union (IOU) for riverbank collapse, scarps, scour pits, and dunes reached 93.7%, 92.2%, 87.2%, and 94.2%, respectively. Furthermore, this study found that low entropy and a high topographic position index (TPI) were the main triggering factors for riverbank collapse. In addition, low elevation and high contrast have been found to increase the probability of dune-related hazards. This study provides a significant reference for accurate shallow water geomorphological identification and timely disaster risk warnings.
Förderung
- National Natural Science Foundation of China
- China Geological Survey Project
Zitieren
APA
Ding, W., Cheng, H., Wang, Z., Wu, J., Yang, Q., Zhao, X., Li, Z., Xu, Y., Dong, J., & Jin, Y. (2025). Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition. IEEE Sensors Journal, 25(7), 11545-11562. https://doi.org/10.1109/jsen.2025.3540562
Vancouver
Ding W, Cheng H, Wang Z, Wu J, Yang Q, Zhao X, et al. Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition. IEEE Sensors Journal. 2025;25(7):11545-11562. doi:10.1109/jsen.2025.3540562
BibTeX
@article{Ding2025,
author = {Ding, Weijie and Cheng, Heqin and Wang, Zhaocai and Wu, Junhao and Yang, Qian and Zhao, Xiaolong and Li, Zhenqi and Xu, Yijun and Dong, Jinghan and Jin, Yang},
title = {Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition},
journal = {IEEE Sensors Journal},
year = {2025},
volume = {25},
number = {7},
pages = {11545--11562},
doi = {10.1109/jsen.2025.3540562}
}
EUVIMED