Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net
Remote Sensing · 24.08.2023 · 15 (17) · 4142
DOI: 10.3390/rs15174142 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 24.08.2023. Verfasst von Shuo Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, Funded Project of Fundamental Scientific Research Business Expenses of Chinese Academy of Surveying and Mapping.
Abstract (Originalsprache)
With the continuous development of surface observation methods and technologies, we can acquire multiple sources of data more effectively in the same geographic area. The quality and availability of these data have also significantly improved. Consequently, how to better utilize multi-source data to represent ground information has become an important research question in the field of geoscience. In this paper, a novel model called multi-modal transformer cascaded fusion net (MMTCFN) is proposed for fusion and classification of multi-modal remote sensing data, Hyperspectral Imagery (HSI) and LiDAR data. Feature fusion and feature extraction are the two stages of the model. First, in the feature extraction stage, a three-branch cascaded Convolutional Neural Network (CNN) framework is employed to fully leverage the advantages of convolutional operators in extracting shallow-level local features. Based on this, we generated multi-modal long-range integrated deep features utilizing the transformer-based vectorized pixel group transformer (VPGT) module during the feature fusion stage. In the VPGT block, we designed a vectorized pixel group embedding that preserves the global features extracted from the three branches in a non-overlapping multi-space manner. Moreover, we introduce the DropKey mechanism into the multi-head self-attention (MHSA) to alleviate overfitting caused by insufficient training samples. Finally, we employ a probabilistic decision fusion strategy to integrate multiple class estimations, assigning a specific category to each pixel. This model was experimented on three HSI-LiDAR datasets with balanced and unbalanced training samples. The proposed model outperforms the other seven SOTA approaches in terms of OA performance, proving the superiority of MMTCFN for the HSI-LiDAR classification task.
Förderung
- National Key Research and Development Program of China
- Funded Project of Fundamental Scientific Research Business Expenses of Chinese Academy of Surveying and Mapping
Zitieren
APA
Wang, S., Hou, C., Chen, Y., Liu, Z., Zhang, Z., & Zhang, G. (2023). Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net. Remote Sensing, 15(17), 4142. https://doi.org/10.3390/rs15174142
Vancouver
Wang S, Hou C, Chen Y, Liu Z, Zhang Z, Zhang G. Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net. Remote Sensing. 2023;15(17):4142. doi:10.3390/rs15174142
BibTeX
@article{Wang2023,
author = {Wang, Shuo and Hou, Chengchao and Chen, Yiming and Liu, Zhengjun and Zhang, Zhenbei and Zhang, Geng},
title = {Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net},
journal = {Remote Sensing},
year = {2023},
volume = {15},
number = {17},
pages = {4142},
doi = {10.3390/rs15174142}
}
EUVIMED