EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net

Shuo Wang, Chengchao Hou, Yiming Chen, Zhengjun Liu, Zhenbei Zhang, Geng Zhang

Remote Sensing · 24.08.2023 · 15 (17) · 4142

DOI: 10.3390/rs15174142 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 24.08.2023. Verfasst von Shuo Wang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, Funded Project of Fundamental Scientific Research Business Expenses of Chinese Academy of Surveying and Mapping.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the continuous development of surface observation methods and technologies, we can acquire multiple sources of data more effectively in the same geographic area. The quality and availability of these data have also significantly improved. Consequently, how to better utilize multi-source data to represent ground information has become an important research question in the field of geoscience. In this paper, a novel model called multi-modal transformer cascaded fusion net (MMTCFN) is proposed for fusion and classification of multi-modal remote sensing data, Hyperspectral Imagery (HSI) and LiDAR data. Feature fusion and feature extraction are the two stages of the model. First, in the feature extraction stage, a three-branch cascaded Convolutional Neural Network (CNN) framework is employed to fully leverage the advantages of convolutional operators in extracting shallow-level local features. Based on this, we generated multi-modal long-range integrated deep features utilizing the transformer-based vectorized pixel group transformer (VPGT) module during the feature fusion stage. In the VPGT block, we designed a vectorized pixel group embedding that preserves the global features extracted from the three branches in a non-overlapping multi-space manner. Moreover, we introduce the DropKey mechanism into the multi-head self-attention (MHSA) to alleviate overfitting caused by insufficient training samples. Finally, we employ a probabilistic decision fusion strategy to integrate multiple class estimations, assigning a specific category to each pixel. This model was experimented on three HSI-LiDAR datasets with balanced and unbalanced training samples. The proposed model outperforms the other seven SOTA approaches in terms of OA performance, proving the superiority of MMTCFN for the HSI-LiDAR classification task.

Förderung

  • National Key Research and Development Program of China
  • Funded Project of Fundamental Scientific Research Business Expenses of Chinese Academy of Surveying and Mapping

Zitieren

APA

Wang, S., Hou, C., Chen, Y., Liu, Z., Zhang, Z., & Zhang, G. (2023). Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net. Remote Sensing, 15(17), 4142. https://doi.org/10.3390/rs15174142

Vancouver

Wang S, Hou C, Chen Y, Liu Z, Zhang Z, Zhang G. Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net. Remote Sensing. 2023;15(17):4142. doi:10.3390/rs15174142

BibTeX
@article{Wang2023,
  author = {Wang, Shuo and Hou, Chengchao and Chen, Yiming and Liu, Zhengjun and Zhang, Zhenbei and Zhang, Geng},
  title = {Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net},
  journal = {Remote Sensing},
  year = {2023},
  volume = {15},
  number = {17},
  pages = {4142},
  doi = {10.3390/rs15174142}
}
Kostenlos starten →