EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification

Haimiao Ge, Liguo Wang, Haizhu Pan, Yanzhong Liu, Cheng Li, Dan Lv, Huiyu Ma

Remote Sensing · 31.10.2024 · 16 (21) · 4073

DOI: 10.3390/rs16214073 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 31.10.2024. Verfasst von Haimiao Ge und 6 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Heilongjiang Provincial Natural Science Foundation of China, Fundamental Research Funds in Heilongjiang Provincial Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, deep learning-based multi-source data fusion, e.g., hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion, has gained significant attention in the field of remote sensing. However, the traditional convolutional neural network fusion techniques always provide poor extraction of discriminative spatial–spectral features from diversified land covers and overlook the correlation and complementarity between different data sources. Furthermore, the mere act of stacking multi-source feature embeddings fails to represent the deep semantic relationships among them. In this paper, we propose a cross attention-based multi-scale convolutional fusion network for HSI-LiDAR joint classification. It contains three major modules: spatial–elevation–spectral convolutional feature extraction module (SESM), cross attention fusion module (CAFM), and classification module. In the SESM, improved multi-scale convolutional blocks are utilized to extract features from HSI and LiDAR to ensure discriminability and comprehensiveness in diversified land cover conditions. Spatial and spectral pseudo-3D convolutions, pointwise convolutions, residual aggregation, one-shot aggregation, and parameter-sharing techniques are implemented in the module. In the CAFM, a self-designed local-global cross attention block is utilized to collect and integrate relationships of the feature embeddings and generate joint semantic representations. In the classification module, average polling, dropout, and linear layers are used to map the fused semantic representations to the final classification results. The experimental evaluations on three public HSI-LiDAR datasets demonstrate the competitiveness of the proposed network in comparison with state-of-the-art methods.

Förderung

  • National Natural Science Foundation of China
  • Heilongjiang Provincial Natural Science Foundation of China
  • Fundamental Research Funds in Heilongjiang Provincial Universities

Zitieren

APA

Ge, H., Wang, L., Pan, H., Liu, Y., Li, C., Lv, D., & Ma, H. (2024). Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification. Remote Sensing, 16(21), 4073. https://doi.org/10.3390/rs16214073

Vancouver

Ge H, Wang L, Pan H, Liu Y, Li C, Lv D, et al. Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification. Remote Sensing. 2024;16(21):4073. doi:10.3390/rs16214073

BibTeX
@article{Ge2024,
  author = {Ge, Haimiao and Wang, Liguo and Pan, Haizhu and Liu, Yanzhong and Li, Cheng and Lv, Dan and Ma, Huiyu},
  title = {Cross Attention-Based Multi-Scale Convolutional Fusion Network for Hyperspectral and LiDAR Joint Classification},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {21},
  pages = {4073},
  doi = {10.3390/rs16214073}
}
Kostenlos starten →