EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Joint Classification of Hyperspectral and LiDAR Data Based on Adaptive Gating Mechanism and Learnable Transformer

Minhui Wang, Yaxiu Sun, Jianhong Xiang, Rui Sun, Yu Zhong

Remote Sensing · 19.03.2024 · 16 (6) · 1080

DOI: 10.3390/rs16061080 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 19.03.2024. Verfasst von Minhui Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities, National Key R&D Program of China, National Key Laboratory of Communication Anti Jamming Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Utilizing multi-modal data, as opposed to only hyperspectral image (HSI), enhances target identification accuracy in remote sensing. Transformers are applied to multi-modal data classification for their long-range dependency but often overlook intrinsic image structure by directly flattening image blocks into vectors. Moreover, as the encoder deepens, unprofitable information negatively impacts classification performance. Therefore, this paper proposes a learnable transformer with an adaptive gating mechanism (AGMLT). Firstly, a spectral–spatial adaptive gating mechanism (SSAGM) is designed to comprehensively extract the local information from images. It mainly contains point depthwise attention (PDWA) and asymmetric depthwise attention (ADWA). The former is for extracting spectral information of HSI, and the latter is for extracting spatial information of HSI and elevation information of LiDAR-derived rasterized digital surface models (LiDAR-DSM). By omitting linear layers, local continuity is maintained. Then, the layer Scale and learnable transition matrix are introduced to the original transformer encoder and self-attention to form the learnable transformer (L-Former). It improves data dynamics and prevents performance degradation as the encoder deepens. Subsequently, learnable cross-attention (LC-Attention) with the learnable transfer matrix is designed to augment the fusion of multi-modal data by enriching feature information. Finally, poly loss, known for its adaptability with multi-modal data, is employed in training the model. Experiments in the paper are conducted on four famous multi-modal datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and Houston2013 (HU). The results show that AGMLT achieves optimal performance over some existing models.

Förderung

  • Fundamental Research Funds for the Central Universities
  • National Key R&D Program of China
  • National Key Laboratory of Communication Anti Jamming Technology

Zitieren

APA

Wang, M., Sun, Y., Xiang, J., Sun, R., & Zhong, Y. (2024). Joint Classification of Hyperspectral and LiDAR Data Based on Adaptive Gating Mechanism and Learnable Transformer. Remote Sensing, 16(6), 1080. https://doi.org/10.3390/rs16061080

Vancouver

Wang M, Sun Y, Xiang J, Sun R, Zhong Y. Joint Classification of Hyperspectral and LiDAR Data Based on Adaptive Gating Mechanism and Learnable Transformer. Remote Sensing. 2024;16(6):1080. doi:10.3390/rs16061080

BibTeX
@article{Wang2024,
  author = {Wang, Minhui and Sun, Yaxiu and Xiang, Jianhong and Sun, Rui and Zhong, Yu},
  title = {Joint Classification of Hyperspectral and LiDAR Data Based on Adaptive Gating Mechanism and Learnable Transformer},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {6},
  pages = {1080},
  doi = {10.3390/rs16061080}
}
Kostenlos starten →