Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer
Sensors · 29.01.2024 · 24 (3) · 867
DOI: 10.3390/s24030867 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 29.01.2024. Verfasst von Qingyan Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Measurement-Control and Communication Engineering, Harbin University of Science and Technology, Harbin 150080, China. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Heilongjiang Province Outstanding Young Teacher Basic ResearchSupport Program, Heilongjiang Province Outstanding Young Teacher Basic Research Support Program.
Abstract (Originalsprache)
In the face of complex scenarios, the information insufficiency of classification tasks dominated by a single modality has led to a bottleneck in classification performance. The joint application of multimodal remote sensing data for surface observation tasks has garnered widespread attention. However, issues such as sample differences between modalities and the lack of correlation in physical features have limited the performance of classification tasks. Establishing effective interaction between multimodal data has become another significant challenge. To fully integrate heterogeneous information from multiple modalities and enhance classification performance, this paper proposes a dual-branch cross-Transformer feature fusion network aimed at joint land cover classification of hyperspectral imagery (HSI) and Light Detection and Ranging (LiDAR) data. The core idea is to leverage the potential of convolutional operators to represent spatial features, combined with the advantages of the Transformer architecture in learning remote dependencies. The framework employs an improved self-attention mechanism to aggregate features within each modality, highlighting the spectral information of HSI and the spatial (elevation) information of LiDAR. The feature fusion module based on cross-attention integrates deep features from two modalities, achieving complementary information through cross-modal attention. The classification task is performed using jointly obtained spectral and spatial features. Experiments were conducted on three multi-source remote sensing classification datasets, demonstrating the effectiveness of the proposed model compared to existing methods.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Measurement-Control and Communication Engineering, Harbin University of Science and Technology, Harbin 150080, China.
- School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China.
- College of Physics and Electronic Engineering, Hainan Normal University, Haikou 571158, China.
ORCID-Profile: Qingyan Wang, Junping Zhang
Förderung
- National Natural Science Foundation of China
- Heilongjiang Province Outstanding Young Teacher Basic ResearchSupport Program
- Heilongjiang Province Outstanding Young Teacher Basic Research Support Program
Zitieren
APA
Wang, Q., Zhou, B., Zhang, J., Xie, J., & Wang, Y. (2024). Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer. Sensors, 24(3), 867. https://doi.org/10.3390/s24030867
Vancouver
Wang Q, Zhou B, Zhang J, Xie J, Wang Y. Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer. Sensors. 2024;24(3):867. doi:10.3390/s24030867
BibTeX
@article{Wang2024,
author = {Wang, Qingyan and Zhou, Binbin and Zhang, Junping and Xie, Jinbao and Wang, Yujing},
title = {Joint Classification of Hyperspectral Images and LiDAR Data Based on Dual-Branch Transformer},
journal = {Sensors},
year = {2024},
volume = {24},
number = {3},
pages = {867},
doi = {10.3390/s24030867}
}
EUVIMED