A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification
Remote Sensing · 28.03.2024 · 16 (7) · 1180
DOI: 10.3390/rs16071180 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 28.03.2024. Verfasst von Xinyu Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 42-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Jiangsu key R&D plan.
Abstract (Originalsprache)
Recently, with the remarkable advancements of deep learning in the field of image processing, convolutional neural networks (CNNs) have garnered widespread attention from researchers in the domain of hyperspectral image (HSI) classification. Moreover, due to the high performance demonstrated by the transformer architecture in classification tasks, there has been a proliferation of neural networks combining CNNs and transformers for HSI classification. However, the majority of the current methods focus on extracting spatial–spectral features from the HSI data of a single size for a pixel, overlooking the rich multi-scale feature information inherent to the data. To address this problem, we designed a novel transformer network with a CNN-enhanced cross-attention (TNCCA) mechanism for HSI classification. It is a dual-branch network that utilizes different scales of HSI input data to extract shallow spatial–spectral features using a multi-scale 3D and 2D hybrid convolutional neural network. After converting the feature maps into tokens, a series of 2D convolutions and dilated convolutions are employed to generate two sets of Q (queries), K (keys), and V (values) at different scales in a cross-attention module. This transformer with CNN-enhanced cross-attention explores multi-scale CNN-enhanced features and fuses them from both branches. Experimental evaluations conducted on three widely used hyperspectral image (HSI) datasets, under the constraint of limited sample size, demonstrate excellent classification performance of the proposed network.
Förderung
- Jiangsu key R&D plan
Zitieren
APA
Wang, X., Sun, L., Lu, C., & Li, B. (2024). A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification. Remote Sensing, 16(7), 1180. https://doi.org/10.3390/rs16071180
Vancouver
Wang X, Sun L, Lu C, Li B. A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification. Remote Sensing. 2024;16(7):1180. doi:10.3390/rs16071180
BibTeX
@article{Wang2024,
author = {Wang, Xinyu and Sun, Le and Lu, Chuhan and Li, Baozhu},
title = {A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification},
journal = {Remote Sensing},
year = {2024},
volume = {16},
number = {7},
pages = {1180},
doi = {10.3390/rs16071180}
}
EUVIMED