EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CSTC: Visual Transformer Network with Multimodal Dual Fusion for Hyperspectral and LiDAR Image Classification

Yong Mei, Jinlong Fan, Xiangsuo Fan, Qi Li

Remote Sensing · 11.09.2025 · 17 (18) · 3158

DOI: 10.3390/rs17183158 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 11.09.2025. Verfasst von Yong Mei und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Field Scientific Observation and Research Station for Ecological Protection Red line in Poyang Lake Basin of the Ministry of Natural Resources Open Project Fun…, Guangxi Key Research and Development Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Convolutional neural networks have made significant progress in multimodal remote sensing image classification, but traditional convolutional neural networks are limited by fixed-size convolutional kernels, which are unable to effectively model and adequately extract contextual information; hyperspectral imagery and LiDAR data have comparatively large information differences, which do not allow for effective information interaction and fusion. Based on this, this paper proposes a multimodal dual fusion network (CSTC) based on the Vision Transformer for the collaborative classification of HSI and LiDAR data. The model is designed through a two-branch architecture: the HSI branch extracts spectral–spatial features by dimensionality reduction using principal component analysis and inputs them into the cross-connectivity feature fusion module; the LiDAR branch mines spatial elevation features through the stacked MobileNetV2 module. The features of the two branches are encoded by a Transformer, and the modal interaction fusion is realized by the cross-attention module for the first time. Then, the features are spliced and input into the secondary Transformer for deep cross-modal fusion, and finally, the classification is completed by the multilayer perceptron. Experiments show that the CSTC model achieves overall classification accuracies of 92.32%, 99.81%, 97.90%, and 99.37% on the publicly available MUUFL dataset, Trento dataset, Augsburg dataset, and Houston2013 dataset, respectively, which is superior to the latest HSI–LiDAR separate classification algorithms. The ablation experiments and model performance evaluation experiments further show that the proposed CSTC model achieves excellent results in terms of robustness, adaptability, and parameter scale.

Förderung

  • Field Scientific Observation and Research Station for Ecological Protection Red line in Poyang Lake Basin of the Ministry of Natural Resources Open Project Fun…
  • Guangxi Key Research and Development Program

Zitieren

APA

Mei, Y., Fan, J., Fan, X., & Li, Q. (2025). CSTC: Visual Transformer Network with Multimodal Dual Fusion for Hyperspectral and LiDAR Image Classification. Remote Sensing, 17(18), 3158. https://doi.org/10.3390/rs17183158

Vancouver

Mei Y, Fan J, Fan X, Li Q. CSTC: Visual Transformer Network with Multimodal Dual Fusion for Hyperspectral and LiDAR Image Classification. Remote Sensing. 2025;17(18):3158. doi:10.3390/rs17183158

BibTeX
@article{Mei2025,
  author = {Mei, Yong and Fan, Jinlong and Fan, Xiangsuo and Li, Qi},
  title = {CSTC: Visual Transformer Network with Multimodal Dual Fusion for Hyperspectral and LiDAR Image Classification},
  journal = {Remote Sensing},
  year = {2025},
  volume = {17},
  number = {18},
  pages = {3158},
  doi = {10.3390/rs17183158}
}
Kostenlos starten →