EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multimodal Semantic Collaborative Classification for Hyperspectral Images and LiDAR Data

Aili Wang, Shiyu Dai, Haibin Wu, Yuji Iwahori

Remote Sensing · 21.08.2024 · 16 (16) · 3082

DOI: 10.3390/rs16163082 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 21.08.2024. Verfasst von Aili Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Key Research and Development Plan Project of Heilongjiang, Natural Science Foundation of Heilongjiang Province, high-end foreign expert introduction program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Although the collaborative use of hyperspectral images (HSIs) and LiDAR data in land cover classification tasks has demonstrated significant importance and potential, several challenges remain. Notably, the heterogeneity in cross-modal information integration presents a major obstacle. Furthermore, most existing research relies heavily on category names, neglecting the rich contextual information from language descriptions. Visual-language pretraining (VLP) has achieved notable success in image recognition within natural domains by using multimodal information to enhance training efficiency and effectiveness. VLP has also shown great potential for land cover classification in remote sensing. This paper introduces a dual-sensor multimodal semantic collaborative classification network (DSMSC2N). It uses large language models (LLMs) in an instruction-driven manner to generate land cover category descriptions enriched with domain-specific knowledge in remote sensing. This approach aims to guide the model to accurately focus on and extract key features. Simultaneously, we integrate and optimize the complementary relationship between HSI and LiDAR data, enhancing the separability of land cover categories and improving classification accuracy. We conduct comprehensive experiments on benchmark datasets like Houston 2013, Trento, and MUUFL Gulfport, validating DSMSC2N’s effectiveness compared to various baseline methods.

Förderung

  • Key Research and Development Plan Project of Heilongjiang
  • Natural Science Foundation of Heilongjiang Province
  • high-end foreign expert introduction program
  • Science and Technology Project of Heilongjiang Provincial Department of Transportation
  • Key Research and Development Program Guidance Project of Heilongjiang
  • Japan Society for the Promotion of Science (JSPS) Grant-in-Aid Scientific Research(C)
  • Chubu University Grant

Zitieren

APA

Wang, A., Dai, S., Wu, H., & Iwahori, Y. (2024). Multimodal Semantic Collaborative Classification for Hyperspectral Images and LiDAR Data. Remote Sensing, 16(16), 3082. https://doi.org/10.3390/rs16163082

Vancouver

Wang A, Dai S, Wu H, Iwahori Y. Multimodal Semantic Collaborative Classification for Hyperspectral Images and LiDAR Data. Remote Sensing. 2024;16(16):3082. doi:10.3390/rs16163082

BibTeX
@article{Wang2024,
  author = {Wang, Aili and Dai, Shiyu and Wu, Haibin and Iwahori, Yuji},
  title = {Multimodal Semantic Collaborative Classification for Hyperspectral Images and LiDAR Data},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {16},
  pages = {3082},
  doi = {10.3390/rs16163082}
}
Kostenlos starten →