EUVIMEDEUVIMED entdecken
Crossref

T-APT: Text-Guided Modality-Aware Prompt Tuning for Arbitrary Multimodal Remote Sensing Data Joint Classification

Qinghao Gao, Jiahui Qu, Wenqian Dong

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (6) · 4158-4166

DOI: 10.1609/aaai.v40i6.42411 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qinghao Gao und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal remote sensing image joint classification has achieved significant progress. However, existing methods primarily focus on designing modality-specific networks, lacking adaptive generalization capabilities in diverse and dynamic modality combinations encountered in real-world scenarios. Inspired by the generalization capabilities of visual foundation model in downstream tasks, we propose a unified Text-guided Arbitrary Modalitiy Prompting (T-APT) framework, which leverages complementary fused features to drive the foundation model and employs text-guided modality-specific prior knowledge as cross-modal prompts to fine-tune a pretrained Vision Transformer (ViT) model. Specifically, a Mamba-Based Arbitrary Modal-Focused Feature Capture (MAMF-FC) module is designed to extract complementary joint features and modality-specific prior knowledge from arbitrary modalities through a shared-specific scanning encoder-decoder architecture. Subsequently, a Text-Guided Modality-Aware Prompt Tuning (TMPT) module is proposed to support the adaptation of fused features to the foundation model, enabling our arbitrary remote sensing image classification task. Extensive experiments on public datasets spanning multispectral (MS), hyperspectral (HS), light detection and ranging (LiDAR), and synthetic aperture radar (SAR) modalities demonstrate that our T-APT achieves classification performance comparable to specialized networks across arbitrary modal combinations.

Zitieren

APA

Gao, Q., Qu, J., & Dong, W. (2026). T-APT: Text-Guided Modality-Aware Prompt Tuning for Arbitrary Multimodal Remote Sensing Data Joint Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4158-4166. https://doi.org/10.1609/aaai.v40i6.42411

Vancouver

Gao Q, Qu J, Dong W. T-APT: Text-Guided Modality-Aware Prompt Tuning for Arbitrary Multimodal Remote Sensing Data Joint Classification. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(6):4158-4166. doi:10.1609/aaai.v40i6.42411

BibTeX
@article{Gao2026,
  author = {Gao, Qinghao and Qu, Jiahui and Dong, Wenqian},
  title = {T-APT: Text-Guided Modality-Aware Prompt Tuning for Arbitrary Multimodal Remote Sensing Data Joint Classification},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {6},
  pages = {4158--4166},
  doi = {10.1609/aaai.v40i6.42411}
}
Kostenlos starten →