EUVIMEDEUVIMED entdecken
Crossref

Alliance: All-in-One Spectral-Spatial-Frequency Awareness Foundation Model

Boyu Zhao, Wei Li, Junjie Wang, Yuxiang Zhang, Hong Yang, Haitao Zhao, Ran Tao, Qian Du

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-03 · 48 (3) · 3586-3603

DOI: 10.1109/tpami.2025.3639595 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-03. Verfasst von Boyu Zhao und 7 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 76 Literaturangaben. Gefördert von: NSFC Projects of International Cooperation and Exchanges.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Frequency domain analysis reveals fundamental image patterns difficult to observe in raw pixel values, while avoiding redundant information in original image processing. Although recent remote sensing foundation models (FMs) have made progress in leveraging spatial and spectral information, they have limitations in fully utilizing frequency characteristics that capture hidden features. Existing FMs that incorporate frequency properties often struggle to maintain connections with the original image content, creating a semantic gap that affects downstream performance. To address these challenges, we propose the All-in-One Spectral-Spatial-Frequency Awareness Foundation Model (Alliance), a framework that effectively integrates information across all three domains. Alliance introduces several key innovations: (1) a progressive frequency decoding mechanism inspired by human visual cognition that minimizes multi-domain information gaps while preserving connections between general image information and frequency characteristics, progressively reconstructing from low to mid to high frequencies to extract patterns difficult to observe in raw pixel values; (2) a triple-domain fusion attention module that separately processes amplitude, phase, and spectral-spatial relationships for comprehensive feature integration; and (3) frequency embedding with frequency-aware Cls token initialization and frequency-specific mask token initialization that achieves fine-grained modeling of different frequency band information. Additionally, to evaluate FMs generalizability, we construct the Yellow River dataset, a large-scale multi-temporal collection that introduces challenging cross-domain tasks and establishes more rigorous standards for FMs assessment. Extensive experiments across six downstream tasks demonstrate Alliance's superior performance.

Förderung

  • NSFC Projects of International Cooperation and Exchanges

Zitieren

APA

Zhao, B., Li, W., Wang, J., Zhang, Y., Yang, H., Zhao, H., Tao, R., & Du, Q. (2026). Alliance: All-in-One Spectral-Spatial-Frequency Awareness Foundation Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3), 3586-3603. https://doi.org/10.1109/tpami.2025.3639595

Vancouver

Zhao B, Li W, Wang J, Zhang Y, Yang H, Zhao H, et al. Alliance: All-in-One Spectral-Spatial-Frequency Awareness Foundation Model. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(3):3586-3603. doi:10.1109/tpami.2025.3639595

BibTeX
@article{Zhao2026,
  author = {Zhao, Boyu and Li, Wei and Wang, Junjie and Zhang, Yuxiang and Yang, Hong and Zhao, Haitao and Tao, Ran and Du, Qian},
  title = {Alliance: All-in-One Spectral-Spatial-Frequency Awareness Foundation Model},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {3},
  pages = {3586--3603},
  doi = {10.1109/tpami.2025.3639595}
}
Kostenlos starten →