EUVIMEDEUVIMED entdecken
Crossref

RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation

Hanbo Bi, Yingchao Feng, Boyuan Tong, Mengyu Wang, Haichen Yu, Yongqiang Mao, Hao Chang, Wenhui Diao, Peijin Wang, Yue Yu, Hanyang Peng, Yehong Zhang und 2 weitere

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-04 · 48 (4) · 4388-4405

DOI: 10.1109/tpami.2025.3643453 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-04. Verfasst von Hanbo Bi und 13 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 90 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid advancement of foundation models has revolutionized visual representation learning in a self-supervised manner. However, their application in remote sensing (RS) remains constrained by a fundamental gap: existing models predominantly handle single or limited modalities, overlooking the inherently multi-modal nature of RS observations. Optical, synthetic aperture radar (SAR), and multi-spectral data offer complementary insights that significantly reduce the inherent ambiguity and uncertainty in single-source analysis. To bridge this gap, we introduce RingMoE, a unified multi-modal RS foundation model with 14.7 billion parameters, pre-trained on 400 million multi-modal RS images from nine satellites. RingMoE incorporates three key innovations: 1) A hierarchical Mixture-of-Experts (MoE) architecture comprising modal-specialized, collaborative, and shared experts, effectively modeling intra-modal knowledge while capturing cross-modal dependencies to mitigate conflicts between modal representations; 2) Physics-informed self-supervised learning, explicitly embedding sensor-specific radiometric characteristics into the pre-training objectives; 3) Dynamic expert pruning, enabling adaptive model compression from 14.7B to 1B parameters while maintaining performance, facilitating efficient deployment in Earth observation applications. Evaluated across 23 benchmarks spanning six key RS tasks (i.e., classification, detection, segmentation, tracking, change detection, and depth estimation), RingMoE outperforms existing foundation models and sets new SOTAs, demonstrating remarkable adaptability from single-modal to multi-modal scenarios. Beyond theoretical progress, it has been deployed and trialed in multiple sectors, including emergency response, land management, marine sciences, and urban planning.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Bi, H., Feng, Y., Tong, B., Wang, M., Yu, H., Mao, Y., Chang, H., Diao, W., Wang, P., Yu, Y., Peng, H., Zhang, Y., Fu, K., & Sun, X. (2026). RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(4), 4388-4405. https://doi.org/10.1109/tpami.2025.3643453

Vancouver

Bi H, Feng Y, Tong B, Wang M, Yu H, Mao Y, et al. RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(4):4388-4405. doi:10.1109/tpami.2025.3643453

BibTeX
@article{Bi2026,
  author = {Bi, Hanbo and Feng, Yingchao and Tong, Boyuan and Wang, Mengyu and Yu, Haichen and Mao, Yongqiang and Chang, Hao and Diao, Wenhui and Wang, Peijin and Yu, Yue and Peng, Hanyang and Zhang, Yehong and Fu, Kun and Sun, Xian},
  title = {RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {4},
  pages = {4388--4405},
  doi = {10.1109/tpami.2025.3643453}
}
Kostenlos starten →