EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MoSA: Modality-Aware Spatially-Adaptive Adaptation for RGB-X Semantic Segmentation

Xing Zhang, Kunlei Dong, Wei Wang, Huasheng Yang, Lei Chen, Zhengpin Li, Jian Zhang

IEEE Access · 2026 · 14 · 81631-81644

DOI: 10.1109/access.2026.3694496 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Xing Zhang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

RGB-X semantic segmentation enhances semantic perception by fusing complementary features from two modalities. However, the contribution of each modality to segmentation predictions varies across spatial locations depending on local content characteristics, an issue that has been insufficiently addressed by existing methods. To tackle this challenge, we present MoSA, a novel framework for adapting vision foundation models to RGB-X semantic segmentation via Modality-aware Spatially-Adaptive learning. MoSA introduces a Modality-Specific Spatially-Modulated Adapter (MS-SMA) to capture modality-specific representations while dynamically adjusting adaptation intensity based on both global scene context and local content information. Furthermore, MoSA proposes Reliability-Guided Cross-Modal Fusion (RGCF), which estimates per-location modality reliability using feature statistics and performs adaptively weighted fusion. MoSA is a model-agnostic framework that can be seamlessly applied to various multimodal combinations without task-specific modifications. MoSA outperforms state-of-the-art methods in mIoU by 2.3% on the NYU Depth V2 dataset and by 2.1% on the MFNet dataset, requiring only 7.7% of the trainable parameters compared to OmniSegmentor.

Zitieren

APA

Zhang, X., Dong, K., Wang, W., Yang, H., Chen, L., Li, Z., & Zhang, J. (2026). MoSA: Modality-Aware Spatially-Adaptive Adaptation for RGB-X Semantic Segmentation. IEEE Access, 14, 81631-81644. https://doi.org/10.1109/access.2026.3694496

Vancouver

Zhang X, Dong K, Wang W, Yang H, Chen L, Li Z, et al. MoSA: Modality-Aware Spatially-Adaptive Adaptation for RGB-X Semantic Segmentation. IEEE Access. 2026;14:81631-81644. doi:10.1109/access.2026.3694496

BibTeX
@article{Zhang2026,
  author = {Zhang, Xing and Dong, Kunlei and Wang, Wei and Yang, Huasheng and Chen, Lei and Li, Zhengpin and Zhang, Jian},
  title = {MoSA: Modality-Aware Spatially-Adaptive Adaptation for RGB-X Semantic Segmentation},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {81631--81644},
  doi = {10.1109/access.2026.3694496}
}
Kostenlos starten →