EUVIMEDEUVIMED entdecken
Crossref

Invariance, Stability, and Diversity: A Unified Framework for Multimodal Domain Generalization

Hongzhao Li, Shizhe Hu, Xianglong Shen, Hualei Wan, Mingyuan Jiu, Shupan Li, Muhammad Haris Khan, Mingliang Xu

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026 · 1-18

DOI: 10.1109/tpami.2026.3750284 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. Verfasst von Hongzhao Li und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Domain Generalization (MMDG) aims to learn models that generalize to unseen domains under unpredictable, modality-dependent distribution shifts. Despite recent progress, existing methods often rely on heuristic fusion strategies and lack a unified theoretical perspective, limiting their robustness under domain shifts. In this paper, we identify three fundamental challenges in MMDG: (I) insufficient cross-domain invariance across individual modalities and cross-modal interaction subspaces; (II) instability induced by asynchronous modality degradation, through which unreliable modalities impair joint representations; and (III) limited representation diversity, which exacerbates source-target support mismatch. To characterize these challenges, we derive a generalization error bound for a stability-weighted ensemble of modality-wise hypotheses, revealing three corresponding factors: Invariance (Source-Source Divergence), Stability (Stability-Weighted Error), and Diversity (Target-Convex Hull Divergence). Guided by this analysis, we develop a unified nonlinear fusion framework comprising Multi-Granularity Adversarial Alignment to promote invariance across modality-specific, interaction, and fusion spaces, Stability-Aware Adaptive Learning to prioritize reliable modalities during source training and mitigate asynchronous degradation, and Manifold Expansion to increase normalized source-feature dispersion as a source-side regularizer. Extensive experiments on multiple benchmarks show that our framework achieves the best overall average rank across 43 transfer tasks on seven datasets spanning four task families and eight diverse modalities.

Zitieren

APA

Li, H., Hu, S., Shen, X., Wan, H., Jiu, M., Li, S., Khan, M. H., & Xu, M. (2026). Invariance, Stability, and Diversity: A Unified Framework for Multimodal Domain Generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1-18. https://doi.org/10.1109/tpami.2026.3750284

Vancouver

Li H, Hu S, Shen X, Wan H, Jiu M, Li S, et al. Invariance, Stability, and Diversity: A Unified Framework for Multimodal Domain Generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026:1-18. doi:10.1109/tpami.2026.3750284

BibTeX
@article{Li2026,
  author = {Li, Hongzhao and Hu, Shizhe and Shen, Xianglong and Wan, Hualei and Jiu, Mingyuan and Li, Shupan and Khan, Muhammad Haris and Xu, Mingliang},
  title = {Invariance, Stability, and Diversity: A Unified Framework for Multimodal Domain Generalization},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  pages = {1--18},
  doi = {10.1109/tpami.2026.3750284}
}
Kostenlos starten →