EUVIMEDEUVIMED entdecken
Crossref

Reconcile Gradient Modulation for Harmony Multimodal Learning

Xiyuan Gao, Bing Cao, Baoquan Gong, Pengfei Zhu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (25) · 21225-21233

DOI: 10.1609/aaai.v40i25.39267 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiyuan Gao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal learning frequently faces two coupled challenges: modality imbalance, where dominant modalities suppress others during training, and modality conflict, where opposing gradient directions hinder optimization. Existing methods typically address these issues in isolation, yet they are intrinsically correlated and most fundamentally reflected in the gradient space—severe imbalance may obscure conflicts, while suppressing conflict may homogenize features and worsen imbalance, affecting fusion performance. To jointly address this coupled challenge, we propose Reconcile Gradient Modulation (RGM), a unified framework that adaptively adjusts gradient magnitude and direction for harmony multimodal learning. The core of RGM is SynOrth Grad, which minimizes Dirichlet energy to perform minimal-gradient surgery. It enhances cooperation synergy when modalities are aligned and enforces orthogonality to preserve uniqueness in conflict situations, thus promoting stable and balanced learning. To guide this modulation, we propose Cumulative Gradient Energy (CGE) as a convergence-guaranteed measure of modality-wise progress, and construct a Balance-nonConflict Plane (BCP) for real-time diagnosis and control of training dynamics. Experiments on diverse benchmarks validate our effectiveness and generalizability, consistently outperforming counterparts that are designed to handle multimodal imbalance or conflict independently.

Zitieren

APA

Gao, X., Cao, B., Gong, B., & Zhu, P. (2026). Reconcile Gradient Modulation for Harmony Multimodal Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(25), 21225-21233. https://doi.org/10.1609/aaai.v40i25.39267

Vancouver

Gao X, Cao B, Gong B, Zhu P. Reconcile Gradient Modulation for Harmony Multimodal Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(25):21225-21233. doi:10.1609/aaai.v40i25.39267

BibTeX
@article{Gao2026,
  author = {Gao, Xiyuan and Cao, Bing and Gong, Baoquan and Zhu, Pengfei},
  title = {Reconcile Gradient Modulation for Harmony Multimodal Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {25},
  pages = {21225--21233},
  doi = {10.1609/aaai.v40i25.39267}
}
Kostenlos starten →