EUVIMEDEUVIMED entdecken
Crossref

Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning

Heqiang Wang, Weihong Yang, Xiaoxiong Zhong, Jia Zhou, Fangming Liu, Weizhe Zhang, Keqin Li

IEEE Transactions on Signal Processing · 2026 · 74 · 2933-2948

DOI: 10.1109/tsp.2026.3712709 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Signal Processing, 2026. Verfasst von Heqiang Wang und 6 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Gefördert von: Major Key Project of Pengcheng Laboratory, Shenzhen Science and Technology Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The Internet of Things (IoT) ecosystem produces massive volumes of multimodal data from diverse sources, including sensors, cameras, and microphones. With advances in edge intelligence, IoT devices have evolved from simple data acquisition units into computationally capable nodes, enabling localized processing of heterogeneous multimodal data. This evolution necessitates distributed learning paradigms that can efficiently handle such data. Furthermore, the continuous nature of data generation and the limited storage capacity of edge devices demand an online learning framework. Multimodal Online Federated Learning (MMO-FL) has been identified as a compelling strategy to address these requirements. However, MMO-FL faces new challenges due to the inherent instability of IoT devices, which often results in modality quantity and quality imbalance (QQI) during data collection. In this study, we systematically investigate the impact of QQI within the MMO-FL framework and present a comprehensive theoretical analysis quantifying how both types of imbalance degrade learning performance. To tackle these issues, we propose the Modality Quantity and Quality Rebalanced (QQR) algorithm, a prototype learning based method designed to operate in parallel with the training process. Extensive experiments on two real-world multimodal datasets show that the proposed QQR algorithm consistently outperforms benchmarks under modality imbalance conditions with promising learning performance.

Förderung

  • Major Key Project of Pengcheng Laboratory
  • Shenzhen Science and Technology Program

Zitieren

APA

Wang, H., Yang, W., Zhong, X., Zhou, J., Liu, F., Zhang, W., & Li, K. (2026). Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning. IEEE Transactions on Signal Processing, 74, 2933-2948. https://doi.org/10.1109/tsp.2026.3712709

Vancouver

Wang H, Yang W, Zhong X, Zhou J, Liu F, Zhang W, et al. Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning. IEEE Transactions on Signal Processing. 2026;74:2933-2948. doi:10.1109/tsp.2026.3712709

BibTeX
@article{Wang2026,
  author = {Wang, Heqiang and Yang, Weihong and Zhong, Xiaoxiong and Zhou, Jia and Liu, Fangming and Zhang, Weizhe and Li, Keqin},
  title = {Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning},
  journal = {IEEE Transactions on Signal Processing},
  year = {2026},
  volume = {74},
  pages = {2933--2948},
  doi = {10.1109/tsp.2026.3712709}
}
Kostenlos starten →