EUVIMEDEUVIMED entdecken
Crossref

Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities

Junchi Chen, Richong Zhang, Junfan Chen

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (15) · 15812-15820

DOI: 10.1609/aaai.v39i15.33736 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Junchi Chen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Supervised multimodal classification has been proven to outperform unimodal classification in the image-text domain. However, this task is highly dependent on abundant labeled data. To perform multimodal classification in data-insufficient scenarios, in this study, we explore semi-supervised multimodal classification (SSMC) that only requires a small amount of labeled data and plenty of unlabeled data. Specifically, we first design baseline SSMC models by combining known semi supervised pseudo-labeling methods with the two most commonly used modal fusion strategies, i.e. feature-level fusion and label-level aggregation. Based on our investigation and empirical study of the baselines, we discover two complementarities that may benefit SSMC if properly exploited: the predictions from different modalities (modal complementarity) and modal fusion strategies for pseudo-labeling (strategic complementarity). Therefore, we propose a Modal and Strategic Complementarity (MSC) framework for SSMC. Concretely, to exploit modal complementarity, we propose to learn reliability weights for the predictions from different modalities and refine the fusion scores. To learn from strategic complementarity, we introduce a dual KL divergence loss to guide the balance of quantity and quality of pseudo-labeled data selection. Extensive empirical studies demonstrate the effectiveness of the proposed framework.

Zitieren

APA

Chen, J., Zhang, R., & Chen, J. (2025). Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities. Proceedings of the AAAI Conference on Artificial Intelligence, 39(15), 15812-15820. https://doi.org/10.1609/aaai.v39i15.33736

Vancouver

Chen J, Zhang R, Chen J. Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(15):15812-15820. doi:10.1609/aaai.v39i15.33736

BibTeX
@article{Chen2025,
  author = {Chen, Junchi and Zhang, Richong and Chen, Junfan},
  title = {Semi-Supervised Multimodal Classification Through Learning from Modal and Strategic Complementarities},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {15},
  pages = {15812--15820},
  doi = {10.1609/aaai.v39i15.33736}
}
Kostenlos starten →