Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint
IEEE Transactions on Circuits and Systems for Video Technology · 2025-02 · 35 (2) · 1070-1083
DOI: 10.1109/tcsvt.2024.3470996 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-02. Verfasst von Pinhan Fu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China, Science and Technology Major Project of Shanxi.
Abstract (Originalsprache)
Multi-modal neural architecture search (MNAS) is an effective approach to obtain task-adaptive multi-modal classification models. Deep neural networks, as currently main-stream feature extractors, can provide hierarchical features for each modality. Existing MNAS methods face difficulty in exploiting such hierarchical features due to their different form coexistence such as tensorial multi-scale features and vectorized penultimate features. Moreover, existing methods always focus on the evolution of fusion operators or vectorized features of all modalities, constraining search space. In this paper, a novel two-stage method called multi-modal multi-scale evolutionary neural architecture search (MM-ENAS) is proposed. The first stage unifies the representation form of hierarchical features by the proposed evolutionary statistics strategy. The second stage identifies the optimal combination of basic fusion operations for all unified hierarchical features by the evolutionary algorithm. MM-ENAS increases search space by simultaneously searching for feature statistical extraction methods, basic fusion operators and feature representation set consisting of tensorial multi-scale features and vectorized penultimate features. Experimental results on three multi-modal tasks demonstrate that the proposed method achieves competitive performance in terms of accuracy, search time, and number of parameters compared to existing representative MNAS methods. Additionally, the method exhibits fast adaptation to various multi-modal tasks.
Förderung
- National Natural Science Foundation of China
- National Key Research and Development Program of China
- Science and Technology Major Project of Shanxi
- Young Scientists Fund of the Natural Science Foundation of Shanxi
- Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province
- Science and Technology Innovation Plan for Colleges and Universities of Shanxi Province
- Taiyuan University of Science and Technology Doctoral Research Start-Up Fund Project
Zitieren
APA
Fu, P., Liang, X., Qian, Y., Guo, Q., Zhang, Y., Huang, Q., & Tang, K. (2025). Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint. IEEE Transactions on Circuits and Systems for Video Technology, 35(2), 1070-1083. https://doi.org/10.1109/tcsvt.2024.3470996
Vancouver
Fu P, Liang X, Qian Y, Guo Q, Zhang Y, Huang Q, et al. Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(2):1070-1083. doi:10.1109/tcsvt.2024.3470996
BibTeX
@article{Fu2025,
author = {Fu, Pinhan and Liang, Xinyan and Qian, Yuhua and Guo, Qian and Zhang, Yayu and Huang, Qin and Tang, Ke},
title = {Multi-Scale Features Are Effective for Multi-Modal Classification: An Architecture Search Viewpoint},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {2},
pages = {1070--1083},
doi = {10.1109/tcsvt.2024.3470996}
}
EUVIMED