VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (14) · 16085-16093
DOI: 10.1609/aaai.v38i14.29541 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yi Xin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 59-mal zitiert (Crossref).
Abstract (Originalsprache)
Large-scale pre-trained models have achieved remarkable success in various computer vision tasks. A standard approach to leverage these models is to fine-tune all model parameters for downstream tasks, which poses challenges in terms of computational and storage costs. Recently, inspired by Natural Language Processing (NLP), parameter-efficient transfer learning has been successfully applied to vision tasks. However, most existing techniques primarily focus on single-task adaptation, and despite limited research on multi-task adaptation, these methods often exhibit suboptimal training/inference efficiency. In this paper, we first propose an once-for-all Vision Multi-Task Adapter (VMT-Adapter), which strikes approximately O(1) training and inference efficiency w.r.t task number. Concretely, VMT-Adapter shares the knowledge from multiple tasks to enhance cross-task interaction while preserves task-specific knowledge via independent knowledge extraction modules. Notably, since task-specific modules require few parameters, VMT-Adapter can handle an arbitrary number of tasks with a negligible increase of trainable parameters. We also propose VMT-Adapter-Lite, which further reduces the trainable parameters by learning shared parameters between down- and up-projections. Extensive experiments on four dense scene understanding tasks demonstrate the superiority of VMT-Adapter(-Lite), achieving a 3.96% (1.34%) relative improvement compared to single-task full fine-tuning, while utilizing merely ~1% (0.36%) trainable parameters of the pre-trained model.
Zitieren
APA
Xin, Y., Du, J., Wang, Q., Lin, Z., & Yan, K. (2024). VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 38(14), 16085-16093. https://doi.org/10.1609/aaai.v38i14.29541
Vancouver
Xin Y, Du J, Wang Q, Lin Z, Yan K. VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(14):16085-16093. doi:10.1609/aaai.v38i14.29541
BibTeX
@article{Xin2024,
author = {Xin, Yi and Du, Junlong and Wang, Qiang and Lin, Zhiwen and Yan, Ke},
title = {VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {14},
pages = {16085--16093},
doi = {10.1609/aaai.v38i14.29541}
}
EUVIMED