LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (24) · 19978-19986
DOI: 10.1609/aaai.v40i24.39082 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Chang Che und 5 weiteren Autor:innen.
Abstract (Originalsprache)
Continual Visual Instruction Tuning (CVIT) enables Multimodal Large Language Models (MLLMs) to incrementally learn new tasks over time. However, this process is challenged by catastrophic forgetting, where performance on previously learned tasks deteriorates as the model adapts to new ones. A common approach to mitigate forgetting is architecture expansion, which introduces task-specific modules to prevent interference. Yet, existing methods often expand entire layers for each task, leading to significant parameter overhead and poor scalability. To overcome these issues, we introduce LoRA in LoRA (LiLoRA), a highly efficient architecture expansion method tailored for CVIT in MLLMs. LiLoRA shares the LoRA matrix A across tasks to reduce redundancy, applies an additional low-rank decomposition to matrix B to minimize task-specific parameters, and incorporates a cosine-regularized stability loss to preserve consistency in shared representations over time. Extensive experiments on a diverse CVIT benchmark show that LiLoRA consistently achieves superior performance in sequential task learning while significantly improving parameter efficiency compared to existing approaches.
Zitieren
APA
Che, C., Wang, Z., Yang, P., Wang, C., Ma, H., & Shi, Z. (2026). LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(24), 19978-19986. https://doi.org/10.1609/aaai.v40i24.39082
Vancouver
Che C, Wang Z, Yang P, Wang C, Ma H, Shi Z. LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(24):19978-19986. doi:10.1609/aaai.v40i24.39082
BibTeX
@article{Che2026,
author = {Che, Chang and Wang, Ziqi and Yang, Pengwan and Wang, Cheems and Ma, Hui and Shi, Zenglin},
title = {LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {24},
pages = {19978--19986},
doi = {10.1609/aaai.v40i24.39082}
}
EUVIMED