FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks
Machine Learning and Knowledge Extraction · 19.11.2026 · 8 (11) · 389
DOI: 10.3390/make8110389 ↗Kurz gefasst
Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 19.11.2026. Verfasst von Aleksei Samarin und 10 weiteren Autor:innen. Die Arbeit nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Economic Development of the Russian Federation.
Abstract (Originalsprache)
Low-rank adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet uniform rank assignment ignores the varying contribution of layers to a downstream task. Adaptive methods such as AdaLoRA allocate ranks via gradient and singular-value importance, but they do not exploit the representation change induced by the task. We introduce FLowRA, a calibration-based rank-allocation framework that combines a lightweight flow-change descriptor (velocity, acceleration, and directional alignment of hidden representations) with gradient activity to solve a constrained integer-budget problem. FLowRA complements existing LoRA variants, requires only two user-configurable parameters, and supports multiple architectures through architecture-specific implementations. On GLUE, SuperGLUE, instruction-following, code, vision, and vision-language benchmarks, FLowRA consistently outperforms uniform LoRA, DoRA, and AdaLoRA under low trainable-parameter budgets, reducing the gap to full fine-tuning by 48.5% on GLUE relative to the strongest baseline. We compare FLowRA with AdaLoRA using sensitivity-score correlations and report component ablations to assess the contribution of flow-guided allocation.
Förderung
- Ministry of Economic Development of the Russian Federation
Zitieren
APA
Samarin, A., Nazarenko, A., Kotenko, E., Toropov, A., Nazarenko, D., Afonina, J., Dozortseva, A., Savelev, A., Mikhailova, E., Motyko, A., & Malykh, V. (2026). FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks. Machine Learning and Knowledge Extraction, 8(11), 389. https://doi.org/10.3390/make8110389
Vancouver
Samarin A, Nazarenko A, Kotenko E, Toropov A, Nazarenko D, Afonina J, et al. FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks. Machine Learning and Knowledge Extraction. 2026;8(11):389. doi:10.3390/make8110389
BibTeX
@article{Samarin2026,
author = {Samarin, Aleksei and Nazarenko, Artem and Kotenko, Egor and Toropov, Aleksei and Nazarenko, Dmitry and Afonina, Julia and Dozortseva, Aleksandra and Savelev, Alexander and Mikhailova, Elena and Motyko, Alexander and Malykh, Valentin},
title = {FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {11},
pages = {389},
doi = {10.3390/make8110389}
}
EUVIMED