EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks

Aleksei Samarin, Artem Nazarenko, Egor Kotenko, Aleksei Toropov, Dmitry Nazarenko, Julia Afonina, Aleksandra Dozortseva, Alexander Savelev, Elena Mikhailova, Alexander Motyko, Valentin Malykh

Machine Learning and Knowledge Extraction · 2026-11-19 · 8 (11) · 389

DOI: 10.3390/make8110389 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-11-19. Written by Aleksei Samarin and 10 co-authors. 72 references. The full text is freely available (open access, CC BY 4.0). Funded by: Ministry of Economic Development of the Russian Federation.

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Low-rank adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet uniform rank assignment ignores the varying contribution of layers to a downstream task. Adaptive methods such as AdaLoRA allocate ranks via gradient and singular-value importance, but they do not exploit the representation change induced by the task. We introduce FLowRA, a calibration-based rank-allocation framework that combines a lightweight flow-change descriptor (velocity, acceleration, and directional alignment of hidden representations) with gradient activity to solve a constrained integer-budget problem. FLowRA complements existing LoRA variants, requires only two user-configurable parameters, and supports multiple architectures through architecture-specific implementations. On GLUE, SuperGLUE, instruction-following, code, vision, and vision-language benchmarks, FLowRA consistently outperforms uniform LoRA, DoRA, and AdaLoRA under low trainable-parameter budgets, reducing the gap to full fine-tuning by 48.5% on GLUE relative to the strongest baseline. We compare FLowRA with AdaLoRA using sensitivity-score correlations and report component ablations to assess the contribution of flow-guided allocation.

Funding

  • Ministry of Economic Development of the Russian Federation

Cite

APA

Samarin, A., Nazarenko, A., Kotenko, E., Toropov, A., Nazarenko, D., Afonina, J., Dozortseva, A., Savelev, A., Mikhailova, E., Motyko, A., & Malykh, V. (2026). FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks. Machine Learning and Knowledge Extraction, 8(11), 389. https://doi.org/10.3390/make8110389

Vancouver

Samarin A, Nazarenko A, Kotenko E, Toropov A, Nazarenko D, Afonina J, et al. FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks. Machine Learning and Knowledge Extraction. 2026;8(11):389. doi:10.3390/make8110389

BibTeX
@article{Samarin2026,
  author = {Samarin, Aleksei and Nazarenko, Artem and Kotenko, Egor and Toropov, Aleksei and Nazarenko, Dmitry and Afonina, Julia and Dozortseva, Aleksandra and Savelev, Alexander and Mikhailova, Elena and Motyko, Alexander and Malykh, Valentin},
  title = {FLowRA: Flow-Guided Rank Allocation for Parameter-Efficient Fine-Tuning of Deep Neural Networks},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {11},
  pages = {389},
  doi = {10.3390/make8110389}
}
Start for free →