Zero-Shot Evolutionary Architecture Search for Low-Rank Adaptation
International Journal of Neural Systems · 2026-07 · 36 (07)
DOI: 10.1142/s0129065726500255 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Neural Systems, online erschienen am 19.03.2026. Verfasst von Pengjin Wu und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Electronic Engineering, University of Surrey, Guildford, UK. Hauptthemen laut MEDLINE: Neural Networks, Computer. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: China Scholarship Council.
Abstract (Originalsprache)
Fine-tuning large-scale Transformer-based models is computationally expensive due to the enormous parameter space. Low-Rank Adaptation (LoRA) substantially reduces the number of trainable parameters while maintaining performance; however, identifying the optimal LoRA configuration — such as rank r, scaling factor [Formula: see text], and insertion positions — remains challenging. To address this issue, we propose a zero-shot proxy metric, termed Gradient Projection Score (GPS), which enables rapid evaluation of candidate configurations using only a few forward and backward passes. Building upon this metric, we further introduce EvoLoRA, a zero-shot evolutionary architecture search method that jointly optimizes three objectives: performance proxy, evaluation stability, and trainable parameter size. EvoLoRA automatically discovers effective LoRA configurations across different models and datasets. Experimental results demonstrate that GPS is strongly correlated with final model performance; moreover, on tasks such as image classification and object detection, EvoLoRA markedly reduces search and training costs while generally outperforming other fine-tuning methods and manually designed LoRA configurations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Electronic Engineering, University of Surrey, Guildford, UK.
- School of Software, Nanjing University of Information Science and Technology, Nanjing, P. R. China.
- School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, P. R. China.
ORCID-Profile: Pengjin Wu, Ferrante Neri, Zhenhua Feng
Förderung
- China Scholarship Council
Zitieren
APA
Wu, P., Neri, F., & Feng, Z. (2026). Zero-Shot Evolutionary Architecture Search for Low-Rank Adaptation. International Journal of Neural Systems, 36(07). https://doi.org/10.1142/s0129065726500255
Vancouver
Wu P, Neri F, Feng Z. Zero-Shot Evolutionary Architecture Search for Low-Rank Adaptation. International Journal of Neural Systems. 2026;36(07). doi:10.1142/s0129065726500255
BibTeX
@article{Wu2026,
author = {Wu, Pengjin and Neri, Ferrante and Feng, Zhenhua},
title = {Zero-Shot Evolutionary Architecture Search for Low-Rank Adaptation},
journal = {International Journal of Neural Systems},
year = {2026},
volume = {36},
number = {07},
doi = {10.1142/s0129065726500255}
}
EUVIMED