A Survey on Memory-Efficient Fine-Tuning for Large Language Models
Transactions of the Association for Computational Linguistics · 01.06.2026 · 14 · 960-982
DOI: 10.1162/tacl.a.692 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.06.2026. Verfasst von Yeachan Kim und 2 weiteren Autor:innen. Die Arbeit nennt 152 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Fine-tuning large language models (LLMs) is a crucial process to align them with human intentions, yet this process remains memoryintensive, varying across tasks and model architectures. These huge and variable memory costs complicate scaling and deployment of LLMs, especially on limited hardware. However, existing surveys on memory efficiency are often either superficial or too narrow in scope, typically focusing on specific subfields. To address this gap, this survey presents the first systematic review of memory-efficient fine-tuning (MEFT) tailored for LLMs. To structure the research landscape, we first categorize existing approaches by their optimization environments (i.e., model itself and systems) and further classify model-based approaches by their specific optimization targets. We also discuss evaluation strategies for assessing MEFT methods and provide empirical analyses. By highlighting challenges and future directions based on current methods, this survey aims to serve as a practical guide for developing MEFT methods.
Zitieren
APA
Kim, Y., Lee, M., & Lee, S. (2026). A Survey on Memory-Efficient Fine-Tuning for Large Language Models. Transactions of the Association for Computational Linguistics, 14, 960-982. https://doi.org/10.1162/tacl.a.692
Vancouver
Kim Y, Lee M, Lee S. A Survey on Memory-Efficient Fine-Tuning for Large Language Models. Transactions of the Association for Computational Linguistics. 2026;14:960-982. doi:10.1162/tacl.a.692
BibTeX
@article{Kim2026,
author = {Kim, Yeachan and Lee, Mingyu and Lee, SangKeun},
title = {A Survey on Memory-Efficient Fine-Tuning for Large Language Models},
journal = {Transactions of the Association for Computational Linguistics},
year = {2026},
volume = {14},
pages = {960--982},
doi = {10.1162/tacl.a.692}
}
EUVIMED