EUVIMEDOpdag EUVIMED
Crossref

Towards Lifelong Learning of Large Language Models: A Survey

Junhao Zheng, Shengjie Qiu, Chengming Shi, Qianli Ma

ACM Computing Surveys · 2025-08-31 · 57 (8) · 1-35

DOI: 10.1145/3716629 ↗

At a glance

Journal article in ACM Computing Surveys, published online on 2025-03-07. Written by Junhao Zheng and 3 co-authors. Cited 52 times so far (Crossref); 270 references. Funded by: National Natural Science Foundation of China, Natural Science Foundation of Guangdong Province, Science and Technology Planning Project of Guangdong Province.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

As the applications of large language models (LLMs) expand across diverse fields, their ability to adapt to ongoing changes in data, tasks, and user preferences becomes crucial. Traditional training methods with static datasets are inadequate for coping with the dynamic nature of real-world information. Lifelong learning, or continual learning, addresses this by enabling LLMs to learn continuously and adapt over their operational lifetime, integrating new knowledge while retaining previously learned information and preventing catastrophic forgetting. Our survey explores the landscape of lifelong learning, categorizing strategies into two groups based on how new knowledge is integrated: Internal Knowledge, where LLMs absorb new knowledge into their parameters through full or partial training, and External Knowledge, which incorporates new knowledge as external resources such as Wikipedia or APIs without updating model parameters. The key contributions of our survey include: (1) introducing a novel taxonomy to categorize the extensive literature of lifelong learning into 12 scenarios; (2) identifying common techniques across all lifelong learning scenarios and classifying existing literature into various technique groups; (3) highlighting emerging techniques such as model expansion and data selection, which were less explored in the pre-LLM era. Resources are available at https://github.com/qianlima-lab/awesome-lifelong-learning-methods-for-llm .

Funding

  • National Natural Science Foundation of China
  • Natural Science Foundation of Guangdong Province
  • Science and Technology Planning Project of Guangdong Province

Cite

APA

Zheng, J., Qiu, S., Shi, C., & Ma, Q. (2025). Towards Lifelong Learning of Large Language Models: A Survey. ACM Computing Surveys, 57(8), 1-35. https://doi.org/10.1145/3716629

Vancouver

Zheng J, Qiu S, Shi C, Ma Q. Towards Lifelong Learning of Large Language Models: A Survey. ACM Computing Surveys. 2025;57(8):1-35. doi:10.1145/3716629

BibTeX
@article{Zheng2025,
  author = {Zheng, Junhao and Qiu, Shengjie and Shi, Chengming and Ma, Qianli},
  title = {Towards Lifelong Learning of Large Language Models: A Survey},
  journal = {ACM Computing Surveys},
  year = {2025},
  volume = {57},
  number = {8},
  pages = {1--35},
  doi = {10.1145/3716629}
}
Start for free →