EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Mitigating Catastrophic Forgetting in Fine-Tuned Large Language Models: An Experimental Study of LoRA and O-LoRA

Xinlan Zhang

Artificial Intelligence and Digital Technology · 2026

DOI: 10.70088/aa3p5364 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Artificial Intelligence and Digital Technology, 2026. Verfasst von Xinlan Zhang. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have become a hot topic in AI, and since the GPT series they have achieved remarkable success across many domains. However, directly using a general-purpose model often fails to meet the needs of specific applications, which motivates fine-tuning with domain-specific data. Nevertheless, parameter-efficient fine-tuning (PEFT) methods such as LoRA may perform poorly on certain algorithmic benchmarks, raising concerns about cat-astrophic forgetting. In this paper, we conduct extensive experiments to confirm this phenomenon and investigate O-LoRA as a mitigation strategy. Results show that O-LoRA can effectively alleviate catastrophic forgetting under continual instruction fine-tuning, but its effectiveness can be sensitive to hyperparameters on some datasets. Overall, O-LoRA provides a practical direction for mitigating catastrophic forgetting during continual fine-tuning of LLMs.

Zitieren

APA

Zhang, X. (2026). Mitigating Catastrophic Forgetting in Fine-Tuned Large Language Models: An Experimental Study of LoRA and O-LoRA. Artificial Intelligence and Digital Technology. https://doi.org/10.70088/aa3p5364

Vancouver

Zhang X. Mitigating Catastrophic Forgetting in Fine-Tuned Large Language Models: An Experimental Study of LoRA and O-LoRA. Artificial Intelligence and Digital Technology. 2026. doi:10.70088/aa3p5364

BibTeX
@article{Zhang2026,
  author = {Zhang, Xinlan},
  title = {Mitigating Catastrophic Forgetting in Fine-Tuned Large Language Models: An Experimental Study of LoRA and O-LoRA},
  journal = {Artificial Intelligence and Digital Technology},
  year = {2026},
  doi = {10.70088/aa3p5364}
}
Kostenlos starten →