EUVIMEDEUVIMED entdecken
Crossref

Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting

Yunhun Nam, Jaehyung Kim, Jongheon Jeong

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (38) · 32537-32545

DOI: 10.1609/aaai.v40i38.40530 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yunhun Nam und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Language models (LMs) are often adapted through supervised fine-tuning (SFT) to specialize their capabilities for downstream tasks. However, in typical scenarios where the fine-tuning data is limited, e.g., compared to pre-training, SFT can lead LMs to overfit, causing them to rely on spurious patterns within the target task or to compromise other broadly useful capabilities as a side effect of narrow specialization. In this paper, we propose Learning-from-the-Undesirable (LfU), a simple yet effective regularization scheme for SFT to mitigate overfitting issues when fine-tuning LMs with limited data. Specifically, we aim to regularize the fine-tuning process to favor solutions that are resilient to “undesirable” model updates, e.g., gradient ascent steps that steer the model toward undesirable behaviors. To this end, we propose a novel form of consistency regularization that directly aligns internal representations of the model with those after an undesirable update. By leveraging representation-level data augmentation through undesirable updates, LfU effectively promotes generalization under limited data. Our experiments on diverse LM downstream tasks show that LfU serves as an effective prior that enhances adaptability while preserving pretrained knowledge. For example, our LM from LfU achieves a 16.8% average improvement on math tasks compared to vanilla SFT on the same dataset, where the latter even leads to degraded performance on those tasks. Furthermore, LfU exhibits improved robustness to prompt variations, e.g., yielding a 92.1% lower standard deviation in output performances compared to SFT, highlighting its versatile effects.

Zitieren

APA

Nam, Y., Kim, J., & Jeong, J. (2026). Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting. Proceedings of the AAAI Conference on Artificial Intelligence, 40(38), 32537-32545. https://doi.org/10.1609/aaai.v40i38.40530

Vancouver

Nam Y, Kim J, Jeong J. Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(38):32537-32545. doi:10.1609/aaai.v40i38.40530

BibTeX
@article{Nam2026,
  author = {Nam, Yunhun and Kim, Jaehyung and Jeong, Jongheon},
  title = {Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {38},
  pages = {32537--32545},
  doi = {10.1609/aaai.v40i38.40530}
}
Kostenlos starten →