Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (38) · 32537-32545
DOI: 10.1609/aaai.v40i38.40530 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yunhun Nam und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Language models (LMs) are often adapted through supervised fine-tuning (SFT) to specialize their capabilities for downstream tasks. However, in typical scenarios where the fine-tuning data is limited, e.g., compared to pre-training, SFT can lead LMs to overfit, causing them to rely on spurious patterns within the target task or to compromise other broadly useful capabilities as a side effect of narrow specialization. In this paper, we propose Learning-from-the-Undesirable (LfU), a simple yet effective regularization scheme for SFT to mitigate overfitting issues when fine-tuning LMs with limited data. Specifically, we aim to regularize the fine-tuning process to favor solutions that are resilient to “undesirable” model updates, e.g., gradient ascent steps that steer the model toward undesirable behaviors. To this end, we propose a novel form of consistency regularization that directly aligns internal representations of the model with those after an undesirable update. By leveraging representation-level data augmentation through undesirable updates, LfU effectively promotes generalization under limited data. Our experiments on diverse LM downstream tasks show that LfU serves as an effective prior that enhances adaptability while preserving pretrained knowledge. For example, our LM from LfU achieves a 16.8% average improvement on math tasks compared to vanilla SFT on the same dataset, where the latter even leads to degraded performance on those tasks. Furthermore, LfU exhibits improved robustness to prompt variations, e.g., yielding a 92.1% lower standard deviation in output performances compared to SFT, highlighting its versatile effects.
Zitieren
APA
Nam, Y., Kim, J., & Jeong, J. (2026). Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting. Proceedings of the AAAI Conference on Artificial Intelligence, 40(38), 32537-32545. https://doi.org/10.1609/aaai.v40i38.40530
Vancouver
Nam Y, Kim J, Jeong J. Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(38):32537-32545. doi:10.1609/aaai.v40i38.40530
BibTeX
@article{Nam2026,
author = {Nam, Yunhun and Kim, Jaehyung and Jeong, Jongheon},
title = {Learning from the Undesirable: Robust Adaptation of Language Models Without Forgetting},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {38},
pages = {32537--32545},
doi = {10.1609/aaai.v40i38.40530}
}
EUVIMED