EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

From Explicit to Implicit: A Theoretical Framework and Transfer Method for Preference Internalization in Language Models

Binrui Wang, Yongping Du, Yu Pei, Zikai Wang

Transactions of the Association for Computational Linguistics · 01.06.2026 · 14 · 1074-1095

DOI: 10.1162/tacl.a.697 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.06.2026. Verfasst von Binrui Wang und 3 weiteren Autor:innen. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transforming explicit preference signals into implicit and parameterized behaviors is pivotal for enabling prompt-free, human-aligned generation and improving the usability, efficiency and robustness of large language models. However, existing methods align model preference well but still rely on explicit user instructions to convey specific preferences, leading to cumbersome user experiences and undermining natural, frictionless interaction with the model. To fill the gap between explicit and implicit preference representation, this paper introduces a theoretical framework that establishes both necessary and sufficient conditions for effective preference recognition. Based on this framework, we propose a novel Two-Stage Progressive Preference Transfer (TSPPT) method, which decomposes preference internalization into two manageable stages: preference representation learning and preference internalization transfer. The proposed method fills the gap between explicit and implicit preferences while maintaining the model’s general capabilities. The experiments across multiple models (Qwen2.5, Qwen3, Llama-3.2, DeepSeek-R1-Distill) and datasets (UltraFeedback, HelpSteer) demonstrate superior performance. The proposed method achieves 79.2% win rate on UltraFeedback (vs. 59.2–67.6% for baselines), substantial improvements on MT-Bench (7.86 vs. 7.34 for best baseline), and significant reductions in implicit social bias (0.165 vs. 0.185–0.325 for baselines). Notably, the method maintains comparable performance between implicit and explicit settings, confirming successful preference internalization.1

Zitieren

APA

Wang, B., Du, Y., Pei, Y., & Wang, Z. (2026). From Explicit to Implicit: A Theoretical Framework and Transfer Method for Preference Internalization in Language Models. Transactions of the Association for Computational Linguistics, 14, 1074-1095. https://doi.org/10.1162/tacl.a.697

Vancouver

Wang B, Du Y, Pei Y, Wang Z. From Explicit to Implicit: A Theoretical Framework and Transfer Method for Preference Internalization in Language Models. Transactions of the Association for Computational Linguistics. 2026;14:1074-1095. doi:10.1162/tacl.a.697

BibTeX
@article{Wang2026,
  author = {Wang, Binrui and Du, Yongping and Pei, Yu and Wang, Zikai},
  title = {From Explicit to Implicit: A Theoretical Framework and Transfer Method for Preference Internalization in Language Models},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  volume = {14},
  pages = {1074--1095},
  doi = {10.1162/tacl.a.697}
}
Kostenlos starten →