Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34133-34141
DOI: 10.1609/aaai.v40i40.40708 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hefei Xu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).
Abstract (Originalsprache)
With the rapid advancement of large language models (LLMs), aligning them with human values for safety and ethics has become a critical challenge. This problem is especially challenging when multiple, potentially conflicting human values must be considered and balanced. Although several variants of existing alignment methods (such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO)) have been proposed to address multi-value alignment, they suffer from notable limitations: 1) they are often unstable and inefficient in multi-value optimization; and 2) they fail to effectively handle value conflicts. As a result, these approaches typically struggle to achieve optimal trade-offs when aligning multiple values. To address this challenge, we propose a novel framework called Multi-Value Alignment (MVA). It mitigates alignment degradation caused by parameter interference among diverse human values by minimizing their mutual information. Furthermore, we propose a value extrapolation strategy to efficiently explore the Pareto frontier, thereby constructing a set of LLMs with diverse value preferences. Extensive experiments demonstrate that MVA consistently outperforms existing baselines in aligning LLMs with multiple human values.
Zitieren
APA
Xu, H., Wu, L., Cheng, C., & Liu, H. (2026). Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34133-34141. https://doi.org/10.1609/aaai.v40i40.40708
Vancouver
Xu H, Wu L, Cheng C, Liu H. Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34133-34141. doi:10.1609/aaai.v40i40.40708
BibTeX
@article{Xu2026,
author = {Xu, Hefei and Wu, Le and Cheng, Chen and Liu, Hao},
title = {Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {40},
pages = {34133--34141},
doi = {10.1609/aaai.v40i40.40708}
}
EUVIMED