EUVIMEDEUVIMED entdecken
Crossref

Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation

Hefei Xu, Le Wu, Chen Cheng, Hao Liu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34133-34141

DOI: 10.1609/aaai.v40i40.40708 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hefei Xu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the rapid advancement of large language models (LLMs), aligning them with human values for safety and ethics has become a critical challenge. This problem is especially challenging when multiple, potentially conflicting human values must be considered and balanced. Although several variants of existing alignment methods (such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO)) have been proposed to address multi-value alignment, they suffer from notable limitations: 1) they are often unstable and inefficient in multi-value optimization; and 2) they fail to effectively handle value conflicts. As a result, these approaches typically struggle to achieve optimal trade-offs when aligning multiple values. To address this challenge, we propose a novel framework called Multi-Value Alignment (MVA). It mitigates alignment degradation caused by parameter interference among diverse human values by minimizing their mutual information. Furthermore, we propose a value extrapolation strategy to efficiently explore the Pareto frontier, thereby constructing a set of LLMs with diverse value preferences. Extensive experiments demonstrate that MVA consistently outperforms existing baselines in aligning LLMs with multiple human values.

Zitieren

APA

Xu, H., Wu, L., Cheng, C., & Liu, H. (2026). Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34133-34141. https://doi.org/10.1609/aaai.v40i40.40708

Vancouver

Xu H, Wu L, Cheng C, Liu H. Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34133-34141. doi:10.1609/aaai.v40i40.40708

BibTeX
@article{Xu2026,
  author = {Xu, Hefei and Wu, Le and Cheng, Chen and Liu, Hao},
  title = {Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {34133--34141},
  doi = {10.1609/aaai.v40i40.40708}
}
Kostenlos starten →