SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31220-31228
DOI: 10.1609/aaai.v40i37.40384 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yue Huang und 2 weiteren Autor:innen.
Abstract (Originalsprache)
In high-stakes scenarios—such as self-harm, legal, or medical queries—LLMs must be both trustworthy and helpful. However, these goals often conflict. We propose priority alignment, a new alignment paradigm that enforces a strict “trustworthy-before-helpful” ordering: optimization of helpfulness is conditioned on first meeting trustworthy thresholds (e.g., harmlessness or honesty). To realize this, we introduce Self-Priority Alignment (SPA)—a fully unsupervised framework that generates diverse responses, self-evaluates them and refines them by the model itself, and applies dual-criterion denoising to remove inconsistency and control variance. From this, SPA constructs lexicographically ordered preference pairs and fine-tunes the model using an uncertainty-weighted alignment loss that emphasizes high-confidence, high-gap decisions. Experiments across multiple benchmarks show that SPA improves helpfulness without compromising safety, outperforming strong baselines while preserving general capabilities. Our results demonstrate that SPA provides a scalable and interpretable alignment strategy for critical LLM applications.
Zitieren
APA
Huang, Y., Wang, X., & Zhang, X. (2026). SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31220-31228. https://doi.org/10.1609/aaai.v40i37.40384
Vancouver
Huang Y, Wang X, Zhang X. SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31220-31228. doi:10.1609/aaai.v40i37.40384
BibTeX
@article{Huang2026,
author = {Huang, Yue and Wang, Xiangqi and Zhang, Xiangliang},
title = {SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {37},
pages = {31220--31228},
doi = {10.1609/aaai.v40i37.40384}
}
EUVIMED