EUVIMEDEUVIMED entdecken
Crossref

SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization

Yue Huang, Xiangqi Wang, Xiangliang Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31220-31228

DOI: 10.1609/aaai.v40i37.40384 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yue Huang und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In high-stakes scenarios—such as self-harm, legal, or medical queries—LLMs must be both trustworthy and helpful. However, these goals often conflict. We propose priority alignment, a new alignment paradigm that enforces a strict “trustworthy-before-helpful” ordering: optimization of helpfulness is conditioned on first meeting trustworthy thresholds (e.g., harmlessness or honesty). To realize this, we introduce Self-Priority Alignment (SPA)—a fully unsupervised framework that generates diverse responses, self-evaluates them and refines them by the model itself, and applies dual-criterion denoising to remove inconsistency and control variance. From this, SPA constructs lexicographically ordered preference pairs and fine-tunes the model using an uncertainty-weighted alignment loss that emphasizes high-confidence, high-gap decisions. Experiments across multiple benchmarks show that SPA improves helpfulness without compromising safety, outperforming strong baselines while preserving general capabilities. Our results demonstrate that SPA provides a scalable and interpretable alignment strategy for critical LLM applications.

Zitieren

APA

Huang, Y., Wang, X., & Zhang, X. (2026). SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31220-31228. https://doi.org/10.1609/aaai.v40i37.40384

Vancouver

Huang Y, Wang X, Zhang X. SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31220-31228. doi:10.1609/aaai.v40i37.40384

BibTeX
@article{Huang2026,
  author = {Huang, Yue and Wang, Xiangqi and Zhang, Xiangliang},
  title = {SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31220--31228},
  doi = {10.1609/aaai.v40i37.40384}
}
Kostenlos starten →