Diffusion Preference Alignment via Attenuated Kullback–Leibler Regularization
Electronics · 23.07.2025 · 14 (15) · 2939
DOI: 10.3390/electronics14152939 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 23.07.2025. Verfasst von Xinjian Zhang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Direct preference optimization (DPO) has been successfully applied to align large language models (LLMs) with human preferences. In recent years, DPO has also been used to improve the generation quality of text-to-image diffusion models. However, existing techniques often rely on a single type of reward model. They are also prone to overfitting to inaccurate reward signals. As a result, model quality cannot be continuously improved. To address these limitations, we propose xDPO. This method introduces a novel regularization approach that implicitly defines reward functions for both preferred and non-preferred samples. This design greatly enhances the flexibility of reward modeling. The experimental results show that, after fine-tuning Stable Diffusion v1.5, xDPO achieves significant improvements in human preference evaluations compared to previous DPO methods. It also improves training efficiency by approximately 1.5 times. Meanwhile, xDPO maintains image–text alignment performance that is comparable to the original model.
Zitieren
APA
Zhang, X., & Xiang, W. (2025). Diffusion Preference Alignment via Attenuated Kullback–Leibler Regularization. Electronics, 14(15), 2939. https://doi.org/10.3390/electronics14152939
Vancouver
Zhang X, Xiang W. Diffusion Preference Alignment via Attenuated Kullback–Leibler Regularization. Electronics. 2025;14(15):2939. doi:10.3390/electronics14152939
BibTeX
@article{Zhang2025,
author = {Zhang, Xinjian and Xiang, Wei},
title = {Diffusion Preference Alignment via Attenuated Kullback–Leibler Regularization},
journal = {Electronics},
year = {2025},
volume = {14},
number = {15},
pages = {2939},
doi = {10.3390/electronics14152939}
}
EUVIMED