Rethinking Direct Preference Optimization in Diffusion Models
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (7) · 5611-5619
DOI: 10.1609/aaai.v40i7.37480 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Junyong Kang und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Aligning text-to-image (T2I) diffusion models with human preferences has emerged as a critical research challenge. While Direct Preference Optimization (DPO) has established a foundation for preference learning in large language models (LLMs), its extension to diffusion models remains limited in alignment performance. In this work, we propose an enhanced version of Diffusion-DPO by introducing a stable reference model update strategy. This strategy facilitates the exploration of better alignment solutions while maintaining training stability. Moreover, we design a timestep-aware optimization strategy that further boosts performance by addressing preference learning imbalance across timesteps. Through the synergistic combination of our exploration and timestep-aware optimization, our method significantly improves the alignment performance of Diffusion-DPO on human preference evaluation benchmarks, achieving state-of-the-art results.
Zitieren
APA
Kang, J., Lim, S., Baek, K., & Shim, H. (2026). Rethinking Direct Preference Optimization in Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(7), 5611-5619. https://doi.org/10.1609/aaai.v40i7.37480
Vancouver
Kang J, Lim S, Baek K, Shim H. Rethinking Direct Preference Optimization in Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(7):5611-5619. doi:10.1609/aaai.v40i7.37480
BibTeX
@article{Kang2026,
author = {Kang, Junyong and Lim, Seohyun and Baek, Kyungjune and Shim, Hyunjung},
title = {Rethinking Direct Preference Optimization in Diffusion Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {7},
pages = {5611--5619},
doi = {10.1609/aaai.v40i7.37480}
}
EUVIMED