AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models
Electronics · 2026-10-25 · 15 (21) · 4927
DOI: 10.3390/electronics15214927 ↗At a glance
Journal article in Electronics, published online on 2026-10-25. Written by Zhe Jing and 2 co-authors. 38 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Natural Science Foundation of China, Beijing Natural Science Foundation.
Abstract (original language)
Diffusion Direct Preference Optimization (DPO) provides a reward-free route for aligning text-to-image diffusion models with human preferences, but its supervision is commonly restricted to fixed winner–loser pairs. When dispreferred examples cover only a limited subset of failure modes, optimization can over-emphasize local dataset preferences and provide insufficient corrective contrasts. We present AdvDPO, an adversarial candidate-negative preference optimization framework for diffusion models. AdvDPO couples the policy with an auxiliary adversarial diffusion network that produces a local perturbation direction from the preferred diffusion state. The resulting candidate augments the original preference pair while preserving the prompt, timestep, and noise target. The policy learns from both real and synthesized preference pairs, whereas the adversarial network is alternately trained with reversed preference supervision and an attack objective that keeps synthesized candidates challenging for the current policy. On three test sets and two diffusion backbones, AdvDPO achieves the strongest automatic scores among the compared methods and the largest preference share in the anonymous user study.
Funding
- National Natural Science Foundation of China
- Beijing Natural Science Foundation
Cite
APA
Jing, Z., Wang, P., & Huang, H. (2026). AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models. Electronics, 15(21), 4927. https://doi.org/10.3390/electronics15214927
Vancouver
Jing Z, Wang P, Huang H. AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models. Electronics. 2026;15(21):4927. doi:10.3390/electronics15214927
BibTeX
@article{Jing2026,
author = {Jing, Zhe and Wang, Pin and Huang, Huaibo},
title = {AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models},
journal = {Electronics},
year = {2026},
volume = {15},
number = {21},
pages = {4927},
doi = {10.3390/electronics15214927}
}
EUVIMED