EUVIMEDScopri EUVIMED
Open access · CC BY 4.0

AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models

Zhe Jing, Pin Wang, Huaibo Huang

Electronics · 2026-10-25 · 15 (21) · 4927

DOI: 10.3390/electronics15214927 ↗

At a glance

Journal article in Electronics, published online on 2026-10-25. Written by Zhe Jing and 2 co-authors. 38 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Natural Science Foundation of China, Beijing Natural Science Foundation.

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Diffusion Direct Preference Optimization (DPO) provides a reward-free route for aligning text-to-image diffusion models with human preferences, but its supervision is commonly restricted to fixed winner–loser pairs. When dispreferred examples cover only a limited subset of failure modes, optimization can over-emphasize local dataset preferences and provide insufficient corrective contrasts. We present AdvDPO, an adversarial candidate-negative preference optimization framework for diffusion models. AdvDPO couples the policy with an auxiliary adversarial diffusion network that produces a local perturbation direction from the preferred diffusion state. The resulting candidate augments the original preference pair while preserving the prompt, timestep, and noise target. The policy learns from both real and synthesized preference pairs, whereas the adversarial network is alternately trained with reversed preference supervision and an attack objective that keeps synthesized candidates challenging for the current policy. On three test sets and two diffusion backbones, AdvDPO achieves the strongest automatic scores among the compared methods and the largest preference share in the anonymous user study.

Funding

  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation

Cite

APA

Jing, Z., Wang, P., & Huang, H. (2026). AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models. Electronics, 15(21), 4927. https://doi.org/10.3390/electronics15214927

Vancouver

Jing Z, Wang P, Huang H. AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models. Electronics. 2026;15(21):4927. doi:10.3390/electronics15214927

BibTeX
@article{Jing2026,
  author = {Jing, Zhe and Wang, Pin and Huang, Huaibo},
  title = {AdvDPO: Adversarial Negative-Augmented Preference Optimization for Text-to-Image Diffusion Models},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {4927},
  doi = {10.3390/electronics15214927}
}
Start for free →