EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CtrlAdapt: an adaptive multi-control framework for text-to-image diffusion models

Ionuț-Valentin Șerban, Andrei-Ioan Țarcă, Remus-Andrei Bujor, Dragoș-Ștefan Văcărașu, Alexandru Manole

Neural Computing and Applications · 2026-07 · 38 (13)

DOI: 10.1007/s00521-026-12261-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neural Computing and Applications, online erschienen am 26.06.2026. Verfasst von Ionuț-Valentin Șerban und 4 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Romanian Hub for Artificial Intelligence - HRIA.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Diffusion models now deliver state-of-the-art image synthesis, but vanilla Stable Diffusion offers limited spatial control. ControlNet injects structure via single-condition maps, leaving multi-condition training underexplored. In this work, we introduce CtrlAdapt, an adaptive multi-control framework for diffusion models that learns to dynamically fuse multiple control signals during image generation. CtrlAdapt extends ControlNet with a lightweight trainable adaptive fusion module that predicts data-dependent, spatially adaptive fusion weights over learned control features for multiple conditions. The combined conditioning is injected at all feature levels in the decoder, enabling adaptive modulation of conditioning information. Our approach uses semantic segmentation and depth maps and trains the adaptive fusion module jointly with the control branches while keeping the base diffusion model fixed. On the Cityscapes dataset, the proposed adaptive fusion achieves a FID of 65.96, compared to 67.95 for fixed-weight dual control and 69.11–70.79 for single-control variants. On the UrbanSyn dataset, adaptive fusion attains a FID of 70.38, substantially lower than the 90.99–96.09 range observed for single-control models. Across both datasets, the proposed method maintains competitive structural fidelity, with PSNR, SSIM, and LPIPS values comparable to fixed-weight fusion approaches.

Förderung

  • Romanian Hub for Artificial Intelligence - HRIA

Zitieren

APA

Șerban, I., Țarcă, A., Bujor, R., Văcărașu, D., & Manole, A. (2026). CtrlAdapt: an adaptive multi-control framework for text-to-image diffusion models. Neural Computing and Applications, 38(13). https://doi.org/10.1007/s00521-026-12261-2

Vancouver

Șerban I, Țarcă A, Bujor R, Văcărașu D, Manole A. CtrlAdapt: an adaptive multi-control framework for text-to-image diffusion models. Neural Computing and Applications. 2026;38(13). doi:10.1007/s00521-026-12261-2

BibTeX
@article{erban2026,
  author = {Șerban, Ionuț-Valentin and Țarcă, Andrei-Ioan and Bujor, Remus-Andrei and Văcărașu, Dragoș-Ștefan and Manole, Alexandru},
  title = {CtrlAdapt: an adaptive multi-control framework for text-to-image diffusion models},
  journal = {Neural Computing and Applications},
  year = {2026},
  volume = {38},
  number = {13},
  doi = {10.1007/s00521-026-12261-2}
}
Kostenlos starten →