Fed-DiffLoRA: Personalized Federated Style Transfer for T2I Diffusion Models
IEEE Transactions on Image Processing · 2026 · 35 · 8034-8047
DOI: 10.1109/tip.2026.3709450 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Fan Qi und 3 weiteren Autor:innen. Die Arbeit nennt 51 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Low-Rank Adaptation (LoRA) merging enables efficient customization of T2I diffusion models; nevertheless, centralized aggregation raises serious privacy concerns. While federated LoRA adaptations mitigate these risks, diffusion models present unique challenges: structural heterogeneity and vulnerability to member inference attacks. To overcome these limitations, we propose Fed-DiffLoRA, a privacy-preserving framework that securely aggregates LoRA adapters. At the client level, we disentangle user-specific adaptations into two orthogonal subspaces: content LoRAs preserving semantic fidelity and style LoRAs encoding stylistic features, isolating sensitive attributes from stylistic components. We design a learnable aggregation operator that dynamically optimizes cross-client style LoRA fusion based on the semantic vectors of clients' content LoRAs, achieving high-fidelity style blending while suppressing client-identifiable patterns. We also provide theoretical guarantees for convergence and privacy. Extensive experiments validate the effectiveness of the proposed approach, achieving substantial reductions in attack success rates and consistently high stylization fidelity.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Qi, F., Cheng, Y., Yang, X., & Xu, C. (2026). Fed-DiffLoRA: Personalized Federated Style Transfer for T2I Diffusion Models. IEEE Transactions on Image Processing, 35, 8034-8047. https://doi.org/10.1109/tip.2026.3709450
Vancouver
Qi F, Cheng Y, Yang X, Xu C. Fed-DiffLoRA: Personalized Federated Style Transfer for T2I Diffusion Models. IEEE Transactions on Image Processing. 2026;35:8034-8047. doi:10.1109/tip.2026.3709450
BibTeX
@article{Qi2026,
author = {Qi, Fan and Cheng, Yue and Yang, Xiaoshan and Xu, Changsheng},
title = {Fed-DiffLoRA: Personalized Federated Style Transfer for T2I Diffusion Models},
journal = {IEEE Transactions on Image Processing},
year = {2026},
volume = {35},
pages = {8034--8047},
doi = {10.1109/tip.2026.3709450}
}
EUVIMED