EUVIMEDEUVIMED entdecken
Crossref

Personalized Federated Direct Preference Optimization via LoRA Adapters

Zhanhong Jiang, Soumik Sarkar

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 1040-1044

DOI: 10.1609/aaaiss.v10i1.43524 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Zhanhong Jiang und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated Direct Preference Optimization (FedDPO) enables privacy-preserving alignment of language models across distributed agents without centralizing sensitive preference data. However, existing FedDPO analysis assumes all clients converge to a single shared policy, which is fundamentally at odds with the heterogeneous preference distributions present in practice. We propose Personalized FedDPO (pFedDPO), in which each client maintains a small private LoRA adapter atop a shared global base model. In pFedDPO, the base model is synchronized via FedAvg each round, while LoRA adapters remain local and capture client-specific preference patterns. We prove that this decomposition replaces the gradient diversity in the FedDPO bound with a residual heterogeneity, which is no larger than the former when adapters absorb client-specific directions, at the cost of an additive adapter-bias term that vanishes as adapters converge. Experiments on preference datasets show that pFedDPO consistently attains a lower gradient norm than vanilla FedDPO while remaining competitive in training loss and per-client reward accuracy.

Zitieren

APA

Jiang, Z., & Sarkar, S. (2026). Personalized Federated Direct Preference Optimization via LoRA Adapters. Proceedings of the AAAI Symposium Series, 10(1), 1040-1044. https://doi.org/10.1609/aaaiss.v10i1.43524

Vancouver

Jiang Z, Sarkar S. Personalized Federated Direct Preference Optimization via LoRA Adapters. Proceedings of the AAAI Symposium Series. 2026;10(1):1040-1044. doi:10.1609/aaaiss.v10i1.43524

BibTeX
@article{Jiang2026,
  author = {Jiang, Zhanhong and Sarkar, Soumik},
  title = {Personalized Federated Direct Preference Optimization via LoRA Adapters},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {1040--1044},
  doi = {10.1609/aaaiss.v10i1.43524}
}
Kostenlos starten →