EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Federated Spectral Regularization for Convergence Acceleration: A Random Matrix Theory Perspective

Shengyu Cai, Jianchao Bai

Mathematics · 05.08.2026 · 14 (15) · 2819

DOI: 10.3390/math14152819 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 05.08.2026. Verfasst von Shengyu Cai und 1 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Federated learning enables privacy-preserving distributed training but suffers from client drift and slow convergence under statistical data heterogeneity. Most existing federated optimization methods address client drift via parameter-space constraints or aggregation-level corrections, while fewer works directly shape the gradient covariance spectral structure of the optimization landscape. This paper analyzes the convergence problem from a spectral perspective, revealing that non-IID data causes spectral diffusion in the gradient covariance matrix and degrades convergence. Guided by random matrix theory, we propose federated spectral regularization (Fed-SR), a computationally efficient method that indirectly constrains spectral spread via gradient norm regularization. Although computing the regularizer gradient requires Hessian vector products, our optimized auto-differentiation implementation avoids storing full Hessian matrices and restricts extra computational overhead to a negligible level. Experiments on CIFAR-10, CIFAR-100, and other benchmarks show that Fed-SR outperforms baselines including FedAvg, FedProx, and SCAFFOLD in non-IID scenarios, reducing communication rounds and improving accuracy and stability. Ablation studies, spectral analysis, and controlled spectral feature manipulation experiments provide consistent empirical evidence showing a strong empirical association between the “spectral concentration” effect and performance gains, offering mechanistic interpretability consistent with our proposed theoretical framework within the tested experimental settings.

Zitieren

APA

Cai, S., & Bai, J. (2026). Federated Spectral Regularization for Convergence Acceleration: A Random Matrix Theory Perspective. Mathematics, 14(15), 2819. https://doi.org/10.3390/math14152819

Vancouver

Cai S, Bai J. Federated Spectral Regularization for Convergence Acceleration: A Random Matrix Theory Perspective. Mathematics. 2026;14(15):2819. doi:10.3390/math14152819

BibTeX
@article{Cai2026,
  author = {Cai, Shengyu and Bai, Jianchao},
  title = {Federated Spectral Regularization for Convergence Acceleration: A Random Matrix Theory Perspective},
  journal = {Mathematics},
  year = {2026},
  volume = {14},
  number = {15},
  pages = {2819},
  doi = {10.3390/math14152819}
}
Kostenlos starten →