EUVIMEDEUVIMED entdecken
Crossref

Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance

Hongxu Chen, Ke Wei, Luo Luo

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (24) · 20067-20075

DOI: 10.1609/aaai.v40i24.39092 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hongxu Chen und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Decentralized optimization is critical for solving large-scale machine learning problems over distributed networks, where multiple nodes collaborate through local communication. In practice, the variances of stochastic gradient estimators often differ across nodes, yet their impact on algorithm design and complexity remains unclear. To address this issue, we propose D-NSS, a decentralized algorithm with node-specific sampling, and establish its sample complexity depending on the arithmetic mean of local standard deviations, achieving tighter bounds than existing methods that rely on the worst-case or quadratic mean. We further derive a matching sample complexity lower bound under heterogeneous variance, thereby proving the optimality of this dependence. Moreover, we extend the framework with a variance reduction technique and develop D-NSS-VR, which under the mean-squared smoothness assumption attains an improved sample complexity bound while preserving the arithmetic-mean dependence. Finally, numerical experiments validate the theoretical results and demonstrate the effectiveness of the proposed algorithms.

Zitieren

APA

Chen, H., Wei, K., & Luo, L. (2026). Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance. Proceedings of the AAAI Conference on Artificial Intelligence, 40(24), 20067-20075. https://doi.org/10.1609/aaai.v40i24.39092

Vancouver

Chen H, Wei K, Luo L. Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(24):20067-20075. doi:10.1609/aaai.v40i24.39092

BibTeX
@article{Chen2026,
  author = {Chen, Hongxu and Wei, Ke and Luo, Luo},
  title = {Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {24},
  pages = {20067--20075},
  doi = {10.1609/aaai.v40i24.39092}
}
Kostenlos starten →