EUVIMEDEUVIMED entdecken
Crossref

Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNs

Zhongjie Shi, Puyu Wang, Chenyang Zhang, Yuan Cao

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25374-25382

DOI: 10.1609/aaai.v40i30.39731 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhongjie Shi und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Modern deep learning techniques focus on extracting intricate information from data to achieve accurate predictions. However, the training datasets may be crowdsourced and include sensitive information, such as personal contact details, financial data, and medical records. As a result, there is a growing emphasis on developing privacy-preserving training algorithms for neural networks that maintain good performance while preserving privacy. In this paper, we investigate the generalization and privacy performances of the differentially private gradient descent (DP-GD) algorithm, which is a private variant of the gradient descent (GD) by incorporating additional noise into the gradients during each iteration. Moreover, we identify a concrete learning task where DP-GD can achieve superior generalization performance compared to GD in training two-layer Huberized ReLU convolutional neural networks (CNNs). Specifically, we demonstrate that, under mild conditions, a small signal-to-noise ratio can result in GD producing training models with poor test accuracy, whereas DP-GD can yield training models with good test accuracy and privacy guarantees if the signal-to-noise ratio is not too small. This indicates that DP-GD has the potential to enhance model performance while ensuring privacy protection in certain learning tasks. Numerical simulations are further conducted to support our theoretical results.

Zitieren

APA

Shi, Z., Wang, P., Zhang, C., & Cao, Y. (2026). Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNs. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25374-25382. https://doi.org/10.1609/aaai.v40i30.39731

Vancouver

Shi Z, Wang P, Zhang C, Cao Y. Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNs. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25374-25382. doi:10.1609/aaai.v40i30.39731

BibTeX
@article{Shi2026,
  author = {Shi, Zhongjie and Wang, Puyu and Zhang, Chenyang and Cao, Yuan},
  title = {Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNs},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25374--25382},
  doi = {10.1609/aaai.v40i30.39731}
}
Kostenlos starten →