EUVIMEDEUVIMED entdecken
Crossref

Improving Fast Adversarial Training With Prior-Guided Knowledge

Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, Xiaochun Cao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-09 · 46 (9) · 6367-6383

DOI: 10.1109/tpami.2024.3381180 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-09. Verfasst von Xiaojun Jia und 6 weiteren Autor:innen. Die Arbeit wurde bisher 56-mal zitiert (Crossref) und nennt 70 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, Shenzhen Science and Technology Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Fast adversarial training (FAT) is an efficient method to improve robustness in white-box attack scenarios. However, the original FAT suffers from catastrophic overfitting, which dramatically and suddenly reduces robustness after a few training epochs. Although various FAT variants have been proposed to prevent overfitting, they require high training time. In this paper, we investigate the relationship between adversarial example quality and catastrophic overfitting by comparing the training processes of standard adversarial training and FAT. We find that catastrophic overfitting occurs when the attack success rate of adversarial examples becomes worse. Based on this observation, we propose a positive prior-guided adversarial initialization to prevent overfitting by improving adversarial example quality without extra training time. This initialization is generated by using high-quality adversarial perturbations from the historical training process. We provide theoretical analysis for the proposed initialization and propose a prior-guided regularization method that boosts the smoothness of the loss function. Additionally, we design a prior-guided ensemble FAT method that averages the different model weights of historical models using different decay rates. Our proposed method, called FGSM-PGK, assembles the prior-guided knowledge, i.e., the prior-guided initialization and model weights, acquired during the historical training process. The proposed method can effectively improve the model's adversarial robustness in white-box attack scenarios. Evaluations of four datasets demonstrate the superiority of the proposed method.

Förderung

  • National Key R&D Program of China
  • National Natural Science Foundation of China
  • Shenzhen Science and Technology Program
  • Guangdong Major Project of Basic and Applied Basic Research

Zitieren

APA

Jia, X., Zhang, Y., Wei, X., Wu, B., Ma, K., Wang, J., & Cao, X. (2024). Improving Fast Adversarial Training With Prior-Guided Knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 6367-6383. https://doi.org/10.1109/tpami.2024.3381180

Vancouver

Jia X, Zhang Y, Wei X, Wu B, Ma K, Wang J, et al. Improving Fast Adversarial Training With Prior-Guided Knowledge. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(9):6367-6383. doi:10.1109/tpami.2024.3381180

BibTeX
@article{Jia2024,
  author = {Jia, Xiaojun and Zhang, Yong and Wei, Xingxing and Wu, Baoyuan and Ma, Ke and Wang, Jue and Cao, Xiaochun},
  title = {Improving Fast Adversarial Training With Prior-Guided Knowledge},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {9},
  pages = {6367--6383},
  doi = {10.1109/tpami.2024.3381180}
}
Kostenlos starten →