EUVIMEDEUVIMED entdecken
Crossref

Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation

Shiji Zhao, Xizhe Wang, Xingxing Wei

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9338-9352

DOI: 10.1109/tpami.2024.3416308 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Shiji Zhao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Adversarial Training is a practical approach for improving the robustness of deep neural networks against adversarial attacks. Although bringing reliable robustness, the performance towards clean examples is negatively affected after Adversarial Training, which means a trade-off exists between accuracy and robustness. Recently, some studies have tried to use knowledge distillation methods in Adversarial Training, achieving competitive performance in improving the robustness but the accuracy for clean samples is still limited. In this paper, to mitigate the accuracy-robustness trade-off, we introduce the Balanced Multi-Teacher Adversarial Robustness Distillation (B-MTARD) to guide the model's Adversarial Training process by applying a strong clean teacher and a strong robust teacher to handle the clean examples and adversarial examples, respectively. During the optimization process, to ensure that different teachers show similar knowledge scales, we design the Entropy-Based Balance algorithm to adjust the teacher's temperature and keep the teachers' information entropy consistent. Besides, to ensure that the student has a relatively consistent learning speed from multiple teachers, we propose the Normalization Loss Balance algorithm to adjust the learning weights of different types of knowledge. A series of experiments conducted on three public datasets demonstrate that B-MTARD outperforms the state-of-the-art methods against various adversarial attacks.

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Zhao, S., Wang, X., & Wei, X. (2024). Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9338-9352. https://doi.org/10.1109/tpami.2024.3416308

Vancouver

Zhao S, Wang X, Wei X. Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9338-9352. doi:10.1109/tpami.2024.3416308

BibTeX
@article{Zhao2024,
  author = {Zhao, Shiji and Wang, Xizhe and Wei, Xingxing},
  title = {Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {12},
  pages = {9338--9352},
  doi = {10.1109/tpami.2024.3416308}
}
Kostenlos starten →