EUVIMEDEUVIMED entdecken
Crossref

Metrics and methods for robustness evaluation of neural networks with generative models

Igor Buzhinsky, Arseny Nerinovsky, Stavros Tripakis

Machine Learning · 2023-10 · 112 (10) · 3977-4012

DOI: 10.1007/s10994-021-05994-9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning, online erschienen am 06.07.2021. Verfasst von Igor Buzhinsky und 2 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: Government of the Russian Federation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent studies have shown that modern deep neural network classifiers are easy to fool, assuming that an adversary is able to slightly modify their inputs. Many papers have proposed adversarial attacks, defenses and methods to measure robustness to such adversarial perturbations. However, most commonly considered adversarial examples are based on perturbations in the input space of the neural network that are unlikely to arise naturally. Recently, especially in computer vision, researchers discovered “natural” perturbations, such as rotations, changes of brightness, or more high-level changes, but these perturbations have not yet been systematically used to measure the performance of classifiers. In this paper, we propose several metrics to measure robustness of classifiers to natural adversarial examples, and methods to evaluate them. These metrics, called latent space performance metrics, are based on the ability of generative models to capture probability distributions. On four image classification case studies, we evaluate the proposed metrics for several classifiers, including ones trained in conventional and robust ways. We find that the latent counterparts of adversarial robustness are associated with the accuracy of the classifier rather than its conventional adversarial robustness, but the latter is still reflected on the properties of found latent perturbations. In addition, our novel method of finding latent adversarial perturbations demonstrates that these perturbations are often perceptually small.

Förderung

  • Government of the Russian Federation

Zitieren

APA

Buzhinsky, I., Nerinovsky, A., & Tripakis, S. (2023). Metrics and methods for robustness evaluation of neural networks with generative models. Machine Learning, 112(10), 3977-4012. https://doi.org/10.1007/s10994-021-05994-9

Vancouver

Buzhinsky I, Nerinovsky A, Tripakis S. Metrics and methods for robustness evaluation of neural networks with generative models. Machine Learning. 2023;112(10):3977-4012. doi:10.1007/s10994-021-05994-9

BibTeX
@article{Buzhinsky2023,
  author = {Buzhinsky, Igor and Nerinovsky, Arseny and Tripakis, Stavros},
  title = {Metrics and methods for robustness evaluation of neural networks with generative models},
  journal = {Machine Learning},
  year = {2023},
  volume = {112},
  number = {10},
  pages = {3977--4012},
  doi = {10.1007/s10994-021-05994-9}
}
Kostenlos starten →