Provable observation noise robustness for neural network control systems
Research Directions: Cyber-Physical Systems · 2024
DOI: 10.1017/cbp.2023.5 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Directions: Cyber-Physical Systems, 2024. Verfasst von Veena Krish und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Neural networks are vulnerable to adversarial perturbations: slight changes to inputs that can result in unexpected outputs. In neural network control systems, these inputs are often noisy sensor readings. In such settings, natural sensor noise – or an adversary who can manipulate them – may cause the system to fail. In this paper, we introduce the first technique to provably compute the minimum magnitude of sensor noise that can cause a neural network control system to violate a safety property from a given initial state. Our algorithm constructs a tree of possible successors with increasing noise until a specification is violated. We build on open-loop neural network verification methods to determine the least amount of noise that could change actions at each step of a closed-loop execution. We prove that this method identifies the unsafe trajectory with the least noise that leads to a safety violation. We evaluate our method on four systems: the Cart Pole and LunarLander environments from OpenAI gym, an aircraft collision avoidance system based on a neural network compression of ACAS Xu, and the SafeRL Aircraft Rejoin scenario. Our analysis produces unsafe trajectories where deviations under $1{\rm{\% }}$ of the sensor noise range make the systems behave erroneously.
Zitieren
APA
Krish, V., Mata, A., Bak, S., Hobbs, K., & Rahmati, A. (2024). Provable observation noise robustness for neural network control systems. Research Directions: Cyber-Physical Systems. https://doi.org/10.1017/cbp.2023.5
Vancouver
Krish V, Mata A, Bak S, Hobbs K, Rahmati A. Provable observation noise robustness for neural network control systems. Research Directions: Cyber-Physical Systems. 2024. doi:10.1017/cbp.2023.5
BibTeX
@article{Krish2024,
author = {Krish, Veena and Mata, Andrew and Bak, Stanley and Hobbs, Kerianne and Rahmati, Amir},
title = {Provable observation noise robustness for neural network control systems},
journal = {Research Directions: Cyber-Physical Systems},
year = {2024},
doi = {10.1017/cbp.2023.5}
}
EUVIMED