EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks

Shuo Feng, Jian Cao, Zehong Ou, Guang Chen, Yi Zhong, Zilin Wang, Juntong Yan, Jue Chen, Bingsen Wang, Chenglong Zou, Zebang Feng, Yuan Wang

Advanced Intelligent Systems · 2024-09 · 6 (9)

DOI: 10.1002/aisy.202400075 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Intelligent Systems, online erschienen am 04.08.2024. Verfasst von Shuo Feng und 11 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As the third‐generation network succeeding artificial neural networks (ANNs), spiking neural networks (SNNs) offer high robustness and low energy consumption. Inspired by biological systems, the limitations of low robustness and high‐power consumption in deep reinforcement learning (DRL) are addressed by introducing SNNs. The Brain Q‐network (BrainQN) is proposed, which replaces the neurons in the classic Deep Q‐learning (DQN) algorithm with SNN neurons. BrainQN is trained using surrogate gradient learning (SGL) and ANN‐to‐SNN conversion methods. Robustness tests with input noise reveal BrainQN's superior performance, achieving an 82.14% increase in rewards under low noise and 71.74% under high noise compared to DQN. These findings highlight BrainQN's robustness and superior performance in noisy environments, supporting its application in complex scenarios. SGL‐trained BrainQN is more robust than ANN‐to‐SNN conversion under high noise. The differences in network output correlations between noisy and original inputs, along with training algorithm distinctions, explain this phenomenon. BrainQN successfully transitioned from a simulated Pong environment to a ball‐catching robot with dynamic vision sensors (DVS). On the neuromorphic chip PAICORE, it shows significant advantages in latency and power consumption compared to Jetson Xavier NX.

Zitieren

APA

Feng, S., Cao, J., Ou, Z., Chen, G., Zhong, Y., Wang, Z., Yan, J., Chen, J., Wang, B., Zou, C., Feng, Z., & Wang, Y. (2024). BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks. Advanced Intelligent Systems, 6(9). https://doi.org/10.1002/aisy.202400075

Vancouver

Feng S, Cao J, Ou Z, Chen G, Zhong Y, Wang Z, et al. BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks. Advanced Intelligent Systems. 2024;6(9). doi:10.1002/aisy.202400075

BibTeX
@article{Feng2024,
  author = {Feng, Shuo and Cao, Jian and Ou, Zehong and Chen, Guang and Zhong, Yi and Wang, Zilin and Yan, Juntong and Chen, Jue and Wang, Bingsen and Zou, Chenglong and Feng, Zebang and Wang, Yuan},
  title = {BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks},
  journal = {Advanced Intelligent Systems},
  year = {2024},
  volume = {6},
  number = {9},
  doi = {10.1002/aisy.202400075}
}
Kostenlos starten →