EUVIMEDEUVIMED entdecken
Crossref

Fully neuromorphic vision and control for autonomous drone flight

F Paredes-Vallés iD, J J Hagenaars iD, J Dupeyroux iD, S Stroobants iD, Y Xu iD, G C H E de Croon iD

Science Robotics · 15.05.2024 · 9 (90)

DOI: 10.1126/scirobotics.adi0591 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Robotics, erschienen am 15.05.2024. Verfasst von F Paredes-Vallés und 5 weiteren Autor:innen. Beteiligte Einrichtung: Micro Air Vehicle Laboratory, Faculty of Aerospace Engineering, Delft University of Technology, Delft, Netherlands. Die Arbeit wurde bisher 105-mal zitiert (Crossref) und nennt 88 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Biological sensing and processing is asynchronous and sparse, leading to low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware for event-based vision and spiking neural networks promises to exhibit similar characteristics. However, robotic implementations have been limited to basic tasks with low-dimensional sensory inputs and motor actions because of the restricted network size in current embedded neuromorphic processors and the difficulties of training spiking neural networks. Here, we present a fully neuromorphic vision-to-control pipeline for controlling a flying drone. Specifically, we trained a spiking neural network that accepts raw event-based camera data and outputs low-level control actions for performing autonomous vision-based flight. The vision part of the network, consisting of five layers and 28,800 neurons, maps incoming raw events to ego-motion estimates and was trained with self-supervised learning on real event data. The control part consists of a single decoding layer and was learned with an evolutionary algorithm in a drone simulator. Robotic experiments show a successful sim-to-real transfer of the fully learned neuromorphic pipeline. The drone could accurately control its ego-motion, allowing for hovering, landing, and maneuvering sideways—even while yawing at the same time. The neuromorphic pipeline runs on board on Intel’s Loihi neuromorphic processor with an execution frequency of 200 hertz, consuming 0.94 watt of idle power and a mere additional 7 to 12 milliwatts when running the network. These results illustrate the potential of neuromorphic sensing and processing for enabling insect-sized intelligent robots.

Autor:innen und Institutionen

  • Micro Air Vehicle Laboratory, Faculty of Aerospace Engineering, Delft University of Technology, Delft, Netherlands.

ORCID-Profile: F Paredes-Vallés, J J Hagenaars, J Dupeyroux, S Stroobants, Y Xu, G C H E de Croon

Zitieren

APA

Paredes-Vallés, F., Hagenaars, J. J., Dupeyroux, J., Stroobants, S., Xu, Y., & de Croon, G. C. H. E. (2024). Fully neuromorphic vision and control for autonomous drone flight. Science Robotics, 9(90). https://doi.org/10.1126/scirobotics.adi0591

Vancouver

Paredes-Vallés F, Hagenaars JJ, Dupeyroux J, Stroobants S, Xu Y, de Croon GCHE. Fully neuromorphic vision and control for autonomous drone flight. Science Robotics. 2024;9(90). doi:10.1126/scirobotics.adi0591

BibTeX
@article{ParedesValls2024,
  author = {Paredes-Vallés, F and Hagenaars, J J and Dupeyroux, J and Stroobants, S and Xu, Y and de Croon, G C H E},
  title = {Fully neuromorphic vision and control for autonomous drone flight},
  journal = {Science Robotics},
  year = {2024},
  volume = {9},
  number = {90},
  doi = {10.1126/scirobotics.adi0591}
}
Kostenlos starten →