EUVIMEDEUVIMED entdecken
Crossref

Reaching the limit in autonomous racing: Optimal control versus reinforcement learning

Yunlong Song, Angel Romero, Matthias Müller, Vladlen Koltun, Davide Scaramuzza

Science Robotics · 13.09.2023 · 8 (82)

DOI: 10.1126/scirobotics.adg1462 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Science Robotics, erschienen am 13.09.2023. Verfasst von Yunlong Song und 4 weiteren Autor:innen. Beteiligte Einrichtung: University of Zurich, Zurich, Switzerland. Die Arbeit wurde bisher 254-mal zitiert (Crossref) und nennt 51 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A central question in robotics is how to design a control system for an agile mobile robot. This paper studies this question systematically, focusing on a challenging setting: autonomous drone racing. We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting. We then investigated which fundamental factors have contributed to the success of RL or have limited OC. Our study indicates that the fundamental advantage of RL over OC is not that it optimizes its objective better but that it optimizes a better objective. OC decomposes the problem into planning and control with an explicit intermediate representation, such as a trajectory, that serves as an interface. This decomposition limits the range of behaviors that can be expressed by the controller, leading to inferior control performance when facing unmodeled effects. In contrast, RL can directly optimize a task-level objective and can leverage domain randomization to cope with model uncertainty, allowing the discovery of more robust control responses. Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour. Our policy achieved superhuman control within minutes of training on a standard workstation. This work presents a milestone in agile robotics and sheds light on the role of RL and OC in robot control.

Autor:innen und Institutionen

  • University of Zurich, Zurich, Switzerland.
  • Intel Labs, Jackson, WY, USA.

Zitieren

APA

Song, Y., Romero, A., Müller, M., Koltun, V., & Scaramuzza, D. (2023). Reaching the limit in autonomous racing: Optimal control versus reinforcement learning. Science Robotics, 8(82). https://doi.org/10.1126/scirobotics.adg1462

Vancouver

Song Y, Romero A, Müller M, Koltun V, Scaramuzza D. Reaching the limit in autonomous racing: Optimal control versus reinforcement learning. Science Robotics. 2023;8(82). doi:10.1126/scirobotics.adg1462

BibTeX
@article{Song2023,
  author = {Song, Yunlong and Romero, Angel and Müller, Matthias and Koltun, Vladlen and Scaramuzza, Davide},
  title = {Reaching the limit in autonomous racing: Optimal control versus reinforcement learning},
  journal = {Science Robotics},
  year = {2023},
  volume = {8},
  number = {82},
  doi = {10.1126/scirobotics.adg1462}
}
Kostenlos starten →