Deep Reinforcement Learning for Real-World Humanoid Robot Locomotion Control with Automatic Reward Learning
Research · 01.01.2026 · 9
DOI: 10.34133/research.1123 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research, online erschienen am 06.02.2026. Verfasst von Renzhi Lu und 9 weiteren Autor:innen. Beteiligte Einrichtung: School of Artificial Intelligence and Automation, Key Laboratory of Image Processing and Intelligent Control, Engineering Research Center of Autonomo…. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 73 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Humanoid robots possess the potential to solve complex problems across diverse environments, such as nuclear-contaminated zones, epidemic-affected areas, and extraterrestrial missions. However, a humanoid robot is an inherently complex system that integrates multiple disciplines, including perception, mechanical design, materials science, and motion control, each of which requires comprehensive and in-depth investigation. Among these aspects, motion control plays a crucial role, as it directly determines the robot’s motion accuracy, stability, and flexibility. In recent years, with the rapid evolution of graphics-processing-unit-based parallel computing and high-fidelity simulation environments, various deep-reinforcement-learning (DRL)-based approaches have been proposed to achieve precise and robust motion control due to its flexibility and adaptability in uncertain and dynamic environments. However, the inherent complexity and uncertainty of real-world tasks pose substantial challenges when designing effective reward functions for DRL agents. Most current methods typically rely on manually engineered or externally tuned reward signals and therefore require considerable domain expertise, associated with considerable human efforts and a long convergence time; these issues may even trigger mission failure. This work proposes an automatic reward learning method to derive reward functions for DRL in humanoid robot locomotion control. Specifically, a bilevel optimization framework is developed to enable automatic reward learning during policy learning. The reward learning mechanism in the upper level adaptively constructs and optimizes the reward function. The DRL framework in the lower level learns the locomotion control policy using the learned reward function. Three sets of experiments are conducted to verify the effectiveness of the proposed approach: training soft actor-critic and proximal policy optimization agents in MuJoCo environments, training the proximal policy optimization agent in a humanoid robot environment built with Isaac Lab, and transferring the agent to the real-world Unitree G1 humanoid robot through sim-to-real. The experimental results demonstrate that the proposed automatic reward learning method substantially improves learning efficiency and achieves superior performance to manually designed reward functions in both simulation and real-world deployment. By enhancing the success rate of transferring control policies from simulation to real-world humanoid robots, this approach provides a promising pathway toward accelerating the deployment of stable and adaptive humanoid robots in practical applications.
Autor:innen und Institutionen
- School of Artificial Intelligence and Automation, Key Laboratory of Image Processing and Intelligent Control, Engineering Research Center of Autonomous Intelligent Unmanned Systems, Chinese Ministry of Education, Huazho…
- School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China.
- Research Center for Applied Mathematics and Interdisciplinary Science, School of Mathematics and Statistics, Wuhan Textile University, Wuhan 430200, Hubei, China.
- State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China.
- Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK.
- Department of Mechanical Engineering, University of Victoria, Victoria, BC, V8W 3P6, Canada.
ORCID-Profile: Jie Wang
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Lu, R., Wang, J., Shao, Z., Chen, R., Zhu, L., Jiang, Y., Pang, Y., Liang, D., Shi, Y., & Ding, H. (2026). Deep Reinforcement Learning for Real-World Humanoid Robot Locomotion Control with Automatic Reward Learning. Research, 9. https://doi.org/10.34133/research.1123
Vancouver
Lu R, Wang J, Shao Z, Chen R, Zhu L, Jiang Y, et al. Deep Reinforcement Learning for Real-World Humanoid Robot Locomotion Control with Automatic Reward Learning. Research. 2026;9. doi:10.34133/research.1123
BibTeX
@article{Lu2026,
author = {Lu, Renzhi and Wang, Jie and Shao, Zonghe and Chen, Ruijuan and Zhu, Lijun and Jiang, Yuzhi and Pang, Yunyi and Liang, Dongfang and Shi, Yang and Ding, Han},
title = {Deep Reinforcement Learning for Real-World Humanoid Robot Locomotion Control with Automatic Reward Learning},
journal = {Research},
year = {2026},
volume = {9},
doi = {10.34133/research.1123}
}
EUVIMED