KinematicRL: A Sim-to-Real Reinforcement Learning Framework for Social Navigation With Kinodynamic Feasibility
IEEE Transactions on Automation Science and Engineering · 2026 · 23 · 11824-11839
DOI: 10.1109/tase.2026.3703136 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2026. Verfasst von Zhiming Xu und 4 weiteren Autor:innen. Die Arbeit nennt 48 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Deep Reinforcement Learning (DRL) has shown promise for social navigation, yet its real-world deployment remains hindered by a persistent sim-to-real gap arising from simplified first-order dynamics and context-specific human state estimation pipelines. This work presents a unified framework that addresses these limitations to produce dynamically feasible navigation policies suitable for real-world deployment. First, theoretical analysis reveals that tracking error between simulated and actual robot position decays exponentially with increased control order, motivating the use of higher-order control inputs as DRL action space. A second-order control formulation tailored to differential drive robots is developed, complemented by a stochastic iterative Linear Quadratic Regulator (iLQR) that pretrains the policy via a divergence minimization objective. Second, to avoid the added system complexity of camera-LiDAR fusion, a cluster-based human tracking pipeline using only 2D LiDAR is introduced. Human detections are associated according to both spatial proximity and velocity similarity, enabling reliable differentiation of nearby pedestrians and yielding stable velocity estimates through temporal aggregation. Third, we introduce an unbiased residual gating block to balance reaction- and memory-based behaviors while handling time-varying crowd sizes, both critical for social navigation. The resulting policy, KinematicRL, consistently improves kinematic performance and adapts to varying number of detected humans. Experiments in real-world environments demonstrate that, when combined with the proposed tracking pipeline, KinematicRL can be deployed on a real differential drive robot with minimal modifications. Note to Practitioners—This work is motivated by the potential of deep reinforcement learning in social navigation and the sim-to-real gap widely witnessed during industrial applications. A comprehensive framework is proposed, ensuring dynamic feasibility in simulation, while effectively providing robust position and velocity estimates from 2D LiDAR data. By incorporating second-order control and warm-starting the policy with a stochastic iterative Linear Quadratic Regulator, our approach ensures dynamic feasibility and enhances kinematic performance for differential drive robots. The proposed cluster-based tracking pipeline relies solely on 2D LiDAR, utilizing online clustering with velocity similarity for robust matching, and historical averaging for smooth velocity estimation. Additionally, a transformer architecture is proposed to handle varying number of detected humans resulting from sensor limitations. Real-world experiments further suggest that a relatively high control rate effectively mitigates jerk-related approximation errors in second-order models, enabling the policy to be directly transferred to a real physical robot with minimal modifications. The proposed methods are computationally efficient and require only standard 2D LiDAR sensors, making them accessible for real-world deployment in industrial and service robotics settings.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Xu, Z., Yang, H., Liu, C., Chen, Q., & Yao, C. (2026). KinematicRL: A Sim-to-Real Reinforcement Learning Framework for Social Navigation With Kinodynamic Feasibility. IEEE Transactions on Automation Science and Engineering, 23, 11824-11839. https://doi.org/10.1109/tase.2026.3703136
Vancouver
Xu Z, Yang H, Liu C, Chen Q, Yao C. KinematicRL: A Sim-to-Real Reinforcement Learning Framework for Social Navigation With Kinodynamic Feasibility. IEEE Transactions on Automation Science and Engineering. 2026;23:11824-11839. doi:10.1109/tase.2026.3703136
BibTeX
@article{Xu2026,
author = {Xu, Zhiming and Yang, Haodong and Liu, Chengju and Chen, Qijun and Yao, Chenpeng},
title = {KinematicRL: A Sim-to-Real Reinforcement Learning Framework for Social Navigation With Kinodynamic Feasibility},
journal = {IEEE Transactions on Automation Science and Engineering},
year = {2026},
volume = {23},
pages = {11824--11839},
doi = {10.1109/tase.2026.3703136}
}
EUVIMED