This paper proposes DRL-RVO, a Deep Reinforcement Learning (DRL) framework incorporating the traditional Reciprocal Velocity Obstacles (RVO) for safe navigation. In this framework, the conventional RVO algorithm, which is widely used for dynamic obstacle avoidance in autonomous systems, is enhanced by integrating a DRL model to dynamically adapt to the behaviors of other human participants in the shared environment. Specifically, the weight coefficient of the RVO algorithm is updated using DRL in real time, improving navigation performance in dynamic environments shared with human participants. To evaluate the effectiveness of the proposed DRL-RVO algorithm, a comparative analysis is performed against state-of-the-art algorithms. The experimental results demonstrate the superior performance of the DRL-RVO algorithm in terms of navigation success rate, real-time adaptability, and efficiency, highlighting the potential benefits of integrating DRL into traditional RVO-based methods for autonomous navigation in dynamic environments.
- Open Access
- Article
DRL-RVO: Learning to Avoid Collision in Crowd Using Reciprocal Velocity Obstacles
- Liyunong Yang,
- Liang Hu *
Author Information
Received: 28 Dec 2025 | Revised: 14 Mar 2026 | Accepted: 15 Apr 2026 | Published: 22 Jun 2026
Abstract
References
- 1.
Borenstein, J.; Koren, Y. Real-time obstacle avoidance for fast mobile robots. IEEE Trans. Syst. Man Cybern. 1989, 19, 1179–1187.
- 2.
Borenstein, J.; Koren, Y. Real-time obstacle avoidance for fast mobile robots in cluttered environments. In Proceedings of the IEEE International Conference on Robotics and Automation, Cincinnati, OH, USA, 13–18 May 1990; pp. 572–577.
- 3.
Borenstein, J.; Koren, Y.; et al. The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Trans. Robot. Autom. 1991, 7, 278–288.
- 4.
Fox, D.; Burgard,W.; Thrun, S. The dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 1997, 4, 23–33.
- 5.
Van den Berg, J.; Lin, M.; Manocha, D. Reciprocal velocity obstacles for real-time multi-agent navigation. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; pp. 1928–1935.
- 6.
Van Den Berg, J.; Guy, S.J.; Lin, M.; et al. Reciprocal n-body collision avoidance. In Proceedings of the Robotics Research: The 14th International Symposium ISRR, Springer, Lucerne, Switzerland, 31 August–3 September 2009; pp. 3–19.
- 7.
Snape, J.; Van Den Berg, J.; Guy, S.J.; et al. The hybrid reciprocal velocity obstacle. IEEE Trans. Robot. 2011, 27, 696–706.
- 8.
Fong, T.; Nourbakhsh, I.; Dautenhahn, K. A survey of socially interactive robots. Rob. Auton. Syst. 2003, 42, 143–166.
- 9.
Kruse, T.; Pandey, A.K.; Alami, R.; et al. Human-aware robot navigation: A survey. Rob. Auton. Syst. 2013, 61, 1726–1743.
- 10.
Kuderer, M.; Kretzschmar, H.; Sprunk, C.; et al. Feature-Based Prediction of Trajectories for Socially Compliant Navigation. In Robotics: Science and Systems VIII; MIT Press: Cambridge, MA, USA, 2013.
- 11.
Alahi, A.; Goel, K.; Ramanathan, V.; et al. Social LSTM: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 961–971.
- 12.
Altche, F.; de La Fortelle, A. An LSTM network for highway trajectory prediction. In Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, 16–19 October 2017; pp. 353–359.
- 13.
Vemula, A.; Muelling, K.; Oh, J. Social attention: Modeling attention in human crowds. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 4601–4607.
- 14.
Rudenko, A.; Palmieri, L.; Herman, M.; et al. Human motion trajectory prediction: A survey. Int. J. Rob. Res. 2020, 39, 895–935.
- 15.
Bennewitz, M.; Burgard, W.; Cielniak, G.; et al. Learning motion patterns of people for compliant robot motion. Int. J. Rob. Res. 2005, 24, 31–48.
- 16.
Trautman, P.; Krause, A. Unfreezing the robot: Navigation in dense, interacting crowds. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 797–803.
- 17.
Fan, T.; Cheng, X.; Pan, J.; et al. Getting robots unfrozen and unlost in dense pedestrian crowds. IEEE Robot. Autom. Lett. 2019, 4, 1178–1185
- 18.
Sathyamoorthy, A.J.; Patel, U.; Guan, T.; et al. Frozone: Freezing-free, pedestrian-friendly navigation in human crowds. IEEE Robot. Autom. Lett. 2020, 5, 4352–4359.
- 19.
Chen, Y.F.; Liu, M.; Everett, M.; et al. Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May–3 June 2017; pp. 285–292.
- 20.
Chen, Y.F.; Everett, M.; Liu, M.; et al. Socially aware motion planning with deep reinforcement learning. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 1343–1350.
- 21.
Long, P.; Fan, T.; Liao, X.; et al. Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 6252–6259.
- 22.
Everett, M.; Chen, Y.F.; How, J.P. Motion planning among dynamic, decision-making agents with deep reinforcement learning. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3052–3059.
- 23.
Chen, C.; Liu, Y.; Kreiss, S.; et al. Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 6015–6022.
- 24.
Wang, S.; Gao, R.; Han, R.; et al. Adaptive environment modeling based reinforcement learning for collision avoidance in complex scenes. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022; pp. 9011–9018.
- 25.
Han, R.; Chen, S.; Wang, S.; et al. Reinforcement learned distributed multi-robot navigation with reciprocal velocity obstacle shaped rewards. IEEE Robot. Autom. Lett. 2022, 7, 5896–5903.
- 26.
Xie, Z.; Dames, P. DRL-VO: Learning to navigate through crowded dynamic scenes using velocity obstacles. IEEE Trans. Robot. 2023, 39, 2700–2719.
- 27.
Chen, L.; Wang, Y.; Miao, Z.; et al. Reciprocal velocity obstacle spatial-temporal network for distributed multirobot navigation. IEEE Trans. Ind. Electron. 2024, 71, 14470–14480.
- 28.
Ban, J.; Li, G. Training is execution: A reinforcement learning-based collision avoidance algorithm for volatile scenarios. IEEE Access 2024, 12, 116956–116967.
- 29.
Fiorini, P.; Shiller, Z. Motion planning in dynamic environments using velocity obstacles. Int. J. Rob. Res. 1998, 17, 760–772.
- 30.
Gopalakrishnan, B.; Singh, A.K.; Kaushik, M.; et al. PRVO: Probabilistic reciprocal velocity obstacle for multi robot navigation under uncertainty. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 1089–1096.
- 31.
Fan, T.; Long, P.; Liu, W.; et al. Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. Int. J. Rob. Res. 2020, 39, 856–892.
- 32.
Han, R.; Chen, S.; Hao, Q. Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May–31 August 2020; pp. 448–454.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Suite 4002 Level 4, 447 Collins Street, Melbourne, Victoria 3000, Australia
General Inquiries: info@sciltp.com