Education, Science, Technology, Innovation and Life
Open Access
Sign In
  1. Home
  2. Journals
  3. Advances in Computer, Signals and Systems
  4. Vol 9, Issue 1, 2025

Map-less Navigation Algorithm for Autonomous Vehicles Based on Deep Reinforcement Learning

Download as PDF

DOI: 10.23977/acss.2025.090117 | Downloads: 18 | Views: 438

Author(s)

Dayu Guo 1, Yuan Zhu 1, Ke Lu 1

Affiliation(s)

1 School of Automotive Studies, Tongji University, No. 4800 Caoan Road, Shanghai, China

Corresponding Author

Ke Lu

ABSTRACT

This paper focuses on the map-less navigation problem of autonomous vehicles based on deep reinforcement learning, and proposes a map-less navigation method for autonomous vehicles based on an improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Aiming at the problems of navigation success rate, exploration performance, and training time of existing map-less navigation algorithms based on deep reinforcement learning, the following innovations are used to optimize the above performance: ① Optimize the neural network structure of the TD3 algorithm to enhance the exploration ability of autonomous vehicles in complex environments. ② Construct a composite reward function to integrate dense rewards and sparse rewards, which significantly speeds up the training speed of the algorithm. Finally, the algorithm in this paper only needs 12% of the training amount of the comparison algorithm to achieve the same success rate. A comprehensive test environment and a special test environment were built in a simulation environment for comparative experiments. The results show that the navigation success rate of the algorithm in this paper is increased by 11.80% in the comprehensive test environment; the obstacle avoidance success rate is increased by 40% and 70% in the special test environment, and the exploration success rate is increased by 100%. In the test of real complex environment, the navigation algorithm is not adjusted, and it can effectively drive the autonomous vehicle to perform map-less navigation. The navigation effect and portability of the algorithm are verified.

KEYWORDS

Map-less Navigation, Deep Reinforcement Learning

CITE THIS PAPER

Dayu Guo, Yuan Zhu, Ke Lu, Map-less Navigation Algorithm for Autonomous Vehicles Based on Deep Reinforcement Learning. Advances in Computer, Signals and Systems (2025) Vol. 9: 123-130. DOI: http://dx.doi.org/10.23977/acss.2025.090117.

REFERENCES

[1] CHAI R, NIU H, CARRASCO J, et al. Design and experimental validation of deep reinforcement learning-based fast trajectory planning and control for mobile robot in unknown environment[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 35(4): 5778-5792.
[2] CAO X, REN L, SUN C. Research on obstacle detection and avoidance of autonomous underwater vehicle based on forward-looking sonar[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 34(11): 9198-9208.
[3] WANG H, HAO J, WU W, et al. A New AGV Path Planning Method Based On PPO Algorithm[C]. 2023 42nd Chinese Control Conference (CCC), 2023: 3760-3765.
[4] WANG R, XU L. Application of Deep Reinforcement Learning in UAVs: A Review[C]. 2022 34th Chinese Control and Decision Conference (CCDC), 2022: 4096-4103.
[5] KAHN G, ABBEEL P, LEVINE S. Badgr: An autonomous self-supervised learning-based navigation system[J]. IEEE Robotics and Automation Letters, 2021, 6(2): 1312-1319.
[6] SHAH D, SRIDHAR A, BHORKAR A, et al. Gnm: A general navigation model to drive any robot[C]//2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023: 7226-7233.
[7] HUANG W, ZHOU Y, HE X, et al. Goal-guided transformer-enabled reinforcement learning for efficient autonomous navigation[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 25(2): 1832-1845.
[8] CIMURS R, SUH I H, LEE J H. Goal-driven autonomous exploration through deep reinforcement learning[J]. IEEE Robotics and Automation Letters, 2021, 7(2): 730-737.
[9] CIMURS R, LEE J H, SUH I H. Goal-oriented obstacle avoidance with deep reinforcement learning in continuous action space[J]. Electronics, 2020, 9(3): 411.
[10] FUJIMOTO S, HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[C]// International conference on machine learning. PMLR, 2018: 1587-1596.

Subscription
E-Mail Alert
Most popular papers
Call for Special Issue
Publication Ethics & OA Statement
Plagiarism Policy
Contact Us
Downloads: 38553
Visits: 697923

Sponsors, Associates, and Links

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.