Developing Decision-Making Algorithm for Unmanned Vessel Navigation Using Markov Processes
Download as PDF
DOI: 10.23977/mastic.023
Author(s)
Ruolan Zhang and Masao Furusho
Corresponding Author
Ruolan Zhang
ABSTRACT
In this study, the autonomous decision-making architecture of unmanned vessel navigation has been formulated. The aim of this study is the advancement of mathematical methods in the ship transportation field with relevance to collision avoidance scenario applications. The process of seafarers safely navigating a vessel at sea entails enacting appropriate decision-making at the appropriate time. In our model, we do not input the
appropriate action order based on a seafarer’s experience. The model scores each step’s reward by its action behaviour and learns how to avoid obstacles by itself. By deploying decision timing, state, reward, and digitizing the seafarer’s decision, we establish a reinforcement learning algorithm based on Markov decision processes. In the model training, under a single factor influence, the vessel tends to change course with the best appropriate action behaviour, which is almost consistent with decision-making behaviour based on actual
experience at sea.
KEYWORDS
Markov decision processes, optimization algorithm, path planning, risk perception, unmanned vessel