Complementary Reward Function Based Learning Enhancement for Deep Reinforcement Learning

Abstract

Several complex sequential decision-making problems have been successfully implemented using reinforcement learning (RL) for continuous optimal control. However, the sample efficiency of data collection process during the learning phase is still not well addressed. The convergence rate to the optimal policy as well as the time of the learning process are strongly influenced by the efficiency of the data collected by the agent during the learning phase. This paper proposes a method to generate efficient sample data which allows the agent to collect high reward trajectories more frequently, decreasing the learning phase time. The proposed method consists of Complementary reward (CR) function augmented to the traditional reward function. The CR tends to infinity when the control input leads to the system performance that meets the given requirements very accurately. Consequently, the control policy which maximizes the reward function can render the system to optimal performance. The main contributions of this study include the following aspects: (1) a new proposed Complementary reward that is augmented to the reward function which improves performance of the reinforcement learning based controller in terms of system requirements; (2) speed-up of training phase via generation of more efficient data resulting in a better learned policy.