Abstract
The sequential and uncontrolled punishments in social life may lead to what psychologists call learned helplessness or depression. Like learning in social life, agents based on Fuzzy Reinforcement Learning (FRL) sometimes cannot learn well. Experiments show that if an agent continuously performs actions that cause sequential punishments in the beginning of learning, then it does not usually behave well and often selects actions that evoke punishments. Therefore, the learning takes so long or not successful. In this paper, we address this issue called faulty learning in RL algorithm by exploiting learned helplessness. We demonstrate learned helplessness in the training of an agent by FRL algorithm and analyze it. The result of analysis shows that since the action value function is approximated by a fuzzy system; continuous punishments lead all weight parameters of the approximator toward negative amounts. Hence, the agent cannot learn well. To prevent this problem, we propose a new reinforcement function. The proposed reinforcement function is adaptive and depends on the number of visit of the state. Simulation results show that new reinforcement function prevents learned helplessness and improves the learning in terms of learning speed and action quality. The proposed ideas can be used and extended to our social and psychology life.