A navigation method for mobile robots using interval type-2 fuzzy neural network fitting Q-learning in unknown environments

Abstract

This paper presents a navigation algorithm based on interval type-2 fuzzy neural network fitting Q-learning (IT2FNN-Q), and succeeds in providing a solution for mobile robot navigation in complex environments. The algorithm utilizes the fuzzy reasoning adaptive ability and extensive functional approximation features of IT2FNN to solve this problem, mapped from state space to action space, of the Q-learning algorithm in unknown environments. Compared with the BP fitting Q-learning algorithm (BP-Q), IT2FNN-Q endows the robot with better adaptive and real-time decision-making abilities and solves the slow convergence and nonconvergence problems, through its local approximation. By comparison with the fuzzy neural network fitting Q-learning algorithm (FNN-Q), this proposed algorithm has more advantages for dealing with the external uncertainty, enabling the robot to complete a better path with less fuzzy rules. The results of the simulation and comparison of the proposed method with FNN-Q and BP-Q revealed that the mobile robot can navigate itself in complex environments with fewer steps, obtaining more reward values by adopting the algorithm presented in this paper.

Keywords

Mobile robots Q-learning robot navigation interval type-2 fuzzy neural network (IT2FNN)

1 Introduction

Mobile robot navigation in unknown environments is of current interest. Besides a complicated and changeable real-time system, robots in unknown complex dynamic environments also face other issues, such as uncertainties in the obstacle position, size, and scale as well as incorrect or incompletely measured data from the sensors [1, 2]. Therefore, a robot can only complete navigation by itself through self-learning in complex environments. Reinforcement learning enables an intelligent agent to perceive the state of the external environment, making it possible for the agent to select an optimal action through trial and error [3]. Furthermore, it does not need a priori knowledge to establish accurate mathematical models, and, as a consequence, is widely used in navigation systems.

In practical applications, when the state and action spaces of the external environment are in succession or excessive in number, the reinforcement learning training time becomes too long and the convergence rate is too slow. The effective way to solve this problem is to use a function approximation algorithm to realize the mapping from the state space to the action space [4 –7], e.g., a fuzzy inference system (FIS) or neural network (NN). Paper [8] proposed a Q-learning algorithm based on FIS. The inference of FIS, which is more suitable for expressing fuzzy or qualitative knowledge, is similar to that of human thinking. The traditional fuzzy control method depicts the fuzzy set by the precise membership function, but the uncertain external factors widely exist, such as the uncertainty of fuzzy system inputs, fuzzy rule inputs, and training data. To deal with these uncertainties, Zadeh proposed an expansion of the membership of the fuzzy sets; from the exact values to fuzzy sets for further enhanced fuzziness [9]. Hagras presented a hierarchical Type-2 fuzzy logic control architecture for autonomous mobile robots that considered the uncertainties facing mobile robots in unstructured environments [10]. The type-2 fuzzy system includes a stronger description and processing of uncertain relationship. In addition, the type-2 fuzzy system lacks self-learning and self-adaptation abilities. By contrast, the neural network has the advantages of parallel computation, strong fault-tolerance, and adaptive learning; however, it does not make good use of existing and prior knowledge [11]. Therefore, the type-2 fuzzy system and neural network are combined to form a type-2 fuzzy neural network (T2FNN) so that the control system has both the advantages of the fuzzy reasoning adaptive abilities and extensive function approximation characteristics [12 –14].

In order to reduce the complexity, the KM algorithms and error reverse transmissions are used to perform the type-reduction of the system [15] and to adjust the model parameters and weights, respectively. The interval type-2 fuzzy neural network (IT2FNN) was applied to the navigation system [16], which not only solves the problem of reinforcement learning for a large number of state spaces and difficult convergences, but also uses less rules to achieve a higher navigation accuracy.

The remainder of this paper is organized as follows: Section 2 provides an introduction to Q-learning and T2FNN; Section 3 describes the proposed method of IT2FNN fitting Q-learning and its application to robot navigation; Section 4 provides simulation results to demonstrate the advantages of the proposed method; the results are discussed in Section 4; finally, the concluding remarks are demonstrated in Section 5.

2 Q-learning and type-2 fuzzy neural network

2.1 Q-learning

Reinforcement learning is a principle of “trial and error” based on animal learning psychology. It can optimize decision sequences based on evaluative feedback signals in the process of interacting with the environment. Q-learning is a major reinforcement learning algorithm. It is not related to the external environment model and can provide the agent with the ability to select the best action. The best action is obtained by using the sequence of actions that have been experienced in the Markov decision process [17]. The algorithm achieves certain convergence conditions by continuously reducing the difference between the Q-estimates and adjacent states. Q (s_t, a_t) is updated as follows: $Q (s_{t}, a_{t}) = R (s_{t}, a_{t}) +$ $γ P (s_{t}, a_{t}, s_{t + 1}) \cdot max Q (s_{t + 1}, a_{t + 1}),$ (1) where (s_t, s_t+1) ∈ S, (a_t, a_t+1) ∈ A; S and A are the state and action sets,respectively; Q (s_t, a_t) is an estimation function corresponding to the state s_t and action a_t; R (s_t, a_t) is the obtained reward value after the action a_t is performed under state s_t; γ is the discount factor; and P (s_t, a_t, s_t+1) is the probability of the state s_t transition to the state s_t+1 after performing the action a_t. The initialized Q value is arbitrarily specified after each execution of the action corresponding to the maximum Q value. The Q value is updated according to $\begin{matrix} Q (s_{t}, a_{t}) \leftarrow Q (s_{t}, a_{t}) + \\ α [R + γ max_{a_{t}} Q (s_{t + 1}, a_{t + 1}) - Q (s_{t}, a_{t})], \end{matrix}$ (2) where α is the learning rate.

2.2 Type-2 fuzzy neural network

The type-2 fuzzy set is an extension of the traditional type-1 fuzzy set. The language information of the original data and uncertainty of the data are translated into the uncertainty of the fuzzy rules to be described by the type-2 fuzzy set. Its membership itself contains fuzzy sets. First, the exact input variable is subjected to type-2 fuzzification to obtain the type-2 fuzzy set input. Then, the type-2 fuzzy set output is obtained by fuzzy reasoning. Finally, the system requires precise output variables. Therefore, the demodulator is used to process the fuzzy set output before the defuzzifier. The principle diagram of the type-2 fuzzy system is shown in Fig. 1.

Fig.1

Principle diagram of the type-2 fuzzy system.

The type-2 fuzzy system and neural network are combined to form the T2FNN. This algorithm has advantages [18] and can approximate any nonlinear and uncertain systems. This paper presents the IT2FNN, which can rapidly generate fuzzy rules and adjust the internal structure and parameters. IT2FNN has the interval type-2 membership functions of an adjustable standard deviation and uncertainty center value. Figure 2 shows the type-2 Gaussian membership function. The type-2 fuzzy set is a domain that describes the uncertain combination of the membership degree function. The shaded part of Fig. 2 is the footprint uncertainty (FOU) of the interval type-2 fuzzy set.

Fig.2

Gaussian membership function.

3 Reinforcement learning based on IT2FNN and its application in robot navigation

An IT2FNN has a strong ability of self-adaptability and a wide range of function approximation properties. Here, IT2FNN was combined with reinforcement learning. Such a network can-not only obtain the conclusion part of the fuzzy rule and parameters of the fuzzy membership function, but also simultaneously solve the reinforcement learning problem of continuous state and action spaces, realizing reinforcement learning using the relatively few fuzzy rule bases. Therefore, this network further improves the accuracy of the system. The input variables of IT2FNN are the state vectors of reinforcement learning. The outputs of the fuzzy rule are used as the action spaces of reinforcement learning. The state to action mapping is implemented by an extensive function approximation of IT2FNN. The reinforcement signals obtained by Q-learning in the environment constitute the error cost function of the network outputs. The network system determines the fuzzy rule by the back propagation of errors and adjusts the parameters of the fuzzy membership function. The corresponding block diagram is shown in Fig. 3.

Fig.3

Structure of type-2 fuzzy neural network and reinforcement learning.

The external environment information obtained by the mobile robot body’s sensors and cameras is used for the network inputs. The number of inputs is n = 4. The output is the Q value corresponding to the set action. The number of outputs is m = 5. The fuzzy rule number is K = 3. The layers are mathematically described as follows.

Layer 1 (Input Layer): For each node i in this layer, s_i denotes the inputs of IT2FNN. s = (s₁, s₂, s₃, s₄).

Layer 2 (Membership Layer): The interval type-2 Gaussian membership function is represented by

$μ_{i}^{j} (s_{i}) = [{\underline{μ}}_{i}^{j} (s_{i}), {\bar{μ}}_{i}^{j} (s_{i})]$ (3) $\begin{matrix} {\underline{μ}}_{i}^{j} (s_{i}) = {\begin{matrix} exp [\frac{- {(s_{i} - {\underline{ξ}}_{i}^{j})}^{2}}{2 {(σ_{i}^{j})}^{2}}] & s_{i} < {\underline{ξ}}_{i}^{j} \\ 1 & {\underline{ξ}}_{i}^{j} < s_{i} < {\bar{ξ}}_{i}^{j} \\ exp [\frac{- {(s_{i} - {\bar{ξ}}_{i}^{j})}^{2}}{2 {(σ_{i}^{j})}^{2}}] & s_{i} > {\bar{ξ}}_{i}^{j} \end{matrix} \\ {\bar{μ}}_{i}^{j} (s_{i}) = {\begin{matrix} exp [\frac{- {(s_{i} - {\bar{ξ}}_{i}^{j})}^{2}}{2 {(σ_{i}^{j})}^{2}}] & s_{i} \leq \frac{{\underline{ξ}}_{i}^{j} + {\bar{ξ}}_{i}^{j}}{2} \\ exp [\frac{- {(s_{i} - {\underline{ξ}}_{i}^{j})}^{2}}{2 {(σ_{i}^{j})}^{2}}] & s_{i} > \frac{{\underline{ξ}}_{i}^{j} + {\bar{ξ}}_{i}^{j}}{2} . \end{matrix} \end{matrix}$

Where i = 1, 2, . . . , N and j = 1, 2, . . . , K. ${\bar{μ}}_{i}^{j}$ and ${\underline{μ}}_{i}^{j}$ are the uncertain footprints of the upper and lower limits of membership functions, respectively. ${\bar{ξ}}_{i}^{j}$ and ${\underline{ξ}}_{i}^{j}$ are the center values of the Gaussian function. $σ_{i}^{j}$ is the standard deviation of the Gaussian function.

Layer 3 (Rule Layer): Each node of the layer represents a fuzzy logic rule, and each node output corresponds to the strength of the fuzzy logic rule F^j. F^j is the interval type-1 fuzzy set;

$F^{j} = [{\underline{f}}^{j}, {\bar{f}}^{j}] = [Π_{i}^{n} {\underline{μ}}_{i}^{j} (s_{i}), Π_{i}^{n} {\bar{μ}}_{i}^{j} (s_{i})] .$ (4)

Layer 4 (Type-Reduction Layer): This layer calculates the left and right points through the KM algorithm;

${y_{l}}_{k} = \frac{\sum_{j = 1}^{L} {\bar{f}}^{j} w_{lk}^{j} + \sum_{j = L + 1}^{N} {\underline{f}}^{j} w_{lk}^{j}}{\sum_{j = 1}^{L} {\bar{f}}^{j} + \sum_{j = L + 1}^{N} {\underline{f}}^{j}}$ (5)

${y_{r}}_{k} = \frac{\sum_{j = 1}^{M} {\bar{f}}^{j} w_{rk}^{j} + \sum_{j = M + 1}^{N} {\underline{f}}^{j} w_{rk}^{j}}{\sum_{j = 1}^{M} {\bar{f}}^{j} + \sum_{j = M + 1}^{N} {\underline{f}}^{j}} .$ (6)

Here [ $w_{l}^{j}$ , $w_{r}^{j}$ ](j = 1, 2, 3, …, N) and (k = 1, 2, …, m) are the weight vectors of interval set, and the variables L and M are the values of the KM iterative algorithm.

Layer 5 (Output Layer): This is based on the center value of the left and right points, and the exact fuzzy value of the system is calculated by;

$Q (s_{t}, a_{k}) = y_{k} = \frac{y_{lk} + y_{rk}}{2},$ (7) where k = 1, 2, …, m.

From (7), the maximum Q value of the corresponding action is chosen as the output with a certain selection strategy. we define;

$π (a_{t}) = arg max Q (s_{t}, a_{t}) .$ (8)

The mobile robot reaches the new state s_t at this moment, and the corresponding reward value, R, will be simultaneously obtained from the environment. In accordance with the above method, the next action value, Q (s_t+1, a_t+1), is calculated. The output error is expressed as follows:

$e = α [R + γ max Q (s_{t + 1}, a_{t + 1}) - Q (s_{t}, a_{t})] .$ (9)

Here, the error function is $E = \frac{1}{2} e^{2}$ . The learning algorithm based on the gradient descent is used to adjust the structure and consequent parameters in this study. The structure parameters $({\underline{ξ}}_{i}^{j}, {\bar{ξ}}_{i}^{j}, σ_{i}^{j})$ and the consequent parameters $(w_{l}^{j}, w_{r}^{j})$ are updated as follows:

$w_{l}^{j} (k + 1) = w_{l}^{j} (k) + \frac{1}{2} η_{w} e (k) [\frac{f_{l}^{j}}{\sum_{j = 1}^{N} f_{l}^{j}}]$ (10)

w_{r}^{j} (k + 1) = w_{r}^{j} (k) + \frac{1}{2} η_{w} e (k) [\frac{f_{r}^{j}}{\sum_{j = 1}^{M} f_{r}^{j}}]

(11)

{\underline{ξ}}_{i}^{j} (k + 1) = {\underline{ξ}}_{i}^{j} (k) + \frac{1}{2} η_{ξ} e (k) [\frac{(w_{l}^{j} - y_{l})}{\sum_{j = 1}^{N} f_{l}^{j}} \times

(x_{i} (k) - {\underline{ξ}}_{i}^{j} (k)) \times {\underline{f}}^{j} / (σ_{i}^{j} (k))^{2}]

(12)

{\bar{ξ}}_{i}^{j} (k + 1) = {\bar{ξ}}_{i}^{j} (k) + \frac{1}{2} η_{ξ} e (k) [\frac{(w_{r}^{j} - y_{r})}{\sum_{j = 1}^{N} f_{r}^{j}} \times

(x_{i} (k) - {\bar{ξ}}_{i}^{j} (k)) \times {\bar{f}}^{j} / (σ_{i}^{j} (k))^{2}]

(13)

σ_{i}^{j} (k + 1) = σ_{i}^{j} (k) + \frac{1}{2} η_{σ} e (k) [\frac{(w_{r}^{j} - y_{r})}{\sum_{j = 1}^{N} f_{r}^{j}} \times

(x_{i} (k) - {\bar{ξ}}_{i}^{j} (k)) \times f^{j} / (σ_{i}^{j} (k))^{3}] .

(14)

The main purpose of IT2FNN learning is to optimize the network step by step, maximize the system’s reward value through the online parameters of fuzzy membership function, and mapping the relationship of the fuzzy rules online. The structure of the IT2FNN fitting Q-learning is shown in Fig. 4.

Fig.4

Structure of IT2FNN fitting Q-learning.

The proposed IT2FNN-Q algorithm is presented in Table 1.

Table 1

IT2FNN-Q algorithm

1. t = 0, initialize $({\underline{ξ}}_{i}^{j}, {\bar{ξ}}_{i}^{j}, σ_{i}^{j})$ , $(w_{l}^{j}, w_{r}^{j})$ , and Q_ij.
2. Observe the current world state S_t.
3. Input the processed states into the IT2FNN network,
and compute the Q-values.
4. Choose a_t in S_t using the adequate policy (e.g., ɛ-greedy).
5. Observe the new state S_t+1.
6. Compute the obtained reward value R.
7. Update the Q-values by (2).
8. Compute the output error e by (9).
9. Adjust the weights, center values and variances by
error back-propagation.
10. t ← t + 1, go to 2, and repeat all steps until the target
is achieved.

4 Simulation and results analysis

4.1 System input and output settings

In this study, the IT2FNN system is used to approximate the Q (s, a) value. The network input is a four-dimensional vector, which corresponds to the state variables of the mobile robot state space. We define s = [s_left, s_front, s_right, s_goal], where s_left, s_front, and s_right are the minimum distances between the left, front, and right sides, respectively, of the mobile robot and obstacle. s_goal is the angle between the body direction of the mobile robot and target point. In order to smoothen the robot movement path, this paper proposes the setting of five output actions, which include left turn 30°, left turn 15°, straight, right turn 15°, and right turn 30° respectively. The mobile robot running speed is determined by the distance between the robot body and its surrounding obstacles. If there are no obstacles in front, the mobile robot will move at full speed. The mobile robot slows down when encountering obstacles. The calculation formula of the mobile robot velocity is defined as follows:

$v = β \cdot d_{\min}, d_{\min} \leq D .$ (15)

Here, v is the velocity of the mobile robot; β is the velocity scale coefficient; D is the maximum distance measured by the sensor; and d_min is the shortest distance between the robot and obstacle. If the value of d_min is greater than D, then d_min = D, in which case the mobile robot has a maximum travelling speed.

4.2 Fuzzifying input states

To make better use of the expertise, when setting the structure parameters the input variables are fuzzified based on expertise. The universe of discourse of the distances s_left, s_front, and s_right between the body and obstacle are divided into {danger, moderation, and safety}, respectively. The universe of discourse of the angles between the body direction and target point are divided into {negative, zero, and positive}. In this study, the interval type-2 Gaussian membership function with uncertain central value is used. The result of the fuzzification is shown in Fig. 5.

Fig.5

Fuzzy membership functions of input parameters.

4.3 Reward signal setting

Generally, a mobile robot needs to complete two tasks; avoid obstacles and approach the target. To approach the target, the closer the robot is to the target, the larger the reward value. In contrast, if stays away from the target point, it will be penalized. As for obstacle avoidance it is desirable that the distance between the mobile robot and obstacle is maximized, and the rule is that the closer the robot is to the obstacle, the more it will be penalized, i.e., the mobile robot is rewarded for staying away from obstacles. Setting a reasonable reward value is a key factor in Q-learning. Considering the above two behaviors, the reward values are set as follows:

$R = R_{od} + R_{gd}$ (16) $R_{od} = {\begin{matrix} 3 & Away from obstacles \\ - & 5 & Near obstacles \\ 0 & Other situations \end{matrix}$ $R_{gd} = {\begin{matrix} \begin{matrix} 2 Close to the target \\ 5 Arrive at the target \end{matrix} \\ - 2 Away from the target . \end{matrix}$

4.4 Simulation results in different environments

First, various network parameters are initialized. The parameters are as follows: D = 5, γ = 0.9, η_w = 0.5, η_ξ = 0.2, η_σ = 0.1, and ɛ = 0.85. The algorithm combining the above-mentioned IT2FNN and Q-learning is applied to the robot navigation system. Different simulation environments are established in the MATLAB GUI. When the mobile robot is set to collide with obstacles, it is set to return to the origin and retrain the operation. After 100 training periods, the consumption steps and consumption time to the target are shown in Figs. 6 and 7.

Fig.6

Consumption steps to the target.

Fig.7

Consumption time to the target.

Figures 6 and 7 show that the robot reached the target point after 247 steps, and it ran for 23.78 s in 40 training periods.

A simulation verification was performed in different environments, and the results are shown in the Fig. 8.

Fig.8

Navigation simulation results in different environments (No interference).

Figure 8 shows that the mobile robot can quickly and effectively avoid obstacles from the starting point to the target point after using the combined algorithm. The navigation algorithm has a good generalization ability in different environments. When the starting position of the robot is changed, that is, when the external state changes, the navigation system can still output a reasonable response quickly.

In the previous analysis, the sensor measurement was assumed to be ideal and unbiased; however, there would be unavoidable measurement errors in actual operation. Therefore, assuming that the measurement error of the sensor is less than 5%, random noise was added to the ideal input measurement. Through multiple simulation training experiments, the simulation results using MATLAB are shown in Fig. 9.

Fig.9

Navigation simulation results after adding interference.

Figure 9 shows the robot behavior after the input of the network is added with strong noise interference. As shown in Fig. 9, the running path of the mobile robot is not optimal, but it can successfully realize the navigation of the mobile robot without collision and avoiding obstacles. Simulation experiments show that the greater number of experimental training times, the smoother the walking path of the mobile robot. Thus, the superiority of the algorithm in dealing with interference input is achieved.

4.5 Comparisons of different navigation algorithms

The simulation results show that the navigation algorithm combined with IT2FNN and Q-learning enables mobile robots to complete collision-free navigation. The performance of reinforcement learning is mainly measured by the convergence. However, the convergence can be reflected by the reward value of the reinforcement learning. To verify the superiority of the proposed algorithm in the same environment, the comparative experiments for three algorithms were conducted, namely the IT2FNN-Q, FNN-Q [11], and BP-Q [19] algorithms. One hundred training cycles are considered. Figure 10 shows the normalized reward value variation curves obtained by the three algorithms through the control strategy.

Fig.10

Normalized reward values for the three algorithms.

For an increasing learning number, the cumulative reward value obtained by each algorithm is gradually increased and finally reaches the convergence, as shown in Fig. 10. Each input variable is divided into three fuzzy rules in IT2FNN. However, it is divided into six fuzzy rules in FNN. The IT2FNN-Q navigation algorithm can acquire more accumulated reward values with fewer fuzzy rules, and it achieves a faster convergence rate. FNN-Q is better than BP-Q in terms of generalization ability. Thus, the results show that the BP network has the worst convergence rate. For the case of setting the same starting and target points, the number of iteration steps required for IT2FNN-Q to complete the navigation task is minimal, as shown in Table 2. It shows that the proposed algorithm (IT2FNN-Q) has better practicability and accuracy.

Table 2

Iterative steps comparison

Distance	IT2FNN-Q	FNN-Q	BP-Q
82	152	157	165
98	179	189	201
30	46	51	53
41	73	79	81
87	228	230	234
71	141	146	148

5 Conclusions

A mobile robot navigation algorithm for complex environments was proposed using IT2FNN fitting Q-learning. The function approximation feature of IT2FNN is used to solve the mapping relationship between the state and action spaces in Q-learning. The KM algorithm was used to reduce the order of the two fuzzy inputs, and the gradient descent method was used to adjust the parameters of the network. The result shows that the proposed algorithm (IT2FNN-Q) can still enable mobile robots to complete navigation tasks, even when there is interference from the external surroundings. Compared with FNN-Q and BP-Q, the proposed algorithm enables Q-learning to achieve a faster convergence rate by using fewer fuzzy rules. By changing the travelling speed, the mobile robot completes the navigation task in a complex environment with fewer iteration steps. The proposed algorithm has strong obstacle avoidance and a self-adaptive ability and robustness, which enables mobile robots to complete autonomous navigation tasks in complex environments in a short time.

Footnotes

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (grant no. 31760342) and Guangxi University Experimental Teaching Reform Project in 2017.

References

Cai

Z.X.

, Cai

H.G.

and He

, Chen, Some issues for mobile robots navigation under unknown environments, Control and Decision 17(4) (2002), 385–390.

Savkin

A.V.

and Wang

, Seeking a path through the crowd: Robot navigation in unknown dynamic environments with moving obstacles based on an integrated environment representation, Robotics and Autonomous Systems 62 (2014), 1568–1580.

Mohammad Abdel

K.J.

, Mohammad

Al.-R.

and Lara

Reinforcement based mobile robot navigation in dynamic environment, Robotics and Computer-Integrated Manufacturing 27 (2011), 135–149.

Gao

, Chen

S.F.

and Lu

, Research on reinforcement learning technology: A review, Acta Automatica Sinica 30(1) (2004), 86–100.

Xiao

H.R.

, Liao

and Zhou

F.Y.

, Mobile robot path planning based on Q-ANN, IEEE International Conference on Automation and Logistics (2007), pp. 2650–2654.

H.Y.

, Liu

S.H.

and Zhang

, Application of reinforcement learning based on radial basis function neural networks in robot navigation, Journal of Jilin University 27 (2009), 185–190.

, Yung

N.H.C.

and Wang

D.W.

, A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance, IEEE Transactions on Systems Man and Cybernetics - Part B: Cybernetics 33 (2003), 17–27.

Boubertakha

, Tadjineb

and Glorennecc

P.Y.

, A new mobile robot navigation method using fuzzy logic and a modified Q-learning algorithm, Journal of Intelligent & Fuzzy Systems 21 (2010), 113–119.

Zadeh

L.A.

, The concept of a linguistic variable and its application to approximate reasoning-I, Information sciences 8 (1975), 199–249.

10.

Hagras

, A hierarchical Type-2 fuzzy logic control architecture for autonomous mobile robots, IEEE Transactions on Fuzzy Systems 12(4) (2004), 524–539.

11.

Duan

and Xu

X.H.

, Reinforcement learning based on FNN and its application in robot navigation, Control and Decision 22 (2005), 525–529.

12.

B.Q.

and Wang

C.Y.

, On type-2 fuzzy relations and interval-valued type-2 fuzzy sets, Fuzzy Sets and Systems 236 (2014), 1–32.

13.

Castro

J.R.

, Castillo

, Melin

and Rodriguez-Diaz

, A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks, Information Sciences 179 (2009), 2175–2193.

14.

Valavanis

K.P.

, Doitsidis

, Long

and Murphy

R.R.

, A case study of fuzzy-logic-based robot navigation, IEEE Robotics & Automation Magazine 13 (2006), 93–107.

15.

and Mendel

J.M.

, Enhanced Karnik-Mendel algorithms, IEEE Transactions on Fuzzy Systems 17 (2009), 923–934.

16.

Kim

and Chwa

, Obstacle avoidance method for wheeled mobile robots using interval type-2 fuzzy neural network, IEEE Transactions on Fuzzy Systems 23 (2015), 677–687.

17.

Watkins

and Dayan

, Q-learning, Machine Learning 8 (1992), 279–292.

18.

Lee

, Hong

, Lin

and Lai

, Type-2 fuzzy neural network systems and learning, International Journal of Computation and Cognition 1 (2003), 79–90.

19.

Mahadevan

and Connell

, Automatic programming of behavior-based robots using reinforcement learning, Artificial Intelligence 55 (1992), 311–365.