Neural network updating via argument Kalman filter for modeling of Takagi-Sugeno fuzzy models

Abstract

In this article, an argument Kalman filter is exposed for the fast updating of a neural network. The argument Kalman filter is developed based on the extended Kalman filter, but the recommended scheme has the next two advantages: first, it has less computational complexity because it only employs the Jacobian argument instead of the full Jacobian, second, its gain is ensured to be uniformly stable based on the Lyapunov approach. The commented scheme is applied for the modeling of two Takagi-Sugeno fuzzy models.

Keywords

Argument Kalman filter modeling fuzzy models

1 Introduction

In the fuzzy models, the fuzzy logic employs values between 0 (it represents totally false facts) and 1 (it represents totally true facts). The binary logic is a particular case of the fuzzy logic. The concepts are associated to fuzzy sets (they are associated to membership functions) in the fuzzification. Once the fuzzified values are gotten, the fuzzy rules are employed to get a fuzzy output. Later, the fuzzy output can be defuzzified to get a discrete value [2, 9, 12, 25]. And the neural networks are computing systems vaguely inspired by the biological neural networks that constitute animal brains. Such models learn dynamic behaviors by considering examples, generally without a specific programming [1, 17, 19, 23]. It would be interesting to take into account neural networks to improve the capacities of fuzzy models, some examples of this goal are described in [7, 21, 22, 29].

On the other hand, there is some development about filters applied for updating of neural networks. The search filters of neural networks are focused in [4] and [28]. In [8, 13, 14], nonlinear filters of neural networks are mentioned. Single-pass active filters of neural networks are employed in [10, 11]. From all filters, the Kalman filter is one of the most famous because it is fast.

The Kalman filter is an approach that exploits a series of measurements observed over time, containing statistical noise and other inaccuracies, and it estimates unknown variables [5]. There is some development about Kalman filters for fast updating of neural networks. The extended Kalman filter of neural networks is discussed in [3, 15, 26]. In [6, 27, 30, 31], the unscented Kalman filter of neural networks is considered. The decoupled Kalman filter of neural networks is addressed in [3, 24, 32]. The aforementioned studies show that several variants of Kalman filters have been recently employed for the fast updating of neural networks. Consequently, any new develop in this creative topic will be well received.

Other Kalman filters are conformed by the decoupled Kalman filter, unscented Kalman filter, extended Kalman filter, and are characterized to employ the full Jacobian, while the argument Kalman filter is characterized to employ the Jacobian argument. Other Kalman filters have two difficulties for fast updating of a neural network: first, they have high computational complexity because they compute many operations in their full Jacobians, second, some of their elements could become unstable. In this article, an argument Kalman filter is exposed to solve the above mentioned difficulties as follows: first, it only uses the Jacobian argument instead of the full Jacobian to compute less operations, second, its gain is ensured to be uniformly stable.

In this article, an argument Kalman filter is employed for the fast updating a neural network. The develop of the exposed approach is based on three aspects. First, the Taylor series is employed to get the modeling dynamic equality, the mathematical expectation of the factors error and the modeling dynamic equality are exploited to get the extended Kalman filter, and the mathematical expectation of factors error and modeling dynamic equality are employed to get the argument Kalman filter. Second, the gain of the argument Kalman filter is ensured to be uniformly stable based on the Lyapunov approach. And third, the computational complexity comparison between the other Kalman filters and argument Kalman filter is analyzed based on the algebraic operations number employed by both approaches.

Finally, the argument Kalman filter is compared with the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter for the modeling of two Takagi-Sugeno fuzzy models. The importance of mentioned fuzzy models is that they can be applied to chaos or pendulums [15].

The remainder of the article is as follows. The neural network, is explained in the sections 2, In section 3, the extended Kalman filter is developed. The argument Kalman filter, its stability analysis, and its computational complexity are explained in Section 4. In Section 5, the commented scheme is summarized. The argument Kalman filter is applied for modeling of two examples in Section 6. In Section 7, conclusions and future develop areexpressed.

2 The neural network and its modeling dynamic equality

In this part of the article: first, the neural network will be explained, second, the Taylor series is employed to get the modeling dynamic equality, it will be exploited after in the article.

2.1 The neural network

Take into account the Takagi-Sugeno fuzzy models: $D_{k + 1} = H_{k} = h (D_{k}) D_{k}$ (1) with $D_{k} \in R^{N \times 1}$ are states, N is the number of states, and $h (\cdot) \in ℜ^{N \times N}$ is the unknown behavior of the Takagi-Sugeno fuzzy model, it is a smooth and continuous function.

In this study, a special neural network is developed which only has one hidden layer. It could be extended to a general multilayer neural network; nevertheless, this study is concentrated in a smaller neural network.

The neural network with the input, hidden, and output layers is: $\begin{matrix} {\hat{D}}_{k + 1} = {\hat{H}}_{k} = W_{k} Φ_{k} \\ Φ_{k} = {[φ_{1, k}, \dots, φ_{j, k}, \dots, φ_{M, k}]}^{T} \\ φ_{j, k} = tanh (V_{j, k} D_{k}) \end{matrix}$ (2) with j = 1, …, M, M is the neurons number in the hidden layer, ${\hat{D}}_{k + 1} \in ℜ^{N \times 1}$ are neural network outputs, $D_{k} \in R^{N \times 1}$ are states, $Φ_{k} \in R^{M \times 1}$ are the activation maps, $V_{k} \in R^{M \times N}$ and $W_{k} \in R^{N \times M}$ are the hidden layer and output parameters, $V_{j, k} \in R^{1 \times N}$ , $φ_{j, k} \in R$ , tanh(⋅) is the hyperbolic tangent map.

The modeling error ${\tilde{D}}_{k + 1} \in R^{N \times 1}$ is: ${\tilde{D}}_{k + 1} = {\hat{D}}_{k + 1} - D_{k + 1}$ (3) with D_k+1 and ${\hat{D}}_{k + 1}$ are explained in (1) and (2).

Figure 1 shows the architecture of the neural network where the input layer, hidden layer, and output layer are observed.

Fig.1

Architecture of the neural network.

Remark 1. Since there is a result in [3] which proves that a neural network with one hidden layer of (2) is sufficient for the admissible estimation of a Takagi-Sugeno fuzzy models. In this article, this kind of neural network is exploited.

2.2 The modeling dynamic equality

The linearization of the neural network is required to get the modeling dynamic equality, this equality will be employed for the develop of the extended Kalman filter and argument Kalmanfilter.

According to the Stone-Weierstrass theorem, the unknown complex map H_k of (1) is approximated: $\begin{matrix} D_{k + 1} = H_{k} = W_{*} Φ_{* k} + \in_{k} \\ Φ_{* k} = {[φ_{* 1, k}, \dots, φ_{* j, k}, \dots, φ_{* M, k}]}^{T} \\ φ_{* j, k} = \tanh (V_{* j} D_{k}) \end{matrix}$ (4) where $Φ_{* k} \in R^{M \times 1}$ , $\in_{k} = D_{k + 1} - W_{*} Φ_{* k} \in R^{N \times 1}$ is the unmodeled dynamics error, $ϕ_{* j, k} \in ℜ V_{* j} \in ℜ^{1 \times N}$ , $V_{*} \in R^{M \times N}$ and $W_{*} \in R^{N \times M}$ are the optimal factors that can minimize the unmodeled dynamics error ∈_k, tanh(⋅) is the hyperbolic tangent map. In the case of two independent variables, a map has the next Taylor series: $\begin{matrix} \hat{H} (ϑ_{1}, ϑ_{2}) = H (ϑ_{1^{0}}, ϑ_{2^{0}}) \\ + (ϑ_{1} - ϑ_{1^{0}}) \frac{\partial \hat{H} (ϑ_{1}, ϑ_{2})}{\partial ϑ_{1}} \\ + (ϑ_{2} - ϑ_{2^{0}}) \frac{\partial \hat{H} (ϑ_{1}, ϑ_{2})}{\partial ϑ_{2}} + r_{k} \end{matrix}$ (5) where $r_{k} \in R^{N \times 1}$ is the remainder of the Taylor series. ϑ₁ and ϑ₂ correspond to $V_{k} \in R^{M \times N}$ and $W_{k} \in R^{N \times M}$ , ϑ_1⁰ and ϑ_2⁰ correspond to $V_{*} \in R^{M \times N}$ and $W_{*} \in R^{N \times M}$ , $\hat{H} (ϑ_{1}, ϑ_{2})$ and H (ϑ_1⁰, ϑ_2⁰) correspond to ${\hat{H}}_{k}$ and H_k, represent ${\tilde{V}}_{k} = V_{k} - V_{*} \in R^{M \times N}$ and ${\tilde{W}}_{k} = W_{k} - W_{*} \in R^{N \times M}$ ; consequently, the Taylor series is applied to linearize (2): $\begin{matrix} W_{k} Φ_{k} = W_{*} Φ_{* k} + {\tilde{W}}_{k} \frac{\partial W_{k} Φ_{k}}{\partial W_{k}} \\ + {\tilde{V}}_{k}^{T} \frac{\partial W_{k} Φ_{k}}{\partial V_{k}} + r_{k} \end{matrix}$ (6) Taking into account (2) and the chain rule yields the second Jacobian: $\begin{matrix} \frac{\partial W_{k} Φ_{k}}{\partial V_{k}} = W_{k} \frac{\partial Φ_{k}}{\partial V_{k}} = W_{k} \frac{\partial \tanh (V_{j, k} D_{k})}{\partial V_{k}} \\ = Δ_{k} W_{k}^{T} D_{k} = Σ_{k} \end{matrix}$ (7) where $Σ_{k} \in R^{M \times 1}$ , $Δ_{k} = diag (δ_{j, k}) \in R^{M \times M}$ and $δ_{j, k} = {sech}^{2} (V_{j, k} D_{k}) \in ℜ$ , sech(⋅) is the hyperbolic secant map. Subsequently, the first Jacobian is: $\frac{\partial W_{k} Φ_{k}}{\partial W_{k}} = Φ_{k}$ (8) where $Φ_{k} = {[φ_{1, k}, \dots, φ_{j, k}, \dots, φ_{M, k}]}^{T} \in R^{M \times 1}$ , and $φ_{j, k} = tanh (V_{j, k} D_{k}) \in R$ . Replacing the Jacobians $\frac{\partial W_{k} Φ_{k}}{\partial V_{k}}$ of (7), and $\frac{\partial W_{k} Φ_{k}}{\partial W_{k}}$ of (8) into (6): $\begin{matrix} W_{k} Φ_{k} = W_{*} Φ_{* k} + {\tilde{W}}_{k} Φ_{k} \\ + {\tilde{V}}_{k}^{T} Σ_{k} + r_{k} \end{matrix}$ (9)

Replacing (2) and (4) into (9): $\begin{matrix} {\hat{D}}_{k + 1} = D_{k + 1} - \in_{k} + {\tilde{W}}_{k} Φ_{k} \\ + {\tilde{V}}_{k} Σ_{k} + r_{k} \\ \Rightarrow {\hat{D}}_{k + 1} - D_{k + 1} = {\tilde{W}}_{k} Φ_{k} \\ + {\tilde{V}}_{k} Σ_{k} + r_{k} - \in_{k} \end{matrix}$ (10) and taking into account (3) in (10): ${\tilde{D}}_{k + 1} = {\tilde{W}}_{k} Φ_{k} + {\tilde{V}}_{k}^{T} Σ_{k} + μ_{k}$ (11) where μ_k = r_k - ∈ _k is the unmodeled dynamics error.

From (11), the next modeling dynamic equality can be gotten: $\begin{matrix} {\tilde{D}}_{k + 1} = {\tilde{Ξ}}_{k}^{T} ψ_{k} + μ_{k} \\ {\tilde{Ξ}}_{k}^{T} = [\begin{matrix} {\tilde{W}}_{k} & {\tilde{V}}_{k}^{T} \end{matrix}] \\ ψ_{k} = {[\begin{matrix} Φ_{k} & Σ_{k} \end{matrix}]}^{T} \end{matrix}$ (12) where $ψ_{k} \in R^{2 M \times 1}$ , ${\tilde{Ξ}}_{k}^{T} \in R^{N \times 2 M}$ .

In the subsequent two sections, two alternative schemes for the updating of the neural network will be represented: the extended Kalman filter and argument Kalman filter.

3 The extended Kalman filter

The extended Kalman filter is developed in this section for the updating of a neural network.

From (12), the states space model is: $\begin{matrix} Ξ_{k + 1} = Ξ_{k} + η_{k} \\ D_{k + 1} = Ξ_{k}^{T} ψ_{k} + μ_{k} \end{matrix}$ (13)

with $\begin{matrix} Ξ_{k} = {[\begin{matrix} W_{k}^{T} & V_{k} \end{matrix}]}^{T} \\ ψ_{k} = {[\begin{matrix} Φ_{k} & Σ_{k} \end{matrix}]}^{T} \end{matrix}$ (14)

η_k $\in R^{2 M \times 2 M}$ and $μ_{k} \in R^{N \times 1}$ are uncorrelated noises.

Take into account the observer: $\begin{matrix} {\hat{Ξ}}_{k + 1} = {\hat{Ξ}}_{k} - G {\tilde{D}}_{k + 1}^{T} \\ {\hat{D}}_{k + 1} = {\hat{Ξ}}_{k}^{T} ψ_{k} \end{matrix}$ (15)

Represent the next states error: ${\tilde{Ξ}}_{k} = {\hat{Ξ}}_{k} - Ξ_{k}$ (16)

Taking into account (3), the next closed loop equality of the observer (15) applied to the model (13) is gotten by the subtraction of (13) to (15): $\begin{matrix} {\hat{Ξ}}_{k + 1} - Ξ_{k + 1} = {\hat{Ξ}}_{k} - Ξ_{k} \\ - η_{k} - G {[{\hat{D}}_{k + 1} - D_{k + 1}]}^{T} \\ \Rightarrow {\hat{Ξ}}_{k + 1} - Ξ_{k + 1} = {\hat{Ξ}}_{k} - Ξ_{k} \\ - η_{k} - G {[{\hat{Ξ}}_{k}^{T} ψ_{k} - Ξ_{k}^{T} ψ_{k} - μ_{k}]}^{T} \\ \Rightarrow [{\hat{Ξ}}_{k + 1} - Ξ_{k + 1}] = [{\hat{Ξ}}_{k} - Ξ_{k}] \\ - η_{k} - G [ψ_{k}^{T} {\hat{Ξ}}_{k} - ψ_{k}^{T} Ξ_{k} - μ_{k}^{T}] \\ \Rightarrow [{\hat{Ξ}}_{k + 1} - Ξ_{k + 1}] = [{\hat{Ξ}}_{k} - Ξ_{k}] \\ - η_{k} - G ψ_{k}^{T} [{\hat{Ξ}}_{k} - Ξ_{k}] + G μ_{k}^{T} \\ \Rightarrow [{\hat{Ξ}}_{k + 1} - Ξ_{k + 1}] = - η_{k} + G μ_{k}^{T} \\ + (I - G ψ_{k}^{T}) [{\hat{Ξ}}_{k} - Ξ_{k}] \end{matrix}$ (17)

Tacking into account (16), the next closed loop equality (17) can be represented: ${\tilde{Ξ}}_{k + 1} = (I - G ψ_{k}^{T}) {\tilde{Ξ}}_{k} - η_{k} + G μ_{k}^{T}$ (18)

C1: It is taken into account: $\begin{matrix} E {η_{k}} = 0 \\ E {η_{k} η_{k}^{T}} = R_{1} \\ E {μ_{k}} = 0 \\ E {μ_{k}^{T} μ_{k}} = r_{2} \\ cross - terms ({\tilde{Ξ}}_{k}, μ_{k}, η_{k}) = 0 \end{matrix}$ (19)

where E{ · } is the mathematical expectation.

Theorem 1. Taking into account C1 of (19), the extended Kalman filter employed as the updating map of the neural network (2) for the modeling of Takagi-Sugeno fuzzy models (1) is: $\begin{array}{l} {\hat{Ξ}}_{k + 1} = {\hat{Ξ}}_{k} - \frac{1}{r (k)} Q_{k} ψ_{k} {\tilde{D}}_{k + 1}^{T} \\ Q_{k + 1} = Q_{k} - \frac{1}{r (k)} Q_{k} ψ_{k} ψ_{k}^{T} Q_{k} + R_{1} \\ r (k) = r_{2} + ψ_{k}^{T} Q_{k} ψ_{k} \end{array}$ (20)

with $\begin{array}{l} {\hat{Ξ}}_{k} = {[{\hat{W}}_{k}^{T} \hat{V}]}^{T} \\ ψ_{k} = [Φ_{k} Σ_{k}] \end{array}$ (21) $\begin{matrix} Σ_{k} = Δ_{k} V_{k}^{T} D_{k} \\ Δ_{k} = diag (δ_{j, k}) \\ δ_{j, k} = {sech}^{2} ({\hat{V}}_{j, k} D_{k}) \end{matrix}$ (22) $\begin{matrix} Φ_{k} = {[φ_{1, k}, \dots, φ_{j, k}, \dots, φ_{M, k}]}^{T} \\ φ_{j, k} = tanh ({\hat{V}}_{j, k} D_{k}) \end{matrix}$ (23) $0 < r_{2} \in R$ is a forgetting factor, ${\tilde{D}}_{k + 1} \in R^{N \times 1}$ is the modeling error of (3), ${\hat{V}}_{k} \in R^{M \times N}$ and ${\hat{W}}_{k} \in R^{N \times M}$ are the hidden layer and output terms of (2), ${\hat{Ξ}}_{k} \in R^{2 M \times N}$ , $ψ_{k} \in R^{2 M \times 1}$ , $Σ_{k} \in R^{M \times 1}$ is the second Jacobian, $Δ_{k} \in R^{M \times M}$ , $δ_{j, k} \in R$ , $Φ_{k} \in R^{M \times 1}$ is the first Jacobian, $φ_{j, k} \in R$ . $Q_{k + 1} \in R^{2 M \times 2 M}$ is the gain which is a positive definite covariance matrix, Q₁ = q₁I is the initial gain, and q₁ > 0 is a scalar constant commonly big enough to ensure an admissible convergence, and $I \in R^{2 M \times 2 M}$ is the identity matrix. R₁ = r₁I, $0 < r_{1} \in R$ , tanh(⋅) is the hyperbolic tangent map and sech(⋅) is the hyperbolic secant map.

Proof 1. Represent the next gain Q_k: $Q_{k} = E {{\tilde{Ξ}}_{k} {\tilde{Ξ}}_{k}^{T}}$ (24) where E{ · } is the mathematical expectation. Taking into account the closed loop equality (18), the subsequent gain Q_k+1 is gotten: $\begin{matrix} Q_{k + 1} = E {{\tilde{Ξ}}_{k + 1} {\tilde{Ξ}}_{k + 1}^{T}} \\ = E {[(I - G ψ_{k}^{T}) {\tilde{Ξ}}_{k} - η_{k} + G μ_{k}^{T}] \\ * {[(I - G ψ_{k}^{T}) {\tilde{Ξ}}_{k} - η_{k} + G μ_{k}^{T}]}^{T}} \\ = [I - G ψ_{k}^{T}] E {{\tilde{Ξ}}_{k} {\tilde{Ξ}}_{k}^{T}} {[I - G ψ_{k}^{T}]}^{T} \\ + E {η_{k} η_{k}^{T}} + G E {μ_{k}^{T} μ_{k}} G^{T} \\ + cross-terms ({\tilde{Ξ}}_{k}, μ_{k}, η_{k}) \end{matrix}$ (25)

Replacing (19) and (24) in the last equality of (25): $\begin{matrix} Q_{k + 1} = [I - G ψ_{k}^{T}] Q_{k} {[I - G ψ_{k}^{T}]}^{T} \\ + R_{1} + {Gr}_{2} G^{T} \\ = Q_{k} - G ψ_{k}^{T} Q_{k} - Q_{k} ψ_{k}^{T} G^{T} \\ + K ψ_{k}^{T} Q_{k} ψ_{k} G^{T} + R_{1} + {Gr}_{2} G^{T} \\ = Q_{k} + R_{1} \\ + G (r_{2} + ψ_{k}^{T} Q_{k} ψ_{k}) G^{T} \\ - G ψ_{k}^{T} Q_{k} - Q_{k} ψ_{k} G^{T} \end{matrix}$ (26)

Adding and subtracting $Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} ψ_{k}^{T} Q_{k}$ to the right side of (26): $\begin{matrix} Q_{k + 1} = Q_{k} + R_{1} \\ + G (r_{2} + ψ_{k}^{T} Q_{k} ψ_{k}) G^{T} \\ - G ψ_{k}^{T} Q_{k} - Q_{k} ψ_{k} G^{T} \\ + Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} ψ_{k}^{T} Q_{k} \\ - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} ψ_{k}^{T} Q_{k} \\ = Q_{k} + R_{1} \\ - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} ψ_{k}^{T} Q_{k} \\ + [G - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1}] \\ * (r_{2} + ψ_{k}^{T} Q_{k} ψ_{k}) \\ * {[G - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1}]}^{T} \end{matrix}$ (27)

From the definition of Q_k in (24), the minimum value of Qk + 1 is desired to assure the convergence to a minimum value of the factors error ${\tilde{Ξ}}_{k + 1}$ , it is gotten by taking into account the smallest value in the last term of (27): $\begin{array}{l} G - Q_{k ψ k} {(r_{2} + ψ_{k}^{T} Q_{k ψ k})}^{- 1} = 0 \\ \Rightarrow G = Q_{k ψ k} {(r_{2} + ψ_{k}^{T} Q_{k ψ k})}^{- 1} \end{array}$ (28)

Replacing G of (28) into the last equality of (27): $\begin{matrix} Q_{k + 1} = Q_{k} + R_{1} \\ - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} ψ_{k}^{T} Q_{k} \end{matrix}$ (29)

It is the second equality of (20) by taking into account $r (k) = r_{2} + ψ_{k}^{T} Q_{k} ψ_{k}$ . Now, replacing G of (28) into the observer of the first equality of (15): $\begin{matrix} {\hat{Ξ}}_{k + 1} = {\hat{Ξ}}_{k} \\ - Q_{k} ψ_{k} {(r_{2} + ψ_{k}^{T} Q_{k} ψ_{k})}^{- 1} {\tilde{D}}_{k + 1}^{T} \end{matrix}$ (30)

It is the first equality of (20) by taking into account $r (k) = r_{2} + ψ_{k}^{T} Q_{k} ψ_{k}$ . Then, the result is proved.

4 The argument Kalman filter

The argument Kalman filter is developed in this section for the updating of a neural network. The difference between the argument Kalman filter and extended Kalman filter is that the extended Kalman filter uses the full Jacobian, while the argument Kalman filter only uses the Jacobian argument instead of the full Jacobian. After, the Lyapunov approach of [16, 18, 20] is employed to analyze the gain stability in the argument Kalman filter. And finally, the computational complexity comparison between the extended Kalman filter and argument Kalman filter will be analyzed.

4.1 Approach

The argument Kalman filter developed as the updating map of the neural network (2) for the modeling of Takagi-Sugeno fuzzy models (1) is: $\begin{array}{l} {\hat{Ξ}}_{k + 1} = {\hat{Ξ}}_{k} - \frac{1}{r (k)} Q_{k} {\hat{ψ}}_{k} {\tilde{D}}_{k + 1}^{T} \\ Q_{k + 1} = Q_{k} - \frac{1}{r (k)} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} + R_{1} \\ r (k) = r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k} \end{array}$ (31) with $\begin{matrix} {\hat{Ξ}}_{k} = {[\begin{matrix} {\hat{W}}_{k}^{T} & {\hat{V}}_{k} \end{matrix}]}^{T} \\ {\hat{ψ}}_{k} = {[\begin{matrix} {\hat{Φ}}_{k} & {\hat{Σ}}_{k} \end{matrix}]}^{T} \end{matrix}$ (32) $\begin{matrix} {\hat{Σ}}_{k} = {[{\hat{δ}}_{1, k}, \dots, {\hat{δ}}_{j, k}, \dots, {\hat{δ}}_{M, k}]}^{T} \\ {\hat{δ}}_{j, k} = {\hat{V}}_{j, k} D_{k} \end{matrix}$ (33) $\begin{matrix} {\hat{Φ}}_{k} = {[{\hat{φ}}_{1, k}, \dots, {\hat{φ}}_{j, k}, \dots, {\hat{φ}}_{M, k}]}^{T} \\ {\hat{φ}}_{j, k} = {\hat{V}}_{j, k} D_{k} \end{matrix}$ (34) $0 < r_{2} \in R$ is a forgetting factor, ${\tilde{D}}_{k + 1} \in R^{N \times 1}$ is the modeling error of (3), ${\hat{V}}_{k} \in R^{M \times N}$ and ${\hat{W}}_{k} \in R^{N \times M}$ are the hidden layer and output terms of (2), ${\hat{Ξ}}_{k} \in R^{2 M \times N}$ , ${\hat{ψ}}_{k} \in R^{2 M \times 1}$ , ${\hat{Σ}}_{k} \in R^{M \times 1}$ is the second Jacobian argument, ${\hat{Φ}}_{k} \in R^{M \times 1}$ is the first Jacobian argument, ${\hat{φ}}_{j, k} \in R$ , ${\hat{δ}}_{j, k} \in R$ . $Q_{k + 1} \in R^{2 M \times 2 M}$ is the gain which is a positive definite covariance matrix, Q₁ = q₁I is the initial gain, and q₁ > 0 is a scalar constant commonly big enough to ensure an admissible convergence, and $I \in R^{2 M \times 2 M}$ is the identity matrix. R₁ = r₁I, $0 < r_{1} \in R$ .

Remark 2. The difference between the extended Kalman filter and the argument Kalman filter is in (22), (23) and (33), (34). The extended Kalman filter employs the full Jacobian (22), (23) which has high computational complexity, while the argument Kalman filter only employs the Jacobian argument (33), (34) which has less computational complexity. This fact, produces two results: first, it yields the argument Kalman filter smaller in size than the extended Kalman filter, and second, it yields the argument Kalman filter more suitable for a stabilityanalysis.

Remark 3. It is better to employ the Jacobian argument than to employ the full Jacobian in the Kalman filter because employing the full Jacobian is a first order approximation of a nonlinear function producing the unmodeled dynamics error μ_k of (11), while employing the Jacobian argument avoids the unmodeled dynamics error μ_k. Thus, the argument Kalman filter is more precise than the extended Kalmanfilter.

Remark 4. There exists works as [3, 15] where the full Jacobian is employed to get a coincidence in the matrices dimensions between the Kalman filter and neural network. Since in this article, the matrices of the Kalman filter and neural network have the same dimension, the Jacobian argument is employed instead of the full Jacobian.

4.2 Stability

The stability and convergence of the gain in the argument Kalman filter are analyzed by the next Theorem.

Theorem 2. The gain of the argument Kalman filter (31) for the updating of the neural network (2), (3) is uniformly stable and the next convergence is satisfied: $\underset{T \to \infty}{lim sup} \frac{1}{T} \sum_{k = 1}^{T} s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s = r_{1}$ (35) with s = [1, 1 …, 1] ^T, Q_k is the gain of the argument Kalman filter expressed in (31), R₁ = r₁I and r₂ are expressed in (31).

Proof 2. Chose the next Lyapunov function: $S_{k} = s^{T} Q_{k} s$ (36) getting ΔS_k: $Δ S_{k} = S_{k + 1} - S_{k} = s^{T} Q_{k + 1} s - s^{T} Q_{k} s$ (37) Taking into account (31) and that s^Ts = 1: $\begin{matrix} c Δ L_{k} = s^{T} Q_{k + 1} s - s^{T} Q_{k} s \\ = s^{T} [Q_{k} - \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} + r_{1} I] s - s^{T} Q_{k} s \\ = s^{T} Q_{k} s - s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s + s^{T} r_{1} s - s^{T} Q_{k} s \\ Δ L_{k} = - s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s + r_{1} \end{matrix}$ (38) The next result is gotten: $Δ L_{k} = - s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s + r_{1}$ (39) since $s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s \geq 0$ and r₁ is small and positive, the gain of the approach is uniformly stable. Now, summarize (39) from 1 to T: $\sum_{k = 1}^{T} s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s = S_{1} - S_{T} + T r_{1}$ (40) multiplying by $\frac{1}{T}$ and getting the limit: $\underset{T \to \infty}{\lim \sup} \frac{1}{T} \sum_{k = 1}^{T} s^{T} \frac{1}{r_{2} + {\hat{ψ}}_{k}^{T} Q_{k} {\hat{ψ}}_{k}} Q_{k} {\hat{ψ}}_{k} {\hat{ψ}}_{k}^{T} Q_{k} s = r_{1}$ (41)

4.3 Computational complexity

The equations to be compared in the other Kalman filters are (22) and (23) where: $\begin{matrix} ϕ_{j, k} = \tanh ({\hat{V}}_{j, k} D_{k}) = \frac{e^{{\hat{V}}_{j, k} D_{k}} - e^{- {\hat{V}}_{j, k} D_{k}}}{e^{{\hat{V}}_{j, k} D_{k}} + e^{- {\hat{V}}_{j, k} D_{k}}} \\ δ_{j, k} = {sech}^{2} ({\hat{V}}_{j, k} D_{k}) = \frac{1}{{(e^{{\hat{V}}_{j, k} D_{k}} + e^{- {\hat{V}}_{j, k} D_{k}})}^{2}} \\ j = \bar{1, M} \end{matrix}$ (42) where M is the neurons number in the hidden layer of the neural network (2), (3).

The equations to be compared in the argument Kalman filter are (33) and (34) where: $\begin{matrix} {\hat{ϕ}}_{j, k} = {\hat{V}}_{j, k} D_{k} \\ {\hat{δ}}_{j, k} = {\hat{V}}_{j, k} D_{k} \\ j = \bar{1, M} \end{matrix}$ (43)

In the Table 1, the comparison results for the computational complexity between the argument Kalman filter of equation (43) and the other Kalman filters of equation (42) are described, where the argument Kalman filter is denoted as AKF, the other Kalman filters are denoted as OKF, the multiplications number are denoted as m, the algebraic additions number are denoted as a, and the total operations number is denoted as To.

Table 1

Computational complexity

AKF	a	m	OKF	a	m
${\hat{φ}}_{j, k}$	0	M	φ _j,k	2M	5M
${\hat{δ}}_{j, k}$	0	M	δ _j,k	2M	5M
To	0	2M	To	4M	10M

From Table 1, it can be observed for the other Kalman filters the algebraic additions number is 4M and the multiplications number is 10M, and for the argument Kalman filter the algebraic additions number is 0 and the multiplications number is 2M. Then, the argument Kalman filter has less computational complexity than the other Kalman filters because the first has less operations number.

Fig.2

The training for the example 1.

Fig.3

The testing for the example 1.

5 Steps of the argument Kalman filter

The steps of the argument Kalman filter for the updating of a neural network are:

The Takagi-Sugeno fuzzy model output D_k+1 is gotten with (1). The Takagi-Sugeno fuzzy model should has the form represented by (1); the term N is chosen in concordance with the inputs number in this Takagi-Sugeno fuzzy model.

Take into account the elements; chose terms ${\hat{W}}_{1}$ and ${\hat{V}}_{1}$ for (2) as random numbers between 0 and 1; chose the number of hidden layer neurons M for (2) with an integer value, chose the initial gain q₁ with a positive value, and elements r₁ and r₂ for (31) with positive values.

For each iteration k, the neural network output ${\hat{D}}_{k + 1}$ is gotten with (2), the modeling error ${\tilde{D}}_{k + 1}$ is gotten with (3), ${\hat{Ξ}}_{k}$ is gotten with terms ${\hat{W}}_{k}$ and ${\hat{V}}_{k}$ employing (32), ${\hat{ψ}}_{k}$ is gotten with the Jacobians arguments ${\hat{Σ}}_{k}$ and ${\hat{Φ}}_{k}$ employing (32), (33), (34), the term ${\hat{Ξ}}_{k + 1}$ is updated with (31), terms of the neural network ${\hat{W}}_{k + 1}$ and ${\hat{V}}_{k + 1}$ with ${\hat{Ξ}}_{k + 1}$ are updated exploiting (32).

The behavior of the scheme could be altered by the selection of different values in the elements M ∈ [N, 5N], q₁ ∈ [1 × 10⁰, 1 × 10⁴], r₁ ∈ [5 × 10^-5, 5 × 10^-1], or r₂ ∈ [1 × 10^-2, 5 × 10^-1]. The mentioned intervals are chosen based on that they yield the best modeling of the recommended approach.

6 Simulations

In this part of the article, the exposed scheme is applied for the modeling of two examples. The two examples have the two main characteristics: first, they are Takagi-Sugeno fuzzy models with the structure of equation (1), second, they let to show the advantages of the recommended approach. In all cases, the argument Kalman filter will be compared with the decoupled Kalman filter of [24, 32], the unscented Kalman filter of [6, 30], and the extended Kalman filter of [15, 26]. The differences between the schemes were expressed in the past sections. The goal of the Kalman filters is the fast updating of a neural network, it can produce that the Kalman filters with neural networks reach faster the signal of the Takagi-Sugeno fuzzy models. The next root mean square error denoted as MSE is employed for comparisons [31]: $M S E = {(\frac{1}{O} \sum_{k = 1}^{O} {\tilde{D}}_{k + 1}^{T} {\tilde{D}}_{k + 1})}^{\frac{1}{2}}$ (44) with ${\tilde{D}}_{k + 1}$ as the modeling error of (3), O is the final iteration. One run with a training stage and a testing stage is employed to compute the MSE.

6.1 The example 1

The Takagi-Sugeno fuzzy model of the example 1 is: $\begin{matrix} D_{k + 1} = h (D_{k}) D_{k} \\ h (D_{k}) = [\begin{matrix} [c] cc h_{11, k} & h_{12, k} \\ h_{21, k} & h_{22, k} \end{matrix}] \end{matrix}$ (45) with $\begin{matrix} h_{11, k} = \frac{0.1 sech (D_{1, k})}{0.4 \tanh^{2} (D_{1, k} + D_{2, k})} + 0.8 rand D_{2, k} \\ h_{22, k} = \frac{0.1 sech (D_{2, k})}{0.4 \tanh^{2} (D_{1, k} + D_{2, k})} + 0.8 rand D_{2, k} \\ h_{12, k} = h_{21, k} = 0 \end{matrix}$

The Takagi-Sugeno fuzzy model of (1), (45) is developed where the inputs are D_1,k = D_1,k, D_2,k = D_2,k, the outputs are D_1,k+1 = D_1,k+1, D_2,k+1 = D_2,k+1, and the initial conditions are D_1,1 = D_2,1 = 0.5, rand is a random number between 0 and 1, sech(⋅) and tanh(⋅) are complex maps. The data of 2000 iterations is developed for the training stage and the data of the least 200 iterations is developed for the testing stage.

The decoupled Kalman filter of [24, 32] is represented with elements N = 2, M = 2, q₁ = 1 ×10¹, r₁ = 1 ×10^-2, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

The unscented Kalman filter of [15, 26] is represented with elements N = 2, M = 2, q₁ = 1 ×10¹, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

The extended Kalman filter of [24, 32] is represented as (2), (3), (20)-(23) with elements N = 2, M = 2, q₁ = 1 ×10¹, r₁ = 1 ×10^-2, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

The argument Kalman filter of this article is represented as (2), (3), (31)-(34) with elements N = 2, M = 2, q₁ = 1 ×10¹, r₁ = 3 ×10^-2, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

Figures 2 and 3 show the comparisons for the training and testing stages of the decoupled Kalman filter, unscented Kalman filter, extended Kalman filter, and argument Kalman filter. The training and testing MSE comparisons of (44) are shown in Table 2.

Table 2

MSE Comparisons for the example 1

Schemes	Training	Testing
Decoupled Kalmanfilter	0.2990	0.0944
Unscented Kalmanfilter	0.0337	0.0062
Extended Kalmanfilter	0.0269	0.0084
Argument Kalmanfilter	0.0269	0.0059

From Figures 2 and 3, it is observed that the argument Kalman filter upgrades the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter because the signal of the first reaches faster the signal of the Takagi-Sugeno fuzzy model than the signal of the others. From Table 2, it is observed that the argument Kalman filter achieves better simulation results when compared with the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter because the MSE is smaller for the first. Then, the argument Kalman filter is the best selection for the modeling in the example 1.

Fig.4

The training for the example 2.

6.2 The example 2

The Takagi-Sugeno fuzzy model of the example 2 is: $\begin{matrix} D_{k + 1} = h (D_{k}) D_{k} \\ h (D_{k}) = [\begin{matrix} h_{11, k} & h_{12, k} \\ h_{21, k} & h_{22, k} \end{matrix}] \end{matrix}$ (46) with $\begin{matrix} h_{11, k} = \frac{0.2 rand + 0.5 \sin (π D_{2, k})}{0.2 rand + 0.1 \cos (π D_{1, k}) + D_{1, k}^{2} + D_{2, k}^{2}} \\ h_{22, k} = \frac{0.2 rand + 0.5 \sin (π D_{1, k})}{0.2 rand + 0.1 \cos (π D_{2, k}) + D_{1, k}^{2} + D_{2, k}^{2}} \\ h_{12, k} = h_{21, k} = 0 \end{matrix}$

The Takagi-Sugeno fuzzy model of (1), (46) is developed where the inputs are D_1,k = D_1,k, D_2,k = D_2,k, the outputs are D_1,k+1 = D_1,k+1, D_2,k+1 = D_2,k+1, and the initial conditions are D_1,1 = D_2,1 = 0.5, rand is a random number between 0 and 1, sin(⋅) and cos(⋅) are complex maps. The data of 2000 iterations is developed for the training stage and the data of the least 200 iterations is developed for the testing stage.

The unscented Kalman filter of [6, 30] is represented with elements N = 2, M = 2, q₁ = 1 ×10¹, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

The extended Kalman filter of [15, 26] is represented as (2), (3), (20)-(23) with elements N = 2, M = 2, q₁ = 1 ×10¹, r₁ = 1 ×10^-2, r₂ = 0.2, $W_{1} = diag (W_{i, 1}) \in R^{2 \times 2}$ , $V_{1} = diag (V_{i, 1}) \in R^{2 \times 2}$ , W_i,1 = rand, V_i,1 = rand, rand is a random number between 0 and 1.

Figures 4 and 5 show the comparisons for the training and testing stages of the decoupled Kalman filter, unscented Kalman filter, extended Kalman filter, and argument Kalman filter. The training and testing MSE comparisons of (44) are shown in Table 3.

Fig.5

The training for the example 2.

From Figures 4 and 5, it is observed that the argument Kalman filter upgrades the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter because the signal of the first reaches faster the signal of the Takagi-Sugeno fuzzy model than the signal of the second. From Table 3, it is observed that the argument Kalman filter achieves better simulation results when compared with the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter because the MSE is smaller for the first. Then, the argument Kalman filter is the best selection for the modeling in the example 2.

Remark 5. From examples 1 and 2, it can be observed the MSE for the testing stage is around the 50% of the MSE for the training stage. It is because the training stage is the starting of the updating in the neural network, the MSE starts with a big value, this MSE is decreasing through the time. After, the testing stage starts with a small value which is maintained.

Table 3

MSE Comparisons for the example 2

Schemes	Training	Testing
Decoupled KalmanFilter	0.4998	0.1562
Unscented KalmanFilter	0.1149	0.0300
Extended KalmanFilter	0.1142	0.0579
Argument KalmanFilter	0.1129	0.0290

7 Conclusion

In this study, a argument Kalman filter was recommended for the fast updating of a neural network. The develop of the exposed scheme was expressed based on the exploitation of the Taylor series and mathematical expectation approaches. From simulations, it was observed that the commented scheme achieves better results when compared with the decoupled Kalman filter, unscented Kalman filter, and extended Kalman filter for the modeling of two Takagi-Sugeno fuzzy models. The recommended scheme could be employed as the fast updating of a fuzzy model or an evolving intelligent model. In the future studies, the recommended approach will be applied to real world models, or it will be developed for the fault detection and diagnosis, or the Baysian filters will be studied to be employed instead of the full Jacobians.

Footnotes

14

Authors are grateful to the editors and the reviewers for their valuable comments. Authors thank the Instituto Politécnico Nacional, the Secretaría de Investigación y Posgrado, the Comisión de Operación y Fomento de Actividades Académicas, and Consejo Nacional de Ciencia y Tecnología for their help in this study. The second author acknowledges the support of the Austrian COMET-K2 programme of the Linz Center of Mechatronics (LCM), funded by the Austrian federal government and the federal state of Upper Austria. This publication reflects only the authors’ views.

References

Abbasi

, Akbari

, Rahmani

and Askari

M.R.

, Optimal placement of distributed generation in radial networks considering reliability and cost indices, Journal of Intelligent & Fuzzy Systems 30(2) (2016), 1077–1086.

de la Cruz-Alejo

, Antonio-Mendez

and Salazar-Pereyra

, Fuzzy logic control on fpga for two axes solar tracking, Neural Computing & Applications (2017). 10.1007/s00521-017-3207-1.

Haykin

, Neural networks: A comprehensive foundation, Prentice-Hall, Second Edition, ISBN: 0-13-273350-1, 1999.

Iglesias

J.A.

, Tiemblo

, Ledezma

and Sanchis

, Web news mining in an evolving framework, Information Fusion 28 (2016), 90–98.

Kalman

R.E.

, A new approach to linear filtering and prediction problems, Transaction of the ASME, Journal of Basic Engineering 82 (1960), 35–45.

Kontorovich

V.Y.

, Lovtchikova

, Meda-Campaña

J.A.

and Tinsley

, Nonlinear filtering algorithms for chaotic signals: A comparative study, ISAST Transactions on Computers and Intelligent Systems 1(2) (2010), 34–43.

Leite

, Ballini

, Costa

and Gomide

, Evolving fuzzy granular modeling from nonstationary fuzzy data streams, Evolving Systems 3 (2012), 65–79.

Leite

D.F.

, Hell

M.B.

, Costa

Jr and Gomide

, Real-time fault diagnosis of nonlinear systems,, Nonlinear Analysis 71 (2009), e2665–e2673.

Lughofer

, Pratama

and Skrjanc

, Incremental rule splitting in generalized evolving fuzzy systems for autonomous drift compensation, IEEE Transactions on Fuzzy Systems (2017). DOI: 10.1109/TFUZZ.2017.2753727.

10.

Lughofer

, Single-pass active learning with conflict and ignorance, Evolving Systems 3(4) (2012), 251–271.

11.

Lughofer

, Weigl

, Heidl

, Eitzinger

and Radauer

, Recognizing input space and target concept drifts in data streams with scarcely labeled and unlabelled instances, Information Sciences 355-356 (2016), 127–151.

12.

Mandal

, Sahoo

, Ghorai

and Pal

, Genus value of mpolar fuzzy graphs, Journal of Intelligent & Fuzzy Systems 34(3) (2018), 1947–1957.

13.

Maciel

, Gomide

and Ballini

, A differential evolution algorithm for yield curve estimation, Mathematics and Computers in Simulation 129 (2016), 10–30.

14.

Mohamad

, Sayed-Mouchaweh

and Bouchachia

, Active learning for classifying data streams with unknown number of classes, Neural Networks 98 (2018), 1–15.

15.

Paramo-Carranza

L.A.

, Meda-Campaña

J.A.

, de Rubio

J.J.

, Tapia Herrera

, Curtidor Lopez

A.V.

, Grande Meza

, and Cázares-Ramirez

, Discrete-time kalman filter for takagi-sugeno fuzzy models, Evolving Systems 8(3) (2017), 211–219.

16.

Pan

and Yu

, Composite learning from adaptive dynamic surface control, IEEE Transactions on Automatic Control 61(9) (2016), 2603–2609.

17.

Pan

and Yu

, Biomimetic hybrid feedback feedforward neural-network learning control, IEEE Transactions on Neural Networks and Learning Systems 28(6) (2017), 1481–1487.

18.

Pan

, Guo

, Li

and Yu

, Output-feedback adaptive neural control of a compliant differential SMA actuator, IEEE Transactions on Control Systems Technology 25(6) (2017), 2202–2210.

19.

Pan

, Liu

, Xu

and Yu

, Hybrid feedback feedforward: An efficient design of adaptive neural network control, Neural Networks 76 (2016), 122–134.

20.

Pan

, Sun

and Yu

, Composite adaptive dynamic surface control using online recorded data, International Journal of Robust and Nonlinear Control 26(18) (2016), 3921–3936.

21.

Pratama

, Dimla

, Lai

C.Y.

and Lughofer

, Metacognitive learning approach for online tool condition monitoring, Journal of Intelligent Manufacturing (2017). DOI: 10.1007/s10845-017-1348-9.

22.

Pratama

, Lughofer

, Er

M.J.

, Anavatti

and Lim

C.P.

, Data driven modelling based on recurrent interval-valued metacognitive scaffolding fuzzy neural network, Neurocomputing 262 (2017), 4–27.

23.

Pratama

, Angelov

P.P.

, Lughofer

and Er

M.J.

, Parsimonious random vector functional link network for data streams, Information Sciences 430–431 (2018), 519–537.

24.

Puskorius

G.V.

and Feldkamp

L.A.

, Decoupled extended Kalman filter training of feedforward layered networks, International Joint Conference on Neural Networks (1991), 771–777.

25.

Rakovic

, Anil

, Mihajlovic

, Savic

, Naik

, Borovac

and Gottscheber

, Fuzzy position-velocity control of underactuated finger of FTN robot hand, Journal of Intelligent & Fuzzy Systems 34(4) (2018), 2723–2736.

26.

Rubio

J.J.

, Stable Kalman filter and neural network for the chaotic systems identification, Journal of the Franklin Institute 354(16) (2017), 7444–7462.

27.

Rubio

J.J.

, Least square neural network model of the crude oil blending process, Neural Networks 78 (2016), 88–96.

28.

Skrjanc

, Andonovski

, Ledezma

, Sipele

, Iglesias

J.A.

and Sanchis

, Evolving cloud-based system for the recognition of drivers actions, Expert Systems With Applications 99 (2018), 231–238.

29.

Venkatesan

, Er

M.J.

, Dave

, Pratama

and Wu

, A novel online multi-label classifier for high-speed streaming data applications, Evolving Systems 8 (2017), 303–315.

30.

Wan

E.A.

and Merwe

R.V.D.

, The unscented Kalman filter for nonlinear estimation, The IEEE Adaptive Systems for Signal Processing, Communications, and Control Symposium, 2000, pp. 153–158.

31.

Wang

, Er

M.J.

and Han

, Parsimonious extreme learning machine using recursive orthogonal least squares, IEEE Transactions on Neural Networks and Learning Systems 25(10) (2014), 1828–1841.

32.

Wang

, Er

M.J.

and Han

, Generalized single-hidden layer feedforward networks for regression problems, IEEE Transactions on Neural Networks and Learning Systems 26(6) (2015), 1161–1176.