A fast learning method for data-driven design of interval type-2 fuzzy logic system

Abstract

In this study, we propose a novel fast learning data-driven method for the design of interval type-2 fuzzy logic system (IT2FLS). In order to accelerate the learning speed, we present a parallel mechanism to generate the antecedents of the IT2FLS and the least square method based learning algorithm to optimize the consequents. Firstly, driven by different sub-data sets, corresponding type-1 fuzzy logic systems (T1FLSs) which have the same initial fuzzy partition (thus the same initial fuzzy rule base) are parallelly obtained through the popular ANFIS method. Then, an ensembling strategy is proposed to form the type-2 fuzzy partition for each input variable through merging corresponding type-1 fuzzy sets (T1FSs) in the type-1 fuzzy partitions of the learned T1FLSs. By this strategy, the antecedents of the IT2FLS are determined and then fixed, however, the consequent parameters still need to be optimized. To achieve both excellent performance and fast training speed, a least square method based learning algorithm is provided for the optimization of the consequent parameters. Finally, three benchmark problems and one real-world application are given, and detailed comparisons with some well performed methods are made. Simulation and comparison results have verified the effectiveness and superiorities of the proposed method.

Keywords

Data-driven method fast learning fuzzy logic system ANFIS type-2 fuzzy

1 Introduction

As a popular extension of the conventional fuzzy logic system (type-1 fuzzy logic system: T1FLS), type-2 fuzzy logic system (T2FLS) nowadays is attracting more and more attentions [1 –6]. In the T2FLS, type-2 fuzzy sets (T2FSs) including interval T2FSs (IT2FSs) [7, 8] and generalized T2FSs (GT2FSs) [9, 10] are adopted to replace the type-1 fuzzy sets (T1FSs). Compared to T1FSs, T2FSs have more parameters and enhanced ability to handle high levels of uncertainties [1, 3]. Naturally, T2FLSs have shown better performance than T1FLSs in many research fields, especially in the modeling, prediction and control applications. Due to the complexity of the generalized T2FLSs, till now, the most widely used T2FLSs are the interval ones. In the interval T2FLSs (IT2FLSs), IT2FSs which have interval-valued membership grades are utilized, and the computational complexity can be greatly reduced.

Generally, two ways exist in designing satisfactory IT2FLSs. One is by the manual setting while the other is by data-driven methods. In the control applications, IT2FLSs used as controllers are usually designed by the first way [11]. In the prediction and modeling applications, IT2FLSs utilized as predictors or system models are often obtained by data-driven methods. For example, to train the IT2FLSs or interval type-2 fuzzy neural networks, different kinds of data-driven learning algorithms were proposed, such as the meta-cognitive sequential learning algorithm [12], the gradient descent algorithm [13, 14], the least squares algorithms [15, 16], the hybrid algorithm [17 –19], the vectorization-optimization-method [20], the multi-objective optimization method [21] and the genetic algorithm [22]. However, among all these studies, IT2FLSs are directly designed without using the popular design methods of T1FLSs. Some other works have tried to generate IT2FLS from T1FLS. In [23, 24], T1FLS was firstly designed, and then transformed to the type-2 case. In such studies, the trained T1FLS is used to initialize the antecedent and consequent parameters of the corresponding IT2FLS, and then such parameters are further optimized by learning algorithms. In this study, we present one different scheme for transforming several learned T1FLSs to one IT2FLS. In the proposed scheme, we firstly divide the whole data set to some sub-data sets, and for each sub-data set, one T1FLS will be generated by the popular ANFIS method [25, 26]. This process will be accomplished in parallel. The obtained different T1FLSs are then ensembled to form the antecedents of the IT2FLS. This is realized by the proposed ensembling strategy which can merge different type-1 fuzzy partitions into one corresponding interval type-2 fuzzy partition for each input variable.

On the other aspect, no matter which kind of method is used to design an IT2FLS, it is a time-consuming task due to its complex input-output mapping and its large number of parameters to be tuned, especially the parameters in the antecedent parts of type-2 fuzzy rules that are usually optimized by the steepest descent methods. One approach to solve this problem is to use the extreme learning method (ELM) [27, 28] which is based on the least square method that has fast learning speed and good performance. However, the ELM for IT2FLS randomly generates the parameters of the antecedent IT2FSs. Hence, this can not guarantee the coverage of the input domain, and the linguistic meanings will be lost. In this study, we utilize the above-mentioned strategy to generate the antecedent IT2FSs to solve such problems. And, the training processes of different T1FLSs for sub-data sets are done in parallel so as to accelerate the learning speed. Although the antecedents of the IT2FLS can be determined by the previous strategy, its consequent parameters still need to be learned to improve the performance. In the consequent parameter learning, a least square method based learning algorithm is presented in order to further fasten the training speed.

The main novelties and contributions of this study are listed as follows:

For the sub-data sets which are divided from the training data set, T1FLSs are generated in parallel by ANFIS method. The parallel mechanism can fasten the learning speed of the proposed data-driven method.

An ensembling strategy is proposed to form the type-2 fuzzy partition for each input variable through merging corresponding T1FSs in the type-1 fuzzy partitions of the learned T1FLSs. Detailed analysis has shown us how to ensemble different triangular T1FSs to corresponding trapezoidal IT2FS.

The consequent parameters of the IT2FLS is optimized by the least square method. As well known, the least square method can achieve both smallest training error and smallest scaled parameters which can assure the approximation and generalization performances of the IT2FLS. Also, the learning speed of the least square method is very fast.

Three benchmark problems including the prediction of the Mackey-Glass time series, the identification of one second-order time-varying system, the identification of one nonlinear plant, and one real-world wind speed prediction application are given. Detailed comparisons with some well performed methods, e.g. ANFIS [25, 26], BPNN [29, 30], RBF-AFS [31], DFNN [32], GEBF-OSFNN [33], TSCIT2FNN [34], TSK FNS [35], SANFIS [36] and Simpl-eTS [37] are made. Simulation and comparison results demonstrate that the proposed method not only has better performance but also has greatly reduced learning time.

The rest of this paper is organized as follows. In Section 2, preliminaries on IT2FS and IT2FLS will be briefly given. In Section 3, the proposed data-driven fast learning method will be presented in detail. In Section 4, simulation and comparisons will be made. At last, conclusions will be drawn in Section 5.

2 Preliminaries

In this section, we will briefly introduce the definition of IT2FS and the inference process of the IT2FLS.

2.1 IT2FS

The FS with crisp membership function (MF) is called T1FS while the one with fuzzy MF is named T2FS [1, 3]. Below, we formally introduce the definition of T2FS.

Definition of T2FS [1, 3]: A T2 FS, denoted as $\tilde{A}$ in universe of discourse X, is commonly characterized as

$\begin{matrix} \tilde{A} & = & \int_{X} μ_{\tilde{A}} (x) / x \\ = & \int_{X} \int_{u \in J_{x}} f_{x} (u) / u / x, J_{x} \subseteq [0, 1], \end{matrix}$ (1) where $μ_{\tilde{A}} (x)$ is the fuzzy MF grade of a generic element x, ∫∫ denotes union over all admissible x and u, f_x (u) is the secondary MF and J_x is the primary membership of x and the domain of the secondary MF.

When the fuzzy MF grade $μ_{\tilde{A}} (x)$ degrades to a crisp MF grade, T2FS becomes the well-known T1FS.

Nowadays, due to the complexity of the general T2FS, its special case – IT2FS is widely used as a substitute and has found lots of applications in many research fields. The MF grade of an IT2FS is an interval. One IT2FS $\tilde{A}$ can be completely described by its lower MF (LMF) ${\underline{μ}}_{\tilde{A}} (x)$ and upper MF (UMF) ${\bar{μ}}_{\tilde{A}} (x)$ , i.e. $μ_{\tilde{A}} (x) = [{\underline{μ}}_{\tilde{A}} (x), {\bar{μ}}_{\tilde{A}} (x)] .$ (2)

In this study, triangular T1FSs and trapezoidal IT2FSs as shown in Fig. 1 will be adopted.

2.2 IT2FLS

As analyzed in many papers on fuzzy logic [1 , 40], the multiple-input-multiple-output (MIMO) fuzzy logic system (FLS), including the type-2 case, can be decomposed into several multiple-input-single-output (MISO) FLSs. Hence, in this study, we only consider the IT2FLS with n inputs and one output. It can be readily extended to the multi-output case. The inference structure of an IT2FLS is shown in Fig. 2. Compared with the T1FLS, the IT2FLS has a type-reducer to transmit IT2FSs to T1FSs.

Suppose that, for the sth input variable x_s, there are m_s IT2FSs ${\tilde{A}}_{s}^{1}, {\tilde{A}}_{s}^{2}, \dots, {\tilde{A}}_{s}^{m_{s}}$ to partition the domain of x_s. Hence, to be a complete fuzzy rule base, there are $\prod_{s = 1}^{n} m_{s}$ type-2 fuzzy rules, one of which can be expressed as

$\begin{matrix} Rule & (j_{1}, j_{2}, \dots, j_{n}) : if x_{1} = {\tilde{A}}_{1}^{j_{1}}, \dots, x_{n} = {\tilde{A}}_{n}^{j_{n}}, \\ then y = [{\underline{y}}^{j_{1} j_{2} \dots j_{n}}, {\bar{y}}^{j_{1} j_{2} \dots j_{n}}] \end{matrix}$ (3) where j_s ∈ {1, 2, ⋯ , m_s}, s = 1, 2, ⋯ , n, and $[{\underline{y}}^{j_{1} j_{2} \dots j_{n}}, {\bar{y}}^{j_{1} j_{2} \dots j_{n}}]$ is the consequent part (interval weight) of Rule (j₁, i₂, ⋯ , j_n).

When a crisp input x = (x₁, x₂, . . . , x_n) is given to the IT2FLS, with the singleton fuzzifier and type-2 inference process, we can obtain the interval firing strength of Rule (j₁, i₂, ⋯ , j_n) as [1 , 8] $f^{j_{1} \dots j_{n}} (x) = [{\underline{f}}^{j_{1}, i_{2}, \dots, j_{n}} (x), {\bar{f}}^{j_{1}, i_{2}, \dots, j_{n}} (x)],$ (4) in which ${\underline{f}}^{j_{1}, i_{2}, \dots, j_{n}} (x) = \prod_{s = 1}^{n} {\underline{μ}}_{{\tilde{A}}_{s}^{j_{s}}} (x_{s}),$ (5) ${\bar{f}}^{j_{1}, i_{2}, \dots, j_{n}} (x) = \prod_{s = 1}^{n} {\bar{μ}}_{{\tilde{A}}_{s}^{j_{s}}} (x_{s}),$ (6) where ${\underline{μ}}_{{\tilde{A}}_{s}^{j_{s}}}$ and ${\bar{μ}}_{{\tilde{A}}_{s}^{j_{s}}}$ are the LMF and UMF of the IT2FS ${\tilde{A}}_{s}^{j_{s}}$ . Here, in the inference process, the product T-norm is adopted.

In order to obtain a crisp output, the output processing including type-reduction and defuzzification is needed in an IT2FLS. There exist several popular type-reduction and defuzzification methods. For simplicity, in this study, we adopt the Begian-Melek-Mendel (BMM) method [41]. With this method, the crisp output of the IT2FLS can be computed as [41]

$\begin{matrix} y (x) & = & α \frac{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\underline{f}}^{j_{1} \dots j_{n}} (x) {\underline{y}}^{j_{1} \dots j_{n}}}{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\underline{f}}^{j_{1} \dots j_{n}} (x)} \\ + β \frac{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\bar{f}}^{j_{1} \dots j_{n}} (x) {\bar{y}}^{j_{1} \dots j_{n}}}{\sum_{j_{1} = 1}^{m_{n}} \dots \sum_{j_{n} = 1}^{m_{n}} {\bar{f}}^{j_{1} \dots j_{n}} (x)} . \end{matrix}$ (7)

where α ≥ 0, β ≥ 0 and α + β = 1. Usually, α and β are respectively set to be 0.5.

3 Fast learning of IT2FLS

To design an IT2FLS, the first thing is to generate its fuzzy rules including the antecedent parts and consequent parts. Then, the parameters in the fuzzy rules need to be optimized to achieve satisfactory performance. In this section, we will give a novel learning method to realize such objectives. Here, we firstly list the main steps of the proposed method, and then we will discuss each step in detail in the following subsections.

Step 1: Determine the numbers of FSs in the fuzzy partitions of the n input variables as [m₁, m₂, ⋯, m_n]. And, set up the division number K for dividing the training data set to sub-data sets.

Step 2: Based on the whole training data set, generate the initial fuzzy partitions of the n input variables to form the initial T1FLS.

Step 3: Randomly divide the training data set to K sub-data sets. For each sub-data set, train a T1FLS using the ANFIS method. And then, extract the antecedent T1FSs from the K trained T1FLSs.

Step 4: Merge the antecedent T1FSs in the K T1FLSs to form the interval type-2 fuzzy partitions for the n input variables. Then, generate the initial IT2FLS.

Step 5: Optimize the consequent parameters (interval weights) of the IT2FLS, and obtain the final IT2FLS.

In this algorithm, we adopt two strategies to guarantee the learning speed. Firstly, the parallel mechanism is utilized to generate the antecedent parts of the IT2FLS. And then, the least square method which has fast learning speed is used to optimize the consequent parameters of the IT2FLS.

There are only two kinds of parameters to be manually determined before the operation of this algorithm. The first kind is the parameter m_i – the number of FSs for the input variable x_i (i = 1, ⋯ , n). To prevent the fuzzy rule explosion problem, the value of m_i should be decided according to the number of input variables. Generally, 2 ≤ m_i ≤ 5 is ok. The other one is the parameter K – the number of the sub-data sets. Both the size of the training data and the number of the computer cores will influence the determination of the value K.

3.1 Generating initial fuzzy partitions for input variables

Suppose that the training data set is ${(x^{t}, y^{t})}_{t = 1}^{N}$ , where $x^{t} = (x_{1}^{t}, x_{2}^{t}, \dots, x_{n}^{t})^{T}$ and N is the number of the training samples.

From the training data, the input domain $[{\underline{U}}_{s}, {\bar{U}}_{s}]$ of the input variable x_s can be estimated as ${\underline{U}}_{s} = {min}_{t = 1}^{N} {x_{s}^{t}},$ (8) ${\bar{U}}_{s} = {max}_{t = 1}^{N} {x_{s}^{t}} .$ (9)

In our study, triangular T1FSs are adopted. The triangular fuzzy set ${\tilde{A}}_{s}^{j_{s}} (x_{s})$ can be expressed as $μ_{{\tilde{A}}_{s}^{j_{s}}} (x_{s}) = {\begin{matrix} \frac{x - a_{s}^{j_{s}}}{b_{s}^{j_{s}} - a_{s}^{j_{s}}}, & a_{s}^{j_{s}} \leq x < b_{s}^{j_{s}} \\ \frac{x - c_{s}^{j_{s}}}{b_{s}^{j_{s}} - c_{s}^{j_{s}}}, & b_{s}^{j_{s}} \leq x < c_{s}^{j_{s}} \\ 0, & else \end{matrix}$ (10)

The initial fuzzy partition of the input variable x_s is shown in Fig. 3, where the input domain $[{\underline{U}}_{s}, {\bar{U}}_{s}]$ is partitioned uniformly by m_s triangular fuzzy sets. In other words, the parameters of the triangular fuzzy set ${\tilde{A}}_{s}^{j_{s}} (x_{s})$ can be expressed as $(a_{s}^{j_{s}}, b_{s}^{j_{s}}, c_{s}^{j_{s}}) = (e_{s}^{j_{s} - 1}, e_{s}^{j_{s}}, e_{s}^{j_{s} + 1}) .$ (11) where $e_{s}^{j_{s}} = {\underline{U}}_{s} + (j_{s} - 1) * \frac{{\bar{U}}_{s} - {\underline{U}}_{s}}{m_{s} - 1} .$ (12)

Then, we can generate the initial T1FLS with the following fuzzy rules

$\begin{matrix} Rule & (j_{1}, j_{2}, \dots, j_{n}) : if x_{1} = {\tilde{A}}_{1}^{j_{1}}, x_{2} = {\tilde{A}}_{2}^{j_{2}}, \dots, \\ x_{n} = {\tilde{A}}_{n}^{j_{n}}, then y = y^{j_{1} j_{2} \dots j_{n}} \end{matrix}$ (13) where j_s = 1, 2, ⋯ , m_s, and s = 1, 2, ⋯ , n.

3.2 Training the antecedent T1FSs using ANFIS

In order to improve efficiency, random division will be applied to the whole data set to obtain K sub-data sets. Each sub-data set will be used to optimize the initial T1FLS to generate corresponding trained T1FLS. In this study, ANFIS will be adopted to train the T1FLSs. These T1FLSs begin from the same initial T1FLS whose antecedent T1FSs are determined as discussed in Subsection 3.1. The form of the fuzzy rules in the initial T1FLS is as shown in (13).

From this initial T1FLS, the K sub-data sets are used to train the K T1FLSs in parallel by the ANFIS method in which the antecedent parameters of T1FLSs are learned using the back-propagation algorithm (steepest descent method) while the consequent parameters are optimized through both the back-propagation algorithm and the least square method. For each sub-data set, one trained T1FLS will be generated, i.e. K trained T1FLSs will be obtained finally. Notice that at this stage, the whole data set has been divided. The sizes of the sub-data sets are one-Kth of the size of the whole data set. Also, the learning processes are run in parallel. So, the learning speed will be faster than generating T1FLS using the whole data set.

Suppose that, after being trained by the ANFIS method, the fuzzy rule base of the kth T1FLS is as follows

$\begin{matrix} Rule & (j_{1, k}, \dots, j_{n, k}) : if x_{1} = A_{1, k}^{j_{1, k}}, \dots, \\ x_{n} = A_{n, k}^{j_{n, k}}, then y = y^{j_{1, k} \dots j_{n, k}} \end{matrix}$ (14) where k = 1, 2, ⋯ , K, j_s,k ∈ {1, 2, ⋯ , m_s}. Totally, in each T1FLS, there are $\prod_{s = 1}^{n} m_{s}$ fuzzy rules. The fuzzy set $A_{s, k}^{j_{s, k}}$ is the trained triangular T1FS in the antecedents of the kth T1FLS. The optimized parameters of the triangular T1FS $A_{s, k}^{j_{s, k}}$ are expressed as $(a_{s, k}^{j_{s, k}}, b_{s, k}^{j_{s, k}}, c_{s, k}^{j_{s, k}})$ which are learned by ANFIS according to the kth sub-data set.

3.3 Generating type-2 fuzzy partition by an ensembling strategy

In this study, we obtain the type-2 fuzzy partition for each variable through ensembling the K type-1 fuzzy partitions in the K learned T1FLSs. Here, trapezoidal IT2FS is adopted to ensemble the triangular T1FSs in the type-1 fuzzy partitions. In detail, a trapezoidal IT2FS ${\tilde{A}}_{s}^{j_{s}}$ in the type-2 fuzzy partition for the sth input variable is the ensemble of the corresponding triangular T1FSs $A_{s, 1}^{j_{s, 1}}, A_{s, 2}^{j_{s, 2}}, \dots, A_{s, K}^{j_{s, K}}$ . And, the IT2FS ${\tilde{A}}_{s}^{j_{s}}$ can be depicted by eight parameters $(z_{s}^{j_{s}^{1}}, z_{s}^{j_{s}^{2}}, z_{s}^{j_{s}^{3}}, t_{s}^{j_{s}^{1}}, t_{s}^{j_{s}^{2}}, t_{s}^{j_{s}^{3}}, t_{s}^{j_{s}^{4}}, h_{s}^{j_{s}})$ where $t_{s}^{j_{s}^{1}},$ $t_{s}^{j_{s}^{2}}, t_{s}^{j_{s}^{3}}, t_{s}^{j_{s}^{4}}$ are the parameters of its UMF while $z_{s}^{j_{s}^{1}}, z_{s}^{j_{s}^{2}}, z_{s}^{j_{s}^{3}}, h_{s}^{j_{s}}$ are the parameters of its LMF. The detailed ensembling strategy is shown in Fig. 4.

From Fig. 4, we can determine the parameters of the UMF as $t_{s}^{j_{s}^{1}} = {min}_{k = 1}^{K} {a_{s, k}^{j_{s, k}}},$ (15) $t_{s}^{j_{s}^{2}} = {min}_{k = 1}^{K} {b_{s, k}^{j_{s, k}}},$ (16) $t_{s}^{j_{s}^{3}} = {max}_{k = 1}^{K} {b_{s, k}^{j_{s, k}}},$ (17) $t_{s}^{j_{s}^{4}} = {max}_{k = 1}^{K} {c_{s, k}^{j_{s, k}}} .$ (18)

The endpoints of the triangular LMF can be computed as $z_{s}^{j_{s}^{1}} = {max}_{k = 1}^{K} {a_{s, k}^{j_{s, k}}},$ (19) $z_{s}^{j_{s}^{3}} = {min}_{k = 1}^{K} {c_{s, k}^{j_{s, k}}} .$ (20)

Determining the vertex $(z_{s}^{j_{s}^{2}}, h_{s}^{j_{s}})$ of the LMF is not straightforward. From Fig. 4, the vertex $(z_{s}^{j_{s}^{2}}, h_{s}^{j_{s}})$ is the lowest one of the K * K intersecting points of the left and right lines of the K T1FSs. Such K * K intersecting points denoted as $(v_{s}^{p, q}, h_{s}^{p, q})$ can be derived as $v_{s}^{p, q} = \frac{b_{s, p}^{j_{s, p}} c_{s, q}^{j_{s, q}} - a_{s, p}^{j_{s, p}} b_{s, q}^{j_{s, q}}}{b_{s, p}^{j_{s, p}} + c_{s, q}^{j_{s, q}} - a_{s, p}^{j_{s, p}} - b_{s, q}^{j_{s, q}}},$ (21) $h_{s}^{p, q} = min {max {\frac{v_{s}^{p, q} - a_{s, p}^{j_{s, p}}}{b_{s, p}^{j_{s, p}} - a_{s, p}^{j_{s, p}}}, 0}, 1} .$ (22) where p, q = 1, 2, ⋯ , K.

As a result, the vertex $(z_{s}^{j_{s}^{2}}, h_{s}^{j_{s}})$ can then be determined as $z_{s}^{j_{s}^{2}} = v_{s}^{\tilde{p}, \tilde{q}},$ (23) $h_{s}^{j_{s}} = h_{s}^{\tilde{p}, \tilde{q}}$ (24) where $\tilde{p}, \tilde{q} = arg {min}_{p, q = 1}^{K} {h_{s}^{p, q}} .$ (25)

From above discussion, the antecedent IT2FSs in the type-2 fuzzy rules in the fuzzy rule base are obtained by the ensembling strategy. However, the parameters (interval weights) in its consequent part still need to be optimized. Below, we will use the least square algorithm to realize this objective.

3.4 Optimization of consequent parameters using least square method

To begin, the input-output mapping of the IT2FLS in (7) is rewritten in the matrix form as $y (x) = F (x)^{T} W$ (26) where $F (x) = [\begin{matrix} \frac{α {\underline{f}}^{1 \dots 1} (x)}{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\underline{f}}^{j_{1} \dots j_{n}} (x)} \\ ⋮ \\ \frac{α {\underline{f}}^{m_{1} \dots m_{n}} (x)}{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\underline{f}}^{j_{1} \dots j_{n}} (x)} \\ \frac{β {\bar{f}}^{1 \dots 1} (x)}{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\bar{f}}^{j_{1} \dots j_{n}} (x)} \\ ⋮ \\ \frac{β {\bar{f}}^{m_{1} \dots m_{n}} (x)}{\sum_{j_{1} = 1}^{m_{1}} \dots \sum_{j_{n} = 1}^{m_{n}} {\bar{f}}^{j_{1} \dots j_{n}} (x)} \end{matrix}] .$ (27) $\begin{matrix} W & = & [{\underline{y}}^{1 \dots 1}, \dots, {\underline{y}}^{m_{1} \dots m_{n}}, {\bar{y}}^{1 \dots 1}, \dots, {\bar{y}}^{m_{1} \dots m_{n}}]^{T} . \end{matrix}$ (28)

For the training samples ${(x^{t}, y^{t})}_{t = 1}^{N}$ , we need to train the IT2FLS to approximate these samples, i.e. $y (x^{t}) = F (x^{t})^{T} W = y^{t}, t = 1, 2, \dots, N .$ (29) which can be expressed in the matrix form as $H W = Y,$ (30) where $H = [F (x^{1}), F (x^{2}), \dots, F (x^{N})]^{T},$ (31) $Y = [y^{1}, y^{2}, \dots, y^{N}]^{T} .$ (32)

The training of the consequent parameters of the IT2FLS can be achieved by solving the previous linear equation under the constraint of the minimum norm least square as $min_{W} | | W | | and min_{W} | | H W - Y | |$ (33)

The smallest norm least-square solution of this optimization problem can be obtained as $\hat{W} = H^{+} Y$ (34) where H⁺ is the Moore-Penrose generalized inverse of matrix H.

From (34), we can see that the least square method only needs to compute the Moore-Penrose generalized inverse of the training matrix. As a result, its learning speed will be very fast. On the other aspect, from (33), we can observe that the least square method can minimize the prediction or identification errors. Hence, satisfactory optimization results can be expected.

4 Simulations and comparisons

In this section, we will use three benchmark problems and one real-world application to verify the effectiveness and advantages of the proposed method. The three benchmark problems are the chaotic Mackey-Glass time series prediction, one second-order time-varying system identification, and one nonlinear plant identification. And, the real-world application is to predict the wind speed based on historical wind speed time series.

Also, the proposed method for IT2FLS will be compared with some popular and well performed methods, such as the ANFIS, BPNN, RBF-AFS [31], DFNN [32], GEBF-OSFNN [33], TSCIT2FNN [34], and Simpl-eTS [37]. For comparison, we will use the following root mean squared error (RMSE) index to evaluate the performances of different methods $RMSE = \sqrt{\frac{\sum_{t = 1}^{N} (y (X^{t}) - y^{t})^{2}}{N}},$ (35) where N is the size of the training data or testing data, y (X^t) is the predicted value from the models constructed by different methods.

In order to show the fast learning properties of the proposed method, we run the IT2FLS, the ANFIS trained by the whole data set, and the BPNN at the same simulation platform which has four 3.20 GHz Cores and 4 GB RAM. Among the compared methods (IT2FLS, BPNN, ANFIS), only the proposed one adopts the stochastic strategy. Hence, we run the proposed method 100 times in each example and then compute their 95% confidence intervals for both the RMSE and the training time.

4.1 Prediction of Mackey-Glass time series

In this first example, we consider the chaotic Mackey-Glass time series prediction problem. The IT2FLS, ANFIS and BPNN are applied to the Mackey-Glass time series generated by the time-delay differential equation as follows $\frac{dx (t)}{dt} = \frac{0.2 x (t - τ)}{1 + x^{1} 0 (t - τ)} - 0.1 x (t) .$ (36) where τ = 17.

In this simulation, we generate the simulation data from the initial conditions x (0) =1.2, x (k) =0 (k < 0). As in [31 –34], 1000 input-output data pairs are generated. And, the first 500 are selected for training while the left 500 are for testing. Four inputs [x (t - 24) , x (t - 18) , x (t - 12) , x (t - 6)] are used to predict the current value x (t) by the models.

Prediction results of the proposed method for the testing data are shown in Fig. 5. From this figure, we can observe that the performance of the IT2FLS is very good, and the prediction errors lie in a very small scale. Comparison results of different models are shown in Table 1 where the learning epoch for the IT2FLS is 100(1) which means that the epoch of the ANFIS method for the T1FLSs is set to be 100 and the consequent parameters are determined by the least square method only once. Also, in this and following tables, the performance indices of IT2FLS is the 95% confidence intervals of the corresponding indices in the 100 running times. From the statistical point of view, the IT2FLS performs best compared with the ANFIS, BPNN, RBF-AFS [31], DFNN [32], GEBF-OSFNN [33], TSCIT2FNN [34]. According to the training time, the IT2FLS with the proposed learning method has the fastest learning speed compared to the ANFIS and BPNN.

4.2 Second-order time-varying system identification

In this example, we use the proposed method to identify one second order time-varying system which is expressed by the following equation [35]

$\begin{matrix} y (t) = \frac{y (t - 1) y (t - 2) y (t - 3) u (t - 1) y (t - 3)}{a (t) + y (t - 2)^{2} + y (t - 3)^{2}} \\ - \frac{y (t - 1) y (t - 2) y (t - 3) u (t - 1) b (t) + c (t) u (t)}{a (t) + y (t - 2)^{2} + y (t - 3)^{2}}, \end{matrix}$ (37) where the time-varying parameters a (t) , b (t) and c (t) are given by $a (t) = 1.2 - 0.2 cos (2 π t / T),$ (38) $b (t) = 1.0 - 0.4 sin (2 π t / T),$ (39) $c (t) = 1.0 + 0.4 sin (2 π t / T) .$ (40) in which T is the time span of the test.

In this case, we use two variables y (t - 1) and u (t) to predict the output y (t). We generate the training data in the same way as in [35]. In the training, the control input u (t) is randomly generated in [-1, 1] when t = 1, ⋯ , 400 and is given by sin(πt/45) for the rest ones. In the testing, the control input is given as $u (t) = {\begin{matrix} sin (π t / 25), & 1 \leq t < 250 \\ 1.0, & 250 \leq t < 500 \\ - 1.0, & 500 \leq t < 750 \\ 0.3 sin (\frac{π t}{25}) + & 750 \leq t \leq 1000 \\ 0.1 sin (\frac{π t}{32}) + 0.6 sin (\frac{π t}{10}), \end{matrix}$ (41)

For the testing data, the identified result and errors of the proposed method are shown in Fig. 6. Again, the performance of the IT2FLS is satisfied. Comparisons with ANFIS, BPNN, T2 TSK FNS and T1 TSK FNS [35] are listed in Table 2. In this example, the proposed method has similar or better performance than the comparison methods. From the statistical point of view, it still spends much less time to obtain better performance compared with the ANFIS and BPNN.

4.3 Nonlinear plant identification

This example shows the identification of one nonlinear plant using the proposed method. The nonlinear plant is depicted as follows [36, 37]

$\begin{matrix} y (t + 1) & = & f (y (t), y (t - 1), u (t)) \\ = & \frac{y (t) y (t - 1) (y (t) - 0.5)}{1 + y^{2} (t) + y^{2} (t - 1)} + u (t) . \end{matrix}$ (42)

There are three inputs [y (t) , y (t - 1) , u (t)] and one output y (t + 1) in this identification problem. The training and testing data are collected in the same way as in [36, 37]. Totally, 5000 and 200 data pairs are respectively generated in the training and testing.

In this example, the identification results and errors of the IT2FLS are demonstrated in Fig. 7. From this figure, the identified errors are very fine. Comparisons with ANFIS, BPNN, SANFIS [36] and Simpl-eTS [37] are given in Table 3. Again, the performance of the proposed method is the best whereas it spends much less training time.

4.4 Wind speed prediction

Accurate prediction of wind speed is very helpful to the maintenance scheduling and dispatch planning [42, 43]. However, accurate wind speed prediction is a challenging task due to the wind’s unstable and random natures.

In this experiment, the wind speed data were downloaded from the website: “http://climate. weather.gc.ca/”. The data were collected every hour between July 1, 2014 and July 31, 2015 in Regina, Saskatchewan, Canada. In the data set, there are 9504 samples. We select the data during July 1, 2014 and June 30, 2015 (totally 8760 samples) for training, and those during July 1, 2015 and July 31, 2015 (totally 744 samples) for testing. And, we use the previous four hours’ wind speeds to predict that of the next hour.

After being trained, the prediction results of the IT2FLS for the testing data are depicted in Fig. 8. From this figure, we can observe that satisfactory performance can be achieved by the proposed method. Comparison results of the IT2FLS, ANFIS and BPNN are listed in Table 4 in detail. From this table, it is obvious that, compared with ANFIS and BPNN, the IT2FLS can give similar performance. However, the learning speed of the proposed method is greatly improved. The proposed method has about 60 times faster learning speed than ANFIS and about 50 times faster learning speed than BPNN.

5 Conclusion remarks

In this paper, we presented a fast learning approach for the data-driven design of IT2FLSs. In the proposed approach, we provide two strategies to accelerate the learning speed. The first one is for the antecedent IT2FSs generation in which parallel learning mechanism is used. The second one is for the consequent parameters learning which is realized by the least square method. Such method has fast learning speed and can achieve excellent approximation and generalization performances as proven by our four examples.

In the proposed IT2FLS, its antecedent parts are generated by an ensembling strategy. This strategy abandons the steepest descent method or back-propagation algorithm based parameter learning schemes which are widely-used in many existing studies on the design of IT2FLSs. Therefore, the drawback of long training time in the steepest descent methods can be avoided.

The performance and fast learning speed have been verified by three benchmark problems and one real-world application. This tells us that the proposed method may be more suitable for the problems with large-scaled data, such as the traffic flow prediction and the stock market prediction applications.

Based upon the satisfactory approximation and generalization performances of the proposed strategy, it can also be applied to more general nonlinear systems introduced in [44, 45] and more complex industrial processes studied in [46, 47]. This will also be one of our future research directions. On the other aspect, data preprocessing methods including the clustering analysis may be expected to further improve the performance of the proposed strategy. We will also try to explore the clustering analysis based learning strategies in the near future.

Footnotes

Acknowledgments

This work is supported by National Natural Science Foundation of China (61473176, 61273149, 61573225), and the Natural Science Foundation of Shandong Province for Outstanding Young Talents in Provincial Universities (ZR2015JL021).

References

Mendel

J.M.

, Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions, Upper Saddle River, NJ: Prentice-Hall, 2001.

Mendel

J.M.

and Wu

, Perceptual Computing: Aiding People in Making Subjective Judgments, Hoboken, NJ: Wiley-IEEE Press, 2010.

Castillo

and Melin

, Type-2 Fuzzy Logic Theory and Applications, Berlin: Springer-Verlag, 2008.

Philip Chen

C.L.

, Wang

C.-H.

and Chen

, A new learning algorithm for a fully connected neuro-fuzzy inference system, IEEE Transactions Neural Networks and Learning Systems 25(10) (2014), 1741–1757.

, Wang

F.-Y.

, Zhou

, Li

and Xiao

, Footprint of uncertainty for type-2 fuzzy sets, Information Science 272 (2014), 96–110.

and Wang

F.-Y.

, Linguistic dynamic systems based on computing with words and their stabilities, Science China-F series: Information Science 52(5) (2009), 780–796.

Liang

and Mendel

J.M.

, Interval type-2 fuzzy logic systems: Theory and design, IEEE Transactions on Fuzzy Systems 8(5) (2000), 535–550.

Mendel

J.M.

, John

and Liu

, Interval type-2 fuzzy logic systems made simple, IEEE Transactions on Fuzzy Systems 14(6) (2006), 808–821.

Karnik

N.N.

, Mendel

J.M.

and Liang

, Type-2 fuzzy logic systems, IEEE Transactions on Fuzzy Systems 7(6) (1999), 643–658.

10.

Wagner

and Hagras

, Toward general type-2 fuzzy logic systems based on zSlices, IEEE Transactions on Fuzzy Systems 18(4) (2010), 637–660.

11.

Castillo

and Melin

, A review on the design and optimization of interval type-2 fuzzy controllers, Applied Soft Computing 12(4) (2012), 1267–1278.

12.

Das

A.K.

, Subramanian

and Sundaram

, An evolving interval type-2 neurofuzzy inference system and its metacognitive sequential learning algorithm, IEEE Transactions on Fuzzy Systems 23(6) (2015), 2080–2093.

13.

Lin

Y.Y.

, Chang

J.Y.

and Lin

C.T.

, A TSK-type-based selfevolving compensatory interval type-2 fuzzy neural network (TSCIT2FNN) and its applications, IEEE Transactions on Industrial Electronics 61(1) (2014), 447–459.

14.

Lin

Y.Y.

, Liao

S.H.

, Chang

J.Y.

and Lin

C.T.

, Simplified interval type-2 fuzzy neural networks, IEEE Transactions on Neural Networks and Learning Systems 25(5) (2014), 959–969.

15.

Juang

C.F.

and Chen

C.Y.

, Data-driven interval type-2 neural fuzzy system with high learning accuracy and improved model interpretability, IEEE Transactions on Cybernetics 43(6) (2013), 1781–1795.

16.

, Zhang

, Wang

and Yi

, Data-driven modeling and optimization of thermal comfort and energy consumption using type-2 fuzzy method, Soft Computing 17(11) (2013), 2075–2088.

17.

Yeh

C.Y.

, Jeng

W.H.R.

and Lee

S.J.

, Data-based system modeling using a type-2 fuzzy neural network with a hybrid learning algorithm, IEEE Transactions on Neural Networks 22(12) (2011), 2296–2309.

18.

Mendez

G.M.

and De

, Los Angeles Hernandez, Hybrid learning mechanism for interval A2-C1 type-2 non-singleton type-2 Takagi-Sugeno-Kang fuzzy logic systems, Information Sciences 220 (2013), 149–169.

19.

Mendez

G.M.

and De

, Los Angeles Hernandez, Hybrid learning for interval type-2 fuzzy logic systems based on orthogonal least-squares and back-propagation methods, Information Sciences 179(13) (2009), 2146–2157.

20.

G.D.

and Huang

P.H.

, A vectorization-optimizationmethod-based type-2 fuzzy neural network for noisy data classification, IEEE Transactions on Fuzzy Systems 21(1) (2013), 1–15.

21.

Cara

A.B.

, Wagner

, Hagras

, Pomares

and Rojas

, Multiobjective optimization and comparison of nonsingleton type-1 and singleton interval type-2 fuzzy logic systems, IEEE Transactions on Fuzzy Systems 21(3) (2013), 459–476.

22.

Hidalgo

, Melin

and Castillo

, An optimization method for designing type-2 fuzzy inference systems based on the footprint of uncertainty using genetic algorithms, Expert Systems with Applications 39(4) (2012), 4590–4598.

23.

Nguyen

S.D.

, Choi

S.-B.

and Nguyen

Q.H.

, An optimal design of interval type-2 fuzzy logic system with various experiments including magnetorheological fluid damper, Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 228 (2014), 3090–3106.

24.

Juang

C.-F.

and Jang

W.-S.

, A type-2 neural fuzzy system learned through type-1 fuzzy rules and its FPGA-based hardware implementation, Applied Soft Computing 18 (2014), 302–313.

25.

Jang

J.-S.R.

, ANFIS: Adaptive-network-based fuzzy inference system, IEEE Transactions on Systems, Man and Cybernetics 23(3) (1993), 665–685.

26.

Bagheri

, Peyhani

H.M.

and Akbari

, Financial forecasting using ANFIS networks with quantum-behaved particle swarm optimization, Expert Systems with Applications 41(14) (2014), 6235–6250.

27.

Deng

, Choi

K.S.

, Cao

and Wang

, T2FELA: Type-2 fuzzy extreme learning algorithm for fast training of interval type-2 TSK fuzzy logic system, IEEE Transactions on Neural Networks and Learning Systems 25(4) (2014), 664–676.

28.

Olatunji

S.O.

, Selamat

and Abdulraheem

, A hybrid model through the fusion of type-2 fuzzy logic systems and extreme learning machines for modelling permeability prediction, Information Fusion 16 (2014), 29–45.

29.

Goh

A.T.C.

, Back-propagation neural networks for modeling complex systems, Artificial Intelligence in Engineering 9(3) (1995), 143–151.

30.

Wang

, Zeng

and Chen

, Back propagation neural network with adaptive differential evolution algorithm for time series forecasting, Expert Systems with Applications 42(2) (2015), 855–863.

31.

Cho

K.B.

and Wang

B.H.

, Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction, Fuzzy Sets and Systems 83(3) (1996), 325–339.

32.

S.Q.

and Er

M.J.

, Dynamic fuzzy neural networks – a novel approach to function approximation, IEEE Transactions on System, Man and Cybernetics: Part B Cybernetics 30(3) (2000), 58–364.

33.

Wang

, A generalized ellipsoidal basis function based online selfconstructing fuzzy neural network, Neural Processing Letters 34(1) (2011), 13–37.

34.

Lin

Y.Y.

, Chang

J.Y.

and Lin

C.T.

, A TSK-type-based selfevolving compensatory interval type-2 fuzzy neural network (TSCIT2FNN) and its applications, IEEE Transactions on Industrial Electronics 61(1) (2014), 447–459.

35.

Abiyev

R.H.

and Kaynak

, Type 2 fuzzy neural structure for identification and control of time-varying plants, IEEE Transactions on Industrial Electronics 57(12) (2010), 4147–4159.

36.

Rong

H.-J.

, Sundararajan

, Huang

G.-B.

and Saratchandran

, Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction, Fuzzy Sets and Systems 157(9) (2006), 1260–1275.

37.

Angelov

and Filev

, Simpl-eTS: A simplified method for learning evolving Takagi-Sugeno fuzzy models, Proceedings 2005 of IEEE International Conference on Fuzzy Systems, 2005, pp. 1068–1073.

38.

Wang

L.X.

, A Course in Fuzzy Systems, Prentice-Hall Press, USA, 1999.

39.

and Mendel

J.M.

, On the continuity of type-1 and interval type-2 fuzzy logic systems, IEEE Transactions on Fuzzy Systems 19(1) (2011), 179–192.

40.

, Yi

and Zhang

, On the monotonicity of interval type-2 fuzzy logic systems, IEEE Transactions on Fuzzy Systems 22(5) (2014), 1197–1212.

41.

Biglarbegian

, Melek

and Mendel

J.M.

, On the stability of interval type-2 TSK fuzzy logic control systems, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 41(5) (2010), 798–818.

42.

Soder

, Hofmann

, Orths

, Holttinen

, Wan

Y.H.

and Tuohy

, Experience from wind integration in some high penetration areas, IEEE Transactions on Energy Conversion 22(1) (2007), 4–12.

43.

Tascikaraoglu

and Uzunoglu

, A review of combined approaches for prediction of short-term wind speed and power, Renewable and Sustainable Energy Reviews 34 (2014), 243–254.

44.

Wang

, Qiu

and Gao

, Adaptive neural control of stochastic nonlinear time-delay systems with multiple constraints, IEEE Transactions on Systems, Man, and Cybernetics: Systems. 10.1109/TSMC.2016.2562511

45.

Qiu

, Ding

S.X.

, Gao

and Yin

, Fuzzy-model-based reliable static output feedback HąÞ control of nonlinear hyperbolic PDE systems, IEEE Transactions on Fuzzy Systems 24(2) (2016), 388–400.

46.

Wang

, Gao

and Qiu

, A combined adaptive neural network and nonlinear model predictive control for multirate networked industrial process control, IEEE Transactions on Neural Networks and Learning Systems 27(2) (2016), 416–425.

47.

Wang

, Gao

and Qiu

, A combined fault-tolerant and predictive control for network-based industrial processes, IEEE Transactions on Industrial Electronics 63(4) (2016), 2529–2536.