Obtaining interval estimates of nonlinear model parameters based on combined soft computing tools

Abstract

Obtaining interval estimates of nonlinear model parameters is as important as point estimates of model parameters. Because the estimated value of the parameters cannot always be expressed as a single numerical quantity exactly. In this study, it is aimed to propose an interval estimation procedure for nonlinear model parameters with combining soft computing methods instead of using probabilistic assumptions. For this purpose, response and model parameters were presented as triangular fuzzy numbers (TFNs) in nonlinear regression model. The errors were defined as intervals through alpha-cut operations and minimized according to the least absolute deviation (LAD) metric. The novelty of the study is achieving the minimization in a multi-objective framework in which the objective functions are lower and upper bound of interval type error functions. The NSGA-II (Non-dominated Sorting Genetic Algorithm-II) and the TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) methods were used for multi-objective optimization (MOO) and multi-criteria decision making (MCDM) stages, respectively. Innovatively, in order to obtain reasonable interval estimates, predefined sized compromise solution set was composed and the fuzzy C-means (FCM) clustering algorithm was applied to the compromise set of interval estimates according to the predicted alpha-cut values. The proposed interval estimation approach is applied on a synthetic and a real data sets for application purpose.

Keywords

Interval estimates of nonlinear model parameters fuzzy nonlinear regression fuzzy alpha-cut NSGA-II TOPSIS FCM

1 Introduction

Fuzzy regression analysis has been commonly used as a modeling tool since uncertainty always exists in real life problems. In the presence of vague nonlinear relationship between input and response variables, fuzzy nonlinear regression (FNR) becomes a powerful tool for modeling of the uncertain data sets. There have been several types of FNR application studies in the literature, e.g. FNR with fuzzy numbers [5, 26], FNR using a random weight network [31, 32], FNR with fuzzy functional networks [21], Artificial Neural Network for FNR [7 , 27], Evolutionary Algorithms for FNR [13 , 16– 18], Support Vector Machines for FNR [3 , 11], Takagi-Sugeno fuzzy model for FNR [20 , 28].

The FNR models usually consider fuzzy inputs-fuzzy response or crisp inputs-fuzzy response to define the fuzzy nonlinear functional relationship between inputs and response variables. Besides, the model parameters are assumed as crisp or fuzzy numbers. Using fuzzy model parameters helps to present the uncertainty of the parameter estimates in the FNR models. However, it is sometimes difficult to make analytical calculations with fuzzy numbers. In this case, it is proper to use interval valued fuzzy numbers obtained by α-cuts. The α-cut of a fuzzy number, a kind of threshold cut that restrict the domain of the fuzzy numbers, is always closed and bounded interval. In this manner, the α-cut of fuzzy model parameters can be considered as interval estimates for a confidence of 1-α for nonlinear model parameters.

Even though many authors have studied about FNR model, there have been only a few studies for the FNR model, based on interval valued data, obtained from the α-cuts of fuzzy numbers. Akrami et al. [1] presented a new method for solving fuzzy nonlinear optimization problems where all coefficients of the problem are triangular fuzzy numbers (TFNs) which are converted to intervals by using α-cuts. In [29], Choi et al. proposed fuzzy regression model with the monotonic response function by using the α-cuts of fuzzy number. The current studies make calculations for specifically defined an α-level. However, it is important to see the uncertainty of model parameters according to all α-levels where the α is not predefined.

In this study, the interval estimates of nonlinear regression model parameters were obtained by using α-cuts of model parameters which were considered as TFNs. It was supposed that the nonlinear functional form of the model was known explicitly. The response variable was dealed as TFN where the input variable was crisp. It was aimed to minimize the distance between observed and predicted interval valued fuzzy responses according to the least absolute deviation (LAD) metric. The main contribution of this study is achieving the minimization of errors in multi-objective framework. For this purpose, NSGA-II (Non-dominated Sorting Genetic Algorithm-II), a well-known population based stochastic multi-objective optimization (MOO) method, was used as a MOO tool to obtain Pareto set for interval estimates of model parameters with corresponding α-levels. Then, a compromise solution was chosen among the Pareto set by using TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) which is a popular multi-criteria decision making (MCDM) method. As a matter of fact that the MOO procedure gives different compromise interval estimates since the NSGA-II uses stochastic search mechanism. Innovatively, in this paper, compromise set of interval estimates for parameters was composed with the application of TOPSIS to many independent runs of the NSGA-II. Then, fuzzy C-means (FCM) clustering algorithm was applied to obtain compromise set of interval estimates. The interval estimates were categorized according to the α-levels to make decision easier.

The rest of the paper is organized as follows: In Section 2, some preliminary concept of fuzzy sets were reviewed and the FNR with fuzzy α-cuts was explained in detail. In Section 3, proposed fuzzy MOO and fuzzy decision making were presented. The NSGA-II, the TOPSIS, the FCM methods were explained briefly. In Section 4, a synthetic and a real data sets were used to illustrate the applicability of proposed interval estimates approach. Finally, conclusion was given in Section 5.

2 Fuzzy nonlinear regression with fuzzy α-cuts

2.1 Preliminaries

Fuzzy set theory, introduced by Zadeh [19], is generally used to deal with vague data and uncertain structure. A fuzzy number $\tilde{A}$ is defined as a fuzzy set on the real line $ℝ$ with the membership function $μ_{\tilde{A}} (x)$ such that (i) $\tilde{A}$ is normal, i.e. $μ_{\tilde{A}} (x_{0}) = 1$ for $\exists x_{0} \in ℝ$ , (ii) $μ_{\tilde{A}} (x)$ is convex, i.e. $μ_{\tilde{A}} (λ x_{1} + (1 - λ) x_{2}) \geq min {μ_{\tilde{A}} (x_{1}), μ_{\tilde{A}} (x_{2})},$ ∀ x₁, x₂ ∈ R, ∀ λ ∈ [0, 1]. Any fuzzy number can be described by specifying its α-cuts which are important for application of fuzzy sets. An α-level set of a fuzzy number is described as ${\tilde{A}}_{α} = {x \in ℝ : μ_{\tilde{A}} (x) \geq α}, \forall α \in [0, 1]$ (1)

Even there have been several types of fuzzy numbers in the literature, the TFN is the most popular one with wide application. The TFN is presented by two endpoints, a₁ and a₃, and one peak-point, a₂, denoted as $\tilde{A} = (a_{1}, a_{2}, a_{3})$ . Its membership function is piece-wise linear, defined as $μ_{\tilde{A}} (x) = {\begin{matrix} \frac{x - a_{1}}{a_{2} - a_{1}} & , & a_{1} \leq x \leq a_{2} \\ \frac{a_{3} - x}{a_{3} - a_{2}} & , & a_{2} \leq x \leq a_{3} \\ 0 & , & x < a_{1} and x > a_{3} . \end{matrix}$ (2)

The definitions of arithmetic operations can be seen in the study of Shyamal and Pal [2]. An α-cut operation of the TFN, generates a finite closed interval, called as α-cut interval. This α-cut interval is denoted as ${\tilde{A}}_{α} = [A_{α}^{L}, A_{α}^{R}] = [α a_{2} + (1 - α) a_{1}, α a_{2} + (1 - α) a_{3}]$ where $A_{α}^{L}$ and $A_{α}^{R}$ are the left and right end points of ${\tilde{A}}_{α}$ , respectively. It is possible to define some basic mathematical operations of TFNs by using α-cut operations.

Let $\tilde{A} = (a_{1}, a_{2}, a_{3})$ and $\tilde{B} = (b_{1}, b_{2}, b_{3})$ be two non-negative TFNs. From above-mentioned analysis, the α-cut intervals of $\tilde{A}$ and $\tilde{B}$ can be written as ${\tilde{A}}_{α} = [α a_{2} + (1 - α) a_{1}, α a_{2} + (1 - α) a_{3}]$ and ${\tilde{B}}_{α} = [α b_{2} + (1 - α) b_{1}, α b_{2} + (1 - α) b_{3}]$ , respectively. The α-cut operations can be summarized as follows:

Addition: ${\tilde{A}}_{α} \oplus {\tilde{B}}_{α} = [α (a_{2} + b_{2}) + (1 - α) (a_{1} +$ b₁) , α (a₂ + b₂) + (1 - α) (a₃ + b₃)]

Subtraction: ${\tilde{A}}_{α} (-) {\tilde{B}}_{α} = [α (a_{2} - b_{2}) + (1 - α)$ (a₁ - b₃) , α (a₂ - b₂) + (1 - α) (a₃ - b₁)]

Multiplication: ${\tilde{A}}_{α} \otimes {\tilde{B}}_{α} = [(α a_{2} + (1 - α) a_{1}) (α b_{2}$ + (1 - α) b₁) , (αa₂ + (1 - α) a₃) (αb₂ + (1 - α) b₃)]

Division: ${\tilde{A}}_{α} (\div) {\tilde{B}}_{α} = [\frac{α a_{2} + (1 - α) a_{1}}{α b_{2} + (1 - α) b_{3}}, \frac{α a_{2} + (1 - α) a_{3}}{α b_{2} + (1 - α) b_{1}}]$

Multiplication by a crisp number: Let k be a crisp number. $k {\tilde{A}}_{α} = [k {\tilde{A}}_{α}^{L}, k {\tilde{A}}_{α}^{R}], k > 0$ and $k {\tilde{A}}_{α} = [k {\tilde{A}}_{α}^{R}, k {\tilde{A}}_{α}^{L}], k < 0$

2.2 Interval estimates of parameters based on fuzzy α-cuts

Let consider a nonlinear regression model as $Y = f (X, θ) + ɛ$ (3) where Y is the response vector, X is the vector of crisp input variables, θ is the vector of unknown model parameters and ɛ is the error vector. Here, f is assumed to be known as nonlinear mathematical function and proper for modelling of replicated response measured data set. The replicated response measured data set is presented in Table 1.

Table 1

Data set with replicated response measures

No	Inputs				Response
	X ₁	X ₂	…	X _p	Y
1	x ₁₁	x ₁₂	…	x _1p	y ₁₁	y ₁₂	…	y _1r
2	x ₂₁	x ₂₂	…	x _2p	y ₂₁	y ₂₂	…	y _2r
⋮	⋮	⋮	…	⋮	⋮	⋮	…	⋮
n	x _n1	x _n2	…	x _np	y _n1	y _n2	…	y _nr

In Table 1, n denotes the number of experimental units and r is the number of replications for the response. It should be noted that each replication is measured for each setting of a group of p input variables. In order to get interval estimates of model parameters, it is aimed to apply the FNR to the data set given in Table 1. For this purpose, the replicated response measures are transformed to the TFNs by using order and descriptive statistics, e.g. minimum, median, maximum. The fuzzy valued response is denoted as ${\tilde{Y}}_{i} = (Y_{i}^{l}, Y_{i}^{c}, Y_{i}^{u})$ , i = 1, 2, . . . , n in which $Y_{i}^{l} = min (Y_{i 1}, Y_{i 2}, . . ., Y_{ir}),$ $Y_{i}^{c} = median (Y_{i 1}, Y_{i 2}, . . ., Y_{ir}),$ $Y_{i}^{u} = max (Y_{i 1}$ , Y_i2, . . . , Y_ir). The data set with fuzzy observed response variable is shown in Table 2.

Table 2

Data set with triangular fuzzy response variable

No	Inputs				Fuzzy Response
	X ₁	X ₂	…	X _p	$\tilde{Y}$
1	x ₁₁	x ₁₂	…	x _1p	${\tilde{Y}}_{1}$
2	x ₂₁	x ₂₂	…	x _2p	${\tilde{Y}}_{2}$
⋮	⋮	⋮	…	⋮	⋮
n	x _n1	x _n2	…	x _np	${\tilde{Y}}_{n}$

The membership function of the triangular observed fuzzy response vector, $\tilde{Y} = (Y^{l}, Y^{c}, Y^{u}),$ with α-cut is shown in Fig. 1. It should be noted here that the fuzzy response vector may not be symmetric.

The general form of the fuzzy nonlinear regression model can be defined as $\tilde{Y} = f (X, \tilde{θ}) + \tilde{ɛ}$ (4) where $\tilde{Y} = (Y^{l}, Y^{c}, Y^{u})$ , $\tilde{θ} = (θ^{l}, θ^{c}, θ^{u})$ , and $\tilde{ɛ} = (ɛ^{l}, ɛ^{c}, ɛ^{u})$ are TFNs, X is a crisp input variables vector, and f is assumed to be known as a nonlinear mathematical function. The Equation (4) can be written as intervals $[Y_{α}^{L}, Y_{α}^{R}] = f (X, [θ_{α}^{L}, θ_{α}^{R}]) + [ɛ_{α}^{L}, ɛ_{α}^{R}]$ (5) where $[Y_{α}^{L}, Y_{α}^{R}]$ , $[θ_{α}^{L}, θ_{α}^{R}]$ and $[ɛ_{α}^{L}, ɛ_{α}^{R}]$ are α-cut forms of $\tilde{Y}$ , $\tilde{θ}$ and $\tilde{ɛ}$ respectively. In this study, $θ_{α}^{L}$ and $θ_{α}^{R}$ are considered as lower and upper bounds of the parameter vector, θ , denoted as α θ ^c + (1 - α) θ ^l and α θ ^c + (1 - α) θ ^u, respectively. The most common method to estimate the $θ_{α}^{L}$ and $θ_{α}^{R}$ , given in Equation (5), is the minimization of the difference between the observed and the predicted α-cut response values. For this purpose, in this study, Least Absolute Deviation (LAD) metric is used for minimizing the interval type errors to estimate the parameters of FNR model. The estimation procedure is considered in a multi-objective perspective. The proposed multi-objective model can be expressed as follows: $\begin{matrix} min_{θ_{α}^{L}} f_{1} (X, θ_{α}^{L}) = | Y_{α}^{L} - f (X, [θ_{α}^{L}]) | \\ min_{θ_{α}^{R}} f_{2} (X, θ_{α}^{R}) = | Y_{α}^{R} - f (X, [θ_{α}^{R}]) | \\ θ_{α}^{L} < θ_{α}^{R} \\ α \in [0, 1] \end{matrix}$ (6) where the objective functions, f₁ and f₂, present the lower and upper distance functions, respectively. The Equation (6) can be represented explicitly as

$\begin{matrix} min_{θ^{c}, θ^{l}, α} f_{1} (X, θ^{c}, θ^{l}, α) = | (α Y^{c} + (1 - α) Y^{l}) \\ - f (X, α θ^{c} + (1 - α) θ^{l}) | \\ min_{θ^{c}, θ^{u}, α} f_{2} (X, θ^{c}, θ^{u}, α) = | (α Y^{c} + (1 - α) Y^{u}) \\ - f (X, α θ^{c} + (1 - α) θ^{u}) | \\ θ^{l} < θ^{c} < θ^{u}, \\ α \in [0, 1] \end{matrix}$ (7)

Fig.1

A triangular fuzzy response variable, $\tilde{Y}$ , with α-cut.

It is clear that the solution of the problem, given in Equation (7), is the LAD estimates of $\tilde{θ}$ , denoted as $\hat{\tilde{θ}}$ . In addition, considering with the estimated α-cut value ( $\hat{α}$ ), one can easily obtain interval estimates of θ , $[{\hat{θ}}_{\hat{α}}^{L}$ , ${\hat{θ}}_{\hat{α}}^{R}]$ .

3 Fuzzy multi-objective optimization and decision making

In this section, MOO methodology is presented to optimize the problem, given in Equation (7). The NSGA-II, introduced by Deb [15], is preferred to use as a MOO tool since it has a capability of getting Pareto solution set which is composed of alternative solutions. The NSGA-II uses a fast non-dominated sorting mechanism and predefined tunable parameters which are population size (n_pop), crossover probability (Pr _cr), mutation probability (Pr _mut) and maximum number of generations (maxgen). The basic algorithmic steps of the NSGA-II can be seen in the study of Türkşen [23]. After obtaining the Pareto solution set, it is necessary to apply a MCDM approach to choose a compromise solution among many alternative solutions. For this purpose, TOPSIS method, presented by Chen and Hwang [30], is preferred to use as a MCDM method. It should be noted here that different compromise solution may be chosen at each run of the NSGA-II since the NSGA-II finds the Pareto solution set by using probabilistic rules in a random search process. From this perspective, in this study, it is suggested to use fuzzy knowledge at decision making stage. It is achieved by applying FCM clustering algorithm, presented by Bezdek [12], with the predefined number of NSGA-II runs. Here, running number of NSGA-II corresponds to the size of compromise solutions set, denoted as ncs. The composed compromise solution set is clustered according to the α-cut values by using the FCM clustering algorithm. The brief algorithmic steps of the FCM can be seen in the study of Türkşen [23], Türkşen and Güler [24]. Here, the number of clusters is chosen equal to three because of possible basic decision scenarios for objective functions f₁ and f₂, e.g. f₁ ≺ f₂, f₂ ≺ f₁ and f₁ ≅ f₂. It is assumed that the objective functions have equal importance. So, the median of the non-dominated solutions, which belong to the equal importance cluster, are considered as interval estimates of the model parameters. The proposed interval estimation procedure for fuzzy response valued data set is summarized as below:

Step 1. Define a proper nonlinear regression function for the replicated response measured data set.

Step 2. Create a FNR function by using α-cuts of triangular fuzzy response and triangular fuzzy model parameters.

Step 3. Compose MOO problem as given in Equation (7).

Step 4. Define the size of compromise solution set, ncs, and initialize the tunable parameters of the NSGA-II.

Step 5. Apply the NSGA-II to the problem composed in Step 3.

Step 6. Choose a compromise solution, considering that the two objectives have equal importance, by using TOPSIS method. Set ncs = ncs+1.

Step 7. If the ncs is not reached, go to Step 5, else go to Step 8.

Step 8. Define the cluster numbers, taken to be three, and compute the clusters according to the predicted α-cut values by using FCM clustering algorithm.

Step 9. Choose the equal importance cluster and calculate the median of the estimates of the bounds.

4 Application

In this section, a hypothetical and a real data sets were used to illustrate the suggested fuzzy α-cut based interval estimation procedure for nonlinear regression model parameters. The hypothetical data set was studied by Duncan [8] and the real data, called nitrite utilization data set, can be seen in the study of Bates and Watts [6]. Throughout the work, the calculations for interval estimation procedure were conducted in MatLab R2013a.

Example 1. (Hypothetical data set) The hypothetical data set includes a response variable (Y) and an input variable (X) with six-point design. The response variable was replicated four times at each design points. The hypothetical data set is given in Table 3.

Table 3
The hypothetical data set [8]

No X Y

1 0.5 0.00530 0.04356 0.00603 0.05198

2 1 0.15303 0.17526 0.15337 0.20580

3 2 0.36962 0.18513 0.25143 0.25610

4 4 0.18093 0.19627 0.26221 0.15962

5 8 0.11619 0.20856 0.18540 0.09583

6 16 0.05278 0.01473 0.05738 0.02519

No	X	Y
1	0.5	0.00530	0.04356	0.00603	0.05198
2	1	0.15303	0.17526	0.15337	0.20580
3	2	0.36962	0.18513	0.25143	0.25610
4	4	0.18093	0.19627	0.26221	0.15962
5	8	0.11619	0.20856	0.18540	0.09583
6	16	0.05278	0.01473	0.05738	0.02519

The plotted data set is presented in Fig. 2. It can be easily seen from Fig. 2 that the functional relationship between input and response variables is not linear.

Fig.2

Plot of hypothetical data set input vs. response.

The appropriate nonlinear model is assumed as a first order chemical reaction function similar to study of Duncan [8]. Under certain conditions, the nonlinear regression model can be written as $Y_{i} (X_{i}, θ) = \frac{θ_{1}}{θ_{1} - θ_{2}} (e^{- θ_{2} X_{i}} - e^{- θ_{1} X_{i}}) + ɛ_{i}$ (8) where θ = [θ₁ θ₂] (>0), X_i ≥ 0, i = 1, 2, . . . , 6. In order to obtain the point estimates of nonlinear regression model parameters, θ₁ and θ₂, the least squares method is used. Duncan [8] obtained the point estimates of unknown parameters as $\hat{θ} = [{\hat{θ}}_{1}, {\hat{θ}}_{2}] = [0.21162, 0.44614] .$ The interval estimates of parameters are calculated by using conventional procedure and Jackknife procedure for 95% confidence level in the study of [8]. The results are given in Table 4.

Table 4

Interval estimates of parameters [8]

	Interval estimates
Parameters	[ $θ_{1}^{L}$ , $θ_{1}^{R}$ ]	[ $θ_{2}^{L}$ , $θ_{2}^{L}$ ]
Conventional procedure	[0.15832, 0.26492]	[0.33614, 0.55614]
Jackknife procedure	[0.15013, 0.27051]	[0.33683, 0.55173]

It can be easily seen from Table 4 that the interval estimates of conventional and jackknife procedures are quite close for the 5% nominal significance level and also both intervals cover the point estimates, $\hat{θ}$ . However, sometimes, the probabilistic modeling assumptions cannot be satisfied, e.g. the data size can be small, the data may have uncertain response measured, Jackknife computations can be time consuming. In this case, the proposed fuzzy approach for interval estimates of nonlinear model parameters can be convenient to use.

In order to create the FNR function, the data set, given in Table 3, is transformed to fuzzy response valued data set by using the minimum, median and maximum statistics of the replicated response measures. The fuzzified data set is given in Table 5.

Table 5

The hypothetical data set with fuzzy response

No	X	Y
1	0.5	(0.0053, 0.1530, 0.1851)
2	1	(0.1596, 0.0958, 0.0147)
3	2	(0.0248, 0.1643, 0.2538)
4	4	(0.1886, 0.1508, 0.0413)
5	8	(0.0520, 0.2058, 0.3696)
6	16	(0.2622, 0.2086, 0.5278)

The Equation (8) can be written according to the α-cuts of parameters and response as below:

$\begin{matrix} [Y_{α}^{L}, Y_{α}^{R}] = \frac{[θ_{1, α}^{L}, θ_{1, α}^{R}]}{[θ_{1, α}^{L}, θ_{1, α}^{R}] - [θ_{2, α}^{L}, θ_{2, α}^{R}]} \\ (e^{- [θ_{2, α}^{L}, θ_{2, α}^{R}] X} - e^{- [θ_{1, α}^{L}, θ_{1, α}^{R}] X}) + [ɛ_{α}^{L}, ɛ_{α}^{R}] \end{matrix}$ (9) where $θ_{1, α}^{L} = α θ_{1}^{c} + (1 - α) θ_{1}^{l}$ , $θ_{1, α}^{R} = α θ_{1}^{c} + (1 - α)$ $θ_{1}^{u}$ , $θ_{2, α}^{L} = α θ_{2}^{c} + (1 - α) θ_{2}^{l}$ and $θ_{2, α}^{R} = α θ_{2}^{c} +$ $(1 - α) θ_{2}^{u}$ .

By using the basic α-cut arithmetic operations, the Equation (9) is rewritten explicitly as following calculations, given in two-part,

$\begin{matrix} \frac{[θ_{1, α}^{L}, θ_{1, α}^{R}]}{[θ_{1, α}^{L}, θ_{1, α}^{R}] - [θ_{2, α}^{L}, θ_{2, α}^{R}]} = \frac{[α θ_{1}^{c} + (1 - α) θ_{1}^{l}, α θ_{1}^{c} + (1 - α) θ_{1}^{u}]}{[α θ_{1}^{c} + (1 - α) θ_{1}^{l}, α θ_{1}^{c} + (1 - α) θ_{1}^{u}] - [α θ_{2}^{c} + (1 - α) θ_{2}^{l}, α θ_{2}^{c} + (1 - α) θ_{2}^{u}]} \\ = \frac{[α θ_{1}^{c} + (1 - α) θ_{1}^{l}, α θ_{1}^{c} + (1 - α) θ_{1}^{u}]}{[α (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{l} - θ_{2}^{u}), α (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{u} - θ_{2}^{l})]} \\ = [\frac{α θ_{1}^{c} + (1 - α) θ_{1}^{l}}{α (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{u} - θ_{2}^{l})}, \frac{α θ_{1}^{c} + (1 - α) θ_{1}^{u}}{α (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{l} - θ_{2}^{u})}] \end{matrix}$ (10) and

$\begin{matrix} (e^{- [θ_{2, α}^{L}, θ_{2, α}^{R}] X} - e^{- [θ_{1, α}^{L}, θ_{1, α}^{R}] X}) = e^{- [α θ_{2}^{c} + (1 - α) θ_{2}^{l}, α θ_{2}^{c} + (1 - α) θ_{2}^{u}] X} - e^{- [α θ_{1}^{c} + (1 - α) θ_{1}^{l}, α θ_{1}^{c} + (1 - α) θ_{1}^{u}] X} \\ = [e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{u})}, e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{l})}] - [e^{X (- α θ_{1}^{c} - (1 - α) θ_{1}^{u})}, e^{X (- α θ_{1}^{c} - (1 - α) θ_{1}^{l})}] \\ = [e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{u})} - e^{X (- α θ_{1}^{c} - (1 - α) θ_{1}^{l})}, e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{l})} - e^{X (- α θ_{1}^{c} - (1 - α) θ_{1}^{u})}] . \end{matrix}$ (11)

Fig.3

The clustered non-dominated parameter estimates.

According to the proposed multi-objective model, given in Equation (7), the objective functions are formed as

$\begin{matrix} f_{1} (X, θ^{c}, θ^{l}, α) = | (α Y^{c} + (1 - α) Y^{l}) \\ - (\frac{α θ_{1}^{c} + (1 - α) θ_{1}^{l}}{α} (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{u} - θ_{2}^{l}) \\ \times (e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{u})} - e^{X} (- α θ_{1}^{c} - (1 - α) θ_{1}^{l}))) | \end{matrix}$ (12) and

$\begin{matrix} f_{2} (X, θ^{c}, θ^{u}, α) = | (α Y^{c} + (1 - α) Y^{u}) \\ - (\frac{α θ_{1}^{c} + (1 - α) θ_{1}^{u}}{α (θ_{1}^{c} - θ_{2}^{c}) + (1 - α) (θ_{1}^{l} - θ_{2}^{u})} \\ \times (e^{X (- α θ_{2}^{c} - (1 - α) θ_{2}^{l})} - e^{X (- α θ_{1}^{c} - (1 - α) θ_{1}^{u})})) | . \end{matrix}$ (13)

It is clear from the Equations (12)-(13) that the objective functions are composed based on LAD metric. Thus, the MOO problem is described as $\begin{matrix} min_{θ^{c}, θ^{l}, α} f_{1} (X, θ^{c}, θ^{l}, α) \\ min_{θ^{c}, θ^{u}, α} f_{2} (X, θ^{c}, θ^{u}, α) \\ 0 < θ^{l} < θ^{c} < θ^{u} < 1 \\ α \in [0, 1] \end{matrix}$ (14)

To obtain the interval estimates of model parameters, the multi-objective constrained model, given in Equation (14), is solved by using the NSGA-II

with the initialized tunable parameters as n_pop = 70, Pr _cr = 0.90, Pr _mut = 0.01, and maxgen = 50. The NSGA-II is runned maxgen times and at the end of 50 generations, Pareto solution set, sized 70, is obtained. The compromise solution set is composed by applying TOPSIS method to the Pareto set. The size of compromise solution set (ncs) is chosen equal to 300. Then, the FCM clustering algorithm is used to compute the interval estimate clusters according to the predicted α-cut values. The obtained clusters for non-dominated parameter estimates are presented in Fig. 3. It can be said from Fig. 3 that the Cluster-2 is the most preferable one according to the assumption of equal importance for f₁ and f₂.

The predicted α-cut values are given according to the interval parameter estimates of clusters in Fig. 4. From Fig. 4, it is possible to categorize the predicted α-cut values as [0.05– 0.4], [0.4– 0.75], and [0.75– 0.99] for Cluster-3, Cluster-2, and Cluster-1, respectively. The predicted α-cut values which are between 0.4 and 0.75 are more reasonable since the Cluster-2 is preferred. The box-plots of calculated interval estimates are presented for each cluster in Fig. 5.

Fig.4

The predicted α-cut values according to the clusters.

Fig.5

Box-plots for the interval estimates of parameters according to the clusters.

It is possible to say from Fig. 5 that if predicted α-cut values to close 0 in the range of interval estimates gets larger, if predicted α-cut values to close 1 in the range of interval estimates gets narrower. The obtained interval estimates of Cluster-2 are more reasonably acceptable since the objective functions have equal importance. The median of the interval estimates are [0.1829, 0.2781] and [0.4009, 0.5894] for θ₁ and θ₂, respectively. It can be said that these interval estimates also cover the point estimates and can be considered as an alternative interval estimates of parameters. Furthermore, the interval estimates of first parameter is narrower than the obtained interval estimates which are given in Table 4.

Example 2. (Nitrit utilization data set) The utilization of nitrite in bush beans as a function of light intensity is considered as a real data set to illustrate the proposed interval estimation approach. In the data set, the portions of primary leaves from three 16-day-old bean plants were subjected to eight levels of light intensity (μE/m²s), and the nitrite utilization (nmol/g hr) was measured on two days. The detailed information about the experiment is given in the study of Bates and Watts [6]. Here, the nitrite utilization and the light intensity are considered as the response variable and the input variable, respectively. The experimental results with three replicated response measures for each day are given in Table 6.

Table 6

The replicated measured data set for nitrite utilization versus light intensity [6]

No	Light intensity	Nitrite utilization ((nmol/g hr))
	(μE/m²s)	Day-1			Day-2
		Rep. 1	Rep. 2	Rep. 3	Rep. 1	Rep. 2	Rep. 3
1	2.2	256	685	1537	549	1550	1882
2	5.5	2148	2583	3376	1888	3372	2362
3	9.6	3634	4960	3814	4561	4939	4356
4	17.5	6986	6903	7636	7548	7471	7642
5	27	9884	11597	10221	9684	8988	8385
6	46	17319	16539	15047	13505	15324	15430
7	94	19250	20282	18357	17842	18185	17331
8	170	19638	19043	17475	18202	18315	15605

In order to understand the functional relationship between the utilization of nitrite in bush beans and the light intensity, plot of replicated nitrite utilization measures versus light intensity is presented in Fig. 6. It can be seen from the Fig. 6 that the nitrite utilization amounts have nonlinear trend considering all the light intensities.

Fig.6

Plot of nitrite utilization data set input vs. response.

According to the behavior of the replicated response measured data, the appropriate nonlinear model is preferred as Michaelis-Menten model, which met the researcher’s beliefs about nitrite utilization and light intensity, given below $f (X_{i}, θ) = \frac{θ_{1} X_{i}}{θ_{2} + X_{i} + θ_{3} X_{i}^{2}}, i = 1, 2, . . ., 8$ (15) where, X_i, i = 1, 2, . . . , 8, is the light intensity and θ = [θ₁, θ₂, θ₃] is the model parameter vector. Following the proposed interval estimate approach for θ , the replicated response measures are transformed to TFNs by using minimum, median, and maximum statistics of replicates. The data set with triangular fuzzy response is given in Table 7.

It is possible to rewrite Equation (15) according to the α-cuts of parameters and response $f (X, [θ_{α}^{L}, θ_{α}^{R}]) = \frac{[θ_{1, α}^{L}, θ_{1, α}^{R}] X}{[θ_{2, α}^{L}, θ_{2, α}^{R}] + X + [θ_{3, α}^{L}, θ_{3, α}^{R}] X^{'} X}$ (16)

Table 7

The data set with fuzzy valued nitrite utilization versus light intensity

No	Light intensity	Fuzzy valued nitrite utilization
	(μE/m²s)	(nmol/g hr)
1	2.2	(256, 1111, 1882)
2	5.5	(1888, 2473, 3376)
3	9.6	(3634, 4459, 4960)
4	17.5	(6903, 7510, 7642)
5	27	(8385, 9784, 11597)
6	46	(13505, 15377, 17319)
7	94	(17331, 18271, 20282)
8	170	(15605, 18259, 19638)

The Equation (16) can be written explicitly by using α-cut arithmetic operations as below: $\begin{array}{l} f (X, [θ_{α}^{L}, θ_{α}^{R}]) = \frac{[α θ_{1}^{c} + (1 - α) θ_{1}^{l}, α θ_{1}^{c} + (1 - α) θ_{1}^{u}] X}{[α θ_{2}^{c} + (1 - α) θ_{2}^{l}, α θ_{2}^{c} + (1 - α) θ_{2}^{u}] + X + [α θ_{3}^{c} + (1 - α) θ_{3}^{l}, α θ_{3}^{c} + (1 - α) θ_{3}^{u}] X^{'} X} \\ = \frac{[(α θ_{1}^{c} + (1 - α) θ_{1}^{l}) X, (α θ_{1}^{c} + (1 - α) θ_{1}^{u}) X]}{[α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{l} + θ_{3}^{l} X^{'} X) + X, α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{u} + θ_{3}^{u} X^{'} X) + X]} \\ = [\frac{(α θ_{1}^{c} + (1 - α) θ_{1}^{l}) X}{α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{u} + θ_{3}^{u} X^{'} X) + X}, \frac{(α θ_{1}^{c} + (1 - α) θ_{1}^{u}) X}{α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{l} + θ_{3}^{l} X^{'} X) + X}] \end{array}$ (17) By using the Equation (17), the MOO problem, given inEquation (7), can be composed as $\begin{matrix} min_{θ^{c}, θ^{l}, α} f_{1} (X, θ^{c}, θ^{l}, α) = | (α Y^{c} + (1 - α) Y^{l}) \\ - \frac{(α θ_{1}^{c} + (1 - α) θ_{1}^{l}) X}{α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{u} + θ_{3}^{u} X^{'} X) + X} | \\ min_{θ^{c}, θ^{u}, α} f_{2} (X, θ^{c}, θ^{u}, α) = | (α Y^{c} + (1 - α) Y^{u}) \\ - \frac{(α θ_{1}^{c} + (1 - α) θ_{1}^{u}) X}{α (θ_{2}^{c} + θ_{3}^{c} X^{'} X) + (1 - α) (θ_{2}^{l} + θ_{3}^{l} X^{'} X) + X} | \\ 50000 < θ_{1}^{l} < θ_{1}^{c} < θ_{1}^{u} < 100000 \\ 50 < θ_{2}^{l} < θ_{2}^{c} < θ_{2}^{u} < 210 \\ 0 < θ_{3}^{l} < θ_{3}^{c} < θ_{3}^{u} < 1 \\ α \in [0, 1] \end{matrix}$ (18) where the lower and upper bounds of parameters are defined by using expert knowledge. The NSGA-II method is applied to the MOO problem, given in Equation (18), with the tunable parameters defined as n_pop = 70, Pr _cr = 0.90, Pr _mut = 0.01, maxgen = 70 and ncs = 300. The obtained non-dominated parameter estimates are clustered according to the predicted α-cut values by using FCM clustering algorithm. The clustered results are presented in Fig. 7. It can be said from Fig. 7 that the Cluster-2 is the most preferable one according to the equal importance of f₁ and f₂.

Fig.7

The clustered non-dominated parameter estimates.

The predicted α-cut values are given according to the interval parameter estimates of clusters in Fig. 8. From Fig. 8, it is possible to categorize the predicted α-cut values as [0.75–1], [0.25– 0.75], and [0– 0.25] for Cluster-1, Cluster-2, and Cluster-3, respectively.

Fig.8

The predicted α-cut values according to the clusters.

Fig.9

Box-plots for the interval estimates of parameters according to the clusters.

The predicted α-cut values which are between 0.25 and 0.75 are more reasonable since the Cluster-2 is preferred (Fig. 8). In Fig. 9, the box-plots of calculated interval estimates are presented. The median of the interval estimates are obtained as [97093, 99896], [50.01, 50.02] and [0.001, 0.002] for θ₁, θ₂ and θ₃, respectively. The interval estimates can be considered as a soft computing based interval estimates.

5 Conclusion

This paper presents a flexible interval estimation approach for nonlinear regression model parameters by using α-cut operation of fuzzy numbers without using any probabilistic assumptions. The obtained interval estimates show that the proposed interval estimation approach, which is based on combination of soft computing tools, e.g. fuzzy α-cut, NSGA-II, TOPSIS and FCM, can be used as an alternative interval estimation procedure. It is seen from the results of hypothetical data set that the proposed approach gives similar interval estimates which are previously obtained by using conventional and Jackknife procedures. In order to obtain more reasonable interval estimates of model parameters, MOO, MCDM and clustering tools can be improved for future work.

Footnotes

Acknowledgments

The author thanks to Dr. Nilüfer Vural (Ankara University, Chemical Engineering Department) for her valuable contributions, helpful suggestions and enjoyable discussion.

References

Akrami

, Hosseini

M.M.

and Karbassi

, An Efficient Method for Solving a Class of Nonlinear Fuzzy Optimization Problems, Journal of Mathematical Extension 10(2) (2016)101–116.

Shyamal

A.K.

and Pal

, Triangular Fuzzy Matrices, Iranian Journal of Fuzzy Systems 4 (2007)75–87.

Yao

C.C.

and Yu

P.T.

, Fuzzy regression based on asymmetric support vector machines, Applied Mathematics and Computation 182 (2006)175–193.

Hong

D.H.

and Hwang

, Support fuzzy regression machines, Fuzzy Set and Systems 138 (2003)271–281.

Hong

D.H.

and Hwang

, Ridge Regression Procedures For Fuzzy Models Using Triangular Fuzzy Numbers, International Journal of Uncertainty Fuzziness and Knowledge-Based Systems 12(2) (2004)145–160.

Bates

D.M.

and Watts

D.G.

, Nonlinear Regression Analysis and Its Applications John Willey and Sons, New York, 1988.

Nasrabadi

and Hashemi

S.M.

, Robust fuzzy regression analysis using neural networks, International Journal of Uncertainty Fuzziness Knowledge Based Systems 16 (2008)579–598.

Duncan

G.T.

, and An, Empirical Study of Jackknife-Constructed Confidence Regions in Nonlinear Regression, Technometrics 20 (1978)123–129.

Ishibuchi

and Nii

, Fuzzy regression using asymmetric fuzzy coefficients and fuzzified neural networks, Fuzzy Sets and Systems 119 (2011)273–290.

10.

Han

H-G.

, Chen

Z-Y.

, Liu

H-X.

and Qiao

J-F.

, A Self organizing interval Type-2 fuzzy neural network for modeling nonlinear systems, Neurocomputing 290 (2018)196–207.

11.

Boulkaibet

, Belarbi

, Bououden

, Chadli

and Marwala

, An adaptive fuzzy predictive control of nonlinear processes based on Multi-Kernel least squares support vector regression, Applied Soft Computing Journal 73 (2018)572–590.

12.

Bezdek

J.C.

, Pattern Recognition With Fuzzy Objective Function, Plenum Press, New York, 1981.

13.

Buckley

J.J.

and Feuring

, Linear and non-linear fuzzy regression: Evolutionary algorithm solutions, Fuzzy Set and Systems 112 (2000)381–394.

14.

Dunyak

J.P.

and Wunsch

, Fuzzy regression by fuzzy number neural networks, Fuzzy Set and Systems 112(3) (2000)371–380.

15.

Deb

, Multi-Objective Optimization Using Evolutionary Algorithms, John-Wiley and Sons, New-York, 2004.

16.

Chan

K.Y.

and Kwong

C.K.

, Modeling of epoxy dispensing process using a hybrid fuzzy regression approach, International Journal of Advanced Manufacturing Technology 65 (2013)589–600.

17.

Chan

K.Y.

, Kwong

C.K.

and Fogarty

T.C.

, Modeling manufacturing processes using a genetic programming-based fuzzy regression with detection of outliers, Information Science 180 (2010)506–518.

18.

Chan

K.Y.

, Kwong

C.K.

, Dillon

T.S.

and Fung

K.Y.

, An Intelligent fuzzy regression approach for affective product design that captures nonlinearity and fuzziness, Journal of Engineering Design 22(8) (2011)523–542.

19.

Zadeh

L.A.

, The concept of a Linguistic variable and applications to approximate reasoning-part-I, II, III, Information Science 8 (1965)199–249.

20.

Bouhentala

, Ghanai

and Chafaa

, Interval-valued membership function estimation for fuzzy modeling, Fuzzy Sets and Systems 361 (2019)101–113.

21.

Gaeta

, Loilai

and Tomasiello

, A Fuzzy Functional Network for nonlinear regression problems, International Journal of Knowledge Engineering and Soft Data Paradigms 4(4) (2014).

22.

Kamali

M.Z.M.

, Kumaresan

and Ratnavelu

, Takagi-Sugeno fuzzy modelling of some nonlinear problems using ant colony programming, Applied Mathematical Modelling 48 (2017)635–654.

23.

Türkşen

, Optimization and Decision Making Stages for Multiple Responses: An Application of NSGA-II and FCM Clustering Algorithm, Gazi University Journal of Science 28 (2015)321–330.

24.

Türkşen

and Güler

, Comparison of fuzzy logic based models for the multi-response surface problems with replicated response measures, Applied Soft Computing 37 (2015)887–896.

25.

Pandit

P.K.

, Fuzzy nonlinear regression using artificial neural networks, International Journal of Advances in Applied Mathematics and Mechanics 2(1) (2014)53–63.

26.

Shrestha

R.R.

and Simonovic

S.P.

, Fuzzy Nonlinear Regression Approach to Stage-Discharge Analyses: Case Study, Journal of Hydrological Engineering (2010)49–56.

27.

Yeylaghi

, Otadi

and Imankhan

, A new fuzzy regression model based on interval-valued fuzzy neural network and its applications to management, Beni-Suef University Journal of Basic and Applied Sciences 6 (2017)106–111.

28.

Zaidi

and Kroll

, NOE TS fuzzy modelling of nonlinear dynamic systems with uncertainties using symbolic interval-valued data, Applied Soft Computing 57 (2017)353–362.

29.

Choi

S.H.

, Jung

H.Y.

, Lee

W.J.

and Yoon

J.H.

, Fuzzy Regression Model With Monotonic Response Function, Commun Korean Math Soc 33 (2018)973–983.

30.

Chen

S.J.

and Hwang

C.L.

, Fuzzy Multiple Attribute Decision Making: Methods and Applications, Springer-Verlag, Berlin, 1992.

31.

Y.L.

, Wang

X.Z.

and Huang

J.Z.

, Fuzzy nonlinear regression analysis using a random weight network, Information Sciences 364 (2016)222–240.

32.

Y-L.

, Wei

C-H.

, Long

, Aamir

, Ashfaq

and Hwang

J.Z.

, Random weight network based fuzzy nonlinear regression for trapezoidal fuzzy number data, Applied Soft Computing 70 (2018)959–979.

Obtaining interval estimates of nonlinear model parameters based on combined soft computing tools

Abstract

Keywords

1 Introduction

2 Fuzzy nonlinear regression with fuzzy α-cuts

2.1 Preliminaries

4 Application

Table 3 The hypothetical data set [8] No X Y 1 0.5 0.00530 0.04356 0.00603 0.05198 2 1 0.15303 0.17526 0.15337 0.20580 3 2 0.36962 0.18513 0.25143 0.25610 4 4 0.18093 0.19627 0.26221 0.15962 5 8 0.11619 0.20856 0.18540 0.09583 6 16 0.05278 0.01473 0.05738 0.02519

Footnotes

Acknowledgments

References

Table 3
The hypothetical data set [8]

No X Y

1 0.5 0.00530 0.04356 0.00603 0.05198

2 1 0.15303 0.17526 0.15337 0.20580

3 2 0.36962 0.18513 0.25143 0.25610

4 4 0.18093 0.19627 0.26221 0.15962

5 8 0.11619 0.20856 0.18540 0.09583

6 16 0.05278 0.01473 0.05738 0.02519