Clustering regression based on interval-valued fuzzy outputs and interval-valued fuzzy parameters

Abstract

In this paper, a new approach is presented to the problem of the clustering regression models with imprecise quantities. In this approach, the response variable and the parameters of model are assumed to be the interval-valued fuzzy numbers. We introduce two indices to investigate the goodness-of-fit of such models based on the similarity measure and the squared errors. In addition to, the predictive ability of the proposed clustering models is evaluated by using the cross-validation method. Finally, the application of the proposed approach in modeling some soil characteristics is studied.

Keywords

Clustering regression cross-validation goodness of fit interval-valued fuzzy set least squares method similarity measure

1 Introduction

In classical regression analysis, we model a relationship between a dependent variable and some independent variables. It is supposed that the observed data are homogeneous and the parameters of model are as precise quantities. In practice, we should formulate and analyze the regression models in the case of heterogeneity of observations and to exist imprecise quantities. In this paper, the main goal is to apply the clustering techniques to regression analysis, when the response variable and the parameters of model are supposed to be as the interval-valued fuzzy numbers. In this approach, we present the one-stage generalized method to model interval-valued fuzzy linear regression. Also, the goodness-of-fit and the predictive ability of proposed models are investigated by some indices (similarity measure, squared errors, and cross-validation).

The topic of regression analysis in non-precise environments has been studied by many authors. Let us review some recent works in this topic. Arabpour and Tata [1], Chen and Hsueh [12], Coppi et al. [14], Kratschmer [30], Mohammadi and Taheri [32], Nasibov [34], Tutmez and Kaymak [43] investigated some approaches for estimating the coefficients of the linear regression model based on least squares method when the available data and/or the parameters of model are fuzzy. Several methods in linear regression model in fuzzy environment based on least-absolutes method are studied by Chachi and Taheri [11], Choi and Buckley [13], Kelkinnama and Taheri [27], Kim et al. [28], Taheri and Kelkinnama [40, 41]. Hasanpour et al. [24, 25] and Pourahmad et al. [37] presented some new approaches to formulate the linear regression models based on linear/non-linear/goal programming methods. Some approaches to model the fuzzy linear regression based on fuzzy clustering procedures are investigated by Yang and Ko [46] and Yang and Lin [47]. The regression analysis in intuitionistic/type-2 fuzzy environment studied by Arefi and Taheri [3], Hosseinzadeh et al. [26], Parvathi et al. [35], and Poleshchuk and Komarov [36].

This paper is organized as follows: In Section 2, we recall some preliminary concepts about interval-valued fuzzy sets. A new distance between interval-valued fuzzy numbers is defined in Section 3. In Section 4, based on clustering techniques, we present a least squares approach to construct a regression model when the dependent variable and the parameters of model are assumed to be as interval-valued fuzzy numbers. Inside, we introduce two indices to evaluate the goodness of fit of such regression models. Also, the predictive ability of the proposed clustering regression models is examined based on the cross-validation method. Application of the proposed approach to analyze some soil characteristics is presented in Section 5. In Section 6, we compare our method with six other methods in the topic of regression modeling in imprecise environment. A brief conclusion is provided in Section 7.

2 Preliminary concepts

In this section, we review some notations and preliminary concepts of interval-valued fuzzy sets. For more details, the reader is referred to Atanassov [5 –7] and Atanassov and Gargov [8].

Definition 2.1. An interval-valued fuzzy set (IVFS) $\tilde{A}$ on the universal set X is defined as $\tilde{A} = {〈 x, μ_{\tilde{A}} (x), ν_{\tilde{A}} (x) 〉 | x \in X},$ where $μ_{\tilde{A}} : X \to [0, 1]$ is the “degree of membership”, $ν_{\tilde{A}} : X \to [0, 1]$ is the “degree of nonmembership”, and $0 \leq μ_{\tilde{A}} (x) + ν_{\tilde{A}} (x) \leq 1$ for all x ∈ X. Also, the value $τ_{\tilde{A}} (x) = 1 - μ_{\tilde{A}} (x) - ν_{\tilde{A}} (x)$ is called the “degree of indeterminacy” of the element x in $\tilde{A}$ .

In the above definition, $μ_{\tilde{A}} (x)$ is the lower bound for degree of membership of x into $\tilde{A}$ , and $ν_{\tilde{A}} (x)$ is the lower bound for negation of membership of x into $\tilde{A}$ . Therefore, the degree of membership of x into the IVFS $\tilde{A}$ is characterized by the interval $[μ_{\tilde{A}} (x), 1 - ν_{\tilde{A}} (x)]$ (see also [8, 20]).

Generally, the idea of intuitionistic fuzzy sets was introduced by Atanassov [5, 7]. Some well-known similar generalizations of a fuzzy set are, the so-called interval-valued fuzzy set theory, introduced by Sambuc [38] (see also Gorzalczany [20]), and the vague set theory, defined by Gau and Buehrer [18]. These approaches are in general not independent and there exist relationships among them. Sometimes they are even mathematically equivalent, however they have arisen on different ground and they have different semantics. For instance, Atanassov’s construct is isomorphic to interval-valued fuzzy sets and other similar notions, even if their interpretive settings and motivation are quite different, the latter capturing the idea of ill-known membership grade, while the former starts from the idea of evaluating degrees of membership and non-membership independently (for more details, see [5 , 18]).

Definition 2.2. An IVFN $\tilde{A}$ is called a LR-IVFN if the membership and nonmembership functions are given as follows $\begin{matrix} μ_{\tilde{A}} (x) & = & {\begin{matrix} L (\frac{m - x}{s_{1}}) & x \leq m, \\ R (\frac{x - m}{s_{2}}) & m < x, \end{matrix} \\ ν_{\tilde{A}} (x) & = & {\begin{matrix} 1 - L (\frac{m - x}{s_{1}^{'}}) & x \leq m, \\ 1 - R (\frac{x - m}{s_{2}^{'}}) & m < x, \end{matrix} \end{matrix}$ where the reference functions L (.) and R (.) are strictly decreasing functions from R⁺ to [0, 1], and L (0) = R (0) =1. Meanwhile, s₁, $s_{1}^{'} \in R^{+} \cup 0$ (with $s_{1}^{'} > s_{1}$ ) are called left spreads, and s₂, $s_{2}^{'} \in R^{+} \cup 0$ (with $s_{2}^{'} > s_{2}$ ) are called right spreads of $\tilde{A}$ . We denote a LR-IVFN by $\tilde{A} = (m; s_{1}, s_{2}, s_{1}^{'}, s_{2}^{'})_{LR}$ . If L (.) = R (.), s₁ = s₂ and $s_{1}^{'} = s_{2}^{'}$ , then, it is a symmetric LR-IVFN, and is denotedas $\tilde{A} = (m; s_{1}, s_{1}^{'})_{LR}$ .

In especial case, $\tilde{A}$ is called a triangular IVFN (TIVFN) if L (x) = R (x) = max {0, 1 - x}, for all x ∈ [0, ∞). It is denoted by $(m; s_{1}, s_{2}, s_{1}^{'}, s_{2}^{'})_{T}$ (see [22]).

We will denote the set of all LR-IVFNs of R by LR - IVFN (R).

Some arithmetic operations on LR-IVFNs are defined as follows (see [9 , 42]).

Proposition 2.3. Let $\tilde{X} = (m_{x}; α_{x}, β_{x}, α_{x}^{'}, β_{x}^{'})_{LR}$ and $\tilde{Y} = (m_{y}; α_{y}, β_{y}, α_{y}^{'}, β_{y}^{'})_{LR}$ be two LR-IVFNs and λ ∈ R - {0}. Then, $\begin{matrix} \tilde{X} \oplus \tilde{Y} \\ = (m_{x}; α_{x}, β_{x}, α_{x}^{'}, β_{x}^{'})_{LR} \oplus (m_{y}; α_{y}, β_{y}, α_{y}^{'}, β_{y}^{'})_{LR} \\ = (m_{x} + m_{x}; α_{x} + α_{y}, β_{x} + β_{y}, α_{x}^{'} + α_{y}^{'}, β_{x}^{'} + β_{y}^{'})_{LR}, \end{matrix}$ and $\begin{matrix} λ \otimes \tilde{X} \\ = {\begin{matrix} (λ m_{x}; λ α_{x}, λ β_{x}, λ α_{x}^{'}, λ β_{x}^{'})_{LR} & λ \geq 0, \\ (λ m_{x}; - λ β_{x}, - λ α_{x}, - λ β_{x}^{'}, - λ α_{x}^{'})_{RL} & λ < 0 . \end{matrix} \end{matrix}$

3 Distance between interval-valued fuzzy numbers

In this section, we define a new distance between IVFNs, which is an extended version of the distance between fuzzy numbers introduced by Yang and Ko [45]. For more details on some other distances between IVFSs, see [7 , 44].

Definition 3.1. Let $\tilde{X} = (m_{x}; α_{x}, β_{x}, α_{x}^{'}, β_{x}^{'})_{LR}$ and $\tilde{Y} = (m_{y}; α_{y}, β_{y}, α_{y}^{'}, β_{y}^{'})_{LR}$ be two LR-IVFNs. Then, the distance between $\tilde{M}$ and $\tilde{N}$ is defined as follows

$\begin{matrix} d (\tilde{X}, \tilde{Y}) & = & [\frac{1}{5} ((m_{x} - m_{y})^{2} \\ + {((m_{x} - l α_{x}) - (m_{y} - l α_{y}))}^{2} \\ + {((m_{x} - l α_{x}^{'}) - (m_{y} - l α_{y}^{'}))}^{2} \\ + {((m_{x} + r β_{x}) - (m_{y} + r β_{y}))}^{2} \\ + {((m_{x} + r β_{x}^{'}) - (m_{y} + r β_{y}^{'}))}^{2})]^{1 / 2}, \end{matrix}$ where $l = \int_{0}^{1} L^{- 1} (w) dw$ and $r = \int_{0}^{1} R^{- 1} (w) dw$ .

Theorem 3.2. (LR - IVFN (R) , d) is a metric space.

Proof. Let $\tilde{X} = (m_{x}; α_{x}, β_{x}, α_{x}^{'}, β_{x}^{'})_{LR}$ , $\tilde{Y} = (m_{y}; α_{y}, β_{y}, α_{y}^{'}, β_{y}^{'})_{LR}$ , and $\tilde{Z} = (m_{z}; α_{z}, β_{z}, α_{z}^{'}, β_{z}^{'})_{LR}$ be three LR-IVF numbers in LR - IVFN (R). It suffices to prove the following items

$d (\tilde{X}, \tilde{Y}) \geq 0$ ,

$d (\tilde{X}, \tilde{Y}) = 0 \Leftrightarrow \tilde{X} = \tilde{Y}$ ,

$d (\tilde{X}, \tilde{Y}) = d (\tilde{Y}, \tilde{X})$ ,

$d (\tilde{X}, \tilde{Y}) \leq d (\tilde{X}, \tilde{Z}) + d (\tilde{Z}, \tilde{Y})$ .

Items (i), (ii), and (iii) are obviously held. We prove item (iv). We have $\begin{matrix} d^{2} (\tilde{X}, \tilde{Y}) = \frac{1}{5} ((m_{x} - m_{y})^{2} \\ + {((m_{x} - l α_{x}) - (m_{y} - l α_{y}))}^{2} \\ + {((m_{x} - l α_{x}^{'}) - (m_{y} - l α_{y}^{'}))}^{2} \end{matrix}$

$\begin{matrix} + {((m_{x} + r β_{x}) - (m_{y} + r β_{y}))}^{2} \\ + {((m_{x} + r β_{x}^{'}) - (m_{y} + r β_{y}^{'}))}^{2}) \\ = \frac{1}{5} ((m_{x} - m_{z} + m_{z} - m_{y})^{2} \\ + ((m_{x} - l α_{x}) - (m_{z} - l α_{z}) \\ {+ (m_{z} - l α_{z}) - (m_{y} - l α_{y}))}^{2} \\ + ((m_{x} - l α_{x}^{'}) - (m_{z} - l α_{z}^{'}) \\ {+ (m_{z} - l α_{z}^{'}) - (m_{y} - l α_{y}^{'}))}^{2} \\ + ((m_{x} + r β_{x}) - (m_{z} + r β_{z}) \\ {+ (m_{z} + r β_{z}) - (m_{y} + r β_{y}))}^{2} \\ + ((m_{x} + r β_{x}^{'}) - (m_{z} + r β_{z}^{'}) \\ {+ (m_{z} + r β_{z}^{'}) - (m_{y} + r β_{y}^{'}))}^{2}) \\ = \frac{1}{5} ((m_{x} - m_{z})^{2} \\ + (m_{z} - m_{y})^{2} + 2 (m_{x} - m_{z}) (m_{z} - m_{y}) \\ + [(m_{x} - l α_{x}) - (m_{z} - l α_{z})]^{2} \\ + [(m_{z} - l α_{z}) - (m_{y} - l α_{y})]^{2} \\ + 2 [(m_{x} - l α_{x}) - (m_{z} - l α_{z})] [(m_{z} - l α_{z}) \\ - (m_{y} - l α_{y})] \\ + [(m_{x} - l α_{x}^{'}) - (m_{z} - l α_{z}^{'})]^{2} \\ + [(m_{z} - l α_{z}^{'}) - (m_{y} - l α_{y}^{'})]^{2} \\ + 2 [(m_{x} - l α_{x}^{'}) - (m_{z} - l α_{z}^{'})] [(m_{z} - l α_{z}^{'}) \\ - (m_{y} - l α_{y}^{'})] \\ + [(m_{x} + r β_{x}) - (m_{z} + r β_{z})]^{2} \\ + [(m_{z} + r β_{z}) - (m_{y} + r β_{y})]^{2} \\ + 2 [(m_{x} + r β_{x}) - (m_{z} + r β_{z})] [(m_{z} + r β_{z}) \\ - (m_{y} + r β_{y})] \\ + [(m_{x} + r β_{x}^{'}) - (m_{z} + r β_{z}^{'})]^{2} \\ + [(m_{z} + r β_{z}^{'}) - (m_{y} + r β_{y}^{'})]^{2} \\ + 2 [(m_{x} + r β_{x}^{'}) - (m_{z} + r β_{z}^{'})] [(m_{z} + r β_{z}^{'}) \\ - (m_{y} + r β_{y}^{'})]) \\ = d^{2} (\tilde{X}, \tilde{Z}) + d^{2} (\tilde{Z}, \tilde{Y}) \end{matrix}$

$\begin{matrix} + \frac{2}{5} [(m_{x} - m_{z}) (m_{z} - m_{y}) \\ + [(m_{x} - l α_{x}) - (m_{z} - l α_{z})] [(m_{z} - l α_{z}) \\ - (m_{y} - l α_{y})] \\ + [(m_{x} - l α_{x}^{'}) - (m_{z} - l α_{z}^{'})] \\ [(m_{z} - l α_{z}^{'}) - (m_{y} - l α_{y}^{'})] \\ + [(m_{x} + r β_{x}) - (m_{z} + r β_{z})] \\ [(m_{z} + r β_{z}) - (m_{y} + r β_{y})] \\ + [(m_{x} + r β_{x}^{'}) - (m_{z} + r β_{z}^{'})] \\ [(m_{z} + r β_{z}^{'}) - (m_{y} + r β_{y}^{'})]] \\ (by Cauchy - Schwarz inequality) \\ \leq d^{2} (\tilde{X}, \tilde{Z}) + d^{2} (\tilde{Z}, \tilde{Y}) \\ + 2 d^{2} (\tilde{X}, \tilde{Z}) d^{2} (\tilde{Z}, \tilde{Y}) \\ = {(d^{2} (\tilde{X}, \tilde{Z}) + d^{2} (\tilde{Z}, \tilde{Y}))}^{2} . \end{matrix}$

Hence, $d (\tilde{X}, \tilde{Y}) \leq d (\tilde{X}, \tilde{Z}) + d (\tilde{Z}, \tilde{Y})$ and the claim is proved.

Example 3.3. [3] Let $\tilde{M} = (3; 1, 1; 2, 2)_{T}, \tilde{N} = (7; 2, 1; 3, 2)_{T},$ and $\tilde{A} = (12; 1, 2; 2, 3)_{T}$ be three triangular IVFNs. Here, L (x) = R (x) =1 - x, 0 ≤ x ≤ 1, and l = r = 0.5. Hence, the distances between $\tilde{M}$ , $\tilde{N}$ , and $\tilde{A}$ are obtained as $d (\tilde{M}, \tilde{N}) = 3.8079$ , $d (\tilde{N}, \tilde{A}) = 5.4037$ , and $d (\tilde{M}, \tilde{A}) = 9.2033$ .

4 Clustering interval-valued fuzzy regression

In this section, we introduce an approach to estimate the coefficients of the clustering multivariate linear regression models based on the interval-valued fuzzy output data. In the proposed models, we suppose that the coefficients of models are also considered as the interval-valued fuzzy quantities.

Therefore, we assume to have a set of observed data $(x_{1 j}, x_{2 j}, . . ., x_{kj}, {\tilde{Y}}_{j})$ , j = 1, 2, . . . , n, where ${\tilde{Y}}_{j} = (m_{y_{j}}; α_{y_{j}}, β_{y_{j}}; α_{y_{j}}^{'}, β_{y_{j}}^{'})_{LR}$ , are LR-IVFNs. The aim is to fit some clustering regression models with IVF coefficients to the data set, as follows $\begin{matrix} {\tilde{Y}}_{j}^{i} & = & {\tilde{A}}_{0 i} \oplus ({\tilde{A}}_{1 i} \otimes x_{1 j}) \\ \oplus . . . \oplus ({\tilde{A}}_{ki} \otimes x_{kj}), \begin{matrix} i = 1, . . ., c, \\ j = 1, . . ., n, \end{matrix} \end{matrix}$ where the parameters ${\tilde{A}}_{pi}$ are LR-IVFNs as ${\tilde{A}}_{pi} = (m_{a_{pi}}; α_{a_{pi}}, β_{a_{pi}}; α_{a_{pi}}^{'}, β_{a_{pi}}^{'})_{LR}$ , p = 0, 1, . . . , k. Using the arithmetic operations on LR-IVFNs (Propositions 2.3), we obtain ${\hat{\tilde{Y}}}_{ij} = ({\hat{m}}_{ij}; {\hat{α}}_{ij}, {\hat{β}}_{ij}; {\hat{α}}_{ij}^{'}, {\hat{β}}_{ij}^{'})_{LR},$ where $\begin{matrix} {\hat{m}}_{ij} & = & m_{a_{0 i}} + \sum_{p = 1}^{k} m_{a_{pi}} x_{pj}, \\ {\hat{α}}_{ij} & = & α_{a_{0 i}} + \sum_{p = 1}^{k} [α_{a_{pi}} | x_{pj} | s_{pj} + β_{a_{pi}} | x_{pj} | (1 - s_{pj})], \\ {\hat{β}}_{ij} & = & β_{a_{0 i}} + \sum_{p = 1}^{k} [β_{a_{pi}} | x_{pj} | s_{pj} + α_{a_{pi}} | x_{pj} | (1 - s_{pj})], \\ {\hat{α}}_{ij}^{'} & = & α_{a_{0 i}}^{'} + \sum_{p = 1}^{k} [α_{a_{pi}}^{'} | x_{pj} | s_{pj} + β_{a_{pi}}^{'} | x_{pj} | (1 - s_{pj})], \\ {\hat{β}}_{ij}^{'} & = & β_{a_{0 i}}^{'} + \sum_{p = 1}^{k} [β_{a_{pi}}^{'} | x_{pj} | s_{pj} + α_{a_{pi}}^{'} | x_{pj} | (1 - s_{pj})], \end{matrix}$ where $s_{pj} = {\begin{matrix} 1 & x_{pj} \geq 0, \\ 0 & x_{pj} < 0, \end{matrix} j = 1, . . ., n, p = 1, . . ., k .$

For simplicity, we consider some matrix forms as follows (x_0j = 1, j = 1, . . . , n) $\begin{matrix} X & = & [\begin{matrix} 1 & x_{11} & \dots & x_{k 1} \\ 1 & x_{12} & \dots & x_{k 2} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 1 & x_{1 n} & \dots & x_{kn} \end{matrix}], \\ X_{s} & = & [\begin{matrix} 1 & | x_{11} | s_{11} & \dots & | x_{k 1} | s_{k 1} \\ 1 & | x_{12} | s_{12} & \dots & | x_{k 2} | s_{k 2} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 1 & | x_{1 n} | s_{1 n} & \dots & | x_{kn} | s_{kn} \end{matrix}], \end{matrix}$

$\begin{matrix} X_{0 s} & = & [\begin{matrix} 0 & | x_{11} | (1 - s_{11}) & \dots & | x_{k 1} | (1 - s_{k 1}) \\ 0 & | x_{12} | (1 - s_{12}) & \dots & | x_{k 2} | (1 - s_{k 2}) \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & | x_{1 n} | (1 - s_{1 n}) & \dots & | x_{kn} | (1 - s_{kn}) \end{matrix}], \\ μ & = & [\begin{matrix} μ_{11} & μ_{21} & \dots & μ_{c 1} \\ μ_{12} & μ_{22} & \dots & μ_{c 2} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ μ_{1 n} & μ_{2 n} & \dots & μ_{cn} \end{matrix}], \\ μ^{*} & = & [\begin{matrix} \frac{1}{μ_{11}} & \frac{1}{μ_{21}} & \dots & \frac{1}{μ_{c 1}} \\ \frac{1}{μ_{12}} & \frac{1}{μ_{22}} & \dots & \frac{1}{μ_{c 2}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ \frac{1}{μ_{1 n}} & \frac{1}{μ_{2 n}} & \dots & \frac{1}{μ_{cn}} \end{matrix}] \\ μ^{(i)} & = & [\begin{matrix} μ_{i 1}^{m} & 0 & \dots & 0 \\ 0 & μ_{i 2}^{m} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & μ_{in}^{m} \end{matrix}], \\ D & = & [\begin{matrix} d_{11}^{\frac{2}{m - 1}} & d_{21}^{\frac{2}{m - 1}} & \dots & d_{c 1}^{\frac{2}{m - 1}} \\ d_{12}^{\frac{2}{m - 1}} & d_{22}^{\frac{2}{m - 1}} & \dots & d_{c 2}^{\frac{2}{m - 1}} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ d_{1 n}^{\frac{2}{m - 1}} & d_{2 n}^{\frac{2}{m - 1}} & \dots & d_{cn}^{\frac{2}{m - 1}} \end{matrix}] \\ S_{d} & = & [\begin{matrix} \sum_{t = 1}^{c} \frac{1}{d_{t 1}^{\frac{2}{m - 1}}} & 0 & \dots & 0 \\ 0 & \sum_{t = 1}^{c} \frac{1}{d_{t 2}^{\frac{2}{m - 1}}} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 0 & 0 & \dots & \sum_{t = 1}^{c} \frac{1}{d_{tn}^{\frac{2}{m - 1}}} \end{matrix}] \\ m_{a}^{(i)} & = & [\begin{matrix} m_{a_{0 i}} \\ m_{a_{1 i}} \\ ⋮ \\ m_{a_{ki}} \end{matrix}], α_{a}^{(i)} = [\begin{matrix} α_{a_{0 i}} \\ α_{a_{1 i}} \\ ⋮ \\ α_{a_{ki}} \end{matrix}], \\ β_{a}^{(i)} & = & [\begin{matrix} β_{a_{0 i}} \\ β_{a_{1 i}} \\ ⋮ \\ β_{a_{ki}} \end{matrix}], {α^{'}}_{a}^{(i)} = [\begin{matrix} α_{a_{0 i}}^{'} \\ α_{a_{1 i}}^{'} \\ ⋮ \\ α_{a_{ki}}^{'} \end{matrix}], \end{matrix}$

$\begin{matrix} {β^{'}}_{a}^{(i)} & = & [\begin{matrix} β_{a_{0 i}}^{'} \\ β_{a_{1 i}}^{'} \\ ⋮ \\ β_{a_{ki}}^{'} \end{matrix}], m_{y} = [\begin{matrix} m_{y_{1}} \\ m_{y_{2}} \\ ⋮ \\ m_{y_{n}} \end{matrix}], \\ α_{y} & = & [\begin{matrix} α_{y_{1}} \\ α_{y_{2}} \\ ⋮ \\ α_{y_{n}} \end{matrix}], β_{y} = [\begin{matrix} β_{y_{1}} \\ β_{y_{2}} \\ ⋮ \\ β_{y_{n}} \end{matrix}], \\ α_{y}^{'} & = & [\begin{matrix} α_{y_{1}}^{'} \\ α_{y_{2}}^{'} \\ ⋮ \\ α_{y_{n}}^{'} \end{matrix}], β_{y}^{'} = [\begin{matrix} β_{y_{1}}^{'} \\ β_{y_{2}}^{'} \\ ⋮ \\ β_{y_{n}}^{'} \end{matrix}] . \end{matrix}$

4.1 Estimation the model parameters

Let $G = {(x_{11}, . . ., x_{k 1}, {\tilde{Y}}_{1}), . . ., (x_{1 n}, . . ., x_{kn}, {\tilde{Y}}_{n})}$ be a set of n interval-valued fuzzy subsets. Let c > 1 be a constant integer. A partition of G into c parts can be represented by the membership functions $μ_{ij} = μ_{i} (x_{1 j}, x_{2 j}, . . ., x_{kj}, {\tilde{Y}}_{j})$ such that μ_ij ∈ [0, 1] and $\sum_{i = 1}^{c} μ_{ij} = 1$ for all j = 1, 2, . . . , n in G. Here, μ_ij is the membership degree of jth observation into ith cluster.

For estimating the parameters of the clustering models, the sum of the squared errors based on the proposed distance is defined as follows: $\begin{matrix} H & = & \sum_{i = 1}^{c} \sum_{j = 1}^{n} μ_{ij}^{m} d^{2} ({\hat{\tilde{Y}}}_{ij}, {\tilde{Y}}_{j}) \\ = & \frac{1}{5} \sum_{i = 1}^{c} \sum_{j = 1}^{n} μ_{ij}^{m} [[{\hat{m}}_{ij} - m_{y_{j}}]^{2} \\ + [({\hat{m}}_{ij} - l {\hat{α}}_{ij}) - (m_{y_{j}} - l α_{y_{j}})]^{2} \\ + [({\hat{m}}_{ij} + r {\hat{β}}_{ij}) - (m_{y_{j}} + r β_{y_{j}})]^{2} \\ + [({\hat{m}}_{ij} - l {\hat{α}}_{ij}^{'}) - (m_{y_{j}} - l α_{y_{j}}^{'})]^{2} \\ + [({\hat{m}}_{ij} + r {\hat{β}}_{ij}^{'}) - (m_{y_{j}} + r β_{y_{j}}^{'})]^{2}], \end{matrix}$ where m ≥ 1 is the fuzziness constant (in this paper, we assume m = 2). For minimizing the sum of the squared errors H with respect to c-partitions, we consider the Lagrangian function as follows $\begin{matrix} L & = & \sum_{i = 1}^{c} \sum_{j = 1}^{n} μ_{ij}^{m} d^{2} ({\hat{\tilde{Y}}}_{ij}, {\tilde{Y}}_{j}) \\ - \sum_{j = 1}^{n} λ_{j} (\sum_{i = 1}^{c} μ_{ij} - 1) . \end{matrix}$

Hence, we have

$\begin{matrix} \frac{\partial L}{\partial m_{a_{pi}}} & = & \frac{2}{5} \sum_{j = 1}^{n} μ_{ij}^{m} x_{pj} [[{\hat{m}}_{ij} - m_{y_{j}}] \\ + [({\hat{m}}_{ij} - l {\hat{α}}_{ij}) - (m_{y_{j}} - l α_{y_{j}})] \\ + [({\hat{m}}_{ij} + r {\hat{β}}_{ij}) - (m_{y_{j}} + r β_{y_{j}})] \\ + [({\hat{m}}_{ij} - l {\hat{α}}_{ij}^{'}) - (m_{y_{j}} - l α_{y_{j}}^{'})] \\ + [({\hat{m}}_{ij} + r {\hat{β}}_{ij}^{'}) - (m_{y_{j}} + r β_{y_{j}}^{'})]], \end{matrix}$

$\begin{matrix} \frac{\partial L}{\partial α_{a_{pi}}} & = & \frac{2}{5} \sum_{j = 1}^{n} μ_{ij}^{m} | x_{pj} | \\ [{ls}_{pj} [(m_{y_{j}} - l α_{y_{j}}) - ({\hat{m}}_{ij} - l {\hat{α}}_{ij})] \\ + r (1 - s_{pj}) [({\hat{m}}_{ij} + r {\hat{β}}_{ij}) - (m_{y_{j}} + r β_{y_{j}})]], \\ \frac{\partial L}{\partial β_{a_{pi}}} & = & \frac{2}{5} \sum_{j = 1}^{n} μ_{ij}^{m} | x_{pj} | \\ [{rs}_{pj} [({\hat{m}}_{ij} + r {\hat{β}}_{ij}) - (m_{y_{j}} + r β_{y_{j}})] \\ - l (1 - s_{pj}) [({\hat{m}}_{ij} - l {\hat{α}}_{ij}) - (m_{y_{j}} - l α_{y_{j}})]], \\ \frac{\partial L}{\partial α_{a_{pi}}^{'}} & = & \frac{2}{5} \sum_{j = 1}^{n} μ_{ij}^{m} | x_{pj} | \\ [{ls}_{pj} [(m_{y_{j}} - l α_{y_{j}}^{'}) - ({\hat{m}}_{ij} - l {\hat{α}}_{ij}^{'})] \\ + r (1 - s_{pj}) [({\hat{m}}_{ij} + r {\hat{β}}_{ij}^{'}) - (m_{y_{j}} + r β_{y_{j}}^{'})]], \\ \frac{\partial L}{\partial β_{a_{pi}}^{'}} & = & \frac{2}{5} \sum_{j = 1}^{n} μ_{ij}^{m} | x_{pj} | \\ [{rs}_{pj} [({\hat{m}}_{ij} + r {\hat{β}}_{ij}^{'}) - (m_{y_{j}} + r β_{y_{j}}^{'})] \\ - l (1 - s_{pj}) [({\hat{m}}_{ij} - l {\hat{α}}_{ij}^{'}) - (m_{y_{j}} - l α_{y_{j}}^{'})]], \end{matrix}$ $\begin{matrix} \frac{\partial L}{\partial λ_{j}} & = & \sum_{i = 1}^{c} μ_{ij} - 1, \\ \frac{\partial L}{\partial μ_{ij}} & = & m μ_{ij}^{m - 1} d^{2} ({\hat{\tilde{Y}}}_{ij}, {\tilde{Y}}_{j}) - λ_{j} . \end{matrix}$

Considering $\frac{\partial L}{\partial m_{a_{pi}}} = 0$ , $\frac{\partial L}{\partial α_{a_{pi}}} = 0$ , $\frac{\partial L}{\partial β_{a_{pi}}} = 0$ , $\frac{\partial L}{\partial α_{a_{pi}}^{'}} = 0$ , $\frac{\partial L}{\partial β_{a_{pi}}^{'}} = 0$ , $\frac{\partial L}{\partial λ_{j}} = 0$ , and $\frac{\partial L}{\partial μ_{ij}} = 0$ for i = 1, . . . , c, j = 1, . . . , n, and p = 0, 1, . . . , k, and using the matrix notations, the estimations of parameters are obtained as follows

$\begin{matrix} {\hat{m}}_{a}^{(i)} & = & (X^{'} μ^{(i)} X)^{- 1} [X^{'} μ^{(i)} m_{y} - \frac{1}{5} {lX}^{'} μ^{(i)} (α_{y} + α_{y}^{'}) \\ + \frac{1}{5} {rX}^{'} μ^{(i)} (β_{y} + β_{y}^{'}) \\ - \frac{1}{5} X^{'} μ^{(i)} ({rX}_{0 s} - {lX}_{s}) ({\hat{α}}_{a}^{(i)} + {\hat{α^{'}}}_{a}^{(i)}) \\ - \frac{1}{5} X^{'} μ^{(i)} ({rX}_{s} - {lX}_{0 s}) ({\hat{β}}_{a}^{(i)} + {\hat{β^{'}}}_{a}^{(i)})], \end{matrix}$ (1) $\begin{matrix} {\hat{α}}_{a}^{(i)} & = & (l^{2} X_{s}^{'} μ^{(i)} X_{s} + r^{2} X_{0 s}^{'} μ^{(i)} X_{0 s})^{- 1} \\ [({lX}_{s}^{'} - {rX}_{0 s}^{'}) μ^{(i)} X {\hat{m}}_{a}^{(i)} \\ - (l^{2} X_{s}^{'} μ^{(i)} X_{0 s} + r^{2} X_{0 s}^{'} μ^{(i)} X_{s}) {\hat{β}}_{a}^{(i)} \\ - ({lX}_{s}^{'} - {rX}_{0 s}^{'}) μ^{(i)} m_{y} \\ + l^{2} X_{s}^{'} μ^{(i)} α_{y} + r^{2} X_{0 s}^{'} μ^{(i)} β_{y}], \end{matrix}$ (2) $\begin{matrix} {\hat{β}}_{a}^{(i)} & = & (r^{2} X_{s}^{'} μ^{(i)} X_{s} + l^{2} X_{0 s}^{'} μ^{(i)} X_{0 s})^{- 1} \\ [({lX}_{0 s}^{'} - {rX}_{s}^{'}) μ^{(i)} X {\hat{m}}_{a}^{(i)} \\ - (r^{2} X_{s}^{'} μ^{(i)} X_{0 s} + l^{2} X_{0 s}^{'} μ^{(i)} X_{s}) {\hat{α}}_{a}^{(i)} \\ - ({lX}_{0 s}^{'} - {rX}_{s}^{'}) μ^{(i)} m_{y} \\ + l^{2} X_{0 s}^{'} μ^{(i)} α_{y} + r^{2} X_{s}^{'} μ^{(i)} β_{y}], \end{matrix}$ (3) $\begin{matrix} {\hat{α^{'}}}_{a}^{(i)} & = & (l^{2} X_{s}^{'} μ^{(i)} X_{s} + r^{2} X_{0 s}^{'} μ^{(i)} X_{0 s})^{- 1} \\ [({lX}_{s}^{'} - {rX}_{0 s}^{'}) μ^{(i)} X {\hat{m}}_{a}^{(i)} \\ - (l^{2} X_{s}^{'} μ^{(i)} X_{0 s} + r^{2} X_{0 s}^{'} μ^{(i)} X_{s}) {\hat{β^{'}}}_{a}^{(i)} \\ - ({lX}_{s}^{'} - {rX}_{0 s}^{'}) μ^{(i)} m_{y} \\ + l^{2} X_{s}^{'} μ^{(i)} α_{y}^{'} + r^{2} X_{0 s}^{'} μ^{(i)} β_{y}^{'}], \end{matrix}$ (4)

$\begin{matrix} {\hat{β^{'}}}_{a}^{(i)} & = & (r^{2} X_{s}^{'} μ^{(i)} X_{s} + l^{2} X_{0 s}^{'} μ^{(i)} X_{0 s})^{- 1} \\ [({lX}_{0 s}^{'} - {rX}_{s}^{'}) μ^{(i)} X {\hat{m}}_{a}^{(i)} \\ - (r^{2} X_{s}^{'} μ^{(i)} X_{0 s} + l^{2} X_{0 s}^{'} μ^{(i)} X_{s}) {\hat{α^{'}}}_{a}^{(i)} \\ - ({lX}_{0 s}^{'} - {rX}_{s}^{'}) μ^{(i)} m_{y} \\ + l^{2} X_{0 s}^{'} μ^{(i)} α_{y}^{'} + r^{2} X_{s}^{'} μ^{(i)} β_{y}^{'}], \end{matrix}$ (5) ${\hat{μ}}^{*} = S_{d} D .$ (6)

Now, the parameters of the proposed models is obtained based on the following algorithm (we use the R software in my calculations).

Algorithm:

Fix m = 2, c ∈ {2, . . . , n - 1}, and ɛ > 0;

For each i = 1, 2, . . . , c and j = 1, 2, . . . , n, choose the initial values $μ_{ij}^{0}$ in matrix μ, denoted by μ⁰, and the initial spreads in vectors $α_{a}^{0 (i)}$ , $β_{a}^{0 (i)}$ , ${α^{'}}_{a}^{0 (i)}$ , and ${β^{'}}_{a}^{0 (i)}$ ;

Based on equations (1) - (5) and for each i = 1, 2, . . . , c, calculate the new values $m_{a}^{1 (i)}$ , $α_{a}^{1 (i)}$ , $β_{a}^{1 (i)}$ , ${α^{'}}_{a}^{1 (i)}$ , and ${β^{'}}_{a}^{1 (i)}$ ;

If the values of spreads in vectors $α_{a}^{1 (i)}$ , $β_{a}^{1 (i)}$ , ${α^{'}}_{a}^{1 (i)}$ , and ${β^{'}}_{a}^{1 (i)}$ are negative, then they are substituted to zero;

Based on the presented values in items (A3) and (A4) and equation (6), update the values $μ_{ij}^{0}$ in matrix μ⁰ to the values $μ_{ij}^{1}$ in μ¹;

Using $max_{i, j} {| μ_{ij}^{0} - μ_{ij}^{1} |} \leq ɛ$ , compare μ⁰ to μ¹. If it is correct, then algorithm is stopped. Otherwise, set $μ_{ij}^{0} = μ_{ij}^{1}$ , $α_{a}^{0 (i)} = α_{a}^{1 (i)}$ , $β_{a}^{0 (i)} = β_{a}^{1 (i)}$ , α′_a^0 (i) = α′_a^1 (i), and β′_a^0 (i) = β′_a^1 (i) and go to item (A3).

Remark 4.1. For controlling the time of computations in above algorithm, we can change the value of ɛ in the item A6. Note that if ɛ is very small (for example ɛ = 0.0001 or ɛ = 0.00001), then the obtained results are creditable, but the time of computations are long. In this paper, we suppose ɛ = 0.0001.

Definition 4.2. Based on the proposed clustering models, for the observed data (x_1j, x_2j, . . . , x_kj) j = 1, 2, . . . , n, with the membership degrees μ_ij, the estimated values ${\hat{\tilde{Y}}}_{j}$ are defined as follows $\begin{matrix} {\hat{\tilde{Y}}}_{j} & = & (μ_{1 j} \otimes {\hat{\tilde{Y}}}_{1 j}) \oplus (μ_{2 j} \otimes {\hat{\tilde{Y}}}_{2 j}) \\ \oplus . . . \oplus (μ_{cj} \otimes {\hat{\tilde{Y}}}_{cj}) . \end{matrix}$

Definition 4.3. Assume that we have the new observation ${\underline{x}}^{new} = (x_{1}^{new}, x_{2}^{new}, . . ., x_{k}^{new})$ . The predicted value ${\hat{\tilde{Y}}}_{new}$ is obtained based on the following procedure:

First, based on the proposed clustering regression models, we calculate the response values ${\tilde{Y}}_{new}^{1}$ , ${\tilde{Y}}_{new}^{2}$ ,..., ${\tilde{Y}}_{new}^{c}$ under ${\underline{x}}^{new}$ .

Calculate d₁, . . . , d_n, and d_new as $d_{j} = \sqrt{\sum_{i = 1}^{k} x_{ij}}$ , j = 1, . . . , n, and $d_{new} = \sqrt{\sum_{i = 1}^{k} x_{i}^{new}}$ , respectively. Also, we order d₁, d₂, . . . , d_n, denoted by d₍₁₎, d₍₂₎, . . . , d_(n).

Estimate the membership degrees $μ_{i}^{new}$ , i = 1, . . . , c, as follows

if d_new ≤ d₍₁₎, then $μ_{i}^{new} = \frac{μ_{i 2} (d_{new} - d_{(1)}) - μ_{i 1} (d_{new} - d_{(2)})}{d_{(2)} - d_{(1)}},$

if d_(j-1) < d_new ≤ d_(j), then $μ_{i}^{new} = \frac{μ_{ij} (d_{new} - d_{(j - 1)}) - μ_{i (j - 1)} (d_{new} - d_{(j)})}{d_{(j)} - d_{(j - 1)}},$

if d_(n) < d_new, then $μ_{i}^{new} = \frac{μ_{in} (d_{new} - d_{(n - 1)}) - μ_{i (n - 1)} (d_{new} - d_{(n)})}{d_{(n)} - d_{(n - 1)}} .$

Finally, the response value ${\hat{\tilde{Y}}}_{new}$ is predicted as ${\hat{\tilde{Y}}}_{new} = (μ_{1}^{new} \otimes {\tilde{Y}}_{new}^{1}) \oplus . . . \oplus (μ_{c}^{new} \otimes {\tilde{Y}}_{new}^{c}) .$

4.2 Goodness of fit of the model

To evaluate the goodness of fit of the proposed models, two indices are introduced based on the similarity measure between two IVFSs and also based on the distance between two IVFSs.

Definition 4.4. [2] Let $\tilde{A}$ and $\tilde{B}$ be two IVFNs. The similarity measure between $\tilde{A}$ and $\tilde{B}$ is defined as $\begin{matrix} SM (\tilde{A}, \tilde{B}) = 1 - \frac{1}{2} [\frac{\int_{- \infty}^{\infty} | μ_{\tilde{A}} (x) - μ_{\tilde{B}} (x) | dx}{\int_{- \infty}^{\infty} μ_{\tilde{A}} (x) dx + \int_{- \infty}^{\infty} μ_{\tilde{B}} (x) dx} \\ + \frac{\int_{- \infty}^{\infty} | ν_{\tilde{A}} (x) - ν_{\tilde{B}} (x) | dx}{\int_{- \infty}^{\infty} (1 - ν_{\tilde{A}} (x)) dx + \int_{- \infty}^{\infty} (1 - ν_{\tilde{B}} (x)) dx}] . \end{matrix}$

Definition 4.5. For the proposed clustering regression models, the mean of similarity measures (MSM) between the observed values ${\tilde{Y}}_{j}$ and the estimated values ${\hat{\tilde{Y}}}_{j}$ , j = 1, 2, . . . , n, is defined as $MSM = \frac{1}{n} \sum_{j = 1}^{n} SM ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}) .$

Definition 4.6. For the proposed clustering regression models, the mean of distance (MD) between the observed values ${\tilde{Y}}_{j}$ and the estimated values ${\hat{\tilde{Y}}}_{j}$ , j = 1, 2, . . . , n, is defined as follows $MD = \frac{1}{n} \sum_{j = 1}^{n} d ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}) .$

Remark 4.7. Note that the performance of the proposed clustering regression models to estimate the response variable was assessed using $SM ({\tilde{Y}}_{i}, {\hat{\tilde{Y}}}_{i})$ and $d ({\tilde{Y}}_{i}, {\hat{\tilde{Y}}}_{i})$ , i = 1, . . . , n. The optimal models are the models with maximum value of MSM (or minimum value of MD).

4.3 Cross-validation

Cross-validation [19, 29] is a model validation technique for assessing how the results of an analysis will generalize to a data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice. It is worth highlighting that in a prediction problem, a model is usually given a data set of known data on which training is run (training data set), and a data set of unknown data against which the model is tested (testing data set).

To investigate the performance of the proposed clustering models based on the cross-validation method with the testing data set, we apply the following procedure:

The jth observation is left out from the data set.

Based on the remaining observations, we develop the new clustering interval-valued fuzzy regression models.

Based on obtained models, the values ${\hat{\tilde{Y}}}_{1 j}, . . ., {\hat{\tilde{Y}}}_{cj}$ are predicted.

Based on the distances between the predicted values ${\hat{\tilde{Y}}}_{1 j}, . . ., {\hat{\tilde{Y}}}_{cj}$ and the jth observed response value ${\tilde{Y}}_{j}$ , the membership degrees μ_1j, . . . , μ_cj are estimated as follows $μ_{ij} = {(\sum_{t = 1}^{c} {(\frac{d^{2} ({\hat{\tilde{Y}}}_{ij}, {\tilde{Y}}_{j})}{d^{2} ({\hat{\tilde{Y}}}_{tj}, {\tilde{Y}}_{j})})}^{1 / (m - 1)})}^{- 1}, i = 1, . . ., c .$

The value of jth observation is predicted as follows $\begin{matrix} {\hat{\tilde{Y}}}_{j}^{(- j)} & = & (μ_{1 j} \otimes {\hat{\tilde{Y}}}_{1 j}) \oplus (μ_{2 j} \otimes {\hat{\tilde{Y}}}_{2 j}) \\ \oplus . . . \oplus (μ_{cj} \otimes {\hat{\tilde{Y}}}_{cj}) . \end{matrix}$

Repeat items (1)– (5) to calculate ${\hat{\tilde{Y}}}_{1}^{(- 1)}, . . ., {\hat{\tilde{Y}}}_{n}^{(- n)}$ .

Calculate the similarity measures and distances between ${\tilde{Y}}_{j}$ and ${\hat{\tilde{Y}}}_{j}^{(- j)}$ , j = 1, . . . , n.

Obtain the mean of similarity measures (MSM₍₁₎) and the mean of distances (MD₍₁₎) between the response observations and the predicted values as follows $\begin{matrix} {MSM}_{(1)} & = & \frac{1}{n} \sum_{j = 1}^{n} SM ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)}), \\ {MD}_{(1)} & = & \frac{1}{n} \sum_{j = 1}^{n} d ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)}) . \end{matrix}$

Finally, if the value of MSM₍₁₎ (or MD₍₁₎) of the testing data set is near to the value of MSM (or MD) of the model with full data set (in Definition 4.6), then the clustering regression models are the optimal models.

Note that if in between data set, we have some outliers, then the clustering linear regression methods can present the suitable models (even before removing of outliers), but if we can recognize them in between data set, then the obtained models after removing outliers are better. For recognizing outliers, we can use from the results of cross-validation as follows (see for example, Tables 4 and 5).

Remark 4.8. In cross-validation, it is remarkable that the observations with high amounts of $d ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)})$ (or low amounts of $SM ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)})$ ) may be detected as the outlier observations.

5 Application in soil characteristics

One of the classical problem in soil science is the measurement of physical, chemical and/or biological soil properties. The problem results from the difficulty, time, and cost of direct measurements. In this section, some clustering pedomodels are studied to develop some relationships between different chemical and physical soil properties. The study area is a part of Silakhor plain (situated in a province west of Iran). A total of 24 core samples were obtained from 0.0 to 25-cm depth. Different soil physical and chemical properties were measured using standard procedures [32]. But due to some impreciseness in related experimental environment, the observed response variables were reported as IVFNs (see Table 1).

5.1 Clustering regression models of CEC-OM-SAND

The data set in Table 1 shows cation exchange capacity (CEC) (as a triangular IVFN response variable), sand content percentage (SAND), and organic matter content (OM) (as two independent variables). Based on this data set, we want to model the relationship between the response variable (CEC) and the explanatory variables (SAND and OM) in three (c = 3) clustering models as follows ${\tilde{Y}}_{j}^{i} = {\tilde{A}}_{0 i} \oplus ({\tilde{A}}_{1 i} \otimes x_{1 j}) \oplus ({\tilde{A}}_{2 i} \otimes x_{2 j}), \begin{matrix} i = 1, 2, 3, \\ j = 1, . . ., n, \end{matrix}$ where, the parameters ${\tilde{A}}_{pi}$ are assumed to be the symmetric triangular IVFNs as ${\tilde{A}}_{pi} = (m_{a_{pi}}; α_{a_{pi}}, α_{a_{pi}}^{'})_{T}$ , p = 0, 1, 2 and i = 1, 2, 3.

5.1.1 Estimation of the model parameters

Using the matrix forms given in Subsection 4.1, we have $m_{y} = [\begin{matrix} 16.5 \\ 18.6 \\ ⋮ \\ 12.8 \end{matrix}], α_{y} = [\begin{matrix} 0.82 \\ 0.93 \\ ⋮ \\ 0.64 \end{matrix}], α_{y}^{'} = [\begin{matrix} 1.65 \\ 1.86 \\ ⋮ \\ 1.28 \end{matrix}],$ $X = X_{s} = [\begin{matrix} 1 & 35 & 0.88 \\ 1 & 37 & 1.13 \\ ⋮ & ⋮ & ⋮ \\ 1 & 42 & 1.08 \end{matrix}], X_{0 s} = [\begin{matrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ ⋮ & ⋮ & ⋮ \\ 0 & 0 & 0 \end{matrix}],$ $m_{a}^{(i)} = [\begin{matrix} m_{a_{0 i}} \\ m_{a_{1 i}} \\ m_{a_{2 i}} \end{matrix}], α_{a}^{(i)} = [\begin{matrix} α_{a_{0 i}} \\ α_{a_{1 i}} \\ α_{a_{2 i}} \end{matrix}], {α^{'}}_{a}^{(i)} = [\begin{matrix} α_{a_{0 i}}^{'} \\ α_{a_{1 i}}^{'} \\ α_{a_{2 i}}^{'} \end{matrix}] .$

Based on Algorithm 1 for m = 2, the parameters of clustering IVF regression models for c = 3 are obtained as follows: $\begin{matrix} m_{a}^{(1)} = [\begin{matrix} 20.5550 \\ - 0.1751 \\ 2.6331 \end{matrix}], & α_{a}^{(1)} = [\begin{matrix} 1.0271 \\ 0 \\ 0.1303 \end{matrix}], \\ {α^{'}}_{a}^{(1)} = [\begin{matrix} 2.0555 \\ 0 \\ 0.2633 \end{matrix}], & m_{a}^{(2)} = [\begin{matrix} 18.7833 \\ - 0.2181 \\ 2.9420 \end{matrix}], \\ α_{a}^{(2)} = [\begin{matrix} 0.9475 \\ 0 \\ 0.1501 \end{matrix}], & {α^{'}}_{a}^{(2)} = [\begin{matrix} 1.8783 \\ 0 \\ 0.2942 \end{matrix}], \\ m_{a}^{(3)} = [\begin{matrix} 24.6282 \\ - 0.1831 \\ 1.1974 \end{matrix}], & α_{a}^{(3)} = [\begin{matrix} 1.2314 \\ 0 \\ 0.0600 \end{matrix}], \end{matrix}$

${α^{'}}_{a}^{(3)} = [\begin{matrix} 2.4628 \\ 0 \\ 0.1197 \end{matrix}] .$

Consequently, the optimal models for c = 3 are given as follows: $\begin{matrix} {\tilde{Y}}^{1} & = & {\tilde{A}}_{01} \oplus ({\tilde{A}}_{11} \otimes x_{1}) \oplus ({\tilde{A}}_{21} \otimes x_{2}) \\ = & (20.5550, 1.0271, 2.0555)_{T} \oplus (- 0.1751 x_{1}) \\ \oplus ((2.6331, 0.1303, 0.2633)_{T} \otimes x_{2}), \\ {\tilde{Y}}^{2} & = & {\tilde{A}}_{02} \oplus ({\tilde{A}}_{12} \otimes x_{1}) \oplus ({\tilde{A}}_{22} \otimes x_{2}) \\ = & (18.7833, 0.9475, 1.8783)_{T} \oplus (- 0.2181 x_{1}) \\ \oplus ((2.9420, 0.1501, 0.2942)_{T} \otimes x_{2}), \\ {\tilde{Y}}^{3} & = & {\tilde{A}}_{03} \oplus ({\tilde{A}}_{13} \otimes x_{1}) \oplus ({\tilde{A}}_{23} \otimes x_{2}) \\ = & (24.6282, 1.2314, 2.4628)_{T} \oplus (- 0.1831 x_{1}) \\ \oplus ((1.1974, 0.0600, 0.1197)_{T} \otimes x_{2}) . \end{matrix}$

The membership degree of jth observation into ith cluster, i.e. μ_ij, given in Table 2. For example, the 3th observation belongs to clusters 1– 3 with the membership degrees μ₁₃ = 0.9887, μ₂₃ = 0.0042, and μ₃₃ = 0.0071, respectively. Based on Definition 4.2, the estimated values are obtained in second column of Table 3. Also, 3th and 4th columns of Table 3 show the goodness of fit of models based on the mean of similarity measures (MSM) and the mean of distance (MD).

Now, suppose that we observe the new values ${\underline{x}}^{new} = (x_{1}^{new}, x_{2}^{new}) = (25, 1.5)$ . Based on Definition 4.1, we obtain the response values for c = 1, 2, 3 as $\begin{matrix} {\tilde{Y}}_{new}^{1} & = & (20.1272, 1.2226, 2.4505)_{T}, \\ {\tilde{Y}}_{new}^{2} & = & (17.7438, 1.1726, 2.3196)_{T}, \\ {\tilde{Y}}_{new}^{3} & = & (21.8468, 1.3214, 2.6423)_{T}, \end{matrix}$ and also the membership degrees $(μ_{1}^{new}, μ_{2}^{new}, μ_{3}^{new}) = (0.7884, 0.0150, 0.1966)$ . Hence, the new response value ${\hat{\tilde{Y}}}_{new}$ is predicted as $\begin{matrix} {\hat{\tilde{Y}}}_{new} & = & (μ_{1}^{new} \otimes {\tilde{Y}}_{new}^{1}) \oplus (μ_{2}^{new} \\ \otimes {\tilde{Y}}_{new}^{2}) \oplus (μ_{3}^{new} \otimes {\tilde{Y}}_{new}^{3}) \\ = & (20.4295, 1.2413, 2.4862)_{T} . \end{matrix}$ Therefore, if the sand content percentage (SAND) and the organic matter content (OM) are observed as $x_{1}^{new} = 25$ and $x_{2}^{new} = 1.5$ , respectively, then, the cation exchange capacity (CEC) is equal to approximately 20.4295.

5.1.2 Cross-validation and outliers

We apply the cross-validation to the CEC-OM-SAND models. The results are given in Table 4. Since the value of MSM₍₁₎ = 0.6349 (or MD₍₁₎ = 0.7720) is near to 0 (or near to 1), the predictive ability of the models is convenient.

Based on Remark 4.8, two observations (No.s 10 and 17) with the high amounts of $d ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)})$ (or low amounts of $SM ({\tilde{Y}}_{j}, {\hat{\tilde{Y}}}_{j}^{(- j)})$ ) can consider as outliers. To investigate the effects of possible outliers on clustering models performance, these points were removed from data set. As it is shown in Table 5, after removing the outliers, the MSM₍₁₎ increases from 0.6349 to 0.7098, and the MD₍₁₎ decreases from 0.7720 to 0.5330, which indicate the improvement of the clustering models.

6 Discussion

In some real systems, we may encounter with the data that are heterogeneous and we wish to obtain some suitable regression models. In such situations, we may model them based on clustering regression. Clustering regression is a technique about the domain and the data set that improves the accuracy of classical regression by partitioning training space into subspaces. For study some approaches on this topic, see Ari and G $\ddot{u}$ venir [4], Lindgren and Ljung [31], and Motoyoshi et al. [33].

In addition to the heterogeneous observations, suppose that we encounter with some imprecise quantities. In such situations, we need to develop some suitable approaches for analyzing the regression models with the existence of these restrictions. Two approaches by Yang and Ko [46] and Yang and Lin [47] are presented to model the linear regression in fuzzy environments when the observations are heterogeneous. In the following, we review these two approaches.

In comparing with proposed approach in this paper, we can review six approaches as follows. The results of comparing are summarized in Table 6.

Arefi and Taheri [3] presented a least squares approach to regression analysis when the input and output data and also the parameters of model are assumed to be the intuitionistic fuzzy numbers.

Hosseinzadeh et al. [26] studied a fuzzy linear regression model with type-2 fuzzy output data and type-2 fuzzy coefficients based on the goal programming.

Poleshchuk and Komarov [36] investigated a least squares regression model for interval type-2 fuzzy sets when the coefficients of model are assumed to be triangular fuzzy numbers. The basic idea is to determine aggregation intervals for type-1 fuzzy sets, the membership functions of whose are low and upper membership functions of interval type-2 fuzzy set.

Parvathi et al. [35] studied a linear regression analysis based on a linear programming problem when the output data and the coefficients of model are the intuitionistic fuzzy numbers.

Yang and Ko [46] applied the fuzzy clustering techniques to fuzzy simple regression analysis when the observations are heterogeneous. They presented the cluster-wise fuzzy regression analysis in two approaches: the two-stage weighted fuzzy regression and the one-stage generalized fuzzy regression.

A least-squares linear regression analysis with the fuzzy inputs-outputs and the fuzzy parameters is studied by Yang and Lin [47]. Since in this approach, we meet the heterogeneous problem in observations, they use the cluster-wise fuzzy regression analysis.

7 Conclusions

In this paper, a new approach is presented to the problem of the clustering linear regression models. Some certain merits of the proposed approach are provided as follows:

It is a new approach to formulate some clustering linear regression models based on a weighted least squares method with the weighted values $μ_{ij}^{m}$ (the membership degree of jth observation into ith cluster).

It is an extended version of Yang and Ko’s [46] and Yang and Lin’s [47] approaches to the clustering linear regression models when the response variable and the parameters of models are assumed to be the interval-valued fuzzy numbers.

To evaluate the goodness-of-fit of clustering regression models, some indices are provided based on the similarity measure and the squared errors. We have also introduced a new cross-validation method to evaluate the predictive ability of the proposed clustering models.

The extension of proposed approach can be investigated to formulate the clustering linear regression models in interval-valued fuzzy environment using the least absolutes method.

References

Arabpour

A.R.

and Tata

, Estimating the parameters of a fuzzy linear regression model, Iranian Journal of Fuzzy Systems 5 (2008), 1–19.

Arefi

and Taheri

S.M.

, Weighted similarity measure on interval-valued fuzzy sets and its application to pattern recognition, Iranian Journal of Fuzzy Systems 11 (2014), 67–79.

Arefi

and Taheri

S.M.

, Least squares regression based on Atanassov’s intuitionistic fuzzy inputs-outputs and Atanassov’s intuitionistic fuzzy parameters, IEEE Transactions on Fuzzy Systems 23 (2015), 1142–1154.

Ari

and Güvenir

H.A.

, Clustered linear regression, Knowledge-Based Systems 15 (2002), 169–175.

Atanassov

, Intuitionistic fuzzy sets, Fuzzy Sets and Systems 20 (1986), 87–96.

Atanassov

, New operations defined over the intuitionistic fuzzy sets, Fuzzy Sets and Systems 61 (1994), 137–142.

Atanassov

, Intuitionistic Fuzzy Sets: Theory and Applications, Physica-Verlag, Heidelberg, 1999.

Atanassov

and Gargov

, Interval valued intuitionistic fuzzy sets, Fuzzy Sets and Systems 31 (1989), 343–349.

Beliakov

, Bustince

, James

, Calvo

and Fernandez

, Aggregation for Atanassov’s intuitionistic and interval valued fuzzy sets: The median operator, IEEE Transactions On Fuzzy Systems 20 (2012), 487–498.

10.

Bustince

and Burillo

, Vague sets are intuitionistic fuzzy sets, Fuzzy Sets and Systems 79 (1996), 403–405.

11.

Chachi

, Taheri

S.M.

, A least-absolutes approach to multiple fuzzy regression, Proceedings of the 58th World Statistics Congress of the ISI (International Statistical Institute), Dublin, Ireland, CPS077-01, 2011.

12.

Chen

L.H.

and Hsueh

C.C.

, Fuzzy regression models using the least-squares method based on the concept of distance, IEEE Transactions On Fuzzy Systems 17 (2009), 1259–1272.

13.

Choi

S.H.

and Buckley

J.J.

, Fuzzy regression using least absolute deviation estimators, Soft Computing 12 (2008), 257–263.

14.

Coppi

, D’Urso

, Giordani

and Santoro

, Least squares estimation of a linear regression model with LR fuzzy response, Computatinal Statistics and Data Analysis 51 (2006), 267–287.

15.

Deschrijver

, Arithmetic operators in interval-valued fuzzy set theory, Information Sciences 177 (2007), 2906–2924.

16.

Deschrijver

and Kerre

E.E.

, On the position of intuitionistic fuzzy set theory in the framework of theories modelling imprecision, Information Sciences 177 (2007), 1860–1866.

17.

Dubois

, Gottwald

, Hajek

, Kacprzyk

and Prade

, Terminological difficulties in fuzzy set theory - The case of “Intuitionistic Fuzzy Sets”, Fuzzy Sets and Systems 156 (2005), 485–491.

18.

Gau

W.L.

and Buehrer

D.J.

, Vague sets, IEEE Transactions on Systems, Man, and Cybernetics 23 (1993), 610–614.

19.

Geisser

, Predictive Inference, Chapman and Hall, New York, NY, 1993.

20.

Gorzalczany

M.B.

, An Interval-valued fuzzy inference method some basic properties, Fuzzy Sets and Systems 31 (1989), 243–251.

21.

Grzegorzewski

, Distances between intuitionistic fuzzy sets and/or interval-valued fuzzy sets based on the Hausdorff metric, Fuzzy Sets and Systems 148 (2004), 319–328.

22.

Guha

and Chakraborty

, A theoretical development of distance measure for intuitionistic fuzzy numbers, International Journal of Mathematics and Mathematical Sciences 2010 (2010), 25. Article ID 949143, doi: 10.1155/2010-949143.

23.

Guha

and Chakraborty

, A new approach to fuzzy distance measure and similarity measure between two generalized fuzzy numbers, Applied Soft Computing 10 (2010), 90–99.

24.

Hasanpour

, Maleki

H.R.

and Yaghoubi

M.A.

, Fuzzy linear regression model with crisp coefficients: A goal programing approach, Iranian Journal of Fuzzy Systems 7 (2010), 19–39.

25.

Hasanpour

, Maleki

H.R.

and Yaghoubi

M.A.

, A goal programming approach to fuzzy linear regression with fuzzy inputoutput data, Soft Computing 15 (2011), 1569–1580.

26.

Hosseinzadeh

, Hassanpour

and Arefi

, A weighted goal programming approach to fuzzy linear regression with crisp inputs and type-2 fuzzy outputs, Soft Computing 19 (2015), 1143–1151.

27.

Kelkinnama

and Taheri

S.M.

, Fuzzy least-absolutes regression using shape preserving operations, Information Sciences 214 (2012), 105–120.

28.

Kim

K.J.

, Kim

D.H.

and Choi

S.H.

, Least absolute deviation estimator in fuzzy regression, Journal of Applied Mathematics & Computing 18 (2005), 649–656.

29.

Kohavi

, A study of cross-validation and bootstrap for accuracy estimation and model selection,pp, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence 2 (1995), 1137–1143.

30.

Kratschmer

, Strong consistency of least squares estimation in linear regression models with vague concepts, Journal of Multivariate Analysis 97 (2006), 633–654.

31.

Lindgren

and Ljung

, Clustered regression analysis, pp, In: Proceedings of the 41st IEEE Conference on Decision and Control (2002), 1838–1844.

32.

Mohammadi

and Taheri

S.M.

, Pedomodels fitting with fuzzy least squares regression, Iranian Journal of Fuzzy Systems 1 (2004), 45–61.

33.

Motoyoshi

, Miura

and Shioya

, Clustering by regression analysis, pp, In: Proceedings of Conference on DataWarehousing and Knowledge Discovery (DaWaK) (2003), 202–211.

34.

Nasibov

E.N.

, Fuzzy least squares regression model based of weighted distance between fuzzy numbers, Automatic Control and Computer Sciences 41 (2007), 10–17.

35.

Parvathi

, Malathi

, Akram

and Atanassov

K.T.

, Intuitionistic fuzzy linear regression analysis, Fuzzy Optimization and Decision Making 12 (2013), 215–229.

36.

Poleshchuk

and Komarov

, A fuzzy linear regression model for interval type-2 fuzzy sets, Conference of the North American Fuzzy Information Processing Society (NAFIPS), 2012, pp, 1–5.

37.

Pourahmad

, Ayatollahi

S.M.T.

and Taheri

S.M.

, Fuzzy logistic regression: A new possibilistic model and its application in clinical vague status, Iranian Journal of Fuzzy Systems 8 (2011), 1–17.

38.

Sambuc

, Fonctions φ-floues: Application a I’Aide au Diagnostic en Pathologie Thyroidienne, Ph. D. Thesis, University of Marseille, France, 1975.

39.

Szmidt

and Kacprzyk

, Distances between intuitionistic fuzzy sets, Fuzzy Sets and Systems 114 (2000), 505–518.

40.

Taheri

S.M.

and Kelkinnama

, Fuzzy least absolutes regression, Varna, Bulgaria, 11, pp. , In: Proceding of the Fourth International IEEE Conference on Intelligent Systems (2008), 55–58.

41.

Taheri

S.M.

and Kelkinnama

, Fuzzy linear regression based on least absolute deviations, Iranian Journal of Fuzzy Systems 9 (2012), 121–140.

42.

Taheri

S.M.

and Zarei

, Extension principle for vague sets and its applications, Advances in Fuzzy Mathematics 6 (2011), 17–26.

43.

Tutmez

, Kaymak

, Hybrid least-squares regression modelling using confidence bounds, In Borgelt

et al. (Eds.), Towards Advanced Data Analysis by Combining Soft Computing and Statistics, Springer, Berlin, 285, 2013, pp. 53–63.

44.

Wang

and Xin

, Distance measure between intuitionistic fuzzy sets, Pattern Recognition Letters 26 (2005), 2063–2069.

45.

Yang

M.S.

and Ko

C.H.

, On a class if fuzzy c-numbers clustering procidures for fuzzy data, Fuzzy Sets and Syatems 84 (1996), 49–60.

46.

Yang

M.S.

and Ko

C.H.

, On cluster-wise fuzzy regression analysis, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 27 (1997), 1–13.

47.

Yang

M.S.

and Lin

T.S.

, Fuzzy least-squares linear regression analysis for fuzzy input-output data, Fuzzy Sets and Systems 126 (2002), 389–399.