Kernel-based orthogonal quantile regression model

Abstract

Quantile regression models with errors in variables have received a great deal of attention in the social and natural sciences. Some efforts have been devoted to develop effective estimation methods for such quantile regression models. In this paper we propose a kernel-based orthogonal quantile regression model that effectively considers the errors on both input and response variables. We also provide a generalized cross validation method for choosing the hyperparameters and the ratios of the error variances which affect the performance of the proposed models. The proposed method is evaluated through simulations.

Keywords

Errors-in-variables generalized cross validation kernel measurement error orthogonal residual quantile regression support vector machine support vector quantile regression

1. Introduction

A great deal of attention has been focused on the problem of quantile regression (QR) estimation. Most of this attention has been paid to data measured exactly without error. The introductions and current research areas of the quantile regression can be found in Koenker (2005) and Takeuchi et al. (2006). On the other hand, QR analysis with errors in variables (EIV) is evolving, albeit slowly. See, for example, He and Liang (2000), Chesher (2001), Barnes and Hughes (2002), Steinwart and Christmann (2008), Ioannidesa and Matzner-Løber (2009), Wei and Carroll (2009), Ma and Yin (2011), Montes-Rojas (2011), Wang et al. (2012). Many areas of applied statistics have become aware of the problem of measurement error-prone variables and their appropriate analysis. However, less attention has been paid to QR with EIV than to mean regression with EIV because of two main difficulties for correcting the bias in QR caused by EIV (Wang et al. 2012). One is that a parametric regression-error likelihood is usually not specified in QR. The other is that the quantile of the sum of two random variables is not necessarily the sum of the two marginal quantiles. In addition, most literature has centered around the parametric approach in which the QR function is assumed to take on a particular functional form. The desire to investigate the effect of EIV in nonparametric QR leads to the subject of this paper.

In this paper we propose a kernel-based orthogonal QR (KBOQR) model with EIV by applying quantile loss function of orthogonal residuals to the formulation of support vector QR (SVQR) of Takeuchi and Furuhashi (2004). Unlike He and Liang (2000), the KBOQR avoids the assumption that the random errors in the response variable and the measurement errors in the input variables follow the same symmetric distribution. This is the first paper which utilizes the idea of support vector machine (SVM) for QR when the input variables have measurement errors. The SVM, first developed by Vapnik (1995) and his group at AT&T Bell Laboratories, has been successfully applied to a number of real world problems related to classification and regression problems. Takeuchi and Furuhashi (2004) first considered SVQR. Takeuchi et al. (2006) discussed several types of extensions including an approach to solve the quantile crossing problems, as well as a method to incorporate prior qualitative knowledge such as monotonicity constraints. Li et al. (2007) proposed a SVQR and derived a simple formula for the effective dimension of the SVQR model, which allows convenient selection of the hyperparameters.

The rest of this paper is organized as follows. Section 2 briefly describes the basic principle of orthogonal QR (OQR). Section 3 proposes the KBOQR and also presents a generalized cross validation (GCV) technique in order to choose the hyperparameters in the proposed KBOQR. Sections 4 and 5 present simulation study and conclusion, respectively.

2. Principle of OQR

In this section we briefly illustrate the principle of OQR which utilizes the quantile loss function of orthogonal residuals of Van Gorp et al. (2000).

Suppose that we have a sequence of samples $\left\{(\mbox{\boldmath$x$}_{i},y_{i}),i=1,\ldots,n\right\}$ satisfying the following model with EIV

$\displaystyle\begin{cases}y_{i}=f(\mbox{\boldmath$x$}_{i}^{*})+\epsilon_{i},&i% =1,\ldots,n,\\ \mbox{\boldmath$x$}_{i}=\mbox{\boldmath$x$}_{i}^{*}+\mbox{\boldmath$u$}_{i},% \end{cases}$ (1)

where $f$ is an unknown smooth regression function, $\mbox{\boldmath$x$}_{i}^{*}$ is $d\times 1$ vector of unknown and unobservable input variables, $\mbox{\boldmath$u$}_{i}$ is $d\times 1$ vector of measurement errors, and $\mbox{\boldmath$x$}_{i}$ is observed. Here, the random errors $\epsilon_{i}$ ’s are independent and identically distributed with zero mean and finite variance $\sigma_{\epsilon}^{2}$ . The measurement errors $\mbox{\boldmath$u$}_{i}$ ’s are independent and identically distributed with mean zero and covariance matrix $\mbox{\boldmath$\Sigma$}_{\mbox{\boldmath$u$}}$ . We assume that $\epsilon_{i}$ and $\mbox{\boldmath$u$}_{i}$ are uncorrelated. The recent literature has become aware of the inadequacy of assumption that $\epsilon_{i}$ and $\mbox{\boldmath$u$}_{i}$ have a joint distribution that is spherically symmetric, and proposes accounting for different uncertainties of the two types of errors. In general, it is usually assumed that $\mbox{\boldmath$\Sigma$}_{\mbox{\boldmath$u$}}$ is a diagonal matrix, i.e., $\mbox{\boldmath$\Sigma$}_{\mbox{\boldmath$u$}}=\textrm{diag}\{\sigma_{u_{1}}^{% 2},\ldots,\sigma_{u_{d}}^{2}\}$ , and $\sigma_{u_{i}}^{2}$ ’s are different from $\sigma_{\epsilon}^{2}$ .

In typical statistical modeling, the form of $f(\cdot)$ is not known exactly and thus the set of functions $f(\cdot)$ is given in a parametric form $\{f(\cdot;\mbox{\boldmath$\theta$}),\mbox{\boldmath$w$}\in\Theta\}$ . Here $\theta$ is a parameter vector from some parameter set $\Theta$ . The study of only parametric sets of functions is not a restriction on the problem, since the set $\Theta$ , to which the parameter $\theta$ belongs, is arbitrary (Vapnik 1998). Thus we describe the principle of OQR through using $f(\cdot;\mbox{\boldmath$\theta$})$ .

In the case of least squares estimation for the conditional mean, some authors proposed methods for correction of the measurement error (Van Gorp et al. 2000; Carroll et al. 2006). For convenience of illustration, we restrict ourselves to the case that $\mbox{\boldmath$x$}_{i}^{*}$ ’s are fixed unknown. Orthogonal residual argument of Van Gorp et al. (2000) leads to the orthogonal regression (OR) problem such as

$\displaystyle\arg\min_{\mbox{\boldmath$w$}}\sum_{i=1}^{n}\left(\frac{y_{i}-f(% \mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})}{\displaystyle\sqrt{\sigma_{% \epsilon}^{2}+\sum_{j=1}^{d}\left(\frac{\partial f(\mbox{\boldmath$x$}_{i};% \mbox{\boldmath$\theta$})}{\partial x_{ij}}\right)^{2}\sigma_{u_{j}}^{2}}}% \right)^{2},$ (2)

where $x_{ij}$ is the $j$ th element of $\mbox{\boldmath$x$}_{i}$ . We note that this OR problem does not involve the true $\mbox{\boldmath$x$}_{i}^{*}$ values and so there is no need to estimate these true values. Here,

$\left(y_{i}-f(\mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})\right)\Bigg{/}% \sqrt{\sigma_{\epsilon}^{2}+\sum_{j=1}^{d}\left(\frac{\partial f(\mbox{% \boldmath$x$}_{i};\mbox{\boldmath$\theta$})}{\partial x_{ij}}\right)^{2}\sigma% _{u_{j}}^{2}}$

can be considered as the orthogonal residual rather than the vertical distance in regression space. In order to implement the estimator given by the OR problem, the error variances’ ratios must be specified a priori. Hence, we posit the parameter $\nu_{j}=\sigma_{u_{j}}^{2}/\sigma_{\epsilon}^{2}$ that transforms the OR problem Eq. (2) into the following expression

$\displaystyle\arg\min_{\mbox{\boldmath$w$}}\sum_{i=1}^{n}\left(\frac{y_{i}-f(% \mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})}{\displaystyle\sqrt{1+\sum_{% j=1}^{d}\left(\frac{\partial f(\mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$% })}{\partial x_{ij}}\right)^{2}\nu_{j}}}\right)^{2}.$ (3)

For estimating the conditional quantile function we apply to orthogonal residuals the quantile loss function which is called check function and defined as

$\displaystyle\rho_{\tau}(r)=\tau rI{(r\geqslant 0)}+(\tau-1)rI{(r<0)},∼{}\tau% \in(0,1),$ (4)

where $I(\cdot)$ is the indicator function. Thus, we are led to specifying the $\tau$ th conditional quantile function of $y$ given $𝒙$ as $q_{\tau}(\mbox{\boldmath$x$})=f(\mbox{\boldmath$x$};\mbox{\boldmath$\theta$}_{% \tau})$ , and to consideration of $\hat{\mbox{\boldmath$\theta$}}_{\tau}$ solving the following OQR problem

$\displaystyle\arg\min_{\mbox{\boldmath$\theta$}}\sum_{i=1}^{n}\rho_{\tau}\left% (\frac{y_{i}-f(\mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})}{% \displaystyle\sqrt{1+\sum_{j=1}^{d}\left(\frac{\partial f(\mbox{\boldmath$x$}_% {i};\mbox{\boldmath$\theta$})}{\partial x_{ij}}\right)^{2}\nu_{j}}}\right).$ (5)

For our purpose we now reexpress the OQR problem Eq. (5) as follows. Since the check function ${\rho}_{\tau}(\cdot)$ in Eq. (4) can be written as

$\displaystyle\rho_{\tau}(r_{i})=w_{i}(\tau){r_{i}^{2}},$ (6)

where

$w_{i}(\tau)=\frac{\tau}{|r_{i}|}I{(r_{i}>0)}+\frac{(1-\tau)}{|r_{i}|}I{(r_{i}<% 0)},$

the OQR problem Eq. (5) can be written as

$\displaystyle\arg\min_{\mbox{\boldmath$\theta$}}\sum_{i=1}^{n}w_{i}(\tau)v_{i}% \left(y_{i}-f(\mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})\right)^{2},$ (7)

where $v_{i}$ is defined as

$\displaystyle\frac{1}{\displaystyle\sqrt{1+\sum_{j=1}^{d}\left(\frac{\partial f% (\mbox{\boldmath$x$}_{i};\mbox{\boldmath$\theta$})}{\partial x_{ij}}\right)^{2% }\nu_{j}}}.$ (8)

In the next section we will use Eq. (7) instead of Eq. (5) when deriving the KBOQR.

3. KBOQR and model selection

In this section we present a learning algorithm and a model selection procedure for KBOQR. For convenience, we illustrate the KBOQR under the setting that $\sigma_{u_{j}}^{2}=\sigma_{u}^{2}$ , i.e., $\nu_{j}=\nu$ for $j=1,\ldots,d$ . The development of KBOQR is straightforward. From now on we denote the QR function at quantile level $\tau$ given $𝒙$ by $q_{\tau}(\mbox{\boldmath$x$})$ . We basically use an iterative method for learning KBOQR.

3.1 Learning algorithm of KBOQR

Using the connection between Reproducing Kernel Hilbert Space (RKHS) and feature spaces we write the model

$\displaystyle f(\mbox{\boldmath$x$};\mbox{\boldmath$\theta$})=\mbox{\boldmath$% w$}^{t}\mbox{\boldmath$\phi$}(\mbox{\boldmath$x$})+b,$ (9)

where the nonlinear function $\mbox{\boldmath$\phi$}(\cdot):R^{d}\rightarrow R^{d_{h}}$ maps the input space to a so-called higher dimensional feature space, and the term $b$ is a bias term. It is important to note that the dimension $d_{h}$ of feature space is only defined in an implicit way. Then the KBOQR optimization problem for the $\tau$ th QR function becomes as follows:

$\displaystyle(\hat{\mbox{\boldmath$w$}},\hat{b})=\arg\min_{(\mbox{\boldmath$w$% },b)}\frac{\gamma}{2}{\mbox{\boldmath$w$}}^{t}{\mbox{\boldmath$w$}}+\frac{1}{2% }\sum_{i=1}^{n}{w_{i}(\tau)}v_{i}(y_{i}-\mbox{\boldmath$w$}^{t}\mbox{\boldmath% $\phi$}(\mbox{\boldmath$x$}_{i})-b)^{2},$ (10)

where $\gamma>0$ is the penalty parameter. This KBOQR optimization problem Eq. (10) is constructed by incorporating the model Eq. (9) into the optimization problem Eq. (7) and utilizing the regularization technique of SVM.

We now propose an iterative procedure for learning KBOQR. The idea is to obtain $𝒘$ and $b$ at the current iteration step through the KBOQR optimization problem Eq. (10) based on $w_{i}(\tau)$ and $v_{i}$ obtained via the previous iteration step. Under this idea the loss function associated with Eq. (10) becomes a weighted convex loss function. Therefore, by representer theorem (see Theorem 5.5 of Steinwart & Christmann (2008)), the $\tau$ th QR function becomes

$\displaystyle q_{\tau}(\mbox{\boldmath$x$})=\sum_{i=1}^{n}\alpha_{i}K(\mbox{% \boldmath$x$},\mbox{\boldmath$x$}_{i})+b,$ (11)

where $\alpha_{i}$ ’s are the dual parameters, $\mbox{\boldmath$\alpha$}=(\alpha_{1}\ldots,\alpha_{n})^{t}$ , and $K(\cdot,\cdot)=\mbox{\boldmath$\phi$}(\cdot)^{t}\mbox{\boldmath$\phi$}(\cdot)$ is a kernel function obtained from the Mercer’s condition (Mercer, 1909). We notice that $q_{\tau}({\mbox{\boldmath$x$}})$ depends implicitly on $\tau$ through $(\alpha_{i},b)$ depending on $\tau$ . Several choices of the kernel function are possible. One popular choice of kernel function in practice is Gaussian kernel

$\displaystyle K(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$x$}_{j})=\exp\left(-\|% \mbox{\boldmath$x$}_{i}-\mbox{\boldmath$x$}_{j}\|^{2}/2\kappa\right),\;i,j=1,% \ldots,n,$ (12)

where $\kappa>0$ is prespecified kernel parameter. Throughout the paper we use this Gaussian kernel.

Let $𝑲$ be the $n\times n$ kernel matrix with $(i,j)$ th elements $K_{ij}=K(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$x$}_{j})$ and $\mbox{\boldmath$K$}_{i}$ be the $i$ th row of the kernel matrix $𝑲$ . Then the KBOQR optimization problem Eq. (10) for the $\tau$ th QR function becomes as follows:

$\displaystyle(\hat{\mbox{\boldmath$\alpha$}},\hat{b})=\arg\min_{(\mbox{% \boldmath$\alpha$},b)}\frac{\gamma}{2}{\mbox{\boldmath$\alpha$}}^{t}\mbox{% \boldmath$K$}{\mbox{\boldmath$\alpha$}}+\frac{1}{2}\sum_{i=1}^{n}{w_{i}(\tau)}% v_{i}(y_{i}-\mbox{\boldmath$K$}_{i}{\mbox{\boldmath$\alpha$}}-b)^{2},$ (13)

where

$\displaystyle w_{i}(\tau)=\frac{\tau}{|y_{i}-\mbox{\boldmath$K$}_{i}\mbox{% \boldmath$\alpha$}-b|}∼{}I{(|y_{i}-\mbox{\boldmath$K$}_{i}{\mbox{\boldmath$% \alpha$}}-b|>0)}+\frac{(1-\tau)}{|y_{i}-\mbox{\boldmath$K$}_{i}{\mbox{% \boldmath$\alpha$}}-b|}∼{}I{(|y_{i}-\mbox{\boldmath$K$}_{i}{\mbox{\boldmath$% \alpha$}}-b|<0)},$ $\displaystyle v_{i}=\frac{1}{\sqrt{1+\nu\mbox{\boldmath$\alpha$}^{t}{\mbox{% \boldmath$\dot{K}$}_{i}}^{t}\mbox{\boldmath$\dot{K}$}_{i}\mbox{\boldmath$% \alpha$}}}.$

Here $\mbox{\boldmath$\dot{K}$}_{i}^{l}$ and $\mbox{\boldmath$\dot{K}$}_{i}$ are $1\times n$ vector and $d\times n$ matrix related with the differentiations of Gaussian kernel Eq. (12) with regard to $x_{il}$ , which are defined as

$\displaystyle\mbox{\boldmath$\dot{K}$}_{i}^{l}=\left(-\frac{1}{\kappa}(x_{il}-% x_{1l})K_{i1},\ldots,-\frac{1}{\kappa}(x_{il}-x_{nl})K_{in}\right),∼{}∼{}l=1,% \ldots,d,$ $\displaystyle\mbox{\boldmath$\dot{K}$}_{i}=\left({\mbox{\boldmath$\dot{K}$}_{i% }^{1}}^{t},\ldots,{\mbox{\boldmath$\dot{K}$}_{i}^{d}}^{t}\right)^{t}.$

We now describe an iterative re-weighted least squares (IRWLS) procedure for solving the minimization problem Eq. (13). Similar IRWLS procedures were used in Shim and Hwang (2009) and Reiss and Huang (2012). Given the $k$ th iteration estimate $(\hat{\mbox{\boldmath$\alpha$}}^{(k)},\hat{b}^{(k)})$ , the updated estimate is

$\displaystyle(\hat{\mbox{\boldmath$\alpha$}}^{(k+1)},\hat{b}^{(k+1)})=\arg\min% _{(\mbox{\boldmath$\alpha$},b)}\frac{\gamma}{2}{\mbox{\boldmath$\alpha$}}^{t}% \mbox{\boldmath$K$}{\mbox{\boldmath$\alpha$}}+\frac{1}{2}\sum_{i=1}^{n}{w_{i}^% {(k)}(\tau)}v_{i}^{(k)}(y_{i}-\mbox{\boldmath$K$}_{i}{\mbox{\boldmath$\alpha$}% }-b)^{2}$ (14)

with weights $w_{1}^{(k)}(\tau),\ldots,w_{n}^{(k)}(\tau),v_{1}^{(k)},\ldots,v_{n}^{(k)}$ chosen so that the following estimating equations for this minimization problem Eq. (14),

$\displaystyle\mbox{\boldmath$0$}=\gamma\mbox{\boldmath$K$}\mbox{\boldmath$% \alpha$}-\mbox{\boldmath$K$}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath$V$}^{(k)}% \mbox{\boldmath$y$}+\mbox{\boldmath$K$}\mbox{\boldmath$W$}^{(k)}\mbox{% \boldmath$V$}^{(k)}\mbox{\boldmath$K$}\mbox{\boldmath$\alpha$}+b\mbox{% \boldmath$K$}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath$V$}^{(k)}\mbox{\boldmath% $1$}$ (15) $\displaystyle 0=\mbox{\boldmath$1$}^{t}\mbox{\boldmath$W$}^{(k)}\mbox{% \boldmath$V$}^{(k)}\mbox{\boldmath$y$}-\mbox{\boldmath$1$}^{t}\mbox{\boldmath$% W$}^{(k)}\mbox{\boldmath$V$}^{(k)}\mbox{\boldmath$K$}\mbox{\boldmath$\alpha$}-% b\mbox{\boldmath$1$}^{t}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath$V$}^{(k)}% \mbox{\boldmath$1$},$ (16)

are approximately equivalent to the estimating equations for the minimization problem Eq. (13). Here $1$ represents all one column vector of $n$ dimension, $\mbox{\boldmath$W$}^{(k)}$ denotes the diagonal matrix with the $i$ th diagonal element $w_{i}^{(k)}(\tau)$ , and $\mbox{\boldmath$V$}^{(k)}$ denotes the diagonal matrix with the $i$ th diagonal element $v_{i}^{(k)}$ . The estimating Eqs (15) and (16) can be written in a simpler form as follows:

$\displaystyle\begin{pmatrix}{\mbox{\boldmath$\alpha$}}\\ b\end{pmatrix}=\begin{pmatrix}\mbox{\boldmath$K$}\mbox{\boldmath$W$}^{(k)}% \mbox{\boldmath$V$}^{(k)}\mbox{\boldmath$K$}+\gamma\mbox{\boldmath$K$}&\mbox{% \boldmath$K$}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath$V$}^{(k)}\mbox{\boldmath% $1$}\\ \mbox{\boldmath$1$}^{t}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath$V$}^{(k)}\mbox% {\boldmath$K$}&\mbox{\boldmath$1$}^{t}\mbox{\boldmath$W$}^{(k)}\mbox{\boldmath% $V$}^{(k)}\mbox{\boldmath$1$}\end{pmatrix}^{-1}\begin{pmatrix}{\mbox{\boldmath% $K$}}{\mbox{\boldmath$W$}^{(k)}}{\mbox{\boldmath$V$}^{(k)}}\\ {\mbox{\boldmath$1$}^{t}}{\mbox{\boldmath$W$}^{(k)}}{\mbox{\boldmath$V$}^{(k)}% }\end{pmatrix}{\mbox{\boldmath$y$}}$ (17)

Since the solutions to the linear equation system Eq. (17) cannot be obtained in a single step, we need to apply an iterative method which starts with initialized values of $\alpha$ and $b$ . Summing up, we describe the algorithm for learning the KBOQR for given $\nu$ and hyperparameters at quantile level $\tau$ as follows:

0. 0.

Set the initial values $\hat{\mbox{\boldmath$\alpha$}}^{(0)}$ and $\hat{b}^{(0)}$ .

Calculate $\mbox{\boldmath$W$}^{(k)}$ and $\mbox{\boldmath$V$}^{(k)}$ with $(\hat{\mbox{\boldmath$\alpha$}}^{(k)},\hat{b}^{(k)})$ obtained at the $k$ th iteration.

Obtain $\hat{\mbox{\boldmath$\alpha$}}^{(k+1)}$ and $\hat{b}^{(k+1)}$ from Eq. (17).

Iterate steps until convergence.

The algorithm is iterated until the following stop criterion is satisfied:

$\displaystyle\frac{1}{n+1}\left\|\left(\begin{array}[]{l}\hat{\mbox{\boldmath$% \alpha$}}^{(k)}\\ \hat{b}^{(k)}\end{array}\right)-\left(\begin{array}[]{l}\hat{\mbox{\boldmath$% \alpha$}}^{(k+1)}\\ \hat{b}^{(k+1)}\end{array}\right)\right\|^{2}<\epsilon,$

where $n+1$ is the number of parametres and $\epsilon=10^{-4}$ is taken as the tolerance level. The algorithm converges somewhat fast according to our experience. Refer to Pérez-Cruz et al. (2005) for convergence proof of the IRWLS algorithm to SVM.

3.2 Model selection

We now illustrate the model selection method which chooses the appropriate values of the error variances’ ratio $\nu$ and the hyperparameters $\gamma,\kappa$ of KBOQR. The functional structure of KBOQR is characterized by $\nu$ , $\gamma$ and $\kappa$ . To choose $\nu$ , $\gamma$ and $\kappa$ of KBOQR we first need to consider the cross validation (CV) function as follows:

$\displaystyle CV(\mbox{\boldmath$\lambda$})=\frac{1}{n}\sum_{i=1}^{n}w_{i}(% \tau)\left(y_{i}-\hat{q}_{\tau}^{(-i)}({\mbox{\boldmath$x$}}_{i}|\mbox{% \boldmath$\lambda$})\right)^{2},$ (18)

where $\lambda$ is the set of parameters $\nu,\gamma,\kappa$ and $\hat{q}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})$ is the quntile function estimated without $i$ th observation. But the computational cost associated with CV function is formidable since $\hat{q}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})$ should be evaluated for each candidate set of parameters. By leaving-out-one lemma (Craven & Wahba 1979),

$\displaystyle({y}_{i}-{\hat{q}}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_{i}|\mbox{% \boldmath$\lambda$}))-(y_{i}-{\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{% \boldmath$\lambda$}))$ $\displaystyle={\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda% $})-{\hat{q}}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})% \simeq\frac{\partial{\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$% \lambda$})}{\partial y_{i}}(y_{i}-{\hat{q}}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_% {i}|\mbox{\boldmath$\lambda$})),$

we have

$\displaystyle(y_{i}-{\hat{q}}_{\tau}^{(-i)}(\mbox{\boldmath$x$}_{i}|\mbox{% \boldmath$\lambda$}))\simeq\frac{y_{i}-{\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i% }|\mbox{\boldmath$\lambda$})}{1-{\displaystyle\frac{\partial{\hat{q}}_{\tau}(% \mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})}{\partial y_{i}}}}.$

Then the ordinary cross validation (OCV) function can be obtained as

$\displaystyle\textit{OCV}({\mbox{\boldmath$\lambda$}})=\frac{1}{n}\sum_{i=1}^{% n}w_{i}(\tau)\left(\frac{y_{i}-{\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{% \boldmath$\lambda$})}{1-{\displaystyle\frac{\partial{\hat{q}}_{\tau}(\mbox{% \boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})}{\partial y_{i}}}}\right)^{2}=% \frac{1}{n}\sum_{i=1}^{n}w_{i}(\tau)\left(\frac{y_{i}-{\hat{q}}_{\tau}(\mbox{% \boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})}{1-h_{ii}}\right)^{2},$ (19)

where $𝑯$ is the hat matrix such that ${\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})=\mbox{% \boldmath$H$}\mbox{\boldmath$y$}$ with the $(i,j)$ th element $h_{ij}=\partial{\hat{q}}_{\tau}(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$% \lambda$})/\partial y_{j}$ and

$\displaystyle\mbox{\boldmath$H$}=(\mbox{\boldmath$K$},\mbox{\boldmath$1$})% \left(\begin{array}[]{cc}\mbox{\boldmath$KWVK$}+\gamma\mbox{\boldmath$K$}&∼{}∼% {}∼{}\mbox{\boldmath$KWV1$}\\ \mbox{\boldmath$1$}^{t}\mbox{\boldmath$WVK$}&\mbox{\boldmath$1$}^{t}\mbox{% \boldmath$WV1$}\end{array}\right)^{-1}\left(\begin{array}[]{l}\mbox{\boldmath$% KWV$}\\ \mbox{\boldmath$1$}^{t}\mbox{\boldmath$WV$}\end{array}\right).$ (20)

Replacing $h_{ii}$ by their average $tr(\mbox{\boldmath$H$})/n$ , the generalized cross validation (GCV) function can be obtained as

$\displaystyle GCV({\mbox{\boldmath$\lambda$}})=\frac{n}{\left(n-tr(\mbox{% \boldmath$H$})\right)^{2}}\sum_{i=1}^{n}w_{i}(\tau)\left(y_{i}-{\hat{q}}_{\tau% }(\mbox{\boldmath$x$}_{i}|\mbox{\boldmath$\lambda$})\right)^{2}.$ (21)

4. Simulation study

In this section we perform simulation study to understand the effects of measurement errors and to demonstrate the performance of KBOQR under different error distributions and quantile levels. We are concerned with the KBOQR in which $\mbox{\boldmath$x$}^{*}$ is 1-dimensional, 2-dimensional or 3-dimensional. We consider normal and $t$ -distributions for the associated error distributions. For the error variances’ ratio $\lambda$ , we only consider $\nu=$ 0.75, 1.0, 1.25. We will focus $\tau=$ 0.1, 0.5, 0.9 for quantile levels. We compare the proposed KBOQR with SVQR by Li et al. (2007). This competing model does not consider measurement errors in input variables.

4.1 Design

We generate 100 data sets of size 50 from each of the following 3 nonlinear EIV models:

•
Model 1: $y_{i}=1+\sin(x_{i}^{})+\epsilon_{i}$ , $x_{i}=x_{i}^{}+u_{i}$ , $x_{i}^{}\sim\;i.i.d.\;U(-\pi,\pi)$
•
Model 2: $y_{i}=x_{1i}^{}\exp(-0.5x_{2i}^{2})+\epsilon_{i}$ , $\mbox{\boldmath$x$}_{i}=\mbox{\boldmath$x$}_{i}^{}+\mbox{\boldmath$u$}_{i}$ , $x_{1i}^{},x_{2i}^{}\sim\;i.i.d.\;U(0,2)$
•
Model 3: $y_{i}=0.5\left(4.5-64x_{1i}^{2}(1-x_{1i}^{})^{2}-16(x_{1i}^{}-0.5)^{2}% \right)+0.4\exp(x_{2i}^{}/2)+0.7\sin(\pi x_{3i}^{}/4)+\epsilon$ , $\mbox{\boldmath$x$}_{i}=\mbox{\boldmath$x$}_{i}^{}+\mbox{\boldmath$u$}_{i}$ , $x_{1i}^{}\sim\;i.i.d.\;U(0,1)$ , $x_{2i}^{}\sim\;i.i.d.\;N(0,1)$ , $x_{3i}^{}\sim\;i.i.d.\;3[U(0,1)]^{\frac{1}{3}}$

Here, we assume that $\epsilon_{i}$ and $u_{i}$ are independent of each other. For distribution of $\epsilon_{i}$ , we consider normal distribution $N(0,\sigma_{\epsilon}^{2})$ and $t$ -distribution $\frac{\sigma_{\epsilon}}{\sqrt{2}}t_{4}$ , where $\sigma_{\epsilon}^{2}=0.1$ . For distribution of $u_{i}$ or each component of $\mbox{\boldmath$u$}_{i}$ , we also consider normal distribution $N(0,\sigma_{u}^{2})$ and $t$ -distribution $\frac{\sigma_{u}}{\sqrt{2}}t_{4}$ , where $\sigma_{u}^{2}$ is determined such that $\nu=\sigma_{u}^{2}/\sigma_{\epsilon}^{2}$ for given $\nu$ . Although there are 4 combinations of distributions for two errors $\sigma_{u}^{2}$ and $\sigma_{\epsilon}^{2}$ , we here consider only two combinations $\left(N(0,\sigma_{u}^{2}),N(0,\sigma_{\epsilon}^{2})\right)$ and $\left(\frac{\sigma_{u}}{\sqrt{2}}t_{4},\frac{\sigma_{\epsilon}}{\sqrt{2}}t_{4}\right)$ because the results show very similar pattern.

For each simulated data set we compare the proposed KBOQR with SVQR. We are basically interested in estimating $q_{\tau}(\mbox{\boldmath$x$}^{})$ which is the $\tau$ th quantile of $y$ conditional on $\mbox{\boldmath$x$}^{}$ , the true unobserved value of the input vector. Thus, for comparison we calculate the mean and standard deviation of 100 mean square errors (MSEs) for each estimated QR function as follows:

$\displaystyle\textit{MSE}={1\over 50}\sum_{i=1}^{50}\left(q_{\tau}(\mbox{% \boldmath$x$}_{i}^{})-\hat{q}_{\tau}(\mbox{\boldmath$x$}_{i}^{})\right)^{2}.$ (22)

For 3 nonlinear EIV models the $\tau$ th QR functions of $y$ are given as follows:

$\displaystyle q_{\tau}(x^{})=1+\sin(x^{})+\sigma_{\epsilon}F_{\epsilon}^{-1}% (\tau),$ $\displaystyle q_{\tau}(\mbox{\boldmath$x$}^{})=x_{1}^{}\exp(-0.5x_{2}^{2})+% \sigma_{\epsilon}F_{\epsilon}^{-1}(\tau),$ $\displaystyle q_{\tau}(\mbox{\boldmath$x$}^{})=0.5\left(4.5-64x_{1i}^{2}(1-x% _{1i}^{})^{2}-16(x_{1i}^{}-0.5)^{2}\right)+0.4\exp(x_{2i}^{}/2)+0.7\sin(\pi x% _{3i}^{}/4)+\sigma_{\epsilon}F_{\epsilon}^{-1}(\tau),$

where $F_{\epsilon}^{-1}(\cdot)$ is the $\tau$ th QR of $\epsilon$ .

Table 1
Comparison of MSEs for 100 $\hat{q}_{\tau}$ ’s for training and test data for Model 1

$\nu$ Distribution Method Training Test

$(u,\epsilon)$ $\hat{q}_{\tau}$ $\hat{q}_{\tau}$

0.1 0.5 0.9 0.1 0.5 0.9

0.75 $(N,N)$ KBOQR 0.0654 0.0237 0.0648 0.0761 0.0274 0.0735

(0.0045) (0.0013) (0.0040) (0.0059) (0.0018) (0.0043)

SVQR 0.1060 0.0535 0.1041 0.1348 0.0710 0.1309

(0.0051) (0.0035) (0.0078) (0.0088) (0.0055) (0.0103)

$(t_{4},t_{4})$ KBOQR 0.1777 0.0353 0.1635 0.1838 0.0411 0.1773

(0.0215) (0.0017) (0.0169) (0.0223) (0.0024) (0.0215)

SVQR 0.2563 0.0911 0.2318 0.2949 0.1156 0.2791

(0.0310) (0.0111) (0.0172) (0.0391) (0.0155) (0.0320)

1.0 $(N,N)$ KBOQR 0.0690 0.0262 0.0705 0.0820 0.0300 0.0779

(0.0043) (0.0014) (0.0045) (0.0061) (0.0019) (0.0047)

SVQR 0.1165 0.0638 0.1157 0.1465 0.0837 0.1384

(0.0060) (0.0053) (0.0068) (0.0121) (0.0084) (0.0101)

$(t_{4},t_{4})$ KBOQR 0.1702 0.0382 0.1640 0.1774 0.0444 0.1729

(0.0214) (0.0018) (0.0172) (0.0221) (0.0025) (0.0215)

SVQR 0.2678 0.0813 0.2439 0.3074 0.0962 0.2728

(0.0295) (0.0054) (0.0182) (0.0353) (0.0071) (0.0281)

1.25 $(N,N)$ KBOQR 0.0782 0.0297 0.0690 0.0919 0.0349 0.0780

(0.0049) (0.0016) (0.0042) (0.0066) (0.0025) (0.0045)

SVQR 0.1278 0.0669 0.1244 0.1565 0.0834 0.1396

(0.0060) (0.0043) (0.0068) (0.0111) (0.0057) (0.0074)

$(t_{4},t_{4})$ KBOQR 0.1567 0.0418 0.1635 0.1591 0.0492 0.1766

(0.0140) (0.0021) (0.0166) (0.0130) (0.0029) (0.0218)

SVQR 0.2643 0.0912 0.2509 0.2846 0.1083 0.2770

(0.0243) (0.0058) (0.0190) (0.0238) (0.0071) (0.0266)

Table 2
Comparison of MSEs for 100 $\hat{q}_{\tau}$ ’s for training and test data for Model 2

$\nu$ Distribution Method Training Test

$(u,\epsilon)$ $\hat{q}_{\tau}$ $\hat{q}_{\tau}$

0.1 0.5 0.9 0.1 0.5 0.9

0.75 $(N,N)$ KBOQR 0.0565 0.0275 0.0472 0.0633 0.0322 0.0525

(0.0026) (0.0014) (0.0029) (0.0034) (0.0019) (0.0035)

SVQR 0.2093 0.1327 0.2222 0.2839 0.1802 0.2807

(0.0086) (0.0127) (0.0099) (0.0159) (0.0179) (0.0160)

$(t_{4},t_{4})$ KBOQR 0.1143 0.0389 0.0920 0.1306 0.0453 0.1036

(0.0089) (0.0019) (0.0084) (0.0107) (0.0024) (0.0099)

SVQR 0.3466 0.2052 0.3643 0.3975 0.2570 0.4244

(0.0147) (0.0195) (0.0189) (0.0185) (0.0245) (0.0233)

1.0 $(N,N)$ KBOQR 0.0640 0.0310 0.0524 0.0720 0.0364 0.0598

(0.0031) (0.0015) (0.0030) (0.0039) (0.0020) (0.0040)

SVQR 0.2283 0.1810 0.2643 0.2909 0.2361 0.3192

(0.0098) (0.0177) (0.0140) (0.0173) (0.0270) (0.0242)

$(t_{4},t_{4})$ KBOQR 0.1179 0.0451 0.0951 0.1286 0.0525 0.1050

(0.0083) (0.0023) (0.0086) (0.0096) (0.0029) (0.0096)

SVQR 0.3852 0.2406 0.3677 0.4346 0.2824 0.4261

(0.0193) (0.0262) (0.0176) (0.0220) (0.0249) (0.0220)

1.25 $(N,N)$ KBOQR 0.0742 0.0354 0.0589 0.0801 0.0414 0.0662

(0.0038) (0.0016) (0.0038) (0.0044) (0.0022) (0.0048)

SVQR 0.2324 0.1804 0.2734 0.2980 0.2244 0.3289

(0.0118) (0.0155) (0.0151) (0.0199) (0.0224) (0.0325)

$(t_{4},t_{4})$ KBOQR 0.1279 0.0505 0.0985 0.1379 0.0586 0.1091

(0.0091) (0.0026) (0.0086) (0.0106) (0.0034) (0.0095)

SVQR 0.4119 0.2968 0.4184 0.4598 0.3342 0.4731

(0.0209) (0.0343) (0.0244) (0.0234) (0.0336) (0.0280)

Table 3
Comparison of MSEs for 100 $\hat{q}_{\tau}$ ’s for training and test data for Model 3

$\nu$ Distribution Method Training Test

$(u,\epsilon)$ $\hat{q}_{\tau}$ $\hat{q}_{\tau}$

0.1 0.5 0.9 0.1 0.5 0.9

0.75 $(N,N)$ KBOQR 0.0677 0.0505 0.0597 0.0816 0.0689 0.0771

(0.0023) (0.0013) (0.0018) (0.0034) (0.0026) (0.0032)

SVQR 0.3955 0.3517 0.3682 0.1348 0.0710 0.1309

(0.0140) (0.0190) (0.0148) (0.0168) (0.0336) (0.0206)

$(t_{4},t_{4})$ KBOQR 0.0952 0.0642 0.1002 0.1038 0.0761 0.1045

(0.0043) (0.0018) (0.0091) (0.0049) (0.0029) (0.0073)

SVQR 0.7402 0.6848 0.7115 0.7987 0.8251 0.8149

(0.0344) (0.0374) (0.0312) (0.0411) (0.0487) (0.0483)

1.0 $(N,N)$ KBOQR 0.0684 0.0520 0.0594 0.0816 0.0700 0.0762

(0.0024) (0.0014) (0.0016) (0.0032) (0.0027) (0.0030)

SVQR 0.3944 0.3756 0.3992 0.4662 0.4914 0.4783

(0.0124) (0.0200) (0.0154) (0.0149) (0.0293) (0.0164)

$(t_{4},t_{4})$ KBOQR 0.0981 0.0667 0.0942 0.1063 0.0779 0.1004

(0.0045) (0.0020) (0.0062) (0.0049) (0.0030) (0.0054)

SVQR 0.7258 0.6880 0.6981 0.7518 0.7314 0.7409

(0.0353) (0.0389) (0.0278) (0.0336) (0.0365) (0.0329)

1.25 $(N,N)$ KBOQR 0.0693 0.0532 0.0601 0.0817 0.0709 0.0766

(0.0024) (0.0014) (0.0016) (0.0032) (0.0027) (0.0029)

SVQR 0.4021 0.3554 0.4122 0.4651 0.4629 0.4955

(0.0126) (0.0174) (0.0150) (0.0142) (0.0235) (0.0180)

$(t_{4},t_{4})$ KBOQR 0.0962 0.0690 0.0955 0.1059 0.0798 0.1032

(0.0039) (0.0021) (0.0058) (0.0047) (0.0031) (0.0056)

SVQR 0.7192 0.7083 0.7326 0.7623 0.7850 0.7486

(0.0313) (0.0485) (0.0304) (0.0358) (0.0522) (0.0302)

The way of computing MSEs for training and test data sets can be explained as follows. First, we obtain the estimated QR function $\hat{q}_{\tau}(\cdot)$ using each training data set consisting of 50 noisy input and output pairs $(\mbox{\boldmath$x$}_{i},y_{i})$ ’s. Then, for the training data set we compute MSE using 50 noiseless input $\mbox{\boldmath$x$}_{i}^{}$ ’s already generated. For each test data set we generate again 50 $\mbox{\boldmath$x$}_{i}^{}$ ’s totally different from $\mbox{\boldmath$x$}_{i}^{*}$ ’s for the training data, and then compute MSE using these new noiseless input data.
4.2 Results

$\nu$	Distribution	Method	Training	Test
	$(u,\epsilon)$			$\hat{q}_{\tau}$			$\hat{q}_{\tau}$
			0.1	0.5	0.9	0.1	0.5	0.9
0.75	$(N,N)$	KBOQR	0.0654	0.0237	0.0648	0.0761	0.0274	0.0735
			(0.0045)	(0.0013)	(0.0040)	(0.0059)	(0.0018)	(0.0043)
		SVQR	0.1060	0.0535	0.1041	0.1348	0.0710	0.1309
			(0.0051)	(0.0035)	(0.0078)	(0.0088)	(0.0055)	(0.0103)
	$(t_{4},t_{4})$	KBOQR	0.1777	0.0353	0.1635	0.1838	0.0411	0.1773
			(0.0215)	(0.0017)	(0.0169)	(0.0223)	(0.0024)	(0.0215)
		SVQR	0.2563	0.0911	0.2318	0.2949	0.1156	0.2791
			(0.0310)	(0.0111)	(0.0172)	(0.0391)	(0.0155)	(0.0320)
1.0	$(N,N)$	KBOQR	0.0690	0.0262	0.0705	0.0820	0.0300	0.0779
			(0.0043)	(0.0014)	(0.0045)	(0.0061)	(0.0019)	(0.0047)
		SVQR	0.1165	0.0638	0.1157	0.1465	0.0837	0.1384
			(0.0060)	(0.0053)	(0.0068)	(0.0121)	(0.0084)	(0.0101)
	$(t_{4},t_{4})$	KBOQR	0.1702	0.0382	0.1640	0.1774	0.0444	0.1729
			(0.0214)	(0.0018)	(0.0172)	(0.0221)	(0.0025)	(0.0215)
		SVQR	0.2678	0.0813	0.2439	0.3074	0.0962	0.2728
			(0.0295)	(0.0054)	(0.0182)	(0.0353)	(0.0071)	(0.0281)
1.25	$(N,N)$	KBOQR	0.0782	0.0297	0.0690	0.0919	0.0349	0.0780
			(0.0049)	(0.0016)	(0.0042)	(0.0066)	(0.0025)	(0.0045)
		SVQR	0.1278	0.0669	0.1244	0.1565	0.0834	0.1396
			(0.0060)	(0.0043)	(0.0068)	(0.0111)	(0.0057)	(0.0074)
	$(t_{4},t_{4})$	KBOQR	0.1567	0.0418	0.1635	0.1591	0.0492	0.1766
			(0.0140)	(0.0021)	(0.0166)	(0.0130)	(0.0029)	(0.0218)
		SVQR	0.2643	0.0912	0.2509	0.2846	0.1083	0.2770
			(0.0243)	(0.0058)	(0.0190)	(0.0238)	(0.0071)	(0.0266)

$\nu$	Distribution	Method	Training	Test
	$(u,\epsilon)$			$\hat{q}_{\tau}$			$\hat{q}_{\tau}$
			0.1	0.5	0.9	0.1	0.5	0.9
0.75	$(N,N)$	KBOQR	0.0565	0.0275	0.0472	0.0633	0.0322	0.0525
			(0.0026)	(0.0014)	(0.0029)	(0.0034)	(0.0019)	(0.0035)
		SVQR	0.2093	0.1327	0.2222	0.2839	0.1802	0.2807
			(0.0086)	(0.0127)	(0.0099)	(0.0159)	(0.0179)	(0.0160)
	$(t_{4},t_{4})$	KBOQR	0.1143	0.0389	0.0920	0.1306	0.0453	0.1036
			(0.0089)	(0.0019)	(0.0084)	(0.0107)	(0.0024)	(0.0099)
		SVQR	0.3466	0.2052	0.3643	0.3975	0.2570	0.4244
			(0.0147)	(0.0195)	(0.0189)	(0.0185)	(0.0245)	(0.0233)
1.0	$(N,N)$	KBOQR	0.0640	0.0310	0.0524	0.0720	0.0364	0.0598
			(0.0031)	(0.0015)	(0.0030)	(0.0039)	(0.0020)	(0.0040)
		SVQR	0.2283	0.1810	0.2643	0.2909	0.2361	0.3192
			(0.0098)	(0.0177)	(0.0140)	(0.0173)	(0.0270)	(0.0242)
	$(t_{4},t_{4})$	KBOQR	0.1179	0.0451	0.0951	0.1286	0.0525	0.1050
			(0.0083)	(0.0023)	(0.0086)	(0.0096)	(0.0029)	(0.0096)
		SVQR	0.3852	0.2406	0.3677	0.4346	0.2824	0.4261
			(0.0193)	(0.0262)	(0.0176)	(0.0220)	(0.0249)	(0.0220)
1.25	$(N,N)$	KBOQR	0.0742	0.0354	0.0589	0.0801	0.0414	0.0662
			(0.0038)	(0.0016)	(0.0038)	(0.0044)	(0.0022)	(0.0048)
		SVQR	0.2324	0.1804	0.2734	0.2980	0.2244	0.3289
			(0.0118)	(0.0155)	(0.0151)	(0.0199)	(0.0224)	(0.0325)
	$(t_{4},t_{4})$	KBOQR	0.1279	0.0505	0.0985	0.1379	0.0586	0.1091
			(0.0091)	(0.0026)	(0.0086)	(0.0106)	(0.0034)	(0.0095)
		SVQR	0.4119	0.2968	0.4184	0.4598	0.3342	0.4731
			(0.0209)	(0.0343)	(0.0244)	(0.0234)	(0.0336)	(0.0280)

$\nu$	Distribution	Method	Training	Test
	$(u,\epsilon)$			$\hat{q}_{\tau}$			$\hat{q}_{\tau}$
			0.1	0.5	0.9	0.1	0.5	0.9
0.75	$(N,N)$	KBOQR	0.0677	0.0505	0.0597	0.0816	0.0689	0.0771
			(0.0023)	(0.0013)	(0.0018)	(0.0034)	(0.0026)	(0.0032)
		SVQR	0.3955	0.3517	0.3682	0.1348	0.0710	0.1309
			(0.0140)	(0.0190)	(0.0148)	(0.0168)	(0.0336)	(0.0206)
	$(t_{4},t_{4})$	KBOQR	0.0952	0.0642	0.1002	0.1038	0.0761	0.1045
			(0.0043)	(0.0018)	(0.0091)	(0.0049)	(0.0029)	(0.0073)
		SVQR	0.7402	0.6848	0.7115	0.7987	0.8251	0.8149
			(0.0344)	(0.0374)	(0.0312)	(0.0411)	(0.0487)	(0.0483)
1.0	$(N,N)$	KBOQR	0.0684	0.0520	0.0594	0.0816	0.0700	0.0762
			(0.0024)	(0.0014)	(0.0016)	(0.0032)	(0.0027)	(0.0030)
		SVQR	0.3944	0.3756	0.3992	0.4662	0.4914	0.4783
			(0.0124)	(0.0200)	(0.0154)	(0.0149)	(0.0293)	(0.0164)
	$(t_{4},t_{4})$	KBOQR	0.0981	0.0667	0.0942	0.1063	0.0779	0.1004
			(0.0045)	(0.0020)	(0.0062)	(0.0049)	(0.0030)	(0.0054)
		SVQR	0.7258	0.6880	0.6981	0.7518	0.7314	0.7409
			(0.0353)	(0.0389)	(0.0278)	(0.0336)	(0.0365)	(0.0329)
1.25	$(N,N)$	KBOQR	0.0693	0.0532	0.0601	0.0817	0.0709	0.0766
			(0.0024)	(0.0014)	(0.0016)	(0.0032)	(0.0027)	(0.0029)
		SVQR	0.4021	0.3554	0.4122	0.4651	0.4629	0.4955
			(0.0126)	(0.0174)	(0.0150)	(0.0142)	(0.0235)	(0.0180)
	$(t_{4},t_{4})$	KBOQR	0.0962	0.0690	0.0955	0.1059	0.0798	0.1032
			(0.0039)	(0.0021)	(0.0058)	(0.0047)	(0.0031)	(0.0056)
		SVQR	0.7192	0.7083	0.7326	0.7623	0.7850	0.7486
			(0.0313)	(0.0485)	(0.0304)	(0.0358)	(0.0522)	(0.0302)

Tables 1–3 show the results for the mean and standard deviation of 100 MSEs for each estimated QR function. Standard deviations are given in parenthesis. Boldfaced values indicate best performance/result in the particular categories of $\tau$ , $\nu$ and error distributions. We use a grid-search approach with GCV function Eq. (21) to find the optimal values of three parameters $\nu,\gamma,\kappa$ of the KBOQR. As seen from Tables 1–3, the proposed KBOQR yields the smaller means of MSEs and smaller standard deviations of MSEs for all cases. Therefore, the KBOQR performs better than SVQR in estimating QR function when the input variables are contaminated with noise.

5. Concluding remarks

In this paper, we dealt with estimating QR function of the nonlinear EIV model with KBOQR. We found that the KBOQR provides good results in estimating QR function for the given examples. The KBOQR also makes the model selection easier and faster than a leave-one-out cross validation or $k$ -fold cross validation technique by using GCV function. In general, the ODR analysis requires knowledge of $\nu_{j}$ ’s. As long as the error variances’ ratios $\nu_{j}$ ’s are specified correctly, the ODR fitting method is an acceptable method. By the way, the model selection process of KBOQR makes it possible to obtain the estimates of $\nu_{j}$ ’s. Thus, the proposed KBOQR appears to be useful in estimating QR function of the nonlinear EIV model.

To conclude, the KBOQR basically have two advantages. One is that this method takes over advantages that SVM works very well for a number of real world problems and overcomes the curse of dimensionality. Thus, the KBOQR can be applied easily and effectively to the nonlinear EIV model with high dimensional input vector. The other is that this method can estimate QR function without knowledge of $\nu_{j}$ ’s in advance, because the estimates of $\nu_{j}$ ’s are obtained during the model selection process.

Footnotes

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology with grant no. (NRF-2014R1A1A 2054917, NRF-2015R1D1A1A01056582). This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2015S1A3A2046715). The present research was conducted by the research fund of Dankook University in 2017.

References

Barnes

M. L.

, & Hughes

A. W.

(2002). A quantile regression analysis of the cross section of stock market returns. Working Paper 02-2, Federal Reserve Bank of Boston.

Carroll

R. J.

Ruppert

Stefanski

, & Crainiceanu

C. M.

(2006). Measurement error in nonlinear models: A modern perspective. Chapman & Hall/CRC, New York.

Chesher

(2001). Parameter approximations for quantile regressions with measurement error. Working Paper CWP02/01, University College London, Dept. of Economics.

Craven

, & Wahba

(1979). Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerical Mathematics, 31, 377-403.

, & Liang

(2000). Quantile regression estimates for a class of linear and partially linear errors-in-variables models. Statistica Sinica, 10, 129-140.

Ioannidesa

D. A.

, & Matzner-Løber

(2009). Regression quantiles with errors-in-variables. Journal of Nonparametric Statistics, 21(8), 1003-1015.

Kimeldorf

G. S.

, & Wahba

(1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and its Applications, 3, 82-95.

Koenker

(2005). Quantile regression. Cambridge University Press, London.

Kuhn

H. W.

, & Tucker

A. W.

(1951). Nonlinear programming. Proceedings of 2nd Berkeley Symposium, 481-492.

10.

Liu

, & Zhu

(2007). Quantile regression in reproducing kernel Hilbert spaces. Journal of the American Statistical Association, 102(477), 255-268.

11.

, & Yin

(2011). Censored quantile regression with covariate measurement errors. Statistica Sinica, 21, 949-971.

12.

Mercer

(1909). Function of positive and negative type and their connection with theory of integral equations. Philosophical Transactions of Royal Society A, 415-446.

13.

Montes-Rojas

(2011). Quantile regression with classical additive measurement errors. Economics Bulletin, 31(4), 2863-2868.

14.

Pérez-Cruz

Bousoño-Calzón

, & Artés-Rodríguez

(2005). Convergence of the IRWLS procedure to the support vector machine solution. Neural Computation, 17(1), 7-18.

15.

Reiss

P. T.

, & Huang

(2012). Smoothness selection for penalized quantile regression splines. The International Journal of Biostatistics, 8(1), 1.

16.

Schennach

S. M.

(2008). Quantile regression with mismeasured covariates. Econometric Theory, 24, 1010-1043.

17.

Shim

, & Hwang

(2009). Support vector censored quantile regression under random censoring. Computational Statistics and Data Analysis, 53(2), 912-919.

18.

Steinwart

, & Christmann

(2008). Support vector machines. Springer, New York.

19.

Takeuchi

, & Furuhashi

(2004). Non-crossing quantile regressions by SVM. Proceedings of 2004 IEEE International Joint Conference on Neural Networks, 401-406.

20.

Takeuchi

Q. V.

Sears

, & Smola

A. J.

(2006). Nonparametric quantile regression. Journal of Machine Learning Research, 7, 1231-1264.

21.

Van Gorp

Schoukens

, & Pintelon

(2000). Learning neural networks with noisy inputs using the errors-in-variables approach. IEEE Transactions on Neural Networks, 11(2), 402-414.

22.

Vapnik

(1995). The Nature of Statistical Learning Theory. Springer, Berlin.

23.

Vapnik

(1998). Statistical Learning Theory. John Wiley and Sons, New York.

24.

Wang

H. J.

Stefanski

L. A.

, & Zhu

(2012). Corrected-loss estimation for quantile regression with covariate measurement errors. Biometrika, 99(2), 405-421.

25.

Wei

, & Carroll

R. J.

(2009). Quantile regression with measurement error. Journal of the American Statistical Association, 104(487), 1129-1143.