An estimating parameter of nonparametric regression model based on smoothing techniques

Abstract

This paper studies the estimating parameter of a nonparametric regression model that consists of the function of independent variables and observation of dependent variables. The smoothing spline, penalized spline, and B-spline methods in a class of smoothing techniques are considered for estimating the unknown parameter on nonparametric regression model. These methods use a smoothing parameter to control the smoothing performance on data set by using a cross-validation method. We also compare these methods by fitting a nonparametric regression model on simulation data and real data. The nonlinear model is a simulation data which is generated in two different models in terms of mathematical function based on statistical distribution. According to the results, the smoothing spline, the penalized spline, and the B-spline methods have a good performance to fit nonlinear data by considering the hypothesis testing of biased estimator. However the penalized spline method shows the minimum mean square errors on two models. As real data, we use the data from a light detection and ranging (LIDAR) experiment that contained the range distance travelled before the light as an independent variable and the logarithm of the ratio of received light from two laser sources as a dependent variable. From the mean square errors of fitting data, the penalized spline again shows the minimum values.

Keywords

B-spline nonparametric regression penalized spline smoothing spline

1. Introduction

In statistical modelling, regression analysis is a statistical process for estimating parameters of the relationships between dependent and independent variables in terms of a regression function. However, regression analysis requires an assumption of the underlying regression function to be met. If an inappropriate assumption is used, it is possible to produce misleading results. To overcome this problem, the nonparametric regression is a choice to analyze data when the data are not meeting the assumption of regression analysis. The nonparametric regression is an alternative way for looking at scatter diagram smoothing to depict the relationship between dependent and independent variables. The single independent variable is called scatterplot smoothing it can be used to enhance the visual appearance to help our eyes pick out the trend in the plot.

The smoothing technique is a part of a method to estimate unknown parameters (trend or smoothing estimators) of nonparametric regression models. There are many popular smoothing techniques such as the smoothing spline [1, 2], the penalized spline [3], and the B-spline [4]. The estimating parameters of these methods depend on the smoothing parameter which is controlled the trade-off between fidelity to the data and roughness of function. Smoothing Spline (SS) is a technique that estimates the natural polynomial spline by minimizing the penalized sum of squares based on a smoothing parameter. The penalized Spline (PS) smoother is approximated by minimizing the truncated power function on a low rank thin-plate spline depended on the smoothing parameter. The concept of the B-spline is similar to the smoothing spline and penalized spline. This requires the piecewise constant B-spline that can be obtained from truncated counterparts by differencing the B-spline function.

In this paper, we consider the nonparametric regression model in Section 2, and use the smoothing spline, penalized spline, and B-spline methods to estimate the unknown parameter of nonparametric regression model in Section 3. In Sections 4 and 5, we show the estimation of these methods for simulation data and real data. The conclusion is presented in Section 6.

2. The nonparametric regression model

The nonparametric regression model consists of the cubic spline of piecewise polynomials function based on a function of independent variables ( $S(x_{t})$ ), error process ( $\varepsilon_{t}$ ), and dependent variables ( $y_{t}$ ) following

$\displaystyle y_{t}=S(x_{t})+\varepsilon_{t},t=1,2,3,...,n.$ (1)

The error process is assumed to follow the normal distribution with mean zero and variance one.

3. Method of smoothing techniques

The following smoothing techniques show the process to estimate parameters based on nonparametric regression model.

3.1 Smoothing spline method

Wahba [1] defined the natural polynomial spline $S(x_{t})=S_{K}^{m}(x_{t})$ as a real-valued function on $[{a,b}]$ with the aid of $K$ so-called knots $-\infty\leqslant a<x_{1}<x_{2}<...<x_{K}<b\leqslant\infty$ . The class of $m$ -order splines with domain $[{a,b}]$ will be denoted by $W^{m}[{a,b}]$ .

The natural measure associated with the function $f\in W^{m}[{a,b}]$ that used to measure the roughness of curve which is called the quadratic penalty function given by

$\displaystyle\int_{b}^{a}{\left\{{S^{(m)}({x_{t}})}\right\}^{2}}dx$ (2)

where $S^{(m)}({x_{t}})$ is the $m$ th derivative of $S(x)$ with respect to $x$ .

Consider the simple nonparametric regression model, to estimate $\hat{S}(\cdot)$ minimizes $S_{K}^{(m)}({x_{t}})$ over the class of function $S(\cdot)$ following

$\displaystyle S_{K}^{(m)}({x_{t}})=\min\sum\limits_{t=1}^{n}{\left\{{y_{t}-S({% x_{t}})}\right\}^{2}}{}+\lambda\int_{a}^{b}{\left\{{S^{({{}^{\prime\prime}})}(% {x_{t}})}\right\}^{2}}dx$ (3)

where $\lambda>$ 0 denotes a smoothing parameter. In this study, we emphasize $m=$ 2 so-called the natural cubic spline which is commonly considered in the statistical literature [2].

The natural cubic spline is given the value and second derivatives at each knots $y_{t}$ as

$\displaystyle\ut{$S$}=S({x_{t}})=\beta_{0}+\beta_{1}S_{1}(x)+\ldots\beta_{n+3}% S_{n+3}(x),$ $\displaystyle\ut{$\gamma$}=S^{{}^{\prime\prime}}({x_{t}}),\quad t=1,2,3,\ldots% ,n.$

Let $S$ be the vector $({S_{1},\ldots,S_{n+3}})^{\text{T}}$ and let $\gamma$ be the vector $({\gamma_{1},\ldots,\gamma_{n+3}})^{\text{T}}$ .

The condition of natural cubic spline depends on two matrices $Q$ and $R$ below

$\displaystyle Q=\left({{\begin{array}[]{*{20}c}{h_{1}^{-1}}&0&\cdots&0\\ \begin{array}[]{c}-h_{1}^{-1}-\\ h_{2}^{-1}\end{array}&{h_{2}^{-1}}&\cdots&0\\ {h_{2}^{-1}}&\begin{array}[]{c}-h_{2}^{-1}-\\ h_{3}^{-1}\end{array}&\cdots&0\\ 0&{h_{3}^{-1}}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&{h_{n-1}^{-1}}\\ \end{array}}}\right)_{(n)\times({n-2}),}$

where $h_{t}=x_{t+1}-x_{t}$ , for $t=1,2,\ldots,n$ , then $Q$ is a $n\times({n-2})$ matrix.

Matrix $R$ is a symmetric $({n-2})\times({n-2})$ matrix with elements below

$\displaystyle R=\left({{\begin{array}[]{*{20}c}\begin{array}[]{c}\frac{1}{3}(h% _{1}\\ +h_{3})\end{array}&{\frac{1}{6}h_{2}}&\cdots&0\\ {\frac{1}{6}h_{2}}&\begin{array}[]{c}\frac{1}{3}(h_{2}\\ +h_{3})\end{array}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\begin{array}[]{c}\frac{1}{3}(h_{n-2}\\ +h_{n-1})\end{array}\\ \end{array}}}\right)_{({n-2})\times({n-2}).}$

The matrix $K$ can be decomposed by

$\displaystyle K=QR^{-1}Q^{\text{T}}.$ (4)

The roughness penalty will satisfy

$\displaystyle\int\limits_{a}^{b}{\left\{{S^{{}^{\prime\prime}}({x_{t}})}\right% \}^{2}dx_{t}}=\gamma^{\text{T}}R\gamma=S^{\text{T}}KS=\Omega_{N}({i,j}).$ (5)

To illustrate, it can be written in matrix form introduced by [2] as residual sum of squares (RSS)

$\displaystyle\textit{RSS}=\sum\limits_{t=2}^{n}{\left\{{y_{t}-S({x_{t}})}% \right\}^{2}}=(\ut{$y$}-\ut{$S$})^{\text{T}}(\ut{$y$}-\ut{$S$}),$ (6)

where $\ut{$y$}\!=\!({y_{1},\ldots,y_{n}})^{\text{T}}$ and $S\!=\!({S({x_{1}}),\ldots,S({x_{n}})})^{\text{T}}$ . Letting $N$ be a matrix with $N(i,j)=S_{j}(x_{i})$ and $\ut{$S$}=N\beta$ .

The roughness penalty term $\int{S^{{}^{\prime\prime}2}}$ as $\Omega_{N}$ in Eq. (5) to obtain

$\displaystyle\hat{S}_{\lambda}({x_{t}})=(\ut{$y$}-N\beta)^{\text{T}}(\ut{$y$}-% X\beta)+\lambda\beta^{\text{T}}\Omega_{N}\beta$ (7)

It therefore follows that Eq. (7) has a unique minimum, other smoothing spline estimator is obtained by

$\displaystyle\hat{\beta}=(N^{\text{T}}N+\lambda\Omega_{N})^{-1}N^{\text{T}}\ut% {$y$},$

then

$\displaystyle\hat{S}_{\lambda}({x_{t}})=N(N^{\text{T}}N+\lambda\Omega_{N})^{-1% }N^{\text{T}}.$ (8)

In this paper, we also select the smoothing parameter using the method of generalized cross-validation (GCV) suggested by Wahba [5] and Craven and Wahba [6]. In practice, this step can be implemented by using the function of smooth.spline in the software R.

3.2 Penalized spline method

Eubank [7, 8] introduced the regression spline that the local neighbourhoods are specified by a group of locations:

$\displaystyle\tau_{0},\tau_{1},\tau_{2},...,\tau_{K},\tau_{K+1}$ (9)

in the range of interval $[{a,b}]$ , where $a=\tau_{0}<\tau_{1}<...<\tau_{K}<\tau_{K+1}<b$ . These locations are known as knots, and $\tau_{r},r=1,2,...,K$ are called interior knots.

A regression spline can be constructed using the $k$ -th degree truncated power basis or called the B-spline basis with $K$ knots $\tau_{1},\tau_{2},...,\tau_{K}$ :

$\displaystyle 1,x_{t},...,x_{t}^{k},({x_{t}-\tau_{1}})_{+}^{k},...,({x_{t}-% \tau_{K}})_{+}^{k},$ (10)

where $w_{+}^{k}$ denotes $k$ -th power of the positive part of $w$ where $w_{+}=\max({0,w})$ . The first $({k+1})$ basis functions of the truncated power basis Eq. (12) are polynomials of degree up to $k$ , and the others are all the truncated power functions of degree $k$ . A regression spline can be expressed as

$\displaystyle S({x_{t}})=\sum\limits_{s=0}^{k}{\beta_{s}x_{t}^{s}}+\sum\limits% _{r=1}^{K}\beta_{k+r}({x_{t}-\tau_{r}})_{+}^{k},$ (11)

where $\beta_{0},\beta_{1},...,\beta_{k+K}$ are the unknown coefficients to be estimated by a suitable loss minimization.

The penalized spline is a method to estimate a unknown smooth function using the truncated power function [9], and the penalized spline can be expressed as

$\displaystyle S({x_{t}})=\sum\limits_{j=0}^{m-1}{\alpha_{j}x_{s}^{j}}+\sum% \limits_{k=1}^{K}\beta_{k}({x_{t}-\tau_{k}})^{2m-1},$ (12)

where $\beta=({\beta_{1},...,\beta_{K}})^{\text{T}}\sim N\left({0,\sigma_{\beta}^{2}% \Omega^{-1/2}({\Omega^{1/2}})^{\text{T}}}\right)$ , and the $({l,k})$ th entry of $\Omega$ is $|{\tau_{1}-\tau_{k}}|^{2m-1}$ and only the coefficient of $|{x_{t}-\tau_{k}}|^{2m-1}$ are penalized so that a reasonably large order $K$ can be used.

In this case, we focus $m=$ 2, as the natural cubic spline, or called low-rank thin-plate spline which present of $S(\cdot)$ as

$\displaystyle S({x_{t}})=\alpha_{0}+\alpha_{1}x_{t}+\sum\limits_{k=1}^{K}\beta% _{k}|{x_{t}-\tau_{k}}|^{3},$ (13)

where $\theta=({\alpha_{0},\alpha_{1},\beta_{1},...,\beta_{K}})^{\text{T}}$ is the vector of regression coefficients, and $\tau_{1}<\tau_{2}<...<\tau_{K}$ are fixed knots. The number of knots, $K$ can be selected using a cross-validation method or information theoretic methods (e.g., BIC or AIC).

This class of penalized spline smoothers $({\hat{S}(\cdot)})$ may also be expressed as

$\displaystyle\hat{S}(x_{t})=C({C^{\text{T}}C+\lambda^{3}D})^{-1}C^{\text{T}}% \ut{$y$},$ (14)

where

$\displaystyle C=\left[{{\begin{array}[]{*{20}c}1&{x_{t}}&{|{x_{t}-\tau_{k}}|_{% 1\leqslant k\leqslant K}^{3}}\\ \end{array}}}\right]_{1\leqslant t\leqslant n},$ $\displaystyle D=\left[{{\begin{array}[]{*{20}l}{0_{2\times 2}}&{0_{2\times K}}% \\ {0_{K\times 2}}&{({\Omega_{K}^{1/2}})^{\text{T}}\Omega_{K}^{1/2}}\\ \end{array}}}\right],$

and $\lambda=\sigma_{\beta}^{2}/\sigma_{\varepsilon}^{2}$ is a smoothing parameter. The penalized spline smoothers are estimated by using the SemiPar package in the software R.

3.3 B-spline method

B-splines are very interesting as a basic function for univariate independent variable of nonparametric regression function. De Boor [10] gave an algorithm to compute B-spline of lower degree on piece wise polynomials function.

The $m^{\text{th}}$ degree of B-spline function are evaluated from $(m-1)^{\text{th}}$ degree as

$\displaystyle B_{j}^{m}(x)=\frac{x_{t}-\tau_{j}}{\tau_{j+m-1}-\tau_{j}}B_{j}^{% m-1}{}+\frac{\tau_{j+m}-x_{t}}{\tau_{j+m}-\tau_{j+1}}B_{j+1}^{m-1},$ (15)

where basis of order $m$ with knots $\{B_{i}^{m}|i=M-m+1,...,M+K\}$ , and auxiliary knots $\tau_{j}$ . B-splines base on non-zero over domain spanned by at most $M+1$ knots. In this case, we focus the $m=$ 4 or called the cubic B-spline with $K$ knots has basis expansion as

$\displaystyle S(x_{t})=\sum\limits_{j=1}^{K+4}B_{j}^{4}(x_{t})\beta_{j}.$

The nonparametric regression model can written in form of B-splines as

$\displaystyle y_{t}=\sum\limits_{j=1}^{K+4}B_{j}^{4}(x_{t})\beta_{j}+% \varepsilon_{t},t=1,2,3,...,n.$ (16)

Figure 1.

The scatter plot of dependent and independent variables on model 1.

In matrix form, B-splines can be written in form a linear model

$\displaystyle B=\left[{{\begin{array}[]{*{20}c}{B_{1}^{4}(x_{1})}&\ldots&{B_{K% +4}^{4}(x_{1})}\\ \vdots&&\vdots\\ {B_{1}^{4}(x_{n})}&\ldots&{B_{K+4}^{4}(x_{n})}\\ \end{array}}}\right],\ut{$y$}=\left[{{\begin{array}[]{*{20}c}{y_{1}}\\ {y_{2}}\\ \vdots\\ {y_{n}}\\ \end{array}}}\right],$ $\displaystyle\text{ and }\ut{$\varepsilon$}=\left[{{\begin{array}[]{*{20}c}{% \varepsilon_{1}}\\ {\varepsilon_{2}}\\ \vdots\\ {\varepsilon_{n}}\\ \end{array}}}\right].$

The B-splines estimators are approximated by least square problems as

$\displaystyle\ut{${\hat{\beta}}$}=\left[{{\begin{array}[]{*{20}c}{\hat{\beta}_% {1}}\\ {\hat{\beta}_{2}}\\ \vdots\\ {\hat{\beta}_{K+4}}\\ \end{array}}}\right]=({B^{\text{T}}B})^{-1}B^{\text{T}}\ut{$y$}.$

The B-spline and penalties are studied by Eilers and Marx [4] that advocate the use of the equally spaced knots, instead of the order statistics of the independent variable. The B-spline coefficients can be estimated as

$\displaystyle\ut{${\hat{\beta}}$}=({B^{\text{T}}B+\lambda D^{\text{T}}D})^{-1}% B^{\text{T}}\ut{$y$},$ (17)

where $D$ is a banded matric which correspond to the difference penalty and denote by

$\displaystyle D=\left[{\begin{array}[]{*{20}c}0&0&{0\ldots 0}&0\\ {-1}&1&{0\ldots 0}&0\\ 0&{-1}&{1\ldots 0}&0\\ {\ldots}&{\ldots}&{\ldots}&{\ldots}\\ 0&0&0\ldots-1&1\\ \end{array}}\right].$

The fitting cubic B-splines are $\hat{S}(x_{t})=\sum_{j=1}^{K+4}B_{j}^{4}\linebreak(x_{t})\hat{\beta}_{j}$ . The smoothing parameter $\lambda$ choosing by minimizing the ordinary function or the generalized cross-validation function.

Figure 2.

The scatter plot of dependent and independent variables on model 2.

Table 1

The summary statistics of simulation studies with model 1 based on smoothing spline (SS), penalized spline (PS), and B-spline (BS)

Sample sizes	Methods	Mean		S.D.		LCI		UCI		$t$ -statistic		$p$ -values
$n=$ 50	SS	$-$ 0.	3344	6.	5444	$-$ 0.	9095	0.	2405	$-$ 1.	1428	0.	2537
	PS	$-$ 0.	3484	6.	7454	$-$ 0.	9411	0.	2442	$-$ 1.	1552	0.	2468
	BS	0.	3666	6.	6935	$-$ 0.	2214	0.	9548	1.	2249	0.	2212
$n=$ 100	SS	0.	0719	4.	2680	$-$ 0.	3031	0.	4469	0.	3767	0.	7065
	PS	0.	0837	4.	2029	$-$ 0.	2855	0.	4530	0.	4454	0.	6562
	BS	$-$ 0.	0895	4.	2152	$-$ 0.	4599	0.	2807	$-$ 0.	4751	0.	6349
$n=$ 200	SS	$-$ 0.	0954	2.	8932	$-$ 0.	3504	0.	1595	$-$ 0.	7353	0.	4625
	PS	$-$ 0.	0921	2.	9123	$-$ 0.	3480	0.	1637	$-$ 0.	7072	0.	4798
	BS	0.	0149	3.	2685	$-$ 0.	2299	0.	2598	0.	1200	0.	9045
$n=$ 300	SS	$-$ 0.	1574	4.	2673	$-$ 0.	5347	0.	2197	$-$ 0.	8201	0.	4125
	PS	$-$ 0.	1441	4.	1982	$-$ 0.	5130	0.	2247	$-$ 0.	7676	0.	4431
	BS	0.	1475	1.	8514	$-$ 0.	2786	0.	5738	0.	6802	0.	4966

Table 2

The summary statistics of simulation studies with model 2 based on smoothing spline (SS), penalized spline (PS), and B-spline (BS)

Sample sizes	Methods	Mean		S.D.		LCI		UCI		$t$ -statistic		$p$ -values
$n=$ 50	SS	$-$ 0.	0037	24.	2710	$-$ 2.	1363	2.	1288	$-$ 0.	0034	0.	9972
	PS	$-$ 0.	2582	23.	8736	$-$ 2.	3559	1.	9893	$-$ 0.	2419	0.	8089
	BS	$-$ 0.	3230	27.	1906	$-$ 2.	7121	2.	0660	$-$ 0.	2656	0.	7906
$n=$ 100	SS	$-$ 0.	3966	15.	8261	$-$ 1.	7871	0.	9939	$-$ 0.	5603	0.	5755
	PS	$-$ 0.	2489	16.	4557	$-$ 1.	6947	1.	1969	$-$ 0.	3382	0.	7353
	BS	0.	5461	16.	2381	$-$ 0.	8806	1.	9729	0.	7520	0.	4524
$n=$ 200	SS	$-$ 0.	3704	8.	1031	$-$ 1.	0838	0.	3430	$-$ 1.	0201	0.	3082
	PS	$-$ 0.	3447	7.	8716	$-$ 1.	0363	0.	3468	$-$ 0.	9792	0.	3279
	BS	0.	4035	8.	5116	$-$ 0.	3443	1.	1514	1.	0602	0.	2896
$n=$ 300	SS	0.	0411	7.	3081	$-$ 0.	6055	0.	6878	0.	1250	0.	9005
	PS	0.	0408	7.	0664	$-$ 0.	5800	0.	6617	0.	1291	0.	8973
	BS	0.	0344	7.	2205	$-$ 0.	5999	0.	6688	0.	1066	0.	9151

4. Simulation study

The nonlinear data of this study is simulated in two models for estimating the performance of smoothing techniques based on independent variables which are considered in the class of uniform distribution. These models in the process of construction a curve on mathematical function, that show the best fit to a series of data points. Figures 1 and 2 show the scatter plot of $x_{t}$ and $y_{t}$ on models 1 and 2 with 50, 100, 200, 300 sample sizes.

Model 1

$\displaystyle S(x_{t})=(x_{t}^{3})-\cos(x_{t})-\exp\left\{{\frac{x_{t}}{1+|{x_% {t}}|}}\right\},$ $\displaystyle x_{t}\sim\textit{Uniform}(-2,2),t=1,2,3,...,n$ $\displaystyle y_{t}=S(x_{t})+\varepsilon_{t},\varepsilon_{t}\sim\textit{Normal% }(0,1),$ $\displaystyle t=1,2,3,...,n$

Model 2

$\displaystyle S({x_{t}})=\sin(x_{t})-\exp\left\{{\frac{x_{t}}{1+|{x_{t}}|}}% \right\}-x_{t}^{3},$ $\displaystyle x_{t}\sim\textit{Uniform}(-2,2),t=1,2,3,...,n$ $\displaystyle y_{t}=S(x_{t})+\varepsilon_{t},\varepsilon_{t}\sim\textit{Normal% }(0,1),$ $\displaystyle t=1,2,3,...,n$

The next step, the estimates of $S({x_{t}})$ or called $\hat{S}({x_{t}})$ are approximated from smoothing spline (SS), penalized spline (PS), and B-spline (BS) that used to compute the bias and MSE of $S({x_{t}})$ following

$\displaystyle S_{\textit{bias}}=\sum\limits_{t=1}^{n}{\left({\frac{\hat{S}({x_% {t}})-S({x_{t}})}{S({x_{t}})}}\right)},$ $\displaystyle\textit{MSE}=\frac{\sum\limits_{t=1}^{n}{\left[{\hat{S}({x_{t}})-% S({x_{t}})}\right]^{2}}}{n}.$

The data are generated and repeated for fitting the model 500 times. A $t$ -statistic is adopted to test that the mean of bias is equal the zero or called unbiased estimator. Tables 1 and 2 present the various summary statistics for the smoothing estimator obtained from three methods. The third and the fourth columns of these tables represent the sample mean and standard deviation of biases. The sample mean for the lower and upper bounds of the 95% confidence interval are given in the next two columns. The last two columns of these tables list the $t$ -statistic, and $p$ -values for hypothesis testing ( $H_{0}:\mu_{S_{\textit{bias}}}=0\mbox{ versus }H_{0}:\mu_{S_{\textit{bias}}}\neq 0$ ) that means when reject $H_{0}:\mu_{S_{\textit{bias}}}=0$ the estimator SS, PS, and BS with bias. The histogram of the bias estimator of SS, PS, and BS in model 1 are presented in Figs 3–5, and model 2 are presented in Figs 6–8.

Figure 3.

Histogram of bias for fitting data of smoothing spline method with model 1.

Figure 4.

Histogram of bias for fitting data of penalized spline method with model 1.

Figure 5.

Histogram of bias for fitting data of B-spline method with model 1.

Figure 6.

Histogram of bias for fitting data of smoothing spline method with model 2.

Figure 7.

Histogram of bias for fitting data of penalized spline method with model 2.

Figure 8.

Histogram of bias for fitting data of B-spline method with model 2.

From Tables 1 and 2, by observing the $p$ -values, the SS, PS, and BS provide asymptotically unbiased estimates for estimating parameter of $S({x_{t}})$ nearly for all sample sizes of two models. From the $p$ -values for the two tables it is seen that are seen that the SS, PS, and BS of smoothing method have a good performance to fit data in a class of nonlinear data. From the histogram it is apparent that a standard deviation of relative biases increase with increasing sample sizes, so it makes the leptokurtic distribution. The average of MSE can answer the final question which smoothing method is the best estimator. Table 3 shows the average MSE for fitting 500 times on two models, and it can be seen that the PS method shows the minimum of average MSE for all sample sizes and models.

Table 3

The average MSE of simulation studies with 3 models based on smoothing spline (SS), penalized spline (PS), and B-spline (BS)

Sample sizes	Methods	Model 1	Model2
$n=$ 50	SS	0.8225	0.8348
	PS	0.7712	0.7846
	BS	0.9174	0.9364
$n=$ 100	SS	0.9087	0.9029
	PS	0.8810	0.8719
	BS	0.9632	0.9508
$n=$ 200	SS	0.9528	0.9521
	PS	0.9387	0.9439
	BS	0.9784	0.9840
$n=$ 300	SS	0.9666	0.9648
	PS	0.9640	0.9615
	BS	0.9899	0.9877

Figure 9.

The plot of LIDAR data frame and model fitting of SS, PS, and BS methods.

5. Application of real data

In this section, we consider the application of smoothing method based on SS, PS, and BS methods that we developed in the previous section. As the real data, we use the data frame which consists of 221 observations from a light detection and ranging (LIDAR) experiment. This data frame contains the range distance travelled before the light is reflected back to its source and logarithm of the ratio of received light from two laser sources as shown in the plot in Fig. 9.

After fitting the model, the estimating values play on a plot of light detection of ranging. It can be seen that the SS and PS interpolate in mass data more than the BS method that followed the MSE values such as SS $=$ 0.006016, PS $=$ 0.006010, and BS $=$ 0.009288. The minimum of MSE is the PS which is closed the SS as the result on Table 3.

6. Conclusion

In this section, we used the smoothing techniques of SS, PS, and BS methods based on nonparametric regression models. Through a Monte Carlo simulation study, we evaluated the smoothing estimator of SS, PS, and BS methods. For hypothesis testing based on the $p$ -value, the fitting values supported the null value, and showed that the smoothing estimators work reasonably well for all methods, but the PS shows the minimum of average MSE.

Footnotes

Acknowledgments

This work was supported by Faculty of Science Fund, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, Thailand.

References

Wahba

. Spline Models for Observational Data, SIAM: Philadelphia; 1990.

Green

Silverman

. Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach, Chapman and Hall: London; 1994.

Ruppert

Wand

Carroll

. Semiparametric Regression, Cambridge University Press: New Your; 2003.

Eilers

PHC

Marx

. Flexible Smoothing with B-splies and Penalties, Statistical Science 11(2) (1996), 89–102.

Wahba

. A survey of some smoothing problems and the method of generalized cross-validation for solving them, In Proceeding of the Conference on the Application of Statistic, 1976, pp. 507–523.

Craven

Wahba

. Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation, Numerische Mathematik 31 (1979), 377–403.

Eubank

. Spline Smoothing and Nonparametric Regression, Marcel Dekker: New York; 1988.

Eubank

. Nonparametric Regression and Spline Smoothing, Marcel Dekker: New York; 1999.

Ruppert

Carroll

. Spatial-adaptive penalties for spline fitting Australian and New Zealand Journal of Statistics 42 (2000), 205–224.

10.

De Boor

. A Practical Guide to Splines, Springer: Berlin; 1978.