Semiparametric spatiotemporal variable coefficient regression model

Abstract

In order to study the semi-parametric spatiotemporal variable coefficient regression model, firstly, the parametric and non-parametric models are introduced. Based on the spatiotemporal variable coefficient model and estimation method, a semi-parametric spatiotemporal variable coefficient model is established. According to the two-step estimation simulation experiment of semi-parametric space-time variable coefficient model, three groups of numerical experiments of semi-parametric spatiotemporal variable coefficient model are carried out. The experimental results show that the semi-parametric regression model has both the advantages of parametric model and non-parametric model, and the estimated value is very close to the real value. After combining the semi-parametric and time-varying coefficient regression model, the fitting effect of the model is better, and the estimation accuracy is obviously improved.

Keywords

Semi-parameter variable coefficient regression model

1. Introduction

Statistical inference of spatiotemporal models has always been an important and widely used branch of statistical and econometric research. However, the main spatiotemporal models are generally parametric models. Although standard parameter models, such as spatiotemporal lag regression model and spatiotemporal error regression model, are very useful in testing problems, their model forms are strictly dependent on subjective assumptions and are easy to be set incorrectly [1]. In recent 20 years, with the rapid development of computer computing technology, in the field of spatiotemporal statistics and spatiotemporal econometrics, a variety of semi-parametric modeling methods have been proposed to explore the complex relationship between dependent variables and independent variables, including geographically additive model, geographically weighted regression model and spatiotemporal non-parametric regression model [2]. Semi-parametric regression model is a new regression model introduced by Engle et al. when studying the relationship between electricity demand and climate. So far, the model has been widely concerned and applied from theoretical research to practical data analysis. Semi-parametric model combines parametric model and non-parametric model. On the one hand, the parametric part can still concentrate the main information and has strong explanatory ability. On the other hand, it not only alleviates the problem of dimension, but also improves the accuracy of the model [3].

With the continuous development of science and technology, regression analysis, as the basis of mathematical statistics in the era of big data, has been widely used in many fields such as geology, meteorology, hydrology, medicine, industry, agriculture and economy. It has become one of the important tools of statistical analysis, an effective method to solve practical problems, and a hot research direction in the field of statistics [4]. The arrival of the data age promotes the further exploration of science and technology, brings more changes to our life and work, brings opportunities to the application and development of regression analysis theory, and also makes it face more significant challenges [5, 6]. As the basic model of regression analysis, linear regression model has been widely used in data research because of its complete system. However, linear regression models generally require the form of dependent variables and the uniform normal distribution of random variables, which has limitations. Therefore, the non-linear regression model has been developed and applied, the most representative of which is the generalized linear model, which mainly represents the exponential family distribution [7]. It can deal with the regression analysis of discrete distribution of dependent variables such as binomial distribution and Poisson distribution.

The research of regression model is an important part of mathematical statistics. Because of its fixed form, the parametric regression model cannot satisfy the flexibility of data in practical problems. Although the non-parametric regression model does not need to select the model beforehand and can reflect the information more accurately, it faces the problem of high dimension. Semi-parametric model has the advantages of both the above two models, and contains both parametric components and non-parametric components. It can reduce the risk of misjudgement and overcome the “dimension disaster”, which has become a hot research topic in recent years and has been widely used in various fields.

2. Method

2.1 Parametric and non-parametric models

In real life, many variables have interdependent relationships, which are composed of some variables $x_{1}$ , $x_{2}$ , $\ldots$ , $x_{p}$ . For example, the sales volume of a company will be affected by the advertising intensity $x_{1}$ , the number of districts $x_{2}$ , and the number of preferences $x_{3}$ . In addition, there are some uncontrolled random factors which can also affect $y$ . They are regarded as random errors, which are recorded as $\varepsilon$ . Therefore, the relationship between $y$ and $x_{1}$ , $x_{2}$ , $\ldots$ , $x_{p}$ can be expressed as:

$\displaystyle y=m\left({x_{1},x_{2},\ldots,x_{p}}\right)+\varepsilon$ (1)

The relationship between $y$ and $x$ is called regression. If it contains some unknown parameters, that is:

$\displaystyle y=m\left({x_{1},x_{2},\ldots,x_{p}}\right)=g\left({x_{1},x_{2},% \ldots,x_{p};\beta_{1},\beta_{2},\ldots,\beta_{r}}\right)$ (2)

In the above formula, $g$ is a known function, $\beta_{1}$ , $\beta_{2}$ , $\ldots$ , $\beta_{r}$ is an unknown parameter, the modulus is called a parametric regression model. If the regression function is the linear function of $x_{1}$ , $x_{2}$ , $\ldots$ , $x_{p}$ , that is:

$\displaystyle y=m\left({x_{1},x_{2},\ldots,x_{p}}\right)=\beta_{0}+\beta_{1}x_% {1}+\ldots+\beta_{p}x_{p}$ (3)

The model Eq. (1) is called a linear regression model; otherwise, it is called a non-linear regression model.

If the specific form of regression function $m$ ( $x_{1}$ , $x_{2}$ , $\ldots$ , $x_{p}$ ) is not set, which only requires continuity or smoothness, then model Eq. (1) is called as a non-parametric regression model.

Non-parametric statistics mainly use large sample method. When the distribution of test statistics is too complex, limit distribution is generally used instead. For example, when estimating density function $f(x)$ using histogram, one or two data cannot get detailed estimates. In practice, in order to increase the number of data to improve the accuracy of estimates, it usually requires sample size $n>$ 50. This is because the key of histogram method is to use frequency $f(i)/n$ as approximation of probability $P$ . It can be seen that the approximation of $n$ is more accurate when $n$ is larger, which is also one of the reasons for using large sample method.

2.2 Spatiotemporal variable coefficient model and estimation method

In many practical applications, with the increasing complexity of the data, it is not enough to consider a single regression coefficient. Taking the price of urban commercial housing as an example, the price of the house is not only related to the area, but also depends on the location ( $u, v$ ) (such as longitude and latitude). At this time, it is necessary to consider the spatial and temporal factors on the basis of the variable coefficient model. The coefficient B can be expressed as a function of ( $u, v$ ) and different geographical locations can be expressed as different coefficient functions. This method overcomes the non-stationarity of space-time and can better use data to reflect the actual situation. At this point, the model can be expressed as:

$\displaystyle Y=\beta\left({u,v}\right)X+\varepsilon$ (4)

Spatiotemporal variable coefficient model can accurately analyze the change of house price with the change of space-time location. Spatiotemporal information is introduced into the model, and the function of space-time location of observation points is used as regression coefficient. The spatiotemporal characteristics of regression relationship are analyzed by using the estimated values of regression coefficients. Assuming that the coordinates of independent variable $X$ and space-time geographic location are precisely measurable non-random variables, the sample form of the regression model of spatiotemporal variable coefficients is as follows:

$\displaystyle Yi=\rho_{0}\left({u_{i},v_{i}}\right)+\sum\limits_{m}^{p}{\rho_{% m}\left({u_{i},v_{i}}\right)X_{IM}+\varepsilon_{i}}$ (5)

( $Y_{i}$ , $X_{i1}$ , $X_{i2}$ , $\ldots$ , $X_{in}$ ) refers to the observation value of $Y$ and $X$ in the location of ( $u_{i}$ , $v_{i})$ , and $\rho_{m}$ ( $u, v$ ) ( $m=$ 1, 2, $\ldots$ , $p$ ) is the location parameter of the observation point ( $u, v$ ) and also its location function; $\varepsilon_{i}$ represents the normal error terms of independent and identical distribution, recorded as N (0, $\sigma^{2}$ ), $E(N)=$ 0, Var(N) $=$ $\sigma^{2}$ .

Set ( $u_{0}$ , $v$ %) to be any point in the space. The function of weight matrix is to have different parameter functions according to the difference of spatiotemporal observations. Because these observations are related to the spatiotemporal position, in order to exclude the influence of remote data, d ${}_{0i}$ is the distance between the observations at ( $u_{0}$ , $v_{0}$ ) and those at ( $u_{i}$ , $v_{i}$ ), then there is:

$\displaystyle d_{0i}=\sqrt{\left({u_{0}-u_{i}}\right)^{2}+\left({v_{0}-v_{i}}% \right)^{2}},i=1,2,\ldots,n$ (6)

The Gauss kernel function used here is in the form of a single-valued function.

$\displaystyle w_{i}\left({u_{0},v_{0}}\right)=\exp\left({-\frac{1}{2}\left({% \frac{d_{0i}}{h}}\right)^{2}}\right),i=1,2\ldots n$ (7)

Construct weighted least squares function:

$\displaystyle\sum\limits_{i=1}^{n}{\left({Y_{i}-\rho_{0}\left({u_{0},v_{0}}% \right)-\sum\limits_{m}^{p}{\rho_{m}\left({u_{i},v_{i}}\right)X_{im}}}\right)^% {2}}w_{i}\left({u_{o},v_{0}}\right)$ (8)

To minimize it, the estimation of parameter $\rho_{m}$ ( $u_{0}$ , $v_{0}$ ) is obtained at this moment.

$\displaystyle f(\rho)=\sum\limits_{i=1}^{n}{\left({Y_{i}-\rho_{0}\left({u_{0},% v_{0}}\right)-\sum\limits_{m}^{p}{\rho_{m}\left({u_{i},v_{i}}\right)X_{im}}}% \right)^{2}}w_{i}\left({u_{o},v_{0}}\right)$ (9)

Solve partial derivative of $\rho_{m}$ ( $u_{0}$ , $v_{0})$ ( $m=$ 1, 2, $\ldots$ , $p$ ) and let it be 0, and finally solve the estimated value $P\rho_{m}$ ( $u_{0}$ , $v_{0}$ ) of $\rho_{m}$ ( $u_{0}$ , $v_{0}$ ).

$\displaystyle x=\left({\begin{array}[]{l}1X_{11}\ldots X_{1P}\\ 1X_{21}\ldots X_{2P}\\ \ldots\\ 1X_{n1}\ldots X_{nP}\\ \end{array}}\right),Y=\left({\begin{array}[]{l}Y_{1}\\ Y_{2}\\ \ldots\\ Y_{n}\\ \end{array}}\right),\varepsilon=\left({\begin{array}[]{l}\varepsilon_{1}\\ \varepsilon_{2}\\ \ldots\\ \varepsilon_{n}\\ \end{array}}\right)$ (10)

$\displaystyle W\left({u_{0},v_{0}}\right)=\left({\begin{array}[]{l}W_{1}\left(% {u_{0},v_{0}}\right)\ldots 0\\ \ldots\\ 0\ldots W_{n}\left({u_{0},v_{0}}\right)\\ \end{array}}\right)$ (11)

$\displaystyle\rho_{0}\left({u_{0},v_{0}}\right)=\left({\rho_{1}\left({u_{0},v_% {0}}\right),\rho_{2}\left({u_{0},v_{0}}\right),\ldots\rho_{p}\left({u_{0},v_{0% }}\right)}\right)^{T}$ (12)

Then the solution is:

$\displaystyle\hat{\rho}_{0}\left({u_{0},v_{0}}\right)=\left({\hat{\rho}_{1}% \left({u_{0},v_{0}}\right),\hat{\rho}_{2}\left({u_{0},v_{0}}\right),\ldots\hat% {\rho}_{p}\left({u_{0},v_{0}}\right)}\right)^{T}=\left({X^{T}W\left({u_{0},v_{% 0}}\right)X}\right)^{-1}X^{T}W\left({u_{0},v_{0}}\right)Y$ (13)

The fitting of $y(u_{0},v_{0})$ is obtained:

$\displaystyle\hat{y}_{0}\left({u_{0},v_{0}}\right)=x_{0}^{T}\hat{\rho}\left({u% _{0},v_{0}}\right)=X_{0}^{T}\left({X^{T}W\left({u_{0},v_{0}}\right)X}\right)^{% -1}X^{T}W\left({u_{0},v_{0}}\right)Y$ (14)

Let ( $u_{0}$ , $v_{0}$ ) be ( $u_{i}$ , $v_{i}$ ):

$\displaystyle\hat{\rho}\left({u_{i},v_{i}}\right)=\left({\hat{\rho}_{1}\left({% u_{i},v_{i}}\right),\hat{\rho}_{2}\left({u_{i},v_{i}}\right),\ldots\hat{\rho}_% {p}\left({u_{i},v_{i}}\right)}\right)^{T}=\left({X^{T}W\left({u_{i},v_{i}}% \right)X}\right)^{-1}X^{T}W\left({u_{i},v_{i}}\right)Y,i=1,2\ldots n$ (15)

Then there is:

$\displaystyle Y=\left({\begin{array}[]{l}Y_{1}\\ Y_{2}\\ \ldots\\ Y_{n}\\ \end{array}}\right)=\left({\begin{array}[]{l}\left({X^{T}W\left({u_{1},v_{1}}% \right)X}\right)^{-1}X^{T}W\left({u_{1},v_{1}}\right)Y\\ \left({X^{T}W\left({u_{2},v_{2}}\right)X}\right)^{-1}X^{T}W\left({u_{2},v_{2}}% \right)Y\\ \ldots\\ \left({X^{T}W\left({u_{2},v_{2}}\right)X}\right)^{-1}X^{T}W\left({u_{2},v_{2}}% \right)Y\\ \end{array}}\right)=LY$ (16)

$\displaystyle L=\left({\begin{array}[]{l}\left({X^{T}W\left({u_{1},v_{1}}% \right)X}\right)^{-1}X^{T}W\left({u_{1},v_{1}}\right)\\ \left({X^{T}W\left({u_{2},v_{2}}\right)X}\right)^{-1}X^{T}W\left({u_{2},v_{2}}% \right)\\ \ldots\\ \left({X^{T}W\left({u_{n},v_{n}}\right)X}\right)^{-1}X^{T}W\left({u_{n},v_{n}}% \right)\\ \end{array}}\right)$ (17)

$X_{i}^{T}=(1,x_{i0},\ldots,x_{ip})$ , $i=$ 1, 2, $\ldots$ , $n$ , suggests the ith row of $X$ , so the residual vector is:

$\displaystyle\hat{\varepsilon}=Y-\hat{Y}=\left({I-L}\right)Y$ (18)

The estimation of the error square is usually recorded as:

$\displaystyle\hat{\sigma}^{2}=\frac{\hat{\varepsilon}^{T}\hat{\varepsilon}}{tr% \left({\left({I-L}\right)^{T}\left({I-L}\right)}\right)}$ (19)

For spatiotemporal variable coefficient model, because two-dimensional geographic area is much larger than one-dimensional, the boundary effect of GWR method will be more serious, which will make the estimation of coefficient function in the boundary area inaccurate. Considering that the local polynomial fitting method has many advantages such as automatically correcting the boundary effect, the local linear fitting method is applied to the spatiotemporal variable coefficient model, and the improved effect of the local linear GWR method is given.

2.3 Establishment of semi-parametric spatiotemporal variable coefficient model

At present, the semi-parametric models studied by scholars mainly focus on partial variable coefficient models. However, in some practical problems, people will encounter such problems. Some variables have temporal and spatial differences in the impact of dependent variables, while other variables have no temporal and spatial differences in the impact of dependent variables, but they are non-linear. The linear part of the combined model will be extended to the non-parametric model, and the estimation method of the semi-parametric model which is composed of an independent variable non-parametric model and a time-space varying coefficient model will be further explored. Assuming a semiparametric spatiotemporal variable coefficient model:

$\displaystyle Y=m\left({X_{1}}\right)+\sum\limits_{i=2}^{k}{\beta_{1}\left(V% \right)X_{1}+\varepsilon}$ (20)

$Y$ is a dependent variable, $X_{1}$ , $X_{2}$ , $\ldots$ , $X_{k}$ is the independent variable and V is the spatiotemporal variable $V=(u,v)\times m$ ( $X_{1}$ ) is an unknown non-linear function, and $\beta_{1}$ ( $V$ ) is a coefficient parameter function of the part of the space-time variable coefficient model. $\varepsilon$ is an error term, satisfying the following conditions:

$\displaystyle E\left({\varepsilon\left|{V,X_{1,},\ldots,X_{k}}\right.}\right)=% 0,\textit{Var}\left({\varepsilon\left|{V,X_{1,},\ldots,X_{k}}\right.}\right)=% \sigma^{2}$ (21)

$m(X_{1})$ is sufficiently smooth functions, and there are arbitrary derivatives.

If ( $x_{i1}$ , $x_{i2}$ , $\ldots$ , $X_{ik}$ , $V_{i}$ , $y_{i}$ ,) is n groups of independent observation data, then the sample form of the semi-parametric spatiotemporal variable coefficient model is

$\displaystyle y_{i}=m\left({x_{i1}}\right)+\sum\limits_{i=2}^{k}{\beta_{1}% \left({V_{i}}\right)x_{i1}+\varepsilon_{i}i=1,2,\ldots n}$ (22)

$\varepsilon_{i}$ meets:

$\displaystyle E\left({\varepsilon\left|{V,X_{1,},\ldots,X_{k}}\right.}\right)=0,$ (23)

$\textit{Var}\left({\varepsilon\left|{V,X_{1,},\ldots,X_{k}}\right.}\right)=% \sigma^{2}$

2.4 Two-step estimation method for semi-parametric spatiotemporal variable coefficient model

Next, a series of simulation experiments will be carried out to investigate the accuracy and stability of linear function and spatiotemporal variable coefficient parameter estimation in the nonlinear semi-parametric variable coefficient model proposed in the second part.

In the simulation experiment, the spatiotemporal region edge length is $m-1$ unit square, and the lower left corner of the region is the origin of coordinates. The square edge length $m-1$ is divided into equal parts, and then $m*m$ lattice points are obtained. If $u$ and $v$ are used to represent the horizontal and vertical coordinates of lattice points, then ( $u_{i}$ , $v_{i}$ ) is the geographical location of the first independent variable and dependent variable:

$\displaystyle u_{i}=\text{mod}\left({\left({i-1}\right)/m}\right)$ (24)

Figure 1.

Fitting figures under three groups of model experiments (A: (i) Fitting figure under variance of 0.2; B: (i) Fitting figure under variance of 0.6; C: (i) Fitting figure under variance of 1; D: (ii) Fitting figure under variance of 0.2; E: (ii) Fitting figure under variance of 0.6: F: (ii) Fitting figure under variance of 1; G: (iii) Fitting figure under variance of 0.2; H: (iii) Fitting figure under variance of 0.6; I: (iii) Fitting figure under variance of 1.

$u_{i}$ represents the remainder of $i-1$ divided by $m$ , $v_{i}=(i-1)/m$ represents the integer part of quotient, $i=$ 1, 2, $\ldots$ , $n$ , $n=m^{2}$ . The values of other independent variables in the model are random numbers generated independently and uniformly distributed on the interval [0,1]. Otherwise, the actual problem variables can be standardized. Take $x_{0}=$ 0.5.

3. Results and discussion

Three sets of semi-parametric spatiotemporal variable coefficient models are numerically tested.

$\displaystyle y=\sin 2x_{1}+\sin v\cdot x_{2}+\varepsilon$ (25) $\displaystyle y=6x_{1}\cdot\cos(6\pi x_{1})+(u+v)x_{2}+\varepsilon$ (26) $\displaystyle y=\sin(6\pi x_{1})+(u^{2}+v^{2})x_{2}+\varepsilon$ (27)

The error term $\varepsilon$ obeys the normal distribution $N$ (0, $\sigma^{2}$ ), and the standard deviation is $\sigma=$ 1, 0.6, 0.2, $m=$ 10, 9, 8, respectively. For each $\sigma$ , m only repeats 300 times with random error $\varepsilon$ to obtain the test results. The mean square error of the non-linear function $m(x)$ is MSE1, the mean square error of the variable coefficient part is MSE2, and the mean square error of the dependent variable estimation is MSE3:

$\displaystyle\text{(i) MSE1}=\frac{1}{n}\sum\limits_{i=1}^{n}{(m(x_{1i})-\hat{% m}(x_{1i}))^{2}}$ (28) $\displaystyle\text{(ii) MSE2}=\frac{1}{nk}\sum\limits_{i=1}^{n}{\sum\limits_{i% =2}^{k}{\left({\beta_{i}\left({V_{i}}\right)-\hat{\beta}_{i}\left({V_{i}}% \right)}\right)}}^{2}$ (29) $\displaystyle\text{(iii) MSE3}=\frac{1}{n}\sum\limits_{i=1}^{n}{(y_{i}-\hat{y}% _{i})^{2}}$ (30)

Using MATLAB software programming, after running, the following results are obtained.

The numerical results show that under different $\sigma$ and $m$ , the estimated values of the semi-parametric model are very close to the real values, and the fitting effect is better. As the $\sigma$ decreases, that is, the variance of noise decreases, the disturbance to the model decreases, and the accuracy of estimation increases significantly. The larger m is, the more observation points are, and the smaller the degree of approximation of the estimated value to the exact value is.

4. Conclusion

Based on the definition and difference of parametric model and non-parametric model, the expression and estimation method of variable coefficient regression model and the generalized spatiotemporal variable coefficient regression model, two-step estimation method is used to estimate the semi-parametric spatiotemporal variable coefficient model combined with one-variable non-parametric and spatiotemporal variable coefficient model, and the estimation expression is given. The simulation experiment shows that the semi-parametric regression model has the advantages of both parametric and non-parametric models. The estimated value is very close to the real value and the fitting effect is good. As the variance decreases, the disturbance to the model decreases, and the accuracy of estimation increases significantly. Combining the non-parametric and time-varying coefficient model, an estimation method is proposed. The estimated value fits well with the real value, and the accuracy of the method is high.

References

Wang

and Sun

, Efficient parameter estimation and variable selection in partial linear varying coefficient quantile regression model with longitudinal data, Statistical Papers (2017), 1–29.

Kurata

and Matsuura

, Best equivariant estimator of regression coefficients in a seemingly unrelated regression model with known correlation matrix, Annals of the Institute of Statistical Mathematics68(4) (2016), 1–19.

and Zhou

, Nonparametric inference for time-varying coefficient quantile regression, Journal of Business & Economic Statistics35(1) (2017), 98–109.

and Zhang

, Semi-varying coefficient multinomial logistic regression for disease progression risk prediction, Statistics in Medicine35(26) (2016), 4764–4778.

Zheng

and Peng

, Consistent model identification of varying coefficient quantile regression with BIC tuning parameter selection, Communications in Statistics46(3) (2017), 1031–1049.

Sedghi

Sadeghian

and Huang

, Mixture Semisupervised Probabilistic Principal Component Regression Model with Missing Inputs, Computers & Chemical Engineering103 (2017), 176–187.

Zheng

and Song

, Semisupervised learning for probabilistic partial least squares regression model and soft sensor application, Journal of Process Control64 (2018), 123–131.