An accurate estimation algorithm for structural change points of multi-dimensional stochastic models

Abstract

In order to improve the estimation accuracy of structural change points of multi-dimensional stochastic model, the accurate estimation algorithm of structural change points of multi-dimensional stochastic model is studied. A multi-dimensional stochastic Graphical Modeling model based on multivariate normal hypothesis is constructed, and the relationship between the Graphical Gaussian model and the linear regression model is determined. The parameters of the multi-dimensional stochastic model are estimated by using the parameter estimation algorithm of the multi-dimensional stochastic model containing intermediate variables. According to the parameter estimation results of the multi-dimensional stochastic model, the structural change point estimation results of the multi-dimensional stochastic model are obtained by using the accurate estimation algorithm of the structural change point based on the MLE identification local drift time. The experimental results show that the proposed algorithm has higher estimation accuracy of structural change points than the control algorithms, which shows that it can effectively estimate the structural change points of multi-dimensional random models and has higher practicability.

Keywords

Multidimensional stochastic model structure change point accurate estimation algorithm

1 Introduction

The stochastic model is to treat the uncertain variable as a random variable or a random process, and study the uncertain phenomenon by using probability theory and statistical methods. Therefore, it is also called the probabilistic model. If the structural uncertainty parameter is a random variable or random field [13], the structural response is also a random variable or random field, and its statistical characteristic values such as mean and variance can be calculated according to the relevant theories in probability statistics. The probability model needs to know the statistical laws of the uncertain parameters [10]. When there is not enough data to verify the correctness of the probability distribution of these random parameters, the probability method will be difficult to reliably obtain the calculation results that meet the accuracy requirements. Stochastic model, also known as “uncertain and probabilistic model”, is a model established according to random variables. Its characteristics are that the model parameters, the conditions and state characteristics of the simulation object are random variables, and their contact information is also random, or the original information is represented by random variables. The so-called random variable is a variable with random property [22]. If it wants to truly reflect the causal relationship of random variables in the system, it should use a stochastic model. For example, in insurance statistics tables, insurance companies use age as a variable to represent the probability of death. To construct a stochastic model, correlation analysis, regression analysis and other statistical algorithms need to be applied. The common stochastic models in the economic system include stochastic storage model, queuing model, game decision model, Markov decision model, etc. [29]. The relationship between variables is given in the form of statistical values. Non deterministic models are called stochastic models [19]. If any exogenous variable in the model is uncertain and changes with the change of specific conditions, this model is called a stochastic model.

In the real world, uncertainty is widespread. For example, the tiny particles floating on the liquid surface constantly move in a disorderly manner, and the position of the particles at any time is uncertain. Another example is that the number of people waiting for a bus at the bus stop is also uncertain at any time, because passengers may come and leave at any time. On the surface, this kind of uncertainty can not be grasped. In fact, behind its uncertainty, there is often a certain probability law. Therefore, the stochastic model based on probability and mathematical statistics has become one of the most effective tools to solve such problems [21]. According to whether the random law changes with time, the stochastic model can be divided into static and dynamic [1]. The former only involves the probability distribution and numerical characteristics of random variables (vectors), while the latter deals with stochastic processes and stochastic differential equations. The common applications of static stochastic models are: fishing problem, newsboy’s strategy, elevator problem, economic steel rolling problem, etc. Each treatment of the stochastic model test is a group of random samples [18], so the treatment effect τ is random and varies with different trials. If the experiment is repeated, a new group of samples must be randomly selected from the population. The purpose of the analysis is not to study the treatment effect, but to study degree of variability of τ. Therefore, the inference is not about some of the tested treatments, but about extracting the whole of these treatments.

Some scholars have studied the structural change point test. In Reference [5], an algorithm of change point detection based on Bayesian posteriori reasoning and genetic algorithm for multiple structural change points of linear regression model is proposed. Firstly, the non-information prior of the parameter is introduced and the Bayesian posterior probability of the change point information is obtained based on the posterior distribution. Then, a Schwarz Bayesian information criterion is defined based on a posteriori probability, and the number and location of change points are quickly obtained by using genetic algorithm. The effectiveness of the new algorithm and the rapidity of calculation are verified by numerical simulation. However, this algorithm does not consider the problem of multi variable point identification in high-dimensional case. In Reference [9], the calculation algorithm of offset effect by combining the principal component with the projection matrix is proposed. This study finds that common factors make the change point estimation of panel data structure offset, and the offset size is related to the strength of the common force. Though this algorithm solves the cross section correlation problem ignored the change point test in the traditional panel data structure, it cannot adapt to multi-dimensional time series. In Reference [6], Li et al. conduct a sequential test on the structural change points of the cumulative logistic regression model. Based on the partial likelihood scoring process, a sequential test algorithm of change points is proposed to monitor whether the structure of the cumulative logistic regression model changes. The limit distribution of test statistics is derived under the original hypothesis, and its consistency is proved under alternative hypothesis. In Reference [16], Yin and Wang estimate the panel model of structural change. The main research results of parametric and nonparametric panel model algorithms for structural change are introduced, including the parameter estimation and hypothesis testing of threshold panel model, panel smooth transition model, nonparametric time-varying coefficient panel model, and semi parametric trend panel model. The research of structural change panel model is prospected. The above two algorithms can realize the estimation of structural change points of different types of models, but they have the disadvantage of poor estimation accuracy.

Based on the existing problems in the research of structural change point estimation, an accurate estimation algorithm of structural change point in multi-dimensional stochastic model is proposed. The innovation of this algorithm lies in:

Multivariate normal hypothesis is made, the calculation of structural change points to multidimensional time series is expanded, and multidimensional random graph modeling model is constructed.

Intermediate variables are introduced between independent variables and dependent variables to make the direct causal relationship between variables more accurate with adding mediation effect.

The LISREL method is replaced by PLS algorithm and corresponding solution software to avoid strict assumptions.

The local feature recognition conditions are simplified, and then the local difference problem is transformed into the classical change point recognition problem.

The purpose of the proposed algorithm is to improve the accuracy of structural change point estimation of stochastic models in multidimensional time series, so that structural change point estimation can be applied to more fields. The experimental results show that the algorithm has higher accuracy in estimating the structural change points of the multidimensional random model, and can achieve the purpose, providing valuable reference for the subsequent related research.

2 Materials and methods

The relationship between multi-dimensional random variables is the basis for accurate estimation of structural change points. The conditional correlation between two variables that are conditionally independent may change over time. In order to improve the accuracy of finding conditional correlations between multidimensional random variables, it is necessary to ensure that there are no change points in the sample sequence. If there is no change point of fluctuation, whether there is no change point in the graphic structure will be made sure. On the basis of this principle, a multi-dimensional random graph modeling model based on multivariate normal hypothesis is established to determine the relationship between the graph Gaussian model and the linear regression model. Secondly, a multidimensional stochastic model with intermediate variables is constructed to estimate the parameters of the multidimensional stochastic model. Then, according to the parameter estimation results, the structural change points based on MLE identification of local drift time are accurately estimated, and finally the structural change point estimation of multi-dimensional random model is completed. The specific process is as follows.

2.1 Basic theory

2.1.1 Basic concepts and lemmas

Definition: let (Z_1t, Z_2t, ⋯ , Z_nt), t ∈ T, be a n-dimensional random process defined on the probability space (Ω, m, P) [27]. Let (ɛ_1t, ɛ_2t, ⋯ , ɛ_nt), t ∈ T be the n-dimensional white noise process on (Ω, m, P).

For any t ⩽ i, j ⩽ n, s, t ∈ T, there are: ${\begin{matrix} E ɛ_{it} = u_{i} \\ E ɛ_{it} ɛ_{it} = δ_{i} δ_{is} \\ E ɛ_{it} ɛ_{js} = 0 \end{matrix}$ (1) where δ_is is 1 or 0.

Let η = (η_ij) _n×n be the n-order matrix of random variable elements, and (ɛ_1t, ɛ_2t, ⋯ , ɛ_nt) and η are independent of each other. If there is a matrix as follows: $[\begin{matrix} Z_{1 t} \\ Z_{2 t} \\ ⋮ \\ Z_{nt} \end{matrix}] = [\begin{matrix} ɛ_{1 t} \\ ɛ_{2 t} \\ ⋮ \\ ɛ_{nt} \end{matrix}] + [\begin{matrix} η_{11} \\ η_{12} \\ ⋮ \\ η_{1 n} \end{matrix}] [\begin{matrix} Z_{1 t - 1} \\ Z_{2 t - 1} \\ ⋮ \\ Z_{nt - 1} \end{matrix}]$ (2)

Formula (2) is called a n-dimensional first-order random coefficient model.

The matrix expression of Formula (2) is as follows: $Z_{t} = ɛ_{1} + η Z_{t - 1}$ (3) where, Z_t = (Z_1t, Z_2t, ⋯ , Z_nt) ^T, ɛ_t = (ɛ_1t, ɛ_2t, ⋯ , ɛ_nt) ^T, η = (η_ij) _n×n, Z_t-1 = (Z_1t-1, Z_2t-1, ⋯ , Z_nt-1) ^T, T represents the transposition of the matrix.

Lemma:

Let {X_t (ω)} and {Y_t (ω)} be two random processes defined in the probability space (Ω, m, P). If {X_t (ω)} and {Y_t (ω)} are independent of each other and Xⁿt (ω) = [X_t (ω)] ⁿ, then {X_t (ω)} and {Y_t (ω)} have the following relationship: $E [X_{t}^{n} (ω) Y_{t} (ω)] = E [X_{t}^{n} (ω)] E Y_{t} (ω)$ (4)

Proof: independence includes: $\begin{matrix} {EX}_{t}^{n} (ω) Y_{t} (ω) \\ = \iint X_{t}^{n} (ω_{1}) Y_{1} (ω_{2}) P (d ω_{1}, d ω_{2}) Ω \times Ω \\ = \iint X_{t}^{n} (ω_{1}) Y_{1} (ω_{2}) P (d ω_{1}) PA (d ω_{2}) Ω \times Ω \\ = \int_{Ω} X_{t}^{n} (ω_{1}) P (d ω_{1}) \int_{Ω} Y_{1} (ω_{2}) P (d ω_{2}) \\ = {EX}_{t}^{n} (ω) E Y_{t} (ω) \end{matrix}$ (5)

This lemma is also true in multi-dimensional cases. Formulas (1)–(5) represent one-dimensional or multi-dimensional stochastic processes in the probability space.

2.1.2 Classification of structural change points

Considering one-dimensional time series {Y_t}, t = 1, 2, ⋯ , n, whose expectation and variance are {u_t} and {σ_t}, respectively. If there are u_t ≡ u and $σ_{t}^{2} = σ^{2}$ , and u and σ² are constants, then {Y_t} comes from the same model. Otherwise, it means that {Y_t} does not come from the same model, and there are structural change points in the sequence [3 , 14]. General structural change points can be divided into three categories: change points where expectations change (mean shift), change points where variances change (volatility change points), and change points where expectations and variances change at the same time. The above structural change point idea can be extended to multi-dimensional time series and other non-time series models.

2.2 Multi-dimensional stochastic graphical modeling model

Graphical Modeling is based on multivariate normal hypothesis, so this section briefly introduces the basic knowledge of this part.

2.2.1 Covariance selection model

Under the assumption of multivariate normality, Dempster firstly proposed a covariance matrix modeling method in which some non-diagonal elements of the precision matrix (i.e., the inverse of the covariance matrix) are zero [12], namely Covariance Selection model. This model studies the conditional independence between multivariate normal variables.

Suppose Y = (Y₁, ⋯ , Y_q) ^T is a q-dimensional random vector subject to multivariate normal distribution, and the expectation vector and covariance matrix of multivariate normal distribution are respectively: $μ = (\begin{matrix} μ^{1} \\ ⋮ \\ μ^{q} \end{matrix})$ (6) $Σ = (\begin{matrix} σ^{11} & \dots & σ^{1 q} \\ ⋮ & ⋱ & ⋮ \\ σ^{q 1} & \dots & σ^{qq} \end{matrix})$ (7)

Since we are more interested in the inverse of the covariance matrix, k = 0 is called the precision matrix. $K = (\begin{matrix} k^{11} & \dots & k^{1 q} \\ ⋮ & ⋱ & ⋮ \\ k^{q 1} & \dots & k^{qq} \end{matrix})$ (8)

According to the properties of multivariate normal distribution, under the given condition of (Y₃, ⋯ , Y_q), the conditional distribution of (Y₁, Y₂) is a binary normal distribution, and its covariance matrix is: $\begin{matrix} {(\begin{matrix} k^{11} & k^{12} \\ k^{21} & k^{22} \end{matrix})}^{- 1} = \\ \frac{1}{k^{11} k^{22} - {(k^{12})}^{2}} (\begin{matrix} k^{22} & - k^{21} \\ - k^{12} & k^{11} \end{matrix}) \end{matrix}$ (9)

Therefore, the correlation coefficient of this binary normal distribution is: $ρ^{1, 2, \dots, q} = \frac{- k^{12}}{{(k^{11} k^{12})}^{1 / 2}}$ (10)

This is the partial correlation coefficient of Y¹ and Y². Obviously, k¹² = 0 is equivalent to ρ^1,2,⋯,q = 0. Therefore, the two variables Yⁱ and Y^j are conditionally uncorrelated (equivalent to conditionally independent in multivariate normal distribution), if and only if k^ij = 0.

The above conclusion can be derived from another angle. The density function of Y can be written as: $\begin{matrix} f (y) = exp {| 2 π Σ |}^{- 1 / 2} \\ exp {- \frac{1}{2} {(y - μ)}^{T} Σ^{- 1} (y - μ)} \end{matrix}$ (11) We can also write it as: $f (y) = exp (α + β^{T} y - \frac{1}{2} y^{T} Ky)$ (12) where, K = Σ^-1, β = Σ^-1 μ, $α = - \frac{1}{2} ln | Σ | - \frac{1}{2} μ^{T} Σ^{- 1} μ - \frac{q}{2} ln (2 π)$ .

Therefore, Formula (11) can be converted as follows: $f (y) = exp (\begin{matrix} α + \sum_{i = 1}^{q} β^{T} y_{i} - \\ \frac{1}{2} \sum_{i = 1}^{q} \sum_{j = 1}^{q} k^{ij} y_{i} y_{j} \end{matrix})$ (13)

It can be seen that if and only if k^ij = 0, f (y) can be expressed as the product of two parts without y_i and y_j, that is, the density function can be decomposed. Therefore: $Y_{i} ∐ Y_{j} | Y ∖ {Y_{i}, Y_{j}} \Leftrightarrow k^{ij} = 0$ (14) where the symbol ∐ denotes conditional independence. Formulas (6)–(14) indicate the acquisition of conditional independence between multivariate normal variables in the process of covariance selection.

2.2.2 Graphical Gaussian model

Whittaker introduced the Covariance Selection model in detail and combined it with graph theory to develop an undirected graph model based on multivariate normal hypothesis, and called it the Graphical Gaussian model. The graph g = (V, E), where V is the node set, E is the edge set, and the edge between variables i and j is denoted as (i, j). Let X = (X_i) _i∈V be a d-dimensional multivariate normal random vector, where each variable is represented by a node of graph g. Let μ and K = Σ^-1 represent the expectation vector and the precision matrix of the multivariate normal random vector X, respectively.

In the graphical Gaussian model, the following three are equivalent:

The edge between the variables X_i and X_j does not exist, i.e. (i, j) ∉ E;

The variables X_i and X_j are conditionally independent under the conditions given by other variables, that is, X_i ∐ X_j|X_V∖{i,j};

The element is k_ij = k_ji = 0 in the precision matrix.

Therefore, from graph g, we can intuitively see the conditional independent relationship between variables.

2.2.3 Relationship between graphical Gaussian model and linear regression model

If x is a group of samples of X, the log likelihood function value is: $\begin{matrix} f (x | μ, K, g) = - \frac{d}{2} log (2 π) + \\ \frac{1}{2} log | K | - \\ \frac{1}{2} \sum_{i \in V} \sum_{j : (i, j) \in E} (x_{i} - μ_{i}) k_{ij} (x_{j} - μ_{j}) \end{matrix}$ (15) where μ_i is the i-th element of μ.

X_-i ={ X_j, j ≠ i }. According to the properties of multivariate normal distribution, the conditional distribution of X_i is also normal under the given condition of X_-i, and its expectation and variance are: $\begin{matrix} E (X_{i} | X_{- i}) = μ_{i} - \sum_{j \neq i} \frac{k_{ij}}{k_{ii}} (X_{j} - μ_{j}) \\ = μ_{i} - \sum_{j : (i, j) \in E} \frac{k_{ij}}{k_{ii}} (X_{j} - μ_{j}) \end{matrix}$ (16) $Var (X_{i} | X_{- i}) = \frac{1}{k_{ii}}$ (17)

It can be regarded as the conditional expectation of the linear regression model with X_-i as the independent variable, X_i as the response variable, and the residuals subject to normal distribution. The coefficient of the regression model is: $β_{ij} = \frac{k_{ij}}{k_{ii}} \times 1 {(i, j) \in E, i \neq j}$ (18)

Intercept items are: $β_{i 0} = μ_{i} - \sum_{j = 1} β_{ij} u_{j}$ (19)

Formulas (15)–(19) reflect the process of obtaining the relationship between the Graphical Gaussian model and the linear regression model.

2.3 Parameter estimation of multi-dimensional stochastic model with intermediate variables

2.3.1 Intermediate variable model

People’s understanding of things is usually based on the correlation of relevant variables or further causality. However, with the gradual deepening of people’s understanding of things, simple causality can no longer accurately describe the laws of things. The “stimulus-organism-response” model with mediating variables believes that the effect of external stimuli on behavior occurs through the internal conversion mechanism of the organism, that is, the organism plays a mediating role in the causal relationship between “stimulus” and “response”.

The so-called intermediate variable is simply the third variable that can change the causal relationship between the independent variable and the dependent variable. It is a more accurate description of the direct causal relationship between variables. The more precise definition of the intermediate variable is: if the independent variable X has an impact on the dependent variable Y, and X affects Y through the variable M, then M is the intermediate variable.

There are two types of mediation variables, one is full mediation variable and the other is partial mediation variable. The so-called complete intermediate variable means that the effect of the independent variable X on the dependent variable Y is completely realized by the intermediate variable M, and M is the complete intermediate variable. Some intermediate variables refer to the direct effect of independent variable X on dependent variable Y and the intermediate effect on Y through intermediate variable M.

The calculation of intermediate variables is based on the action mechanism of intermediate variables. The action mechanism of intermediate variables is as follows:

There is a causal relationship between independent variable X and dependent variable Y;

The independent variable X exerts influence on the variable M, and M exerts influence on Y, thus acting as an intermediary.

Based on the action mechanism of the above intermediary variables, the three steps to test the intermediary variables from the data relationship are as follows:

The independent variable X affects dependent variable Y;

The independent variable X affects the intermediate variable M;

When there is no intermediate variable M, the effect of X on Y will be significantly reduced or even disappear.

Assuming that all variables have been standardized, the relationship between each variable has the following formulas: $Y = wX + e_{1}$ (20) $M = a^{*} X + e_{2}$ (21) $Y = c^{*} X + b^{*} M + e_{3}$ (22) where w is the total effect of X on Y, a^* and b^* are the intermediary effect of X on Y via mediator, and c is the direct effect of X on Y. The following relationships exist among the effects: $w = c + a * b$ (23)

Since the mediating variable acts on the relationship between the independent variable and the dependent variable, before discussing the mediating effect, we must ensure that there is a causal relationship between the independent variable X and the dependent variable Y. Formulas (20)–(23) reflect the judgment process of the causal relationship.

2.3.2 Model parameter estimation of PLS algorithm

Structural equation modeling method has been rapidly and widely used. Although there are many solutions to the early structural equation model, the most widely used method is the LISREL method based on the covariance matrix which expresses the nonlinear structural relationship. The premise of this method is that the covariance matrix of the measured variables is a function of the parameters to be estimated. H is the covariance matrix of the measured variable population [15], which is actually replaced by the covariance matrix of the sample, that is, it is obtained from the collected questionnaire data. If ∑ (w) is the covariance matrix of the parameter to be evaluated, then the corresponding parameter value can be solved by using equation H = ∑ (w) according to the equality of the equations at the corresponding positions in the matrix. If the equation containing the parameter to be estimated in the matrix is linear, the parameter value can be accurately estimated. However, if it is nonlinear, it is necessary to minimize the covariance difference between the sample covariance matrix and the covariance matrix of the parameter to be estimated.

In general, the purpose of LISREL is to minimize the difference between the sample covariance matrix and the covariance matrix of the parameter to be estimated, which is the fitting function f (H, ∑ (w)). The method used to find the minimum value of the fitting function is the maximum likelihood estimation method. The fitting function is: $f_{ML} = log | \sum (w) | + tr (H Σ^{- 1} (w)) - log | H | - σ$ (24)

However, the biggest disadvantage of this method is that the premise assumption is too strict, including the sample must conform to normal distribution and the sample size must reach a certain number. And this method can not check the specified error. Therefore, LISREL method is replaced by PLS algorithm and corresponding solution software. The partial least square path model also includes two basic models: measurement model and structural model [25]. Generally, the measurement model is called the external model, and the structural model is called the internal model. In the external model, the relationship between the measured variable X_i and the corresponding latent variable ξ_i can be expressed by the following formula: $x_{ij} = λ_{ij} * ξ_{i} + ɛ_{ij}$ (25) where, x_ij represents the j-th measurement variable corresponding to the i-th potential variable, ξ_i represents the i-th potential variable, λ_ij represents the corresponding factor load, and ɛ_ij is the residual of x_ij. In the internal model, the causal relationship between potential variables can be expressed by the following formula: $ξ_{i} = \sum_{i \neq j} β_{ij} * ξ_{j} + ς_{i}$ (26)

In the above formula, ξ_i represents the i-th latent variable, ξ_j is the j-th latent variable, β_ij is the path coefficient between the i-th and the j-th latent variables, and ς_i is the residual of ξ_i.

The partial least squares path method estimates the latent variable through multiple iterations, and then estimates the factor load corresponding to the measured variable according to the external model setting. There are two methods to estimate the latent variable ξ_i, one is based on the setting of the external model, and the other is based on the relationship between the latent variables in the internal model [8 , 30]. The former is called external estimation and the latter is called internal estimation.

The estimation method of factor load vector $\vec{W_{i}}$ is: $\vec{W_{i}} = \frac{1}{p} * X_{i}^{T} * Z_{i}$ (27) where, p represents the length of the vector $\vec{W_{i}}$ , Z_i is the internal estimator of ξ_i, and $X_{i}$ is the measurement variable corresponding to the i-th latent variable. Formulas (24)–(27) represent the process of obtaining the estimation results of variable parameters.

When the calculation converges, the estimated values of the measured variable and the latent variable can be obtained. The PLS algorithm does not require the sample data to satisfy strict assumptions such as the LISREL estimation method. The iterative process accurately estimates the relationship between the measured variables and the latent variables in the external model [26], and uses the truncated sample data information as the estimation of the residual of the measured variables through the idea of component extraction. In addition, the causal relationship between latent variables is also well estimated. Therefore, it is a more practical method for estimating SEM parameters.

2.4 Accurate estimation of structural change points based on MLE identification of local drift time

In order to simplify the problem of identifying local feature out of control, it is assumed that there is only a functional relationship of a changed input variable value interval, and then the minimum interest interval with local drift can be obtained. If it is proved that the response function does change in this interval, the problem of local difference is transformed into the classical problem of change point identification. It is assumed that there are m normal production cycles, that is, in the whole input variable value range, the relationship between the output variable X_i and the input variable Y_ji conforms to the function f₁, and the contour segmentation and contour difference density of the observed value are calculated to obtain the average difference density due to the random error in the controllable state [7]. The control limit is 95% of the mean difference density. Because of its poor normality, if μ + 3σ is used as the control line, it is easy to cause false alarm signals. The simulation software MATLAB also shows that the algorithm of setting quantile as control limit is effective. Since only one local section drifts, the maximum value of contour difference density in uncontrollable state can be taken. If the control limit is exceeded, it is judged that the process deviates from the original state and an alarm signal is sent.

Next, for the abnormal interval, the drift from the production cycle is detected. It is assumed that the process starts in a controlled state, the input variable and the output variable follow the polynomial function (y = β₀ + β₁x + β₀x² + ⋯ + β_kx^k + ɛ), and the error is σ₀. However, from the unknown time point τ (i.e., the process change point), the response function changes to (y = β′₀ + β′₁x + β′₂x² + ⋯ + β′_kx^k + ɛ). The changed function is still a polynomial function, but the values of coefficients and constant terms of different orders of the dependent variable X are changed. Once the response function changes but the non-random factors are not identified and removed [17], the subsequent production process will obey the new functional relationship f₂ in this local interval. In this study, we consider that t, which maximizes C1 on 0 ⩽ t ⩽ m, is the estimated value of the structural change point τ of the multi-dimensional stochastic model. The calculation formula is as follows: $C_{t} = (m - t) {(\bar{\bar{Y_{m, t}}} - μ_{0})}^{2}$ (28) where $\bar{\bar{Y_{m, t}}}$ is the reciprocal of the cumulative mean of the first m - t nearest subsets: $\bar{\bar{Y_{m, t}}} = 1 / (m - t) \sum_{i = t + 1}^{m} \bar{Y_{i}}$ (29)

Specifically, in order to identify the structural change points of the multi-dimensional stochastic model, we first calculate the reciprocal of the cumulative sum of t = 0, 1, 2, ⋯ , m - 1 means (m is the total number of profiles, that is, the production cycle). The next step is to calculate the statistic C_t of the maximum likelihood estimation. Since the mean parameter of the output variable Y is unknown, here we use the sample mean as the unbiased estimator of μ₀: $\hat{μ_{0}} = \frac{1}{m} \sum_{i = 1}^{m} y_{i}$ (30)

In order to deeply understand the meaning of the above formula, the following section will give a detailed derivation of the maximum likelihood estimator (MLE) at the change time τ. Here, $\hat{τ}$ is taken as the maximum likelihood estimation of τ [2 , 28]. Considering the mean subgroup ${\bar{y_{1}}, \bar{y_{2}}, \dots, \bar{y_{m}}}$ , the MLE estimation value of τ is τ that maximizes the likelihood function, or τ that maximizes the logarithm. The log form of the likelihood function is: $\begin{matrix} logL (τ, | \bar{y}) = - \frac{n}{2 σ_{0}^{2}} (\sum_{i = 1}^{m} \begin{matrix} {\bar{y_{i}}}^{2} - 2 μ_{0} \sum_{i = 1}^{τ} \bar{y_{i}} + \\ τ μ_{0}^{2} - (m - τ) {\bar{\bar{y_{m, τ}}}}^{2} \end{matrix}) \\ = - \frac{n}{2 σ_{0}^{2}} (\sum_{i = 1}^{m} \begin{matrix} {\bar{y_{i}}}^{2} - 2 μ_{0} \sum_{i = 1}^{τ} \bar{y_{i}} - τ μ_{0}^{2} - \\ 2 μ_{1} \sum_{i = τ + 1}^{m} \bar{y_{i}} + (m - τ) {μ_{1}}^{2} \end{matrix}) \end{matrix}$ (31)

We note that there are two unknowns in the log likelihood function: τ and μ₁. If τ is obtained, the maximum likelihood estimation of μ₁ will be: $\hat{μ_{1}} = \bar{\bar{y_{m, τ}}} = {(m - τ)}^{- 1} \sum_{i = τ + 1}^{m} \bar{y_{i}}$ (32)

$\bar{\bar{y_{m, τ}}}$ is the average of the mean of the m - τ nearest subsets. This brings back Formula (31) to obtain: $\begin{matrix} logL (τ, | \bar{y}) = \\ - \frac{n}{2 σ_{0}^{2}} (\sum_{i = 1}^{m} \begin{matrix} {\bar{y_{i}}}^{2} - 2 μ_{0} \sum_{i = 1}^{τ} \bar{y_{i}} + \\ m μ_{0}^{2} - (m - τ) {(\bar{\bar{x_{m, τ}}} - μ_{0})}^{2} \end{matrix}) \end{matrix}$ (33)

Therefore, it is possible to obtain τ that the log likelihood function is maximum: $\begin{matrix} \hat{τ} = \underset{t}{argmax} {(m - t) {(\bar{\bar{x_{m, τ}}} - μ_{0})}^{2}} = \\ \underset{t}{argmax} {C_{t}} \end{matrix}$ (34) here $C_{t} = (m - t) {(\bar{\bar{y_{m, τ}}} - μ_{0})}^{2}$ . In other words, $\hat{τ}$ is the value of t in the interval 0 ⩽ t < m such that C_t has the maximum value. Formulas (28)-(34) reflect the process of finally obtaining the estimated value of the structural change point of the multidimensional random model.

To sum up, the algorithm flow of accurate estimation of structural change points of multidimensional random model is shown in Fig. 1.

Fig. 1

Algorithm flow for accurate estimation of structural change points of multidimensional random model.

3 Results

In this paper, three evaluation criteria will be used to test the performance of the new algorithm framework for simultaneous identification of abnormal local intervals and change times, namely, the first type of error (type I ERR), the second type of error (type II ERR), and the non-signal probability.

The type I error refers to the probability that the normal observation value under control is considered to be an abnormal condition, that is, true is taken as false. For example, there are 400 contour curves in total, and the local function drift occurs from the 200th contour curve and then remains at the abnormal level. However, through calculation, the 180th contour curve is determined as the time point at which the uncontrolled scene begins to occur. Therefore, the probability of type I error is 0.10. Similarly, the probability of type II error is equal to the possibility of “taking the false as the true”, that is, the abnormal observation value is judged as normal. For example, the alarm signal occurs on the 220th contour curve, but the real drift actually occurs from the 200th contour curve. Therefore, the originally normal process (from the 200th contour curve to the 220th contour curve) is considered to be an uncontrolled state. In this scenario, the probability of type II error is 0.10. It should be noted here that as an important standard, experienced engineers and academic researchers generally believe that the ideal calculation results of the first and second types of error probabilities are better than 0.05. In this study, the third evaluation criterion is non-signal probability. Suppose that we will conduct S = 5000 simulation operations, and the actual process drifts, but there are s simulation cycles in which the uncontrolled process is not identified. The unrecognized rate is equal to s/S. Under such a definition, an unrecognized rate close to zero means that statistical process control can identify almost all abnormal points. In other words, the probability of type I error and the probability of type II error reflect the accuracy of the algorithm in identifying change points, while the unrecognized rate can reflect the timeliness of identifying abnormal changes. It is reasonable to judge the effectiveness and accuracy of the new algorithm through the probability of type I error, the probability of type II error and the unidentified probability.

In order to verify the effectiveness of the algorithm in estimating the structural change points of the multi-dimensional stochastic model, simulation experiments are conducted to investigate the effect of the change point location estimation. Among them, the number of iterations is 2000, and the significance level was α=0.04 and the number of simulations is 200. The investigation results are shown in Fig. 2.

Fig. 2

The position estimation result of the structural change point.

It can be seen from Fig. 2 that when the number of iterations is 190 and 302 respectively, there are change points. And near the change point, the frequency of THL statistics higher than the critical value can be as high as 90%. And the frequency at which the value of THL statistic is greater than the critical value is about 5% at the position far from the change point. This shows that our model can accurately estimate the position of the change point.

In order to further verify the estimation accuracy of the proposed algorithm for structural change points, the algorithm in reference [5] (change point detection algorithm of multiple structural change points of linear regression model based on Bayesian posteriori reasoning and genetic algorithm) and algorithm in reference [9] (calculation algorithm of offset effect combined with principal component and projection matrix) are used as comparison algorithms to compare the simulation results of type I error, type II error and the unidentified rate of different algorithms.

For the change of the primary term coefficient, the simulation results of the probability of type I error and the type II error and the unrecognized rate are shown in Table 1.

Table 1

Drift of the first-order coefficients

Algorithm	Criteria	(a,b,c)	(1,2.4,3)	(1,2.5,3)	(1,2.6,3)
Proposed algorithm	err1	T²	0.2567	0.2752	0.2452
		Segmentation	0.1152	0.0009	0.000
	err2	T²	0.2785	0.2852	0.2561
		Segmentation	0.1785	0.0008	0.000
	Non-signal	T²	0.3685	0.0005	0.000
		Segmentation	0.4865	0.0001	0.000
Algorithm in reference [5]	err1	T²	0.2869	0.2947	0.2615
		Segmentation	0.1934	0.0624	0.0931
	err2	T²	0.2938	0.3168	0.2649
		Segmentation	0.2215	0.0957	0.0387
	Non-signal	T²	0.4586	0.1638	0.0615
		Segmentation	0.6025	0.0849	0.0589
Algorithm in reference [9]	err1	T²	0.2841	0.2899	0.2497
		Segmentation	0.1689	0.0323	0.0237
	err2	T²	0.2972	0.3045	0.2651
		Segmentation	0.2137	0.0574	0.0128
	Non-signal	T²	0.4197	0.0096	0.0092
		Segmentation	0.5365	0.0078	0.0171

Since the influence of the first-order coefficient drift is not as obvious as that of the second-order coefficient drift, the step size of the specified drift is 0.05 instead of 0.001. The algorithm proposed in this paper can be implemented effectively. Especially after b = 2.5, the unrecognized rate close to zero can ensure the timely alarm for the change point. On the other hand, no matter how the coefficient of the primary term changes, the type I and type II errors of the T² statistic algorithm exceed many, which means that they can not accurately identify the change point. Although the unrecognized rate of this algorithm is relatively acceptable, it is meaningless because it alarms in the correct process state. To sum up, compared with algorithm in reference [5] and reference [9], the algorithm proposed in this paper has a lowerer probability of type I error and type II error under all possible combinations of abnormal conditions. As for the unidentified probability, the proposed algorithm can detect the local drift of the process in time. The unidentified probability of the other two algorithms is higher than that of the proposed algorithm.

Because the research object of this paper is about the partial drift of a response variable in a multi-dimensional stochastic model, it is necessary to discuss the effect of the new algorithm under different local proportions. Therefore, we will discuss the estimation effect of the proposed algorithm on structural change points when the proportion of local drift interval changes from 0.2 to 0.4. The comparative analysis is still based on the above three evaluation indicators, the first type error probability, the second type error probability and the unidentified probability. The calculation results of this algorithm are shown in Table 2.

Table 2

Estimation of quadratic coefficient variation under different local scales

Algorithm	Proportion		0.2	0.3	0.4
Proposed algorithm	err1	T²	0.2584	0.2354	0.2354
		Segmentation	0.1165	0.0285	0.0009
	err2	T²	0.2754	0.2784	0.2584
		Segmentation	0.1985	0.0385	0.0028
	Non-signal	T²	0.5946	0.0428	0.0005
		Segmentation	0.3945	0.0945	0.0012
Algorithm in reference [5]	err1	T²	0.2897	0.2538	0.2503
		Segmentation	0.1393	0.0537	0.0489
	err2	T²	0.2934	0.2916	0.2739
		Segmentation	0.2272	0.1131	0.0314
	Non-signal	T²	0.6125	0.0617	0.0097
		Segmentation	0.4137	0.1291	0.0135
Algorithm in reference [9]	err1	T²	0.2693	0.2465	0.2417
		Segmentation	0.1306	0.0391	0.0082
	err2	T²	0.2817	0.2885	0.2637
		Segmentation	0.2075	0.0785	0.0149
	Non-signal	T²	0.5989	0.0524	0.0065
		Segmentation	0.4102	0.1069	0.0046

As shown in Table 2, when the local change ratio of the quadratic term coefficient of the multi-dimensional stochastic model changes from 0.2 to 0.4, the algorithms for data segmentation of the multi-dimensional stochastic model have relatively low type I and type II error probabilities, which means that the accuracy advantage of the proposed precise estimation calculation algorithm for structural change point is robust. However, the type I error probability, type II error probability and unidentified probability of algorithm in Reference [5] and Reference [9] are higher than those of the proposed algorithm. The purpose of the algorithm proposed in this paper is to deal with the local characteristics of the multi-dimensional stochastic model. The algorithm is effective in estimating the structural change points of the multi-dimensional stochastic model.

4 Discussion

The accurate estimation algorithm for structural change point of multi-dimensional stochastic model is studied, and the structural change point of multi-dimensional stochastic model is estimated accurately. In any manufacturing process, no matter how well designed it is, some inherent variations always exist. These “natural variation” or “background noise” are the cumulative effects of many basically unavoidable minor causes. In the framework of statistical quality control, natural error is often referred to as the opportunity factor of stable system. In other words, the opportunity factor is an inherent component of the process. Other errors may occur accidentally in the process. This error generally comes from three sources: improper adjustment and control of equipment, operator error, or defective raw materials. Compared with background noise, such error is relatively large, and it usually represents an unacceptable level of process performance. We call this part which is not an opportunity factor non-random error. If a process has a non-random error, it means that it is in an uncontrolled state.

The process before time t₁ is controlled, that is, there is only random error. Therefore, both the mean and standard deviation are their controlled values (μ₀ and σ₀). When the non-random error occurs at time t₁, the effect is that the process average shifts to a new value μ₁ > μ₀. At time t₂, another nonrandom error occurs, making μ > μ₀, but now the standard deviation of the process shifts to a larger value σ₁ < σ₀. There is another non-random error at time t₃, which causes the process mean and standard deviation to change. From time t₁, the existence of non-random error leads to an uncontrolled process.

The process will run in a controlled state for a relatively long time. However, no process can be truly stable forever, and non-random errors will eventually occur, resulting in an abnormal production state and making the output products to a large extent not meet the requirements. When the process is under control, most of the product characteristic values will fall between the upper and lower control limits. When the process deviates from the original production status, a relatively high proportion of the process exceeds the specified limit. The main objective of statistical process control is to quickly identify the process drift caused by non-random errors, and conduct in-depth investigation and correction of the process before unqualified units are output. Therefore, it is of great importance to accurately estimate the structural change points of multi-dimensional stochastic models.

5 Conclusion

In this paper, the multi-dimensional stochastic model is taken as the research object, and the structural change points of the multi-dimensional stochastic model are accurately estimated. The relationship between multi-dimensional stochastic variables is used as the basis for accurate estimation of structural change points. The correlation mentioned here refers to the conditional correlation of two variables given all other variables. This conditional correlation is not necessarily constant. With the change of time, the two variables that are originally conditionally independent may become conditionally related, and vice versa. Therefore, in order to accurately find the conditional correlation between multi-dimensional stochastic variables, we must ensure that there are no change points in the sample sequence. However, we can directly detect whether there are fluctuating change points in the sample sequence. If there are no fluctuating change points, we can ensure that there are no change points in the graph structure. After finding the position of change point, we find the structure of the graph in a single time block without time change point. The simulation experiments show that the proposed algorithm has certain theoretical and practical value, and can solve the problem of structural change point of multi-dimensional stochastic model. Simulation results show that compared with the comparison algorithm, the proposed algorithm has lower class I error probability, class II error probability and unidentified probability under the change of the coefficients of the primary and secondary terms, which indicates that its estimation performance of structural change points is better. The proposed algorithm makes a multivariate normal hypothesis, extends the calculation of structural change points to multidimensional time series, constructs a multidimensional random graphic modeling model, introduces intermediate variables between independent variables and dependent variables, and replaces LISREL method with PLS algorithm and corresponding solution software. These improvements make the proposed algorithm realize the purpose of solving the structural change point problem of multi-dimensional random model, and have certain theoretical and practical value.

Footnotes

Acknowledgments

This research was supported by the Henan Soft Science Research Plan Project (grant no. 212400410114).

References

Grover

and Lall

, A stochastic model for characterizing fluctuations in chemical sensings, IEEE Transactions on Signal Processing 69 (2021), 1938–1951.

Zhu

, Zhong

, Chen

, Liao

, Li

, Shi

and Sotelo

M.A.

, A novel reconstruction method for temperature distribution measurement based on ultrasonic tomography, IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control 69(7) (2022), 2352–2370.

Zong

and Wan

, Container ship cell guide accuracy checktechnology based on improved 3D point cloud instance segmentation, Brodogradnja 73(1) (2022), 23–35.

Zong

, Wang

and Wan

, An improved 3D point cloud instance segmentation method for overhead catenary height detection, Computers & Electrical Engineering 98 (2022), 107685.

D.Q.

and Zhao

W.H.

, Detection of multi-structure change point based on genetic algorithm and its application, Statistics and Decision 38(6) (2022), 21–25.

F.X.

, Xiao

Y.T.

and Chen

Z.S.

, Sequential change-point detection in cumulative logistic regression model, Mathematica Applicata 35(2) (2022), 402–411.

Ito

, Itasaka

, Takeda

and Kitahata

, Experimental study on the bifurcation of a density oscillator depending on density difference, EPL (Europhysics Letters) 129(1) (2020), 18001.

Zheng

and Jin

, A multi–source fluid queue based stochastic model of the probabilistic offloading strategy in a MEC system with multiple mobile devices and a single MEC server, International Journal of Applied Mathematics and Computer Science 32(1) (2022), 125–138.

Chen

H.Y.

and Feng

, The offset effect of common factors on panel data structure change points, Statistics and Decision 37(13) (2021), 28–31.

10.

Haasler

, Singh

, Zhang

, Karlsson

and Chen

, Multi-marginal optimal transport and probabilistic graphical model, IEEE Transactions on Information Theory 67(7) (2021), 4647–4668.

11.

Chen

, Du

and Guo

, Label constrained convolutional factor analysis for classification with limited training samples, Information Sciences 544 (2021), 372–394.

12.

Chen

, Xu

, Gesbert

, Yang

and Zhi

, Spatially asymptotic behavior of structured covariance matrix estimation for massive MIMO, IEEE Communications Letters 25(8) (2021), 2594–2598.

13.

, Nie

and Zhu

, Characterization and selection of probability statistical parameters in random slope PWM based on uniform distribution, IEEE Transactions on Power Electronics 36(1) (2020), 1184–1192.

14.

Huang

K.W.

, Wang

H.M.

, Towsley

and Poor

H.V.

, LPD communication: A sequential change-point detection perspective, IEEE Transactions on Communications 68(4) (2020), 2474–2490.

15.

Wang

, Wang

W.Q.

and Zhou

, FDA-MIMO radar covariance matrix estimation via shrinkage processing, Digital Signal Processing 118(7) (2021), 103206.

16.

Yin

and Wang

W.G.

, Estimation and testing methods of panel models with structural changes, Statistics and Decision 37(18) (2021), 5–9.

17.

Diego

and Gallego

, Comment on “the WINFREE model with non-infinitesimal phase-response curve: Ott–antonsen theory”, Chaos 31(1) (2021), 018101.

18.

Hislop

P.D.

and Marx

C.A.

, Dependence of the density of states on the probability distribution for discrete random schrdinger operators, International Mathematics Research Notices 2020(17) (2020), 5279–5341.

19.

Mishra

P.K.

, Diwale

S.S.

, Jones

C.N.

and Chatterjee

, Reference tracking stochastic model predictive control over unreliable channels and bounded control actions, Automatica 127(9) (2021), 109512.

20.

Zhang

, Ge

, Hensley

, Isabel Metternicht

, Liu

and Zhang

, PolGAN: A deep-learning-based unsupervised forest height estimation based on the synergy of PolInSAR and LiDAR data, ISPRS Journal of Photogrammetry and Remote Sensing 186 (2022), 123–139.

21.

Jia

Q.Q.

, Hao

M.Z.

and Fu

X.Q.

, Research on regression modeling of high-dimensional complex systems based on statistical machine learning, Computer Simulation 39(1) (2022), 333–338+417.

22.

Mihaescu

, Kampermann

, Gianfelici

, Isar

and Bru

, Detecting entanglement of unknown continuous variable states with random measurements, New Journal of Physics 22(12) (2020), 123041–123047.

23.

Ciccone

, Ferrante

and Zorzi

, Learning latent variable dynamic graphical models by confidence sets selection, IEEE Transactions on Automatic Control 65(12) (2020), 5130–5143.

24.

Elvira

and Santamaria

, Multiple importance sampling for symbol error rate estimation of maximum-likelihood detectors in MIMO channels, IEEE Transactions on Signal Processing 69 (2021), 1200–1212.

25.

Yang

W.Y.

, Wang

W.M.

and Zhang

R.Q.

, A modified moving-window partial least-squares method by coupling with sampling error profile analysis for variable selection in near-infrared spectral analysis, Analytical Sciences 36(3) (2020), 303–309.

26.

, Zhao

and Liu

, Latent variable iterative learning model predictive control for multivariable control of batch processes, Journal of Process Control 94(6) (2020), 1–11.

27.

Yuan

, Liu

, Valdebenito

M.A.

, Gu

and Beer

, Efficient procedure for failure probability function estimation in augmented space, Structural Safety 92(3) (2021), 102104.

28.

Zenggang

, Mingyang

, Xuemin

, Sanyuan

, Fang

, Xiaochao

, Yunyun

and Xiang

, Social similarity routing algorithm based on socially aware networks in the big data environment, Journal of Signal Processing Systems 94 (2022), 1253–1267.

29.

Zeng

, Xu

, Qin

and Yin

, A semi-Markov decision model with inverse reinforcement learning for recognizing the destination of a maneuvering agent in real time strategy games, IEEE Access 8 (2020), 15392–15409.

30.

, Zheng

, Chen

and Yin

, Joint embedding VQA model based on dynamic word vector, PeerJ Computer Science 7 (2021), e353.