Comprehensive evaluation measures of nonlinear estimation algorithm performance

Abstract

Although many scholars say that their algorithms are better than others in the state estimation problem, only a fewer convincing algorithms were applied to engineering practices. The reason is that their algorithms outperform others only in some aspects such as the estimation accuracy or the computation load. To solve the problem of performance evaluation of state estimation algorithms, in this paper, the comprehensive evaluation measures (CEM) for evaluating the nonlinear estimation algorithm (NEA) is proposed, which can comprehensively reflect the performance of the NEAs. First, we introduce three types of the NEAs. Second, the CEM combining the flatness, estimation accuracy and computation time of the NEAs, is designed to evaluate the above NEAs. Finally, the superiority of the CEM is verified by a numerical example, which helps decision makers of nonlinear estimation algorithms theoretically and technically.

Keywords

Performance evaluation nonlinear estimation algorithm comprehensive metrics error spectrum EKF,UKF,PF

1 Introduction

Nonlinear estimation algorithm (NEA) was widely used in many fields, such as target tracking [1, 2], satellite navigation [3], fault detection [4], signal processing [5] and optimal control implementation [33]. Typical NEA for nonlinear dynamical systems include the extended Kalman Filter (EKF) [7, 8], unscented Kalman filter (UKF) [9], particle filter (PF) [10, 11], where the EKF is the most popular one, which is based on the Taylor series expansions of the nonlinear functions in the description of the models. In the last two decades, a number of new filters based on the KF framework have been proposed. Instead of the Taylor series expansions, they employed stochastic linearization to estimate a random variable by a group of points, which are transformed from the nonlinear functions [12]. This filter is called sigma-point filter and its advantage is that it doesn’t to compute the Jacobi matrices of the nonlinear functions. The unscented transform (UT) adopted in the unscented Kalman filter (UKF) [13] is one of the stochastic linearization methods. However, an important assumption for EKF and UKF is that the prior density is Gaussian. As a result, several modifications were presented to account for the non-Gaussianity of the various densities, such as the PF [14, 15] which implements recursive filters via Monte Carlo simulations. The above two approaches can cope well with non-linear and/or non-Gaussian filtering and estimation. [16] proposed a new filtering technique to solve a nonlinear state estimation problem and used the root mean square error, percentage of track loss and execution time to demonstrate the proposed filter provides more accurate results than the cubature Kalman filter, the unscented Kalman filter, and the Gauss-Hermite filter. [17] evaluate the performance of our EKF-STDF against some representative positioning algorithms by numerical simulations in terms of root mean squared error (RMSE) between the estimated positions and the true positions. For the tight integration of low-cost ultra wide band (UWB) ranging sensors, the EKF. [18] used the average time and standard deviation to compare the Double-EKF with the EKF, the indirect Kalman filter (IKF), UKF, the factored quaternion algorithm (FQA), and the algebraic quaternion algorithm (AQUA). [19] applied the extraction error, filtering deviation (DE), and RMSE to compare the performance of the radial basis function-neural network based on the improved UKF algorithm, the traditional UKF algorithm, and long short-term memory algorithm at different SNRs. The position errors and the estimated 3σ² bounds along the East and North directions are utilized to compare the performance of the reduced UKF, consensus-based distributed UKF, federated UKF, sequential UKF and standard UKF [20]. [21] the RMSE is used to evaluation the performance among the UKF, strong tracking UKF, M-estimation UKF, variational Bayesian VBHUKF, removed strong tracking UKF, and generalized strong tracking UKF. The prediction performance is evaluated and compared among the PF, unscented PF, radial basis function, long short-term memory, and PF+LSTM methods based on multiple criteria including absolute error, mean absolute error, RMSE, and Relative accuracy [22]. Furthermore, the cumulative probability distribution (CPD) of errors is used to evaluation the performance of the inertial measurement unit-only method, the EKF algorithm, the UKF algorithm and the Grey wolf optimizer-PF [23].

In practice applications, many scholars say that their algorithms are better than others in the state estimation problem [24]. However, only a fewer convincing algorithms were utilized to engineering application. So, it is important to explore and design more comprehensive metrics for these algorithms evaluation, which leads to an extensive literatures are by now available on performance evaluation of NEA (PE-NEA) (see, e.g. [25 –27]). As shown in Fig. 1(a), the NEA is still a hot topic in current research, but EKF, UKF, and PF are still the mainstream algorithms in solving nonlinear estimation problems. Figure 1(b) to (d) illustrate the research trends and applications of these three commonly used estimation algorithms in various fields.

Fig. 1

Number of publications of the NEA, EKF,UKFAND PFin the past 10 years.

In performance evaluation, error spectrum (ES) is a comprehensive measure because it aggregates several commonly incomprehensive metrics [28]. Therefore, ES has recently attracted lots of researches to improve it [29 –32]. Unfortunately, it is difficult to analyze the NEA’s performance based on ES, because ES is a three dimensional (3D) plot over the total time span. Accordingly, [30] proposed a dynamic error spectrum (DES) based on the average value of ES. Furthermore, it was found that DES suffers from the information loss problem. Therefore, we proposed the range error spectrum induced area (RESA) and the DES induced area (DESA) to overcome the problem of DES [27, 31]. However, it’s still hard to distinguish which estimator performs better by using the DES, RESA, and DESA. So, a volume error spectrum (VES) was proposed to further solve the dynamic systems evaluation problem [32]. Actually, the above proposed metrics (i.e., DES, RESA and DESA) need to combine all the three-dimensional plot (ES curve in dynamic systems) into a single two-dimensional plot. Clearly, it is still not easy to find the better estimator when two ES curves intersect with each other among whole time horizon.

Therefore, this paper proposes the comprehensive evaluation measures (CEM) to evaluate the existed nonlinear estimation algorithms, which provides theoretical and technical support for decision-makers of the NEAs. And, three types of the NEAs are proposed to quantify the flatness, to measure the estimation accuracy and to calculate the computation time of every NEAs. And the CEM combining the flatness, estimation accuracy and computation time of the NEA is designed to evaluate its performance.

This paper is organized as follows. Three NEAs are summarized in Section 2. In Section 3, the CEM are proposed to evaluate the above three NEAs. Finally, a numerical example is provided in Section 4 to illustrate the superiority of the comprehensive metrics of the CEM. Section 5 concludes this paper.

2 Nonlinear estimation algorithm

In this paper, the model is given by ${_{y_{k + 1} = h (x_{k + 1}) + v_{k + 1}}^{x_{k + 1} = g (x_{_{k}}) + w_{k}}$ (1) where g (·) and h (·) are nonlinear functions; w_k represents a non-Gaussian noises, v_k is N (0, R_k) (N (· , ·) is a Gaussian distribution).

As we all know, there exist several types of the NEA. For example, the EKF is a NEA, which use the the linearization of the process and observation equations [33]. Furthermore, the UKF is presented to estimate the above model by calculating the statistics of a random variable which undergoes a nonlinear transformation [13]; and the PF is based on Monte Carlo method with sequential importance sampling, which gives a numerical method for nonlinear non-Gaussian state estimation [34].

However, in practical applications, how to choose the optimal nonlinear estimation algorithm scientifically and reasonably is very important for designing reasonable estimator, to solve this problem, we will proposed the dynamic enhanced error spectrum in the following section.

3 Comprehensive evaluation measures for the PE-NEA

In the following the comprehensive evaluation measures (CEM) to the PE-NEA is proposed, which includes the flatness of the estimator, the estimation accuracy and the computation time.

3.1 Quantifying the flatness of an NEA

In practical applications, the estimated stability mainly means that the estimated errors of the estimator remain within an acceptable range within a certain period of time. To quantify the flatness of a nonlinear estimation algorithm, we previously proposed a range error spectrum induced area, i.e., [31] $A_{RES} (t) = \int_{S (r_{1})}^{S (r_{n})} f^{- 1} (S (r)) dS (r)$ (2) where, S (·) is the error spectrum (ES), which is defined as [28] $\begin{matrix} S (r) = {\begin{matrix} {\int e^{r} f (e) de}^{1 / r} \\ {(\sum p_{i} e_{i}^{r})}^{1 / r} \end{matrix} \begin{matrix} if e is continuous \\ if e is discrete \end{matrix} \end{matrix}$ (3) where r ∈ [- ∞ , + ∞], f (e) and p_i are probability density function (PDF) and probablity mass function of e, respectively. e represents the absolute or relative estimation error norm.

Using the random simulation, at time instant t, the RESA can be calculated as $A_{RES} (t) \approx \frac{S (r_{n}) - S (r_{1})}{n} \sum_{i = 1}^{n} f^{- 1} (S (r_{i}))$ (4)

Since it is difficult to calculate f^-1 (S (r_i)) analytically, r_i ∈ [-1, 2] , i = 1, 2, ⋯ , n, Approximating method is used in Equation (4) as $A_{RES} (t) \approx RES \times \frac{1}{n} \sum_{i = 1}^{n} | r_{i} |$ (5) where RES is called the range error spectrum, i.e., $RES = S (r_{n}) - S (r_{1})$ (6) where, without loss of generality, S (r_n) is the right endpoint of ES curve and S (r₁) is the left endpoint of ES curve. On this basis, RES was proposed to quantify the flatness of an ES curve [27, 31].

Particularly, for r_n = 2, r₁ = -1, we see that $RES = S (r_{n} = 2) - S (r_{1} = - 1)$ (7) where ${\begin{matrix} S (r_{n} = 2) = {(\frac{1}{M} \sum_{q = 1}^{M} {∥ {\tilde{x}}_{q} ∥}^{2})}^{\frac{1}{2}} \\ S (r_{1} = - 1) = {(\frac{1}{M} \sum_{q = 1}^{M} \frac{1}{∥ {\tilde{x}}_{q} ∥})}^{- 1} \end{matrix}$ (8) where ${\tilde{x}}_{q}$ is the estimation error, i.e., ${\tilde{x}}_{q} = x_{q} - {\hat{x}}_{q}$ and ${∥ {\tilde{x}}_{q} ∥}_{2} = ({\tilde{x}}_{q}^{'} {\tilde{x}}_{q})$ , where ${\hat{x}}_{q}$ is the estimator, x_q is the quantity to be estimated; M is the Monte Carlo runs.

In fact, S (r_n = 2) and S (r₁ = -1) are the RMSE and AEE, respectively.

3.2 Measuring the estimation accuracy of an NEA

According to Equation (3), the DESA was defined as follows [31] $A_{DES} (t) = \int_{r_{1}}^{r_{n}} S (r) dr \approx (r_{n} - r_{1}) \frac{1}{n} \sum_{i = 1}^{n} S (r_{i})$ (9) where ${r_{i}}_{i = 1}^{n}$ is the numerical value set of r.

3.3 Calculating the Computation time of an NEA

At time interval [T₁, T_m], over M Monte Carlo runs, the average computation time (ACT) of the nonlinear estimation algorithm is defined as

ACT = \frac{1}{M} \sum_{k = 1}^{M} T_{kt}

(10)

where, M is the Monte Carlo runs; t = 1, 2, ⋯ m.

3.4 Definition of the CEM

To take advantage of the RESA, the DESA and the ACT fully, the CEM was presented to combine the three metrics into a single objective function [31].

Since the RESA, the DESA and the ACT have different magnitudes and units, normalization should be carried out first, i.e., $\begin{matrix} A_{j}^{normal} = \\ \frac{A_{j} - min {A_{RES}, A_{DES}, ACT}}{max {A_{RES}, A_{DES}, ACT} - min {A_{RES}, A_{DES}, ACT}} \end{matrix}$ (11) where $A_{j}^{normal}$ and A_j , j = 1, 2, 3 are the normalized value and the value of the A_RES, A_DES, ACT, respectively.

Thus, the CEM is defined as $CEM (t) = g (A_{RES}^{normal}, A_{DES}^{normal}, AC T^{normal}),$ (12) which is the utility function to be determined.

Furthermore, we propose an additive form and a multiplicative form to the CEM.

(a) Suppose that prior preference about the weights is available, the additive form of the CEM is given as $\begin{matrix} CE S_{A} (t) = \sum_{i = 1}^{3} β_{i} f_{i} (r) \\ = β_{1} A_{RES}^{normal} + β_{2} A_{DES}^{normal} + β_{3} AC T^{normal} \end{matrix}$ (13) where β₁, β₂ and β₃, which satisfy $\sum_{i = 1}^{3} β_{i} = 1$ , are the weights associated with the A_RES, A_DES and ACT, respectively, typically determined by the users. Obviously, an estimator with a larger CEM_A is poor. (b) According to Equation (12), the multiplicative form of the CEM is defined as $\begin{matrix} CE S_{M} (t) = [\prod_{i = 1}^{3} f_{i} (r)]^{1 / 3} \\ = \sqrt[3]{A_{RES}^{normal} \times A_{DES}^{normal} \times AC T^{normal}} \end{matrix}$ (14)

Clearly, the estimator with a smaller CES_M is better. Furthermore, Equation (14) indicates that A_DES, A_RES and ACT are equally important in the PE-NEA.

As stated before, we obtain the architecture diagram of the PE-NEA as shown in Fig. 2 and summarize the above comprehensive evaluation measures in Table 1.

Fig. 2

The architecture diagram of the PE-NEA.

Table 1

Comprehensive evaluation measures

Metric	Pros and Cons
RES	$\begin{matrix} Pros : Quantifying the flatness \\ of an NEA \\ Cons : Difficult to calculate \end{matrix}$
DES	$\begin{matrix} Pros : Measuring the estimation \\ accuracy of an NEA \\ Cons : not popular, scenario sensitive \end{matrix}$
ACT	$\begin{matrix} Pros : clear physical interpretation \\ Cons : Calculating the Computation \\ time of an NEA \end{matrix}$
CEM	$\begin{matrix} Pros : Comprehensive evaluation measures \\ Cons : not popular \end{matrix}$

4 Numerical example

Hereafter, a convincing testing case about performance evaluation of nonlinear estimation algorithm is designed to illustrate the superiority of the CEM in the PE-NEA.

4.1 State equation and measurement equation

In this part, we considered the following state equation and measurement equation $\begin{array}{l} x_{k + 1} = 0.5 x_{k} + \sin (0.04 \times π \times t) + 1 + w_{k} \\ y_{k + 1} = {\begin{cases} \frac{x^{2}_{k + 1}}{5} + \sin (x_{k + 1}) + v_{k + 1} \\ \frac{x^{}_{k + 1}}{2} - 2 + v_{k - 1} \end{cases} \begin{matrix} k < 30 \\ Otherwise \end{matrix} \end{array}$ (15) where w_k follows the Gamma distribution, i.e., w_k ∼ Gamma (2, 3), and v_k follows the Gaussian distribution, i.e., v_k ∼ N (0, 0.01).

4.3 Argument Initialization of the above three NEAs

Table 2 shows the initialization parameters of the above three NEAs.

Table 2
Initialization parameters

Algorithm Initialization parameters Time

EKF $\begin{matrix} {\hat{x}}_{0} = 1, P_{0} = 3 / 4 \\ Q_{0} = 3 / 4, R_{0} = 0.01 \end{matrix}$

UKF $\begin{matrix} {\hat{x}}_{0} = 1, P_{0} = 3 / 4 \\ Q_{0} = 3 / 4, R_{0} = 0.01, \\ α = 1, β = 0, λ = 0 \end{matrix}$ $\begin{matrix} [T_{0}, T_{m}] \\ = [0, 50 s] \end{matrix}$

PF $\begin{matrix} N = 1000, x_{0}^{(i)} = 1 \\ w_{0}^{(i)} = 1, R_{0} = 0.01 \end{matrix}$

4.3 Analysis of the simulation results based on three NEAs

Let [T₀, T_m] = [0, 50s], after M = 500 Monte-Carlo runs, we obtained the tracking results of the above three NEAs shown in Fig. 3, which shows that the measurement y matches the state x within the time interval [0, 5s] and [20, 30s], and mismatches the state x within the time interval [6, 19s] and [31, 50s].

Fig. 3

The estimated results of the above three NEAs.

To illustrate the superiority of the CEM, we used the average of the estimation value among the above 500 Monte-Carlo runs at time instant t, i.e., ${\begin{matrix} {\bar{x}}_{t}^{EKF} = \sum_{k = 1}^{M} {\hat{x}}_{kt}^{EKF} / M \\ {\bar{x}}_{t}^{UKF} = \sum_{k = 1}^{M} {\hat{x}}_{kt}^{UKF} / M \\ {\bar{x}}_{t}^{PF} = \sum_{k = 1}^{M} {\hat{x}}_{kt}^{PF} / M \end{matrix}$ (16)

By the way, the absolute errors were given as ${\begin{matrix} e_{kt}^{EKF} = | | {\hat{x}}_{kt}^{EKF} - x_{t} | |_{1} \\ e_{kt}^{UKF} = | | {\hat{x}}_{kt}^{UKF} - x_{t} | |_{1} \\ e_{kt}^{PF} = | | {\hat{x}}_{kt}^{PF} - x_{t} | |_{1} \end{matrix}$ (17)

Firstly, we used the RMSE, A_DES (t), and A_RES (t) to evaluate the above three NEAs, and their evaluated results were shown in the Figs. 4–6, respectively.

Fig. 4

Evaluated results of the RMSE curves.

Fig. 5

Evaluated results of The A_RES (t) curves.

Fig. 6

Evaluated results of the A_DES (t) curves.

From Fig. 4, we see that RMSE^PF (t) is lowest than RMSE^UKF (t) curve, and RMSE^EKF (t) curve is in the middle of them within the time intervals [0s, 31s], so, we have ${RMSE}^{PF} (t) < {RMSE}^{UKF} (t) < {RMSE}^{EKF} (t)$ (18)

Likewise, Fig. 6 shows that $A_{DES}^{PF} (t)$ curve is still lowest than $A_{DES}^{UKF} (t)$ , and $A_{DES}^{UKF} (t)$ curve is in the middle of them within the time intervals[0s, 50s]. Thus, the PF is still the best NEA if only considering the estimation accuracy.

Clearly, for the estimation accuracy, the evaluation results using the A_DES (t) is the same as the RMSE. However, in practical applications, we may consider not only the accuracy of the estimator, but also the flatness or the estimation speed of the estimator. The evaluation results obtained by this way can provide better a scientific basis for decision-making.

Furthermore, as depicted in Fig. 5, we see that $A_{RES}^{PF} (t)$ is lowest than $A_{RES}^{UKF} (t)$ curve, and $A_{RES}^{EKF} (t)$ curve is in the middle of them within the time intervals, i.e., [0s, 31s], so, we obtained $A_{RES}^{PF} (t) < A_{RES}^{UKF} (t) < A_{RES}^{EKF} (t)$ (19)

Similarly, we can see from the time interval [32s, 50s] that $A_{RES}^{PF} (t) \approx A_{RES}^{UKF} (t) \approx A_{RES}^{EKF} (t)$ (20)

So, compared with these three NEAs, the estimation results of the PF is the most stable, which illustrates that the PF is the best NEAs only considering the flatness of the estimator.

Secondly, at time interval [0, T], each estimation time of the EKF, UKF, and PF were shown in Fig. 7.

Fig. 7

Evaluated results of the the ACT curves.

So, according to the Equation (10), we obtained the ACT, i.e, ${\begin{matrix} AC T^{EKF} = \sum_{k = 1}^{M} t_{k}^{EKF} / M = 0.0143 \\ AC T^{UKF} = \sum_{k = 1}^{M} t_{k}^{UKF} / M = 0.0422 \\ AC T^{PF} = \sum_{k = 1}^{M} t_{k}^{PF} / M = 3.9470 \end{matrix}$ (21)

That is, $AC T^{EKF} < AC T^{UKF} < AC T^{PF}$ (22)

Clearly, the EKF has the shortest computation time; the UKF is next; and the PF is longest computation time among the above three NEAs.

Furthermore, according to Equations (19) and (22), we can see that different metrics lead to different evaluation results. Therefore, to consider the flatness of the estimator synthetically, the estimation accuracy and the computation time in PE-NEA, the CEM_A and the CEM_M were proposed to evaluate the above three NEAs, as shown in the Figs. 8, 9, respectively.

Fig. 8

Evaluated results of the the CEM_A curves.

Fig. 9

Evaluated results of the the CEM_M curves.

Similarly, the CEM_A curves of the above three NEAs have the same results in case of considering only the flatness of the estimator or the estimation accuracy. From Fig. 8, the PF is the best algorithm among the above three metrics because the CEM_A curves is the lowest curve among the above all curves. Actually, the ACT is too small to affect the value of the CEM_A since it is dominated by large terms in the additive form. Fortunately, the above problem can be solve by the CEM_A.

From Fig. 9, in the time interval [1, 9s] and [18, 30], we obtained the following results from the CEM_M curves, i.e., ${CEM}_{M}^{UKF} (t) < {CEM}_{M}^{EKF} (t)$ (23)

However, in the time intervals [10, 17], [31, 50s], we obtained ${CEM}_{M}^{EKF} (t) < {CEM}_{M}^{UKF} (t)$ (24)

Clearly, The interval length t_a = [10, 17s] + [31, 50s] =28s is greater than the interval length t_a = [1, 9s] + [18, 30s] =22s, that is, the EKF wins the UKF too much. So, the EKF is better than the UKF, i.e., ${CEM}_{M}^{EKF} (t) < {CEM}_{M}^{UKF} (t)$ (25)

Similarly, we got ${\begin{matrix} {CEM}_{M}^{PF} < {CEM}_{M}^{UKF} \\ {CEM}_{M}^{PF} > {CEM}_{M}^{UKF} \end{matrix} \begin{matrix} [3, 8 s], [18, 26 s] \\ [1, 2 s], [9, 17 s], [27, 50 s] \end{matrix}$ (26)

}So, we obtained ${CEM}_{M}^{UKF} (t) < {CEM}_{M}^{PF} (t)$ (27)

According to Equations (25) and (27), we got ${CEM}_{M}^{EKF} (t) < {CEM}_{M}^{UKF} (t) < {CEM}_{M}^{PF} (t)$ (28)

In other words, we obtained $EKF ≻ UKF ≻ PF$ (29)

where EKF ≻ UKF means that EKF outperforms UKF.As discussed above, if only considering the flatness of the estimator, we have $PF ≻ UKF ≻ EKF$ (30)

Furthermore, if only considering the estimation accuracy, we obtained $PF ≻ UKF ≻ EKF$ (31)

However, when only considering the computation time, we get $EKF ≻ UKF ≻ PF$ (32)

Finally, if comprehensively considering the flatness, the estimation accuracy and the computation time, we get $EKF ≻ UKF ≻ PF$ (33) In addition, the results in Equation (33) are obtained when based on the equal weights of the flatness, the estimation accuracy and the computation time.Finally, we can see from the performance evaluation results that: (1) In terms of one performance index of calculation time, the EKF and the UKF have obvious advantages; (2) The PF has obvious advantages based on two performance indices of calculation accuracy and estimation stability; (3) the EKF is the best among the above three NEAs based on the three performance indices of the flatness, the estimation accuracy and the computation time.

5 Conclusion

The Primary contribution of this paper is that the comprehensive evaluation measure was proposed to evaluate the existing nonlinear estimation algorithm, which can comprehensively reflect the flatness, estimation accuracy and computation time of the existing nonlinear estimation algorithm. (1) The additive form of the CEM is good for not only can integrate the stability, accuracy, and computational time of translation estimation algorithms, but also can reflect the decision-maker’s intentions by setting different weights. (2) the multiplicative form of the CEM can balance the stability, accuracy, and computational time of estimation algorithms, and is not affected by extreme values. In future work, we will apply the CEM to the other NEAs.

Footnotes

Acknowledgments

We would like to thank anonymous reviewers for their helpful comments and suggestions. This work was supported in part by the national natural science foundation of China through grant No. 72271243, No. 71801222 and No. 61973253, the national science foundation of shaanxi province of China through grant No. 2018JQ6019, the national postdoctoral program for innovative talents through grant No. BX201700104.

References

Sun

, Yi

, Varshney

P.K.

, et al., Resource scheduling for multi-target tracking in multi-radar systems with imperfect detection, IEEE Transactions on Signal Processing 70 (2022), 3878–3893.

Jiang

, Cui

, Cheng

, et al., A ball-shaped target development and pose estimation strategy for a tracking-based scanning system, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–12.

Gong

, Zhang

, Sang

, et al., In-flight performance assessment of the single-frequency GPS/BDS receiver for Yaogan-30 series satellites real-time navigation, IEEE Transactions on Aerospace and Electronic Systems 58(5) (2022), 4495–4505.

Yang

, Shu

, Li

, et al., SILOS: An intelligent fault detection scheme for solar insecticidal lamp IoT with improved energy efficiency, IEEE Internet of Things Journal 10(1) (2023), 920–939.

Zhao

, Xiang

and Lv

, A variable parameter lms algorithmbased on generalized maximum correntropy criterion for graph signalprocessing, IEEE Transactions on Signal and InformationProcessing over Networks 9 (2023), 140–151.

Liu

Z.E.

, Zhou

, Li

, et al., Safe deep reinforcement learning-based constrained optimal control scheme for HEV energy management, IEEE Transactions on Transportation Electrification 1 (2023), 1–1.

Kalman

R.E.

, A new approach to linear filtering and predictionproblems, Journal of Basic Engineering 82 (1960), 35–45.

Al-Masri

, Wadi

, Abdel-Hafez

M.F.

, et al., Partial discharge localization in power transformers using invariant extended kalman filter, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–10.

Deng

, Wang

and Wang

, An improved unscented kalman filter forinterrupted and drift sensor faults of aircrafts, IEEETransactions on Instrumentation and Measurement 72 (2023), 1–10.

10.

Shi

L.F.

, Feng

B.L.

, Dai

Y.F.

, et al., Pedestrian indoor localization method based on integrated particle filter, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–10.

11.

Song

, Wang

, Li

, et al., Protocol-based particle filtering for nonlinear complex networks: handling non-gaussian noises and measurement censoring, IEEE Transactions on Network Science and Engineering 10(1) (2023), 128–139.

12.

Lan

and Li

X.R.

, Multiple conversions of measurements fornonlinear estimation, IEEE Transactions on Signal Processing 65 (2017), 4956–4970.

13.

Xue

Z.W.

, Zhang

, Cheng

, et al., Remaining useful life prediction of lithium-ion batteries with adaptive unscented Kalman filter and optimized support vector regression, Neurocomputing 376 (2020), 95–102.

14.

Zhou

B.H.

and Liao

X.M.

, Particle filter and levy flight-baseddecomposed multi-objective evolution hybridized particle swarm forflexible job shop greening scheduling with crane transportation, Applied Soft Computing Journal 91 (2020), 106217.

15.

, Liang

and Xu

, Hybrid sampling-based particle filteringwith temporal constraints, IEEE Transactions on Systems, Man,and Cybernetics: Systems 53(2) (2023), 1104–1115.

16.

Kundar

, Shreya

and Shovan

, A new method for nonlinear stateestimation problem, Digital Signal Processing 132(2023), 10378.

17.

Cao

, Yang

, Ma

, et al., Cooperative positioning for sparsely distributed high-mobility wireless networks with EKF based spatio-temporal data fusion, IEEE Communications Letters (2023).

18.

Rong

, Peng

, Chen

, et al., An EKF-based attitude estimator for eliminating the effect of magnetometer measurements on pitch and roll angles, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–10.

19.

Zhang

, et al., , An improved UKF algorithm for extracting weak signals based on RBF neural network, IEEE Transactions on Instrumentation and Measurement 71 (2022), 1–14.

20.

Bucci

, Franchi

, Ridolfi

, et al., Evaluation of UKF-based fusion strategies for autonomous underwater vehicles multisensor navigation, IEEE Journal of Oceanic Engineering 48(1) (2023), 1–26.

21.

Liu

, Chen

and Wang

, Strong tracking UKF-based hybridalgorithm and its application to initial alignment of rotating SINSwith large misalignment angles, IEEE Transactions on IndustrialElectronics 70(8) (2023), 8334–8343.

22.

Xue

, Yang

, et al., An improved generic hybrid prognostic method for RUL prediction based on PF-LSTM learning, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–21.

23.

, Li

, Wang

, et al., An inertial magneto-inductive positioning system based on GWO-PF algorithm, IEEE Transactions on Instrumentation and Measurement 71 (2022), 1–10.

24.

Garcia

R.V.

, Pardal

P.C.P.M.

, Kuga

H.K.

, et al., Nonlinear filtering for sequential spacecraft attitude estimation with real data: Cubature Kalman Filter, Unscented Kalman Filter and Extended Kalman Filter., Advances in Space Research 63 (2019), 1038–1050.

25.

Chou

, Boers

, Podt

, Geist

, Performance evaluation for particle filters, In Proceedings of 2011 International Conference Information Fusion, Chicago, IL, USA (2011), 1–7.

26.

Peng

W.S.

, Li

Y.H.

, Fang

Y.W.

, et al., Radar chart for estimation performance evaluation, IEEE Access 7 (2019), 113880–113888.

27.

Peng

W.S.

, Fang

Y.W.

and Chai

, Enhanced dynamic error spectrum for estimation performance evaluation in target tracking, Optick 127 (2016), 3943–3949.

28.

X.R.

, Zhao

Z.L.

, Duan

Z.S.

, Error spectrum and desirability level for estimation performance evaluation, In Proceedings of Workshop on Estimation, Tracking and Fusion: A Tribute to Fred Daum, Monterey, CA, USA (2007), 1–7.

29.

Liu

, Li

X.R.

, Computation of error spectrum for estimation performance evaluation, In Proceedings of 2013 International Conference Information Fusion, Istanbul, Turkey (2013), 477–483.

30.

Peng

W.S.

and Fang

Y.W.

, New dynamic error for estimationperformance evaluation, Journal of Intelligent & FuzzySystems 40(1) (2023), 775–782.

31.

Peng

W.S.

, Fang

Y.W.

, Duan

Z.S.

and Wang

H.B.

, Enhanced error spectrum for estimation performance evaluation, Optik 127 (2016), 5084–5091.

32.

, Peng

W.S.

and Liu

, Volume error spectrum for estimation performance evaluation in dynamic systems, Optik 144(2017), 655–663.

33.

Liu

, Li

, Wang

, et al., Data-driven state of charge estimation for power battery with improved extended kalman filter, IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–10.

34.

Song

, Wang

, et al., Distributed auxiliary particle filtering with diffusion strategy for target tracking: A dynamic event-triggered approach, IEEE Transactions on Signal Processing 69 (2021), 328–340.

Algorithm	Initialization parameters	Time
EKF	$\begin{matrix} {\hat{x}}_{0} = 1, P_{0} = 3 / 4 \\ Q_{0} = 3 / 4, R_{0} = 0.01 \end{matrix}$
UKF	$\begin{matrix} {\hat{x}}_{0} = 1, P_{0} = 3 / 4 \\ Q_{0} = 3 / 4, R_{0} = 0.01, \\ α = 1, β = 0, λ = 0 \end{matrix}$	$\begin{matrix} [T_{0}, T_{m}] \\ = [0, 50 s] \end{matrix}$
PF	$\begin{matrix} N = 1000, x_{0}^{(i)} = 1 \\ w_{0}^{(i)} = 1, R_{0} = 0.01 \end{matrix}$

Comprehensive evaluation measures of nonlinear estimation algorithm performance

Abstract

Keywords

1 Introduction

3.1 Quantifying the flatness of an NEA

4.1 State equation and measurement equation

Table 2 Initialization parameters Algorithm Initialization parameters Time EKF x ˆ 0 = 1 , P 0 = 3 / 4 Q 0 = 3 / 4 , R 0 = 0.01 UKF x ˆ 0 = 1 , P 0 = 3 / 4 Q 0 = 3 / 4 , R 0 = 0.01 , α = 1 , β = 0 , λ = 0 [ T 0 , T m ] = [ 0 , 50 s ] PF N = 1000 , x 0 ( i ) = 1 w 0 ( i ) = 1 , R 0 = 0.01

Footnotes

Acknowledgments

References