Abstract
The traditional Interacting Multiple Model (IMM) filters usually consider that the Transition Probability Matrix (TPM) is known, however, when the IMM is associated with time-varying or inaccurate transition probabilities the estimation of system states may not be predicted adequately. The main methodological contribution of this paper is an approach based on the IMM filter and retention models to determine the TPM adaptively and automatically with relatively low computational cost and no need for complex operations or storing the measurement history. The proposed method is compared to the traditional IMM filter, IMM with Bayesian Network (BNs) and a state-of-the-art Adaptive TPM-based parallel IMM (ATPM-PIMM) algorithm. The experiments were carried out in an artificial numerical example as well as in two real-world health monitoring applications: the PRONOSTIA platform and the Li-ion batteries data set provided by NASA. The Retention Interacting Multiple Model (R-IMM) results indicate that a better prediction performance can be obtained when the TPM is not properly adjusted or not precisely known.
Introduction
Estimating the states of a dynamic system is a fundamental task to ensure certain performance levels in practical industrial applications. However, obtaining some of these measurements may not be possible, which explains the success of Bayesian filtering for real-world problems [1]. In the literature, several estimation problems based on Bayesian filtering assume that one model describes the system dynamics [2–5], but the system state evolution may implicate the changing of modes, and a single model may not describe the system behavior adequately. In this context, Markov Jump System (MJSs) and Fuzzy Logic System (FLSs) play a substantial role in tasks related to condition-based maintenance, such as fault detection and diagnosis [6, 7], fault prognostics [8, 9], Fault Tolerant Control (FTC) [10–15], and event-triggered control [16, 17]. A slide mode observer is used in [10] for FTC of mobile manipulator systems described by time-delay Markov Jump System (MJS). In [11], nonlinear simultaneous additive and multiplicative actuator faults are dealt with through MJS and fuzzy logic systems. The fuzzy adaptive control of an electromechanical system with non-affine nonlinear faults is addressed in [12]. [13] presents fuzzy reconfiguration blocks to hide actuator and sensor fault occurrences from the nominal controller, enabling FTC in nonlinear systems. The design of virtual sensors and virtual actuators is addressed in [14], where a robust FTC methodology is employed for linear parameter varying systems. [8] uses a specific type of MJS, namely Interacting Multiple Model (IMM), to predict accelerated rolling bearings’ degradation state, enabling fault prognostics. In [9], a novel incremental Fuzzy Logic System (FLS) is provided for fault prognostics in accelerated rolling bearings. In [6], an IMM approach based on a nonlinear vehicle dynamics observer is employed to detect and isolate faults in driven-by-wire vehicles. [7] uses parallel particle filter estimations combined through IMM to estimate the states of highly nonlinear dynamic processes, allowing fault detection and diagnosis. The IMM filters are methods for adaptive estimation of states that characterize the behavior of dynamic systems with multiple mode operation [8, 19].
In the IMM filters, the system jumps are modeled by a Markov chain which is reflected by a Transition Probability Matrix (TPM) and it is assumed to be known a priori. However, when this approach is associated with a time-varying or inaccurate transition probabilities it can lead to the misleading state estimation. In order to improve this drawback, different kinds of approaches have been proposed in the literature. In [20], an approximate likelihood function of the TPM is used to determine the TPM according to the maximum likelihood criterion. In [21], the authors present a meta model filter which the transitions are obtained using a Bayesian Network (BN). In [22], a fuzzy decision maker is proposed to choose the TPM from a finite candidate set. In [23], expectation maximization procedure is proposed to estimate the TPM and its performance is shown to be better when compared with the Kullback-Leibler divergence method presented in [24]. Unfortunately, these methods have some disadvantages, as, for example, to be computationally very expensive [21, 23], other ones rely on optimization algorithms [23] or consider that the TPM is obtained from a finite set [22]. Two methodologies to estimate the TPM are presented in [25]; the first technique estimates a continuous Markov chain using missing data and the second uses the full data set to capture the non-Markovian effect of rating momentum. [26] shows four algorithms with minimum mean-square error to recursively estimate the TPM. The first method is used when the TPM is linear in the transition probabilities, but its computational load increases with the data length since the algorithm requires a knowledge of all moments of the Transition Probability Matrix (TPMs) prior probability density function. The second method considers that the third central moments are zero to create a second-order approximation for the TPM, which can yield numerical problems in estimating the TPM. The third method assumes a Dirichlet distribution for the prior probabilities of a finite mixture used to estimate the TPM. At the same time, the fourth approach straightforwardly integrates over a finite grid of the TPM posterior probability density function, increasing its computational burden. [27] provides an algorithm to adapt the TPM in maneuvering target tracking tasks; the computation relies on the speed of the target and the radius of the turn which restricts the application to systems with such sensors. Moreover, TPM uncertainty is considered in [28], where a polytopic set with multiple Transition Probability Matrix (TPMs) is used in the design of robust model predictive control. Less conservative conditions were obtained for state-feedback control, increasing the maximal feasible region for the systems’ states while ensuring mean square stability and satisfaction of hard constraints on system inputs and states. Time-varying Transition Probability Matrix (TPMs) are considered in [29] with values estimated at each time step by solving a linear programming optimization problem. The method imposes that the observed multivariate time series needs to be non-negative and that its normalized values are realizations of the stationary distribution of the modeled Markov process. For the parallel IMM algorithm, [30] proposed a technique to correct the TPM in a recursive way using the gradient of the model probability. This correction is improved by doubling the number of filters to bring in the information of the current model upon model jumps. However, in order to identify these jumps, a constant threshold is introduced; this threshold must be defined according to the problem.
To overcome the problem of unknown Transition Probability Matrix (TPMs) in nonlinear time-varying systems and to reduce the conservativeness regarding previous assumptions and required pre-defined thresholds, this paper proposes an approach based on the concept of retention model presented by [31] which tests hypothesis between constancy and free variation of transition probabilities applied to a panel study of voting intentions prior to the 1940 election in the US. The relatively low cost is very attractive to the estimation of the transition probabilities in the context of the IMM filter for real-world online applications. This work differs from [31] by introducing the adaptive feature and by ensuring that less parameters need to be estimated. The main contributions of this paper are summarized as follows: the unknown transition probability is determined adaptively and automatically at each instant k; a method with relatively low computational cost is presented which has no need for complex operations or storing the measurement history; a new approach is proposed based on IMM and the retention model concept to allow the TPM be time-varying.
The remainder of this paper is organized as follows. Section 2 presents the problem formulation that will be addressed in this paper and a brief overview of the IMM theory. Section 3 provides details on the proposed approach that applies the retention model concept to estimate the TPM used in IMM recursively. Section 4 shows three experiments to test the proposed approach against the traditional IMM, a method that uses Bayesian Network (BNs) with IMM [21] and a state-of-the-art Adaptive TPM-based parallel IMM (ATPM-PIMM) proposed in [30]. The first experiment is a toy example with two different switching transition models where the influence of inaccurate Transition Probability Matrix (TPMs) is studied, showing the robustness of the proposed approach. The second experiment consists of a real-world experiment of monitoring real rolling bearings’ health in the PRONOSTIA platform [32]. The third experiment concerns the degradation of four Li-ion batteries provided by a testbed in the NASA Ames Prognostics Center of Excellence (PCoE) [33]. Finally, Section 5 draws the final remarks.
Problem formulation
Consider the following discrete-time MJS:
where
According to [18], the IMM output uses the law of total probability defined by:
Unfortunately, the IMM filter may not obtain good estimates in cases when the available TPM is not accurate. For instance, consider a state space system with inaccurate transition probability in which the state vector is estimated by model 1 up to instant k = 150 and by model 2 after that. Fig. 1 depicts a two model example with true and predicted states by the IMM algorithm. The figure also shows the model probabilities. In this case, model 1 is expected to have higher probability values (y-axis) than model 2 before k = 150 and, after this instant, model 2 becomes dominant. However, as the TPM is not accurate, Fig. 1 depicts that the dominant model becomes model 2 before the expected instant (k = 150). After k = 150 the model 1 is dominant, and that is the reason IMM does not predict the true system states.

State estimation and model probabilities obtained by the IMM method with wrong TPM matrix.
This uncertainty can be explained by observing the TPM used to estimate these states:
In this case, the matrix shows that the probability of remaining in the same state is less than changing to a new state, which does not correspond to the true matrix, as can be seen:
To overcome the aforementioned problem, a new approach is proposed based on IMM and the retention model concept [31]. This new approach is called Retention Interacting Multiple Model (R-IMM) and it will include Particle Filters since they are very suitable to deal with nonlinear and non-Gaussian scenarios [34–37].
The traditional IMM proposed by [38] has the following steps:
The model mixing and state interacting depend on the predicted model probability from the previous estimates and the TPM R. The model transition is defined by a Markov process and the switch probability p (θ
k
= i|θk-1 = j) = R
ij
denotes the transition probability from model i to model j. However, this matrix is assumed to be known a priori. The Chapman-Kolmogorov equation for the model mixing is:
This paper uses the concept of retention model proposed in [31] which tests hypothesis between constancy and free variation of transition probabilities. This approach can specify the way in which the transition matrices vary in time. Here, we apply the retention model in the context of the IMM filter, with the difference that the TPM is unknown a priori and it is calculated along the IMM procedure. This methodology also ensures that less parameters need to be estimated each time instant. Let {θ
k
, k = 0, 1, …, T} be an N state Markov chain and its transition probabilities are defined by:
According to [31], a Markov chain can be the combination of two processes: the first process determines when a transition between two distinct states will occur; and the second one determines the outcome of the move. Following this approach with two matrices D
k
and M
k
, the decomposition of the transition matrix is defined as follows:
In [31], the matrices M and D are calculated from all available transitions from instant 1 up to T, thus if M k = M, D k is still given by (7), but m ij = n ij /(n i - n ii ) , ThickSpace i ≠ j, where n ij and n i are calculated from all available transitions. Lastly, when D k = D, and M k changes, M k is (8) and d i = n ii /n i .
As the IMM method is calculated recursively and the transition matrix is considered not known, this work proposes a few modifications so that R is also calculated recursively. Consider that matrices D0 and M0, at instant k = 0, are provided by the user, the active model (θ
k
) at instant k is the one with the highest model probability calculated by IMM and Dk-1 and Mk-1 are matrices calculated at instant k - 1. Thus, if M
k
= Mk-1, D
k
is still given by (7), and
Find matching models μk-1 and μ
k
Increment nμk-1,μ
k
,k
Compute D
k
using (7) and Dk-1 using (10) Compute M
k
using (8) and Mk-1 using (9) Run the χ
2 tests in (13) to update the TPM: If Else if Else if Else: choose hypothesis H3
where η = χ
2 (0.95, ν) is the chi-square inverse cumulative distribution function at 95% confidence level and ν degrees of freedom, which it is defined by the number of independent TPM parameters.
In this section, we consider a numerical experiment and two real-world monitoring applications to compare the performance of the proposed algorithm (R-IMM) with the following methods: the IMM algorithm, the IMM with BN (BN-IMM) [21], and the ATPM-PIMM algorithm [30]. According to [30], the threshold for ATPM-PIMM can be set to 0.8 so the algorithm can achieve a good performance on both noise smoothing and response speed to the model jump. The Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE) are employed to measure the performance of the filters. The simulations use thirty independent experiments, 50 particles and the systematic resampling method [34].
Illustrative experiment
Consider the state space models:
where T is the maximum simulation time, x
k
is the system state, y
k
is the measurement,
The TPM R is usually assumed to be known a priori, but, in real-world problems, it may be unknown. To overcome this problem, the transition probability is calculated recursively by R-IMM. Some scenarios are presented in this paper to illustrate that the proposed R-IMM is more robust than the IMM method especially in the situations in which there is uncertainty about matrix R. In this case, the matrices R
s
1
, R
s
2
, R
s
3
and R
s
4
will be used into the traditional IMM. To study the influence of inaccurate TPM, some scenarios are defined:
Table 1 presents the state estimation results for the proposed scenarios considering different methods. The traditional IMM results are worse as the uncertainty increases in the TPM. The BN-IMM results remain stable because they do not depend on the matrix R; however, it calculates the matrix recursively based on all previous measurements, which is computationally costly. As this method depends on the previous data, it takes time to recognize the change of the active model since there is still no information about the model 2. The proposed method R-IMM does not use all previous measurements and yet it can correctly estimate the transition probability and the states. This fact suggests that the R-IMM with unknown matrix R is able to estimate the states close to true value, as can be noticed from the values highlighted in bold in Table 1.
RMSE and MAPE metrics’ mean and standard deviation for 30 independent experiments of state estimation in the illustrative experiment 4.1, with best values in bold
It is also worth noting that scenario 4 provides the true TPM for the IMM, so no uncertainty related to the transition matrix is provided to the traditional method. Yet, the proposed R-IMM can correctly estimate the states, even if its initial TPM is completely different, as can be seen by Rij0.
For illustration purposes, Fig. 2 depicts the state predicted by the IMM, BN-IMM and the proposed R-IMM for scenario 3, which is the scenario with greater uncertainty in the TPM defined a priori.

State estimation of IMM, BN-IMM, ATPM-PIMM, and R-IMM, respectively, for scenario 1. Experiment 4.1.
The database PRONOSTIA [32] is focused on the estimation of the remaining life of bearings under operating conditions. The extraction and selection of bearings features is based on vibration sensor and here it is used the same procedure and the health indicator proposed in [39]. Three different loads data were considered in the dataset. According to [32], theoretical estimated life durations (L10, BPFI, BPFE, etc.) do not provide the same estimations than those reached by the experimental observations. In this case, two models are extracted for each operating condition using a fuzzy inference system, in the same way as it was proposed in [8]. The proposed R-IMM procedures applied to PRONOSTIA database is divided into two phases. In the first, the models are extracted in the offline training step but, in the second phase, the R-IMM only uses the fuzzy models as system models in the filtering step. In the training, Takagi-Sugeno fuzzy models are used for which the input variables xk-1 are assigned the N membership functions. The fuzzy IF-THEN rules generated are presented below:
where xk-1 is the predicted state at instant k - 1, A
j
, j = 1, …, N is the fuzzy set,
Parameters of the fuzzy systems for Experiment 4.2
The filter simulation parameters are σ
ω
= 0.0215 and σ
η
= 0.2, for process and measurements noises, respectively. The inaccurate TPM R is given as:
The initial probability transition matrix for all methods is given as an identity matrix. Table 3 summarizes the analysis results of the three operating conditions, which helps to understand the merits of the proposed approach in terms of RMSE and MAPE metrics. For each bearing, 30 independent experiments were done to allow statistical comparison between the methods; the results are depicted as the average value with the standard deviation for each group of 30 runs and the best results are highlighted in bold. For the PRONOSTIA database, the conventional IMM has great variability of performance and they are significantly worse than the R-IMM in some cases. This diversity of results also shows that IMM is very sensitive to TPM uncertainty. For the sake of simplicity, all operating conditions are dealt with the same parameters. It is important to note that the results in Table 3 shows a consistent improvement of the accuracy when using the proposed approach. The mean value of both metrics is also compared two by two in Table 4 using the two-sample Student’s t-test. All p-values at a 99% confidence level are significantly lower than 0.01, showing strong evidence to reject the null hypothesis. This indicates that the proposed method’s performance is statistically different than the other approaches’ while it is consistently better in terms of RMSE and MAPE.
RMSE and MAPE metrics’ mean and standard deviation for 30 independent experiments of degradation state estimation in the PRONOSTIA dataset, with best values in bold –Experiment 4.2
p-values for the two-sample Student’s t-test comparing the average RMSE two by two between the proposed R-IMM and IMM, BN-IMM, and ATPM-PIMM at a 99% confidence level. Experiment 4.2
The proposed R-IMM presented better performance for all cases without knowing the true TPM and without storing all the measurements. The BN-IMM method did not achieve good results especially in cases in which there were transitions of the states. For example, there was no transition in bearing 2.7, thus the BN-IMM results were the same as the R-IMM.
In the same way as in the previous numerical example, assume that the true TPM, called R, is the transition rate R
ij
= c
ij
/∑
j
c
ij
and it is obtained by the training data. This matrix is defined as
Table 5 shows the comparative results between the IMM with the true TPM (18) and the R-IMM with a different TPM. The results of 30 independent experiments are shown in terms of average and standard deviation for each metric. The p-value of a two-sample Student’s t-test for the mean is also shown to provide a statistical comparison between the results at a 99% confidence levels, where the * symbol in the bearing column denotes p-values greater than 0.01, which indicate a failure in rejecting the null hypothesis. As can be seen in the Table 5, the performance of the proposed method is practically equivalent to the IMM that uses the true transition matrix, which shows the ability of R-IMM to replace the need for prior knowledge of TPM.
Comparative RMSE and MAPE metrics between the IMM using the true TPM and R-IMM from 30 independent experiments with the two-sample Student’s t-test for the mean. The best values are in bold and * denotes results with no statistical difference –Experiment 4.2
The main difference between the results in Tables 3 and 5 is that, in the first, the same TPM (given in (17)) is used for all filters to illustrate their sensitivity to uncertain estimations of the real TPM. The latter shows the robustness of the proposed method; the results are similar to the IMM with the real TPM, even when the initial TPM in R-IMM is wrong.
For illustration purposes, Fig. 3 depicts the state estimation of one simulation of bearing 3-3 obtained by the IMM filter with accurate TPM and the proposed R-IMM filter.

State estimation of IMM and R-IMM, respectively, for bearing 3-3. Experiment 4.2.
Considering the case study, when comparing filter estimates, the proposed R-IMM approach can estimate the states in all testing sets. It can also be observed that the models extracted by the fuzzy inference system were able to model adequately the dynamics of the state derived from the data.
The case study reported in this section concerns the degradation of Li-ion batteries. This type of battery is found in industry and commercially, e.g., in electric vehicles, microgrids and electronic devices [33, 40]. The cycle aging datasets of four Li-ion batteries are provided by a testbed in the NASA Ames Prognostics Center of Excellence (PCoE). The testbed comprises commercial Li-ion 18650-sized rechargeable batteries from the Idaho National Laboratory; a programmable 4-channel DC electronic load and power supply; voltmeters, ammeters, and a thermocouple sensor suite; custom electrochemical impedance spectrometry equipment; and environmental chamber to impose different operational conditions. The batteries run at room temperature (23° C). Charging is done in constant mode at 1.5 A, until the voltage reaches 4.2 V. Discharging is performed at a constant current level of 2 A, until the battery voltage reaches 2.7 V [33].
In the same way as in the previous Experiment 4.2, two models are extracted using a fuzzy inference system as proposed in [8]. These parameters (p,q) are given in Table 6.
Parameters of the fuzzy systems for Experiment 4.3
Parameters of the fuzzy systems for Experiment 4.3
The filter simulation parameters are σ ω = 0.0018 and σ η = 0.002, for process and measurements noises, respectively. The initial inaccurate matrix of transition probabilities R for R-IMM is given as an identity matrix. Table 7 summarizes the analysis results of the batteries datasets, which reinforces the merits of the proposed approach in terms of RMSE and MAPE metrics. For each dataset, 30 independent experiments were done to allow statistical comparison between the methods; the results are depicted as the average value with the standard deviation for each group of 30 runs and the best results are highlighted in bold.
RMSE and MAPE metrics’ mean and standard deviation for 30 independent experiments of state estimation in the Li-on batteries dataset - Experiment 4.3, with best values in bold
The mean value of both metrics is also compared two by two in Table 8 using the two-sample Student’s t-test. All p-values at a 99% confidence level are significantly lower than 0.01, showing strong evidence to reject the null hypothesis. This indicates that the proposed method’s performance is statistically different than the other approaches’ performance while it is consistently better in terms of RMSE and MAPE.
p-values for the two-sample Student’s t-test comparing the average RMSE two by two between the proposed R-IMM and IMM, BN-IMM, and ATPM-PIMM at a 99% confidence level for Experiment 4.3
Likewise, as in the previous experiment, when comparing filter estimates, the proposed R-IMM approach can estimate the states in all testing datasets. It is also worth mentioning that the R-IMM using a fuzzy inference system presented better performance without knowing the true TPM and without knowing the system models in advance.
For illustration purposes, Fig. 4 depicts the state estimation of one simulation of dataset B0007 obtained by the IMM, BN-IMM, R-IMM and ATPM-PIMM filters.

State estimation of IMM, BN-IMM, R-IMM and ATPM-PIMM, respectively, for dataset B0007. Experiment 4.3.
The combination of the estimation using multiple parallel filters is sensitive to the TPM accuracy. This work proposes a new procedure for cases in which the TPM is not precisely known. This new approach, called R-IMM, uses IMM filters and the retention model concept to calculate recursively and online the TPM with relatively low computational cost, no need for complex operations, or storing the measurement history. The comparative results for a numerical example and the real-world application PRONOSTIA indicate that the proposed R-IMM improves standard IMM performance in the presence of uncertainty in the transition matrix.
The main limitations of the proposed method arise from the use of Chi-square statistics to decide how to update the TPM. A strong limitation is regarding the sample size; according to [41], the value of the cell in the frequency table should be 5 or more in at least 80% of the cells, and no cell should have an expected of less than one. Therefore, in the first samples of the process, the TPM updates might be affected by type I or type II errors in the Chi-square tests. A future research direction is investigating the use of the maximum likelihood ratio Chi-square test in the first samples, which have been indicated as a possible solution to this problem [42].
Footnotes
Acknowledgments
This work was supported by the Brazilian agencies CNPq (Grant numbers 309909/2019-8, 312289/2017-0 and 307933/2018-0), FAPEMIG (Grant number PPM-00053-17) and CAPES.
