A novel method for optimal test sequencing under unreliable test based on Markov Decision Process

Abstract

In this paper, a novel method for optimal solution in long-run is proposed to solve the test sequencing problem under unreliable tests that exist widely in real applications. The fault diagnosis process is presented and reformulated as a typical Markov Decision Process model, with the uncertainty of the tests is descripted by false alarm and detection probabilities which are to be the transition probabilities of the model. Moreover, the repeated test is adopted to improve the reliability of fault diagnosis. The cost and information gain are both considered in test chosen to achieve fast diagnosis with minimum cost. An application on the launcher device of a missile is studied in detail, as well as the comparisons in diagnostic performance among the proposed solution and the ones of traditional methods. Both the simulation and results illustrate the validity and feasibility of the proposed method.

Keywords

Optimal test sequencing unreliable tests Markov Decision Process repeated test

1 Introduction

The problem to optimize test sequencing has drawn wide attention of scholars both at home and abroad, and some effective methods have been proposed. Among them, Johnson firstly proposed the greedy algorithm based on information heuristic to optimize the test selection step by step [1]. Pattipati et al. developed a dynamic planning (DP) algorithm [2, 3], but it brings large amount of calculation. And then they combined the AND/OR graph search method with information heuristic functions [4], of which AO* algorithm is widely applied and studied [5 –7] but easy to get stuck in backtracking and searching for a global optimal solution and causes a surge in iterations. The Rollout algorithm is developed [8 –10] to solve the optimal sequential test problem with less complexity. A bottom-up algorithm is proposed [11] to build a decision tree which can get the global optimal solution, but this may cause combinational explosion. Recently, intelligent algorithms such as the genetic algorithm [12], discrete binary particle swarm optimization (DPSO) [13], support vector machine (SVM) [14] applied in the optimal test sequencing have been suggested to solve for complex real-world systems.

Although the test sequencing problem can be optimized in a way by the methods above theoretically, they are based on a hypothesis that the tests are always perfect. In fact, in the practical systems of engineering and marketing applications [15, 16], on account of the uncertainty caused by imprecision or interference of electromagnetic, unreliable sensors or components, abrupt environmental disturbances, changing subsystem interconnections and so on [16 –19] the available measurement data may be incorrect or imprecise as the tests are unreliable. For this, Tao et al. [20] adopted Markov jump systems (MJSs) to characterize the unideal measurements for sensor failures in reliable control. Zhang et al. [21, 22] suggested balanced truncation approach to simplify the MJSs or systems of a Markovian process. Wang and Zuo [23] applied the fuzzy logic theory to solve the uncertainty information in the safety diagnosis process. Various algorithms for test sequencing with unreliable tests are discussed by Raghavan et al. [24]. The evaluation functions with unreliable tests are developed in Ref. [25] along with the diagnostic strategies maximizing the test cost and diagnostic accuracy respectively. The misdiagnosis cost is also proposed with Rollout algorithm [26] to obtain the optimal diagnostic strategy under unreliable tests. Wei et al. [27] developed a tabu search algorithm and a simulation-based evaluation technique to find high-quality solutions in sequential system diagnosis with imperfect tests. All the methods mentioned above, just simply consider the cost or misdiagnosis cost generated by unreliable tests and they are based on traditional algorithms, with large amount of computation and analysis to get the optimal solutions.

The fault diagnosis under unreliable tests is a typical sequential decision problem with uncertainty in the outcomes of all available tests. As an efficient technique for sequential decision problems in dynamic and uncertain environments, Markov Decision Process (MDP) has widely been applied to model systems with probabilistic and nondeterministic behaviors in hospital management [28], marketplace [29], defense against jamming attacks [30], mobile communications [31] and the source of uncertainty can be directly modeled as probabilistic components in the model. So in this paper, we introduce the MDP to get the optimal test sequencing in fault diagnosis with unreliable tests. Firstly, the mathematical description of test sequencing problem is presented and the uncertainty of unreliable tests is descripted as false alarm probability and detection probability. With the Markov property of classic fault diagnosis process, the diagnosis model or framework based on MDP is outlined. And the repeated application of a test is adopted to improve the reliability of fault diagnosis in the proposed model. The test cost and information gain are both considered with weighted sum method to achieve fast fault diagnosis with minimum cost. By solving the proposed model, the optimal test sequencing in long-run with high-reliability and less computation is obtained. Moreover, comparisons in diagnostic performance among different test sequencings of the proposed method and traditional ones are discussed.

2 Test sequencing problem description

In this work, we assume that there is only one or no faults occurring in the system during the fault diagnosis process. The test sequencing problem can be described as a quintuple of {F, p, T, C, D}, the details are as follows.

F = {f₀, f₁, …, f_m} (m ≥ 1) is a finite ambiguity set or group of failure sources before test in a fault diagnosis process, where f₀ is the “no-fault” condition and f_i represents that the i-th failure occurs only.

p = {p (f₀) , p (f₁) , …, p (f_m)} (m ≥ 1) is a set of priori probabilities of all failure sources.

T = {t₁, t₂, …, t_n} (n ≥ 1) is a finite set of all available tests with binary outcomes, where t_j (1 ≤ j ≤ n) can check one or more specific failure sources.

C = {c₁, c₂, …, c_n} (n ≥ 1) is a set of test cost measured of considering execution time, power and some other economic factors. Each c_j (1 ≤ j ≤ n) is one-one correspondence to the test t_j ∈ T (1 ≤ j ≤ n).

D = (d_ij) is a binary matrix of dimension m×n which represents the logic relationships between the set of failure sources F and the test set T, where d_ij = 1 as the failure f_i can be detected by test t_j and 0 otherwise. Besides, this matrix can be obtained by reachability analysis using fault tree model, information flow model or multi-signal model [32].

This problem actually is to find the optimal test sequencing with minimum test cost to achieve quickly failure source isolation with the knowledge above. However, the tests are unreliable for the false alarm and missed detection caused by various interferences existed extensively in real-systems. The reliability of each test t_j can be characterized by detection and false-alarm probability pair (P_dj, P_fj), where P_dj = {t_j fails | any of failure source monitored by t_j has failed} and P_dj = {t_j fails | none of failure source monitored by t_j has failed}. Moreover, P_dj + P_fj = 1. Combined the reliability (P_dj, P_fj) with the correlation matrix D, we can get the correlation matrix under unreliable tests, denoted as B = (b_ij), and is given by: $b_{ij} = d_{ij} P_{dj} + (1 - d_{ij}) P_{fj}$ (1)

Where b_ij represents the probability of test t_j fails when the failure f_i has occurred. By the way, the false-alarm probability or miss-detection probability can be estimated from the historical statistics data or be obtained from the reliability experiment, which will be my next study.

Since the tests are imperfect, it makes sense for applying the same test multiple times. When the results of repeated tests are inconsistent, this fault diagnosis process should go back to the previous step, which means that the next fault state is to be the previous one. This operation could forcefully improve the reliability of the diagnosis results.

3 The diagnosis model based on MDP

Generally, the fault diagnosis problem can be descripted as a process of decreasing the ambiguity in the system fault state, namely fault inference machine. The final malfunction of a system is gradually determined through inferring with the test outcomes of each step, and the principle of inferring is expressed as ${\begin{matrix} F_{jp} = {f_{i} | d_{ij} = 0, \forall f_{i} \in F}, & if t_{j} passed \\ F_{jf} = {f_{i} | d_{ij} = 1, \forall f_{i} \in F}, & if t_{j} failed \end{matrix}$ (2) where d_ij is the corresponding value in matrix D and F = { f₀, f₁, …, f₀ } (n ≥ 0) is the ambiguous fault group of a system state before executing test t_j, and F_jp, F_jf are the subsets of F. Also f_i (f_i ∈ F) is the i-th failure occurred in the system and is to be an element of either F_jp or F_jf. Then the next fault state of this system can be identified to be one of the two subsets according to the outcomes of t_j. Further, considering the fault states and tests before, this process also could be descripted as $P [F_{k + 1} | h_{k - 1}, F_{k}, t_{j}] = P [F_{k + 1} | F_{k}, t_{j}]$ (3) where h_k-1 represents the history of system fault states and chosen tests before time step k, and h_k-1 ={ F₀, t₀, F₁, t₁, …, F_k-1, t_k-1 }. From Equations (2) and (3), it’s not hard to find that the next fault state of the system is deduced just from the current system state and outcomes of the chosen test, and has nothing to do with the fault states and tests before. This is a typical Markovian process with problems of sequential decision [33]. So in this work, the theory of MDP is applied to solve the test sequencing problem and obtain an optimal test sequencing or policy.

The classic model of a discrete time MDP can be descripted as a five-tuple ${S, A (i), K, p_{ij} (a), r (i, a), i, j \in S, a \in A (i)}$ where

S is the state space, a set of states which descript the current situation of the system at each time step.

A(i) is the set of available actions at state i as A is the set of all possible actions which could control the system dynamics or state transitions.

K is the set of time steps where decisions need to be made.

p_ij(a) denotes the probability of state i transfers to state j of taking an action a, which is probabilistic description of the uncertainty in the results of all possible actions.

r(i,a) denotes the instantaneous reward or payoff received at state i of taking action a.

This model successfully shows that the state evolution dynamics of a system is controlled by an agent choosing and executing the action a_k at each time step k ∈ K, and decision sequencing of this procedure is called policy, written as π. To evaluate a policy, the expected cumulative sum of instantaneous rewards along a MDP trajectory, also called the value function, is developed and defined as $\begin{matrix} V (i, π) = E [r_{0} + γ r_{1} + \dots + γ^{k} r_{k} | S_{0} = i] \\ = E [\sum_{k = 0}^{\infty} γ^{k} r_{k} | S_{0} = i] \end{matrix}$ (4) where γ is a discount factor which can weigh the rewards received in the future being discounted according to how far away in time they will be received, and 0< γ<1. In decision-making domain, the future payoff is not as good as the current one. So the later rewards are weighed less than the earlier ones and the rewards of decisions in far distant effect little on the total amount of rewards.

The final target of a MDP is to search for an optimal strategy π^*, which provides the best rewards sequencing and maximizes the value function. Generally, the optimal value function can be expressed by $V (i, π^{*}) = sup_{π \in \prod} V (i, π), i \in S$ (5) where ∏ is the strategy space and consists of all possible sequences of actions in action set A.

Based on the above analysis, we could get the fault state transitions based on MDP as Fig. 1, according to the inference machine as Equation (2) with the possible outcomes of the chosen tests and repeated tests. In Fig. 1, “t₁P” represents that the outcome of test t₁ is “passed” as d_ij = 1, whereas “t₁F” means “failed” and corresponds to d_ij = 0.

Fig.1

Fault state transitions under unreliable test.

Consequently, the fault diagnosis model based on MDP under unreliable tests is built. Similarly, the fault state space S = { F₁, F₂, …, F_q } (q ≥ 1) is composed of all the possible fault states that may occur in a system, where F₁ ={ f₀, f₁, …, f_m } (m ≥ 0) is the initial ambiguous fault group. Specifically, when the diagnosis object is a Line Replaceable Unit (LRU) and the failure source should be isolated to a Shop Reliable Unit (SRU), {f₀, f₁, … , f_m } contains all the failure sources or modes in the SURs attached to the LRU.

And the action space A is equal to test set T, i.e. A = T = { t₁, t₂, …, t_n } (n ≥ 1). The state transition probability of fault state F → F_jp is closely related to the false alarm and missed detection probability pair (P_dj, P_fj) of test t_j, as mentioned in section 2 of Equation (1), and it can be further described as ${\begin{matrix} p (F_{jp} | F, t_{j}) = b_{ij} = P_{fj}, & d_{ij} = 0 or t_{j} passed \\ p (F_{jf} | F, t_{j}) = b_{ij} = P_{dj}, & d_{ij} = 1 or t_{j} failed \end{matrix}$ (6)

Particularly, when t_j is applied for the second time subsequently, at state F_jp or F_jf, the deduction process is expressed as ${\begin{matrix} p (F_{jp} | F_{jp}, t_{j}) = P_{dj}, & d_{ij} = 0 and t_{j} passed \\ p (F | F_{jp}, t_{j}) = P_{fj}, & d_{ij} = 0 and t_{j} failed \end{matrix}$ (7) where F is the fault state of previous time step. For F_jf, there is a similar deducing process.

As known, the goal of fault diagnosis is to realize fast fault detecting and isolating with minimum test cost. So we make the weighed sum of test cost C ={ c₁, c₂, …, c_n } and the information gain be the instantaneous reward of executing test at each step, and they are defined as $r_{c} (s_{k}, a_{k}) = r_{c} (F_{q}, t_{n}) = - C_{n}$ (8)

$\begin{matrix} r_{i} (s_{k}, a_{k}) & = & I (F_{q}, t_{n}) \\ = & - [\frac{P (F_{qnp})}{P (F_{q})} 1 b \frac{P (F_{qnp})}{P (F_{q})} \\ + \frac{P (F_{qnf})}{P (F_{q})} 1 b \frac{P (F_{qnf})}{P (F_{q})}] \end{matrix}$ (9) $r (s_{k}, a_{k}) = {αr}_{c} (F_{q}, t_{n}) + (1 - α) r_{i} (F_{q}, t_{n})$ (10) where r_c (s_k, a_k) and r_i (s_k, a_k) represent the instantaneous rewards of cost and information gain respectively of taking test action t_n at state F_q of time step k, lb is the base-2 logarithm, and α is the weight coefficient of these two kinds of rewards. Besides, test cost is negative for the maximum reward as expected, so we take the negative value of it here. Also F_qnp, F_qnf are the subsets and the possible subsequent states after F_q according to Equations (7)∼(8).

According to Equation (4), the value function of cumulative reward in fault diagnosis process with initial state F₁ can be expanded as $\begin{matrix} V (F_{1}, π) = E [r_{1} + \sum_{k = 1}^{\infty} γ^{k} r_{k + 1} | S_{0} = F_{1}] \\ = \sum_{t_{n} \in T} π (F_{1}, t_{n}) \sum_{F_{q} \in S} p (F_{q} | F_{1}, t_{n}) \\ (r (F_{q} | F_{1}, t_{n}) + γ E [\sum_{k = 0}^{\infty} γ^{k} r_{k + 1} | S_{0}^{'} = F_{q}]) \\ = \sum_{t_{n} \in T} π (F_{1}, t_{n}) [r (F_{1}, t_{n}) + \\ γ \sum_{F_{q} \in S} p (F_{q} | F_{1}, t_{n}) V (F_{q}, π)], π \in Π \end{matrix}$ (11) where F_q is the next fault state of the system after F₁ and r (F₁, t_n) is known as Equation (10). Furthermore, the optimal value function with maximum value in long-run is obtained as $\begin{matrix} V (F_{1}, π^{*}) = sup_{π \in Π} V (F_{1}, π) \\ max_{t_{n} \in T} [r (F_{1}, t_{n}) \\ + γ \sum_{F_{q} \in S} p (F_{q} | F_{1}, t_{n}) V (F_{q}, π^{*})] \end{matrix}$ (12)

Then the optimal diagnostic policy can be deduced as $π^{*} = \underset{π \in Π}{arg max} V (F_{1}, π)$ (13)

By solving Equations (11) and (13) with the classic algorithms, like policy iteration algorithm and value iteration algorithm [33], the optimal test sequencing under unreliable tests can be figured out. In the next section, details of this method will be demonstrated through an application case.

4 Case study

In this work, the optimal test sequencing for the suspension and launcher test on a certain type of missile is obtained using the proposed method. The airborne suspension ejection equipment is an important interface device that realizes the mechanical, electrical, RF and gas connections between the aircraft and missile. After analyzing the signals and test requirements of them, the possible failure sources and available tests are obtained and given in Table 1 specifically, along with the logic relationships among them, namely correlation matrix D, and the probabilities about failure occurrences, false alarm and detection probabilities of the tests. Then the fault diagnosis model based on MDP under unreliable tests could be built smoothly as expressed in Section 3.

Table 1
The dependency matrix, failure occurrence rates, false alarm and detection probabilities, and test expenses of launcher ignition cable module

FT t ₁ t ₂ t ₃ t ₄ t ₅ P(f_n)

f ₁ Radio frequency interface 0 0 0 0 0 0.01

f ₂ Gas propeller 0 1 0 0 1 0.05

f ₃ Transmitter box 0 0 1 1 1 0.08

f ₄ Launch control power box 0 0 0 1 1 0.10

f ₅ The signal component 1 1 1 0 0 0.08

f ₆ Synchronizing mechanism 1 1 1 1 0 0.06

Test cost (C_n) 1 1.2 1.5 1.2 1.3 —

P_dj 0.81 0.85 0.87 0.90 0.78 —

p_fj 0.19 0.15 0.13 0.10 0.22 —

	FT	t ₁	t ₂	t ₃	t ₄	t ₅	P(f_n)
f ₁	Radio frequency interface	0	0	0	0	0	0.01
f ₂	Gas propeller	0	1	0	0	1	0.05
f ₃	Transmitter box	0	0	1	1	1	0.08
f ₄	Launch control power box	0	0	0	1	1	0.10
f ₅	The signal component	1	1	1	0	0	0.08
f ₆	Synchronizing mechanism	1	1	1	1	0	0.06
	Test cost (C_n)	1	1.2	1.5	1.2	1.3	—
	P_dj	0.81	0.85	0.87	0.90	0.78	—
	p_fj	0.19	0.15	0.13	0.10	0.22	—

1) With the matrix D in Table 1, firstly all the possible fault states of this component can be deduced following the inference process as Fig. 1, and they are enumerated in Table 2. Distinctly, the state space of this system is S ={ F₁, F₂, …, F₂₄ }, and the act set is A ={ t₁, t₂, t₃, t₄, t₅ }. Particularly, the terminal states of a fault diagnosis process are confirmed as the state of single failure source can be isolated with no ambiguity finally, in this work, which are F₁₂, F₁₃, F₁₆, F₁₇, F₁₉, F₂₁.

2) All the state transition probabilities can be determined according to Equation (6) with the information in Table 2. Because of the limited space, the probabilities of all state transitions are no longer listed one by one. For t₁, there is a transition probability matrix E₁ to describe all the transfer relationships and probabilities of the state transitions under it and E₁ is given as $\begin{matrix} t_{1} \to E_{1} & = & \begin{matrix} F_{1} \\ F_{2} \\ ⋮ \\ F_{24} \end{matrix} \overset{\begin{matrix} F 1 & F 2 & \dots & F_{24} \end{matrix}}{[\begin{matrix} 0 & P_{f 1} & \dots & 0 \\ P_{f 1} & P_{d 1} & \dots & 0 \\ ⋮ & ⋮ & ⋱ & ⋮ \\ P_{f 1} & 0 & \dots & P_{d 1} \end{matrix}]}, P_{d 1} = 0.81 \end{matrix}$ where each element e_ij in E₁ represents the transition probability of fault state F_i → F_j. Particularly, the state transitions of F₂ in the second row of E are the results of applying t₁ for the 2-th time at state F₁, as well as F₃. And F₁₄, F₁₅ are the results of applying t₁ for the 3-th time. Similarly, the transition probability matrix E₂, E₃, E₄, E₅ under test of t₂, t₃, t₄, t₅ could be acquired.

Table 2

All the possible fault states of the launcher device

Fault state	Failure source	Fault state	Failure source	Fault state	Failure source
F ₁	f ₁ f ₂ f ₃ f ₄ f ₅ f ₆	F ₉	f ₃ f ₄ f ₆	F ₁₇	f ₆
F ₂	f ₁ f ₂ f ₃ f ₄	F ₁₀	f ₁ f ₅ f ₆	F ₁₈	f ₁ f ₄
F ₃	f ₅ f ₆	F ₁₁	f ₂ f ₃ f ₄	F ₁₉	f ₁
F ₄	f ₁ f ₃ f ₄	F ₁₂	f ₂	F ₂₀	f ₂ f ₅
F ₅	f ₂ f ₅ f ₆	F ₁₃	f ₃	F ₂₁	f ₄
F ₆	f ₁ f ₂ f ₄	F ₁₄	f ₁ f ₂	F ₂₂	f ₂ f ₄
F ₇	f ₃ f ₅ f ₆	F ₁₅	f ₃ f ₄	F ₂₃	f ₃ f ₆
F ₈	f ₁ f ₂ f ₅	F ₁₆	f ₅	F ₂₄	f ₁ f ₅

In section 3, the instantaneous reward of a test action has been defined by Equations (7) ∼ (9). And the rewards of each test and state transition are further obtained with the analysis and results above, which are not listed one by one here for limited space. By the way, the weight coefficient is set as α = 0.5 in this work for cost and efficiency are equally important, or other values as appropriate.

Through the above procedures, this diagnosis model is completely built, and the optimal test sequencing of diagnosing this equipment can be obtained by solving the optimal value function of Equation (11) with all the necessary parameters and values are known. Since there is a complete MDP tool box in MATLAB, the policy iteration algorithm is applied and programmed to get the solution of this problem or Equation (11). The results are shown in Figs. 2 and 3, and the diagnostic performance comparisons on different parameters and different test sequencings are also made by simulations.

Fig.2

The best decisions for all states.

Fig.3

The utility values of the best test sequencings.

In Fig. 2, the subplots of red, purple and blue represent the policies of γ = 0.9, 0.8, 0.7 respectively. Easy to see, there is no difference between test sequencings of discounts 0.9 and 0.8, and the best test sequencings of γ = 0.8, 0.7 only have different test selections on state F₅ and F₇. As shown in Fig. 3, the results of optimal value function Equation (12) under different initial fault states for discount 0.9, 0.8 and 0.7 are nearly the same with little difference before F₁₁. So the best sequencing may be different for different weigh expectations on the future rewards in test selection. Moreover, the comparison results among different sequencings are given in Figs. 4–6, and γ = 0.8 in later work.

Fig.4

The best test actions under different test sequencings.

Fig.5

Fault state transitions under different test sequencings, where horizontal axis corresponds to the test or time step(k) of diagnosis process and vertical axis represents the possile fault states.

Fig.6

The terminal states occupancy under different test sequencings, where vertical axis is coordinate to the occupancy of all terminal states (%) and horizontal axis corresponds to the number of experiments.

In Fig. 4, the best test selections for all fault states are given under three different optimal test sequencings, which are based on MDP, greedy algorithm and a random test sequencing. And the test actions of some states may be consistent under those three different sequencings. The Monte Carlo experiments with three different test sequencings are conducted to discuss their diagnostic performances.

Basically, the state transitions of different test sequencings for one single experiment are shown in Fig. 5. It’s obvious that the MDP test sequencing and greedy test sequencing end with the terminal state F₁₇ and F₁₂ respectively within 2 steps, and the system may get back to the initial fault state at the 4-th step under the random sequencing as the blue polyline and cost much more to reach a terminal state.

To compare the diagnostic performance, the occupancy or visiting rate of terminal state which indicates the ability of fault isolation is introduced in this experiment. Then we have done 1600 Mote Carlo experiments for these three test sequencings respectively and calculate their average occupancy of terminal states as shown in Fig. 6. In addition, the test step of each experiment is 4 which is adequate for fault diagnosis in this case from Fig. 5. As known in Fig. 6, the occupancy of terminal states is 50% approximately for MDP test sequencing and 45% for greedy test sequencing. That means, the MDP test sequencing is better than the greedy one for about 0.25 step faster, for the random test sequencing, the occupancy rate is so tiny for successful diagnosis. All the results above indicate that the proposed method is scientific and valid to get the optimal test sequencing under unreliable tests, and the obtained solution has better performance in fault diagnosis than traditional ones. The sample size is small in this case, or the advantage could be more obvious.

5 Conclusion

This work provides a novel intelligent method to generate an optimal test sequencing for fault diagnosis, considering the extensive existence of unreliable tests in real-world applications. The fault diagnosis process is modeled based on classic MDP as a process that the ambiguous fault state transfers to one of less ambiguous and could be identified to be a terminal state at last, with the detection and false alarm probabilities of unreliable tests to be the fault state transition probabilities and two important indexes in fault diagnosis, test cost and the information gain, are adopted to achieve fast diagnosis with minimum cost. By solving this diagnosis model based on MDP under unreliable tests, the optimal test sequencing is obtained consequently. The simulation case has demonstrated the effectiveness and feasibility of the proposed method. The results of comparisons also show that the test sequencing of proposed method has better diagnostic performance than the ones of traditional methods. Besides, how to acquire the precise value of transition probability under unreliable tests is worthy of study.

References

John

R.A.

, An information theory approach to diagnosis, Proceedings of the 6th Annual Conference on reliability and quality control, 1960, pp. 102–109.

Wei

W.-C.

, Coolen

and Leus

, Sequential testing policies for complex systems under precedence constraints, Expert Systems with Applications 40(2) (2013), 611–620.

Pattipati

K.R.

and Alexanderidis

, Application of heuristic search and information theory to sequential fault diagnosis, IEEE Transactions on System, Man, and Cybernetics 20(4) (1990), 872–887.

Pattipati

K.R.

and Deb

, System testability analysis and research tool, Proceedings of the IEEE Autotestcon Conference, 1990, pp. 395–402.

, Shen

S.-T.

, Li

Y.-H.

and Bai

, Sequential diagnostic strategy for the diagnosis of complex systems, in: Conference on Computers, Communications, Control and Power Engineering, 2002, pp. 1662–1666.

Boumen

, Ruan

and de Jong

I.S.M.

, Hierarchical test sequencing for complex systems, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 39(3) (2009), 640–649.

Wang

H.-X.

, Ye

X.-H.

and Tian

S.-X.

, Research on test sequencing problem based on generalized AO* algorithm, Acta Armamentarii, 31(2) (2010), 204–208. (in chinese)

and Pattipati

K.R.

, Rollout strategies for sequential fault diagnosis, IEEE Transactions on System, Man, and Cybernetics: Part A 33(1) (2003), 86–99.

Huang

Y.-F.

and Jing

, Diagnosis strategy for multi-value attribute system based on Rollout algorithm, Control and Decision 26(8) (2011), 1268–1272. (in chinese)

10.

Huang

Y.-F.

, Jing

and Luo

B.-H.

, Sequential multiple fault diagnosis strategy based on Rollout algorithm, Kongzhi Yu Juece/control & Decision 30(3) (2015), 572–576. (in chinese)

11.

Kundakcioglu

O.E.

and Unluyurt

, Bottom-up construction of minimum-cost AND/OR trees for sequential fault diagnosis, IEEE Transactions on System, Man, and Cybernetics 37(5) (2007), 621–629.

12.

J.-S.

, Xu

and Li

X.-S.

, Generation of test strategy for sequential fault diagnosis based on genetic algorithms, Acta Simulata Systematica Sinica 16(4) (2004), 833–836.

13.

Yang

C.-L.

, Yan

J.-H.

and Long

, A novel test optimizing algorithm for sequential fault diagnosis, Microelectronics Journal 45(6) (2014), 719–727.

14.

Z.-L.

, Outbib

and Giurgea

, Online implementation of SVM based fault diagnosis strategy for PEMFC systems, Applied Energy 164(2) (2016), 284–293.

15.

Rosales

M.J.

, Rojas

A.L.

, Chavarría

S.L.

and Elizondo

M.M.

, Forecasting a stores credit plan using the Markov chains. International Journal of Innovative Computing Information and Control, 13(5) (2017), 1537–1547.

16.

Tao

, Lu

R.-Q.

, Su

H.-Y.

, Shi

and Wu

Z.-G.

, Asynchronous filtering of nonlinear Markov jump systems with randomly occurred quantization via T-S fuzzy models, IEEE Transactions on Fuzzy Systems PP(99) (2017), 1.

17.

Shi

, Li

F.-B.

, Wu

L.-G.

and Lim

C.-C.

, Neural network-based passive filtering for delayed neutral-type semi-Markovian jump systems, IEEE Trans on Neural Networks and Learning Systems 28(9) (2017), 2101–2114.

18.

Chen

Z.-S.

, Yang

Y.-M.

and Hu

, A technical framework and roadmap of embedded diagnostics and prognostics for complex mechanical systems in prognostics and health management systems, IEEE Transactions on Reliability 61(2) (2012), 314–322.

19.

Verbert

, BabuŠka

and Schutter

B.D.

, Bayesian and Dempster– Shafer reasoning for knowledge-based fault diagnosis– a comparative study, , Engineering Applications of Artificial Intelligence 60 (2017), 136–150.

20.

Tao

, Lu

R.-Q.

, Wu

Z.-G.

and Wu

Y.-Q.

, Reliable control against sensor failure for markov jump systems with unideal measurements, IEEE Transactions on Systems, Men, and Cybernetics: Systems PP(99) (2017), 1–9.

21.

Zhang

H.-Y.

, Wu

L.-G.

, Shi

and Zhao

Y.-X.

, Model reduction on Markovian jump systems with partially unknown transition probabilities: Balanced truncation approach, IET Control Theory and Applications 9(9) (2015), 1411–1421.

22.

Zhang

H.-Y.

, Wu

L.-G.

, Shi

and Zhao

Y.-X.

, Balanced truncation approach to model reduction of Markovian jump time-varying delay systems, Journal of the Franklin Institute 28(9) (2015), 4205–4224.

23.

Wang

S.-Y.

and Zuo

H.-Y.

, Safety diagnosis on coal mine production system based on fuzzy logic inference, Journal of Central South University 19(2) (2012), 477–481.

24.

Raghavan

, Shakeri

and Pattipati

, Test sequencing algorithms with unreliable tests, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 29(4) (1999), 347–357.

25.

Yang

, Yang

S.-M.

, Qiu

, Liu

G.-J.

and Chen

G.-Y.

, Sequential test strategies with unreliable tests, IEEE AUTOTESTCON 2008, Salt Lake City, 2008, pp. 587–592.

26.

Dong

H.-D.

, He

, Liu

, He

H.-F.

, Qi

H.-Z.

and Ji

Y.-W.

, Study on fault diagnostic strategy under unreliable test, Control Conference, Chengdu, China, 2016, pp. 6718–6721.

27.

Wei

W.-C.

, Li

H.-B.

and Leus

, Test sequencing for sequential system diagnosis with precedence constraints and imperfect tests , Decision Support Systems 103 (2017), 104–116.

28.

Nunes

L.G.N.

, de Carvalho

S.V.

and Rodrigues

R.C.M.

, Markov decision process applied to the control of hospital elective admissions, , Artificial Intelligence in Medicine 47 (2009), 159–171.

29.

Greenwald

, Kannan

and Krishnan

, On evaluating information revelation policies in procurement auctions: A Markov decision process approach, Information System Research 21(1) (2010), 15–36.

30.

Y.-L.

, Wang

B.-B.

and Liu

K.J.R.

, Optimal defense against jamming attacks in cognitive radio networks using the Markov decision process approach, IEEE Global Telecommunications Conference 45(2) (2010), 1–5.

31.

Bokani

, Hassan

and Kanhere

, HTTP-based adaptive streaming for mobile clients using Markov decision process, 20th International Packet Video Workshop (PV) 10(1) (2013), 1–8.

32.

Qiu

, Liu

G.-J.

and Yang

, Testability modeling technology and design for testability of equipment, Beijing: Science Press, 2012.

33.

Russell

S.J.

and Norving

, Artificial intelligence: A modern approach, Third Edition, Beijing: Pearson Education Asia Limited and Tsinghua University Press, 2012.