Bayesian statistical models with uncertainty variables

1 Introduction

Bayesian statistical inference is an important technique in mathematical statistics in which Bayes’ theorem (Albert [1]) is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian statistics was named after Bayes [3], who studied how to compute a distribution for the probability parameter of a binomial distribution. Laplace [8] used Bayesian methods to solve a number of statistical problems. Many Bayesian methods were developed by Jeffreys [7], Berger [4], etc. With the advent of powerful computers and new algorithms like Markov chain Monte Carlo (Brooks and Roberts [5]), Bayesian methods have seen increasing use within statistics, such as Bayesian experimental design, Bayesian networks, Bayesian hierarchical modelling. Hierarchical model is used when information is available on several different levels of observational units. The hierarchical form of analysis and organization helps in the understanding of multiparameter problems and also plays an important role in developing computational strategies. Baladandayuthapani et al. [2] employed Bayesian hierarchical model to analyze colon carcinogenesis data. Shen et al. [19] proposed an improved Bayesian dictionary learning algorithm to effectively recover Aqua Moderate Resolution Imaging Spectroradiometer Band 6.

In Bayesian statistics, it is important to obtain the methods for estimating and testing the unknown parameters form the given observations and prior information. Traditionally, the observations of the sample are assumed to be precise and treated as real numbers. However, the observations are often imprecise by virtue of the uncertainty when acquiring the sample, and cannot be treated as crisp values. For example, the data of the factoriesąŕ carbon emission are collected in an imprecise way. It was shown by many surveys that uncertainty theory is more fitted to model the data with imprecise observations given by the experts (Liu [15]). Thus we should take the imprecisely observed data as uncertain variables and describe them by uncertainty distributions (Liu [10]).

The uncertainty theory is a branch of mathematics to model the human uncertainty. In the framework of the uncertainty theory, uncertain variable is used to model an uncertain quantity, and uncertainty distribution was employed to describe an uncertain variable. Gao [6] discussed some mathematical properties of uncertain measure. Peng and Iwamura [18] showed what function is an uncertainty distribution. Liu [13] proposed the operational law of uncertain variables based on the product measure axiom. The expected value of the function of uncertain variables was proposed by Liu and Ha [17]. The variance of an uncertain variable was provided by Yao [21]. The moment of an uncertain variable was proposed by Sheng and Ha [20] and Liu [16]. You [26] and Zhang [27] studied the convergence of a sequence of uncertain variables. After about ten years of development, the uncertainty theory has found a wide range of applications, for example, uncertain risk analysis (Liu [14]), uncertain programming (Liu [12]), uncertain statistics (Yao and Liu [25]), uncertain process (Yao and Li [24]), etc. Uncertain statistics is a method to collect and interpret expertąŕs data. Liu [13] proposed the least squares estimation of the unknown parameters in the uncertainty distribution. Linear uncertain regression analysis was proposed by Yao and Liu [25] and Lio and Liu [9]. Uncertain time series analysis was documented by Yang and Liu [22]. Uncertain statistical inference with the imprecise observations was discussed by Yao [23].

In this paper, we propose the uncertain Bayesian statistics inference with uncertainty variables based on uncertainty theory and Bayesian inference. Three kinds of Bayesian inference methods including the uncertain Bayesian point estimation, the uncertain Bayesian hypothesis test, and the uncertain Bayesian interval estimation are investigated when observations of the sample are imprecise. The rest of this paper is structured as follows. Some basic knowledge about uncertain theory are introduced in Section 2. Uncertain Bayesian point estimation is proposed in Section 3. Uncertain Bayesian hypothesis test is proposed in Section 4. Uncertain Bayesian interval estimation is proposed in Section 5. Finally, some remarks are made in Section 6.

2 Preliminary

In this section, we review some basic knowledge on uncertainty theory from the related references in order to make it easy for the reader to understand the paper. Definition 2.1. (Liu [10]) Let Ł be a σ-algebra on a nonempty set Γ. A set function M : Ł → [0, 1] is called an uncertain measure if it satisfies the following axioms: Axiom 1. (Normality Axiom) M {Γ} =1 for the universal set Γ;

Axiom 2. (Duality Axiom) M {Λ} + M {Λ ^c } =1 for any event Λ;

Axiom 3. (Subadditivity Axiom) For every countable sequence of events Λ₁, Λ₂, ⋯ , we have M { ⋃ i = 1 ∞ Λ i } ≤ ∑ i = 1 ∞ M { Λ i } .

Besides, in order to provide the operational law, Liu [11] defined the product uncertain measure on the product σ-algebra Ł as follows.

Axiom 4. (Product Axiom) Let (Γ _k , Ł  _k , _Mk) be uncertainty spaces for k = 1, 2, ⋯ . The product uncertain measure M is an uncertain measure satisfying M { ∏ k = 1 ∞ Λ k } = ⋀ k = 1 ∞ M k { Λ k } , where Λ _k are arbitrarily chosen events from Ł _k for k = 1, 2, ⋯, respectively.

Definition 2.2. (Liu[10]) An uncertain variable is a function ξ from an uncertainty space (Γ, eulerL, M) to the set of real numbers such that {ξ ∈ B} is an event for any Borel set B.

In order to describe uncertain variable in practice, uncertainty distribution Φ : R → [ 0 , 1 ] of an uncertain variable ξ is defined as Φ (x) = M{ ξ ≤ x } for any real number x. An uncertainty distribution Φ is called regular if its inverse function Φ^-1 exists. In this case, the inverse function Φ^-1 is called an inverse uncertainty distribution.

An uncertain variable ξ is called linear if it has a linear uncertainty distribution Φ ( x ) = { 0 , if x ≤ a x - a b - a , if a ≤ x ≤ b 1 , if x ≥ b denoted by Ł (a, b), where a and b are real numbers with a < b, and the inverse uncertainty distribution of linear uncertain variable Ł (a, b) is Φ - 1 ( α ) = ( 1 - α ) a + α b .

The uncertain variables ξ₁, ξ₂, ⋯ , ξ _m are said to be independent (Liu[11]) if M { ⋂ i = 1 m { ξ i ∈ B i } } = ⋀ i = 1 m M { ξ i ∈ B i } , for any Borel sets B₁, B₂, …, B _m of real numbers.

Let ξ₁, ξ₂ ⋯ , ξ _n be independent uncertain variables with regular uncertainty distributions Φ₁, Φ₂ ⋯ , Φ _n , respectively. If f is strictly increasing with respect to ξ₁, ξ₂ ⋯ , ξ _m and strictly decreasing with respect to ξ_m+1, ξ_m+2 ⋯ , ξ _n , then ξ = f (ξ₁, ξ₂ ⋯ , ξ _n ) is an uncertain variable, the inverse uncertainty distribution of the ξ = f (ξ₁, ξ₂ ⋯ , ξ _n ) are given by Liu [10], Ψ - 1 ( α ) = f ( Φ 1 - 1 ( α ) , ⋯ , Φ m - 1 ( α ) , Φ m + 1 - 1 ( 1 - α ) , ⋯ , Φ n - 1 ( 1 - α ) ) .(2.1)

As the average value of an uncertain variable in the sense of uncertain measure, expected value can represent the size of the uncertain variable. The expected value and variance of uncertain variable ξ are defined by E [ ξ ] = ∫ 0 + ∞ M { ξ ≥ r } d r - ∫ - ∞ 0 M { ξ ≤ r } d r , provided that at least one of the two integrals is finite, V [ ξ ] = E [ ( ξ - E [ ξ ] ) 2 ] , respectively.

Let ξ be an uncertain variable with an uncertainty distribution Φ. If its kth moment E [ξ ^k ] exists, then E [ ξ k ] = ∫ - ∞ ∞ x k d Φ ( x ) . Furthermore, if Φ is regular, then E [ ξ ] = ∫ 0 1 Φ - 1 ( α ) d α .

lET ξ be an uncertain variable with uncertainty distribution Φ, and f (x) be a strictly monotone (increasing or decreasing), then we have E [ f ( ξ ) ] = ∫ 0 1 f ( Φ - 1 ( a ) ) d a .(2.2)

In Bayesian statistical inference, Bayes’ theorem can be used to estimate the parameters of a probability distribution or statistical model. Since Bayesian statistics treats probability as a degree of belief, we can directly assign a probability distribution that quantifies the belief to the parameter or set of parameters. The prior distribution π (θ) includes information about unknown parameter θ, and the data x includes the information about θ. The information in the prior and data are integrated in the joint distribution function f (x, θ) = f (x|θ) π (θ). The posterior distribution π (θ|x), which is the conditional distribution of the unknown quantities conditional on the observed data, is obtained easily using condition probability formula, π ( θ | x ) = f ( x , θ ) f ( x ) = f ( x | θ ) π ( θ ) ∫ f ( x | θ ) π ( θ ) d θ , where f (x, θ) is the marginal distribution of x, f (x|θ) is the distribution of the data conditional on the parameter. The above formula is called Bayesąŕ theorem. It describes the conditional probability of an event based on data as well as prior information or beliefs about the event or conditions related to the event. The posterior probability distribution π (θ|x) is essential to Bayesian statistical inference for θ. When the observations of the sample from the population are imprecise, we propose uncertain Bayesian statistic inference problems such as Bayesian point estimation, Bayesian hypothesis test, and Bayesian interval estimation.

This section introduces uncertain Bayesian expectation estimator and uncertain Bayesian mode estimation when the observations are uncertain variables but not traditional crisp values.

This section introduces Bayesian hypothesis test when the observations of the sample are imprecise.

Let Θ be the set of all possible values of θ. Let null hypothesis about θ be H₀ : θ ∈ Θ₀ and alternative hypothesis be H₁ : θ ∈ Θ₁, where Θ₀ and Θ₁ are given subsets of Θ. We consider the following test question: H 0 : θ ∈ Θ 0 versus H 1 : θ ∈ Θ 1 . In the Bayesian framework, hypothesis test is straightforward. When acquiring the posterior distribution, we compute posterior probability of null hypothesis H₀ and alternative hypothesis H₁, respectively, α 0 ( ξ ) = P ( Θ 0 | ξ ) = ∫ Θ 0 f ( θ | ξ ) d θ , and α 1 ( ξ ) = P ( Θ 1 | ξ ) = ∫ Θ 1 f ( θ | ξ ) d θ . We make decisions by comparing α₀ and α₁. Note that α₀ ( ξ ) and α₁ ( ξ ) are uncertainty variables. If M { α 0 ( ξ ) > α 1 ( ξ ) } > 1 / 2 ,(4.1) we think uncertainty variable α₀ ( ξ ) > α₁ ( ξ ), and the null hypothesis H₀ : θ ∈ Θ₀ is accepted. If M { α 0 ( ξ ) < α 1 ( ξ ) } > 1 / 2 , we think uncertainty variable α₀ ( ξ ) < α₁ ( ξ ) and H₀ is rejected. If M { α 0 ( ξ ) < α 1 ( ξ ) } ≈ 1 / 2 , then we do not make decision, and need to furthermore acquire expert’s data or collect prior information. Example 4.1: Assume ξ₁, ⋯ , ξ _n are some imprecise observations from a population with normal distribution x|θ ∼ N (θ, σ²), and the prior is normal, θ ∼ N (μ, τ²). The posterior distribution is N ( μ 1 , τ 1 2 ) by Example 3.1. Consider testing H 0 : θ ≤ θ 0 versus H 1 : θ > θ 0 , where θ₀ is a number fixed by some experimenter. Firstly, compute α₀ (ξ) and α₁ (ξ). Since the posterior distribution is N ( μ 1 , τ 1 2 ) , we have α 0 ( ξ ) = P ( θ ≤ θ 0 | ξ ) = P ( θ - μ 1 τ 1 ≤ θ 0 - μ 1 τ 1 | ξ ) = F ( θ 0 - μ 1 τ 1 ) = F ( θ 0 - σ 2 μ + τ 2 ξ ¯ σ 2 + τ 2 τ 1 ) , where F (·) is distribution function of standard normal distribution N (0, 1). By duality of probability, we have α 1 ( ξ ) = 1 - α 0 ( ξ ) = 1 - F ( θ 0 - σ 2 μ + τ 2 ξ ¯ σ 2 + τ 2 τ 1 ) . Nextly, we compute M { α 0 ( ξ ) > α 1 ( ξ ) } = M { 2 F ( θ 0 - σ 2 μ + τ 2 ξ ¯ σ 2 + τ 2 τ 1 ) > 1 } = M { θ 0 - σ 2 μ + τ 2 ξ ¯ σ 2 + τ 2 > F - 1 ( 1 2 ) τ 1 } = M { ξ ¯ < σ 2 + τ 2 τ 2 ( θ 0 - F - 1 ( 1 2 ) τ 1 - σ 2 μ σ 2 + τ 2 ) } .

As a numerical example, we consider the intelligence test on a child in Example 3.1. Suppose the test results follow a normal distribution x ∼ N (θ, 100), where θ is defined as the child’s IQ in psychology. The prior is θ ∼ N (100, 225). the posterior distribution of the child’s IQ can be obtained when an imprecise observation ξ is obtained. The posterior distribution of child’s IQ is θ | ξ ∼ N ( μ 1 , τ 1 2 ) , where μ 1 = 100 × 100 + 225 ξ 100 + 225 = 400 + 9 ξ 13 , τ 1 2 = 8 . 32 2 . If further suppose the child’s score ξ on this test follows linear uncertainty distribution L ( 80 , 110 ) by some expert. Consider testing H 0 : θ ≤ 100 versus H 1 : θ > 100 . We compute posterior probability of null hypothesis H₀ and alternative hypothesis H₁, respectively, α 0 ( ξ ) = P ( θ ≤ 100 | ξ ) = F ( θ 0 - μ 1 τ 1 ) = F ( 100 - 400 + 9 ξ 13 8.32 ) , α 1 ( ξ ) = 1 - α 0 ( ξ ) . Nextly, we compute M { α 0 ( ξ ) > α 1 ( ξ ) } = M { 2 F ( 100 - 400 + 9 ξ 13 8.32 ) > 1 } = M { ξ < 325 225 ( 100 - F - 1 ( 1 2 ) 8.32 - 100 × 100 325 ) } = M [ ξ < 100 ] = 2 3 > 1 2 . Thus, the hypothesis H₀ : θ ≤ 100 should be accepted, the child’s IQ scores is not more than 100.

In Bayesian statistics, a credible interval is a range of values within which an unobserved parameter value falls with a particular subjective probability. It is an interval in the domain of a posterior probability density function. Assume there are n imprecise observations of sample from the population with a probability density function f (x|θ), which we denote by uncertain variables ξ₁, ⋯ , ξ _n . Suppose an unknown parameter θ is known to have a prior distribution π. When posterior distribution π (θ| ξ ) is acquired, we can find a and b, such that P ( a ≤ θ ≤ b | ξ } ) ≥ 1 - α . Note that a and b are functions of uncertain vector ξ , the definition of uncertain Bayesian credible interval is give as follows.

Definition 5.1. (Uncertain credible interval) Assume posterior distribution of parameter θ is f (θ| ξ ), for the uncertainty variables vector ξ and probability 1 - α (0 < α < 1), if the uncertainty variables θ ˆ 1 ( ξ ) and θ ˆ 2 ( ξ ) , such that P ( θ ˆ 1 ( ξ ) ≤ θ ≤ θ ˆ 2 ( ξ ) | ξ } ) ≥ 1 - α , then [ E M [ θ ˆ 1 ( ξ ) ] , E M [ θ ˆ 2 ( ξ ) ] ] is called uncertain Bayesain credible interval where credible level is 1 - α . If θ ˆ L satisfies P ( θ ˆ L ( ξ ) ≤ θ | ξ } ) ≥ 1 - α , then E M [ θ ˆ L ] is called uncertain Bayesian lower credible limit where credible level is 1 - α . If θ ˆ U satisfies P ( θ ≤ θ ˆ U ( ξ ) | ξ } ) ≥ 1 - α , then E M [ θ ˆ U ] is called uncertain Bayesian upper credible limit where credible level is 1 - α .

Example 5.1: Assume ξ₁, ⋯ , ξ _n are some imprecise observations from a population with normal distribution x|θ ∼ N (θ, σ²), and the prior is normal, θ ∼ N (μ, τ²). The posterior distribution is θ | ξ ∼ N ( μ 1 , τ 1 2 ) by Example 3.1. We have P ( a ≤ θ ≤ b | ξ ) = P ( a - μ 1 τ 1 ≤ θ - μ 1 τ 1 ≤ b - μ 1 τ 1 | ξ ) . Let a - μ 1 τ 1 = z α 2 , and b - μ 1 τ 1 = z 1 - α 2 , we have P ( a ≤ θ ≤ b | ξ ) = 1 - α , where z _α is α quantile of standard normal distribution. Hence, a = z α 2 τ 1 + μ 1 , b = z 1 - α 2 τ 1 + μ 1 ,(5.1) and credible interval of 1 - α credible level is [ E M [ a ] , E M [ b ] ] .

As a numerical example, we consider the intelligence test on a child in Example 3.1. Suppose the test results follow a normal distribution x ∼ N (θ, 100), where θ is defined as the child’s IQ in psychology. The prior is θ ∼ N (100, 225). the posterior distribution of the child’s IQ can be obtained when an imprecise observation ξ is obtained. The posterior distribution of children’s IQ is θ | ξ ∼ N ( μ 1 , τ 1 2 ) , where μ 1 = 100 × 100 + 225 ξ 100 + 225 = 400 + 9 ξ 13 , τ 1 2 = 8 . 32 2 . If further suppose the child scores ξ on this test follows linear uncertainty distribution L ( 80 , 110 ) by some expert. we take credible level 95%, by (5.1), a = z α 2 τ 1 + μ 1 = z 0.025 × 8.32 + 400 + 9 ξ 13 , and b = z α 2 τ 1 + μ 1 = z 0.975 × 8.32 + 400 + 9 ξ 13 . Thus, we have E M [ a ] = E M [ z α 2 τ 1 + μ 1 ] = z 0.025 × 8.32 + 400 + 9 E M [ ξ ] 13 = - 1.96 × 8.32 + 400 + 9 × 95 13 = 80.23 , In a similar way, we can obtain E_M [b] =112.80. The 95% credible interval of the child’s IQ is [80.23,112.80].

Uncertain Bayestian statistical inference is a new interdisciplinary to solve Bayestian statistical inference problems when observations of the sample are imprecise. This paper treated the imprecise observations as uncertain variables, and proposed three kinds of Bayesian inference methods such as Bayesian point estimation, Bayesian hypothesis test, and Bayesian interval estimation. When there is no analytic expression for the posterior distribution, the calculations are quite complicated, the proposed methods are challenging or failure and need for further research.