A Comparison of Maximum Likelihood Estimations for Normal Mean under Right Censoring

Abstract

For efficiently estimating the normal mean ( $μ$ ) under right censoring (threshold = $M$ , $σ$ is known), we compare two approaches within the maximum likelihood estimation (MLE) framework. Approach I is a hierarchical MLE for which only the empirical censoring probability is utilized. Approach II is the direct MLE for which expectation-maximization (EM) algorithm is applied to all individual observations. We use discrete approximation to explain that the asymptotic variance of Approach II estimate equals the inverse Fisher information calculated from the full log-likelihood. We prove that Approach II gives a uniformly smaller asymptotic variance than Approach I and the variance ratio is a decreasing function of $(M - μ) / σ$ . We further prove some supportive results and graphically demonstrate that EM algorithm monotonically converges to the unique MLE.

Keywords

EM algorithm convergence maximum likelihood estimation quantile right censoring variance

AMS 2010 subject classifications: 62F10

1 Introduction

We consider normal mean ( $μ$ ) estimation under right censoring (threshold = $M$ ). Suppose we collect $n$ independently and identically distributed random samples:

Y_{i} \sim N (μ, σ^{2}), i = 1, \dots, n .

(1.1)

Among them, there are $n_{1}$ complete observations ( $Y_{1}$ , $\dots$ , $Y_{n_{1}} < M$ ) and $n - n_{1}$ censored ones ( $Y_{n_{1} + 1} = \dots = Y_{n} = M$ ). Our present study depends upon the assumption of $0 < n_{1} < n$ and all expectations and/or variances (e.g., Sections 2.1 and 2.2) are conditional ones (given $0 < n_{1} < n$ ). The censoring probability is $p = 1 - Φ (\frac{M - μ}{σ})$ and Pr $(0 < n_{1} < n) = 1 - p^{n} - (1 - p)^{n} = 1 - o (1)$ with the term $o (1)$ exponentially small (as $n$ grows). All conditional approximations are subsequently justified.

1.1 Approach I: Hierarchical MLE

The following empirical censoring probability is calculated from the observations:

Pr (Y > M | μ, σ^{2}) = 1 - Φ (\frac{M - μ}{σ}) \approx \frac{n - n_{1}}{n} .

$μ$ can be subsequently estimated by the following linear function of the empirical quantile:

{\hat{μ}}_{Q} = M - σ Φ^{- 1} (\frac{n_{1}}{n}) .

This is essentially a hierarchical maximum likelihood estimation as follows:

The “censoring” event ( $Y > M$ ) follows a Bernoulli trial with event probability

p = 1 - Φ (\frac{M - μ}{σ}) .

The total number of events follows a binomial distribution with likelihood function

Pr (# {Y > M} = n - n_{1}) = C_{n}^{n - n_{1}} p^{n - n_{1}} (1 - p)^{n_{1}} .

Since $p = 1 - Φ (\frac{M - μ}{σ})$ , the hierarchical MLE is a two-step procedure:

{\hat{p}}_{MLE} = 1 - \frac{n_{1}}{n} and {\hat{μ}}_{Q} = M - σ Φ^{- 1} (1 - {\hat{p}}_{MLE}) .

For obtaining the MLE for certain function ( $g (θ)$ ) of the unknown parameter ( $θ$ ), we refer to the so-called “invariance principle” in the literature: Suppose $\hat{θ}$ is the MLE of $θ$ and $g (θ)$ is a continuous function of $θ$ , then MLE for $g (θ)$ is $g (\hat{θ})$ . Since ${\hat{p}}_{MLE}$ is the MLE for $p$ in the binomial distribution, ${\hat{μ}}_{Q}$ is a type of hierarchical MLE.

1.2 Approach II: Direct MLE

The direct MLE maximizes the full likelihood of all individual observations and we employ the EM algorithm to achieve this based on the following protocol:

Denote the complete observations as: $Z_{j} = Y_{j}$ for $j = 1, \dots, n_{1}$ .

Initialization (step $i = 0$ ): $μ^{(i)} = {\hat{μ}}_{Q}$ or ${\overset{̅}{Y}}_{n_{1}}$ .

For $j = n_{1} + 1, \dots, n$ , we update the conditional expectations of the censored observations as:

Z_{j} = E_{μ = μ^{(i)}} (Y_{j} | Y_{j} > M) = μ^{(i)} + σ \times \frac{ϕ (\frac{M - μ^{(i)}}{σ})}{1 - Φ (\frac{M - μ^{(i)}}{σ})} .

Update

μ^{(i + 1)} = \frac{\sum_{j = 1}^{n} Z_{j}}{n} .

(1.2)

Let $i = i + 1$ , continue the next round (doing 3) and 4)) till convergence.

We denote such an estimate at convergence by ${\hat{μ}}_{EM}$ .

2 Estimation Variance Comparison

We set $μ = 50$ , $M = 100$ , $σ = 40$ and $n = 100$ for a simulation (refer to Equation [1.1]). Approach I ( ${\hat{μ}}_{Q}$ ) gives mean estimation of 49.26 with variation of 7.05. Approach II ( ${\hat{μ}}_{EM}$ ) gives mean estimation of 49.97 with variation of 4.03. The latter one outperforms the former one in terms of variation. Figure 1 shows the variation comparison as sample size ( $n$ ) increases. As $n$ gets moderately large (e.g., $\approx 1000$ ), the ratio approaches the limit. We now calculate the asymptotic variances of the two estimators (Approaches I and II).

2.1 Approach I

A first-order Taylor expansion around $E (n_{1} / n)$ gives

Φ^{- 1} (\frac{n_{1}}{n}) \approx Φ^{- 1} (E (\frac{n_{1}}{n})) + (Φ^{- 1})^{(1)} (\frac{n_{1}}{n} - E (\frac{n_{1}}{n})),

and

\begin{array}{ll} Var (Φ^{- 1} (\frac{n_{1}}{n})) \approx ((Φ^{- 1} (Φ (\frac{M - μ}{σ})))^{(1)})^{2} Var (\frac{n_{1}}{n}) \\ = & ((ϕ (Φ^{- 1} (Φ (\frac{M - μ}{σ}))))^{- 1})^{2} Var (\frac{n_{1}}{n}) = \frac{Φ (\frac{M - μ}{σ}) (1 - Φ (\frac{M - μ}{σ}))}{n ϕ^{2} (\frac{M - μ}{σ})} . \end{array}

Thus, the variance of ${\hat{μ}}_{Q}$

Var ({\hat{μ}}_{Q}) \approx \frac{σ^{2} Φ (w) (1 - Φ (w))}{n ϕ^{2} (w)} .

(2.1)

Here, $w = (M - μ) / σ$ (we use this notation in the sequal). Equation (2.1) essentially arises from the so-called Delta method which approximately calculates the variance of a function of an original random variable ( $n_{1} / n$ ) with known variance. On the other hand, the exact variance of ${\hat{μ}}_{Q}$ can be well approximated by the following:

\begin{matrix} Var ({\hat{μ}}_{Q}) = σ^{2} \sum_{i = 1}^{n - 1} C_{n}^{i} (Φ (w))^{i} (1 - Φ (w))^{n - i} \\ \times (Φ^{- 1} (\frac{i}{n}) - \sum_{j = 1}^{n - 1} C_{n}^{j} (Φ (w))^{j} (1 - Φ (w))^{n - j} Φ^{- 1} (\frac{j}{n}))^{2} . \end{matrix}

(2.2)

Figure 1:

Source: The authors.

Here, the number of events ( $i$ and $j$ ) ranges from $1$ to $n - 1$ since Pr( $n_{1} \in {0, n}$ ) has an ignorable probability (see the beginning of Section 1). Such an approximation is finite and equals the unconditional case ( $n_{1} \in {0, \dots, n}$ ) up to $o (1)$ errors.

2.2 Approach II

We start with the following likelihood function for all individual observations:

L (μ | Y, M, σ) = \prod_{i = 1}^{n_{1}} \frac{1}{σ} ϕ (\frac{Y_{i} - μ}{σ}) \prod_{i = n_{1} + 1}^{n} (1 - Φ (\frac{M - μ}{σ})) .

(2.3)

The log-likelihood is as follows:

ln L (μ; Y, M, σ) = - \frac{1}{2 σ^{2}} \sum_{i = 1}^{n_{1}} (Y_{i} - μ)^{2} - n_{1} ln (\sqrt{2 π} σ) + (n - n_{1}) ln (1 - Φ (\frac{M - μ}{σ})) .

The first-order derivative of the log-likelihood is as follows:

\frac{\partial ln L}{\partial μ} = - \frac{n_{1}}{σ^{2}} (μ - {\overset{̅}{Y}}_{n_{1}}) + \frac{n - n_{1}}{σ} \times \frac{ϕ (\frac{M - μ}{σ})}{1 - Φ (\frac{M - μ}{σ})} .

(2.4)

The iterative “update” step (Equation (1.2)) reaches for a root of $\frac{\partial ln L}{\partial μ} = 0$ . Either Equation (1.2) or (2.4) involves the following iteration function:

f (μ) = \frac{1}{n} A + (1 - \frac{n_{1}}{n}) μ + \frac{n - n_{1}}{n} \times σ \times \frac{ϕ ((M - μ) / σ)}{1 - Φ ((M - μ) / σ)}, (A = \sum_{i = 1}^{n_{1}} Y_{i}) .

(2.5)

Equation (1.2) amounts to updating “ $f (μ^{(i)}) \to μ^{(i + 1)}$ ” and MLE is a solution of the fixed-point equation: $f (μ) = μ$ .

The second-order derivative of the log-likelihood is

\frac{\partial^{2} ln L}{\partial μ^{2}} = - \frac{n_{1}}{σ^{2}} + \frac{n - n_{1}}{σ^{2} (1 - Φ (w))} \times (w ϕ (w) - \frac{ϕ^{2} (w)}{1 - Φ (w)}) .

For a random variable with a continuous probability density function, general regularity conditions are available in the literature for MLE to be a consistent and asymptotic efficient estimator (the limiting variance equals the inverse of Fisher information). For the special case (Equation (2.3)), our simulation study shows that, the inverse of Fisher information accurately approximates the variance of ${\hat{μ}}_{EM}$ (given any moderately large $n$ ), that is,

Var ({\hat{μ}}_{EM}) \approx - (E (\frac{\partial^{2} ln L}{\partial μ^{2}}))^{- 1} = \frac{σ^{2}}{n} (Φ (w) + \frac{ϕ^{2} (w)}{1 - Φ (w)} - w ϕ (w))^{- 1} .

(2.6)

For Equation (2.6), we present an explanatory justification (not a rigorous proof) through using discrete approximation. The random variable $Y$ regulated by Equation (2.3) amounts to the mixture of a continuous random variable (range= $(- \infty, M)$ , with probability $Φ (w)$ ) and a point ( ${M}$ , with probability $1 - Φ (w)$ ). Given a large integer $K$ , we formulate a discrete random variable $X$ (assume $σ = 1$ without loss of generality) with the following defined probability mass function $p (x; μ)$ , where three cases ((a)–(c)) are considered.

If $Y \in [M - k Δ, M - (k - 1) Δ)$ , then $X = M - k Δ$ and $p (x; μ) = Φ (M - (k - 1) Δ - μ) - Φ (M - k Δ - μ)$ , $(1 \leq k \leq K)$ .

If $Y \in (- \infty, M - K Δ)$ , then $X = M - (K + 1) Δ$ and $p (x; μ) = Φ (M - K Δ - μ)$ . The specification in is case does not have to be rigorous since we can always make $K$ so large that any value of observed $Y$ ( $< M$ ) is to be covered in Case (1) or the probability of not being covered becomes ignorable.

If $Y \geq M$ , then $X = M$ and $p (x; μ) = 1 - Φ (M - μ)$ .

In order for the inverse of Fisher information (of $X$ ) to be the asymptotic variance of the MLE ( ${argmax}_{μ} \sum_{i = 1}^{n} ln p (x_{i}; μ)$ ), the following regularity conditions (for the asymptotic normality of MLE) apply to any discrete random variable.

For any $μ$ , the three derivatives ( $\frac{\partial ln p}{\partial μ}$ , $\frac{\partial^{2} ln p}{\partial μ^{2}}$ and $\frac{\partial^{3} ln p}{\partial μ^{3}}$ ) exist.

For any $μ$ , we can find functions ( $A_{1} (x)$ , $A_{2} (x)$ and $A_{3} (x)$ ) such that $| \frac{\partial p}{\partial μ} | < A_{1} (x)$ , $| \frac{\partial^{2} p}{\partial μ^{2}} | < A_{2} (x)$ and $| \frac{\partial^{3} p}{\partial μ^{3}} | < A_{3} (x)$ . Here, $\sum_{x \in X} A_{1} (x) < \infty$ , $\sum_{x \in X} A_{2} (x) < \infty$ and $\sum_{x \in X} A_{3} (x) p (x; μ)$ $< D$ ( $D$ is a constant not related to $μ$ ).

For any $μ$ , $0 < E (\frac{\partial ln p}{\partial μ})^{2} = \sum_{x \in X} (\frac{\partial ln p}{\partial μ})^{2} p (x; μ) < \infty$ .

It is easy to verify that the probability mass function ( $p (x; μ)$ ) of $X$ meets these conditions ((1)–(3)). When $Δ \to 0$ , $K \to \infty$ and $Δ K \to \infty$ , $X$ (and its distribution) approximates $Y$ (and its distribution) sufficiently well. Thus, the Fisher information of $X$ amounts to that of $Y$ and the variance of MLE from $X$ amounts to that from $Y$ . These pieces of evidence would make Equation (2.6) hold. In Equation (2.6), the left hand side (Var( ${\hat{μ}}_{EM}$ )) is discussed conditioned on the event of $0 < n_{1} < n$ and the right hand side (the inverse Fisher information) involves an expectation over $n_{1} \in {0, \dots, n}$ . The trivial case of EM algorithm (with $n_{1} \in [0, n]$ ) becomes ignorable due to the exponentially small probability of occurrence (see the beginning of Section 1).

The asymptotic variance ratio (Approach II vs. I) is as follows:

\frac{Var ({\hat{μ}}_{EM})}{Var ({\hat{μ}}_{Q})} = \frac{ϕ (w)}{Φ (w) (1 - Φ (w)) (\frac{Φ (w)}{ϕ (w)} + \frac{ϕ (w)}{1 - Φ (w)} - w)} .

(2.7)

In Figure 2, ${\hat{μ}}_{EM}$ appears to uniformly outperform ${\hat{μ}}_{Q}$ as $w$ varies and the domination increases as $w$ increases.

Figure 2:

Source: The authors.

Theorem 1. $\forall$ $w$ , ${\hat{μ}}_{EM}$ has a smaller asymptotic variance than ${\hat{μ}}_{Q}$ does. That is

\frac{ϕ (w)}{Φ (w) (1 - Φ (w)) (\frac{Φ (w)}{ϕ (w)} + \frac{ϕ (w)}{1 - Φ (w)} - w)} < 1 .

Proof. We have the following equivalent statements by simple algebraic operations. To prove the conclusion, we only need to show the following:

Φ (w) (1 - Φ (w)) (\frac{Φ (w)}{ϕ (w)} + \frac{ϕ (w)}{1 - Φ (w)} - w) > ϕ (w) .

We divide both sides by $ϕ (w)$ and have the following:

Φ (w) (1 - Φ (w)) (\frac{Φ (w)}{ϕ^{2} (w)} + \frac{1}{1 - Φ (w)} - \frac{w}{ϕ (w)}) > 1 .

We further divide both sides by ( $1 - Φ (w)$ ) and have the following:

Φ (w) (\frac{Φ (w)}{ϕ^{2} (w)} + \frac{1}{1 - Φ (w)} - \frac{w}{ϕ (w)}) > \frac{1}{1 - Φ (w)} .

That is

\frac{Φ^{2} (w)}{ϕ^{2} (w)} - \frac{w Φ (w)}{ϕ (w)} > 1 .

We multiply both sides by $ϕ^{2} (w) / Φ^{2} (w)$ and have the following:

\begin{matrix} (\frac{ϕ (w)}{Φ (w)})^{2} + w \times (\frac{ϕ (w)}{Φ (w)}) - 1 < 0 . \end{matrix}

(2.8)

That is

\begin{matrix} \frac{1}{{ln}^{(1)} (Φ (w))} - {ln}^{(1)} (Φ (w)) > w . \end{matrix}

(2.9)

The left side of (2.9) decreases as ${ln}^{(1)} (Φ (w))$ increases, so we only need to prove that ${ln}^{(1)} (Φ (w))$ is less than the root of $\frac{1}{x} - x = w$ on domain $x \in (0, \infty]$ , which is $\frac{- w + \sqrt{w^{2} + 4}}{2} = \frac{2}{w + \sqrt{w^{2} + 4}}$ .

Now, we prove the following:

\frac{Φ^{(1)} (w)}{Φ (w)} = \frac{e^{- \frac{w^{2}}{2}}}{\int_{- \infty}^{w} e^{- \frac{t^{2}}{2}} dt} < \frac{2}{w + \sqrt{w^{2} + 4}} .

This statement amounts to the following:

2 \int_{- \infty}^{w} e^{- \frac{t^{2}}{2}} dt - e^{- \frac{w^{2}}{2}} (w + \sqrt{w^{2} + 4}) > 0 for any w .

Define

g (w) = 2 \int_{- \infty}^{w} e^{- \frac{t^{2}}{2}} dt - e^{- \frac{w^{2}}{2}} (w + \sqrt{w^{2} + 4}) .

It is easy to see that, $\lim_{w \to - \infty} g (w) = 0$ .

Its derivative

\begin{array}{ll} g^{'} (w) = 2 e^{- \frac{w^{2}}{2}} + e^{- \frac{w^{2}}{2}} (w^{2} + w \sqrt{w^{2} + 4}) - e^{- \frac{w^{2}}{2}} (1 + \frac{w}{\sqrt{w^{2} + 4}}) \\ = & e^{- \frac{w^{2}}{2}} (1 + w^{2} + w \sqrt{w^{2} + 4} - \frac{w}{\sqrt{w^{2} + 4}}) \\ = & e^{- \frac{w^{2}}{2}} (1 + w^{2} + w \frac{w^{2} + 3}{\sqrt{w^{2} + 4}}) \\ = & e^{- \frac{w^{2}}{2}} \frac{(1 + w^{2}) \sqrt{w^{2} + 4} + w (w^{2} + 3)}{\sqrt{w^{2} + 4}} . \end{array}

We only check that

(1 + w^{2}) \sqrt{w^{2} + 4} > - w (w^{2} + 3) .

(2.10)

When $w \geq 0$ , it is trivial. When $w < 0$ , the square of the left side of (2.10) is

(1 + w^{2})^{2} (w^{2} + 4) = w^{6} + 6 w^{4} + 9 w^{2} + 4,

this value is greater than the square of the right side of (2.10) which is

w^{2} (w^{2} + 3)^{2} = w^{6} + 6 w^{4} + 9 w^{2} .

The proof ends. Some probability inequalities regarding the standard normal random variable are available in the literature. One of them is Mill's ratio inequality which is as following:

\sqrt{\frac{2}{π}} \frac{w}{1 + w^{2}} e^{- w^{2} / 2} \leq \Pr (| Z | > w) \leq \sqrt{\frac{2}{π}} \frac{e^{- w^{2} / 2}}{w}, (\forall w > 0) .

(2.11)

Theorem 1 gives another inequality relevant to standard normal distribution.

3 Some Desirable Properties of the EM Algorithm

Lemma 1. The iteration function $f (μ)$ (Eq.(2.5)) has a constant lower bound as $μ \to - \infty$ and is asymptotic to a line (with slope $< 1$ ) as $μ \to + \infty$ .

Proof. For any $w > 0$ , Mill's ratio inequality (2.11) implies $ϕ (w) / (1 - Φ (w)) \geq w$ . When $μ < M$ , we have the following:

f (μ) \geq \frac{1}{n} A + (1 - \frac{n_{1}}{n}) μ + \frac{n - n_{1}}{n} \times σ \times (M - μ) / σ = \frac{1}{n} A + (1 - \frac{n_{1}}{n}) M .

Thus, the first statement in Lemma 1 holds. Since $ϕ (w) / (1 - Φ (w)) \to 0$ as $w \to - \infty$ , we have

f (μ) \to \frac{1}{n} A + (1 - \frac{n_{1}}{n}) μ, as μ \to + \infty .

Thus, the second statement in Lemma 1 holds. The proof ends.

For example, under configuration

n = 50, n_{1} = 10, A = 2, M = 3 and σ = 2,

(3.1)

$f (μ)$ is plotted in the left panel of Figure 3 ( $μ$ varies along the horizontal axis) along with the reference line ( $y = μ$ ), where intersection (between $y = f (μ)$ and $y = μ$ ) surely exists due to Lemma 1.

Figure 3:

Source: The authors.

Lemma 2. The iteration function (Eq.(2.5)) monotonically increases.

Proof. We define

g (μ) = μ + σ \times \frac{ϕ ((M - μ) / σ)}{1 - Φ ((M - μ) / σ)} .

Its first-order derivative

g^{(1)} (μ) = 1 + \frac{(\frac{M - μ}{σ}) ϕ (\frac{M - μ}{σ}) Φ (\frac{μ - M}{σ}) - ϕ^{2} (\frac{M - μ}{σ})}{Φ^{2} (\frac{μ - M}{σ})} .

Eq.(2.8) gives

1 - \frac{v ϕ (v)}{Φ (v)} - \frac{ϕ^{2} (v)}{Φ^{2} (v)} > 0 (\forall v), i.e., 1 + \frac{w ϕ (w) Φ (- w) - ϕ^{2} (w)}{Φ^{2} (- w)} > 0 (\forall w) .

The proof ends.

Lemma 3. At every $w$ , the first-order derivative of $ϕ (w) / Φ (w)$ is negative and the first-order derivative of $ϕ (w) / (1 - Φ (w))$ is positive.

Proof.

(\frac{ϕ (w)}{Φ (w)})^{'} = \frac{- w ϕ (w) Φ (w) - ϕ (w)^{2}}{Φ^{2} (w)} = \frac{- ϕ (w) (w Φ (w) + ϕ (w))}{Φ^{2} (w)} .

When $w \geq 0$ , Lemma 3 obviously holds. When $w < 0$ , we let $u : = - w > 0$ . By Mill's ratio inequality (Equation [2.11]), we have $2 (1 - Φ (u)) \leq (\sqrt{2 / π} / u) \times e^{- u^{2} / 2}$ , i.e., $- u (1 - Φ (u)) + ϕ (u) > 0$ . Thus, $w Φ (w) + ϕ (w) > 0$ holds for every $w < 0$ to make the first statement in Lemma 3 hold. Since $ϕ (w) / (1 - Φ (w)) = ϕ (- w) / Φ (- w)$ , the second statement in Lemma 3 also holds.

Theorem 2. The EM iteration (Equation [2.5]) surely converges and the resultant ${\hat{μ}}_{EM}$ equals the unique MLE which is the root of Equation (2.4).

Proof.

Existence of a solution: By Lemmas 1 and 2, as $μ \to - \infty$ , $f (μ) \to$ some constant. So $h (μ) : = f (μ) - μ \to + \infty$ . On the other hand, as $μ \to + \infty$ , $f (μ)$ is asymptotic to a line with slope in (0,1) (assuming $0 < n_{1} < n$ ). Hence, $h (μ) \to - \infty$ in this case. Thus, there must be some $μ$ for which $h (μ) = 0$ by continuity. A procedural illustration is given in the right panel of Figure 3, the iteration (see Section 1, under the foregoing configuration [3.1]) starts at ${\hat{μ}}_{Q}$ and appears to converge to the intersection (a fixed point for $f (μ) = μ$ ) which is promised by Lemma 1. The fact that $f (μ)$ is continuous and monotonically increases (Lemma 2) satisfies the sufficient condition for convergence, $^{[4, Ch . 2.1]}$ which can be verified by observing that the reference line ( $y = μ$ , slants downwards from right to left) autonomously pulls the zigzag iteration path towards the intersection. Since the iteration solution ( $f (μ) = μ$ ) is a root for Equation (2.4), MLE arises from such intersections.

Uniqueness: For any $w$ , the first-order derivative of $ϕ (w) / Φ (w)$ is negative and the first-order derivative of $ϕ (w) / (1 - Φ (w))$ is positive (Lemma 3). Thus, $\forall$ $μ$ , $f^{(1)} (μ) < (1 - n_{1} / n) < 1$ . Since existence of multiple intersections (between functions: $y = f (μ)$ and $y = μ$ ) causes $f^{(1)} (μ^{*}) = 1$ (at certain $μ^{*}$ ) due to Lagrange's mean value theorem, EM iteration surely finds the unique MLE. The proof ends.

4 Discussion

For i.i.d. normal variables under right-censoring, the present work investigates MLE from some new perspectives. Since the hierarchical MLE (Approach I) does not take into account the detailed individual observations and loses information compared to the direct MLE (Approach II), we are motivated to quantify the relative information loss by using the asymptotic variance ratio (Equation (2.7)) which is a monotonically decreasing function of $w = (M - μ) / σ$ and free of sample size ( $n$ and/or $n_{1}$ ). Thus, when the prior information on $μ$ implies that the variance ratio is close to one ( $μ$ is much larger than $M$ ), the hierarchical MLE estimation may be employed due to its simplicity. In the future, the multi-dimensional-parameter scenarios and/or more complicated models may need further investigation.

Footnotes

Acknowledgement

The authors are very grateful to Editor-in-Chief (Uttam Bandyopadhyay) and two anonymous referees whose insightful comments have greatly improved the presentation of the materials and motivated us to investigate more aspects of this subject.

Declaration of Conflicting Interests

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

References

Aitkin

Wilson

GT.

Mixture models, outliers and the EM algorithm. Technometrics 1980; 22: 325–331.

Casella

Berger

RL.

Statistical inference. 2nd ed. Pacific Grove, CA: Duxbury Press, 2002.

Mao

Zhou

Probability theory and mathematical statistics. 2nd ed. Beijing Advanced Education Press, 2000.

Chen

Fan

Implicit function theorem. 1st ed. Lanzhou: University Press, 1986.