Alternative evaluation of the Poisson type distribution

Abstract

It is shown that for a small sample size, when the maximum likelihood estimators lose its asymptotic properties, a more accurate point and interval estimation of the Poisson distribution parameter requires a direct investigation of the properties of the normalized likelihood function. Alternative point and interval estimates of the distribution parameter are obtained.

Keywords

Normalized likelihood function Poisson type distribution point estimate confidence interval

1. Introduction

The theory of the Poisson distribution is widely known and fully represented in the educational literature. With a small sample size the estimations of the observed parameters obtained by the maximum likelihood method (MLM) can lose their unbiasedness and efficiency properties, so it can complicate the qualitative application of statistical methods and solving practical problems. In the present research it is proposed to use the method of direct investigation of the likelihood surface for more accurate point and interval estimations of the Poisson distribution parameter and to calculate the Expected value (EV) of the normalized likelihood function (NLF) as an alternative to the solution of the MLM system. The object of this paper is to give a quantitative and qualitative description of the error of such an estimation of the Poisson-type distribution parameter with a small sample size. To illustrate the applicability and simplicity of the proposed method, one example is considered, taken from the classical literature on statistics. A numerical comparison of the estimates obtained by the new method with previously known values is carried out.

2. Formulation of the problem

We consider the random variable (RV), having the Poisson-type distribution with a parameter $\lambda>0$ , $X\sim Po(\lambda)$ , with a probability function (PF) Eq. (1):

$\displaystyle p_{X}=P(X=x)=e^{-\lambda}\cdot\frac{\lambda^{x}}{x!},x=0,1,...,$ (1)

Then we say Patil et al. (1968, p. 14) that the random variable $Y=aX+b$ has the Poisson-type distribution with parameters $\lambda,a,b$ , where $0\neq a$ and $b$ – are constant values. PF of the RV have the form Eq. (2):

$\displaystyle p_{Y}=P(Y=y)=e^{-\lambda}\cdot\frac{\lambda^{x}}{x!},y=b,a+b,2a+% b,...,$ (2)

where $x=\frac{y-b}{a},0<\lambda<\infty,0\neq a,-\infty<b<\infty$ .

The Expected value (EV) and the variance of the random variables $X, Y$ are:

$\displaystyle EX=DX=\lambda,EY=a\lambda+b,DY=a^{2}\lambda.$ (3)

The problem of point and interval estimation of the parameter $\lambda$ of distribution Eq. (2) for a limited $n$ -volume sample of the independent observations of the RV $Y:\{y_{1},\ldots,y_{n}\}$ is important for practice.

3. Method of estimation

Consider $n$ series of observations during a fixed period, in each of which it was recorded $y_{i}$ events, $i=1,\ldots,n$ . We denote the possible estimate $\lambda$ by $\hat{\lambda}$ . The likelihood function (LF) by the results of independent observations has the Eq. (4):

$\displaystyle L(\hat{\lambda})=\prod_{i=1}^{n}p_{y_{i}}=e^{-n\hat{\lambda}}% \cdot\hat{\lambda}^{N_{X}}\cdot\prod_{i=1}^{n}(x_{i}!)^{-1},$ (4)

where

$\displaystyle N_{X}=\sum_{i=1}^{n}x_{i}=\sum_{i=1}^{n}\left(\frac{y_{i}-b}{a}% \right)=\frac{N_{Y}-nb}{a},N_{Y}=\sum_{i=1}^{n}y_{i}.$ (5)

In accordance with the MLM, for the estimation $\hat{\lambda}=\hat{\lambda}_{\textit{MLM}}$ of the parameter $\lambda$ and its variance the following Eq. (6) are taken place:

$\displaystyle\hat{\lambda}_{\textit{MLM}}=\frac{N_{X}}{n}=\frac{N_{Y}-nb}{na},% \sigma_{\hat{\lambda}_{\textit{MlM}}}^{2}=\frac{N_{X}}{n^{2}}=\frac{N_{Y}-nb}{% n^{2}a}=\frac{\hat{\lambda}_{\textit{MLM}}}{n}$ (6)

To carry out the valuation of the LF we define a normalizing factor $C$ . Taking into account the known identity Eq. (7)

$\displaystyle\int\limits_{0}^{\infty}x^{n}e^{-\mu x}dx=n!\frac{1}{\mu^{n+1}},% \mu>0,$ (7)

we have Eq. (8),

$\displaystyle C=\left\{{\int\limits_{0}^{\infty}{L(\hat{\lambda})d\hat{\lambda% }}}\right\}^{-1}=\prod\limits_{i=1}^{n}x_{i}!\cdot\left\{{\int\limits_{0}^{% \infty}{\hat{\lambda}^{N_{X}}e^{-n\hat{\lambda}}d\hat{\lambda}}}\right\}^{-1}=% \prod\limits_{i=1}^{n}x_{i}!\cdot\frac{n^{N_{X}+1}}{N_{X}!}.$ (8)

Then NLF has the form Eq. (9):

$\displaystyle l(\hat{\lambda})=\frac{n^{N_{X}+1}}{N_{X}!}\hat{\lambda}^{N_{X}}% e^{-n\hat{\lambda}}.$ (9)

The Eq. (9) can be considered as the distribution density of estimates $\hat{\lambda}$ . It depends on the number of observations n and the total number $N_{Y}({N_{X}})$ of occurrences of the event during observations.

Let us investigate some properties of the Eq. (9) $l(\hat{\lambda})$ . The first Eq. (10) and the second Eq. (11) derivatives of this function are

$\displaystyle\frac{dl(\hat{\lambda})}{d\hat{\lambda}}=\frac{n^{N_{X}+1}}{N_{X}% !}\hat{\lambda}^{N_{X}-1}e^{-n\hat{\lambda}}({N_{X}-n\hat{\lambda}}),$ (10) $\displaystyle\frac{d^{2}l(\hat{\lambda})}{d\hat{\lambda}^{2}}=\frac{n^{N_{X}+1% }}{N_{X}!}\hat{\lambda}^{N_{X}-2}e^{-n\hat{\lambda}}\cdot({n^{2}\hat{\lambda}^% {2}-2nN_{X}\hat{\lambda}+N_{X}({N_{X}-1})}).$ (11)

After equating first and the second derivatives with zero we obtain extremums $\hat{\lambda}_{E}$ Eq. (12) and inflection points $\hat{\lambda}_{I}$ Eqs (13) and (14):

$\displaystyle\hat{\lambda}_{E}=\frac{N_{X}}{n}=\hat{\lambda}_{\textit{MLM}}$ (12) $\displaystyle\hat{\lambda}_{I}^{(1)}=\frac{N_{X}-\sqrt{N_{X}}}{n}=\hat{\lambda% }_{\textit{MLM}}-\sigma_{\hat{\lambda}_{\textit{MLM}}},$ (13) $\displaystyle\hat{\lambda}_{I}^{(2)}=\frac{N_{X}+\sqrt{N_{X}}}{n}=\hat{\lambda% }_{\textit{MLM}}+\sigma_{\hat{\lambda}_{\textit{MLM}}}.$ (14)

Then it is obvious that $\frac{d^{2}l(\hat{\lambda})}{d\hat{\lambda}^{2}}<0$ and the function graph $l(\hat{\lambda})$ is also convex upward on the interval $(\hat{\lambda}_{I}^{(1)},\hat{\lambda}_{I}^{(2)})$ and within it reaches a maximum at the point $\hat{\lambda}=\hat{\lambda}_{E}$ . On intervals $(0,{\hat{\lambda}_{I}^{(1)}})$ and $({\hat{\lambda}_{I}^{(2)}},\infty)$ , we have $\frac{d^{2}l(\hat{\lambda})}{d\hat{\lambda}^{2}}>0$ and the function graph has a concavity down.

Despite the symmetry of the values of the inflection points relative to $\hat{\lambda}=\hat{\lambda}_{E}=\hat{\lambda}_{\textit{MLM}}$ the value of the NLF at these points are different and its graph is asymmetric with a limited sample size $n$ . Indeed, if we consider the limiting case $N_{X}=1$ or $N_{Y}=a+nb$ , we have Eq. (15):

$\displaystyle\hat{\lambda}_{I}^{(1)}=0,\hat{\lambda}_{E}=\frac{1}{n},\hat{% \lambda}_{I}^{(2)}=\frac{2}{n},l(0)=0,l\left(\frac{1}{n}\right)=ne^{-1},l\left% (\frac{2}{n}\right)=2ne^{-2}.$ (15)

From the asymmetry of NLF follows a difference between the mode $\hat{\lambda}=\hat{\lambda}_{E}=\hat{\lambda}_{\textit{MLM}}$ that corresponds to the MLM-estimate and the expected value density of distribution of the estimates $\hat{\lambda}$ .

We define the expected value of the estimates $\hat{\lambda}$ and its variance, and also more precisely the variance of the MLM-estimate $\hat{\lambda}=\hat{\lambda}_{\textit{MLM}}$ Eq. (6). After a series of transformations, taking into account Eqs (7) and (9) we obtain Eqs (16)–(18):

$\displaystyle E\hat{\lambda}=\int\limits_{0}^{\infty}{\hat{\lambda}l(\hat{% \lambda})}d\hat{\lambda}=\frac{N_{X}+1}{n},$ (16) $\displaystyle D\hat{\lambda}=\int\limits_{0}^{\infty}{({\hat{\lambda}-E\hat{% \lambda}})^{2}l(\hat{\lambda})}d\hat{\lambda}=\frac{N_{X}+1}{n^{2}}=\frac{E% \hat{\lambda}}{n},$ (17) $\displaystyle\bar{\sigma}_{\hat{\lambda}_{\textit{MLM}}}^{2}=D\hat{\lambda}_{% \textit{MLM}}=\int\limits_{0}^{\infty}{({\hat{\lambda}-\hat{\lambda}_{\textit{% MLM}}})^{2}l(\hat{\lambda})d\hat{\lambda}}=\frac{N_{X}+2}{n^{2}}.$ (18)

Comparing Eq. (6) with Eq. (16) and Eq. (18) with Eq. (17), we can see that the MLM-estimate has a negative bias relative to the expected value $\hat{\lambda}_{\textit{MLM}}-E\hat{\lambda}=-n^{-1}$ , and its specified variance exceeds the variance from the EV on the squared bias $\bar{\sigma}_{\hat{\lambda}_{\textit{MLM}}}^{2}-D\hat{\lambda}=n^{-2}$ . The comparisons show that the EV Eq. (16) NLF is the best alternative to the MLM-estimate Eq. (6) for the parameter $\lambda$ , which is especially noticeable for a limited sample size.

4. Parameter estimations

Based on NLF as the density of distribution of estimates, it is possible not only to construct a point estimation of the parameter $\lambda$ , but also a confidence interval $[{\hat{\lambda}_{1},\hat{\lambda}_{2}}]$ for a given confidence probability $0.5\leqslant\alpha<1$ . For this it suffices to use the relations Eq. (19):

$\displaystyle\int\limits_{0}^{\hat{\lambda}_{1}}l(\hat{\lambda})d\hat{\lambda}% =1-\alpha_{1},\int\limits_{0}^{\hat{\lambda}_{2}}l(\hat{\lambda})d\hat{\lambda% }=\alpha_{2},$ (19)

where $0.5\leqslant\alpha_{1},\alpha_{2}<1,\alpha_{1}+\alpha_{2}=1+\alpha$ . In the symmetric case, when the quantities $\alpha_{1},\alpha_{2}$ are assumed to be equal $\alpha_{1}=\alpha_{2}=P$ , we have $\alpha=2P-1$ , and with a given confidence level $\alpha$ , it is easy to determine the value $P=\frac{1+\alpha}{2}$ .

Substituting Eq. (9) into the left-hand sides of the equalities and taking into account the known Eq. (20)

$\displaystyle\int\limits_{0}^{u}x^{n}e^{-\mu x}dx=\frac{n!}{\mu^{n+1}}-e^{-u% \mu}\sum\limits_{i=0}^{n}{\frac{n!}{i!}}\frac{u^{i}}{\mu^{n-i+1}}=\frac{n!}{% \mu^{n+1}}\cdot\left\{{1-e^{-u\mu}\sum\limits_{i=0}^{n}{\frac{({u\mu})^{i}}{i!% }}}\right\},\mu>0,u>0,$ (20)

after the transformations, we obtain equations for computing the limits of the confidence interval $[{\hat{\lambda}_{1},\hat{\lambda}_{2}}]$ of the parameter $\lambda$ of the Poisson type distributions Eq. (21):

$\displaystyle e^{-n\hat{\lambda}_{1}}\sum\limits_{i=0}^{N_{X}}{\frac{({n\hat{% \lambda}_{1}})^{i}}{i!}}=\alpha_{1},e^{-n\hat{\lambda}_{2}}\sum\limits_{i=0}^{% N_{X}}{\frac{({n\hat{\lambda}_{2}})^{i}}{i!}}=1-\alpha_{2}.$ (21)

Since the left-hand sides of the Eq. (21) correspond to the Poisson distribution function with the parameters $n\hat{\lambda}_{1}$ and $n\hat{\lambda}_{2}$ respectively, the confidence interval boundaries $[{\hat{\lambda}_{1},\hat{\lambda}_{2}}]$ can be determined numerically from Eq. (21), or by the numerical integration of NLF Eq. (9), or by using tabulated tables of the Poisson distribution function.

Taking into account the known relations between Poisson and $\chi^{2}$ – distributions Bolshev and Smirnov (1983, p. 70) Eq. (22),

$\displaystyle e^{-\lambda}\sum\limits_{i=0}^{m}{\frac{\lambda^{i}}{i!}}=P({2% \lambda,2m+2}),$ (22)

Equation (21) can be rewritten as Eq. (23),

$\displaystyle P({2n\hat{\lambda}_{1},2N_{X}+2})=\alpha_{1},P({2n\hat{\lambda}_% {2},2N_{X}+2})=1-\alpha_{2},$ (23)

or in the Eq. (24),

$\displaystyle\hat{\lambda}_{1}=\frac{1}{2n}\chi^{2}({100\alpha_{1}\%;2N_{X}+2}% ),\hat{\lambda}_{2}=\frac{1}{2n}\chi^{2}({100({1-\alpha_{2}})\%;2N_{X}+2})$ (24)

where $P({x,\nu})$ is the probability integral, and $\chi^{2}({\alpha,\nu})$ is the $\alpha$ – percentage point of the $\chi^{2}$ – distribution with $\nu$ degrees of freedom, defined as Eq. (4):

$\displaystyle P({x,\nu})=\frac{1}{2^{\nu/2}\Gamma({\nu/2})}\int\limits_{x}^{% \infty}u^{\nu/2-1}e^{-u/2}du,\nu=1,2,...,$ (25) $\displaystyle\alpha=\frac{1}{2^{\nu/2}\Gamma({\nu/2})}\int\limits_{0}^{\chi^{2% }({\alpha,\nu})}u^{\nu/2-1}e^{-u/2}du,\Gamma(x)=\int\limits_{0}^{\infty}t^{x-1% }e^{-t}dt,x>0.$

Thus, the $[{\hat{\lambda}_{1},\hat{\lambda}_{2}}]$ confidence interval boundaries can also be found by interpolation (in terms of the number of degrees of freedom) in the tables of percentage points of $\chi^{2}$ – distribution.

We note that the estimate of the lower confidence boundary $\hat{\lambda}_{1}$ differs from the similar formula given in Bolshev and Smirnov (1983, p. 70). In the notation used, the corresponding Eq. (24) would have the form

$\displaystyle\hat{\lambda}_{1}=\frac{1}{2n}\chi^{2}({100\alpha_{1}\%;2N_{X}}),% \hat{\lambda}_{2}=\frac{1}{2n}\chi^{2}({100({1-\alpha_{2}})\%;2N_{X}+2})$ (26)

This is due to the fact that the basis of the latter rations was founded on the general approaches of the theory of interval estimates. Whereas the interval estimates Eq. (24) are obtained from the NLF. The recalculation of the existing confidence interval tables for the Poisson parameter is not required. It is enough just to make a correction, shifting the line one position down to find the lower bound of the interval.

Thus, the confidence intervals determined by the rations Eq. (26) are not the narrowest. Their determination with a confidence probability $(1-\alpha)$ can be based on the use of NLF Eq. (9), which leads to Eq. (24).

5. Example

The enterprise is conducting the quality control of the technical products. During the working period there were performed 8 series of independent tests of the technical device functioning and the following number of failures were recorded: $\{{y_{1},...,y_{n}}\}=\{{1,0,1,2,0,1,3,2}\}$ . Assuming that the probability of failure is described by Poisson’s law, it is necessary to construct the density of the distribution of the parameter $\lambda$ estimates, to find its point estimate and the confidence interval boundaries for the failure rate $\lambda$ of the output products with confidence probability $\alpha=$ 0.98.

By virtue of the adopted notation, we have obtained the experimental results $a=$ 1, $b=$ 0, $n=$ 8, $N_{Y}=N_{X}=$ 10. We will assume $\alpha_{1}=\alpha_{2}=P=$ 0.99. By virtue of Eqs (6), (18), (16), (17), (24), (26) we obtain

$\displaystyle\hat{\lambda}_{\textit{MLM}}=\frac{10}{8},\quad{\sigma_{\hat{% \lambda}_{\textit{MLM}}}}^{2}=\frac{10}{64},\quad\bar{\sigma}_{\hat{\lambda}_{% \textit{MLM}}}^{2}=\frac{12}{64},\quad E\hat{\lambda}=\frac{11}{8},\quad D\hat% {\lambda}=\frac{11}{64}.$

According to Table 2.2a (Bolshev & Smirnov, 1983, p. 166-167) of the percentage points of the $\chi^{2}$ – distribution, in view of Eq. (24) we have

$\displaystyle n\hat{\lambda}_{1}=\frac{1}{2}\chi^{2}({99\%;22})=\frac{9.542}{2% }=4.771,\quad n\hat{\lambda}_{2}=\frac{1}{2}\chi^{2}({1\%;22})=\frac{40.289}{2% }=20.1445.$

Then, by virtue of Eq. (26), we obtain

$\displaystyle n\hat{\lambda}_{1}=\frac{1}{2}\chi^{2}({99\%;20})=\frac{8.26}{2}% =4.13,\quad n\hat{\lambda}_{2}=\frac{1}{2}\chi^{2}({1\%;22})=\frac{40.289}{2}=% 20.1445.$

In the Table 5.4a (Bolshev & Smirnov, 1983, p. 306) of the confidence limits for the Poisson parameter, exactly the last boundaries [4.13, 20.14] are given. The left border $n\hat{\lambda}_{1}=$ 4.77 is a line below. As a result, we obtain a confidence interval $0.5963<\lambda<2.518$ with the level of confidence $\alpha=2P-1=$ 0.98.

6. Conclusions

So, the paper shows that for a limited sample size, when MLM estimates lose their asymptotic properties, for more precisepoint and interval estimation of the parameter of Poisson type distributions we should use a direct investigation of the graph and the properties of the NLF. The alternative point and interval estimates of the distribution parameter are obtained. For greater clarity of applicability of the proposed method, one example taken from the classical literature on statistics is considered. A numerical comparison of the estimates obtained by the new method with previously known values is carried out. Simplicity of implementation of the above method allows you to include it in software packages on statistics. The detailed estimation method can also be extended to other distributions.

References

Patil,

G.P.

Joshi,

S.W.

, & Rao,

C.R.

A dictionary and Bibliography of discrete distributions. Oliver and Boyd Ltd., Edinburgh, 1968.

Bolshev,

L.N.

, & Smirnov,

N.V.

Tables of Mathematical Statistics (in Russian). Nauka, Moscow, 1983.