Completeness (statistics)
This article needs additional citations for verification. (August 2009) |
In statistics, completeness is a property of a statistic in relation to a parameterised model for a set of observed data.
A complete statistic T is one for which any proposed distribution on the domain of T is predicted by one or more
Put another way: assume that we have an identifiable model space parameterised by , and a statistic (which is effectively just a function of one or more i.i.d. random variables drawn from the model). Then consider the map which takes each distribution on model parameter to its induced distribution on statistic . The statistic is said to be complete when is surjective, and sufficient when is injective.
Definition
Consider a random variable X whose probability distribution belongs to a parametric model Pθ parametrized by θ.
Say T is a statistic; that is, the composition of a measurable function with a random sample X1,...,Xn.
The statistic T is said to be complete for the distribution of X if, for every measurable function g,[1]
The statistic T is said to be boundedly complete for the distribution of X if this implication holds for every measurable function g that is also bounded.
Example 1: Bernoulli model
The Bernoulli model admits a complete statistic.
Observe also that neither p nor 1 − p can be 0. Hence if and only if:
On denoting p/(1 − p) by r, one gets:
First, observe that the range of r is the
It is important to notice that the result that all coefficients must be 0 was obtained because of the range of r. Had the parameter space been finite and with a number of elements less than or equal to n, it might be possible to solve the linear equations in g(t) obtained by substituting the values of r and get solutions different from 0. For example, if n = 1 and the parameter space is {0.5}, a single observation and a single parameter value, T is not complete. Observe that, with the definition:
then, E(g(T)) = 0 although g(t) is not 0 for t = 0 nor for t = 1.
Example 2: Sum of normals
This example will show that, in a sample X1, X2 of size 2 from a
is a complete statistic for θ.
To show this, it is sufficient to demonstrate that there is no non-zero function such that the expectation of
remains zero regardless of the value of θ.
That fact may be seen as follows. The probability distribution of X1 + X2 is normal with expectation 2θ and variance 2. Its probability density function in is therefore proportional to
The expectation of g above would therefore be a constant times
A bit of algebra reduces this to
where k(θ) is nowhere zero and
As a function of θ this is a two-sided Laplace transform of h(X), and cannot be identically zero unless h(x) is zero almost everywhere.[3] The exponential is not zero, so this can only happen if g(x) is zero almost everywhere.
Relation to sufficient statistics
For some parametric families, a complete sufficient statistic does not exist (for example, see Galili and Meilijson 2016 [4]).
For example, if you take a sample sized n > 2 from a N(θ,θ2) distribution, then is a minimal sufficient statistic and is a function of any other minimal sufficient statistic, but has an expectation of 0 for all θ, so there cannot be a complete statistic.
If there is a minimal sufficient statistic then any complete sufficient statistic is also minimal sufficient. But there are pathological cases where a
Importance of completeness
The notion of completeness has many applications in statistics, particularly in the following two theorems of mathematical statistics.
Lehmann–Scheffé theorem
Completeness occurs in the Lehmann–Scheffé theorem,[5] which states that if a statistic that is unbiased, complete and
Examples exists that when the minimal sufficient statistic is not complete then several alternative statistics exist for unbiased estimation of θ, while some of them have lower variance than others.[6]
See also minimum-variance unbiased estimator.
Basu's theorem
Bounded completeness occurs in
Bahadur's theorem
Bounded completeness also occurs in
Notes
- ^ Young, G. A. and Smith, R. L. (2005). Essentials of Statistical Inference. (p. 94). Cambridge University Press.
- ^ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 285–286). Duxbury Press.
- ^ Orloff, Jeremy. "Uniqueness of Laplace Transform" (PDF).
- PMID 27499547.
- ISBN 978-0534243128.
- PMID 27499547.
- ^ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 287). Duxbury Press.
- ^ "Statistical Inference Lecture Notes" (PDF). July 7, 2022.
References
This article includes a list of general references, but it lacks sufficient corresponding inline citations. (February 2012) |