<Sufficient Statistics>
Sufficiency : T(X1, X2,..., Xn) is said to have 'sufficiency' for the parameter, if the conditional of X1, X2,...Xn given T=t does not rely on the value of the parameter. And the T(X1,X2,X3,...,Xn) is called 'Sufficient Statistics.'
That is, by gaining T we need no longer to find knowledge about the parameter from random samples.
<Factorization Theorem - Sufficient Statistics>
Figuring out whether the conditional of X1, X2,...Xn given T=t does not rely on the parameter or not is complicated sometimes.
By using the "Factorization Theorem", you can easily find the sufficient statistics' of distributions.
If the joint pdf or pmf can be divided by the (function of parameter and sufficient statistics) and (the function of X1,X2,X3,...,Xn.), there exists sufficient statistics of the distributions.
<Proof for factorization Theorem>
Necessary and Sufficient conditions for sufficient statistics is f(X1,X2,X3,...,Xn;θ) = g(s;θ)h(X1,X2,X3,...Xn)
I will prove separately how do they make sense.
'S is sufficient statistic' => f(X1,X2,X3,...,Xn;θ) can be expressed as g(s;θ)h(X1,X2,X3,...Xn)
f(X1,X2,X3,...,Xn;θ) can be expressed as g(s;θ)h(X1,X2,X3,...Xn) => 'S is sufficient statistic'
<Examples for finding Sufficient Statistics by using Factorization Theorem>
Bernoulli with parameter p
Gamma with parameter alpha and beta
Uniform distribution with parameter θ1, θ2
Normal distribution with parameters Mean and Variance.
Exponential with a parameter λ
Poisson with a parameter λ
Geometric with a parameter p
Order Statistics always be sufficient statistics for every distribution
<Properties of Sufficient Statistics>
Rao Blackwell Theorem is one of the core theorems in Sufficient statistics and complete sufficient statistics.
It means that If the expectation value of the unbiased estimator of the function of the parameter given sufficient statistics could be another sufficient statistics that has less value of variance than the original unbiased estimator. This theorem is needed to find Minimum variance unbiased estimator.
[Statistics] - Exponential family
