MathIsimple

Sufficient & Complete Statistics Formulas

Comprehensive reference for sufficient statistics, complete statistics, and their applications in optimal statistical inference.

Learn ConceptsPractice Problems

Basic Concepts

Sufficient Statistics Definitions
Fundamentals
definition:
P(X̃ = x̃ | T = t; θ) is independent of θ
interpretation:
T(X̃) contains all information about θ in the sample
sufficiency:
Given T, sample distribution doesn't depend on θ
Complete Statistics Definitions
Fundamentals
definition:
E_θ[φ(T)] = 0 ∀θ ∈ Θ ⟹ P_θ(φ(T) = 0) = 1 ∀θ ∈ Θ
interpretation:
Only zero function has zero expectation for all θ
uniqueness:
Ensures uniqueness of unbiased estimators
UMVUE Definition
Fundamentals
definition:
Var_θ(θ̂*) ≤ Var_θ(θ̂) ∀θ ∈ Θ, ∀ unbiased θ̂
property:
Uniformly Minimum Variance Unbiased Estimator
optimality:
Best unbiased estimator in variance sense

Key Theorems

Fisher-Neyman Factorization Theorem
Core Theorems
Statement:
T(X̃) is sufficient for θ ⟺ joint density can be factored
Formula:
p(x̃;θ) = g(T(x̃);θ) × h(x̃)
components:
g(T(x̃);θ): depends on data only through T and on parameter θ
h(x̃): depends on data but is independent of parameter θ
applications:
0: Normal: T = (∑Xᵢ, ∑Xᵢ²) sufficient for (μ,σ²)
1: Poisson: T = ∑Xᵢ sufficient for λ
2: Uniform U(0,θ): T = X₍ₙ₎ sufficient for θ
components:
g(T(x̃);θ): depends on data only through T and on parameter θ
h(x̃): depends on data but is independent of parameter θ
applications:
0: Normal: T = (∑Xᵢ, ∑Xᵢ²) sufficient for (μ,σ²)
1: Poisson: T = ∑Xᵢ sufficient for λ
2: Uniform U(0,θ): T = X₍ₙ₎ sufficient for θ
Rao-Blackwell Theorem
Core Theorems
Statement:
Sufficient statistics improve unbiased estimators
Formula:
ĝ(T) = E[φ(X̃)|T] where T is sufficient
properties:
unbiasedness: E[ĝ(T)] = E[φ(X̃)] = g(θ)
varianceReduction: Var(ĝ(T)) ≤ Var(φ(X̃))
equality: Equality ⟺ φ(X̃) = ĝ(T) a.s.
implication:
Optimal estimators are functions of sufficient statistics
properties:
unbiasedness: E[ĝ(T)] = E[φ(X̃)] = g(θ)
varianceReduction: Var(ĝ(T)) ≤ Var(φ(X̃))
equality: Equality ⟺ φ(X̃) = ĝ(T) a.s.
implication:
Optimal estimators are functions of sufficient statistics
Lehmann-Scheffé Theorem
Core Theorems
Statement:
Sufficient complete statistics yield unique UMVUE
Formula:
ĝ = E[φ(X̃)|S] is unique UMVUE if S is sufficient complete
conditions:
sufficient: S contains all parameter information
complete: S ensures uniqueness of unbiased functions
unbiased: φ(X̃) is any unbiased estimator of g(θ)
corollary:
If h(S) is unbiased function of sufficient complete S, then h(S) is unique UMVUE
conditions:
sufficient: S contains all parameter information
complete: S ensures uniqueness of unbiased functions
unbiased: φ(X̃) is any unbiased estimator of g(θ)
corollary:
If h(S) is unbiased function of sufficient complete S, then h(S) is unique UMVUE
Basu's Theorem
Independence
Statement:
Sufficient complete statistics are independent of ancillary statistics
Formula:
T ⊥ V where T is sufficient complete, V is ancillary
definitions:
ancillary: Distribution of V doesn't depend on θ
independence: T and V are independent ∀θ ∈ Θ
application: Useful for proving independence in specific problems
example:
Normal: (X̄,S²) ⊥ sample skewness
definitions:
ancillary: Distribution of V doesn't depend on θ
independence: T and V are independent ∀θ ∈ Θ
application: Useful for proving independence in specific problems
example:
Normal: (X̄,S²) ⊥ sample skewness

Distribution Examples

Normal Distribution N(μ,σ²)
Examples
Joint Density/PMF:
p(x̃;μ,σ²) = (2πσ²)^(-n/2) exp{-∑(xᵢ-μ)²/(2σ²)}
Factorization:
g(·): (2πσ²)^(-n/2) exp{μ∑xᵢ/σ² - nμ²/(2σ²) - ∑xᵢ²/(2σ²)}
h(·): 1
sufficient(·): T = (∑Xᵢ, ∑Xᵢ²)
Completeness:
T is sufficient complete for (μ,σ²)
UMVUE:
μ: X̄ = (∑Xᵢ)/n
σ²: S² = ∑(Xᵢ-X̄)²/(n-1)
Poisson Distribution P(λ)
Examples
Uniform Distribution U(0,θ)
Examples
Joint Density/PMF:
p(x̃;θ) = θ^(-n) I{0 ≤ x₍₁₎ ≤ x₍ₙ₎ ≤ θ}
Factorization:
g(·): θ^(-n) I{T ≤ θ} where T = X₍ₙ₎
h(·): I{0 ≤ x₍₁₎ ≤ T}
sufficient(·): T = X₍ₙ₎ (maximum order statistic)
Completeness:
T has density p_T(t;θ) = nt^(n-1)/θ^n, is complete
UMVUE:
θ: θ̂ = ((n+1)/n)X₍ₙ₎ (unbiased version of MLE)
Bernoulli Distribution B(1,p)
Examples

Practical Applications

Constructing UMVUE
Applications
step1:
Identify sufficient complete statistic S using factorization theorem
step2:
Find any unbiased estimator φ(X̃) for parameter g(θ)
step3:
Compute ĝ = E[φ(X̃)|S] (conditional expectation)
step4:
ĝ is the unique UMVUE of g(θ) by Lehmann-Scheffé theorem
Key Formula:
UMVUE = E[unbiased estimator | sufficient complete statistic]
Variance Improvement
Applications
raoblackwell:
If T sufficient, φ unbiased: Var(E[φ|T]) ≤ Var(φ)
improvement:
Var(improved) ≤ Var(original)
when Equality:
Equality ⟺ original estimator is already function of T
application:
Always condition on sufficient statistics to reduce variance
Checking Completeness
Verification
method:
If E_θ[φ(T)] = 0 ∀θ, show P_θ(φ(T) = 0) = 1 ∀θ
techniques:
exponentialFamily: Natural exponential families are typically complete
polynomialArgument: For discrete: polynomial in θ must have all zero coefficients
differentiationTrick: For continuous: differentiate expectation w.r.t. θ
non Complete:
Example: X ~ N(0,σ²), take φ(X) = X has E[φ] = 0 but P(φ = 0) = 0

Important Inequalities

Rao-Blackwell Inequality
Inequalities
condition:
T is sufficient for θ, φ is unbiased for g(θ)
equality:
Holds ⟺ φ(X̃) = E[φ(X̃)|T] almost surely
interpretation:
Conditioning on sufficient statistics never increases variance
Information Inequality (via Sufficiency)
Inequalities
Statement:
Sufficient statistics capture all Fisher information
Formula:
I_T(θ) = I_{X̃}(θ) where T is sufficient
implication:
No information loss when using sufficient statistics
connection:
Links to Cramér-Rao bound: Var(T̂) ≥ 1/I_T(θ)
implication:
No information loss when using sufficient statistics
connection:
Links to Cramér-Rao bound: Var(T̂) ≥ 1/I_T(θ)
Back to Formulas Hub
Learn ConceptsPractice Problems