Distribution Families

Common Distribution Families & Properties

Master probability distribution families, exponential family theory, and their applications in statistical inference and data modeling

6-8 hours12 topicsIntermediate level

Learning Objectives

By the end of this course, you will be able to:

Master fundamental distribution families: binomial, Poisson, normal, uniform, exponential
Understand advanced distributions: gamma, chi-square, t-distribution, F-distribution, beta
Learn exponential family theory and its applications in statistical inference
Explore distribution relationships and transformation properties
Apply distribution knowledge to real-world statistical problems
Recognize when to use specific distributions in statistical modeling

Fundamental Distribution Families

Core probability distributions essential for statistical modeling and inference

Binomial Distribution B(n,p)

Models the number of successes in n independent Bernoulli trials

Probability Function:

P(X=k) = C(n,k)p^k(1-p)^(n-k)

Parameters: n ∈ {1,2,3,...}, p ∈ (0,1)

Key Properties:

Mean: E(X) = np
Variance: Var(X) = np(1-p)
Discrete distribution
Belongs to exponential family

Applications:

Quality control: defective products in batch
Medical trials: success rate of treatments
Marketing: response rates to campaigns
Manufacturing: pass/fail testing

Poisson Distribution P(λ)

Models the number of rare events occurring in a fixed interval

Probability Function:

P(X=k) = (λ^k e^(-λ)) / k!

Parameters: λ > 0 (rate parameter)

Key Properties:

Mean: E(X) = λ
Variance: Var(X) = λ
Discrete distribution
Limiting case of binomial when n→∞, p→0, np→λ

Applications:

Traffic analysis: accidents per time period
Telecommunications: call arrivals
Biology: mutation counts
Finance: market crashes per year

Normal Distribution N(μ,σ²)

The most important continuous distribution in statistics

Probability Function:

f(x) = (1/(σ√(2π))) exp(-(x-μ)²/(2σ²))

Parameters: μ ∈ ℝ (mean), σ > 0 (standard deviation)

Key Properties:

Mean: E(X) = μ
Variance: Var(X) = σ²
Symmetric around μ
68-95-99.7 rule for standard deviations

Applications:

Natural phenomena: heights, weights, measurements
Errors in experiments and measurements
Financial returns and risk modeling
Central Limit Theorem applications

Uniform Distribution U(a,b)

All values in an interval are equally likely

Probability Function:

f(x) = 1/(b-a) for a ≤ x ≤ b

Parameters: a < b (interval endpoints)

Key Properties:

Mean: E(X) = (a+b)/2
Variance: Var(X) = (b-a)²/12
Rectangular probability density
NOT exponential family (support depends on parameters)

Applications:

Random sampling and simulation
Computer random number generation
Modeling uncertainty with known bounds
Quality control with tolerance limits

Exponential Distribution E(λ)

Models waiting times and lifetimes with memoryless property

Probability Function:

f(x) = λe^(-λx) for x > 0

Parameters: λ > 0 (rate parameter)

Key Properties:

Mean: E(X) = 1/λ
Variance: Var(X) = 1/λ²
Memoryless property: P(X>s+t|X>s) = P(X>t)
Special case of Gamma(1,λ)

Applications:

Product lifetime analysis
Service time modeling
Radioactive decay processes
Reliability engineering

Advanced Distribution Families

Specialized distributions derived from fundamental ones, essential for statistical inference

Gamma Distribution Γ(α,λ)

Generalizes exponential distribution for sum of waiting times

Density Function:

f(x) = (λ^α/Γ(α)) x^(α-1) e^(-λx)

Parameters: α > 0 (shape), λ > 0 (rate)

Key Properties:

Additivity: Γ(α₁,λ) + Γ(α₂,λ) = Γ(α₁+α₂,λ)
Special cases: Γ(1,λ) = E(λ), Γ(n/2,1/2) = χ²(n)
Mean: E(X) = α/λ, Variance: Var(X) = α/λ²
Conjugate prior for Poisson rate parameter

Applications:

Reliability modeling: time to k-th failure
Bayesian inference: prior distributions
Queuing theory: service time modeling
Risk analysis and insurance

Chi-Square Distribution χ²(n)

Distribution of sum of squared standard normal variables

Density Function:

f(x) = (1/(2^(n/2)Γ(n/2))) x^(n/2-1) e^(-x/2)

Parameters: n ≥ 1 (degrees of freedom)

Key Properties:

Definition: X₁² + X₂² + ... + Xₙ² where Xᵢ ~ N(0,1)
Additivity: χ²(n₁) + χ²(n₂) = χ²(n₁+n₂)
Mean: E(X) = n, Variance: Var(X) = 2n
Non-central χ²: when Xᵢ ~ N(aᵢ,1), parameter λ = Σaᵢ²

Applications:

Goodness-of-fit testing
Testing independence in contingency tables
Confidence intervals for variance
Sample variance distribution in normal populations

t-Distribution t(n)

Distribution for small sample inference about normal means

Density Function:

f(t) = Γ((n+1)/2)/(√(nπ)Γ(n/2)) (1+t²/n)^(-(n+1)/2)

Parameters: n ≥ 1 (degrees of freedom)

Key Properties:

Definition: T = X/√(K/n) where X ~ N(0,1), K ~ χ²(n)
Symmetric around 0: E(T) = 0 (n ≥ 2)
Variance: Var(T) = n/(n-2) (n ≥ 3), larger than N(0,1)
Limit: t(n) → N(0,1) as n → ∞

Applications:

Confidence intervals for means (σ unknown)
One-sample and two-sample t-tests
Small sample inference in normal populations
Regression coefficient testing

F-Distribution F(m,n)

Ratio of two chi-square variables, used for variance comparison

Density Function:

f(x) = Γ((m+n)/2)/(Γ(m/2)Γ(n/2)) m^(m/2)n^(n/2) x^(m/2-1)/(n+mx)^((m+n)/2)

Parameters: m,n ≥ 1 (degrees of freedom)

Key Properties:

Definition: F = (K₁/m)/(K₂/n) where K₁ ~ χ²(m), K₂ ~ χ²(n)
Reciprocal property: 1/F ~ F(n,m)
Quantile relation: F₁₋α(m,n) = 1/Fα(n,m)
Connection to t-distribution: t²(n) ~ F(1,n)

Applications:

Testing equality of two variances
ANOVA F-tests for mean equality
Regression model significance testing
Comparing model fit in nested models

Exponential Family Theory

Unified framework for many important statistical distributions

General Form:

f(x;θ) = c(θ) exp{∑Qⱼ(θ)Tⱼ(x)} h(x)

Components Explanation:

c(θ)

Normalizing function depending only on parameter θ

Ensures probability density/mass integrates to 1

Qⱼ(θ)

Natural parameter functions

Transform original parameters to canonical form

Tⱼ(x)

Sufficient statistic functions

Capture all information about θ from the data

h(x)

Base measure function independent of θ

Provides reference measure for the distribution

Examples in Exponential Family Form:

Normal N(μ,σ²)

c(μ,σ²) exp{(μ/σ²)x - (1/2σ²)x²} h(x)

Natural parameters: η₁ = μ/σ², η₂ = -1/2σ²

Sufficient statistics: T₁(x) = x, T₂(x) = x²

Binomial B(n,p)

c(p) exp{ln(p/(1-p)) · x} h(x)

Natural parameters: η = ln(p/(1-p))

Sufficient statistics: T(x) = x

Poisson P(λ)

c(λ) exp{ln(λ) · x} h(x)

Natural parameters: η = ln(λ)

Sufficient statistics: T(x) = x

Important Properties:

Sufficient statistics have finite dimension
Maximum likelihood estimation has nice properties
Conjugate priors exist for Bayesian inference
Exponential families are closed under sampling

Non-Examples:

✗Uniform U(0,θ): support depends on parameter
✗Cauchy distribution: no finite moments
✗Some mixture distributions

Distribution Relationships

Understanding how different distributions connect and derive from each other

Gamma Family Relationships

Γ(1,λ) = Exponential E(λ)
Γ(n/2, 1/2) = Chi-square χ²(n)
Sum property: Γ(α₁,λ) + Γ(α₂,λ) = Γ(α₁+α₂,λ)

Normal Distribution Connections

Sample mean: X̄ ~ N(μ, σ²/n) from N(μ,σ²) population
Standardization: (X-μ)/σ ~ N(0,1)
Chi-square: ∑(Xᵢ-μ)²/σ² ~ χ²(n) for X ~ N(μ,σ²)

t and F Distribution Origins

t(n) = N(0,1) / √(χ²(n)/n)
F(m,n) = (χ²(m)/m) / (χ²(n)/n)
t²(n) = F(1,n)

Beta and Fisher Z Connections

Beta(1,1) = Uniform U(0,1)
If Y ~ Beta(a,b), then X = Y/(1-Y) ~ Fisher Z(a,b)
If F ~ F(n,m), then (n/m)F ~ Fisher Z(n/2, m/2)

Practical Applications

Real-world scenarios demonstrating distribution family applications

Quality Control Application

Manufacturing defect analysis using binomial and Poisson distributions

Problem Setup:

Factory produces items with 2% defect rate, inspecting batches of 100 items

Relevant Distributions:

Binomial B(100, 0.02): exact number of defects per batch
Poisson P(2): approximation when n large, p small
Normal N(2, 1.96): further approximation for large samples

Key Insights:

Exact calculation: P(X ≤ 5) using binomial
Approximation: P(X ≤ 5) ≈ P(Y ≤ 5) for Y ~ Poisson(2)
Continuity correction for normal approximation

Reliability Engineering

Product lifetime modeling using exponential and gamma distributions

Problem Setup:

Electronic components with exponential lifetimes, λ = 0.001 failures/hour

Relevant Distributions:

Exponential E(0.001): individual component lifetime
Gamma Γ(k, 0.001): time until k-th failure
Chi-square: related to sum of squared standardized lifetimes

Key Insights:

Mean lifetime: 1/λ = 1000 hours
Memoryless property: P(T > t+s | T > t) = P(T > s)
System reliability: P(all n components work) = e^(-nλt)

Medical Research

Clinical trial analysis using t-distribution and F-distribution

Problem Setup:

Comparing treatment effects in small samples with unknown variance

Relevant Distributions:

t(n-1): confidence intervals for treatment mean
F(1, n₁+n₂-2): comparing variances of two treatment groups
Chi-square: goodness-of-fit for treatment response categories

Key Insights:

Small sample inference when σ unknown
Two-sample t-test for treatment comparison
ANOVA F-test for multiple treatment groups

🎯 Apply Your Knowledge

Practice with distribution calculators, explore formulas, and test your understanding

Distribution Calculator

Transform between different distribution families and compute probabilities.

Try Calculator

Formula Reference

Access comprehensive formulas for all distribution families and their properties.

View Formulas

Practice Problems

Test your understanding with interactive practice problems and explanations.

Practice Now

Continue learning

Point Estimation