Back to Probability Theory Jump to Practice

Core Topic

8-10 Hours

Random Variables & Distributions

Intermediate Level

12 Lessons

Core Concepts

Random Variable Fundamentals

Understanding the mathematical foundation and classification of random variables

Definition & Classification

Random Variable Definition

$X: \Omega \to \mathbb{R}, a measurable function mapping from sample space to real numbers$

Key Point:

Random variable transforms random outcomes into numerical values for mathematical analysis

Properties:

$Measurability: \{\omega: X(\omega) \leq x\} \in \mathcal{F} for all x \in \mathbb{R}$
$Range: Can be finite, countably infinite, or uncountably infinite$
$Induces probability: P_X(A) = P(\{\omega: X(\omega) \in A\})$

Examples:

Dice roll: X(outcome) = number shown
Coin tosses: X = number of heads in n tosses
Lifetime: X = time until device failure

Discrete vs Continuous

$Classification based on the range of possible values$

Characterization:

$Discrete: countable range; Continuous: uncountable range (interval)$

Properties:

$Discrete: described by PMF P(X=x_i)$
$Continuous: described by PDF p(x) where P(X=x)=0$
$Mixed: combination of discrete and continuous parts$

Examples:

Discrete: number of customers, defects count
Continuous: temperature, waiting time, height
Mixed: insurance claim (0 with positive probability, continuous otherwise)

Distribution Functions

The fundamental tool for describing probability distributions

Cumulative Distribution Function

Definition

$F(x) = P(X \leq x) for all x \in \mathbb{R}$

Essential Properties

Monotonicity

$F(x_1) \leq F(x_2) \text{ if } x_1 \leq x_2$

CDF is non-decreasing

Right Continuity

$\lim_{x \to a^+} F(x) = F(a)$

Continuous from the right

Limits

$\lim_{x \to -\infty} F(x) = 0, \lim_{x \to +\infty} F(x) = 1$

Bounds at infinity

Relationships with PMF/PDF

Discrete Case

F(x) = \sum_{x_i \leq x} P(X = x_i)

Sum of probabilities up to x

Continuous Case

F(x) = \int_{-\infty}^x p(t)dt, \quad p(x) = F'(x)

Integral of PDF; PDF is derivative of CDF

Common Discrete Distributions

Essential discrete probability distributions and their applications

Bernoulli Distribution

Ber(p)

Parameters:

p ∈ (0,1) (success probability)

Probability Mass Function:

P(X = k) = p^k(1-p)^{1-k}, k ∈ {0,1}

Application:

Single trial success/failure experiment

Key Properties:

$E[X] = p$
$Var(X) = p(1-p)$
$Special case of binomial with n=1$

Binomial Distribution

B(n,p)

Parameters:

n ∈ ℕ (trials), p ∈ (0,1) (success probability)

Probability Mass Function:

P(X = k) = C_n^k p^k(1-p)^{n-k}, k = 0,1,...,n

Application:

Number of successes in n independent Bernoulli trials

Key Properties:

$E[X] = np$
$Var(X) = np(1-p)$
$Additive: B(n₁,p) + B(n₂,p) = B(n₁+n₂,p)$

Poisson Distribution

P(λ)

Parameters:

λ > 0 (rate parameter)

Probability Mass Function:

P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}, k = 0,1,2,...

Application:

Rare events occurrence (defects, arrivals, accidents)

Key Properties:

$E[X] = Var(X) = λ$
$Additive: P(λ₁) + P(λ₂) = P(λ₁+λ₂)$
$Approximates binomial when n large, p small, np = λ$

Geometric Distribution

Geo(p)

Parameters:

p ∈ (0,1) (success probability)

Probability Mass Function:

P(X = k) = (1-p)^{k-1}p, k = 1,2,3,...

Application:

Number of trials until first success

Key Properties:

$E[X] = 1/p$
$Var(X) = (1-p)/p²$
$Memoryless: P(X > s+t|X > s) = P(X > t)$

Hypergeometric Distribution

H(n,M,N)

Parameters:

n (sample size), M (success states), N (population size)

Probability Mass Function:

P(X = k) = \frac{C_M^k C_{N-M}^{n-k}}{C_N^n}

Application:

Sampling without replacement (defective items, card draws)

Key Properties:

$E[X] = n(M/N)$
$Approximates B(n,M/N) when N → ∞$

Common Continuous Distributions

Fundamental continuous probability distributions and their properties

Uniform Distribution

U(a,b)

Parameters:

$a, b ∈ ℝ, a < b$

Application:

Equal probability over an interval (random timing, rounding errors)

Key Properties:

$E[X] = (a+b)/2$
$Var(X) = (b-a)²/12$
$P(c < X < c+l) = l/(b-a) for any c$

Probability Density Function:

p(x) = \begin{cases}\frac{1}{b-a}, & a < x < b \\ 0, & \text{otherwise}\end{cases}

Normal Distribution

N(μ,σ²)

Parameters:

$μ ∈ ℝ (mean), σ² > 0 (variance)$

Application:

Natural phenomena (heights, measurement errors, test scores)

Key Properties:

$E[X] = μ$
$Var(X) = σ²$
$Standardization: (X-μ)/σ ~ N(0,1)$
$Additive property$

Probability Density Function:

p(x) = \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}

Exponential Distribution

Exp(λ)

Parameters:

$λ > 0 (rate parameter)$

Application:

Lifetime modeling, waiting times between events

Key Properties:

$E[X] = 1/λ$
$Var(X) = 1/λ²$
$Memoryless: P(X > s+t|X > s) = P(X > t)$

Probability Density Function:

p(x) = \begin{cases}\lambda e^{-\lambda x}, & x > 0 \\ 0, & x \leq 0\end{cases}

Gamma Distribution

Γ(α,β)

Parameters:

$α > 0 (shape), β > 0 (rate)$

Application:

Sum of independent exponential variables, reliability modeling

Key Properties:

$E[X] = α/β$
$Var(X) = α/β²$
$Additive: Γ(α₁,β) + Γ(α₂,β) = Γ(α₁+α₂,β)$

Probability Density Function:

p(x) = \begin{cases}\frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x}, & x > 0 \\ 0, & x \leq 0\end{cases}

Chi-squared Distribution

χ²(n)

Parameters:

$n ∈ ℕ (degrees of freedom)$

Application:

Sum of squares of independent standard normal variables

Key Properties:

$E[X] = n$
$Var(X) = 2n$
$Additive: χ²(n₁) + χ²(n₂) = χ²(n₁+n₂)$

Probability Density Function:

p(x) = \begin{cases}\frac{(1/2)^{n/2}}{\Gamma(n/2)}x^{n/2-1}e^{-x/2}, & x > 0 \\ 0, & x \leq 0\end{cases}

Multidimensional Random Variables

Understanding joint distributions and independence of multiple random variables

Joint Distributions

Discrete Case

Definition:

$Joint PMF: P(X = xᵢ, Y = yⱼ) = pᵢⱼ with Σᵢ Σⱼ pᵢⱼ = 1$

Condition:

$Non-negativity: pᵢⱼ ≥ 0$
$Normalization: ΣΣ pᵢⱼ = 1$

Properties:

• $Marginal PMF: P(X=xᵢ) = Σⱼ pᵢⱼ$
• $Conditional PMF: P(Y=yⱼ|X=xᵢ) = pᵢⱼ/pᵢ·$

Continuous Case

Definition:

$Joint PDF: p(x,y) with ∫∫ p(x,y)dxdy = 1$

Condition:

$Non-negativity: p(x,y) ≥ 0$
$Normalization: ∫∫ p(x,y)dxdy = 1$

Properties:

• $Marginal PDF: p_X(x) = ∫ p(x,y)dy$
• $Conditional PDF: p_{Y|X}(y|x) = p(x,y)/p_X(x)$

Independence

Definition

Definition:

$Random variables X and Y are independent if their joint distribution factors into the product of marginals$

Condition:

$Discrete: P(X=x, Y=y) = P(X=x)P(Y=y)$
$Continuous: p(x,y) = p_X(x)p_Y(y)$

Properties:

• $E[XY] = E[X]E[Y]$
• $Var(X+Y) = Var(X) + Var(Y)$
• $Cov(X,Y) = 0$

Functions of Random Variables

Methods for determining the distribution of functions of random variables

Distribution of Functions

Discrete Case

Method:

$P(Y=y) = Σ_{x:g(x)=y} P(X=x)$

Steps:

1 $Identify all x values mapping to y$
2 $Sum probabilities of these x values$

Applications:

• Linear transformations Y = aX + b
• Square of a random variable Y = X²

Continuous Case (Monotonic)

Method:

$p_Y(y) = p_X(h(y))|h'(y)| where x=h(y) is the inverse function$

Steps:

1 $Find inverse function x = h(y)$
2 $Calculate derivative h'(y)$
3 $Substitute into formula$

Applications:

• Linear scaling
• Log-normal distribution derivation

Sampling Distributions

Essential distributions for statistical inference

Chi-squared Distribution

Definition:

$If X₁, X₂, ..., Xₙ ~ N(0,1) independently, then Y = Σᵢ₌₁ⁿ Xᵢ² ~ χ²(n)$

Properties:

$Degrees of freedom: n$
$Additive property$
$E[Y] = n, Var(Y) = 2n$

Statistical Applications:

• Goodness of fit tests
• Variance testing
• Independence testing

t-Distribution

Definition:

$If X ~ N(0,1), Y ~ χ²(n) independently, then T = X/√(Y/n) ~ t(n)$

Properties:

$Degrees of freedom: n$
$Symmetric about 0$
$Approaches N(0,1) as n → ∞$

Statistical Applications:

• Small sample inference
• Confidence intervals for mean
• Hypothesis testing

F-Distribution

Definition:

$If X ~ χ²(m), Y ~ χ²(n) independently, then F = (X/m)/(Y/n) ~ F(m,n)$

Properties:

$Two degrees of freedom: m, n$
$F(m,n) = 1/F(n,m)$
$If T ~ t(n), then T² ~ F(1,n)$

Statistical Applications:

• Variance ratio testing
• ANOVA
• Regression analysis

Worked Examples

Step-by-step solutions to typical random variable problems

Binomial Probability Calculation

Problem:

In 10 independent trials with success probability 0.3, find P(X = 4)

Solution Steps:

1 $Step 1: Identify distribution - X ~ B(10, 0.3)$
2 $Step 2: Apply PMF formula - P(X = k) = C₁₀ᵏ (0.3)ᵏ (0.7)¹⁰⁻ᵏ$
3 $Step 3: Calculate combination - C₁₀⁴ = 10!/(4!×6!) = 210$
4 $Step 4: Compute probability - P(X = 4) = 210 × (0.3)⁴ × (0.7)⁶$
5 $Step 5: Final calculation - P(X = 4) = 210 × 0.0081 × 0.1176 ≈ 0.200$

Key Point:

Binomial distribution applies to fixed number of independent trials with constant success probability

Normal Distribution Standardization

Problem:

If X ~ N(100, 16), find P(96 < X < 108)

Solution Steps:

1 $Step 1: Identify parameters - μ = 100, σ² = 16, so σ = 4$
2 $Step 2: Standardize - Z = (X - 100)/4 ~ N(0,1)$
3 $Step 3: Transform bounds - P(96 < X < 108) = P(-1 < Z < 2)$
4 $Step 4: Use standard normal table - P(-1 < Z < 2) = Φ(2) - Φ(-1)$
5 $Step 5: Calculate - P(-1 < Z < 2) = 0.9772 - 0.1587 = 0.8185$

Key Point:

Standardization allows us to use the standard normal table for any normal distribution

Exponential Distribution Memory Property

Problem:

For X ~ Exp(λ), prove that P(X > s+t|X > s) = P(X > t)

Solution Steps:

1 $Step 1: Write conditional probability - P(X > s+t|X > s) = P(X > s+t, X > s)/P(X > s)$
2 $Step 2: Simplify numerator - P(X > s+t, X > s) = P(X > s+t)$
3 $Step 3: Use exponential CDF - P(X > x) = e⁻λˣ for x > 0$
4 $Step 4: Substitute - P(X > s+t|X > s) = e⁻λ(s+t)/e⁻λs = e⁻λt$
5 $Step 5: Conclude - P(X > s+t|X > s) = e⁻λt = P(X > t)$

Key Point:

Memoryless property means past waiting time doesn't affect future waiting time

Practice Quiz

Questions

Correct

Accuracy

What is the key difference between discrete and continuous random variables?

Not attempted

For a continuous random variable

X

, what is

P(X = x)

for any specific value

x

Not attempted

Which property does NOT hold for a CDF

F(x)

Not attempted

X \sim B(n, p)

, what is

E[X]

Not attempted

What distribution models the number of trials until the first success?

Not attempted

X

and

Y

are independent, which is TRUE?

Not attempted

What is the memoryless property?

Not attempted

X \sim N(0, 1)

and

Y = \sum_{i=1}^n X_i^2

where

X_i

are independent

N(0,1)

, what is

Y

's distribution?

Not attempted

For the transformation

Y = aX + b

where

X

is continuous with PDF

p_X(x)

, what is

p_Y(y)

Not attempted

What does

E[XY] = E[X]E[Y]

imply about

X

and

Y

Not attempted

Frequently Asked Questions

What is the difference between PMF and PDF?

PMF is used for discrete random variables and gives exact point probabilities P(X = x). PDF is used for continuous random variables and gives density, so probabilities come from integration over intervals and P(X = x) = 0.

How do I know which distribution to use for a problem?

Match the context to the defining mechanism: fixed independent trials suggest Binomial, time until first success suggests Geometric, rare counts suggest Poisson, waiting times suggest Exponential, and measurement-type phenomena often suggest Normal.

What does independence mean for random variables?

Random variables X and Y are independent if knowing one gives no information about the other. In formulas, the joint distribution factors into the product of marginals: P(X=x, Y=y) = P(X=x)P(Y=y) in the discrete case, or p(x,y) = pX(x)pY(y) in the continuous case.

Why are sampling distributions (chi-square, t, F) important?

These sampling distributions describe how common statistics behave under repeated sampling. They support variance inference, small-sample mean inference, ANOVA, and many classical confidence intervals and hypothesis tests.

How do I transform a random variable?

For discrete variables, sum probabilities across all original values that map to the transformed value. For continuous monotone transforms, use the inverse function and derivative. For non-monotone or multivariate transforms, use CDF arguments or a Jacobian.