Prerequisites

8-12 Hours

Probability Theory Prerequisites

Master the essential mathematical foundations required for advanced stochastic modeling

Probability Foundations - Probability Space

A probability space is a triple $(\Omega, \mathcal{F}, P)$ where:

Sample Space $\Omega$ : The set of all possible outcomes
$\sigma$ -Algebra $\mathcal{F}$ : A collection of measurable events
Probability Measure $P$ : A function assigning probabilities to events

Kolmogorov's Axioms

Non-negativity: $P(E) \geq 0$ for any event $E$
Normalization: $P(\Omega) = 1$
Countable Additivity: For pairwise disjoint events $E_1, E_2, \dots$ : $P\left(\bigcup_{i=1}^{\infty} E_i\right) = \sum_{i=1}^{\infty} P(E_i)$

Conditional Probability - Definition & Multiplication Rule

The conditional probability of $A$ given $B$ is:

P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad \text{where } P(B) > 0

Multiplication Rule: $P(A \cap B) = P(A) \times P(B|A) = P(B) \times P(A|B)$

Law of Total Probability

If $B_1, B_2, \dots, B_n$ form a partition of the sample space, then:

P(A) = \sum_{i=1}^n P(A|B_i)P(B_i)

Bayes' Theorem

Bayes' theorem provides a way to update probabilities based on new evidence:

P(B_j|A) = \frac{P(A|B_j)P(B_j)}{\sum_{i=1}^n P(A|B_i)P(B_i)} = \frac{P(A|B_j)P(B_j)}{P(A)}

Components: $P(B_j)$ is the prior probability, $P(B_j|A)$ is the posterior probability, and $P(A|B_j)$ is the likelihood.

Independence - Definition

Two events $A$ and $B$ are independent if:

P(A \cap B) = P(A) \times P(B)

Equivalently, $P(A|B) = P(A)$ (knowing $B$ doesn't change the probability of $A$ ).

Note: For three or more events, we need both pairwise independence and the triple intersection condition for mutual independence.

Worked Examples - Example 1: Medical Diagnosis

Problem:

A disease has prevalence $P(D) = 0.01$ . A test has sensitivity $P(+|D) = 0.95$ and specificity $P(-|D^c) = 0.90$ . If the test is positive, what is $P(D|+)$ ?

Solution:

Using Bayes' theorem: $P(D|+) = \frac{P(+|D)P(D)}{P(+)}$

First, find $P(+) = P(+|D)P(D) + P(+|D^c)P(D^c) = 0.95 \times 0.01 + 0.10 \times 0.99 = 0.1085$

Then: $P(D|+) = \frac{0.95 \times 0.01}{0.1085} \approx 0.0876$ or about 8.76%.

Key Insight: Despite a positive test, the probability is low due to the rare disease.

Example 2: Independence Check

Problem:

Given $P(A) = 0.3$ , $P(B) = 0.4$ , and $P(A \cap B) = 0.12$ , are $A$ and $B$ independent?

Solution:

Check: $P(A) \times P(B) = 0.3 \times 0.4 = 0.12 = P(A \cap B)$

Answer: Yes, the events are independent.

Example 3: Law of Total Probability

Problem:

A factory has three machines producing items. Machine 1 produces 50% with defect rate 2%, Machine 2 produces 30% with defect rate 3%, Machine 3 produces 20% with defect rate 4%. What is the overall defect rate?

Solution:

Using the Law of Total Probability:

P(\text{defect}) = 0.5 \times 0.02 + 0.3 \times 0.03 + 0.2 \times 0.04 = 0.01 + 0.009 + 0.008 = 0.027

The overall defect rate is 2.7%.

Practice Quiz

Questions

Correct

Accuracy

What is the formula for conditional probability

P(A|B)

Not attempted

P(A) = 0.4

and

P(B|A) = 0.6

, what is

P(A \cap B)

Not attempted

Two events

A

and

B

are independent if:

Not attempted

What is Bayes' theorem used for?

Not attempted

P(A \cap B) = 0.3

and

P(B) = 0.5

, what is

P(A|B)

Not attempted

A test is 95% accurate for a disease (2% prevalence). If test is positive, what is

P(\text{disease}|\text{positive})

Not attempted

What is the multiplication rule for probability?

Not attempted

P(A) = 0.3

P(B) = 0.4

, and

P(A \cap B) = 0.12

, are

A

and

B

independent?

Not attempted

What does

P(A|B)

represent?

Not attempted

What is the Law of Total Probability?

Not attempted

Frequently Asked Questions

What is the difference between a random variable and a random process?

A random variable is a single function mapping outcomes to numbers (like one roll of a die). A random process is a collection of random variables indexed by time (like the sequence of results from rolling a die repeatedly). You can think of a process as a 'random function of time'.

Why is the axiomatic definition of probability important?

The axiomatic definition, introduced by Kolmogorov, provides a rigorous mathematical foundation that avoids the circularity of the classical definition (which requires 'equally likely' outcomes) and the ambiguity of the frequentist definition. It allows us to prove theorems and handle complex scenarios, including infinite sample spaces, which are crucial for stochastic processes.

What is the difference between mutually exclusive and independent events?

Mutually exclusive events cannot happen at the same time (intersection is empty, P(A∩B)=0). Independent events are such that the occurrence of one does not affect the probability of the other (P(A∩B)=P(A)P(B)). Interestingly, if two events have non-zero probabilities, they cannot be both mutually exclusive and independent.

How do I know when to use Bayes' Theorem?

Use Bayes' Theorem when you have a 'reverse' probability question. For example, if you know the probability of an effect given a cause (likelihood), but you want to find the probability of the cause given the observed effect (posterior). It's the standard tool for updating beliefs with new evidence.

What is a partition of a sample space?

A partition is a collection of events B₁, B₂, ... that are pairwise mutually exclusive (no overlap) and exhaustive (their union covers the entire sample space). This concept is fundamental to the Law of Total Probability, allowing us to break down a complex event into simpler, non-overlapping scenarios.