Problems on Poisson processes, Markov chains, Brownian motion, martingales, and queueing theory
Instructions
Customers arrive at a service counter according to a Poisson process with rate λ = 3 per hour.
(1) What is the probability that exactly 5 customers arrive in the next 2 hours?
(2) Given that 5 customers arrived in 2 hours, what is the probability that exactly 2 arrived in the first hour?
(3) What is the expected waiting time until the 4th customer arrives?
Use exponential interarrival times and Poisson count properties to move between event counts and waiting-time questions.
A Markov chain has state space {1, 2, 3} with transition matrix:
(1) Find the stationary distribution π.
(2) If X₀ = 1, what is P(X₂ = 3)?
(3) Is this chain irreducible? Is it aperiodic?
Analyze the transition matrix, then use recurrence, irreducibility, or stationary-distribution logic to describe the long-run regime.
Consider a birth-death process with birth rates λₙ = nλ and death rates μₙ = nμ for n ≥ 1, μ₀ = 0.
(1) Write the generator matrix Q for states {0, 1, 2} when λ = 2, μ = 1.
(2) Find the stationary distribution if it exists.
(3) What is the physical interpretation of this process?
Write the one-step balance equations carefully and use the birth and death rates to derive hitting or equilibrium probabilities.
A symmetric random walk on integers starts at X₀ = 0. At each step, the walk moves +1 or -1 with equal probability.
(1) Find P(X₄ = 0).
(2) Find E[X₁₀].
(3) What is the probability that the walk first returns to 0 at time n = 4?
Reduce the walk to step probabilities and cumulative increments, then use symmetry or recursion depending on whether the question is local or long-run.
Let {B(t), t ≥ 0} be a standard Brownian motion.
(1) Find P(B(2) > 1 | B(1) = 0.5).
(2) Calculate Cov(B(2), B(5)).
(3) What is the distribution of M = max{B(t): 0 ≤ t ≤ 1}?
Rely on Gaussian increments, scaling, and covariance structure to compute distributions and pathwise consequences of Brownian motion.
Let {Xₙ, n ≥ 0} be a sequence of i.i.d. random variables with E[Xᵢ] = 0 and Var(Xᵢ) = σ².
Define Sₙ = X₁ + X₂ + ... + Xₙ with S₀ = 0.
(1) Show that {Sₙ} is a martingale.
(2) Show that {Sₙ² - nσ²} is a martingale.
(3) Use the optional stopping theorem to find E[Sₜ] where T is a stopping time with E[T] < ∞.
Verify the conditional-expectation definition first, then use martingale tools such as optional stopping or Doob-style decompositions.
A machine breaks down, and the time until breakdown follows an Exponential(λ) distribution. After each breakdown, repair takes a random time with Exponential(μ) distribution, independent of breakdown times.
(1) What is the long-run proportion of time the machine is operating?
(2) If λ = 0.5 failures/hour and μ = 2 repairs/hour, find the expected number of failures in 100 hours.
Track the process through interarrival times and renewal counts, then apply renewal identities to get expectations or asymptotic behavior.
Consider an M/M/1 queue with arrival rate λ = 4 customers/hour and service rate μ = 5 customers/hour.
(1) Find the utilization ρ and verify stability.
(2) Find the average number of customers in the system L.
(3) Find the average time a customer spends in the system W.
(4) What is the probability that an arriving customer finds the system empty?
Translate arrivals and service into birth-death parameters, then use traffic intensity to analyze stability, waiting, and steady-state counts.
Review Probability Theory Prerequisites
Revisit the theory that supports these worked practice problems.
Review Markov Chains
Revisit the theory that supports these worked practice problems.
Review Poisson Processes
Revisit the theory that supports these worked practice problems.
Review Brownian Motion
Revisit the theory that supports these worked practice problems.