Mathematical Statistics

Mathematical Statistics Practice 1

Problems on estimation, sufficiency, Fisher information, and hypothesis testing

8 Problems

Suggested: 2 hours

Instructions

• Try to solve each problem before viewing the solution
• Click "Show Solution" to reveal the answer and detailed explanation
• Focus on understanding the problem-solving methodology

1Sufficient Statistics

Problem

Let $X_1, X_2, \ldots, X_n$ be i.i.d. from $\text{Uniform}(0, \theta)$ where θ > 0 is unknown.

(1) Find a sufficient statistic for θ.

(2) Show that $X_{(n)} = \max(X_1, \ldots, X_n)$ is sufficient for θ.

(3) Is $\bar{X}$ sufficient for θ?

Answer Summary

Look for factorization of the likelihood so the data only enters through a low-dimensional statistic that captures all parameter information.

2Maximum Likelihood Estimation

Problem

Let $X_1, \ldots, X_n$ be i.i.d. from a distribution with density:

f(x|\theta) = \frac{2x}{\theta^2}, \quad 0 < x < \theta

(1) Find the MLE of θ.

(2) Is the MLE unbiased? If not, find an unbiased estimator based on the MLE.

(3) Find the variance of the MLE.

Answer Summary

Write the likelihood, take logs to simplify differentiation, and solve the score equation while checking parameter constraints.

3Fisher Information and CRLB

Problem

Let $X_1, \ldots, X_n$ be i.i.d. Poisson(λ).

(1) Find the Fisher information $I(\lambda)$ for a single observation.

(2) Find the Cramér-Rao lower bound for unbiased estimators of λ.

(3) Show that $\bar{X}$ attains the CRLB.

(4) Find the CRLB for estimating $g(\lambda) = e^{-\lambda}$ .

Answer Summary

Compute Fisher information from the score or Hessian, then compare estimator variance to the Cramer-Rao lower bound.

4Method of Moments Estimation

Problem

Let $X_1, \ldots, X_n$ be i.i.d. from Gamma(α, β) with density:

f(x|\alpha, \beta) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x > 0

Find the method of moments estimators for α and β.

Answer Summary

Match sample moments to theoretical moments and solve the resulting system for the unknown parameter values.

5Hypothesis Testing: Likelihood Ratio Test

Problem

Let $X_1, \ldots, X_n$ be i.i.d. N(μ, σ²) where σ² is known.

Test $H_0: \mu = \mu_0$ vs $H_1: \mu \neq \mu_0$ .

(1) Derive the likelihood ratio test statistic.

(2) Find the rejection region for significance level α.

(3) Show the LRT is equivalent to the z-test.

Answer Summary

Build the unrestricted and restricted likelihoods, form their ratio, and reject when the data looks too unlikely under the null model.

6Confidence Intervals

Problem

Let $X_1, \ldots, X_n$ be i.i.d. from Exponential(λ).

(1) Find a pivot based on $\sum X_i$ .

(2) Construct a 95% confidence interval for λ.

(3) Find the expected length of this confidence interval.

Answer Summary

Choose the right pivot or sampling distribution, then invert the probability statement to produce lower and upper bounds.

7UMVUE and Completeness

Problem

Let $X_1, \ldots, X_n$ be i.i.d. Bernoulli(p).

(1) Show that $T = \sum X_i$ is complete and sufficient.

(2) Find the UMVUE of p.

(3) Find the UMVUE of p(1-p).

Answer Summary

Combine sufficiency, completeness, and unbiasedness so the final estimator is justified by Lehmann-Scheffe rather than guesswork.

8Bayesian Estimation

Problem

Let $X_1, \ldots, X_n$ be i.i.d. Poisson(λ). Assume a Gamma(α, β) prior for λ.

(1) Find the posterior distribution of λ.

(2) Find the Bayes estimator under squared error loss.

(3) Find a 95% Bayesian credible interval for λ.

Answer Summary

Start with the prior and likelihood, form the posterior, and then read off the Bayes estimator under the stated loss function.

Related Theory

Review Common Distribution Families

Revisit the theory that supports these worked practice problems.

Review Point Estimation and Cramer-Rao Bound

Revisit the theory that supports these worked practice problems.

Review Sufficient and Complete Statistics

Revisit the theory that supports these worked practice problems.

Review Bayesian Statistics and Inference

Revisit the theory that supports these worked practice problems.