Let be a sequence of i.i.d. random variables with and . For , define , where are constants satisfying . Prove: (a) The series defining converges almost surely; (b) holds in both the almost sure and convergence senses.

Stochastic Processes Solution – Problem 32: Prove: (a) The series defining converges almost surely; (b) holds in both the almost sure and…

Question

Let $X_n, n \geq 1$ be a sequence of i.i.d. random variables with $EX_1 = 0$ and $\operatorname{var}(X_1) = 1$ . For $n \geq 1$ , define $Y_n = \sum_{i=0}^{\infty} \alpha_i X_{n+i}$ , where $\alpha_i$ are constants satisfying $\sum \alpha_i^2 < \infty$ . Prove: (a) The series defining $Y_n$ converges almost surely; (b) $\lim_{n \to \infty} n^{-1} \sum_{i=1}^{n} Y_i = 0$ holds in both the almost sure and $L^1$ convergence senses.

Step-by-step solution

Step 1. (a) Proof of a.s. convergence of the series defining $Y_n$ . Consider the infinite series $\sum_{i=0}^{\infty} \alpha_i X_{n+i}$ . Since $E[X_k]=0$ and $\mathrm{var}(X_k)=1$ , the partial sums $S_m = \sum_{i=0}^{m} \alpha_i X_{n+i}$ satisfy $\mathrm{var}(S_m) = \sum_{i=0}^{m} \alpha_i^2$ . Since $\sum \alpha_i^2 < \infty$ , by the Kolmogorov three-series theorem (or $L^2$ martingale convergence), the series converges a.s. This guarantees $Y_n$ is well-defined.

Step 2. (b) Show $\{Y_n\}$ is strictly stationary and ergodic. $Y_n$ is generated from the i.i.d. sequence $\{X_k\}$ by linear filtering. Define the shift $T$ by $X_k(\omega') = X_{k+1}(\omega)$ . Since $\{X_k\}$ is i.i.d., $T$ is measure-preserving and ergodic. $Y_n = f(T^n \omega)$ where $f(\omega) = \sum_{i=0}^{\infty} \alpha_i X_i(\omega)$ , so $\{Y_n\}$ is strictly stationary and ergodic.

Step 3. Verify conditions for the Birkhoff ergodic theorem. $E[Y_1^2] = \sum \alpha_i^2 E[X_{1+i}^2] = \sum \alpha_i^2 < \infty$ . Since $L^2$ integrability implies $L^1$ integrability, $E[|Y_1|] < \infty$ . Also $E[Y_1] = \sum \alpha_i E[X_{1+i}] = 0$ .

Step 4. Apply the Birkhoff ergodic theorem. For a strictly stationary ergodic $L^1$ -integrable sequence, the sample mean converges a.s. and in $L^1$ to the expectation. Hence $\lim_{n \to \infty} \frac{1}{n} \sum_{i=1}^{n} Y_i = E[Y_1] = 0$ a.s. and in $L^1$ .

Final answer

(a) QED. (b) QED.

Marking scheme

The following is the grading rubric based on the official solution.

1. Checkpoints (max 7 pts total)

(a) Prove the series converges a.s. (2 pts)

[additive] Use independence and $\sum \alpha_i^2 < \infty$ to show the partial sum variance is bounded or the sequence is $L^2$ -Cauchy. (1 pt)
[additive] Cite the Kolmogorov theorem (e.g., convergence theorem for independent zero-mean series, three-series theorem) or $L^2$ martingale convergence to conclude a.s. convergence. (1 pt)

(b) Prove the sample mean converges to 0 (5 pts)

Score exactly one chain (Chain A or Chain B); do not add points across chains.

Chain A: Ergodic theorem path (official solution)
[additive] Argue $\{Y_n\}$ is strictly stationary and ergodic. *(Must state this is based on the i.i.d. property of $\{X_n\}$ and that $Y_n$ is a measurable function of the shift; stationarity alone without ergodicity is insufficient.)* (2 pts)
[additive] Verify the integrability condition: $E[Y_1^2] = \sum \alpha_i^2 < \infty$ implies $Y_1 \in L^1$ . (1 pt)
[additive] Cite the Birkhoff ergodic theorem and state the limit equals $E[Y_1]$ . (1 pt)
[additive] Compute $E[Y_1] = 0$ , concluding the limit is 0 (covering both a.s. and $L^1$ ). (1 pt)
Chain B: Direct analysis of linear processes (exchange of order method)
[additive] Decompose the sample mean as $\frac{1}{n}\sum Y_k = \sum \alpha_j (\frac{1}{n} \sum X_{k+j})$ . (1 pt)
[additive] Apply SLLN to the inner sum: $\frac{1}{n} \sum X_{k+j} \to 0$ a.s. (1 pt)
[additive] Key step: Rigorously justify the exchange of summation and limit. *(Must provide uniform convergence estimates, tail truncation error analysis, or cite Phillips-Solo type results; exchange without proof gets 0 for this point.)* (3 pts)

Total (max 7)

2. Zero-credit items

In (b), only citing SLLN for i.i.d. variables without addressing the dependence among $Y_n$ .
Only stating definitions or known conditions.
Asserting convergence without any computation or theorem.

3. Deductions

Incorrectly assuming independence: In (b), explicitly assuming $Y_n$ are mutually independent. (-2 pts)
Logical gap: In Chain B, exchanging infinite series and limit without proof. (-2 pts)
Missing condition: Not checking $E|Y_1| < \infty$ when using Birkhoff's theorem. (-1 pt)
Convergence type confusion: Only proving $L^2$ or in-probability convergence without addressing a.s. convergence as required. (-1 pt)

Stochastic Processes – Problem 32: Prove: (a) The series defining converges almost surely; (b) holds in both the almost sure and…