Stochastic Processes – Problem 30: Prove the following results: (a) If , then is a martingale

Consider the asymmetric random walk, i.e., $P(\xi_i=1)=p$, $P(\xi_i=-1)=q=1-p$, and $p\neq q$. Prove the following results:

(a) If $\phi(y)=\left[\frac{1-p}{p}\right]^{y}$, then $\phi(S_n)$ is a martingale.

(b) With initial position $S_0=x$, let $T_z=\inf\{n:S_n=z\}$. Then for $a<x<b$, $$P_x(T_a<T_b)=\frac{\phi(b)-\phi(x)}{\phi(b)-\phi(a)},\quad P_x(T_b<T_a)=\frac{\phi(x)-\phi(a)}{\phi(b)-\phi(a)}.$$

For the following two parts, assume $\frac{1}{2}<p<1$:

(c) If $a<0$, then $P\left(\min_n S_n \leq a\right)=P(T_a<\infty)=\left[\frac{1-p}{p}\right]^{-a}$.

(d) If $b>0$, then $P(T_b<\infty)=1$, and $E(T_b)=\frac{b}{2p-1}$.

Let $S_n=S_0+\sum_{k=1}^n\xi_k$, where $$P(\xi_k=1)=p,\qquad P(\xi_k=-1)=q=1-p,\qquad p\ne q.$$ Define $$\phi(x):=\left(\frac{q}{p}\right)^x.$$

(a) Martingale property of $\phi(S_n)$. Because $S_{n+1}=S_n+\xi_{n+1}$, $$E[\phi(S_{n+1})\mid\mathcal F_n] =\phi(S_n)E\!\left[\left(\frac{q}{p}\right)^{\xi_{n+1}}\right] =\phi(S_n)\Bigl(p\frac{q}{p}+q\frac{p}{q}\Bigr) =\phi(S_n).$$ Hence $(\phi(S_n))$ is a martingale.

(b) Hitting probabilities on $[a,b]$, $a<x<b$. Let $$\tau:=T_a\wedge T_b, \qquad T_z:=\inf\{n\ge0:S_n=z\}.$$ By optional stopping for bounded stopping times $\tau\wedge m$, then $m\to\infty$, $$E_x[\phi(S_\tau)]=\phi(x).$$ Since $S_\tau\in\{a,b\}$, letting $u=P_x(T_a<T_b)$, we have $$\phi(x)=u\phi(a)+(1-u)\phi(b),$$ thus $$P_x(T_a<T_b)=\frac{\phi(b)-\phi(x)}{\phi(b)-\phi(a)}, \qquad P_x(T_b<T_a)=\frac{\phi(x)-\phi(a)}{\phi(b)-\phi(a)}.$$

Now assume $p>1/2$ (so $q/p<1$).

(c) Probability of ever hitting level $a<0$. Apply part (b) with upper barrier $b\to\infty$, starting from $x=0$: $$P_0(T_a<\infty)=\lim_{b\to\infty}\frac{\phi(b)-\phi(0)}{\phi(b)-\phi(a)} =\frac{0-1}{0-\phi(a)} =\phi(a)^{-1} =\left(\frac{q}{p}\right)^{-a}.$$ Since $\{\min_n S_n\le a\}=\{T_a<\infty\}$, $$P\!\left(\min_n S_n\le a\right)=\left(\frac{q}{p}\right)^{-a}.$$

(d) Hitting $b>0$: probability and expectation. Because the walk has positive drift $$E[\xi_1]=p-q=2p-1>0,$$ it tends to $+\infty$ almost surely, hence $$P_0(T_b<\infty)=1.$$ For expectation, define $$M_n:=S_n-(p-q)n.$$ Then $(M_n)$ is a martingale. Apply optional stopping at $\tau_b:=T_b$: $$E[M_{\tau_b}]=E[M_0]=0 \quad\Longrightarrow\quad E[S_{\tau_b}]-(p-q)E[\tau_b]=0.$$ Since $S_{\tau_b}=b$ a.s., $$E[T_b]=\frac{b}{p-q}=\frac{b}{2p-1}.$$ This proves all four claims.

Additional justification for optional stopping in part (b): use the bounded stopping time $$\tau_m:=\tau\wedge m,\qquad \tau=T_a\wedge T_b.$$ Because $a\le S_{\tau_m}\le b$, the variable $\phi(S_{\tau_m})$ is bounded uniformly in $m$, so dominated convergence applies. Therefore $$E_x[\phi(S_{\tau_m})]=\phi(x)\ \Longrightarrow\ E_x[\phi(S_{\tau})]=\phi(x).$$ This removes any integrability ambiguity and gives a fully rigorous passage to the limit.

For part (d), one can also derive $E[T_b]=b/(2p-1)$ from Wald's identity: $$E[S_{T_b\wedge m}]=E[T_b\wedge m]\,E[\xi_1],$$ then let $m\to\infty$ using monotone convergence and the fact that $S_{T_b}=b$ almost surely when $p>1/2$.

QED.