Stochastic Processes – Problem 47: Prove that is a Markov chain and find its invariant distribution

Let $X_{t}$ be a jump process on state space $S$ with transition rate matrix $Q$, assumed to be positive recurrent with invariant distribution $\pi$. Let $S_{n}$ be an independent Poisson process with rate $\lambda$, i.e., $S_{n}=Z_{1}+\cdots+Z_{n}$ where $Z_{i}$ are i.i.d. exponential random variables with parameter $\lambda$. Prove that $Y_{n}:=X_{S_{n}}$ is a Markov chain and find its invariant distribution.

Step 1. Let $(X_t)_{t\ge0}$ be a positive recurrent continuous-time Markov chain with transition semigroup $$P_t(i,j)=P(X_t=j\mid X_0=i),\qquad i,j\in S,$$ and invariant distribution $\pi$, so $\pi P_t=\pi$ for every $t\ge0$.

Step 2. Let $Z_1,Z_2,\dots$ be i.i.d. $\operatorname{Exp}(\lambda)$, independent of $X$, and define $$S_n=Z_1+\cdots+Z_n, \qquad Y_n:=X_{S_n}.$$ We compute the one-step kernel of $Y$: $$K(i,j):=P(Y_{n+1}=j\mid Y_n=i) =\int_0^\infty P_t(i,j)\,\lambda e^{-\lambda t}\,dt.$$ This formula follows from conditioning on $Z_{n+1}$, and it is independent of $n$.

Step 3. Markov property. Given $Y_n=i$, the future state is $Y_{n+1}=X_{S_n+Z_{n+1}}$. By independence of $Z_{n+1}$ and the strong Markov property of $X$ at time $S_n$, the conditional law of $Y_{n+1}$ depends only on $Y_n=i$, not on $(Y_0,\dots,Y_{n-1})$. Therefore $(Y_n)$ is a time-homogeneous discrete-time Markov chain with kernel $K$.

Step 4. Invariance of $\pi$. For each $j\in S$, $$\sum_{i\in S}\pi(i)K(i,j) =\sum_i\pi(i)\int_0^\infty P_t(i,j)\lambda e^{-\lambda t}dt =\int_0^\infty\Bigl(\sum_i\pi(i)P_t(i,j)\Bigr)\lambda e^{-\lambda t}dt.$$ Since $\pi P_t=\pi$, the inner sum equals $\pi(j)$. Hence $$\sum_i\pi(i)K(i,j) =\int_0^\infty \pi(j)\lambda e^{-\lambda t}dt =\pi(j).$$ So $\pi$ is invariant for $Y$.

Conclusion: $Y_n=X_{S_n}$ is a discrete-time Markov chain with transition matrix $$K=\int_0^\infty P_t\,\lambda e^{-\lambda t}dt,$$ and invariant distribution equal to the same $\pi$.

QED.