Stochastic Processes – Problem 40: Prove that form a Markov chain

Let $\beta>0$ and $n\geq1$. Let $X_{1},\ldots,X_{n}$ be random variables taking values in $\{-1,1\}$ such that

$$\mathbb{P}(X_{i}=\sigma_{i}\text{ for }i=1,\dots,n)=\frac{1}{Z}e^{\beta\sum_{i=1}^{n-1}\sigma_{i}\sigma_{i+1}}\,.$$

The above holds for any sequence $(\sigma_{i})_{i}$ taking values $\pm 1$; here $Z$ is a normalizing constant ensuring $P$ has total mass 1 ($Z$ does not depend on $(\sigma_{i})_{i}$).

(a) Prove that $X_{1},\ldots,X_{n}$ form a Markov chain.

(b) Compute its transition probabilities.

(c) Compute $E(X_1 X_n)$.

Step 1. (a) Prove that $X_1, \ldots, X_n$ form a Markov chain. To verify the Markov property: $$\mathbb{P}(X_{k+1} = \sigma_{k+1} \mid X_k = \sigma_k, \dots, X_1 = \sigma_1) = \mathbb{P}(X_{k+1} = \sigma_{k+1} \mid X_k = \sigma_k)$$ From the joint distribution: $$\mathbb{P}(X_1=\sigma_1, \dots, X_k=\sigma_k) = \frac{1}{Z_k} e^{\beta \sum_{i=1}^{k-1} \sigma_i \sigma_{i+1}}$$ The conditional probability is: $$\mathbb{P}(X_{k+1}=\sigma_{k+1} \mid X_1=\sigma_1, \dots, X_k=\sigma_k) = \frac{Z_k}{Z_{k+1}} \exp(\beta \sigma_k \sigma_{k+1})$$ This depends only on $\sigma_k$ and $\sigma_{k+1}$, not on $\sigma_1, \dots, \sigma_{k-1}$, so the Markov property holds.

(b) The transition probability is $P(\sigma_{k+1} \mid \sigma_k) = C(\sigma_k) e^{\beta \sigma_k \sigma_{k+1}}$ where $C(\sigma_k)(e^{\beta} + e^{-\beta}) = 1$, giving $C = \frac{1}{2\cosh\beta}$. Thus: $$P(\sigma_{k+1} \mid \sigma_k) = \frac{e^{\beta \sigma_k \sigma_{k+1}}}{2 \cosh \beta}$$

(c) The one-step conditional expectation is $\mathbb{E}(X_{k+1} \mid X_k) = X_k \tanh\beta$. By iteration, $\mathbb{E}(X_n \mid X_1) = X_1 (\tanh\beta)^{n-1}$. Since $X_1^2=1$: $$\mathbb{E}(X_1 X_n) = (\tanh \beta)^{n-1}$$

(a) QED. (b) $P(\sigma_{k+1} | \sigma_k) = \frac{e^{\beta \sigma_k \sigma_{k+1}}}{2 \cosh \beta}$ (c) $(\tanh \beta)^{n-1}$