Custom Homework

Chapter 1

Problem C1.1

Consider the Markov chain with state space $S = \{0, 1, 2, \ldots \}$ and transition probabilities: \[ p(i, j) =\begin{cases} p, &j = i + 1\\ q, &j = 0\\ 0, &\text{otherwise} \end{cases} \] where $p, q > 0$ and $p + q = 1.1$. This Markov chain counts the lengths of runs of heads in a sequence of independent coin tosses. Let \[ T_y = \min\{n\ge 1: X_n=y\} \] be the time of the first return to $y$.

  1. Show that the chain is irreducible.
  2. Find $P_0(T_0 = n)$ for $n = 1, 2, \ldots$. What is the name of this distribution?
  3. Is the chain recurrent? Explain.