]>
Suppose that is an interval of integers, either finite or infinite. A birth-death chain on is a Markov chain on with transition probability matrix of the form.
where , , and are nonnegative functions on with for . If the interval has a minimum value then of course we must have . If , the boundary point is said to be absorbing and if , then is said to be reflecting. Similarly, if the interval has a maximum value then of course we must have . If , the boundary point is said to be absorbing and if , then is said to be reflecting. Several other special models that we have studied are birth-death chains:
Describe each of the following as a birth-death chain.
If is finite, classification of the states as recurrent or transient is simple, and depends only on the state graph. In particular, if the chain is irreducible, then the chain is recurrent. In this paragraph, we will study the recurrence and transience of birth-death chains when . We assume that for all and that for all (but of course we must have ). Thus, the chain is irreducible. We will use the test for recurrence derived earlier with , the set of positive states. Essentially, we will compute the probability that the chain never hits 0, starting in a positive state.
Show that the functional equation for an unknown function on is equivalent to the following system of equations:
Use the result in the previous exercise to show that
Use the result in the previous exercise to show that
Now use the test for recurrence to conclude that the chain is recurrent if and only if
Note that , the function that assigns to each state the probability of an immediate return to , plays no direct role in whether the chain is transient or recurrent. Indeed all that matters are the ratios for
Suppose that and (and hence ) are constant on . Show that the chain is recurrent if and transient if .
Show that the birth-death chain is positive recurrent if and only if
in which case the invariant probability density function is given by
Note again that , the function that assigns to each state the probability of an immediate return to , plays no direct role in whether the chain is transient or null recurrent, or positive recurrent.
In the positive recurrent case, show that the birth-death chain is reversible.
Suppose that is constant on and is constant on . Prove the following results: