When states and , , of a Markov chain can be reached from one another (that is, there is a positive probability of transitioning to from , and a positive probability of transitioning to from ; note these needn’t be one-step transitions), we say that the two states communicate. Note that every state communicates with itself, as if we’re in state we’ll be in state after zero transitions. Communication of states forms an equivalence relation, and so when we say two states belong to the same class, we mean that they belong to the same equivalence class with this relation. If a chain has only one class of states, the chain is called irreducible.

For instance, consider the chain that simply moves from state one to state two, from state two to state three, then from state three back to state one, and the process repeats. Such a chain would have a transition probability matrix of

This chain has only one class and so is irreducible.

The chain with transition matrix

has two classes, and . The first three states communicate with one another as if you’re any one of , then there is a positive probability of transitioning to any of . Notice that we *can *transition from state two to state one by going through state three first. State four is in a class by itself, though, as state four can only communicate with itself: Once you’re in state four, you stay in state four with probability one.

### Like this:

Like Loading...

*Related*

## Leave a Reply