Академический Документы
Профессиональный Документы
Культура Документы
Winter 2014
The state i comunicates with the state j if it is possible for the chain to go from state i to j in any number
of steps. If i communicates with j and j communicates with i, then the two states intercomunicate.
A state i is persistent if
P chain eventually returns to state i X0 = i = 1.
Denote by Ti the time it takes for the chain to return to state i, given that it begins in state i. That is,
Ti = min n such that Xn = i|X0 = i .
If i is a persistent state, then Ti is a random variable on Z+ , the set of positive integers. If E[Ti ] exists,
we say that the state i is non-null or positive, and we denote E[Ti ] = i Otherwise, i is null persistent.
A subset C of the state space S is called closed if pij = 0 for all i C and all j S \ C.
S = T C1 C2 ,
where T is the set of transient states and each Ci is a closed and irredudible set of persistent states.
For finite state spaces, at least one state is persistent, and all persistent states are non-null.
A stationary distribution
~ = (0 , 1 , 2 , . . . )
is a probability distribution on Xn such that if Xn has distribution ~ , then so does Xn+1 . That is,
~ P = ~ .
1
Stationary distributions exist uniquely whenever all states are positive persistent, in which case j = j
.
For an irreducible ergodic Markov chain, limn Pn exists and is equal to the matrix for which all rows
are ~ .