Академический Документы
Профессиональный Документы
Культура Документы
Fall 2015
Homework 6 - Solutions
October 2, 2015
1. Suppose each morning a factory post the number of days worked in a row without any injuries. Assume
that each day there is injury free with probability 99/100. Let X0 = 0 be the morning the factory first
opened. Let Xn be the number posted on the morning after n full days of work. Is X0 , X1 , . . . a
Markov chain? If so, give its state space, initial distribution, and transition matrix P . If not, show
that it is not a Markov chain.
Note that Xn is the number of consecutive injury-free days posted on the morning after n full days of
work. Lets work on a table of Xn first. Let In = 1 if injury-free on the nth day and In = 0, otherwise.
Xn
0
1
2
..
.
Xn+1
1
0
2
0
3
0
..
.
So from the above table, you can see that Xn+1 is a function of Xn and In+1 as follows:
Xn + 1, if In+1 = 1
Xn+1 =
0,
if In+1 = 0
This proves that Xn is a DTMC.
State space is S = {0, 1, 2, . . .}.
The initial distribution is given by
a(0) = (1, 0, 0, . . .)
because it is given that X0 = 0.
Now we find the transition matrix. Note that
P (Xn+1 = j + 1|Xn = j) = P (In+1 = 1) = 0.99 and
P (Xn+1 = 0|Xn = j) = P (In+1 = 0) = 0.01.
Therefore the transition probability
0.01
0.01
P = 0.01
..
.
matrix is
0.99 0
0
0.99
0
0
..
..
.
.
0
0
0.99
..
.
0
0
0
..
.
...
...
...
...
2. Let X0 , X1 , . . . be a Markov chain with state space {0, 1, 2}, initial distribution a = (?, 0.4, 0.5), and
transition matrix
.4 ? .5
P = .3 ? .1
? .2 .5
Fill in the entries for P and a, and compute the following:
Moreover, if the probability of moving from i to j in one time step is Pr(Xn+1 = j | Xn = i) = pij ,
then each row in the transition matrix must sum up to one, i.e.
X
pij = 1.
j
.4
P = .3
.3
.1
.6
.2
.5
.1
.5
.34
P = .33
.33
2
.2 .46
.41 .26
.25 .42
Pr(X10 = 1 | X8 = 2, X7 = 0) Pr(X8 = 2 | X7 = 0)
Pr(X10 = 1 | X8 = 2) Pr(X8 = 2 | X7 = 0)
(2)
(2)
Note that p21 is the (2, 1)th element in P 2 and thus from above p21 = 0.25.
c)
Pr(X1 = 0)
2
X
Pr(X1 = 0, X0 = i)
i=0
2
X
Pr(X1 = 0 | X0 = i) Pr(X0 = i)
i=0
p00 a0 + p10 a1 + p20 a2 = 0.4 0.1 + 0.3 0.4 + 0.3 0.5 = 0.31.
2
d) Pr(X0 = 2|X1 = 2) =
Pr(X1 = 2)
2
X
(0)
P22 a2
Pr(X1 =2)
0.50.5
0.34
0.735.
Pr(X1 = 2, X0 = i)
i=0
2
X
Pr(X1 = 2 | X0 = i) Pr(X0 = i)
i=0
p02 a0 + p12 a1 + p22 a2 = 0.5 0.1 + 0.1 0.4 + 0.5 0.5 = 0.34.
P2
P2
2
= 0 .33 + 1 0.25 + 4 0.42 = 1.93
e) E X22 |X0 = 2 = i=0 i2 Pr(X2 = i|X0 = 2) = i=0 i2 P2i
P2
f) E X22 = i=0 E X22 |X0 = i Pr(X0 = i) = 2.04 0.1 + 1.45 0.4 + 1.93 0.5 = 1.749.
P2
P2
2
= 0 .34 + 1 0.2 + 4 0.46 = 2.04
E X22 |X0 = 0 = i=0 i2 Pr(X2 = i|X0 = 0) = i=0 i2 P0i
2
P2 2
P2 2 2
E X2 |X0 = 1 = i=0 i Pr(X2 = i|X0 = 1) = i=0 i P1i = 0 .33 + 1 0.41 + 4 0.26 = 1.45
From part (e), E X22 |X0 = 2 = 1.93
g) Pr(X0 = 2|X0 = 0) = 0, since the Markov chain cannot be at two different states at the same time.
h)
Pr(X2 = 1|X0 = 2)
2
X
Pr(X2 = 1, X1 = i|X0 = 2)
i=0
2
X
Pr(X2 = 1 | X1 = i, X0 = 2) Pr(X1 = i | X0 = 2)
i=0
2
X
Pr(X2 = 1 | X1 = i) Pr(X1 = i | X0 = 2)
i=0
2
X
pi1 p2i
i=0
=
=
=
=
Pr(X1 = 1, X0 = 0, X2 = 2)
Pr(X0 = 0, X2 = 2)
Pr(X2 = 2 | X1 = 1, X0 = 0) Pr(X1 = 1, X0 = 0)
Pr(X2 = 2 | X0 = 0) Pr(X0 = 0)
Pr(X2 = 2 | X1 = 1) Pr(X1 = 1 | X0 = 0) Pr(X0 = 0)
Pr(X2 = 2 | X0 = 0) Pr(X0 = 0)
p12 p01 a0
0.1 0.1
1
=
=
0.0217.
(2)
0.46
46
p a0
02
k) The only possible combination of (X0 , X1 , X2 ) to get their product equal to 1 is (1, 1, 1). Thus,
Pr(X0 X1 X2 = 1)
Pr(X2 = 1 | X1 = 1, X0 = 1) Pr(X1 = 1, X0 = 1)
3. A six-sided die is rolled repeatedly. After each roll n = 1, 2, ..., let Xn be the largest number rolled in
the first n rolls. Is {Xn , n 1} a discrete-time Markov chain? If its not, show that it is not. If it is,
what is the state space and the transition probabilities of the Markov chain?
The following table shows how Xn changes.
Xn
1
..
.
Xn+1
1
2
3
4
5
6
2
2
3
4
5
6
3
3
3
4
5
6
..
.
Since Xn+1 = max{Xn , Un+1 } where {Un : n 1} is an i.i.d. sequence of uniform distribution on
{1, 2, 3, 4, 5, 6}, {Xn : n 1} is a Markov chain with state space S = {1, 2, . . . , 6} and transition
probability
Pr{Xn+1 = i | Xn = i} = Pr{Un+1 i} = i/6,
Pr{Xn+1 = j | Xn = i} = 0,
j < i
1
6
P =
0
0
0
0
0
1
6
2
6
0
0
0
0
1
6
1
6
3
6
0
0
0
1
6
1
6
1
6
4
6
0
0
1
6
1
6
1
6
1
6
5
6
1
6
1
6
1
6
1
6
1
6
6
6
j > i
4. In a study of the effect of advertising on brand shifting in the U.S. brewing industry, a Markov chain
model was estimated to represent the chance of consumers shifting preferences among Anheuser-Busch
(state 1), Miller (state 2) and other (state 3) beers from 1978 to 1979. The one-step transition
matrix is
0.9940
0.0000
P =
0.0084
0.0046
0.9981
0.0526
0.0014
0.0019
0.9390
Clearly brand loyalty is quite strong since the diagonal elements are all near 1. Suppose that there
were 6 million beer drinkers in 1978, divided equally among the three brands. Answer the following
questions:
If the transition matrix does not change over time (typically it does) and the number of beer drinkers is
unchanged (certainly not true), how many consumers are expected to prefer Anheuser-Busch products
in the long-run?
Solve P =
1
0.9941 + 0.00843
1 + 2 + 3
1.
0.0418 0.9283
0.0418 0.9283
0.0418 0.9283
0.0299
0.0299
0.0299
, which means, in the long run, 6 0.0418 = 0.2508 million customers prefer Anheuser-Busch.