ISYE 3232B Fall 2013 Test 2 - 01

I,
, do swear that I abide by the Georgia Tech Honor Code. In case
of honor code violation,I will accept an F in this course.
Signature:
You will have 1 hour 15 minutes.
This exam is closed book and closed notes. Calculators are not allowed. No scrap paper is allowed.
Make sure that there is nothing on your desk except pens and erasers.
If you need extra space, use the back of the page and indicate that you have done so.
Do not remove any page from the original staple. Otherwise, there will be 3 points off.
We will not select among several answers. Make sure it is clear what part of your work you want
graded. If two answers are given, zero point will be given for the problem.
Throughout, you will receive full credit if someone with no understanding of probability, set theory,
and calculus could simplify your answer to obtain the correct numerical solution. However, you must
give a numerical answer where asked.
Discrete-Time Markov Chain (DTMC)
Pnij = Pr{Xn = j | X0 = i}.
Stationary probabilities i are (i) the long-run proportion of time that a Markov chain is in state i and
(ii) nonnegative solutions of
X
X
j =
i Pi j and
j = 1.
(1)
all i
all j
Exponential Distribution
An exponential random variable X has f (x) = ex for x 0 and F(x) = 1 ex for x 0. Mean is
1/ and variance is 1/2 . It has a memoryless property that Pr{X > s + t | X > t) = Pr{X > s}.

1. (15 points - no partial points) Let X represent a discrete-time Markov chain with state S = {1, 2} and
one-step transition matrix
!
0.6 0.4
P=
.
0.2 0.8
The initial distribution is given as (0.1, 0.9). Calculate the following quantities.
(a) (3 points) Pr(X10 = 2 | X9 = 1).

(b) (3 points) Pr(X2 = 2 | X0 = 1).

(c) (3 points) Pr(X1 = 2).

(d) (3 points) Pr(X0 = 1 | X1 = 2).

(e) (3 points) E[X10 | X9 = 1].

2. (15 points) A Markov chain has state space {1, 2, 3, 4} and the following one-step transition matrix,

0.0 0.0 0.0

0.0 0.0 0.0
P =
0.5 0.5 0.0
0.0 0.0 1.0

1.0
1.0
0.0
0.0

(a) (3 points) Classify the states as recurrent or transient.

(b) (3 points) Give the period of each recurrent state.

(c) (6 points) Does a unique stationary distribution exist? If not, explain why. If yes, set up all
necessary equations to find (Do not solve the equations. Just set them up.)

(d) (3 points) Does P exist? If not, explain why. If yes, give P .

3. (13 points) For the Markov chain with state space S = {1, 2, 3, 4} and one-step transition matrix

0.1 0.0 0.3

0.0 0.5 0.0
P =
0.0 0.0 0.5
0.0 0.0 0.3

0.6
0.5
0.5
0.7

(a) (3 points) Classify the states as recurrent or transient.

(b) (10 points) Calculate P .

4. (5 points) A discrete-time Markov chain with state space S = {1, 2, 3, 4} has the steady-state probabilities (0.2, 0.3, 0.4, 0.1). In steady-state, on average how many transitions would it take to return to
state 2 given that the system is currently in state 2? (A numerical answer is expected.)

5. (15 points) Consider a discrete time Markov chain on state space {1, 2, 3, 4, 5} with transition matrix

0
0
0.7 0.3 0

0
0
0.4 0.6 0

0 0.2 0.8 0
P = 0

0 0.5 0.5 0
0

0.2 0.1 0.2 0 0.5

Part of the limiting distribution P is given below:

?
?
?
4/7 3/7

?
?
?
?
?

? 5/13 8/13 ?
P = ?

?
?
?
?
?

(a) (b) (c)

(d) (e)
Calculate the values of (a)(e) (i.e., the values in the 5th row in P ).

6. (10 points) A discrete-time Markov chain has the following diagram with state space S
{0, 1, 2, 3, 4, . . .} (infinite number of states).

Given that p = 0.8 and q = 0.2, calculate i for i = 0, 1, 2, 3, 4, . . ..

7. (5 points) Littlefield earns $1500 if an order is delivered in 2 days and $1000 if it is delivered in 4
days. Currently it takes exponential time with mean 1/4 day to deliver a customers order from the
time when the order is received. What is the probability that Littlefield earns $1500 for an order?

8. (22 points) An airline reservation system runs by two computer servers A and B. Both A and B last
exponential time with mean 6 months.
(a) (5 points) If the reservation system is up only when both servers are up, what is the expected up
time of the reservation system? (A numerical answer is expected.)

(b) (5 points) If the reservation system is up when at least one server is up, what is the expected up
time of the reservation system? (A numerical answer is expected.)

(c) (6 points) Now suppose that the reservation system is up when at least one server is up and an
extra server C is purchased as a back up. It will be installed as soon as one of the servers (A
or B) goes down. A down server will not be fixed but sent to a manufacturer. Server C lasts
exponential time with mean 6 months. What is the probability that server C goes down last? (A
numerical answer is expected.)

(d) (6 points) Continued from (c), what is the expected up time of the reservation system when
server C is available as a back up? (A numerical answer is expected.)