Homework 4 for Graduates, Due 04/02

Write up your solutions neatly and coherently. Show all work to qualify for full credits

1. (10 points) You are given the following information about a homogeneous Markov chain:

• Planes move among three states servicing flights: On-time (State 0), Delayed (State
1), and Canceled (State 2).
 
0.8 0.1 0.1
• P = 0.7 0.0 0.3.
0.8 0.2 0.0

• The plane is currently in the state Canceled (State 2) at time 0.

Find the probability that the plane will transit to every states by three transitions, e.g. State
1 at time 1, State 0 at time 2, and State 2 at time 3.

2. (10 points) Assume workers transition through the labor force independently with the
transitions following a homogeneous Markov chain with three states:

• Employed full-time

• Employed part-time

• Unemployed
 
0.90 0.07 0.03
The transition matrix is: 0.05 0.80 0.15.
0.15 0.15 0.70
• Worker Y is currently employed full-time

• Worker Z is currently employed part-time

Find the probability that either Y or Z, but not both will be unemployed after two transitions.

3. (10 points) Immigration depends upon the growth rate of the economy, which follows the
non-homogeneous Markov chain described below:

• State 1: Rapid Growth

• State 2: Normal Growth

• State 3: Slow Growth

• In a year of Rapid Growth, immigration is +10, 000.

1
 • In a year of Normal Growth, immigration is +4, 000.
• In a year of Slow Growth, immigration is zero.
• The following matrices show the probabilities of economy transitions between states
from time t = 0 to time t = 1 and from time t = 1 to time t = 2.
   
0.85 0.10 0.05 0.15 0.65 0.20
Q0 =  0.3 0.5 0.2  ; Q1 = 0.35 0.40 0.25
0.0 0.9 0.1 0.05 0.85 0.10

• The economy just finished a year of Rapid Growth at time 0.

Find the expected total immigration in the next two years.

4. (10 points) You are given the following homogeneous Markov chain with state space
{1, 2, 3, 4, 5, 6, 7} and transition probability matrix:
 
0.5 0.0 0.5 0.0 0.0 0.0 0.0
 0.2 0.4 0.2 0.2 0.0 0.0 0.0
 
0.25 0.0 0.75 0.0 0.0 0.0 0.0
 
P =  0.0 0.2 0.2 0.2 0.2 0.0 0.2

 0.0 0.0 0.0 0.0 2 1
0.0
 3 3 
 0.0 0.0 0.0 0.0 0.5 0.5 0.0
0.0 0.0 0.0 0.0 0.0 0.0 1
(a) Identify its communicating classes.
(b) Classify closed and non-closed classes.
(c) Classify states in recurrence and transience.

5. (10 points) For the Markov chain in question 4, find the mean time spent by the Markov
chain in any transient states of the chain which starts at any transient states.

6. (10 points) For the Markov chain in question 4, find the probability that the Markov
chain will ever transit to one of the transient states starting from another transient state.

7. (10 points) You are given the following Markov chain transition probability matrix:
 
0.2 0.8 0.0 0.0 0.0
0.5 0.5 0.0 0.0 0.0
 
P = 0.2 0.3 0.0 0.3 0.2
0.0 0.0 0.0 1.0 0.0
0.0 0.0 0.4 0.0 0.6

2
Find the mean time spent by the Markov chain in any transient states of the system which
starts at any transient states.

8. (10 points) For the Markov chain in question 7, find the probability that the Markov
chain will ever transit to one of the transient states starting from another transient state.