OPR 330 Review

1. A transit company owns 100 train cars. If a car is operable today, the probability of it being
operable tomorrow is 0.9 and the probability of it being inoperable tomorrow is 0.1. If it is
inoperable today, it will be inoperable tomorrow with a probability of 0.25 and operable with a
probability of 0.75. In the long run (steady-state), how many cars should we expect to be
operable on any particular day?
Let Xt = {0 if car is not operable , 1 if car is operable}
P (Xt+1 = 0 | Xt = 0} =0.25
P (Xt+1 = 1 | Xt = 0} =0.75
P (Xt+1 = 0 | Xt = 1} =0.1
P (Xt+1 = 1 | Xt = 1} =0.9

Using the above, equations we can build the following equations

0 1
P (transition matrix) = 0 0.25 0.75
1 0.1 0.9

[𝜋0 𝜋1 ] = [𝜋0 𝜋1 ] [0.25 0.75]

0.1 0.9

𝜋0 = 𝜋0 0.25 + 𝜋1 0.1
𝜋1 = 𝜋0 0.75 + 𝜋1 0.9

𝜋0 + 𝜋1 = 1

Using the above equations,

We get 0.75 𝜋0 = 𝜋1 0.1

0.75 𝜋0 = (1 − 𝜋0 )0.1

2
𝜋0 = 17

15
Thus, 𝜋1 = 17

The steady state probability for car is operable = 15/17 which will be same for all cars.
Number of cars to be operable = 15/17 *100

2. The employees at the transit company can be classified in several categories: non-management,
management, quit/fired, or retired. The transition matrix P is
Retired Mgmt Non-Mgmt Quit/Fired
Retired 1 0 0 0
Mgmt 0.1 0.7 0.05 0.15
Non-Mgmt 0.05 0.15 0.6 0.2
Quit/Fired 0 0 0 1
a. Find the matrix I-Q that would be used to calculate the matrix NR.

b. If (I-Q)-1 is 3.556 0.444 then find NR.

1.333 2.667

c. If there are currently 50 people employed in a management role, how many should we
expect to eventually quit or be fired?
 Retired Mgmt Non-Mgmt Quit/Fired
Retired 1 0 0 0
Mgmt 0.1 0.7 0.05 0.15
Non-Mgmt 0.05 0.15 0.6 0.2
Quit/Fired 0 0 0 1

Retired and Quit/Fired states are absorbing states since it is impossible to leave it.
Mgmt and Non – Mgmt are transient states.

Using the transition matrix, we rearrange it in the above format. Use the transition matrix to
refill by using the same values. For example, Cell Mgmt Mgmt will have the same value as the
Mgmt Mgmt cell in the above transition matrix

Mgmt Non- Retired Quit/Fired

Mgmt
Mgmt 0.7 0.05 0.1 0.15
Non-Mgmt 0.15 0.6 0.05 0.2
Retired 0 0 1 0
Quit/Fired 0 0 0 1

0.7 0.05 0.1 0.15

Comparing it to the canonical form, we get Q = [ ], R = [ ]
0.15 0.6 0.05 0.2

1 0
I=[ ]
0 1

1 0 0.7 0.05 0.3 −0.05

I–Q=[ ]-[ ] =[ ]
0 1 0.15 0.6 −0.15 0.4

3.556 0.444
b) N = (I – Q) -1 = [ ]
1.333 2.667

3.556 0.444 0.1 0.15 0.378 0.622

NR = [ ][ ]=[ ] (multiply both matrices to get the
1.333 2.667 0.05 0.2 0.267 0.733
answer)
 3. A system with absorbing states A1, A2, and A3 and non-absorbing states T1, T2, and T3 is under
consideration. Below is a portion of the matrix NR:

NR A1 A2 A3
T1 0.5 0.4
T2 0.1 0.3
T3 0.4 0.2

a. Find the steady-state probability of ending up in state A2 if you begin in state T1.
b. Find the steady-state probability of ending up in state A2 if you begin in state T2.
c. Find the steady-state probability of ending up in state T1 if you begin in state T3.
d. Find the steady-state probability of ending up in state A3, if you begin in state A1.

NR A1 A2 A3
T1 0.5 0.4 = 1-(0.5+0.4) =0.1
T2 0.1 =1 – (0.4+0.3) =0.3 0.3
T3 0.4 0.2 =1 –(0.4+0.2) = 0.4

The net probability for every state should be 1. Using that the above table was filled up.
a) Look for T1 row and A2 column = 0.4
b) Look for T2 row and A2 column = 0.3
c) Both are non- absorbing states. It is not possible to move. P =0
d) Both are absorbing states. It is not possible to move. P=0

4. A manufacturer must meet production demands for the next three months. The monthly
demands, production costs, and maximum production levels are given below. We can use a
product produced in the current month to meet that month’s demand, and we can also hold
product from one month to another at a cost of $1/unit. At most 2 units can be held per month.
 Demand Production Cost/Unit Maximum Production
January (Stage 3) 2 $2 3
February (Stage 2) 4 $3 5
March (Stage 1) 6 $5 8
Let xj be the number of units held in storage as we enter stage j and dj be the number of units
that we product in stage j. Use dynamic programming to find the optimal decision in stage 2 for
x2 = 0 and for x2 = 1. Assume that there will be no demand for our product after March. The
following tables may be helpful in arranging your thoughts.

Stage 2 Stage 1
x2 d2 r2 f2 x1 x1 d1 r1=f1
0

Stage 1
Demand = 6, Max production =8

Max goods that could be held from stage 2& 3 = Maximum production – Demand = 3+5 – (2+4)
=2

Thus, d1 can be either 0,1,2

Since the product doesn’t have any demand after March,

x1 + d1 = demand = 6

ri= fi = production cost only as the holding cost was paid the month before

Using the above info, we fill the table below

x1 d1 r1=f1
0 6 6*5=30
1 5 5*5=25
2 4 4*5=20

Stage 2

Demand = 4, Max production =5

Max goods that could be held from stage 3 = Maximum production – Demand = 3– 2 = 1
Thus, d2 can be 0,1
 Max value of x1 = Maximum production – Demand + Max value of d1 = 5-4 +1 =2

Thus, x1 can be 0,1,2

x2 + d2 – x1 = demand = 4

f2 = r2 + r1
f2 when d2 is 4 = 12 + 30 since x1 =0, thus d1 =6 and r1=30
x2 d2 r2 f2 x1
0 4 12 12 + 30=42 0
5 15+1 16+ 25=41
1 3 9 9+30=39 0
4 12+1 13+25=38 1
5 15+2 17+20=37 2

Choose the lowest cost for each case. I have highlighted in bold

5. An advertising agency is conducting a 10 day campaign for a local department store. The agency
determined that the most effective campaign would possibly including placing ads in four
media: internet, print, radio, and television. A total of $8000 has been made available for this
campaign, and the agency would like to distribution this budget in $1000 increments across the
media in such a fashion that an advertising exposure index is maximized:

Suppose that we use dynamic programming to determine how to maximize the department
store’s exposure.

a. Define the problem’s stages.

The stages are the 4 types of media: Internet, Print, Radio and Television

b. Define the problem’s decisions.

Let xi be the amount of money allocated to the media i with i=1,2,3,4 with 1 representing internet, 2
for print, 3 for radio and 4 for television. The decision variable would be x1, x2,x3, x4
c. Define the problem’s states.
The amount of money we have remaining post the allocation of a budget to the type of media would be
the respective state of the media

Exposure $1000 $2000 $3000 $4000 $5000 $6000 $7000 $8000

Internet 24 37 46 59 72 80 82 82
Print 15 55 70 75 90 95 95 95
Radio 20 30 45 55 60 62 63 63
Television 20 40 55 65 70 70 70 70