MS&E 221 Final Examination

Ramesh Johari March 20, 2009

Instructions
1. Take alternate seating.
2. Answer all questions in the blue examination books. Answers given on any other paper will
not be counted.
3. The examination begins at 8:30 am, and ends at 11:30 am.
4. No notes, books, calculators, or other aids are allowed.
5. Show your work! Partial credit will be given for correct reasoning.
Honor Code
In taking this examination, I acknowledge and accept the Stanford University Honor Code.
NAME (signed)
NAME (printed)
1
Useful Formulas
1. If X is a geometric random variable with parameter p, then the pmf of X is:
P(X = k) = (1 p)
k1
p, k 1.
2. If X is a binomial random variable with parameters p and n, then the pmf of X is:
P(X = k) =

n
k

p
k
(1 p)
nk
=
n!
k!(n k)!
p
k
(1 p)
nk
, 0 k n.
3. If T is an exponentially distributed random variable with mean 1/, then the density of T is
given by f
T
, where:
f
T
(t) = e
t
, t 0.
4. If S is a random variable with a gamma distribution of parameters n and , where n is a
positive integer, then the density of S is given by f
S
, where:
f
S
(s) =
e
s
(s)
n1
(n 1)!
, s 0.
5. If N is a Poisson random variable of parameter , then the pmf of N is:
P(N = k) = e

k
k!
, k 0.
6. For an M/M/1 queue with arrival rate and service rate with < , the equilibrium
distribution is:
P(Q = j) = (1 )
j
, j 0,
where = /.
7. For an M/M/queue with arrival rate and service rate , the equilibrium distribution is:
P(Q = j) =
e

j
j!
, j 0,
where = /.
8. For an M/M/K/K queue with arrival rate and service rate , the equilibrium distribution
is:
P(Q = j) =

j
/j!
1 + +
K
/K!
, 0 j K,
where = /.
2
Part A (5 questions; 5 points each; 25 points total)
Answer each of the following short answer questions. Explain your answers!
1. Let {X
n
} and {Y
n
} be independent irreducible discrete time Markov chains on a nite
state space {1, . . . , N} with the same invariant distribution ((1), . . . , (N)). Suppose
both chains start in state 1 at time 0. Let T be the rst time strictly larger than zero that
X
T
= Y
T
= 1. What is E[T]?
2. Suppose X
1
, X
2
, . . . are independent and identically distributed random variables that take
values in the nonnegative integers. Dene Y
n
= max{X
1
, . . . , X
n
}. Are there any condi-
tions under which {Y
n
} has a recurrent communicating class?
3. Suppose that the rate matrix Q of a continuous time Markov chain is symmetric, i.e., it
satises Q
ij
= Q
ji
. Does that imply the transition matrix of the corresponding jump chain
is symmetric as well?
4. Can a discrete time Markov chain on a nite state space have any null recurrent states?
5. Consider a continuous time Markov chain on two states A and B, with transition rate = 1
fromAto B, and transition rate = 2 fromB to A. Let T be the last time the chain changed
state before time t = 1, 000, 000, and let S be the rst time the chain changes state after time
t = 1, 000, 000. Give an approximate value for the mean of S T.
3
Part B (3 questions; 75 points total)
Problem 1 (5 points per part; 25 points total). A company has manufacturing plants in three
cities A, B, and C, and two inspectors. Inspector i requires an exponentially distributed amount
of time with mean 1/
i
(j) to inspect plant j. Assume that when an inspector nishes inspecting a
plant, they immediately visit the plant the other inspector is not currently visiting.
Assume that
1
(A) =
1
(B) =
2
(B) =
2
(C) = 1, and that
1
(C) =
2
(A) = 2.
(a) Give an appropriate state space to describe the movement of the two inspectors, and draw the
corresponding state transition diagram. Which states are recurrent, and which are transient?
(b) Give the jump chain transition matrix corresponding to the model in part (a). Which states
are recurrent, and which are transient? Which are aperiodic?
(c) Without doing any calculations, explain why a unique invariant distribution exists for the
jump chain. Is an invariant distribution for the continuous time chain?
(d) Assume inspector 1 is currently at the plant in city A, and inspector 2 is currently at the plant
in city B; call these their initial plants. Write a system of equations that can be solved to
nd the probability both inspectors visit the plant in city C at least once before the rst time
at which both inspectors simultaneously nd themselves back at their initial plants.
(e) Find the long run fraction of time that inspector 1 spends at the plant in city C.
4
Problem 2 (5 points per part; 30 points total). A store has a parking lot with N spaces, which
are numbered 1, 2, . . . , N; the number of the parking spot denotes its distance from the front door
of the store. Cars arrive according to a Poisson process of rate . Upon arrival, a car parks in the
lowest numbered parking spot that is available; if the parking lot is full, the car leaves immediately.
Assume that each parked car spends an exponentially distributed amount of time with mean 1/
in the parking lot, independently of other cars.
(a) For this part only, suppose that N = 1. What is the long run fraction of time that the single
parking spot is full?
(b) For parts (b) and (c) only, assume that N = 2. What is the long run fraction of time that the
rst parking spot is full?
(c) Assume the parking lot is in equilibrium. If an arriving car is able to park, what is the mean
distance from the car to the front door of the store?
(d) For general N, what is the long run fraction of arriving cars that are turned away?
(e) For 1 n N, let X
n
(t) denote the number of free parking spaces at time t among the
spaces numbered 1, 2, . . . , n. Thus, for example, if X
4
(t) = 2, it means that two of the four
spaces 1, 2, 3, 4 are free. For each n, is {X
n
(t)} a continuous time Markov chain? Explain
your answer.
(f) (Harder complete only after you have nished the rest of the exam!) Now assume that
N = , i.e., the parking lot has innitely many spots.
Let E(x, C) denote the Erlang formula:
E(x, C) =
x
C
/C!
1 +x +x
2
/2 + +x
C
/C!
.
Assume the parking lot is in equilibrium. Show that the mean distance an arriving car parks
from the front of the store is:
1 +

C=1
E

, C

.
5
Problem 3 (5 points per part; 20 points total). Suppose that orders arrive to a tailor shop
according to a Poisson process of rate . Each order consists of a a random number of items
of clothing N, where N is a geometric random variable with parameter p, independent of other
orders. Assume that each garment requires an independent exponentially distributed amount of
time to complete with mean 1/, and that the tailor can only work on one garment at a time.
(a) What is the distribution of the total processing time of a single order?
(b) State an assumption on , p, and to ensure that the tailor does not build an innite backlog
of garments.
(c) Under the assumption of part (b), what is the equilibrium disribution for the number of
garments in the tailors shop?
(d) Consider the following queueing system. Jobs arrive according to a Poisson process of rate ,
and service times are independent exponentially distributed random variables of mean 1/.
Assume that after a job completes service, it departs with probability p or is recirculated
back to the server with probability 1 p. What is the long run average number of jobs in the
system?
6