J. Virtamo 38.

3143 Queueing Theory / Birth-death processes 1

Birth-death processes
General
A birth-death (BD process) process refers to a Markov process with

- a discrete state space

- the states of which can be enumerated with index i=0,1,2,. . . such that
- state transitions can occur only between neighbouring states, i → i + 1 or i → i − 1

l0 l1 l2 li l i+1
0 1 2 ... i i+1
m1 m2 m3 m i+1 m i+2

Transition rates
 
λi when j = i+1 probability of birth in interval ∆t is λi ∆t
 

 

 
 
qi,j =

µi if j = i−1 

 probability of death in interval ∆t is µi ∆t
 
 0


otherwise 
 when the system is in state i
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 2

The equilibrium probabilities of a BD process

We use the method of a cut = global balance condition applied on the set of states 0, 1, . . . , k.
In equilibrium the probability flows across the cut are balanced (net flow =0)

λk πk = µk+1 πk+1 k = 0, 1, 2, . . .

We obtain the recursion

λk
πk+1 = πk
µk+1
By means of the recursion, all the state probabilities can be expressed in terms of that of the
state 0, π0,
λk−1λk−2 · · · λ0 k−1
Y λi
πk = π0 = π0
µk µk−1 · · · µ1 i=0 µi+1

The probability π0 is determined by the normalization condition π0

1 1
π0 = =
λ0 λ0 λ1 ∞ k−1
X Y λi
1+ + +··· 1 +
µ1 µ1 µ2 k=1 i=0 µi+1
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 3

The time-dependent solution of a BD process

Above we considered the equilibrium distribution π of a BD process.
Sometimes the state probabilities at time 0, π(0), are known
- usually one knows that the system at time 0 is precisely in a given state k; then πk (0) = 1
and πj (0) = 0 when j 6= k
and one wishes to determine how the state probabilities evolve as a function of time π(t)
- in the limit we have limt→∞ π(t) = π.

This is determined by the equation

d
π(t) = π(t) · Q where
dt
 

−λ0 λ0 0 ... ... 
 µ −(λ1 + µ1 ) λ1 0 ...
 

 1 
 
Q=

0 µ 2 −(λ 2 + µ2 ) λ 2 0 

...
 
 

 0 µ 3 −(λ 3 + µ3 ) λ 3



... ... 0 µ4 −(λ4 + µ4)

J. Virtamo 38.3143 Queueing Theory / Birth-death processes 4

The time-dependent solution of a BD process (continued)

l0 l1 l2 l i-1 li l i+1
0 1 2 ... i i+1
m1 m2 m3 mi m i+1 m i+2

The equations component wise



 dπi (t)
= −(λ i + µi )πi (t) + λi−1πi−1 (t) + µi+1 πi+1(t) i = 1, 2, . . .








 dt | {z } | {z }

 flows out flows in



 dπ0 (t)
= −λ 0 π0 (t) + µ1 π1 (t)







 dt | {z } | {z }
flow out flow in

J. Virtamo 38.3143 Queueing Theory / Birth-death processes 5

Example 1. Pure death process

 

 λi = 0 
 1 i=n
 µ = iµ
i = 0, 1, 2, . . . πi(0) = 

i
 0 i 6= n

all individuals have the same the system starts from state n
mortality rate µ

State 0 is an absorbing state,

0 1 2 ... n-1 n
m 2m 3m (n-1) m nm other states are transient


d



 dt
πn(t) = −nµπn(t) ⇒ πn(t) = e−nµt

 d


dt πi (t) = (i + 1)µπi+1(t) − iµπi (t) i = 0, 1, . . . , n − 1
Z t 0
d iµt
dt (e πi (t)) = (i + 1)µπi+1 (t)e iµt
⇒ πi (t) = (i + 1)e −iµt
µ 0
πi+1(t0 )eiµt dt0
Z t −nµt0 (n−1)µt0 0
πn−1(t) = ne −(n−1)µt
µ 0
e
| e
{z } dt = n e−(n−1)µt (1 − e−µt)
0
e−µt
  Binomial distribution: the survival
n −µt i
Recursively πi(t) =  (e

 ) (1 − e−µt )n−i probability at time t is e−µt inde-
i
pendent of others
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 6

Example 2. Pure birth process (Poisson process)

 

 λi = λ 
 1 i=0
 µ = 0
i = 0, 1, 2, . . . πi (0) = 

i  0 i>0

birth probability per time unit is initially the population size is 0

constant λ

l l l l l l
0 1 2 ... i-1 i All states are transient


d



 dt
πi (t) = −λπi (t) + λπi−1(t) i>0

d
π0(t) = e−λt



dt
π0(t) = −λπ0(t) ⇒

Z t 0
d λt
dt
(e πi(t)) = λπi−1(t)e λt
⇒ πi(t) = e −λt
λ 0
πi−1(t0 )eλt dt0
Zt −λt0 λt0 0
π1(t) = e −λt
λ 0
e
| {ze } dt = e−λt(λt)
1

(λt)i −λt
Recursively πi(t) = e Number of births in interval (0, t) ∼ Poisson(λt)
i!
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 7

Example 3. A single server system

l
0 1 - constant arrival rate λ (Poisson arrivals)
m - stopping rate of the service µ (exponential distribution)
The states of the system

 0

server free
1  1

server busy
0
ì
í
î
ì
í
î

~ Exp(m) ~ Exp(l)


d



 dt
π0(t) = − λπ0(t) + µπ1(t) 
−λ λ 


Q=



 d
π1(t) = λπ0(t) − µπ1(t) µ −µ
dt

BY adding both sides of the equations

d
dt (π0 (t) + π1(t)) = 0 ⇒ π0 (t) + π1 (t) = constant = 1 ⇒ π1(t) = 1 − π0 (t)
d (λ+µ)t
d
π (t)
dt 0
+ (λ + µ)π0 (t) = µ ⇒ dt
(e π0(t)) = µe(λ+µ)t

µ µ
π0(t) = λ+µ + (π0(0) − λ+µ )e−(λ+µ)t
λ λ
π1(t) = λ+µ
+ (π1(0) − λ+µ )e−(λ+µ)t
| {z } | {z }| {z }
equilibrium deviation from decays expo-
distribution the equilibrium nentially
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 8

Summary of the analysis on Markov processes

1. Find the state description of the system

• no ready recipe
• often an appropriate description is obvious
• sometimes requires more thinking
• a system may Markovian with respect to one state description but non-Markovian
with respect to another one (the state information is not sufficient)
• finding the state description is the creative part of the problem
2. Determine the state transition rates
• a straight forward task when holding times and interarrival times are exponential
3. Solve the balance equations
• in principle straight forward (solution of a set of linear equations)
• the number of unknowns (number of states) can be very great
• often the special structure of the transition diagram can be exploited
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 9

Global balance

q i, j X X i = 0, 1, . . . , n
j
πj qj,i = πi qi,j
j6=i j6=i one equation per each state
i q j, i | {z
flow to state i
} | {z
flow out of
}

state i
n + 1 states
Q
z
 X }| {

− q0,j q0,1 q0,2 ... q0,n 

0




j X


 
 

 q1,0 − q1,j q1,2 ... q1,n 



 0



z
π}|

{ j

 



 X 
(π0 , . . . , π ) 
n 

q2,0 q2,1 − q2,j . . . q2,n 

 =






 j   
... ... ... ...
 


 ... 





... 


   
 X 
qn,0 qn,1, qn,2, ... − qn,j
   

j
0

π·Q=0 π0 + π1 + · · · + πn = 1
one equation normalization condition
is redundant
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 10

Example 1. A queueing system

s palvelinta

m 



 s servers
K odotuspaikkaa 

l m


 K waiting places





λ arrival rate (Poisson)
m


 µ

Exp(µ) holding time (expectation 1/µ)
ì
í
K=5 m
î s=4

The number of customers in system N is an appropriate state variable

- uniquely determines the number of customers in service and in waiting room

- after each arrival and departure the remaining service times
of the customers in service are Exp(µ) distributed (memoryless)

0 l 1 l 2 l 3 l 4 l 5 l 6 l 7 l 8 l 9

m 2m 3m 4m 4m 4m 4m 4m 4m
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 11

Example 2. Call blocking in an ATM network

A virtual path (VP) of an ATM network is offered calls of two different types.
 





R1 = 1Mbps 




R2 = 2Mbps
 


λ1 = arrival rate 

λ2 = arrival rate
 
 µ1 = mean holding time  µ2 = mean holding time

 


a) The capacity of the link is large (infinite)

n2

l2 (n2+1)m2
l1 l1

n1 m1 (n1+1)m1
l2 n2 m 2

n1

The state variable of the Markov process in this example is the pair (N1 , N2), where Ni defines
the number of class-i connections in progress.
J. Virtamo 38.3143 Queueing Theory / Birth-death processes 12

Call blocking in an ATM network (continued)

b) The capacity of the link is 4.5 Mbps

n2

n1