1

241-460 I nt r oduct i on t o Queuei ng

Net wor ks : Engi neer i ng Appr oach
Chapt er 6 Random Pr ocesses or Random Pr ocesses or
Assoc. Prof. Thossaporn Kamolphiwong
Centre for Network Research (CNR)
Department of Computer Engineering, Faculty of Engineering
Prince of Songkla University, Thailand
Chapt er 6 Random Pr ocesses or Random Pr ocesses or
St ochast i c Pr ocesses St ochast i c Pr ocesses
Email : kthossaporn@coe.psu.ac.th
Out l i ne
Random Pr ocesses or St ochast i c Pr ocesses
Definit ions Definit ions
Types of St ochast ic Processes
Random Sequences
Examples of St ochast ic Processes
Bernoulli Process
C t i P Count ing Process
Poisson Process
St at ionary Process
Chapter 6 : Stochastic Processes
2
St ochast i c Pr ocess
Observat ion corresponds t o funct ion of t ime
Outcomes
S
,
Chapter 6 : Stochastic Processes
Random Variable
X(,)
Random Variable
X(t) = X(t,,)
St ochast i c Pr ocess
Definition : A stochastic process X(t) or Random
process is a rule for assigning t o every , a funct ion
x(t,,)
Definition : Sample Function
A sample function x(t,,) is the time function associated
with outcome , of an experiment
Definition : Ensemble
The ensemble of a stochastic process is the set of all
possible time functions that can result from an
experiment
Chapter 6 : Stochastic Processes
3
Exampl e
,
x(t,,
1
)
,
1
,
2
x(t,,
2
)
x(t,,
3
)
Ensemble
Chapter 6 : Stochastic Processes
,
3
( ,
3
)
Sample Space
Sample Function
St ochast i c Symbol s
, : outcome of the experiment , : outcome of the experiment
t : time dependent
x(t,,) : sample functions,
X(t) : name of stochastic process
Chapter 6 : Stochastic Processes
4
Exampl e
Time inst ant s T = 0, 1, 2,,
Roll a die and record N
x(t,S
1
)
Roll a die and record N
T
where 1s N
T
s 6
X(t) = N
T
X(t) for T s t < T + 1,
S
2
x(t,S
2
)
S
1
1,2,6,3,
t
Chapter 6 : Stochastic Processes
t
S
2
4,2,6,5,
Type of St ochast i c pr ocesses
Based on t he paramet er space:
Discret e-t ime st ochast ic process: p
Set I is count able ( t e I)
Cont inuous-t ime st ochast ic process
Set I is cont inuous (t e I)
Based on t he st at e processes:
Discret e-st at e processes:
st at e space discret e
Cont inuous-st at e processes:
st at e space cont inuous
Chapter 6 : Stochastic Processes
5
St ochast i c pr ocesses Exampl e
Discret e-t ime, discret e-st at e processes
The number of occupied channels in a t elephone
link at t he arrival t ime of t he k
t h
cust omer,
k = 1,2,
The number of packet s in t he buffer of a p
st at ist ical mult iplexer at t he arrival t ime of t he k
t h
cust omer, k = 1,2,
Chapter 6 : Stochastic Processes
( Cont i nue)
Cont inuous-t ime, discret e-st at e processes
The number of occupied channels in a t elephone
link at t ime t > 0
The number of packet s in t he buffer of a
st at ist ical mult iplexer at t ime t > 0 st at ist ical mult iplexer at t ime t > 0
Chapter 6 : Stochastic Processes
6
Type of St ochast i c Pr ocesses
Discrete-Time, Continuous-Value
t
)
Continuous-Time, Continuous-Value
t
)
Continuous-Time, Discrete-Value
D
(
t
)
Discrete-Time, Discrete-Value
D
(
t
)
X
D
C
(
t
t
X
C
C
(
t
t
Chapter 6 : Stochastic Processes
X
C
D
t
X
D
D
t
Random Var i abl es f r om
St ochast i c Pr ocesses
x(t,S
1
)
Sample function:
x(t,S
2
)
S
1
1,2,6,3,
t
Random Variable : X(t)
Sample function:
x(0,1), x(1,2), x(2,6), x(3,3),
( )
( )
( )
( ) x P PMF
x f PDF
t X
t X
:
:
Chapter 6 : Stochastic Processes
t
S
2
4,2,6,5,
7
Exampl e
Rolling a die, what is t he PMF of X(3.5)?
T s t < T + 1
x(t,S
1
) 6
Sol ut i on
The random variables X(3.5) is t he value of t he
die roll at t ime 3.
t
2
( )
( )

=
=
otherwise
x
x P
6 ,..., 2 , 1
0
6 / 1
5 . 3
Chapter 6 : Stochastic Processes
Random Sequence
Def i ni t i on :
Random sequence (X ) is an ordered sequence Random sequence (X
n
) is an ordered sequence
of X
0
, X
1
, X
2
,
Independent, Identically Distributed (iid) Random
Sequences is a random sequence X
n
in which
d d bl , X
-2
, X
-1
, X
0
, X
1
, X
2
, are iid Random Variables
Chapter 6 : Stochastic Processes
8
Theorem : Let X
n
is an iid random sequence.
For a discrete value process the joint PMF is
I I D Random Sequences
For a continuous-valued process, the joint PDF is
For a discrete-value process, the joint PMF is
( ) ( )
[
=
i X k X X
x P x x x P
k
,..., ,
2 1 ,...,
1
Chapter 6 : Stochastic Processes
( ) ( )
[
=
=
k
i
i X k X X
x f x x x f
k
1
2 1 ,...,
,..., ,
1
Sum Pr ocess
Many int erest ing random processes are obt ained
as t he sum of sequence of iid random variable, as t he sum of sequence of iid random variable,
X
1
, X
2
,
S
n
= X
1
+ X
2
+ + X
n
n = 1, 2,
Chapter 6 : Stochastic Processes
9
Some I mpor t ant St ochast i c
Pr ocess
Bernoulli Process
Count ing Process Count ing Process
Poisson Process
Chapter 6 : Stochastic Processes
Ber noul l i Pr ocess
Def i ni t i on : A Bernoulli (p) process X
n
is an iid
d i hi h h X i random sequence in which each X
n
is a
Bernoulli (p) random variable
Example
In a common model for communications the
Chapter 6 : Stochastic Processes
In a common model for communications, the
output X
1
, X
2
, of a binary source is modeled as
a Bernoulli (p = ) process
10
Count i ng Pr ocess
Def i ni t i on Counting Process
A t h ti N(t) i ti A stochastic process N(t) is a counting process
if for every sample function, k(t,,) = 0
for t < 0 and k(t,,) is integer-valued and
nondecreasing with time
Chapter 6 : Stochastic Processes
Sampl e pat h of count i ng pr ocess
N(t)
Arrival rate > 0
X
5
S
1
S
2
S
3
S
4
S
5
t
X
4
X
3
X
2
X
1
Chapter 6 : Stochastic Processes
X
n
: Bernoulli process
N(t) = # of customers that arrive at a system
during interval (0,t]
11
N(t)
Arrival rate > 0
( Cont i nue)
Sum of Process
S
1
= X
1
X
5
S
1
S
2
S
3
S
4
S
5
t
X
4
X
3
X
2
X
1
S
2
= X
1
+ X
2
S
3
= X
1
+ X
2
+ X
3
S
4
= X
1
+ X
2
+ X
3
+ X
4
S
5
= X
1
+ X
2
+ X
3
+ X
4
+ X
5
Chapter 6 : Stochastic Processes
# f
N(t)
( Cont i nue)
N(t) = # of customers
X
n
: Bernoulli process
X
5
t
S
1
S
2
S
3
S
4
S
5
Arrival rate > 0
X
4
X
3
X
2
X
1
T=mA
T=mA
A
t
S
1
S
2
S
3
S
4
S
5
Chapter 6 : Stochastic Processes
12
Count i ng Pr ocess
t
S
1
S
2
S
3
S
4
S
5
A 0, t here is only 1 arrival (X
n
= 1)
Choose A < < 1, success probabilit y of T/m
A A
5A 10A
15A mA
( ) ( ) ( )
n m n
N
m T m T
n
m
n P
m

|
|
.
|

\
|
= 1
Chapter 6 : Stochastic Processes
Prob. of N
m
arrival is
Binomial PMF
Count i ng Pr ocess
( ) ( ) ( )
m
|
|

|
Bi nomi al Pr ocess
( ) ( ) ( )
n m n
N
m T m T
n
m
n P
m

|
|
.
|

\
|
= 1
( )

n
e T
T
n
2 1 0

m , A 0,
Chapter 6 : Stochastic Processes
( )
( )

=
=
otherwise
n
n
e T
n P
N
m
,... 2 , 1 , 0
0
!

Poisson Process
13
Poi sson Pr ocess
Any int erval (t
0
,t
1
], # of arrivals is a Poisson PMF
wit h paramet er T where T = t t wit h paramet er T where T = t
1
-t
0
# of arrivals in (t
0
,t
1
] dependent s on t he
independent Bernoulli t rials
A 0, count ing process in which # of arrivals in
any int erval is Poisson process
Chapter 6 : Stochastic Processes
Poi sson Pr ocess
Definition : Poisson Process
A counting process N(t) is a Poisson process of rate if A counting process N(t) is a Poisson process of rate if
a) # of arrivals in any interval (t
0
,t
1
], N(t
1
)
N(t
0
), is a Poisson random variable with
expected value (t
1
-t
0
)
b) For any pair of nonoverlapping intervals (t
0
,t
1
]
and (t ' t '] # of arrivals in each and (t
0
,t
1
], # of arrivals in each
interval, N(t
1
) N(t
0
) and N(t
1
') N(t
0
')
respectively, are independent random variables
Chapter 6 : Stochastic Processes
14
Poi sson Pr ocess
t
t t t
0
N(t
0
)
N(t
1
)
t
1
N(t)
Chapter 6 : Stochastic Processes
N(t) = # of arrivals in the interval (t
0
,t
1
]
N(t
1
) - N(t
0
) = # of arrivals in the interval (t
0
,t
1
]
Poi sson Pr ocess
Process rat e () = E[N(t)]/t
Poisson random variable, N(t) = N(t
1
) N(t
0
)
( )
( )
( ) | |
( )

=

otherwise
m
e
m
t t
m P
t t
m
t N
,..., 1 , 0
!
0 1 0 1

Poisson random variable, N(t) N(t

1
) N(t
0
)
PMF is
( )

otherwise
0
Chapter 6 : Stochastic Processes
15
Exampl e
Suppose t hat t he number of calls t hat arrive at a
company call cent re is a Poisson process wit h a company call cent re is a Poisson process wit h a
rat e of 120 per hour.
a) What is t he probabilit y of 3 calls in a minut e?
b) What is t he probabilit y of at least t wo calls in a
minut e?
Chapter 6 : Stochastic Processes
Sol ut i on
=120 calls/hour
a) t t = 1 minute N(t) = 3 calls P[N(t) = 3] = ?
t
t
0
t
1
N(t)
Chapter 6 : Stochastic Processes
a) t
1
- t
0
= 1 minute, N(t) = 3 calls P[N(t) = 3] = ?
b) t
1
- t
0
= 1 minute, N(t) > 2 calls P[N(t) > 2] = ?
16
Sol ut i on
a) What is t he probabilit y of 3 calls in a minut e?
P[N(t) = 3] = ??
( )
( ) | |
( )
2
2
3 3
0 1
t t
t t

P[N(t) 3] ??
On average t here are 120/ 60 = 2 calls per
minut e. ( = 2)
( )
( ) | |
( )
18 . 0
! 3
2
!
) 3 (
2 0 1 0 1
= = = =

e e
m
t t
t N P
t t

Chapter 6 : Stochastic Processes

Sol ut i on
b) What is t he probabilit y of at least t wo calls in a
minut e? P[N(t) > 2] = ??
( ) ( ) ( ) ( ) 1 0 1 2 1 2 = = = < = > N P N P N P N P
P[N > 2] = P[N = 2] + P[N = 3] + P[N = 4] +
P[N = 0] + P[N = 1] + P[N = 2] + P[N = 3] + = 1
( ) ( ) ( ) 1 0 1 = + = = N P N P
Chapter 6 : Stochastic Processes
( ) 594 . 0 2 1 1
! 1
2
! 0
2
1
2
1
2
0
2
= + =
|
|
.
|

\
|
+ =

e
e e
17
Theor em : For a Poisson process N(t) of rat e ,
t he j oint PMF of N [N(t ) N(t )]' fo
Joi nt PMF
t he j oint PMF of N = [N(t
1
),, N(t
k
)]', for
ordered t ime inst ances t
1
< < t
k
, is
( ) ( ) ( )

s s s

=



otherwise
n n
n n
e
n n
e
n
e
n P
k
k k
n n
k
n n n
N
k k k
, ... 0
0
! ! !
1
1 1 2
2
1
1
1 2 1 2 1 1
o o o
o o o
Chapter 6 : Stochastic Processes
o
1
= t
1
and o
i
= (t
i
t
i-1
), i = 2, 3,
Exampl e

t
t
0
t
4
t
1
t
2
t
3

3
Chapter 6 : Stochastic Processes

2

3
n
1
= 2 n
2
= 3 n
3
= 4
18
( Cont i nue)

t
t
0
t
4
t
1
t
2
t
3

1

2

3
n
1
= 2 n
2
= 3 n
3
= 4
( )

2
3
2
o
o e
P
Chapter 6 : Stochastic Processes
( )

=
! 3
2
2
1 2
t t P
N
( )

=

! 2
1
1
2
1
0 1
o
o e
t t P
N
( )

=

! 4
3
3
4
3
3 4
o
o e
t t P
N
( Cont i nue)

t
t
0
t
4
t
1
t
2
t
3

1

2

3
n
1
= 2 n
2
= 3 n
3
= 4
Chapter 6 : Stochastic Processes
( )

- - =

! 4 ! 3 ! 2
3 2 1
4
3
3
2
2
1
o o o
o o o e e e
t P
N
19
Exampl e
I nquiries arrive at a recorded message device
according t o a Poisson process of rat e 15 according t o a Poisson process of rat e 15
inquiries per minut e.
Find t he probabilit y t hat in a 1-minut e period, 3
inquiries arrive during t he first 10 seconds and 2
inquiries arrive during t he last 15 seconds.
Chapter 6 : Stochastic Processes
( Cont i nue)
3
= 15 inquiries/minute
t(s)
0 10
3
2
= /second
50
60
Chapter 6 : Stochastic Processes
P[N
1
(10) = 3 and N
3
(60)-N
2
(45) = 2] = ??
20
Sol ut i on
Arrival rat e () = 15/ 60 = inquiries per second,
( ) ( )
! 2
4 15
! 3
4 10
4 15 2 4 10 3
e e
=
P[N
1
(10) = 3, N
3
(60)-N
2
(45) = 2] = ??
= P[N(10) = 3]P[N(60 45) = 2]
Chapter 6 : Stochastic Processes
I nt er ar r i val Ti me
Theorem: A count ing process wit h independent
exponent ial () int erarrival X
1
, X
2
, is a Poisson exponent ial () int erarrival X
1
, X
2
, is a Poisson
process of rat e
N(t)
t
Arrival rate > 0
Chapter 6 : Stochastic Processes
X
5
t
S
1
S
2
S
3
S4
S
5
X
4
X
3
X
2
X
1
2
th
Interarrival time
21
Theorem: For a Poisson process of rate ,
I nt er ar r i val t i me
the interarrival times X
1
, X
2
, are an iid
random sequence with the exponential PDF
( )

>
=

otherwise
x e
x f
x
X
, 0
0

Chapter 6 : Stochastic Processes

Rel at i onshi p bet ween t he Poi sson
and Exponent i al Di st r i but i ons
Poisson distribution Poisson distribution
provides an appropriate description provides an appropriate description p pp p p p pp p p
of the number of occurrences of the number of occurrences
per interval per interval
Exponential distribution Exponential distribution
Chapter 6 : Stochastic Processes
Exponential distribution Exponential distribution
provides an appropriate description provides an appropriate description
of the length of the interval of the length of the interval
between occurrences between occurrences
22
Pr oper t y 1 : Memoryless propert y
Pr oper t i es of Poi sson Pr ocess
| | X h X P
| |
| |
| |
h
n
n n
n n
e
t X P
t X h t X P
t X h t X P

=
>
> + >
= > + >
,
|
h
Chapter 6 : Stochastic Processes
If the arrival has not occurred by time t, the additional time until
the arrival, h + t, has the same exponential distribution as X
n
t
t+h time
( Cont i nue)
| |
| |
| | X P
t X h t X P
t X h t X P
n n
n n
> + >
= > + >
,
|
h
X
n
> t + h
X
n
> t
t+h ti
| |
| | t X P
n
n n
>
Chapter 6 : Stochastic Processes
t+h time
t
23
Exampl e
Connect ion request s arrive at a server according t o
a Poisson process wit h int ensit y = 5 request s a Poisson process wit h int ensit y 5 request s
in a minut e.
(a) What is t he probabilit y t hat exact ly 2 new
request s arrive during t he next 30 seconds?
(b) I f a new connect ion request has j ust arrived (b) I f a new connect ion request has j ust arrived
at t he server, what is t he probabilit y t hat it
t akes more t han 30 seconds before next
request arrives?
Chapter 6 : Stochastic Processes
Sol ut i on
N(t) : # of request s arrive at a server at t ime t
(a) What is t he probabilit y t hat exact ly 2 new (a) What is t he probabilit y t hat exact ly 2 new
request s arrive during t he next 30 seconds?
t
30 s
t
Chapter 6 : Stochastic Processes
t
0
t
1
30 s
P[N(t
0
+ 30) - N(t
0
) = 2 ] = ??
24
Sol ut i on
# of new arrivals during a t ime int erval follows
Poisson dist ribut ion wit h t he paramet er
( ) | |
( )
( )
( ) 5 2
! 2
5 . 0 5
2 ) ( 30
2
5 . 0 5
2

= = +

e t N t N P
A=(5/60)30 = 2.5
N(t+30)-N(t) ~ Poisson(2.5)
( )
257 . 0
! 2
5 . 2
5 . 2
=
=

e
Chapter 6 : Stochastic Processes
Sol ut i on
(b) I f a new connect ion request has j ust arrived at
t he server, what is t he probabilit y t hat it t akes t he server, what is t he probabilit y t hat it t akes
more t han 30 seconds before next request
arrives?
t
30 s
t
new connection next connection
P[t
1
-t
0
> 30] = ??
Or P[T > 30+t | T > t] = ??
Chapter 6 : Stochastic Processes
t
0
t
1
30 s
25
Sol ut i on
Consider t he process as a point process. The
int erarrival t ime follows exponent ial dist ribut ion int erarrival t ime follows exponent ial dist ribut ion
wit h paramet er
P[t
1
-t
0
> 30] = 1 P[t
1
-t
0
< 30]
= e
-(5/60)30
= e
-2.5
= 0.82
OR OR
P[T > t+30 | T > t] = e
-(5)(30/60)
= e
-2.5
= 0.82
Chapter 6 : Stochastic Processes
Pr oper t i es of Poi sson Pr ocess
Pr oper t y 2 : Let N
1
(t) and N
2
(t) be t wo
independent Poisson processes wit h rat es
1
independent Poisson processes wit h rat es
1
and
2
. The count ing process N(t) = N
1
(t) + N
2
(t)
is a Poisson process of rat e
1
+
2
.

1
N
1
(t)
N
2
(t)
Chapter 6 : Stochastic Processes

1
+
2
N
2
(t)
N(t)
26
Pr oper t i es of Poi sson Pr ocess
Pr oper t y 3 : The count ing processes N
1
(t) and
N
2
(t) derived from a Bernoulli decomposit ion of N
2
(t) derived from a Bernoulli decomposit ion of
t he Poisson process N(t) are independent
Poisson processes wit h rat e p and (1-p).
N (t)
N(t)= N
1
(t)+ N
2
(t)

Chapter 6 : Stochastic Processes

N
1
(t)
p
N
2
(t) (1-p)
Exampl e
A corporat e Web server records hit s (request for
HTML document ) as a Poisson process at a rat e HTML document ) as a Poisson process at a rat e
of 10 hit s per second. Each page is eit her an
int ernal request (wit h probabilit y 0.7) from t he
corporat e int ranet or an ext ernal request (wit h
probabilit y 0.3) from t he I nt ernet .
Over a 10-minut e int erval, what is t he j oint PMF , j
of I, t he number of int ernal request s, and X, t he
number of ext ernal request s?
Chapter 6 : Stochastic Processes
27
Sol ut i on
I nt ernal and ext ernal request arrival are
independent Poisson processes wit h rat e of 7
d 3 h d
( ) ( ) ( ) = x P i P x i P
and 3 hit s per second
o
I
= 7(600) = 4200 hits
o
X
= 3(600) = 1800 hits
The j oint PMF of I and X is
( ) ( ) ( )
( ) ( )
{ }

e
=
=

otherwise 0
,... 1 , 0 ,
!
1800
!
4200
,
1800 4200
,
x i
x
e
i
e
x P i P x i P
x i
X I X I
Chapter 6 : Stochastic Processes
St at i onar y Pr ocesses
St ochast ic process X(t),
at t : X(t ) wit h PDF f (x) depend on t at t
1
: X(t
1
) wit h PDF f
X(t1)
(x) depend on t
1
.
St at ionary process,
at t
1
: X(t
1
) wit h f
X(t1)
(x) does not depend on t
1
Stationary process
S d i bl t ll ti
Chapter 6 : Stochastic Processes
Same random variable at all time
The statistical properties of the process
do not change with time
28
Definit ion : St at i onar y Pr ocess
A h X( ) f d l
St at i onar y Pr ocess
A stochastic process X(t) is stationary if and only
if for all sets of time instant t
1
,, t
m
, and any
time difference t
( ) ( )
( )
( ) ( )
( )
m t X t X m t X t X
x x f x x f
m m
,..., ,...,
1 ,..., 1 ,...,
1 1
t t + +
=
Chapter 6 : Stochastic Processes
Ref er ences
1. Albert o Leon-Garcia, Probabilit y and Random
Processes for Elect rical Engineering, 3
rd
Processes for Elect rical Engineering, 3
Ed., Addision-Wesley Publishing, 2008
2. Roy D. Yat es, David J. Goodman, Probabilit y
and St ochast ic Processes: A Friendly
I nt roduct ion for Elect rical and Comput er
Engineering, 2nd, John Wiley & Sons, I nc, 2005 g g, , y , ,
3. Jay L. Devore, Probabilit y and St at ist ics for
Engineering and t he Sciences, 3rd
edit ion, Brooks/ Cole Publishing
Company, USA, 1991.
Chapter 6 : Stochastic Processes