Outline

Lecture #4 Stochastic Process

. Anan Phonphoem, Ph.D.
anan@cpe.ku.ac.th http://www.cpe.ku.ac.th/~anan Computer Engineering Department Kasetsart University, Bangkok, Thailand

Stochastic Process Counting Process Poisson Process Brownian Motion Process Autocovarience and Autocorrelation Random Sequence Stationary Process Wide-sense Stationary Process
2
7/9/2003

7/9/2003

Definition

Random Variable
X is a function that maps each outcome, s, in S to a real number X(s), x

Stochastic Process Random Function of Time

S
s

X(s) = x

Random Process
3
7/9/2003

-2.0

-1.5

-1.0

-0.5

0.5

SX
4
7/9/2003

Random Process
X(t) is a function that maps each outcome, s, in S to a time function x(t,s)

Example 1
Taking temperature at the surface of a space shuttle Starting at launch time t = 0 X(t) = temp in degree Celsius on the surface Each launch, record x(t,s)

S
s

X(t,s) = x(t,s)

x(t,s1) x(t,s2) x(t,sn)

Ensemble (Sx)

7/9/2003

Sample Functions

Source:www.inventorsmuseum.com

7/9/2003

Example 1
x(t,e1)
2500.10

Example 1
x(t,e1) x(t,e2)
Average temp at e1 for completed mission is 5000 C E[x(t, e1)] Time Average

Source:www.analyticalsci.com/Astronomy/ Hansen

e1 = 1st launch measuring

7
7/9/2003

At time t = 2500.10 sec x(2500.10, e1) = 1200 C

x(t,en)

Average temp of engine at t = 2500.10 sec is 1320 C E[X(2500.10)] Ensemble Average

Sample functions
7/9/2003

Example 2
Measure the rain fall in a day @Songkla province every day. Let F(t) = random process f(t,y) = a sample function for measuring at day t of the year y

Example 2
f(t,y1) f(t,y2)
A sample function of rain fall in y1 = year 1990 (1 t 365) A sample function of rain fall in y2 = year 1991 (1 t 365)

f(t,yn)

A sample function of rain fall in yn = year 2001 (1 t 365)

7/9/2003

7/9/2003

10

Sample functions

Example 2
f(t,y1) f(t,y2) Therefore, we might want to know

Types of Stochastic Process

W(t) Continuous Time, Continuous value Process Discrete Time, Continuous value Process Y(t) Continuous Time, Discrete value Process Z(n) Discrete Time, Discrete value Process
7/9/2003

11

Sample functions
7/9/2003

f(t,yn)

The average rain fall in year 2001 The average rain fall for Sep 3

X(n)

12

Stochastic Process Examples

W(t) Record temperature as a continuous time Record round(temperature) as a continuous time Y(t) Record temperature every T seconds Z(n) Record round(temperature) every T seconds
7/9/2003

IID Random Sequence

Independent, Identically Distributed Random Sequence Independent trials of an experiment at a constant rate Discrete / Continuous

X(n)

Theorem:
PXn Xn (x1,,xk) = PX(x1)PX(xk) = PX(xi)
1 k

i=1

13

14

7/9/2003

Counting Process
Definition: A Stochastic Process is a Counting Process N(t) if
n(t,s) = 0 for t < 0 n(t,s) = integer valued and non-decreasing

Counting Process
# of customers arrive at (0,t]
N(t)
4 3 2 1

S1 S2
X1 15
7/9/2003

S3
X3
7/9/2003

S4
X4

X2

16

Counting Process
N(t) 4 3 2 1 t S1 S2 X1 X2 X3 S3 X4 S4

Counting Process
For a small step , only one arrival (Xn = 1) Binomial Process PNm(n) =

( m) (T/m)n(1- T/m)m-n n
0 m

n = 0,1,2,.. Otherwise

Success Prob. of Xn= = T/m Binomial PMF

or

Poisson Process

T = m

(T)n e- T
m (T/m)n(1- T/m)m-n n = 0,1,2,.. n

n = 0,1,2,..
Otherwise

PNm(n) =
17

( )
0

PN(t)(n) =
0
18

n!

Otherwise
7/9/2003 7/9/2003

Poisson Process
Poisson Process is
a Counting Process that the # of Arrival during any interval is Poisson RV

Poisson Process
N(t)
4 3 2 1

An arrival during any instant is independent of the past history of the process Memoryless Xn is called Interarrival Time
19
7/9/2003

nth arrival time

S1 S2
X1 X2 X3

S3
X4

S4

t nth interarrival time

20

7/9/2003

Poisson Process
Definition:
A Counting Process N(t) is a Poisson Process N(t) if # of arrivals in (t0,t1], N(t1) N(t0), is a Poisson RV with expected value (t1-t0) # of arrivals in each interval are independent random variable
21
7/9/2003

Poisson Process
Process rate () = E[N(t)] / t M = N(t1) N(t0) = Poisson RV
PM(m) = [(t1-t0)]m e- (t1-t0) m = 0,1,2,..
m! 0 Otherwise

22

7/9/2003

Joint PMF
Theorem: Poisson Process N(t) of rate ,
Joint PMF of N(t1),,N(tk), t1 <<tk

Example
A mobile station transmits data packet as Poisson process with rate 12 packets/sec

P N(t1),,N(tk)(n1,,nk)
=

1n1 e- 1 2(n2-n1) e- 2 k(nk-nk-1) e- k

n1! (n2 n1)! (nk nk-1)! 0

0 n1 nk

Find # of packets transmitted in the kth hour Find Joint PMF of # of packets transmitted in the kth hour and zth hour

Otherwise

i = (ti-ti-1)
23
7/9/2003

24

7/9/2003

Example
Let Nk = # of packets transmitted in kth hour # packets in each hour is IID
[12(3600-0)]n e -12 (3600-0) n = 0,1,2,.. Otherwise n = 0,1,2,.. Otherwise
7/9/2003

Example
Joint PMF of # of packets transmitted in the kth hour and zth hour knk e- k znz e- z nk = 0,1, PNk,Nz(nk,nz) = nz = 0,1, nk! n z!
0 Otherwise nk = 0,1, nz = 0,1, Otherwise
7/9/2003

PNi(n) =
0 =
25

n! [43200]n e -43200 n! 0 =

(nk+nz) e- 2
nk! nz! 0
26

= k = Z = T = 12(3600-0)] = 43200

Interarrival Time
Theorem:
Poisson Process of rate , the interarrival times X1, X2, are an iid random sequence with Exponential PDF

Interarrival Time
Theorem:
A Counting Process with independent exponential interarrival time X1, X2, with E[Xi] = 1/ is a Poisson Process of rate

fX(x) =

e -x 0

x0 Otherwise

27

7/9/2003

28

7/9/2003

Brownian Motion Process

A continuous time, continuous value random process
Definition: A Brownian Motion Process X(t) has the property that X(0) = 0 For > 0 , X(t + ) X(t) = Gaussian RV with E[X(t)] = 0 and Var[X(t)] = that is independent of X(t) for t t
29
7/9/2003

Brownian Motion Process

Gaussian RV fX(x) =
1 22
(x - )2

22

E[X(t)] = 0 and Var[X(t)] =

fX(t)(x) =

1 2

x2
2

30

7/9/2003

Joint PDF
Theorem: For the Brownian motion process X(t), joint PDF of X(t1),,X(tk) f X(t1),,X(tk) (x1,,xk)
1 = n = 1 2(tn tn-1)
k

Expected Value
X(t): X(t1) f X(t 1 ) (x) E[X(t1)]

(xn xn-1)2

Definition: The expected value of a stochastic process X(t) is the deterministic function

2 (tn

tn-1)

x(t) = E[X(t)]
32
7/9/2003

31

7/9/2003

Covariance of X and Y
Definition: Cov[X,Y] = E[(X-x)(Y-Y)]

Autocovariance
Auto = self = same process For a same process X(t), in 2 different times t1 and t2 = t + For high covariance = a sample function, is unlikely to change in interval For near zero covariance = rapid change

Cov[X,Y] = E[XY - xY - YX + x y] = E[XY] - xE[Y] - YE[X] + x y = E[XY] - x y - x y + x y Theorem: Cov[X,Y] = E[XY] - x y

High covariance = an observation of X provides an accurate indication of Y

33
7/9/2003

How much the sample function is likely to change in the interval after t
34
7/9/2003

Autocovarience
Definition:
The autocovariance function of a stochastic process X(t) is

Correlation
Definition: The correlation of X and Y is rX,Y rX,Y = E[XY] Theorem: Cov[X,Y] = rX,Y - x y

CX(t,) = Cov[ X(t), X(t+) ]

Note For = 0
35

CX(t,t) = Var[X(t)]
7/9/2003

36

7/9/2003

Autocorrelation
Definition: The autocorrelation function of a stochastic process X(t) is RX(t,) = E [ X(t) X(t+) ]

Autocovariance & Autocorrelation

Theorem: The autocovariance and autocorrelation functions of a process X(t) satisfy CX(t,) = RX(t,) x(t) x(t+ )
Note: Autocovariance Autocorrelation
use X(t) to predict a future X(t+) describe the power of a random signal
7/9/2003

37

7/9/2003

38

Random Sequence
For a discrete time process, the sample function is described by the ordered sequence of random variable Xn = X(nT)

Autocovariance & Autocorrelation of a Random Sequence

Definition: The autocovariance of a random sequence Xn is CX[m,k] = Cov[Xm, Xm+k] Definition: The autocorrelation of a random sequence Xn is RX[m,k] = E[Xm Xm+k]
m and k are integers
40
7/9/2003

Definition: A random sequence Xn is an ordered sequence of random variable X0,X1,

39

7/9/2003

Autocovariance & Autocorrelation of a Random Sequence

X(t)

CX[m,k] = CX[4,1] = Cov[X4,X4+1]

T

Stationary Process

X4 X5
2 4 6 8

0 X0 = (0T) X1 = (1T) X2 = (2T)

41

7/9/2003

7/9/2003

Stationary Process
For a random process X(t), normally, at t1: X(t1) has pdf = fX( t )(x) [depends on t1] For a random process X(t), at t1: X(t1) has pdf = fX( t )(x) [not depend on t1]
1 1

Stationary Process
Definition: A stochastic process X(t) is stationary
iif for all sets of time t1,,tm and any time different ,

f X( t ),
1

,X( tm ) (x1,,xm)

Stationary Process = same random variable at all time = no statistical properties change with time

fX( t

1+

),, X( tm + ) (x1,,xm)

fX(t1 )(x) = fX(t1 + )(x) = fX(x)

43
7/9/2003

44

7/9/2003

Stationary Random Sequence

Definition: A random sequence Xn is stationary
iif for any finite sets of time instants n1,,nm and any time different k,

Stationary Process
Theorem: A stationary process X(t),
X(t) = X RX(t,) = RX(0,) = RX() CX(t,) = RX() 2X = CX()

f X( n ),
1

,X( nm) (x1,,xm) = fX( n +k ),, X( n

1

m+k

) (x1,,xm)

45

7/9/2003

46

7/9/2003

Stationary Random Sequence

Theorem: A stationary random sequence Xn, for
all m E[Xm] = X RX[m.k] = RX[0,k] = RX[k] CX[m,k] = RX[k] 2X = CX[k]

Example
Telegraph Signal, X(t) take value 1 X(0) = 1 with probability = 0.5 Let X(t) toggles the polarity with each occurrence of an event in a Poisson process rate

47

7/9/2003

48

7/9/2003

Example

Example
Find PMF of X(t), fX(t)(x) P[X(t)] = P[X(t) | X(0) = 1] P[X(0) = 1] + P[X(t) | X(0) = 1 ] P[X(0) = 1] P[X(t) | X(0) = 1] = P[N(t) = even]

+1

1
X1 X2 X3 X4 X5

=
j=0

(t)2j (2j)!

e t

= et (1/2) (et + et ) = (1/2) (1 + e-2t)

49
7/9/2003

50

7/9/2003

Example
P[X(t) | X(0) = 1] = P[N(t) = odd]

Example
=
j=0

(t)2j+1 (2j+1)!

e t

= et (1/2) (et et ) = (1/2) (1 e-2t)

P[X(t) = 1] = P[X(t) | X(0) = 1] P[X(0) = 1] + P[X(t) | X(0) = 1] P[X(0) = 1] = (1/2) (1 + e-2t)(1/2) + (1/2) (1 + e-2t)(1/2) = 1/2 P[X(t) = -1] = 1 P[X(t) = 1] = 1/2

fX(t)(x) =
51
7/9/2003

0
7/9/2003

X(t) = 1 , 1 Otherwise

52

Example
X(t) = 1 (1/2) + ( 1)(1/2) = 0 Var [X(t)] = E[X2(t)] = 12(1/2) + (-1)2(1/2) =1

Wide Sense Stationary

Definition: X(t) is a wide sense stationary random
process iff for all t, E[X(t)] = X RX(t,) = RX(0,) = RX()

Autocovariance, CX(t,) = e -2 f X( t ),
1

,X( tm ) (x1,,xm)

Definition: Xn is a wide sense stationary random

sequence iff for all n, E[Xn] = X RX[n,k] = RX[0,k] = RX[k]
7/9/2003

fX( t

),, X( tm + ) (x1,,xm) 1+

53

7/9/2003

54

Wide Sense Stationary

For every stationary process or sequence, it is also wide sense stationary. However, if it is a wide sense stationary it may or may not be stationary.

Example
Let Xn = 1 with prob = (n = even) For n = odd Xn = -1/3 with prob = 9/10 Xn = 3 with prob = 1/10 Stationary ? No Wide sense stationary ? Mean = 0 for all n CX(t,) = 0 for > 0 CX(t,) = 1 for = 0 Yes , its wide sense stationary
7/9/2003

55

7/9/2003

56

Wide Sense Stationary

Theorem: For a wide sense stationary process
X(t), RX(0) 0 RX() = RX( ) |RX()| RX(0)

Wide Sense Stationary

Theorem: For a wide sense stationary sequence
Xn, RX[0] 0 RX[k] = RX[ k] |RX[k]| RX[0]

57

7/9/2003

58

7/9/2003