Вы находитесь на странице: 1из 32

Probability–An Overview

B. Sainath (BITS, PILANI)

B. Sainath EEE Dept., BITS PILANI

Jan. 2018

Probability–An Overview

Jan. 2018 1 / 29
Jan. 2018
1 / 29

Outline

1 Ordered Random Variables

2 Complex Random Variables

3 Random Vectors

4 Random Signals

5 Random Processes

6 Further Reading & References

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 2 / 29
Jan. 2018
2 / 29

Ordered RVs & Statistics

Applications in wireless communications (eg. antenna selection/relay selection) Eg. Z = Max(X,Y)

(eg. antenna selection/relay selection) Eg. Z = Max(X,Y) Find pdf: Max ( X 1 , ,

Find pdf: Max(X 1 ,

, X n ) and Min(X 1 ,

,

X n )

Eg. Min. of independent RVs

P(Min {X 1 ,

, X n } > y) = P(X 1 > y,

, X n > y)

= P(X 1 > y ) P(X n > y) Exercise: Derive Max. RV pdf
= P(X 1 > y )
P(X
n > y)
Exercise:
Derive Max. RV pdf when X i ∼ exp(1), i = 1,
Find MV, MSV, Variance
, n
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
3 / 29

Complex RVs

Noise waveforms after demodulation to baseband are cplx. Cplx. RV: Z = Z re + jZ Im

Z Gaussian if Z re and Z Im are jointly Gaussian MGF of Z if Z re and Z Im are jointly standard Gaussian ? MAR (Magnitude-Angle representation) R exp(jΘ)

R =? Θ =? B. Sainath (BITS, PILANI) Probability–An Overview Jan. 2018 4 / 29
R =?
Θ =?
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
4 / 29
Example Problem Let Z = X + jY , X ∼ N (0, σ 2
Example Problem
Let Z = X + jY , X ∼ N (0, σ 2 ) and Y ∼ N (0, σ 2 ) are independent RVs.
2
2
It’s magnitude R = √ X 2 + Y 2 and phase Θ = arctan(Y /X )

Find pdfs of R and Θ

Hint: Use Cartesian to Polar transformation

Resultant pdfs are ? and ? (prove)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 5 / 29
Jan. 2018
5 / 29
Example Problem Let Z = X + jY , X ∼ N (0, σ 2
Example Problem
Let Z = X + jY , X ∼ N (0, σ 2 ) and Y ∼ N (0, σ 2 ) are independent RVs.
2
2
It’s magnitude R = √ X 2 + Y 2 and phase Θ = arctan(Y /X )

Find pdfs of R and Θ

Hint: Use Cartesian to Polar transformation

Resultant pdfs are ? and ? (prove)

Magnitude: Rayleigh distribution Phase: Uniform distribution

Distribution of R 2 ?

Exponential distribution

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 5 / 29
Jan. 2018
5 / 29

Gaussian Random Vectors

Standard Gaussian random vector

Collection of n i.i.d. standard Gaussian RVs X 1 ,

X = [X 1 ,

,

X n ] t

, X n ∈ R n

pdf of X ?

P X (x) =

1

(2π)

n

2

exp x 2

2

Result: Let O is orthogonal matrix (transformation). If X is std. Gaussian, OX is also std. Gaussian (proof in class) Qs:

How is x 2 distributed (Ans. χ-square RV with n degrees of freedom)

How is x 2 = x

1

+ x

2

2

distributed?

2

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 6 / 29
Jan. 2018
6 / 29

Gaussian Random Vectors

Standard Gaussian random vector

Collection of n i.i.d. standard Gaussian RVs X 1 ,

X = [X 1 ,

,

X n ] t

, X n ∈ R n

pdf of X ?

P X (x) =

1

(2π)

n

2

exp x 2

2

Result: Let O is orthogonal matrix (transformation). If X is std. Gaussian, OX is also std. Gaussian (proof in class) Qs:

How is x 2 distributed (Ans. χ-square RV with n degrees of freedom)

How is x 2 = x

1

+ x

2

2 distributed? (Ans. Exponential)

2

Y = AX + µ Gaussian random vector

A: Matrix representing linear transformation µ mean vector pdf of Y?

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 6 / 29
Jan. 2018
6 / 29

Further Transformation

For any c ∈ R n , c t Y ?

A std. Gaussian random vector is also Gaussian

Mean: zero vector Covariance matrix: I n

Any linear combination of elements of a Gaussian random vector is also Gaussian

If A invertible, Gaussian random vector is completely characterized by

Mean vector Covariance matrix: Symmetric, non-negative definite

Symmetric n × n real matrix M is said to be non-negative definite if c t Mc is non-negative for every non-zero column vector c ∈ R n

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 7 / 29
Jan. 2018
7 / 29

SOME KEY POINTS

O orthogonal matrix

Covariance matrices of Y and OY same = AA t (show this) White Gaussian random vector

Independent Gaussian RVs and covariance matrix diagonal RVs are uncorrelated

What if A is not invertible?

Some components of AX linearly dependent (LD) Focus on components that are linearly independent (LI)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 8 / 29
Jan. 2018
8 / 29

Complex Gaussian Random Vectors

Cplx. random vector: Z = Z re + jZ Im

Z re and Z Im are real random vectors Distribution completely specified by mean and covariance matrix of real random vector [Z re , Z Im ] t

Mean µ, Covariance matrix K, Pseudo-covariance J

µ E [Z]

K

E [(Z µ)(Z µ) ]

J

E (Z µ)(Z µ) t

Covariance matrix: Hermitian (K = K )

Complete second order statistics: n + n 2 + n 2 = n(2n + 1) Note:

In wireless communication, we are interested in circularly symmetric complex Gaussian random vectors

in circularly symmetric complex Gaussian random vectors B. Sainath (BITS, PILANI) Probability–An Overview Jan.

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 9 / 29
Jan. 2018
9 / 29

Circulary Symmetric Complex Gaussian (CSCG)

Result:

Z is circularly symmetric complex random vector if e jθ Z has same

distribution of Z for any θ

Circularly symmetric complex random vector Z

Mean zero (Proof in class)

Pseudo-covariance zero (Proof in class)

K fully specifies first and second order statistics Z ∼ CN (0, K)

Spl. Cases

CSCG must have i.i.d.zero mean Re. and Im. components

Statistics fully specified by σ 2 , Z ∼ CN(0, σ 2 )

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 10 / 29
Jan. 2018
10 / 29

Density Function of Standard CSCG Random Vector

Standard CSCG RV Z

Z ∼ CN(0, 1), z ∈ C Re. and Im. parts are RVs with variance

1

2

P Z (z) =

1

π

exp z 2

Standard CSCG random vector Z ∼ CN (0, I), z ∈ C n

Collection of n i.i.d.Z ∼ CN (0, 1)

P Z (z) =

1 n exp z 2

π

Property: UZ has same distribution as Z (U: unitary matrix) Q. Let A is an invertible matrix. pdf of X = AZ?

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 11 / 29
Jan. 2018
11 / 29

Random Signals or Waveforms

Deterministic signals modeled by mathematical formula

Not enough to model real-world phenomena

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 12 / 29
Jan. 2018
12 / 29

Random Signals or Waveforms

Deterministic signals modeled by mathematical formula

Not enough to model real-world phenomena

Random signals (due to random processes)

Modeled in probabilistic terms Eg., Speech, Stock market exchange rates, Noise, Interference, ECG, EEG

Stock market exchange rates, Noise, Interference, ECG, EEG Figure: Types of waveforms B. Sainath (BITS, PILANI)

Figure: Types of waveforms

Noise, Interference, ECG, EEG Figure: Types of waveforms B. Sainath (BITS, PILANI) Probability–An Overview Jan.

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 12 / 29
Jan. 2018
12 / 29

Random (stochastic) Processes (RPs)

RV: Assignment of real number X (ω) to each point ω of sample space in such a way that CDF exists

Random process X (t) (evolves with time): Assign a real-time function (called sample function) X (ω, t) to each point ω

(called sample function) X ( ω, t ) to each point ω Figure: Simple random process.

Figure: Simple random process. Source: WC & J [1965]

Probability space: Sample space, waveforms, probability measure

Collection of waveforms with probabilities define RV at observed time instant

with probabilities define RV at observed time instant B. Sainath (BITS, PILANI) Probability–An Overview Jan.

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 13 / 29
Jan. 2018
13 / 29

Examples of Stochastic Processes

Gaussian processes, Brownian motion, Markov chains, Martingales Specification of RP

A rule is given or implied for determining joint pdf P X(t) for any finite set of

observation instants (t 1 ,

, t n )

Three methods of specifying RP

Direct way to find joint pdf, eg. Gaussian process (discuss later) Time function involving one or more parameters, eg.

Y (t) = R sin(2πt + Θ)

Sample functions: Y (ω, t) = R(ω) sin(2πt + Θ(ω)), ω To find P Y(t) , knowledge of P R,Θ required Eg. R, Θ as polar coordinates, is 2D plane A possible joint pdf

B. Sainath (BITS, PILANI)

P R,Θ (r, θ) =

2π exp r 2 ;

r

2

0;

Probability–An Overview

0 r < , 0 θ < 2π otherwise

2 ; r 2 0 ; Probability–An Overview 0 ≤ r < ∞ , 0 ≤
Jan. 2018 14 / 29
Jan. 2018
14 / 29

Third Method

Generate ensemble of RP by applying stated operation to sample functions of known process

Eg. Time translation: Y (ω, t) = X (ω, t + T ), ω

Thus P Y(t) = P X(t+T)

Eg. Linear filtering

h(t) impulse response

Y(t) =

−∞

h(t τ)X(τ)

In general, finding P Y(t) difficult unless X (t) is Gaussian process

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 15 / 29
Jan. 2018
15 / 29

Stationary RPs

Stationary in strict sense: Let p(x t 1

x t n ) denote joint pdf

p(x t 1

x t n ) = p(x t 1 +T

x t n +T ), T&n

Statistics of stationary RPs are invariant to time-axis translation Non-stationary if joint pdfs are different

Statistical (or ensemble) averages:

n th moment of RV X t i :

E X =

n

t

i

−∞

n

t

x

i

p(x t i ) dx t i .

Autocorrelation function (ACF) R X (t 1 , t 2 )

E [X t 1 , X t 2 ] =

−∞ −∞

x t 1 x t 2 p(x t 1 , x t 2 ) dx t 1 dx t 2 .

For stationary RP: R X (t 1 , t 2 ) = R X (t 1 t 2 ) = R X (τ ), where τ = t 1 t 2

2 ) = R X ( τ ) , where τ = t 1 − t

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 16 / 29
Jan. 2018
16 / 29

Some Properties of ACF & Covariance Function

R X (τ) = R X (τ ) Even function

R X (0) = E X

Autocovarinace function

2 (since t 1 = t 2 = t) denotes avg. power in RP

t

C X (t 1 , t 2 ) = E (X t 1 µ X t 1 )(X t 2 µ X t 1 )

Stationary RP: C X

B. Sainath (BITS, PILANI)

(t 1 , t 2 ) = R X (τ) µ 2

Probability–An Overview

Jan. 2018 17 / 29
Jan. 2018
17 / 29

Wide-Sense Stationary (WSS) RP

MV is independent of time (µ X (t) = µ X ) ACF: R X (t 1 , t 2 ) == R X (τ) Various RPs by Venn diagram

B. Sainath (BITS, PILANI) Probability–An Overview Jan. 2018 18 / 29
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
18 / 29

Gaussian RP

X (t) Gaussian RP at t = t i , i = 1

(µ X (t i ), C X (t i , t j ))

Higher order moments can be expressed in terms of first and second moments

n, RVs are jointly Gaussian

Gaussian RP completely specified by µ X (t i ) and C X (t i , t j )

If Gaussian RP is WSS implies also strict sense stationary

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 19 / 29
Jan. 2018
19 / 29

Cross-Correlation Function (CCF) & Cross Covariance

CCF of X (t) and Y (t):

R XY (t 1 , t 2 ) = E [X t 1 Y t 2 ]

Cross covariance:

=

−∞ −∞

x t 1 y t 2 p(x t 1 , y t 2 ) dx t 1 dy t 2 .

C XY (t 1 , t 2 ) = R XY (t 1 , t 2 )

µ X t 1 µ Y t 2

RPs X (t) and Y (t) are jointly and individually stationary:

C XY (t 1 , t 2 ) = R XY (τ) µ X µ Y and R XY (τ) = R YX (τ)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 20 / 29
Jan. 2018
20 / 29

Statistical Independence & Uncorrelated Condition

Statistical independence of X (t) and Y (t): t&t , n&m ∈ Z +

p(x t 1

x t n ; y t

1

m ) = p(x t 1

y

t

x t n )p(y t

1

m )

y

t

X (t) and Y (t) uncorrelated if R XY (t 1 , t 2 ) = µ X t 1 µ Y t 2 ⇒ C XY (t 1 , t 2 ) = 0

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 21 / 29
Jan. 2018
21 / 29

Complex-Valued RP

Z(t i ), i = 1

n (Z (t) = X (t) + jY (t))

Given by joint pdf of X (t i ), Y(t i ): p(x t 1

For instance: encounter in narrowband band-pass noise ACF:

x t n ; y t 1

y t n )

R ZZ (t 1 , t 2 ) = 1 2 E Z t 1 Z

t

2

Express in terms of real and img. parts

X (t), Y (t) jointly & individually stationary (Show this):

B. Sainath (BITS, PILANI)

R ZZ (t 1 , t 2 ) = R ZZ (τ) ∗ R ZZ
R ZZ (t 1 , t 2 ) =
R ZZ (τ)
R ZZ (τ) = R ZZ (−τ)
Probability–An Overview
Jan. 2018
22 / 29

Two Complex-Valued RP

Z (t) = X (t) + jY (t) and W (t) = U(t) + jV (t) two complex valued RPs:

R ZW (τ)R ZW (t 1 , t 2 ) = 1 2 E Z t 1 W

t

2

RPs pairwise-stationary:

R ZW (τ) = R ZW (τ)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 23 / 29
Jan. 2018
23 / 29

Power Spectral Density (PSD)

X (t) RP

PSD: Fourier Transform of ACF

S XX (f) =

−∞

R XX (τ ) exp (j2πf τ ) dτ

If RP real, PSD is real and even E X = S XX (f ) df

2

t

Cross PSD of cross correlation function (CCF):

B. Sainath (BITS, PILANI)

S XY (f) =

−∞

R XY (τ ) exp (j2πf τ ) dτ

Probability–An Overview

Jan. 2018 24 / 29
Jan. 2018
24 / 29

Response of LTI system to RP

linear time-invariant system (filter) characterized by

Impulse response h(t) (t-domain), its Fourier transform H(f )

h ( t ) (t-domain), its Fourier transform H ( f ) Output y ( t

Output y(t): Convolution of X (t) and h(t)

∞ y(t) = x(t − τ)h(τ) dτ −∞ Y(f) = X(f)H(f) B. Sainath (BITS, PILANI)
y(t) =
x(t − τ)h(τ) dτ
−∞
Y(f) = X(f)H(f)
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
25 / 29

LTI Example

AWGN input RP:

R XX (τ) = N 0 δ(τ) S XX (f) = N 0 for all f

2

2

Output RP PSD?

S YY (f) = N 0 |H(f)| 2

2

H(f ): transfer function of LTI system

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 26 / 29
Jan. 2018
26 / 29

Cyclostationary RP

RPs that have periodic statistical averages Let {a n }: Discrete-time sequence of RVs

x(t) =

−∞

a n g p (t nT )

Mean: µ a = E [a n ] for all n

ACF: R aa (k) = 2 E [a

g p (t) is deterministic {a n } represents(in modulation schemes):

1

n a n+k ]

Digital information sequence of symbols transmitted Rate of transmission:

1

T

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 27 / 29
Jan. 2018
27 / 29

Mean, ACF, PSD of x(t) [Proakis 2001]

Determine mean, ACF of x(t)

Show that statistical averages are periodic

How to compute time-average ACF and PSD? Exercise:

Y(t) = G 1 (ω)

cos(ω c t) + G 2 (ω) cos(ω c t)

G 1 ∼ N(µ 1 , σ 2 ) G 2 ∼ N (µ, σ

Find statistical averages of Y (t) Show that Y (t) is periodically (cyclo-) stationary

2

2

) independent RVs

1

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 28 / 29
Jan. 2018
28 / 29

Text Book & References

Text book: Digital communications by J. G. Proakis, 4 th Ed., MGH. References (brief list)

Principles of communication engg, Wozencraft and Jacobs Principles of digital communication by Gallager Wireless communication by Andrea Goldsmith Fundamentals of wireless communication by Tse & Viswanath

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018 29 / 29
Jan. 2018
29 / 29