Вы находитесь на странице: 1из 32

Probability–An Overview

B. Sainath (BITS, PILANI)

B. Sainath EEE Dept., BITS PILANI

Jan. 2018

Probability–An Overview

Jan. 2018
1 / 29

Outline

1 Ordered Random Variables

2 Complex Random Variables

3 Random Vectors

4 Random Signals

5 Random Processes

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
2 / 29

Ordered RVs & Statistics

Applications in wireless communications (eg. antenna selection/relay selection) Eg. Z = Max(X,Y)

Find pdf: Max(X 1 ,

, X n ) and Min(X 1 ,

,

X n )

Eg. Min. of independent RVs

P(Min {X 1 ,

, X n } > y) = P(X 1 > y,

, X n > y)

= P(X 1 > y )
P(X
n > y)
Exercise:
Derive Max. RV pdf when X i ∼ exp(1), i = 1,
Find MV, MSV, Variance
, n
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
3 / 29

Complex RVs

Noise waveforms after demodulation to baseband are cplx. Cplx. RV: Z = Z re + jZ Im

Z Gaussian if Z re and Z Im are jointly Gaussian MGF of Z if Z re and Z Im are jointly standard Gaussian ? MAR (Magnitude-Angle representation) R exp(jΘ)

R =?
Θ =?
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
4 / 29
Example Problem
Let Z = X + jY , X ∼ N (0, σ 2 ) and Y ∼ N (0, σ 2 ) are independent RVs.
2
2
It’s magnitude R = √ X 2 + Y 2 and phase Θ = arctan(Y /X )

Find pdfs of R and Θ

Hint: Use Cartesian to Polar transformation

Resultant pdfs are ? and ? (prove)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
5 / 29
Example Problem
Let Z = X + jY , X ∼ N (0, σ 2 ) and Y ∼ N (0, σ 2 ) are independent RVs.
2
2
It’s magnitude R = √ X 2 + Y 2 and phase Θ = arctan(Y /X )

Find pdfs of R and Θ

Hint: Use Cartesian to Polar transformation

Resultant pdfs are ? and ? (prove)

Magnitude: Rayleigh distribution Phase: Uniform distribution

Distribution of R 2 ?

Exponential distribution

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
5 / 29

Gaussian Random Vectors

Standard Gaussian random vector

Collection of n i.i.d. standard Gaussian RVs X 1 ,

X = [X 1 ,

,

X n ] t

, X n ∈ R n

pdf of X ?

P X (x) =

1

(2π)

n

2

exp x 2

2

Result: Let O is orthogonal matrix (transformation). If X is std. Gaussian, OX is also std. Gaussian (proof in class) Qs:

How is x 2 distributed (Ans. χ-square RV with n degrees of freedom)

How is x 2 = x

1

+ x

2

2

distributed?

2

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
6 / 29

Gaussian Random Vectors

Standard Gaussian random vector

Collection of n i.i.d. standard Gaussian RVs X 1 ,

X = [X 1 ,

,

X n ] t

, X n ∈ R n

pdf of X ?

P X (x) =

1

(2π)

n

2

exp x 2

2

Result: Let O is orthogonal matrix (transformation). If X is std. Gaussian, OX is also std. Gaussian (proof in class) Qs:

How is x 2 distributed (Ans. χ-square RV with n degrees of freedom)

How is x 2 = x

1

+ x

2

2 distributed? (Ans. Exponential)

2

Y = AX + µ Gaussian random vector

A: Matrix representing linear transformation µ mean vector pdf of Y?

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
6 / 29

Further Transformation

For any c ∈ R n , c t Y ?

A std. Gaussian random vector is also Gaussian

Mean: zero vector Covariance matrix: I n

Any linear combination of elements of a Gaussian random vector is also Gaussian

If A invertible, Gaussian random vector is completely characterized by

Mean vector Covariance matrix: Symmetric, non-negative deﬁnite

Symmetric n × n real matrix M is said to be non-negative deﬁnite if c t Mc is non-negative for every non-zero column vector c ∈ R n

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
7 / 29

SOME KEY POINTS

O orthogonal matrix

Covariance matrices of Y and OY same = AA t (show this) White Gaussian random vector

Independent Gaussian RVs and covariance matrix diagonal RVs are uncorrelated

What if A is not invertible?

Some components of AX linearly dependent (LD) Focus on components that are linearly independent (LI)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
8 / 29

Complex Gaussian Random Vectors

Cplx. random vector: Z = Z re + jZ Im

Z re and Z Im are real random vectors Distribution completely speciﬁed by mean and covariance matrix of real random vector [Z re , Z Im ] t

Mean µ, Covariance matrix K, Pseudo-covariance J

µ E [Z]

 K E [(Z − µ)(Z − µ) ∗ ] J E (Z − µ)(Z − µ) t

Covariance matrix: Hermitian (K = K )

Complete second order statistics: n + n 2 + n 2 = n(2n + 1) Note:

In wireless communication, we are interested in circularly symmetric complex Gaussian random vectors

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
9 / 29

Circulary Symmetric Complex Gaussian (CSCG)

Result:

Z is circularly symmetric complex random vector if e jθ Z has same

distribution of Z for any θ

Circularly symmetric complex random vector Z

Mean zero (Proof in class)

Pseudo-covariance zero (Proof in class)

K fully speciﬁes ﬁrst and second order statistics Z ∼ CN (0, K)

Spl. Cases

CSCG must have i.i.d.zero mean Re. and Im. components

Statistics fully speciﬁed by σ 2 , Z ∼ CN(0, σ 2 )

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
10 / 29

Density Function of Standard CSCG Random Vector

Standard CSCG RV Z

Z ∼ CN(0, 1), z ∈ C Re. and Im. parts are RVs with variance

1

2

P Z (z) =

1

π

exp z 2

Standard CSCG random vector Z ∼ CN (0, I), z ∈ C n

Collection of n i.i.d.Z ∼ CN (0, 1)

P Z (z) =

1 n exp z 2

π

Property: UZ has same distribution as Z (U: unitary matrix) Q. Let A is an invertible matrix. pdf of X = AZ?

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
11 / 29

Random Signals or Waveforms

Deterministic signals modeled by mathematical formula

Not enough to model real-world phenomena

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
12 / 29

Random Signals or Waveforms

Deterministic signals modeled by mathematical formula

Not enough to model real-world phenomena

Random signals (due to random processes)

Modeled in probabilistic terms Eg., Speech, Stock market exchange rates, Noise, Interference, ECG, EEG

Figure: Types of waveforms

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
12 / 29

Random (stochastic) Processes (RPs)

RV: Assignment of real number X (ω) to each point ω of sample space in such a way that CDF exists

Random process X (t) (evolves with time): Assign a real-time function (called sample function) X (ω, t) to each point ω

Figure: Simple random process. Source: WC & J [1965]

Probability space: Sample space, waveforms, probability measure

Collection of waveforms with probabilities deﬁne RV at observed time instant

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
13 / 29

Examples of Stochastic Processes

Gaussian processes, Brownian motion, Markov chains, Martingales Speciﬁcation of RP

A rule is given or implied for determining joint pdf P X(t) for any ﬁnite set of

observation instants (t 1 ,

, t n )

Three methods of specifying RP

Direct way to ﬁnd joint pdf, eg. Gaussian process (discuss later) Time function involving one or more parameters, eg.

Y (t) = R sin(2πt + Θ)

Sample functions: Y (ω, t) = R(ω) sin(2πt + Θ(ω)), ω To ﬁnd P Y(t) , knowledge of P R,Θ required Eg. R, Θ as polar coordinates, is 2D plane A possible joint pdf

B. Sainath (BITS, PILANI)

P R,Θ (r, θ) =

2π exp r 2 ;

r

2

0;

Probability–An Overview

0 r < , 0 θ < 2π otherwise

Jan. 2018
14 / 29

Third Method

Generate ensemble of RP by applying stated operation to sample functions of known process

Eg. Time translation: Y (ω, t) = X (ω, t + T ), ω

Thus P Y(t) = P X(t+T)

Eg. Linear ﬁltering

h(t) impulse response

Y(t) =

−∞

h(t τ)X(τ)

In general, ﬁnding P Y(t) difﬁcult unless X (t) is Gaussian process

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
15 / 29

Stationary RPs

Stationary in strict sense: Let p(x t 1

x t n ) denote joint pdf

p(x t 1

x t n ) = p(x t 1 +T

x t n +T ), T&n

Statistics of stationary RPs are invariant to time-axis translation Non-stationary if joint pdfs are different

Statistical (or ensemble) averages:

n th moment of RV X t i :

E X =

n

t

i

−∞

n

t

x

i

p(x t i ) dx t i .

Autocorrelation function (ACF) R X (t 1 , t 2 )

E [X t 1 , X t 2 ] =

−∞ −∞

x t 1 x t 2 p(x t 1 , x t 2 ) dx t 1 dx t 2 .

For stationary RP: R X (t 1 , t 2 ) = R X (t 1 t 2 ) = R X (τ ), where τ = t 1 t 2

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
16 / 29

Some Properties of ACF & Covariance Function

R X (τ) = R X (τ ) Even function

R X (0) = E X

Autocovarinace function

2 (since t 1 = t 2 = t) denotes avg. power in RP

t

C X (t 1 , t 2 ) = E (X t 1 µ X t 1 )(X t 2 µ X t 1 )

Stationary RP: C X

B. Sainath (BITS, PILANI)

(t 1 , t 2 ) = R X (τ) µ 2

Probability–An Overview

Jan. 2018
17 / 29

Wide-Sense Stationary (WSS) RP

MV is independent of time (µ X (t) = µ X ) ACF: R X (t 1 , t 2 ) == R X (τ) Various RPs by Venn diagram

B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
18 / 29

Gaussian RP

X (t) Gaussian RP at t = t i , i = 1

(µ X (t i ), C X (t i , t j ))

Higher order moments can be expressed in terms of ﬁrst and second moments

n, RVs are jointly Gaussian

Gaussian RP completely speciﬁed by µ X (t i ) and C X (t i , t j )

If Gaussian RP is WSS implies also strict sense stationary

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
19 / 29

Cross-Correlation Function (CCF) & Cross Covariance

CCF of X (t) and Y (t):

R XY (t 1 , t 2 ) = E [X t 1 Y t 2 ]

Cross covariance:

=

−∞ −∞

x t 1 y t 2 p(x t 1 , y t 2 ) dx t 1 dy t 2 .

C XY (t 1 , t 2 ) = R XY (t 1 , t 2 )

µ X t 1 µ Y t 2

RPs X (t) and Y (t) are jointly and individually stationary:

C XY (t 1 , t 2 ) = R XY (τ) µ X µ Y and R XY (τ) = R YX (τ)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
20 / 29

Statistical Independence & Uncorrelated Condition

Statistical independence of X (t) and Y (t): t&t , n&m ∈ Z +

p(x t 1

x t n ; y t

1

m ) = p(x t 1

y

t

x t n )p(y t

1

m )

y

t

X (t) and Y (t) uncorrelated if R XY (t 1 , t 2 ) = µ X t 1 µ Y t 2 ⇒ C XY (t 1 , t 2 ) = 0

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
21 / 29

Complex-Valued RP

Z(t i ), i = 1

n (Z (t) = X (t) + jY (t))

Given by joint pdf of X (t i ), Y(t i ): p(x t 1

For instance: encounter in narrowband band-pass noise ACF:

x t n ; y t 1

y t n )

R ZZ (t 1 , t 2 ) = 1 2 E Z t 1 Z

t

2

Express in terms of real and img. parts

X (t), Y (t) jointly & individually stationary (Show this):

B. Sainath (BITS, PILANI)

R ZZ (t 1 , t 2 ) =
R ZZ (τ)
R ZZ (τ) = R ZZ (−τ)
Probability–An Overview
Jan. 2018
22 / 29

Two Complex-Valued RP

Z (t) = X (t) + jY (t) and W (t) = U(t) + jV (t) two complex valued RPs:

R ZW (τ)R ZW (t 1 , t 2 ) = 1 2 E Z t 1 W

t

2

RPs pairwise-stationary:

R ZW (τ) = R ZW (τ)

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
23 / 29

Power Spectral Density (PSD)

X (t) RP

PSD: Fourier Transform of ACF

S XX (f) =

−∞

R XX (τ ) exp (j2πf τ ) dτ

If RP real, PSD is real and even E X = S XX (f ) df

2

t

Cross PSD of cross correlation function (CCF):

B. Sainath (BITS, PILANI)

S XY (f) =

−∞

R XY (τ ) exp (j2πf τ ) dτ

Probability–An Overview

Jan. 2018
24 / 29

Response of LTI system to RP

linear time-invariant system (ﬁlter) characterized by

Impulse response h(t) (t-domain), its Fourier transform H(f )

Output y(t): Convolution of X (t) and h(t)

y(t) =
x(t − τ)h(τ) dτ
−∞
Y(f) = X(f)H(f)
B. Sainath (BITS, PILANI)
Probability–An Overview
Jan. 2018
25 / 29

LTI Example

AWGN input RP:

R XX (τ) = N 0 δ(τ) S XX (f) = N 0 for all f

2

2

Output RP PSD?

S YY (f) = N 0 |H(f)| 2

2

H(f ): transfer function of LTI system

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
26 / 29

Cyclostationary RP

RPs that have periodic statistical averages Let {a n }: Discrete-time sequence of RVs

x(t) =

−∞

a n g p (t nT )

Mean: µ a = E [a n ] for all n

ACF: R aa (k) = 2 E [a

g p (t) is deterministic {a n } represents(in modulation schemes):

1

n a n+k ]

Digital information sequence of symbols transmitted Rate of transmission:

1

T

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
27 / 29

Mean, ACF, PSD of x(t) [Proakis 2001]

Determine mean, ACF of x(t)

Show that statistical averages are periodic

How to compute time-average ACF and PSD? Exercise:

Y(t) = G 1 (ω)

cos(ω c t) + G 2 (ω) cos(ω c t)

G 1 ∼ N(µ 1 , σ 2 ) G 2 ∼ N (µ, σ

Find statistical averages of Y (t) Show that Y (t) is periodically (cyclo-) stationary

2

2

) independent RVs

1

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
28 / 29

Text Book & References

Text book: Digital communications by J. G. Proakis, 4 th Ed., MGH. References (brief list)

Principles of communication engg, Wozencraft and Jacobs Principles of digital communication by Gallager Wireless communication by Andrea Goldsmith Fundamentals of wireless communication by Tse & Viswanath

B. Sainath (BITS, PILANI)

Probability–An Overview

Jan. 2018
29 / 29