You are on page 1of 45

Chapter 5: Signal Space Analysis

CHAPTER 5

SIGNAL SPACE ANALYSIS

Digital Communication Systems 2012


1/45
R.Sokullu
Chapter 5: Signal Space Analysis

Outline

5.1 Introduction
5.2 Geometric Representation of Signals
Gram-Schmidt Orthogonalization Procedure
5.3 Conversion of the AWGN into a Vector Channel
5.4 Maximum Likelihood Decoding
5.5 Correlation Receiver
5.6 Probability of Error

Digital Communication Systems 2012 R.Sokullu 2/45


Chapter 5: Signal Space Analysis

Introduction the Model


We consider the following model of a generic
transmission system (digital source):
A message source transmits 1 symbol every T sec
Symbols belong to an alphabet M (m1, m2, mM)
Binary symbols are 0s and 1s
Quaternary PCM symbols are 00, 01, 10, 11

Digital Communication Systems 2012 R.Sokullu 3/45


Chapter 5: Signal Space Analysis

Transmitter Side
Symbol generation (message) is probabilistic, with
a priori probabilities p1, p2, .. pM. or
Symbols are equally likely
So, probability that symbol mi will be emitted:

i P (mi )
1
= for i=1,2,....,M (5.1)
M

Digital Communication Systems 2012 R.Sokullu 4/45


Chapter 5: Signal Space Analysis

Transmitter takes the symbol (data) mi (digital


message source output) and encodes it into a
distinct signal si(t).
The signal si(t) occupies the whole slot T allotted
to symbol mi.
si(t) is a real valued energy signal (???)

T
E i si2 (t )dt , i=1,2,....,M (5.2)
0

Digital Communication Systems 2012 R.Sokullu 5/45


Chapter 5: Signal Space Analysis

Transmitter takes the symbol (data) mi (digital


message source output) and encodes it into a
distinct signal si(t).
The signal si(t) occupies the whole slot T allotted
to symbol mi.
si(t) is a real valued energy signal (signal with
finite energy)
T
E i si2 (t )dt , i=1,2,....,M (5.2)
0

Digital Communication Systems 2012 R.Sokullu 6/45


Chapter 5: Signal Space Analysis

Channel Assumptions:
Linear, wide enough to accommodate the signal si(t)
with no or negligible distortion
Channel noise is w(t) is a zero-mean white Gaussian
noise process AWGN
additive noise
received signal may be expressed as:

0tT
x(t ) si (t ) w(t ), (5.3)
i=1,2,....,M

Digital Communication Systems 2012 R.Sokullu 7/45


Chapter 5: Signal Space Analysis

Receiver Side
Observes the received signal x(t) for a duration of time T sec
Makes an estimate of the transmitted signal si(t) (eq. symbol mi).
Process is statistical
presence of noise
errors
So, receiver has to be designed for minimizing the average
probability of error (Pe)
What is this?
M
Pe = p P(m m
i 1
i i / mi ) (5.4)
Symbol sent
cond. error probability
given ith symbol was
sent Digital Communication Systems 2012 R.Sokullu 8/45
Chapter 5: Signal Space Analysis

Outline

5.1 Introduction
5.2 Geometric Representation of Signals
Gram-Schmidt Orthogonalization Procedure
5.3 Conversion of the AWGN into a Vector Channel
5.4 Maximum Likelihood Decoding
5.5 Correlation Receiver
5.6 Probability of Error

Digital Communication Systems 2012 R.Sokullu 9/45


Chapter 5: Signal Space Analysis

5.2. Geometric Representation of


Signals
Objective: To represent any set of M energy signals
{si(t)} as linear combinations of N orthogonal
basis functions, where N M
Real value energy signals s1(t), s2(t),..sM(t), each of
duration T sec Orthogonal basis
function

N
0tT
si (t ) sij j (t ), (5.5)
j 1 i==1,2,....,M
coefficient

Energy signal
Digital Communication Systems 2012 R.Sokullu 10/45
Chapter 5: Signal Space Analysis

Coefficients:
T i=1,2,....,M
sij si (t ) j (t )dt , (5.6)
0
j=1,2,....,M
Real-valued basis functions:
T
1 if i j
0 i (t ) j (t )dt ij 0 if i j (5.7)

Digital Communication Systems 2012 R.Sokullu 11/45


Chapter 5: Signal Space Analysis

The set of coefficients can be viewed as a N-


dimensional vector, denoted by si
Bears a one-to-one relationship with the
transmitted signal si(t)

Digital Communication Systems 2012 R.Sokullu 12/45


Chapter 5: Signal Space Analysis

Figure 5.3
(a) Synthesizer for generating the signal si(t). (b) Analyzer
for generating the set of signal vectors si .

Digital Communication Systems 2012 R.Sokullu 13/45


Chapter 5: Signal Space Analysis

So,
Each signal in the set si(t) is completely
determined by the vector of its coefficients
si1
s
i 2
.
si , i 1,2,....,M (5.8)
.
.

siN

Digital Communication Systems 2012 R.Sokullu 14/45


Chapter 5: Signal Space Analysis

Finally,
The signal vector si concept can be extended to 2D, 3D etc. N-
dimensional Euclidian space
Provides mathematical basis for the geometric representation of
energy signals that is used in noise analysis
Allows definition of
Length of vectors (absolute value)
Angles between vectors
Squared value (inner product of s i with itself)

2 Matrix
si siT s i Transposition
N
= sij2 , i 1,2,....,M (5.9)
j 1

Digital Communication Systems 2012 R.Sokullu 15/45


Chapter 5: Signal Space Analysis
Figure 5.4
Illustrating the geometric
representation of signals
for the case when N 2
and M 3.
(two dimensional space,
three signals)

Digital Communication Systems 2012 R.Sokullu 16/45


Chapter 5: Signal Space Analysis

Also,
What is the relation between the vector representation
of a signal and its energy value?

start with the definition T

of average energy in a E i si2 (t )dt (5.10)


0
signal(5.10)
N
Where si(t) is as in (5.5): si (t ) sij j (t ), (5.5)
j 1

Digital Communication Systems 2012 R.Sokullu 17/45


Chapter 5: Signal Space Analysis

T
N N

After substitution: E i sij j (t ) s (t )
ik k dt
0 j 1 k 1

N N T

After regrouping: Ei s s (t ) (t )dt


ij ik j k (5.11)
j 1 k 1 0

j(t) is orthogonal, so N
Ei s 2 2
= si (5.12)
finally we have: j 1
ij

The energy of a signal


is equal to the squared
length of its vector
Digital Communication Systems 2012 R.Sokullu 18/45
Chapter 5: Signal Space Analysis

Formulas for two signals


Assume we have a pair of signals: si(t) and sj(t), each
represented by its vector,
Then:
T
sij si (t ) sk (t )dt s s T
i k (5.13)
0

Inner product is invariant


to the selection of basis
Inner product of the signals functions
is equal to the inner product
of their vector
representations [0,T]
Digital Communication Systems 2012 R.Sokullu 19/45
Chapter 5: Signal Space Analysis

Euclidian Distance
The Euclidean distance between two points
represented by vectors (signal vectors) is equal to
||si-sk|| and the squared value is given by:
N
= (sij -skj ) 2
2
si s k (5.14)
j 1
T
= ( si (t ) sk (t )) 2 dt
0

Digital Communication Systems 2012 R.Sokullu 20/45


Chapter 5: Signal Space Analysis

Angle between two signals


The cosine of the angle ik between two signal vectors si and
sk is equal to the inner product of these two vectors, divided
by the product of their norms:

T
s s
cosik i k
(5.15)
si sk

So the two signal vectors are orthogonal if their inner product


siTsk is zero (cos ik = 0)

Digital Communication Systems 2012 R.Sokullu 21/45


Chapter 5: Signal Space Analysis

Schwartz Inequality
Defined as:


2
s1 (t )s2 (t )dt 2
s (t )dt s22 (t )dt (5.16)
1

accept without proof

Digital Communication Systems 2012 R.Sokullu 22/45


Chapter 5: Signal Space Analysis

Outline

5.1 Introduction
5.2 Geometric Representation of Signals
Gram-Schmidt Orthogonalization Procedure
5.3 Conversion of the AWGN into a Vector Channel
5.4 Maximum Likelihood Decoding
5.5 Correlation Receiver
5.6 Probability of Error

Digital Communication Systems 2012 R.Sokullu 23/45


Chapter 5: Signal Space Analysis

Gram-Schmidt Orthogonalization
Procedure
Assume a set of M energy signals denoted by s1(t), s2(t), .. , sM(t).

1. Define the first basis function s1 (t )


starting with s1 as: (where E is the 1 (t ) (5.19)
energy of the signal) (based on E1
5.12)
2. Then express s1(t) using the basis
function and an energy related s1 (t ) E11 (t ) = s111 (t ) (5.20)
coefficient s11 as:

3. Later using s2 define the T


coefficient s21 as: s21 s2 (t )1 (t )dt (5.21)
0

Digital Communication Systems 2012 R.Sokullu 24/45


Chapter 5: Signal Space Analysis

4. If we introduce the g 2 (t ) s2 (t ) s211 (t ) (5.22)


intermediate function g2 as:
Orthogonal to 1(t)

g 2 (t )
5. We can define the second 2 (t ) (5.23)
T
basis function 2(t) as:
0
g 22 (t )dt

6. Which after substitution of s2 (t ) s211 (t )


2 (t ) (5.24)
g2(t) using s1(t) and s2(t) it E2 s212
becomes:
T
Note that 1(t) and 2(t) are 0
22 (t )dt 1 (Look at 5.23)
orthogonal that means:
T
0
1 (t )2 (t )dt 0
Digital Communication Systems 2012 R.Sokullu 25/45
Chapter 5: Signal Space Analysis

And so on for N dimensional space,


In general a basis function can be defined using the
following formula:
i 1
g i (t ) si (t ) sij - j (t) (5.25)
j 1

where the coefficients can be defined using:


T
sij si (t ) j (t )dt , j 1, 2,....., i 1 (5.26)
0

Digital Communication Systems 2012 R.Sokullu 26/45


Chapter 5: Signal Space Analysis

Special case:
For the special case of i = 1 gi(t) reduces to si(t).

General case:

Given a function gi(t) we can define a set of basis


functions, which form an orthogonal set, as:
gi (t )
i (t ) , i 1, 2,....., N (5.27)
T

0
gi2 (t )dt

Digital Communication Systems 2012 R.Sokullu 27/45


Chapter 5: Signal Space Analysis

Outline

5.1 Introduction
5.2 Geometric Representation of Signals
Gram-Schmidt Orthogonalization Procedure
5.3 Conversion of the AWGN into a Vector Channel
5.4 Maximum Likelihood Decoding
5.5 Correlation Receiver
5.6 Probability of Error

Digital Communication Systems 2012 R.Sokullu 28/45


Chapter 5: Signal Space Analysis

Conversion of the Continuous AWGN


Channel into a Vector Channel
Suppose that the si(t) is
x(t ) si (t ) w(t ),
not any signal, but
specifically the signal at 0tT
(5.28)
the receiver side, defined i=1,2,....,M
in accordance with an
AWGN channel: T
x i x(t ) j (t ) dt
So the output of the 0

correlator (Fig. 5.3b) can =sij wi ,


be defined as: j 1, 2,....., N (5.29)
Digital Communication Systems 2012 R.Sokullu 29/45
Chapter 5: Signal Space Analysis

T T

sij si (t )i (t )dt (5.30) wi w(t )i (t )dt (5.31)


0 0

deterministic quantity random quantity

contributed by the sample value of the


transmitted signal si(t) variable Wi due to noise

Digital Communication Systems 2012 R.Sokullu 30/45


Chapter 5: Signal Space Analysis

Now,
Consider a random
N
process X1(t), with x1(t), a
x(t ) x(t ) x ji (t ) (5.32)
sample function which is j 1
related to the received N
signal x(t) as follows: x(t ) x(t ) ( sij w j ) j (t )
Using 5.28, 5.29 and 5.30 j 1

and the expansion 5.5 we N

get: =w(t ) w j j (t )
j 1

=w(t ) (5.33)
which means that the sample function x1(t) depends only on
the channel noise!
Digital Communication Systems 2012 R.Sokullu 31/45
Chapter 5: Signal Space Analysis

The received signal can


be expressed as:
N
x(t ) x ji (t ) x(t )
j 1
N
x ji (t ) w(t ) (5.34)
j 1

NOTE: This is an expansion similar to the one


in 5.5 but it is random, due to the additive
noise.

Digital Communication Systems 2012 R.Sokullu 32/45


Chapter 5: Signal Space Analysis

Statistical Characterization
The received signal (output of the correlator of
Fig.5.3b) is a random signal. To describe it we need
to use statistical methods mean and variance.
The assumptions are:
X(t) denotes a random process, a sample function of which
is represented by the received signal x(t).
Xj(t) denotes a random variable whose sample value is
represented by the correlator output xj(t), j = 1, 2, N.
We have assumed AWGN, so the noise is Gaussian, so X(t)
is a Gaussian process and being a Gaussian RV, X j is
described fully by its mean value and variance.

Digital Communication Systems 2012 R.Sokullu 33/45


Chapter 5: Signal Space Analysis

Mean Value
Let Wj, denote a random variable, represented by its
sample value wj, produced by the jth correlator in
response to the Gaussian noise component w(t).
So it has zero mean (by definition of the AWGN
model)
x j E X j
then the mean of
=E sij W j
Xj depends only on
=sij E[W j ]
sij:
x j = sij (5.35)

Digital Communication Systems 2012 R.Sokullu 34/45


Chapter 5: Signal Space Analysis

Variance
x2i var[ X j ]
Starting from the definition,
we substitute using 5.29 and =E ( X j sij ) 2
5.31
T =E W j2 (5.36)
wi w(t )i (t )dt (5.31)
0
T T

=E W (t ) j (t )dt W (u ) j (u )du
2
xi
0 0
T T
T T =E j (t )i (u )W (t )W (u )dtdu (5.37)
x2i = o 0
o
i (t ) j (u) E[W (t )W (u)]dtdu
0

T T

=E

0 j (t )i (u) Rw (t , u )dtdu (5.38) Autocorrelation function of
o the noise process
Digital Communication Systems 2012 R.Sokullu 35/45
Chapter 5: Signal Space Analysis
It can be expressed as:
(because the noise is N0
stationary and with a R w (t , u ) (t u ) (5.39)
2
constant power spectral
density) N T T
After substitution for
x2i = 0

2 (t ) (u) (t u)dtdu
o 0
i j

the variance we get: N0 T 2


=
2 0 j (t )dt (5.40)
And since j(t) has unit N0
energy for the variance x = 2
2
i
for all j (5.41)
we finally have:
Correlator outputs, denoted by Xj have variance
equal to the power spectral density N0/2 of the noise
process W(t). Digital Communication Systems 2012 R.Sokullu 36/45
Chapter 5: Signal Space Analysis

Properties (without proof)


Xj are mutually uncorrelated
Xj are statistically independent (follows from above
because Xj are Gaussian)
and for a memoryless channel the following equation
is true:
N
f x ( x / mi ) f x j ( x j / mi ), i=1,2,....,M (5.44)
j 1

Digital Communication Systems 2012 R.Sokullu 37/45


Chapter 5: Signal Space Analysis

Define (construct) a vector X of N random variables, X1, X2,


XN, whose elements are independent Gaussian RV with
mean values sij, (output of the correlator, deterministic part of
the signal defined by the signal transmitted) and variance
equal to N0/2 (output of the correlator, random part,
calculated noise added by the channel).
then the X1, X2, XN , elements of X are statistically
independent.
So, we can express the conditional probability of X, given si(t)
(correspondingly symbol mi) as a product of the conditional
density functions (fx) of its individual elements fxj.
NOTE: This is equal to finding an expression of the probability
of a received symbol given a specific symbol was sent,
assuming a memoryless channel

Digital Communication Systems 2012 R.Sokullu 38/45


Chapter 5: Signal Space Analysis

that is:
N
f x ( x / mi ) f x j ( x j / mi ), i=1,2,....,M (5.44)
j 1

where, the vector x and the scalar xj, are sample


values of the random vector X and the random
variable Xj.

Digital Communication Systems 2012 R.Sokullu 39/45


Chapter 5: Signal Space Analysis

N
f x ( x / mi ) f x j ( x j / mi ), i=1,2,....,M (5.44)
j 1

Vector x is called
observation vector
Vector x and scalar xj Scalar xj is called
are sample values of observable element
the random vector X
and the random
variable Xj

Digital Communication Systems 2012 R.Sokullu 40/45


Chapter 5: Signal Space Analysis

Since, each Xj is Gaussian with mean sj and variance


N0/2

1 2 j=1,2,....,N
f x j ( x / mi ) ( N 0 ) N /2
exp ( x j sij ) , (5.45)
N0 i=1,2,....,M

we can substitute in 5.44 to get 5.46:

1 N 2
f x ( x / mi ) ( N 0 ) exp ( x j sij ) ,
N /2

N 0 j 1 i=1,2,....,M (5.46)

Digital Communication Systems 2012 R.Sokullu 41/45


Chapter 5: Signal Space Analysis

If we go back to the formulation of the received


signal through a AWGN channel 5.34
N
x(t ) x ji (t ) x(t )
j 1
N
x ji (t ) w(t ) (5.34)
j 1

Only projections of the noise onto


the basis functions of the signal set
The vector that we
{si(t)Mi=1 affect the significant
have constructed fully statistics of the detection problem
defines this part

Digital Communication Systems 2012 R.Sokullu 42/45


Chapter 5: Signal Space Analysis

Finally,
The AWGN channel, is equivalent to an N-
dimensional vector channel, described by the
observation vector

x si w, i 1, 2,....., M (5.48)

Digital Communication Systems 2012 R.Sokullu 43/45


Chapter 5: Signal Space Analysis

Outline

5.1 Introduction
5.2 Geometric Representation of Signals
Gram-Schmidt Orthogonalization Procedure
5.3 Conversion of the AWGN into a Vector Channel
5.4 Maximum Likelihood Decoding
5.5 Correlation Receiver
5.6 Probability of Error

Digital Communication Systems 2012 R.Sokullu 44/45


Chapter 5: Signal Space Analysis

Maximum Likelihood Decoding


to be continued.

Digital Communication Systems 2012 R.Sokullu 45/45