1

Lecture 1
Sampling of Signals
by
Graham C. Goodwin
University of Newcastle
Australia
Lecture 1
Presented at the Zaborszky Distinguished Lecture Series
December 3rd, 4th and 5th, 2007

2
Recall Basic Idea of Sampling
and Quantization
Quantization
Sampling
t
1
t
3
t
2
t
4
t

0
1
2
3
4
5
6
3
In this lecture we will ignore quantization
issues and focus on the impact of different
sampling patterns for scalar and
multidimensional signals
4
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
5
Sampling: Assume amplitude quantization
sufficiently fine to be negligible.


Question: Say we are given



Under what conditions can we recover
from the samples?

( )
; f t t
( )
;
i
f t i Z
6
A Well Known Result (Shannons
Reconstruction Theorem for Uniform
Sampling)
Consider a scalar signal f(t) consisting of
frequency components in the range . If
this signal is sampled at period , then the
signal can be perfectly reconstructed from the
samples using:

[ ]
( )
( )
sin
2
( )
2
s
s k
t k
y t y k
t k
w
w

= -

- D

- D

,
2 2
s s
w w

-

2
s
p
w
D <
7
Low pass filter recovers
original spectrum
Hence
or
( )
s
Y w
2
s
w -
2
s
w
s
w
( ) ( ) ( )
( ) 1
2 2
0 otherwise
s s
s s
s
Y H Y
H
w w w
w w
w w
=

-

=
( ) ( ) ( )
( ) [ ] ( )
[ ] ( )
s
s
s
k
s
k
y t h y t d
h y k t k d
y k h t k
s s s
s d s s

-
=-

=-
= -
= - - D
= - D

Proof: Sampling produces folding

8
A Simple (but surprising) Extension
where
[Recurrent Sampling]
is a periodic sequence of integers; i.e.,
Let
Note that the average sampling period is
e.g.
average 5
k k
M D = D
{ }
k
M
k N k
M M
+
=
1
N
k
k
M K
=
=

T K = D
K
N
D
= D
1
2
3
4
9
1
9
1
D =
D =
D =
D =
9
Non-uniform
Uniform
0 9 -1 10 19 20
x x x x x x
0 5 10 15 20
x x x x x
10
Claim:
Provided the signal is bandlimited to
where , then the signal can be
perfectly reconstructed from the periodic
sampling pattern.
where = average sampling period

Proof:
We will defer the proof to later when we
will use it as an illustration of Generalized
Sampling Expansion (GSE) Theorem.
,
2 2
s s
w w

-

2
s
p
w =
D
D
11
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
12
Multidimensional Signals
Digital Photography
Digital Video
x
1

x
2

x
1

x
2

x
3
(time)
13
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
14
How should we define sampling for multi-
dimensional signals?

Utilize idea of Sampling Lattice



Sampling Lattice
nonsingular matrix
D D
T
( )
{ }
:
D
Lat T Tn n Z L = =
15
Also, need multivariable frequency domain
concepts.

These are captured by two ideas
i. Reciprocal Lattice
ii. Unit Cell

16
Unit Cell (Non-unique)
i.



ii.
Reciprocal Lattice
( )
{ }
( )
{ }
1 1
*
2 2 :
T T D
Lat T T n n Z p p
- -
L = =
( ) ( )
{ }
( ) ( )
{ }
1 1
* *
1 2
1 2 1 2
2 2
,
T T
D
UC T n UC T n
n n Z n n
p p
- -
L + L + =

( )
( )
{ }
1
2
D
T D
n Z
UC T n R p
-

L + =
U
( )
*
UC L
17
One Dimensional Example
Sampling Lattice
0
-20 10 20
x x x x
D
{ }
. : n n Z L = D
-10
x
18
Reciprocal Lattice and Unit Cell
Unit Cell
1
2
w
p
0
1
10
2
10
3
10
19
Multidimensional Example
x
1

x
2
1 2 3 4 5
-4 -3 -2 -1
-1
-2
-3
-4
5
4
3
2
1
2 1
0 2
T


=


20
Reciprocal Lattice and Unit Cell for
Example
1/4 1/2 3/4 1
-1/4
-1/2
-3/4
-1
1/2
1/4
( )
1
1
0
2
1 1
4 2
T
T
-



=


-


( )
( )
1
2
T
UC T p
-
1
2
w
p
2
2
w
p
21
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
22
We will be interested here in the situation where
the Sampling Lattice is not a Nyquist Lattice for
the signal (i.e., the signal cannot be perfectly
reconstructed from the original pattern!)

Strategy:
We will generate other samples by filtering
or shifting operations on the original pattern.
23
Consider a bandlimited signal .
Assume the D-dimension Fourier transform has finite
support, S.
Then for given D-dimensional lattice T, there always
exists a finite set , such that support
( )
,
D
f x x
{ }
*
1
P
i
w L
( )
( )
( ) ( )
*
1

.
P
i
i
f S UC w w
=
= L +
U
Heuristically: The idea of Tiling the
area of interest in the frequency domain
24
One Dimensional Example
Our one dimensional example continued.
Sampling Lattice
{ }
; k k Z L = D
Unit Cell
1
2
w
p
0
1
10
2
10
3
10
( )

f w
Bandlimited spectrum
Use
1
2
0
2
10
w
p
w
=

= -

( )
( )
( ) ( )
* *
2

f UC UC w w

= L L +


U
Support
1
12
1
12
- 2
w
p
25
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
26
Generation of Extra Samples
Suppose now we generate a data set
as shown in below
( )
{ }
{ }
1 D
Q
q
q
n Z
g Tn
=

( )
Q P
Q Channel Filter Bank
( ) f x
( )
1

h w
( )

q
h w
( )

Q
h w
L
L
L
( )
1
g x
( )
q
g x
( )
Q
g x
M
M
27
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
28
Define
Let
be the solution (if it exists) of
for
( ) ( ) ( )
( )
( ) ( )
1 1 2 1 1
1 2
1

( )

Q
P Q P
h h h
h
H
h h
w w w w w w
w w
w
w w w w

+ + +


+

=




+ +

L
M
( )
( )
( )
1
,
,
,
Q
x
x
x
w
w
w

F


F =


F


M
( )
*
UC w L
( ) ( )
1
,
T
T
P
j x
j x
e
H x
e
w
w
w w



F =




M
29
Conditions for Perfect
Reconstruction
can be reconstructed from




if and only if has full row rank for all in
the Unit Cell

where
( )
H w
( )
f x
( ) ( ) ( )
1
D
Q
q q
q
k Z
f x g Tk x Tk f
=

= -

( ) ( )
,
T
j x
q q
UC
x x e d
w
f w w = F

GSE Theorem:
w
30
Proof:

Multiply both sides by where (the
Reciprocal Lattice). Then sum over q





Note that tiles the entire
support S
Thus,
( ) ( )
( )
*
, ;
T T
D
j x j Tk
q q
k Z
x e x Tk e UC
w w
w f w
-

F = - L

( )

q i
h w w +
( ) ( ) ( ) ( )
( )
1 1

,
T T
D
T
i
Q Q
j Tk j x
q i q q i q
q q k Z
j x
h x Tk e h x e
e
w w
w w
w w f w w w
-
= =
+
+ - = + F
=

from the Matrix identity that defines
( ) ( ) ( )
( )
( )
( ) ( ) ( )
( )
*
*
1
1 1


T
T
i
T
D
P
j x
j x
i
i
s
UC
Q P
j Tk
i q i q
i q k Z
UC
f x f e d f e d
f h x Tk e d
w w
w
w
w w w w w
w w w w f w
- +
=
L
-
= =
L
= = +
= + + -

*
i
w L
( ) , x w F
{ }
*
; and 1, ,
i
i P w w w + L = K
31
where we have used the fact that

Since is the output of f(x) passing through ,
then


Hence, we finally have
( ) ( ) ( )
( )
( )
( )
* 1 1

T
i
D
P
j Tk
i q i
i
UC
Q
q
q k Z
f x x Tk f h e d
w w
w w w f w w
- +
= =
L


= -



+ +

( )
1
2 for .
T D
i
T Z w p
-
= l l
( )
q
g x
[ ] ( )
q
g Tk =
( ) ( ) ( )
1
D
Q
q q
q k Z
f x g Tk x Tk f
=
= -

( )

q
h w

32
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
33
Special Case: Recurrent Sampling
(where is implemented by a spatial shift )
This amounts to the sampling pattern:



where w.l.o.g.

Now, given the samples , our goal is to
perfectly reconstruct
( )
{ }
1
Q
q
q
Lat T x
=
Y= +
I
{ }
( )
q
x UC T
( ) { }
x
f x
Y %
%
( )
. f x
q
h
q
x
34
Here , and

Thus

To apply the GSE Theorem we require
( )

T
q
j x
q
h e
w
w =
( )
( )
q q
g x f x x = +
( )
{ }
( ) { }
, 1,
P q
x n Z q Q
g Tn f x
Y =
=
%
K
%
Nonsingular
( )
( ) ( )
( ) ( )
1 1 1
1
1
1 1
1
1
0
0
T T
Q
T T
P Q P
T
T
T
Q
T
T
T
Q
P Q
P
j x j x
j x j x
j x
j x
j x
j x
j x
j x
e e
H
e e
e e
e
e
e e
w w w w
w w w w
w
w
w
w
w
w
w
+ +
+ +


=







=






L
L
L
O
L
35
Something to think about
The GSE result depends on inversion of a
particular matrix, H(w). Of course we have
assumed here perfect representation of all
coefficients. An interesting question is
what happens when the representation is
imperfect i.e. coefficients are represented
with finite wordlength (i.e. they are
quantized)
We will not address this here but it is
something to keep in mind.
36
Return to our one-dimensional
example
Recall that we had

so that

support

Say we use recurrent sampling with
1
2
0
2
10
w
p
w
=
= -
1
2
0
0.9 ; 10
x
x
=
= D D =
( )
( )
( ) ( )
* *
2

f UC UC w w

= L L +


37
0 10 20
x x x
0
9
19
x x x
0
x x x
1
0 x =
2
0.9 x = D
-1
-1
x x
19 20
x
9 10
38
Condition for Perfect Reconstruction is
nonsingular
( )( )
1 1 1 2
2 1 2 2
0.9 2
1 1
1
j x j x
j x j x
j
e e
e e
e
w w
w w
p -






=


Hence, the original signal can be recovered
from the sampling pattern given in the
previous slide.
39
Summary
We have seen that the well known
Shannon reconstruction theorem can be
extended in several directions; e.g.
Multidimensional signals
Sampling on a lattice
Recurrent sampling
Given specific frequency domain
distributions, these can be matched to
appropriate sampling patterns.
40
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
41
Application: Video Compression
Source
Introduction to video cameras
Instead of tape, digital cameras use 2D sensor
array (CCD or CMOS)
Image
Processor
Image
Processor
Memory
Image
Processor
Image
Processor
Image
Processor
Image
Processing
Display
( TV or LCD )
Pipeline
DVCD
controller
42
Image Sensor
A 2D array of sensors replaces the traditional
tape

Each sensor records a 'point' of the
continuous image

The whole array records the continuous
image at a particular time instant

43
2D Colours Sensor Array
Data transfer from array is sequential
and has a maximal rate of Q.
* Based on http://www.dpreview.com/learn/
44
Uniform 3D sampling

a sequence of identical frames equally spaced in
time
Current Technology
45
The volume of box depends on the capacity:
pixel rate = (frame rate) x (spatial resolution)
x ex
Video Bandwidth
depends on
spatial resolution
of the frames

depends on
the frame rate

46
1. Data recording on sensor:
Sensor array density
- for spatial resolution
pixels
frame
R
(
(
(

Sensor exposure time
- for frame rate
frames
sec.
F
(
(
(

2. Data reading from sensor:
Data readout time
- for pixel rate
pixels
sec.
Q
(
(
(

Hard Constraints
47
Generally Q << RF
Need: R
1
< R F
1
< F
s.t. R
1
F
1
= Q



Compromise:
spatial resolution R
1
< R
temporal resolution F
1
< F
BUT...
48
x
e
y
e
t
e
1
R
o
1
F
o
volume determined
by
1 1
Q R F =
Actual Capacity (Data Readout)
49
Observation
Most energy of typical video scene
is concentrated around the
plane and the axis.
,
x y
e e
t
e
t
e
x
e
50
t
e
x
e
uniform sampling
- compromise
in frame rate
uniform sampling
- compromise
in spatial resolution
uniform sampling
- no compromise
The Spectrum of this Video Clip
52
y A
x
A
( ) 2 1 N y + A
(
)
2
1
L
x
+
A
frame type A frame type B
t
t A ( )
2
2 1 M t + A
( )
1
2 1 M t + A
Recurrent Non-Uniform Sampling
53
y
e
t
e
x
e
What Does it Buy?
54
Schematic Implementation
t
Filter
bank
t
t
non-uniform data from the sensor
uniform high def. video
'compression at the source'
55
Recurrent Non-Uniform Sampling
A special case of
Generalized Sampling Expansion Theorem
56
Sampling Pattern
)
`

A
+
)
`

A
+ = +

+ = =
t m
U LAT
x l
U LAT U LAT
M
M m
L
l
0
) (
0
) ( ) (
1
2
2
1 2
2
1
{ }
s
M L
s
x U LAT + = +

+ +
=
) (
1 ) ( 2
1
The resulting sampling pattern is given by:
57
Frequency Domain
{ }
r
T
M L
r
U UC S e + A =

+ +
=

) 2 (
1 ) ( 2
1
where:
)
`

A +
<
A +
<
(

= =
=
t M x L
U UC
t x
t
x T
) 1 2 (
,
) 1 2 (
: ) 2 (
1
t
e
t
e
e
e
e t
is the unit cell of the reciprocal lattice

e
(
(
(
(

A +
A +
=
2
1
:
) 1 2 (
0
0
) 1 2 (
) 2 ( Z n n
t M
x L
U LAT
T
t
t
t
58
Reciprocal Lattice
x A

t
x A
t
t M
M
A

+
+

t
) 1 2 (
1 2
1
t M
M
A

+
+ t
) 1 2 (
1 2
1
e
x
e
t
t M A + ) 1 2 (
1
t
x L A + ) 1 2 (
t
Unit cell
59
Apply the GSE Theorem
) ( ) (
1 ) ( 2
2
1
x H
M L
e =
(
(
(
(
(

u
u
u
+ +
where: is uniquely defined by H
1
H
2
()
is a set of 2(L+M)+1 constraints
) ( ) ( ) , (
1
x H x e e

= u
If exists, we can find the reconstruction function
1
H
( )
H w
60
Reconstruction Scheme
H
1
u
1
H
2L+1
u
2L+1
H
2(L+M)+1
u
2(L+M)+1

I(x,t) (x,t)
1 M) 2(L
frequency Nyquist
+ +
(

=
r
r T
r
t
x
j
s r
e H
e
.
The sub-sampled frequency of each filter H is:
61
Reconstruction functions
t
t 1) (2M
sin
) ) 1 ( (
x) 1) - (r ( sin
) 1 2 ( ) , (
t
t
t
t

|
|
.
|

\
|
A +
A
|
.
|

\
|
A
A
A A + =
x r x
x
x
t x L t x
r
) ) 1 2 ( (
t) 1) - 2L - (r ( sin
x
x 1) (2L
sin
) 1 2 ( ) , (
t L r t
t
t
t x M t x
r
A
|
.
|

\
|
A
A
|
|
.
|

\
|
A +
A A + =
t
t
t
t

for r = 2,3,,2L+1
for r = 2(L+1),,2(L+M)+1
Multidimensional sinc like functions
62
Demo
Full resolution
sequence
Reconstructed
sequence
Temporal
decimation
Spacial
decimation
63
Outline
1. One Dimensional Sampling
2. Multidimensional Sampling
3. Sampling and Reciprocal Lattices
4. Undersampled Signals
5. Filter Banks
6. Generalized Sampling Expansion (GSE)
7. Recurrent Sampling
8. Application: Video Compression at Source
9. Conclusions
64
Conclusions
Nonuniform sampling of scalar signals
Nonuniform sampling of multidimensional
signals
Generalized sampling expansion
Application to video compression
A remaining problem is that of joint design of
sampling schemes and quantization strategies
to minimize error for a given bit rate
65
References
One Dimensional Sampling
A. Feuer and G.C. Goodwin, Sampling in Digital Signal Processing and Control.
Birkhuser, 1996.
R.J. Marks II, Ed., Advanced Topics in Shannon Sampling and Interpolation Theory.
New Your: Springer-Verlag, 1993.

Multidimensional Sampling
W.K. Pratt, Digital Image Processing, 3rd ed: John Wiley & Sons, 2001.
B.L. Evans, Designing commutative cascades of multidimensional upsamplers and
downsamplers, IEEE Signal Process Letters, Vol4, No.11, pp.313-316, 1997.

Sampling and Reciprocal Lattices, Undersampled Signals
A.Feuer, G.C. Goodwin, Reconstruction of Multidimensional Bandlimited Signals for
Uniform and Generalized Samples, IEEE Transactions on Signal Processing, Vol.53,
No.11, 2005.
A.K. Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, NJ: Prentice-
Hall, 1989.
66
References
Filter Banks
Y.C. Eldar and A.V. Oppenheim, Filterbank reconstruction of bandlimited signals
from nonuniform and generalized samples, IEEE Transactions on Signal Processing,
Vol.48, No.10, pp.2864-2875, 2000.
P.P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, NJ:
Prentice-Hall, 1993.
H. Blceskei, F. Hlawatsch and H.G. Feichtinger, Frame-theoretic analysis of
oversampled filter banks, IEEE Transactions on Signal Processing, Vol.46, No.12,
pp.3256-3268, 1998.
M. Vetterli and J. Kovaevi, Wavelets and Subband Coding, Englewood Cliffs, NJ:
Prentice Hall, 1995.
67
References
Generalized Sampling Expansions, Recurrent Sampling
A. Papoulis, Generalized sampling expansion, IEEE Transaction on Circuits and
Systems, Vol.CAS-24, No.11, pp.652-654, 1977.
A. Feuer, On the necessity of Papoulis result for multidimensional (GSE), IEEE
Signal Processing Letters, Vol.11, No.4, pp.420-422, 2004.
K.F.Cheung, A multidimensional extension of Papoulis generalized sampling
expansion with application in minimum density sampling, in Advanced Topics in
Shannon Sampling and Interpolation Theory, R.J. Marks II. Ed., New York: Springer-
Verlag, pp.86-119, 1993.

Video Compression at Source
E. Shechtman, Y. Caspi and M. Irani, Increasing space-time resolution in video,
European Conference on Computer Vision (ECCV), 2002.
N. Maor, A. Feuer and G.C. Goodwin, Compression at the source of digital video,
To appear EURASIP Journal on Applied Signal Processing.
68
Lecture 1
Sampling of Signals
by
Graham C. Goodwin
University of Newcastle
Australia
Lecture 1
Presented at the Zaborszky Distinguished Lecture Series
December 3rd, 4th and 5th, 2007