Lecture 11 Fast Fourier Transform (FFT)

Weinan E1,2 and Tiejun Li2

Department of Mathematics,
Princeton University,
weinan@princeton.edu

School of Mathematical Sciences,

Peking University,
tieli@pku.edu.cn
No.1 Science Building, 1575

Examples

Fast Fourier Transform

Outline

Examples

Fast Fourier Transform

Applications

Applications

Examples

Fast Fourier Transform

Applications

Signal processing

Filtering: a polluted signal

1.5

0.5

0.5

1.5

200

400

600

800

1000

1200

High pass and low pass filter (signal and noise)

1.5

0.5

0.5

1.5

200

400

600

800

1000

1200

How to obtain the high frequency and low frequency quickly?

Examples

Fast Fourier Transform

Solving PDEs on rectangular mesh

Solving the Poisson equations

u = f in
u = 0 on
in the rectangular domain

After discretization we will obtain the linear system with about N 2

unknowns

ui+1,j + ui1,j + ui,j+1 + ui,j1 4ui,j

= fij
4h2

The FFT would give a fast algorithm to solve the system above with
computational efforts O(N 2 log2 N ).

Applications

Examples

Fast Fourier Transform

Applications

Computing convolution (
)

Suppose
Z

f (x y)g(y)dy

h(x) =
0

is the convolution of f and g, where f (x), g(x) C2 are period 2

functions.
I

Take xj = j, j = 0, 1, . . . , N 1, =

2
N

and apply simple rectangular

discretization
h(xi )

N
1
X

f (xi xj )g(xj )

i = 0, 1, . . . , N 1

j=0
I

Define fi = f (xi ), gi = g(xi ), and let fi is period N respect to the

subscript i, define
hi =

N
1
X

fij gj

j=0
I

The direct computation is O(N 2 ).

i = 0, 1, . . . , N 1

Examples

Fast Fourier Transform

Fast Fourier Transform

Fast Fourier Transform is one of the top 10 algorithms in 20th century.

But its idea is quite simple, even for a high school student!

Applications

Examples

Fast Fourier Transform

Outline

Examples

Fast Fourier Transform

Applications

Applications

Examples

Fast Fourier Transform

Applications

Fourier Transform

Suppose f (x) is absolutely integrable in (, +), then the Fourier

transform of f (x) is
f(k) =

f (x)eikx dx.

Moreover if f (x) is square integrable, then the inverse Fourier transform of

f(k) is
Z +
1
f (x) =
f(k)eikx dk.
2

Examples

Fast Fourier Transform

Applications

Properties of Fourier transform

Some important properties of Fourier transform:
1. Derivative to coefficient:
\
0 (x))(k) = ik f(k);
(f
2. Translation property:
\
(f (x
a))(k) = eika f(k);
3. Convolution to multiplication:
\
(f
g)(k) = f(k)
g (k);
where (f g)(x) =
4. Parsevals identity:
Z

R +

f (x y)g(y)dy.

|f (x)|2 dx =

1
2

|f(k)|2 dk.

Examples

Fast Fourier Transform

Applications

Discrete Fourier transform (DFT)

Suppose we have a = (a0 , a1 , , aN 1 )T , define DFT of a as

, where
c = (c0 , c1 , , cN 1 )T = a
ck =

N
1
X

aj ejk

2i
N

k = 0, 1, . . . , N 1.

j=0

Here i is the imaginary unit, e

I

2i
N

= is the N -th root of unity.

a is the inverse discrete Fourier transform of c defined as

aj =

N 1
2i
1 X
ck ejk N ,
N

j = 0, 1, . . . , N 1.

k=0

DFT is closely related to the trigonometric interpolation for 2-periodic

function
N

T (x) =

2
X

k= N
2

such that at xj =

2j
,
N

T (xj ) = aj ,

bk eikx .
+1

j = 0, 1, . . . , N 1. The readers

may find the relation between ck and bk .

Examples

Fast Fourier Transform

Applications

Remark on DFT

DFT can be considered as a linear transformation.

Define Fourier matrix

1
F =

N 1

N 1

(N 1)

N 1
= ( jk )j,k=0

where is the N -th root of unity.

I

c is the Fourier transform of a can be represented as

c = Fa

Examples

Fast Fourier Transform

Applications

Remark on DFT

Inverse DFT can also be considered as a linear transformation.

Define inverse Fourier matrix

1
1

F 1 = G =
N

(N 1)

(N 1)
1
= ( jk )N
j,k=0

2
(N 1)

where is the N -th root of unity.

I

a is the inverse Fourier transform of c can be represented as

a = Gc

Examples

Fast Fourier Transform

Applications

Properties of DFT

Convolution to multiplication:
\
(f
g)k = fk gk
where
(f g)l =

N
1
X

k = 0, 1, . . . , N 1

flj gj

l = 0, 1, . . . , N 1,

j=0

and fl is period N with respect to index l, i.e.

f1 = fN 1 , f2 = fN 2 , . . .
I

Parsevals identity:
N

N
1
X
j=0

|aj |2 =

N
1
X
k=0

|ck |2

Examples

Fast Fourier Transform

Applications

FFT idea

FFT is proposed by J.W. Cooley and J.W. Tukey in 1960s, but the idea
may be traced back to Gauss.

The basic motivation is if we compute DFT directly, i.e.

c = Fa
we need N 2 multiplications and N (N 1) additions. Is it possible to
reduce the computation effort?

First consider

1
F =

1
1

the case N = 4
1
i

1
1

i
,

1
i

(a0 + a2 ) + (a1 + a3 )

(a0 a2 ) i(a1 a3 )
Fa =

(a0 + a2 ) (a1 + a3 )
(a0 a2 ) + i(a1 a3 )

Examples

Fast Fourier Transform

FFT idea

From the concrete form of DFT, we actually need 2 multiplications

(timing i) and 8 additions (a0 + a2 , a1 + a3 , a0 a2 , a1 a3 and the
additions in the middle).

This observation may reduce the computational effort from O(N 2 ) into
O(N log2 N )

Because
lim

log2 N
=0
N

It is a typical fast algorithm.

I

Fast algorithms of this type of recursive halving are very typical in

scientific computing.

Applications

Examples

Fast Fourier Transform

Applications

Construction of FFT

Consider N = 2m and denote

p(x) = a0 + a1 x + + aN 1 xN 1 ,
divide p(x) into odd () and even () power parts
p(x)

(a0 + a2 x2 + ) + x(a1 + a3 x2 + )

pe (x2 ) + xpo (x2 )

where
N

pe (t) = a0 + a2 t + . . . + aN 2 t 2 1 , po (t) = a1 + a3 t + . . . + aN 1 t 2 1
I

2i

Define k = e k (k-th root of unity), then when j = 0, 1, . . . ,

2j
j
2j
cj
= pe (N
) + N
po (N
)
cN
+j
2

2( N
+j)
2

= pe (N

) + N2

+j

2( N
+j)
2

po (N

N
2

Examples

Fast Fourier Transform

Applications

Construction of FFT

Pay attention that

N

2j
N
= jN , N2

+j

j
N +2j
= N
, N
= jN
2

then
j
j
cj = vj + N
uj , cj+ N = vj N
uj

j = 0, 1, ...,

N
1
2

where
vj = pe ( jN ), uj = po ( jN )
2

The formula above show that the DFT of N components vector a could
be converted to compute the DFT of two

N
2

components vectors ae , ao

and some simple additions and multiplications. This is called

Danielson-Lanczos algorithm. The recursive application of this idea gives
FFT.

Examples

Fast Fourier Transform

Applications

A simple example: N = 8

Suppose the array

a = (a0 , a1 , , a7 )T
Step A: Splitting (reordering) (odd parts and even parts):
I

Step 1
ae = (a0 , a2 , a4 , a6 )T ,

Step 2
aee = (a0 , a4 )T ,
T

aoe = (a1 , a5 ) ,
I

ao = (a1 , a3 , a5 , a7 )T ;
aeo = (a2 , a6 )T ,
aoo = (a3 , a7 )T ;

Step 3
aeee
a0

aeeo
a4

aeoe
a2

aeoo
a6

aoee
a1

aoeo
a5

aooe
a3

aooo
a7

Examples

Fast Fourier Transform

Applications

A simple example: N = 8

Step B: Combination:
I

Step 1
T

cee = (a0 + 20 a4 , a0 20 a4 ) ,
T

ceo = (a2 + 20 a6 , a2 20 a6 ) ,
T

coe = (a1 + 20 a5 , a1 20 a5 ) ,
T

coo = (a3 + 20 a7 , a3 20 a7 ) ,
I

Define the notations

w4 , (w40 , w41 )T , w8 , (w80 , w81 , w82 , w83 )T ,
and
X Y , (xj yj )j
as the vector product through multiplication by components.

Examples

Fast Fourier Transform

Applications

A simple example: N = 8

Step B: Combination:
I

Step 2
ce =

"
#
cee + w4 ceo
cee w4 ceo

co =

"
#
coe + w4 coo
coe w4 coo

Step 3
c=

"
#
ce + w8 c0
ce w8 c0

Examples

Fast Fourier Transform

Applications

A simple sketch of FFT (N = 8)

a0

a4



a2
cee

" #

reordering a6 combination 1 ceo

combination 2 ce combination 3

c
a

a1
co
ceo


coo
a5

a3

a7

Examples

Fast Fourier Transform

Applications

A remark on the reordering

If we map e to 0, and o to 1we can find the binary representation of the

indices after reordering is just the bit reversal before reordering
0= 0002

0002 = 0

1= 0012

1002 = 4

2= 0102

0102 = 2

3 = 0112

Bit

reversal

1102 = 6

4 = 1002

0012 = 1

5 = 1012

1012 = 5

6 = 1102

0112 = 3

7 = 1112

1112 = 7

Examples

Fast Fourier Transform

Outline

Examples

Fast Fourier Transform

Applications

Applications

Examples

Fast Fourier Transform

Compute the convolution

From the discretization at the beginning, we have

hi =

N
1
X

fij gj

i = 0, 1, . . . , N 1

j=0

thus
= ( f g
)
h = (h)

After using FFT, N 2 + N multiplications and N (N 1) additions are

reduced to

3
N
2

log2 N + 2N multiplications and 3N log2 N additions.

Applications

Examples

Fast Fourier Transform

Applications

Solving the linear system with loop matrix

Let

c0

cN 1

c1

L=

c1

c0

cN 1

cN 2

c2

c0

Solving Lx = b. L is a loop matrix.

I

We have
(Lx)i =

N
1
X

cij xj

j=0

where we assume c is period N with respect to the subscripts, and

x = (x0 , x1 , . . . , xN 1 )T .

Examples

Fast Fourier Transform

Applications

Solving the linear system with loop matrix

I

First consider the Jordan form of L. From the formula before

Lx = c x = x
Take DFT we have
x
=
c
x
then eigenvalues
k
k = c

The eigenvectors
j(k) = kj , (j, k = 0, 1, . . . , N 1)
x
where kj is Kroneckers .

Take inverse transform we obtain

x(0) = (1, 1, . . . , 1)T ,
x(1) = (1, 1 , . . . , (N 1) )T ,
.........
x

(N 1)

= (1,

(N 1)

, . . . , (N 1) )T

Examples

Fast Fourier Transform

Applications

Solving the linear system with loop matrix

Spectral decomposition of L

= (x

(0)

(1)

(N 1)

(0) (1)
(x x x(N 1) )1

1
..

.
N 1

= (N F
I

) (N F

1 1

=F

Solving Lx = b is equivalent to F 1 F x = b, i.e. (F x) = F b. Then it

is composed of three steps:
I
I
I

Step 1: Compute F b i.e. apply FFT to b to obtain b;

;
Step 2: Compute i.e. apply FFT to c to obtain c
Step 3: Compute x
k = bk /
ck , and then compute (
x) to obtainx.

Examples

Fast Fourier Transform

Homework assignment

Familiarize the FFT and IFFT command in MATLAB;

Compute the convolution for

2

sin(x y)ecos y dy

h(x) =
0

Applications