Вы находитесь на странице: 1из 110

CS 473ug: Algorithms

Chandra Chekuri
chekuri@cs.uiuc.edu
3228 Siebel Center
University of Illinois, Urbana-Champaign
Fall 2007
Chekuri CS473ug
Polynomials
Convolutions
FFT
Part I
Polynomials, Convolutions and FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Polynomials
Denition
A polynomial is a function of one variable built from additions,
subtractions and multiplications (but no divisions).
p(x) =
n1

j =0
a
j
x
j
The numbers a
0
, a
1
, . . . , a
n
are the coecients of the polynomial.
The degree is the highest power of x with a non-zero coecient.
Example
p(x) = 3 4x + 5x
3
a
0
= 3, a
1
= 4, a
2
= 0, a
3
= 5 and deg(p) = 3
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Polynomials
Denition
A polynomial is a function of one variable built from additions,
subtractions and multiplications (but no divisions).
p(x) =
n1

j =0
a
j
x
j
The numbers a
0
, a
1
, . . . , a
n
are the coecients of the polynomial.
The degree is the highest power of x with a non-zero coecient.
Representation
Polynomials represented by vector a = (a
0
, a
1
, . . . a
n1
) of
coecients.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial p and a value x, compute p(x)
Add Multiply Given (representation of) polynomials p, q, compute
(representation of) polynomial p q.
Roots Given p nd all roots of p.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial p and a value x, compute p(x)
Add Given polynomials p, q, compute polynomial p + q
Multiply Given (representation of) polynomials p, q, compute
(representation of) polynomial p q.
Roots Given p nd all roots of p.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial p and a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute
(reprsentation of) polynomial p + q
Multiply Given (representation of) polynomials p, q, compute
(representation of) polynomial p q.
Roots Given p nd all roots of p.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial p and a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute
(reprsentation of) polynomial p + q
Multiply Given (representation of) polynomials p, q, compute
(representation of) polynomial p q.
Roots Given p nd all roots of p.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Operations on Polynomials
Evaluate Given a polynomial p and a value x, compute p(x)
Add Given (representations of) polynomials p, q, compute
(reprsentation of) polynomial p + q
Multiply Given (representation of) polynomials p, q, compute
(representation of) polynomial p q.
Roots Given p nd all roots of p.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Evaluation
Compute value of polynomial a = (a
0
, a
1
, . . . a
n1
) at x
power = 1
value = 0
for j = 0 to n 1
// invariant: power = x
j
value = value + a
j
power
power = power x
end for
return value
Uses 2n multiplication and n additions
Horners rule can be used to cut the multiplications in half
a(x) = a
0
+ x(a
1
+ x(a
2
+ + xa
n1
))
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Evaluation
Compute value of polynomial a = (a
0
, a
1
, . . . a
n1
) at x
power = 1
value = 0
for j = 0 to n 1
// invariant: power = x
j
value = value + a
j
power
power = power x
end for
return value
Uses 2n multiplication and n additions
Horners rule can be used to cut the multiplications in half
a(x) = a
0
+ x(a
1
+ x(a
2
+ + xa
n1
))
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Evaluation
Compute value of polynomial a = (a
0
, a
1
, . . . a
n1
) at x
power = 1
value = 0
for j = 0 to n 1
// invariant: power = x
j
value = value + a
j
power
power = power x
end for
return value
Uses 2n multiplication and n additions
Horners rule can be used to cut the multiplications in half
a(x) = a
0
+ x(a
1
+ x(a
2
+ + xa
n1
))
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Evaluation: Numerical Issues
Question
How long does evaluation really take? O(n) time?
Size of x
n
in terms of bits is n log x while size of x is only log x.
Thus, need to pay attention to size of numbers and multiplication
complexity.
Ignore this issue for now. Can get around it for applications of
interest.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Evaluation: Numerical Issues
Question
How long does evaluation really take? O(n) time?
Size of x
n
in terms of bits is n log x while size of x is only log x.
Thus, need to pay attention to size of numbers and multiplication
complexity.
Ignore this issue for now. Can get around it for applications of
interest.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Addition
Compute the sum of polynomials
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
a + b = (a
0
+ b
0
, a
1
+ b
1
, . . . a
n1
+ b
n1
). Takes O(n) time.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Addition
Compute the sum of polynomials
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
a + b = (a
0
+ b
0
, a
1
+ b
1
, . . . a
n1
+ b
n1
). Takes O(n) time.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Multiplication
Compute the product of polynomials
a = (a
0
, a
1
, . . . a
n
) and b = (b
0
, b
1
, . . . b
m
)
Recall a b = (c
0
, c
1
, . . . c
n+m
) where
c
k
=

i ,j : i +j =k
a
i
b
j
for j = 0 to n+m c
j
= 0
for j = 0 to n
for k = 0 to m
c
j +k
= c
j +k
+ a
j
b
k
return (c
0
, c
1
, . . . c
n+m
)
Takes O(n
2
) time.
We will give a better algorithm!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Multiplication
Compute the product of polynomials
a = (a
0
, a
1
, . . . a
n
) and b = (b
0
, b
1
, . . . b
m
)
Recall a b = (c
0
, c
1
, . . . c
n+m
) where
c
k
=

i ,j : i +j =k
a
i
b
j
for j = 0 to n+m c
j
= 0
for j = 0 to n
for k = 0 to m
c
j +k
= c
j +k
+ a
j
b
k
return (c
0
, c
1
, . . . c
n+m
)
Takes O(n
2
) time.
We will give a better algorithm!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Computing with Polynomials
Multiplication
Compute the product of polynomials
a = (a
0
, a
1
, . . . a
n
) and b = (b
0
, b
1
, . . . b
m
)
Recall a b = (c
0
, c
1
, . . . c
n+m
) where
c
k
=

i ,j : i +j =k
a
i
b
j
for j = 0 to n+m c
j
= 0
for j = 0 to n
for k = 0 to m
c
j +k
= c
j +k
+ a
j
b
k
return (c
0
, c
1
, . . . c
n+m
)
Takes O(n
2
) time. We will give a better algorithm!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Convolutions
Denition
The convolution of vectors a = (a
0
, a
1
, . . . a
n
) and
b = (b
0
, b
1
, . . . b
m
) is the vector c = (c
0
, c
1
, . . . c
n+m
) where
c
k
=

i ,j : i +j =k
a
i
b
j
Convolution of vectors a and b is denoted by a b.
In other words,
the convolution is the coecients of the product of the two
polynomials
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Convolutions
Denition
The convolution of vectors a = (a
0
, a
1
, . . . a
n
) and
b = (b
0
, b
1
, . . . b
m
) is the vector c = (c
0
, c
1
, . . . c
n+m
) where
c
k
=

i ,j : i +j =k
a
i
b
j
Convolution of vectors a and b is denoted by a b. In other words,
the convolution is the coecients of the product of the two
polynomials
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Applications: Signal Processing
Let a = (a
0
, a
1
, . . . a
n1
) be a sequence of measurements over
time.
To account for measurement errors or random uctuations,
measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, in Gaussian
smoothing a
i
is replaced by
a

i
=
1
Z
i +k

j =i k
a
j
e
(j i )
2
Smoothing can be thought of as vector of weights
w = (w
k
, w
(k1)
, . . . , w
1
, w
0
, w
1
. . . w
k
) used to average
each entry as a

i
=

k
s=k
w
s
a
i +s
Taking b = (b
0
, b
1
, . . . b
2k
) to be b
j
= w
kj
, a

= a b
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Applications: Signal Processing
Let a = (a
0
, a
1
, . . . a
n1
) be a sequence of measurements over
time.
To account for measurement errors or random uctuations,
measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, in Gaussian
smoothing a
i
is replaced by
a

i
=
1
Z
i +k

j =i k
a
j
e
(j i )
2
Smoothing can be thought of as vector of weights
w = (w
k
, w
(k1)
, . . . , w
1
, w
0
, w
1
. . . w
k
) used to average
each entry as a

i
=

k
s=k
w
s
a
i +s
Taking b = (b
0
, b
1
, . . . b
2k
) to be b
j
= w
kj
, a

= a b
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Applications: Signal Processing
Let a = (a
0
, a
1
, . . . a
n1
) be a sequence of measurements over
time.
To account for measurement errors or random uctuations,
measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, in Gaussian
smoothing a
i
is replaced by
a

i
=
1
Z
i +k

j =i k
a
j
e
(j i )
2
Smoothing can be thought of as vector of weights
w = (w
k
, w
(k1)
, . . . , w
1
, w
0
, w
1
. . . w
k
) used to average
each entry as a

i
=

k
s=k
w
s
a
i +s
Taking b = (b
0
, b
1
, . . . b
2k
) to be b
j
= w
kj
, a

= a b
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Applications: Signal Processing
Let a = (a
0
, a
1
, . . . a
n1
) be a sequence of measurements over
time.
To account for measurement errors or random uctuations,
measurements are smoothed by averaging each value with a
weighted sum of its neighbors. For example, in Gaussian
smoothing a
i
is replaced by
a

i
=
1
Z
i +k

j =i k
a
j
e
(j i )
2
Smoothing can be thought of as vector of weights
w = (w
k
, w
(k1)
, . . . , w
1
, w
0
, w
1
. . . w
k
) used to average
each entry as a

i
=

k
s=k
w
s
a
i +s
Taking b = (b
0
, b
1
, . . . b
2k
) to be b
j
= w
kj
, a

= a b
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Historical Applications
Gauss used convolutions in the 1800s to compute the path of
asteroids from a nite number of equi-spaced observations.
In the mid-60s, Cooley and Tukey used it to detect Soviet
nuclear tests by interpolating o-shore seismic readings
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Historical Applications
Gauss used convolutions in the 1800s to compute the path of
asteroids from a nite number of equi-spaced observations.
In the mid-60s, Cooley and Tukey used it to detect Soviet
nuclear tests by interpolating o-shore seismic readings
Chekuri CS473ug
Polynomials
Convolutions
FFT
Denition
Applications
Many Applications
To mention a few:
Signal and image processing (radar, MRI, astronomy,
compression, ...)
Statistics
Multiplication of numbers
Pattern matching
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a = (a
0
, a
1
, . . . a
n1
) of
coecients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r ) = 0. If r
1
, r
2
, . . . , r
n1
are roots then p(x) = a
n1
(x r
1
)(x r
2
) . . . (x r
n1
).
Theorem (Fundamental Theorem of Algebra)
Every polynomial p(x) of degree d has exactly d roots r
1
, r
2
, . . . , r
d
where the roots can be complex numbers and can be repeated.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a = (a
0
, a
1
, . . . a
n1
) of
coecients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r ) = 0. If r
1
, r
2
, . . . , r
n1
are roots then p(x) = a
n1
(x r
1
)(x r
2
) . . . (x r
n1
).
Theorem (Fundamental Theorem of Algebra)
Every polynomial p(x) of degree d has exactly d roots r
1
, r
2
, . . . , r
d
where the roots can be complex numbers and can be repeated.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a = (a
0
, a
1
, . . . a
n1
) of
coecients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r ) = 0. If r
1
, r
2
, . . . , r
n1
are roots then p(x) = a
n1
(x r
1
)(x r
2
) . . . (x r
n1
).
Theorem (Fundamental Theorem of Algebra)
Every polynomial p(x) of degree d has exactly d roots r
1
, r
2
, . . . , r
d
where the roots can be complex numbers and can be repeated.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Revisiting Polynomial Representations
Representation
Polynomials represented by vector a = (a
0
, a
1
, . . . a
n1
) of
coecients.
Question
Are there other ways to represent polynomials?
Root of a polynomial p(x): r such that p(r ) = 0. If r
1
, r
2
, . . . , r
n1
are roots then p(x) = a
n1
(x r
1
)(x r
2
) . . . (x r
n1
).
Theorem (Fundamental Theorem of Algebra)
Every polynomial p(x) of degree d has exactly d roots r
1
, r
2
, . . . , r
d
where the roots can be complex numbers and can be repeated.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor a
n1
and roots
r
1
, r
2
, . . . , r
n1
.
Evaluating p at a given x is easy. Why?
Multiplication: given p, q with roots r
1
, . . . , r
n1
and
s
1
, . . . , s
m1
the product p q has roots
r
1
, . . . , r
n1
, s
1
, . . . , s
m1
. Easy!
Addition: requires O(n
2
) time??
Given coecient representation, how do we go to root
representation?? No nite algorithm because of potential for
irrational roots.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor a
n1
and roots
r
1
, r
2
, . . . , r
n1
.
Evaluating p at a given x is easy. Why?
Multiplication: given p, q with roots r
1
, . . . , r
n1
and
s
1
, . . . , s
m1
the product p q has roots
r
1
, . . . , r
n1
, s
1
, . . . , s
m1
. Easy!
Addition: requires O(n
2
) time??
Given coecient representation, how do we go to root
representation?? No nite algorithm because of potential for
irrational roots.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor a
n1
and roots
r
1
, r
2
, . . . , r
n1
.
Evaluating p at a given x is easy. Why?
Multiplication: given p, q with roots r
1
, . . . , r
n1
and
s
1
, . . . , s
m1
the product p q has roots
r
1
, . . . , r
n1
, s
1
, . . . , s
m1
. Easy!
Addition: requires O(n
2
) time??
Given coecient representation, how do we go to root
representation?? No nite algorithm because of potential for
irrational roots.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor a
n1
and roots
r
1
, r
2
, . . . , r
n1
.
Evaluating p at a given x is easy. Why?
Multiplication: given p, q with roots r
1
, . . . , r
n1
and
s
1
, . . . , s
m1
the product p q has roots
r
1
, . . . , r
n1
, s
1
, . . . , s
m1
. Easy!
Addition: requires O(n
2
) time??
Given coecient representation, how do we go to root
representation?? No nite algorithm because of potential for
irrational roots.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Roots
Representation
Polynomials represented by vector scale factor a
n1
and roots
r
1
, r
2
, . . . , r
n1
.
Evaluating p at a given x is easy. Why?
Multiplication: given p, q with roots r
1
, . . . , r
n1
and
s
1
, . . . , s
m1
the product p q has roots
r
1
, . . . , r
n1
, s
1
, . . . , s
m1
. Easy!
Addition: requires O(n
2
) time??
Given coecient representation, how do we go to root
representation?? No nite algorithm because of potential for
irrational roots.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Samples
Let p be a polynomial of degree n 1.
Pick n distinct samples x
0
, x
1
, x
2
, . . . , x
n1
Let y
0
= p(x
0
), y
1
= p(x
1
), . . . , y
n1
= p(x
n1
).
Representation
Polynomials represented by (x
0
, y
0
), (x
1
, y
1
), . . . , (x
n1
, y
n1
).
Is the above a valid representation? Why do we use 2n numbers
instead of n numbers for coecient and root representation?
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Samples
Let p be a polynomial of degree n 1.
Pick n distinct samples x
0
, x
1
, x
2
, . . . , x
n1
Let y
0
= p(x
0
), y
1
= p(x
1
), . . . , y
n1
= p(x
n1
).
Representation
Polynomials represented by (x
0
, y
0
), (x
1
, y
1
), . . . , (x
n1
, y
n1
).
Is the above a valid representation?
Why do we use 2n numbers
instead of n numbers for coecient and root representation?
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Representing Polynomials by Samples
Let p be a polynomial of degree n 1.
Pick n distinct samples x
0
, x
1
, x
2
, . . . , x
n1
Let y
0
= p(x
0
), y
1
= p(x
1
), . . . , y
n1
= p(x
n1
).
Representation
Polynomials represented by (x
0
, y
0
), (x
1
, y
1
), . . . , (x
n1
, y
n1
).
Is the above a valid representation? Why do we use 2n numbers
instead of n numbers for coecient and root representation?
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Sample Representation
Theorem
Given a list {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} there is exactly one
polynomial p of degree n 1 such that p(x
j
) = y
j
for
j = 0, 1, . . . , n 1.
So representation is valid.
Can use same x
0
, x
1
, . . . , x
n1
for all polynomials of degree n 1!
No need to store them explicitly! Need only n numbers
y
0
, y
1
, . . . , y
n1
.
Lagrange interpolation formula: Given (x
0
, y
0
), . . . , (x
n1
, y
n1
) the
following polynomial p satises p(x
j
) = y
j
for j = 0, 1, 2, . . . , n 1.
p(x) =
n1

j =0
_
_
y
j

k=j
(x
j
x
k
)

k=j
(x x
k
)
_
_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Sample Representation
Theorem
Given a list {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} there is exactly one
polynomial p of degree n 1 such that p(x
j
) = y
j
for
j = 0, 1, . . . , n 1.
So representation is valid.
Can use same x
0
, x
1
, . . . , x
n1
for all polynomials of degree n 1!
No need to store them explicitly! Need only n numbers
y
0
, y
1
, . . . , y
n1
.
Lagrange interpolation formula: Given (x
0
, y
0
), . . . , (x
n1
, y
n1
) the
following polynomial p satises p(x
j
) = y
j
for j = 0, 1, 2, . . . , n 1.
p(x) =
n1

j =0
_
_
y
j

k=j
(x
j
x
k
)

k=j
(x x
k
)
_
_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Sample Representation
Theorem
Given a list {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} there is exactly one
polynomial p of degree n 1 such that p(x
j
) = y
j
for
j = 0, 1, . . . , n 1.
So representation is valid.
Can use same x
0
, x
1
, . . . , x
n1
for all polynomials of degree n 1!
No need to store them explicitly! Need only n numbers
y
0
, y
1
, . . . , y
n1
.
Lagrange interpolation formula: Given (x
0
, y
0
), . . . , (x
n1
, y
n1
) the
following polynomial p satises p(x
j
) = y
j
for j = 0, 1, 2, . . . , n 1.
p(x) =
n1

j =0
_
_
y
j

k=j
(x
j
x
k
)

k=j
(x x
k
)
_
_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Sample Representation
Theorem
Given a list {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} there is exactly one
polynomial p of degree n 1 such that p(x
j
) = y
j
for
j = 0, 1, . . . , n 1.
So representation is valid.
Can use same x
0
, x
1
, . . . , x
n1
for all polynomials of degree n 1!
No need to store them explicitly! Need only n numbers
y
0
, y
1
, . . . , y
n1
.
Lagrange interpolation formula: Given (x
0
, y
0
), . . . , (x
n1
, y
n1
) the
following polynomial p satises p(x
j
) = y
j
for j = 0, 1, 2, . . . , n 1.
p(x) =
n1

j =0
_
_
y
j

k=j
(x
j
x
k
)

k=j
(x x
k
)
_
_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Lagrange Interpolation
For n = 3
p(x) = y
0
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
+y
1
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
+y
2
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
Easy to verify that p(x
j
) = y
j
! Thus there exists one polynomial of
degree n 1 that interpolates the values (x
0
, y
0
), . . . , (x
n1
, y
n1
).
Can there be two distinct polynomials?
No! Use Fundamental Theorem of Algebra to prove it exercise.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Lagrange Interpolation
For n = 3
p(x) = y
0
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
+y
1
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
+y
2
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
Easy to verify that p(x
j
) = y
j
! Thus there exists one polynomial of
degree n 1 that interpolates the values (x
0
, y
0
), . . . , (x
n1
, y
n1
).
Can there be two distinct polynomials?
No! Use Fundamental Theorem of Algebra to prove it exercise.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Lagrange Interpolation
For n = 3
p(x) = y
0
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
+y
1
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
+y
2
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
Easy to verify that p(x
j
) = y
j
! Thus there exists one polynomial of
degree n 1 that interpolates the values (x
0
, y
0
), . . . , (x
n1
, y
n1
).
Can there be two distinct polynomials?
No! Use Fundamental Theorem of Algebra to prove it exercise.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Addition and Multiplication with Sample Representation
Let {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} and
{(x
0
, y

0
), (x
1
, y

1
), . . . (x
n1
, y

n1
)} be representations of two
polynomials of degree n 1
a + b can be represented by
{(x
0
, (y
0
+ y

0
)), (x
1
, (y
1
+ y

1
)), . . . (x
n1
, (y
n1
+ y

n1
))}
Thus, can be computed in O(n) time
a b can be evaluated at n samples
{(x
0
, (y
0
y

0
)), (x
1
, (y
1
y

1
)), . . . (x
n1
, (y
n1
y

n1
))}
Can be computed in O(n) time!
But what if p, q are given in coecient form? Convolution requires
p, q to be in coecient form.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Addition and Multiplication with Sample Representation
Let {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} and
{(x
0
, y

0
), (x
1
, y

1
), . . . (x
n1
, y

n1
)} be representations of two
polynomials of degree n 1
a + b can be represented by
{(x
0
, (y
0
+ y

0
)), (x
1
, (y
1
+ y

1
)), . . . (x
n1
, (y
n1
+ y

n1
))}
Thus, can be computed in O(n) time
a b can be evaluated at n samples
{(x
0
, (y
0
y

0
)), (x
1
, (y
1
y

1
)), . . . (x
n1
, (y
n1
y

n1
))}
Can be computed in O(n) time!
But what if p, q are given in coecient form? Convolution requires
p, q to be in coecient form.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Addition and Multiplication with Sample Representation
Let {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} and
{(x
0
, y

0
), (x
1
, y

1
), . . . (x
n1
, y

n1
)} be representations of two
polynomials of degree n 1
a + b can be represented by
{(x
0
, (y
0
+ y

0
)), (x
1
, (y
1
+ y

1
)), . . . (x
n1
, (y
n1
+ y

n1
))}
Thus, can be computed in O(n) time
a b can be evaluated at n samples
{(x
0
, (y
0
y

0
)), (x
1
, (y
1
y

1
)), . . . (x
n1
, (y
n1
y

n1
))}
Can be computed in O(n) time!
But what if p, q are given in coecient form? Convolution requires
p, q to be in coecient form.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Addition and Multiplication with Sample Representation
Let {(x
0
, y
0
), (x
1
, y
1
), . . . (x
n1
, y
n1
)} and
{(x
0
, y

0
), (x
1
, y

1
), . . . (x
n1
, y

n1
)} be representations of two
polynomials of degree n 1
a + b can be represented by
{(x
0
, (y
0
+ y

0
)), (x
1
, (y
1
+ y

1
)), . . . (x
n1
, (y
n1
+ y

n1
))}
Thus, can be computed in O(n) time
a b can be evaluated at n samples
{(x
0
, (y
0
y

0
)), (x
1
, (y
1
y

1
)), . . . (x
n1
, (y
n1
y

n1
))}
Can be computed in O(n) time!
But what if p, q are given in coecient form? Convolution requires
p, q to be in coecient form.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Coecient representation to Sample representation
Given p as (a
0
, a
1
, . . . , a
n1
) can we obtain a sample
representation (x
0
, y
0
), . . . , (x
n1
, y
n1
) quickly? Also can we
invert the representation quickly?
Suppose we choose x
0
, x
1
, . . . , x
n1
arbitrarily.
Take O(n) time to evaluate y
j
= p(x
j
) given (a
0
, . . . , a
n1
).
Total time is (n
2
)!
Inversion via Lagrange interpolation also (n
2
).
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Coecient representation to Sample representation
Given p as (a
0
, a
1
, . . . , a
n1
) can we obtain a sample
representation (x
0
, y
0
), . . . , (x
n1
, y
n1
) quickly? Also can we
invert the representation quickly?
Suppose we choose x
0
, x
1
, . . . , x
n1
arbitrarily.
Take O(n) time to evaluate y
j
= p(x
j
) given (a
0
, . . . , a
n1
).
Total time is (n
2
)!
Inversion via Lagrange interpolation also (n
2
).
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Key Idea
Can choose x
0
, x
1
, . . . , x
n1
carefully!
Total time to evaluate p(x
0
), p(x
1
), . . . , p(x
n1
) should be better
than evaluating each separately.
How do we choose x
0
, x
1
, . . . , x
n1
to save work?
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Key Idea
Can choose x
0
, x
1
, . . . , x
n1
carefully!
Total time to evaluate p(x
0
), p(x
1
), . . . , p(x
n1
) should be better
than evaluating each separately.
How do we choose x
0
, x
1
, . . . , x
n1
to save work?
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
A Simple Start
a(x) = a
0
+ a
1
x + a
2
x
2
+ a
3
x
3
+ . . . + a
n1
x
n1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)
2j
= x
2j
. Can we exploit this?
Example
3+4x +6x
2
+2x
3
+x
4
+10x
5
= (3+6x
2
+x
4
) +x(4+2x
2
+10x
4
)
If we have a(x) then easy to also compute a(x)!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
A Simple Start
a(x) = a
0
+ a
1
x + a
2
x
2
+ a
3
x
3
+ . . . + a
n1
x
n1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)
2j
= x
2j
. Can we exploit this?
Example
3+4x +6x
2
+2x
3
+x
4
+10x
5
= (3+6x
2
+x
4
) +x(4+2x
2
+10x
4
)
If we have a(x) then easy to also compute a(x)!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
A Simple Start
a(x) = a
0
+ a
1
x + a
2
x
2
+ a
3
x
3
+ . . . + a
n1
x
n1
Assume n is a power of 2 for rest of the discussion.
Observation: (x)
2j
= x
2j
. Can we exploit this?
Example
3+4x +6x
2
+2x
3
+x
4
+10x
5
= (3+6x
2
+x
4
) +x(4+2x
2
+10x
4
)
If we have a(x) then easy to also compute a(x)!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Odd and Even Decomposition
Let a = (a
0
, a
1
, . . . a
n1
) be a polynomial.
Let a
odd
= (a
1
, a
3
, a
5
, . . .) be the n/2 degree polynomial
dened by the odd coecients; so
a
odd
(x) = a
1
+ a
3
x + a
5
x
2
+
Let a
even
= (a
0
, a
2
, a
4
, . . .) be the n/2 degree polynomial
dened by the even coecients.
Observe
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Thus, evaluating a at x can be reduced to evaluating lower
degree polynomials plus constantly many arithmetic
operations.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Exploiting Odd-Even Decomposition
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose n samples
x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Sucient to evaluate a
even
and a
odd
at x
2
0
, x
2
1
, x
2
2
, . . . , x
2
n/21
!
Pluse O(n) work gives a(x) for all n samples!
Suppose we can make this work recursively. Then
T(n) = 2T(n/2) + O(n) which implies T(n) = O(n log n)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Exploiting Odd-Even Decomposition
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose n samples
x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Sucient to evaluate a
even
and a
odd
at x
2
0
, x
2
1
, x
2
2
, . . . , x
2
n/21
!
Pluse O(n) work gives a(x) for all n samples!
Suppose we can make this work recursively. Then
T(n) = 2T(n/2) + O(n) which implies T(n) = O(n log n)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Exploiting Odd-Even Decomposition
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose n samples
x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Sucient to evaluate a
even
and a
odd
at x
2
0
, x
2
1
, x
2
2
, . . . , x
2
n/21
!
Pluse O(n) work gives a(x) for all n samples!
Suppose we can make this work recursively. Then
T(n) = 2T(n/2) + O(n) which implies T(n) = O(n log n)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Can we recurse?
n samples x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Next step in recursion x
2
0
, x
2
1
, . . . , x
2
n/21
To continue recursion, we need
{x
2
0
, x
2
1
, . . . , x
2
n/21
} = {z
0
, z
1
, . . . , z
n/41
, z
0
, z
1
, . . . , z
n/41
}
If z
0
= x
2
0
and say z
0
= x
2
j
then x
0
=

1x
j
! That is
x
0
= ix
j
where i is the imaginary number.
Can continue recursion but need to go to complex numbers.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Can we recurse?
n samples x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Next step in recursion x
2
0
, x
2
1
, . . . , x
2
n/21
To continue recursion, we need
{x
2
0
, x
2
1
, . . . , x
2
n/21
} = {z
0
, z
1
, . . . , z
n/41
, z
0
, z
1
, . . . , z
n/41
}
If z
0
= x
2
0
and say z
0
= x
2
j
then x
0
=

1x
j
! That is
x
0
= ix
j
where i is the imaginary number.
Can continue recursion but need to go to complex numbers.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Can we recurse?
n samples x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Next step in recursion x
2
0
, x
2
1
, . . . , x
2
n/21
To continue recursion, we need
{x
2
0
, x
2
1
, . . . , x
2
n/21
} = {z
0
, z
1
, . . . , z
n/41
, z
0
, z
1
, . . . , z
n/41
}
If z
0
= x
2
0
and say z
0
= x
2
j
then x
0
=

1x
j
! That is
x
0
= ix
j
where i is the imaginary number.
Can continue recursion but need to go to complex numbers.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Can we recurse?
n samples x
0
, x
1
, x
2
, . . . , x
n/21
, x
0
, x
1
, . . . , x
n/21
Next step in recursion x
2
0
, x
2
1
, . . . , x
2
n/21
To continue recursion, we need
{x
2
0
, x
2
1
, . . . , x
2
n/21
} = {z
0
, z
1
, . . . , z
n/41
, z
0
, z
1
, . . . , z
n/41
}
If z
0
= x
2
0
and say z
0
= x
2
j
then x
0
=

1x
j
! That is
x
0
= ix
j
where i is the imaginary number.
Can continue recursion but need to go to complex numbers.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib
=

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib
=

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib
=

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib =

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib =

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Numbers
Notation
For the rest of lecture, i stands for

1
Denition
Complex numbers are points lying in the complex plane
represented as
Cartesian a + ib =

a
2
+ b
2
e
(arctan(b/a))i
Polar re
i
= r (cos + i sin )
Thus, e
i
= 1 and e
2i
= 1.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Power Series for Functions
What is e
z
when z is a real number? When z is a complex
number?
e
z
= 1 + z/1! + z
2
/2! + . . . + z
j
/j ! + . . .
Therefore
e
i
= 1 + i /1! + (i )
2
/2! + (i )
3
/3! + . . .
= (1
2
/2! +
4
/4! . . . +) + i (
3
/3! + . . . +)
= cos + i sin
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Roots of Unity
What are the roots of the polynomial x
k
1?
Clearly 1 is a root.
Suppose re
i
is a root then r
k
e
ki
= 1 which implies that
r = 1 and k = 2 since e
ki
= cos(k) + i sin(k) = 1
Let
k
= e
2i /k
. The roots are 1 =
0
k
,
2
k
, . . . ,
k1
k
where
w
j
k
= e
2ji /k
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Roots of Unity
What are the roots of the polynomial x
k
1?
Clearly 1 is a root.
Suppose re
i
is a root then r
k
e
ki
= 1 which implies that
r = 1 and k = 2 since e
ki
= cos(k) + i sin(k) = 1
Let
k
= e
2i /k
. The roots are 1 =
0
k
,
2
k
, . . . ,
k1
k
where
w
j
k
= e
2ji /k
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Complex Roots of Unity
What are the roots of the polynomial x
k
1?
Clearly 1 is a root.
Suppose re
i
is a root then r
k
e
ki
= 1 which implies that
r = 1 and k = 2 since e
ki
= cos(k) + i sin(k) = 1
Let
k
= e
2i /k
. The roots are 1 =
0
k
,
2
k
, . . . ,
k1
k
where
w
j
k
= e
2ji /k
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
More on the Roots of Unity
Observations

j
k
=
j mod k
k

k
=
j
jk
; thus, every kth root is also a jkth root.

k1
s=0
(
j
k
)
s
= (1 +
j
k
+
2j
k
+ . . . +
j (k1)
k
) = 0 for j = 0

j
k
is root of x
k
1 = (x 1)(x
k1
+ x
k2
+ . . . + 1)
Thus,
j
k
is root of (x
k1
+ x
k2
+ . . . + 1)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
More on the Roots of Unity
Observations

j
k
=
j mod k
k

k
=
j
jk
; thus, every kth root is also a jkth root.

k1
s=0
(
j
k
)
s
= (1 +
j
k
+
2j
k
+ . . . +
j (k1)
k
) = 0 for j = 0

j
k
is root of x
k
1 = (x 1)(x
k1
+ x
k2
+ . . . + 1)
Thus,
j
k
is root of (x
k1
+ x
k2
+ . . . + 1)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Back to Recursive Idea
Let a = (a
0
, a
1
, . . . a
n1
) be a polynomial. Assume n is a
power of 2.
Recall
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose x
0
, x
1
, . . . , x
n1
to be nth roots of unity. That is
x
j
=
j
n
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Back to Recursive Idea
Let a = (a
0
, a
1
, . . . a
n1
) be a polynomial. Assume n is a
power of 2.
Recall
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose x
0
, x
1
, . . . , x
n1
to be nth roots of unity. That is
x
j
=
j
n
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Back to Recursive Idea
Let a = (a
0
, a
1
, . . . a
n1
) be a polynomial. Assume n is a
power of 2.
Recall
a(x) = a
even
(x
2
) + xa
odd
(x
2
)
Choose x
0
, x
1
, . . . , x
n1
to be nth roots of unity. That is
x
j
=
j
n
.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Evaluating at roots of unity
Problem: Evaluate degree n 1 polynomial
a = (a
0
, a
1
, . . . a
n1
) on the nth roots of unity
Recall
a(
j
n
) = a
even
((
j
n
)
2
) +
j
n
a
odd
((
j
n
)
2
)
Observe that (
j
n
)
2
=
j
n/2
Thus, evaluating a on the nth roots of unity, can be
accomplished by evaluating a
even
and a
odd
on the n/2 roots of
unity!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Evaluating at roots of unity
Problem: Evaluate degree n 1 polynomial
a = (a
0
, a
1
, . . . a
n1
) on the nth roots of unity
Recall
a(
j
n
) = a
even
((
j
n
)
2
) +
j
n
a
odd
((
j
n
)
2
)
Observe that (
j
n
)
2
=
j
n/2
Thus, evaluating a on the nth roots of unity, can be
accomplished by evaluating a
even
and a
odd
on the n/2 roots of
unity!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Evaluating at roots of unity
Problem: Evaluate degree n 1 polynomial
a = (a
0
, a
1
, . . . a
n1
) on the nth roots of unity
Recall
a(
j
n
) = a
even
((
j
n
)
2
) +
j
n
a
odd
((
j
n
)
2
)
Observe that (
j
n
)
2
=
j
n/2
Thus, evaluating a on the nth roots of unity, can be
accomplished by evaluating a
even
and a
odd
on the n/2 roots of
unity!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Evaluating at roots of unity
Problem: Evaluate degree n 1 polynomial
a = (a
0
, a
1
, . . . a
n1
) on the nth roots of unity
Recall
a(
j
n
) = a
even
((
j
n
)
2
) +
j
n
a
odd
((
j
n
)
2
)
Observe that (
j
n
)
2
=
j
n/2
Thus, evaluating a on the nth roots of unity, can be
accomplished by evaluating a
even
and a
odd
on the n/2 roots of
unity!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Divide and Conquer Evaluation
Evaluation Problem
Evaluate n 1-degree polynomial on nth roots of unity
Construct polynomials a
even
and a
odd
Recurisvely evaluate a
even
and a
odd
on the n/2th roots of unity
Compute a(
j
n
) as a
even
(
j
n/2
) +
j
n
a
odd
(
j
n/2
)
Let T(n) denote the running time of evaluating an n 1-degree
polynomial on the nth roots of unity. Then,
T(n) 2T(n/2) + O(n) = O(n log n)
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Discrete Fourier Transform
Denition
Given a polynomial a = (a
0
, a
1
, . . . , a
n1
) the Discrete Fourier
Transform of a is the vector a

= (a

0
, a

1
, . . . , a

n1
) where
a

j
= a(
j
n
) for 0 j < n.
a

is a sample representation of a for nth roots of unity.


We have shown that a

can be computed from a in O(n log n)


time. This divide and conquer algorithm is called the Fast Fourier
Transform (FFT).
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Back to Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at some n sample points.
2
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
).
3
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
How can be compute c from c

? We only have n sample points


and c

has 2n 1 coecients!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Back to Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the nth roots of unity.
2
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
).
3
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
How can be compute c from c

? We only have n sample points


and c

has 2n 1 coecients!
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Pad a with n zeroes to make it a (2n 1) degree polynomial
a = (a
0
, a
1
, . . . , a
n1
, a
n
, a
n+1
, . . . , a
2n1
). Similarly for b.
2
Compute values of a and b at some 2n sample points.
3
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
, . . . , a

2n1
b

2n1
).
4
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
Step 1 takes O(n log n) using divide and conquer algorithm
Step 2 takes O(n) time
Step 3??
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Pad a with n zeroes to make it a (2n 1) degree polynomial
a = (a
0
, a
1
, . . . , a
n1
, a
n
, a
n+1
, . . . , a
2n1
). Similarly for b.
2
Compute values of a and b at the 2nth roots of unity.
3
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
, . . . , a

2n1
b

2n1
).
4
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
Step 1 takes O(n log n) using divide and conquer algorithm
Step 2 takes O(n) time
Step 3??
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Pad a with n zeroes to make it a (2n 1) degree polynomial
a = (a
0
, a
1
, . . . , a
n1
, a
n
, a
n+1
, . . . , a
2n1
). Similarly for b.
2
Compute values of a and b at the 2nth roots of unity.
3
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
, . . . , a

2n1
b

2n1
).
4
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
Step 1 takes O(n log n) using divide and conquer algorithm
Step 2 takes O(n) time
Step 3??
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Pad a with n zeroes to make it a (2n 1) degree polynomial
a = (a
0
, a
1
, . . . , a
n1
, a
n
, a
n+1
, . . . , a
2n1
). Similarly for b.
2
Compute values of a and b at the 2nth roots of unity.
3
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
, . . . , a

2n1
b

2n1
).
4
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
Step 1 takes O(n log n) using divide and conquer algorithm
Step 2 takes O(n) time
Step 3??
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions and Polynomial Multiplication
Convolutions
Compute convolution c = (c
0
, c
1
, . . . , c
2n2
) of
a = (a
0
, a
1
, . . . a
n1
) and b = (b
0
, b
1
, . . . b
n1
)
1
Pad a with n zeroes to make it a (2n 1) degree polynomial
a = (a
0
, a
1
, . . . , a
n1
, a
n
, a
n+1
, . . . , a
2n1
). Similarly for b.
2
Compute values of a and b at the 2nth roots of unity.
3
Compute sample representation of product. That is
c

= (a

0
b

0
, a

1
b

1
, . . . , a

n1
b

n1
, . . . , a

2n1
b

2n1
).
4
Compute coecients of unique polynomial associated with
sample representation of product. That is compute c from c

.
Step 1 takes O(n log n) using divide and conquer algorithm
Step 2 takes O(n) time
Step 3??
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverse Fourier Transform
Input Given the evaluation of a 2n 1-degree polynomial c
on the 2nth roots of unity
Goal Compute the coecients of c
We saw that a

can be computed from a in O(n log n) time. Can


we compute a from a

in O(n log n) time?


Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverse Fourier Transform
Input Given the evaluation of a 2n 1-degree polynomial c
on the 2nth roots of unity
Goal Compute the coecients of c
We saw that a

can be computed from a in O(n log n) time. Can


we compute a from a

in O(n log n) time?


Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
A Matrix Point of View
a

0
= a(x
0
), a

1
= a(x
1
), . . . , a

n1
= a(x
n1
) where x
j
=
j
n
.
Let
1
n
= e
2/n
= .
_

_
1 x
0
x
2
0
. . . x
n1
0
1 x
1
x
2
1
. . . x
n1
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 x
j
x
2
j
. . . x
n1
j
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 x
n1
x
2
n1
. . . x
n1
n1
_

_
_

_
a
0
a
1
.
.
.
a
j
.
.
.
a
n1
_

_
=
_

_
a

0
a

1
.
.
.
a

j
.
.
.
a

n1
_

_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
A Matrix Point of View
a

0
= a(x
0
), a

1
= a(x
1
), . . . , a

n1
= a(x
n1
) where x
j
=
j
n
.
Let
1
n
= e
2/n
= .
_

_
1 1 1 . . . 1
1
2
. . .
n1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
j

2j
. . .
j (n1)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
n1

2(n1)
. . .
(n1)(n1)
_

_
_

_
a
0
a
1
.
.
.
a
j
.
.
.
a
n1
_

_
=
_

_
a

0
a

1
.
.
.
a

j
.
.
.
a

n1
_

_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverting the Matrix
_

_
a
0
a
1
.
.
.
a
j
.
.
.
a
n1
_

_
=
_

_
1 1 1 . . . 1
1
2
. . .
n1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
j

2j
. . .
j (n1)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
n1

2(n1)
. . .
(n1)(n1)
_

_
1
_

_
a

0
a

1
.
.
.
a

j
.
.
.
a

n1
_

_
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverting the Matrix
2
6
6
6
6
6
6
6
6
6
6
6
4
1 1 1 . . . 1
1
2
. . .
n1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
j

2j
. . .
j (n1)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
n1

2(n1)
. . .
(n1)(n1)
3
7
7
7
7
7
7
7
7
7
7
7
5
1
=
1
n
2
6
6
6
6
6
6
6
6
6
6
6
4
1 1 1 . . . 1
1
1

2
. . .
(n1)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
j

2j
. . .
j (n1)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
(n1)

2(n1)
. . .
(n1)(n1)
3
7
7
7
7
7
7
7
7
7
7
7
5
Replace by
1
which is also a root of unity!
Inverse matrix is simply a permutation of the original matrix
modulo scale factor 1/n.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Why does it work?
Can check using simple algebra VV
1
= I where V is the orignal
matrix and I is the n n identity matrix.
(1,
j
,
2j
, . . . ,
j (n1)
)(1,
k
,
2k
, . . . ,
k(n1)
) =
n1

s=0

(j k)s
Note that
j k
is a nth root of unity. If j = k then sum is n,
otherwise by previous observation sum is 0.
Rows of matrix V (and hence also those of V
1
) are orthogonal.
Thus a

= Va can be thought of a transforming the vector a into a


new Fourier basis with basis vectors corresponding to rows of V.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverse Fourier Transform
Input Given the evaluation of a 2n 1-degree polynomial c
on the 2nth roots of unity
Goal Compute the coecients of c
We saw that a

can be computed from a in O(n log n) time. Can


we compute a from a

in O(n log n) time?


Yes! a = V
1
a which is simply a permuted and scaled version of
DFT. Hence can be computed in O(n log n) time.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Inverse Fourier Transform
Input Given the evaluation of a 2n 1-degree polynomial c
on the 2nth roots of unity
Goal Compute the coecients of c
We saw that a

can be computed from a in O(n log n) time. Can


we compute a from a

in O(n log n) time?


Yes! a = V
1
a which is simply a permuted and scaled version of
DFT. Hence can be computed in O(n log n) time.
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions Once More
Convolutions
Compute convolution of a = (a
0
, a
1
, . . . a
n1
) and
b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the 2nth roots of unity
2
Compute sample representation c

of product c = a b
3
Compute c from c

using inverse Fourier transform.


Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Step 3 takes O(n log n) using one FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions Once More
Convolutions
Compute convolution of a = (a
0
, a
1
, . . . a
n1
) and
b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the 2nth roots of unity
2
Compute sample representation c

of product c = a b
3
Compute c from c

using inverse Fourier transform.


Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Step 3 takes O(n log n) using one FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions Once More
Convolutions
Compute convolution of a = (a
0
, a
1
, . . . a
n1
) and
b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the 2nth roots of unity
2
Compute sample representation c

of product c = a b
3
Compute c from c

using inverse Fourier transform.


Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Step 3 takes O(n log n) using one FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions Once More
Convolutions
Compute convolution of a = (a
0
, a
1
, . . . a
n1
) and
b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the 2nth roots of unity
2
Compute sample representation c

of product c = a b
3
Compute c from c

using inverse Fourier transform.


Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Step 3 takes O(n log n) using one FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Convolutions Once More
Convolutions
Compute convolution of a = (a
0
, a
1
, . . . a
n1
) and
b = (b
0
, b
1
, . . . b
n1
)
1
Compute values of a and b at the 2nth roots of unity
2
Compute sample representation c

of product c = a b
3
Compute c from c

using inverse Fourier transform.


Step 1 takes O(n log n) using two FFTs
Step 2 takes O(n) time
Step 3 takes O(n log n) using one FFT
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
FFT Circuit
Algorithms Lecture 3: Fast Fourier Transforms
FFT(n/2)
FFT(n/2)
P P*
U U*
V V*
bit reversal permutation butterfly network
000
001
010
011
100
101
110
111
000
001
010
011
100
101
110
111
000
001
010
011
100
101
110
111
The recursive structure of the FFT algorithm.
If we expand this recursive structure completely, we see that the circuit splits naturally into
two parts. The left half computes the bit-reversal permutation of the input. To nd the position of
P[k] in this permutation, write k in binary, and then read the bits backward. For example, in an
8-element bit-reversal permutation, P[3] = P[011
2
] ends up in position 6 = 110
2
. The right half of
the FFT circuit is a buttery network. Buttery networks are often used to route between processors
in massively-parallel computers, since they allow any processor to communicate with any other in
only O(log n) steps.
Caveat Lector! This presentation is appropriate for graduate students or undergrads with
strong math backgrounds, but it leaves most undergrads confused. You may nd it less
confusing to approach the material in the opposite order, as follows:
First, any polynomial can be split into even-degree and odd-degree parts:
p(x) = p
even
(x
2
) + x p
odd
(x
2
).
We can evaluate p(x) by recursively evaluating p
even
(x
2
) and p
odd
(x
2
) and doing O(1)
arithmetic operations.
Now suppose our task is to evaluate the degree-n polynomial p(x) at n dierent points x,
as quickly as possible. To exploit the even/odd recursive structure, we must choose the n
evaluation points carefully. Call a set X of n values delicious if either (1) X has only one
element, or (2) the set X
2
= {x
2
| x X} has only n/2 elements and X
2
is delicious.
Clearly such a set exists only if N is a power of two. If someone magically handed us a
delicious set X, we could compute {p(x) | x X} in O(nlog n) time using the even/odd
recursive structure. Bit reversal permutation, blah blah blah, buttery network, yadda yadda
yadda.
If n is a power of two, then the set of integers {0, 1, . . . , n 1} is delicious, provided
we perform all arithmetic modulo n. But that only tells us p(x) mod n, and we want
the actual value of p(x). Of course, we can use larger moduli: {0, k, 2k, . . . , (n 1)k}
is delicious mod nk. We can avoid modular arithmetic entirely by using complex roots of
unitythe set {e
2i/n
| i = 0, 1, . . . , n 1} is delicious! The sequence of values p(e
2i/n
)
is called the discrete Fourier transform of p.
Finally, to invert this transformation from coecients to values, we repeat exactly the same
procedure, using the same delicious set but in the opposite order. Blardy blardy, linear
algebra, hi dee hi dee hi dee ho.
c Copyright 2006 Jeff Erickson. Released under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License (http://creativecommons.org/licenses/by-nc-sa/2.5/)
9
Chekuri CS473ug
Polynomials
Convolutions
FFT
Polynomial Representations
Basic Ideas
Complex Roots of Unity
FFT Algorithm
Numerical Issues
As noted earlier evaluating a polynomial a at a point x makes
numbers big
Are we cheating when we say O(n log n) algorithm for
convolution?
Can get around numerical issues work in nite elds and
avoid numbers growing too big.
Outside the scope of class.
Chekuri CS473ug

Вам также может понравиться