CSC373: Algorithm Design, Analysis & Complexity Winter 2017

Tutorial 3: Polynomial Multiplication via Fast Fourier Transforms

TA: Eric Bannatyne January 30, 2017

Today, we’re going to learn about the fast Fourier transform, and we’ll see how it can be applied to
efficiently solve the problem of multiplying two polynomials. The fast Fourier transform is a very
famous algorithm that has tons of applications in areas like signal processing, speech recognition,
and data compression, to name a few. We won’t actually say too much about the Fourier transform
in its full generality; instead, we will focus on seeing how it can be applied to obtain faster algorithms
for polynomial multiplication.

1 Polynomial Multiplication

Suppose that we are given two polynomials of degree n − 1:1

A(x) = a0 + a1 x + a2 x2 + · · · + an−1 xn−1

B(x) = b0 + b1 x + b2 x2 + · · · + bn−1 xn−1 .

We wish to find the product of A and B, a polynomial C such that C(x) = A(x)B(x). Suppose
that we represent A as a vector of its n coefficients a = (a0 , . . . , an−1 ), and likewise B with
b = (b0 , . . . , bn−1 ). Then there is a straightforward algorithm for computing the coefficients of
C, by simply expanding out the algebraic expression A(x)B(x) and collecting like terms. This
procedure takes Θ(n2 ) operations, since we’re essentially multiplying all pairs of coefficients from
A and B. In the rest of this tutorial, we’ll see a faster method of computing the product C.

1.1 Representing Polynomials

So far, we’ve assumed that a polynomial is represented by its list of coefficients. However, the
following fact will suggest an alternative way of representing polynomials.
Fact 1. A set of n point-value pairs (say in R2 or C2 ) uniquely determines a polynomial of degree
n − 1.

This is essentially a generalization of the statement “given two points in the plane, there is a unique
line that passes through them.” Proving this is an exercise in linear algebra; given n point-value
pairs, we can compute the coefficients of a polynomial passing through those points by solving a
system of linear equations.2
This leads us to the point-value representation of a polynomial: Given points x0 , . . . , xn−1 ∈ C, a
degree-n − 1 polynomial A(x) can be represented by the set

{(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn−1 , A(xn−1 ))}.

1
For simplicity, we’ll assume that n is a power of 2.
2
This system of equations is defined using a Vandermonde matrix, which we’ll see in more detail later.

1
Using this representation, it’s now much easier to multiply two polynomials A and B, by simply
multiplying their values on the sample points x0 , . . . , xn−1 pointwise:

{(x0 , A(x0 )B(x0 )), (x1 , A(x1 )B(x1 )), . . . , (xn−1 , A(xn−1 )B(xn−1 ))}. (⋆)

Therefore, if we use the point-value representation for polynomials, then we can multiply two
polynomials of degree n − 1 using only Θ(n) arithmetic operations. However, there’s still a slight
problem: If A(x) and B(x) are both polynomials of degree n − 1, then their product will be a
polynomial C(x) = A(x)B(x) of degree n − 1 + n − 1 = 2n − 2. But the result computed by (⋆) only
contains n sample points; not the 2n − 1 points needed to represent a polynomial of degree 2n − 2.
To fix this, we can represent A and B using an “extended” point-value representation, by using 2n
point-value pairs, instead of just3 n. Asymptotically, this doesn’t affect our overall running time,
since it only increases our input size by a constant factor of 2.

1.2 Plan of Attack

It’s great that we can multiply polynomials in linear time, using their point-value representation,
but we’d really like to do this using their coefficient representation. This gives us the outline of our
algorithm for multiplying polynomials: Given two polynomials A and B in coefficient form, convert
them to point-value form by evaluating them at 2n points, then multiply them pointwise in linear
time, and finally convert the result back to coefficient form, via a process called interpolation.
In general, evaluating a polynomial A at a single point x takes Θ(n) operations, and since we have
to evaluate 2n points, the evaluation step takes time Θ(n2 ). So it seems like we still aren’t doing
any better than the ordinary multiplication algorithm.
The key point is that the evaluation step will take Θ(n2 ) operations if we just choose any old points
x0 , . . . , x2n−1 . But for a particular choice of points, namely, the complex roots of unity, we’ll see
how we can evaluate A on that set of points using only Θ(n log n) operations, using the fast Fourier
transform. It’ll also be possible to use the FFT to do the interpolation step, once we have the
product of A and B in point-value form.

2 Complex Roots of Unity

2πi
A number z ∈ C is an nth root of unity if z n = 1. The principal nth root of unity is ωn = e n . For
n ≥ 1, the n nth roots of unity are ωn0 , ωn1 , . . . , ωnn−1 . For a bit of geometric intuition, the nth roots
of unity form the vertices of a regular n-gon in the complex plane. We’ll use a few basic properties
of complex roots of unity.

Lemma 2 (Cancellation lemma). For integers n ≥ 0, k ≥ 0, d > 0, we have ωdn

dk = ω k .
n

Proof.
( 2πi )dk 2πik
dk
ωdn = e dn = e n = ωnk .
3
We use 2n point-value pairs instead of just 2n − 1, since if n is a power of 2, then so is 2n. This will be useful
later.

2
 Evaluation
Time Θ(n2 )
Coefficient vectors Θ(n log n) with FFT Point-value representation
(a0 , . . . , an−1 ) {(x0 , A(x0 ), . . . , (x2n−1 , A(x2n−1 ))}
(b0 , . . . , bn−1 ) {(x0 , B(x0 ), . . . , (x2n−1 , B(x2n−1 ))}

Ordinary Pointwise
multiplication multiplication
Time Θ(n2 ) Time Θ(n)

Coefficient vector Point-value representation

of product of product
(c0 , . . . , c2n−2 ) Interpolation {(x0 , A(x0 )B(x0 ), . . . , (x2n−1 , A(x2n−1 )B(x2n−1 ))}
Time Θ(n log n)
with FFT

Figure 1: Outline of the approach to efficient polynomial multiplication using the fast Fourier
transform.

Lemma 3 (Halving lemma). If n > 0 is even, then the squares of the n complex nth roots of unity
are the n2 complex n2 th roots of unity.

Proof. For any nonnegative k, we have (ωnk )2 = ωn2k = ωn/2

k .

n
Note that squaring all of the nth roots of unity gives us each 2 th root of unity twice, since
k+n/2 2
(ωn ) = ωn2k+n = ωn2k ωnn = (ωnk )2 .

Lemma 4 (Summation lemma). If n ≥ 1 and k is not divisible by n, then

∑
n−1
(ωnk )j = 0.
j=0

Proof. Using the formula for the sum of a geometric series,

∑
n−1
(ωnk )n − 1 (ω n )k − 1 1k − 1
(ωnk )j = = nk = k = 0.
ωn − 1
k ωn − 1 ωn − 1
j=0

Since k is not divisible by n, we know that ωnk ̸= 1, so that the denominator is not zero.

3
3 The Discrete Fourier Transform for Polynomial Evaluation

Now we are ready to define the discrete Fourier transform, and see how it can be applied to the
problem of evaluating a polynomial on the complex roots of unity.

Definition 5. Let a = (a0 , . . . , an−1 ) ∈ Cn . The discrete Fourier transform of a is the vector
DFTn (a) = (â0 , . . . , ân−1 ), where4

∑
n−1
2πikj ∑
n−1
âk = aj e n = aj ωnkj 0 ≤ k ≤ n − 1.
j=0 j=1

If you look closely, you can see that this definition is in fact exactly what we want. Let A(x) =
a0 + a1 x + a2 x2 + · · · + an−1 xn−1 be the polynomial whose coefficients are given by a. Then for
0 ≤ k ≤ n − 1, the polynomial A evaluated at the root of unity ωnk is exactly

∑
n−1
A(ωnk ) = aj (ωnk )j = âk .
j=0

Here, the n is really the 2n from previous sections, however we will simply use n to denote the
length of the vector a when describing the DFT to simplify matters.
So far though, we still haven’t gained much, since applying this definition and directly computing
the sum for each k takes Θ(n2 ) operations. This is where the fast Fourier transform comes in: this
will allow us to compute DFTn (a) in time Θ(n log n).

4 Fast Fourier Transform

The fast Fourier transform is an algorithm for computing the discrete Fourier transform of a se-
quence by using a divide-and-conquer approach.
As always, assume that n is a power of 2. Given a = (a0 , . . . , an−1 ) ∈ Cn , we have the polynomial
A(x), defined as above, which has n terms. We can also define two other polynomials, each with n2
terms, by taking the even and odd coefficients of A, respectively:

Ae (x) = a0 + a2 x + a4 x2 + · · · + an−2 x 2 −1 ,
n

Ao (x) = a1 + a3 x + a5 x2 + · · · + an−1 x 2 −1 .
n

Then, for any x ∈ C, we can evaluate A at x using the formula

A(x) = Ae (x2 ) + xAo (x2 ). (∗)

So the problem of evaluating A at ωn0 , ωn1 , . . . , ωnn−1 reduces to performing the following steps:
4
If you’ve seen Fourier transforms before in the context of signal processing or other areas, you’ve probably seen
it defined in terms of ωn−1 rather than ωn . The form used here simplifies matters a bit for our specific application,
and the math is almost the same either way.

4
 1. evaluating two polynomials of degree n
2 − 1 at the points (ωn0 )2 , (ωn1 )2 , . . . , (ωnn−1 )2 , and

2. combining the results using (∗).

By the halving lemma, we only actually need to evaluate Ae and Ao at n2 points, rather than at all
n points. Thus, at each step, we divide the initial problem of size n into two subproblems of size
n
2.

Now that we have the general idea behind how the FFT works, we can look at some pseudocode.

Algorithm 1 Recursive version of the fast Fourier transform.

1: function Recursive-FFT(a)
2: n = a.length
3: if n == 1 then
4: return a
5: ae = (a0 , a2 , . . . , an−2 )
6: ao = (a1 , a3 , . . . , an−1 )
7: ye = Recursive-FFT(ae )
8: yo = Recursive-FFT(ao )
2πi
9: ωn = e n
10: ω=1
11: for k = 0 to n2 − 1 do
12: yk = (ye )k + ω(y0 )k
13: yk+ n2 = (ye )k − ω(yo )k
14: ω = ωωn . ▷ ω = ωnk+1
15: y = (y0 , . . . , yn−1 )
16: return y

The pseudocode more or less follows the algorithmic structure outlined above. The main thing that
we need to check is that the for loop actually combines the results using the correct formula. For
k = 0, 1, . . . , n2 − 1, in the recursive calls we compute, by the cancellation lemma,
k
(ye )k = Ae (ωn/2 ) = Ae (ωn2k ),
k
(yo )k = Ao (ωn/2 ) = Ao (ωn2k ).

In the for loop, for each k = 0, . . . , n2 − 1, we compute

yk = Ae (ωn2k ) + ωnk Ao (ωn2k ) = A(ωnk ).

We also compute
k
yk+ n2 = Ae (ωn/2 ) − ωnk Ao (ωn/2
k
)
k+ n
= Ae (ωn2k ) + ωn 2
Ao (ωn2k )
k+ n
= Ae (ωn2k+n ) + ωn 2
Ao (ωn2k+n )
k+ n
2
= A(ωn ).

5
Therefore yk = A(ωnk ) = âk for all 0 ≤ k ≤ n − 1, and therefore this algorithm returns the correct
result. Moreover, let T (n) denote the running time of Recursive-FFT(a) when a has length n.
Since the algorithm makes two recursive calls on subproblems of size n2 , and uses Θ(n) steps to
combine the results, the overall running time of this algorithm is
(n)
T (n) = 2T + Θ(n) = Θ(n log n).
2

5 Inversion Formula and Interpolation via FFT

Now that, given the coefficients of a polynomial, we can evaluate it on the roots of unity by using
the FFT, we want to perform the inverse operation, interpolating a polynomial from its values on
the nth roots of unity, so that we can convert a polynomial from its point-value representation,
back to its coefficient representation.
The first idea here will be to observe that DFTn : Cn → Cn is a linear map. This means that it
can be written in terms of a matrix multiplication:
    
1 1 1 ··· 1 a0 â0
1 ωn ωn 2 ··· n−1
ωn   
a1  â1
    
 .. .. .. .. ..   
=
.. . ..
. . . . .   
.  .
2(n−1) (n−1)2
1 ωnn−1 ωn ··· ωn an−1 ân−1

We can denote the matrix on the left-hand side of this equation by Mn (ωn ). This matrix has
a special name in linear algebra, and is called a Vandermonde matrix. To solve the problem of
performing the reverse of the discrete Fourier transform, and convert from the point-value repre-
sentation to the coefficient representation of a polynomial, we simply need to find the inverse of
this matrix. It turns out that the structure of this particular matrix makes it particularly easy to
invert.
Theorem 6 (Inversion theorem). For n ≥ 1, the matrix Mn (ωn ) is invertible, and
1
Mn (ωn )−1 = Mn (ωn−1 ).
n
′
Proof. The (j, j ′ ) entry of Mn (ωn ) is equal to ωnjj . Similarly, the (j, j ′ ) entry of 1 −1
n Mn (ωn ) is
1 −jj ′
n ωn . Therefore, the (j, j ′ ) entry of the product n1 Mn (ωn−1 )Mn (ωn ) is equal to

1 ∑ −kj kj ′ 1 ∑ k(j ′ −j)

n−1 n−1
ωn ωn = ωn .
n n
k=0 k=0

If j ′ = j, then this sum equals n, and so the entire expression evaluates to 1. On the other hand, if
j ′ ̸= j, then this sum equals 0, by the summation lemma. Therefore n1 Mn (ωn−1 )Mn (ωn ) = In , the
n × n identity matrix, and so Mn (ωn )−1 = n1 Mn (ωn−1 ).

This key takeaway of this inversion formula is that enables us to invert the discrete Fourier transform
by using the exact same algorithm as the fast Fourier transform, but with ωn replaced with ωn−1 ,

6
and then dividing the entire result by n. Therefore it is also possible to perform polynomial
interpolation (from the complex roots of unity) in time Θ(n log n).
Putting it all together, this demonstrates how it is possible to multiply two polynomials, by com-
puting their values on the 2nth roots of unity, doing pointwise multiplication, and interpolating
the result to obtain the coefficients of the product, all in time Θ(n log n), a significant improvement
over the naive Θ(n2 ) algorithm.