Fast Fourier Transform

Polynomial Multiplications and Fast Fourier Transform

Polynomial Multiplication

▪ Problem: Given two polynomials 𝑝(𝑥) and 𝑞(𝑥) with

degree 𝑑 − 1, compute its product 𝑟 𝑥 = 𝑝 𝑥 𝑞(𝑥).
▪ Each polynomial is encoded by its coefficients:
– 𝑝 𝑥 = σ𝑑−1 𝑖
𝑖=0 𝑎𝑖 𝑥 → (𝒂𝟎 , 𝒂𝟏 , … , 𝒂𝒅−𝟏 )
– 𝑞 𝑥 = σ𝑑−1 𝑏
𝑖=0 𝑖 𝑥 𝑖 → (𝒃 , 𝒃 , … , 𝒃
𝟎 𝟏 𝒅−𝟏 )

▪ Need to compute
2𝑑−2 𝑖
𝑟 𝑥 =෍ 𝑐𝑖 𝑥 𝑖 where 𝑐𝑖 = ෍ 𝑎𝑘 𝑏𝑖−𝑘
𝑖=0 𝑘=0

▪ Naïve computation: 𝑂 𝑑2
Polynomial Multiplication

▪ Given 𝑝 𝑥 = σ𝑑−1 𝑎
𝑖=0 𝑖 𝑥 𝑖
and 𝑞 𝑥 = σ 𝑑−1
𝑏
𝑖=0 𝑖 𝑥 𝑖

▪ Compute 𝑟 𝑥 = σ2𝑑−2
𝑖=0 𝑐𝑖 𝑥
𝑖 where 𝑐 = σ𝑖
𝑖 𝑘=0 𝑎𝑘 𝑏𝑖−𝑘

▪ Can we do better than 𝑂 𝑑2 ?

 Divide and Conquer

▪ Adapt Karatsuba Algorithm

▪ Assume 𝑑 is an integer power of 2.
𝑑
▪ Write 𝑝 𝑥 = 𝑝1 𝑥 + 𝑝2 𝑥 ⋅ 𝑥 where 2
𝑑 𝑑
𝑝1 𝑥 = 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎𝑑 𝑥 2 −1 and 𝑝2 𝑥 = 𝑎𝑑 + 𝑎𝑑 𝑥+ ⋯ + 𝑎𝑑−1 𝑥 2 −1
−1 +1
2 2 2
𝑑
▪ Similarly, write 𝑞 𝑥 = 𝑞1 𝑥 + 𝑞2 (𝑥) ⋅ 𝑥 2

𝑑
▪ Then, 𝑟 = 𝑝1 𝑞1 + 𝑝1 𝑞2 + 𝑝2 𝑞1 𝑥 + 𝑝2 𝑞2 𝑥 𝑑 . We need to compute
2
Adapting Karatsuba Algorithm

▪ Need to compute 𝑝1 𝑞1 , 𝑝2 𝑞2 , and 𝑝1 𝑞2 + 𝑝2 𝑞1

▪ 𝑝1 𝑞2 + 𝑝2 𝑞1 = 𝑝1 + 𝑝2 𝑞1 + 𝑞2 − 𝑝1 𝑞1 − 𝑝2 𝑞2
▪ Compute
– 𝑝1 𝑞1
– 𝑝2 𝑞2
– 𝑝1 + 𝑝2 𝑞1 + 𝑞2
𝑑
▪ One size-𝑑 multiplication → Three size- multiplications
2

▪ Time Complexity
𝑑
𝑇 𝑑 = 3𝑇 + 𝑂 𝑑 ⟹ 𝑇 𝑑 = 𝑂 𝑑 log2 3
2
Polynomial Multiplications vs Integer
Multiplications

▪ 23341 = 2 × 104 + 3 × 103 + 3 × 102 + 4 × 10 + 1

▪ 𝑝 𝑥 = 2𝑥 4 + 3𝑥 3 + 3𝑥 2 + 4𝑥 +1
▪ Polynomials and integers are similar!
▪ Perhaps the only difference in multiplications is “carry”.
▪ Some tricky things about computational model.
▪ FFT-based algorithms for integer multiplications:
– Schonhage-Strassen (1971): 𝑂(𝑛 log 𝑛 log log 𝑛)
– Furer (2007): 𝑂(𝑛 log 𝑛 log ∗ 𝑛)
– Harvey and van der Hoeven (2019): 𝑂(𝑛 log 𝑛)
Fast Fourier Transform (FFT)

▪ In this lecture, we will learn a new divide and conquer

algorithm with time complexity 𝑂(𝑑 log 𝑑)!

▪ Fast Fourier Transform (FFT)

▪ Polynomial Interpolation
▪ Complex Numbers
Another Interpretation of A Polynomial

Polynomial Interpolation
▪ Represent a polynomial 𝑝(𝑥) of degree 𝑑 − 1 by 𝑑 points
𝑥0 , 𝑝 𝑥0 , 𝑥1 , 𝑝 𝑥1 , … , 𝑥𝑑−1 , 𝑝 𝑥𝑑−1
where 𝑥0 , 𝑥1 , … , 𝑥𝑑−1 are distinct.
Before we move on…

▪ Let’s prove that 𝑑 distinct points can indeed uniquely

determine a polynomial of degree 𝑑 − 1.

Interpolation Theorem. Given 𝑑 points 𝑥0 , 𝑦0 , 𝑥1 , 𝑦1 , … (𝑥𝑑−1 , 𝑦𝑑−1 )

such that 𝑥𝑖 ≠ 𝑥𝑗 for any 𝑖 ≠ 𝑗, there exists a unique polynomial 𝑝(𝑥)
with degree at most 𝑑 − 1 such that 𝑝 𝑥𝑖 = 𝑦𝑖 for each 𝑖.
Proof of Interpolation Theorem

▪ Let 𝑝 𝑥 = σ𝑑−1
𝑡=0 𝑎𝑡 𝑥 𝑡 . We have 𝑦 = σ𝑑−1 𝑎 𝑥 𝑡 for each 𝑖.
𝑖 𝑡=0 𝑡 𝑖

𝑦0 1 𝑥0 𝑥02 ⋯ 𝑥0𝑑−1 𝑎0
𝑦1 1 𝑥1 𝑥12 ⋯ 𝑥1𝑑−1 ⋅ 𝑎1
⋮ = ⋮
⋮ ⋮ ⋮ ⋱ ⋮
𝑦𝑑−1 1 𝑥𝑑−1 2
𝑥𝑑−1 𝑑−1
⋯ 𝑥𝑑−1 𝑎𝑑−1

▪ We want to show: (𝑎0 , 𝑎1 , … , 𝑎𝑑−1 ) satisfying the above

equation is unique.
▪ The yellow matrix is a Vandermonde matrix with determinant
ς0≤𝑖<𝑗≤𝑑−1(𝑥𝑗 − 𝑥𝑖 ), which is nonzero given 𝑥𝑖 ≠ 𝑥𝑗 .
▪ Uniqueness is proved: 𝐲 = 𝑋𝐚 ⟹ 𝐚 = 𝑋 −1 𝐲
Framework for FFT

▪ Interpolation Step (FFT):

– Choose 2𝑑 − 1 distinct numbers 𝑥0 , 𝑥1 , … , 𝑥2𝑑−2 , and
– Compute the values of
▪ 𝑝 𝑥0 , 𝑝 𝑥1 , … , 𝑝 𝑥2𝑑−2
▪ 𝑞(𝑥0 ), 𝑞(𝑥1 ), … , 𝑞(𝑥2𝑑−2 )

▪ Multiplication Step:
– For each 𝑖 = 0,1, … , 2𝑑 − 2, compute 𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
– Obtain interpolation for 𝑟(𝑥): 𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2

▪ Recovery Step (inverse FFT):

– Recover (𝑐0 , 𝑐1 , … , 𝑐2𝑑−2 ), the polynomial 𝑟 𝑥 = σ2𝑑−2
𝑖=0 𝑐𝑖 𝑥 , from the
𝑖

interpolation obtained in the previous step.

 Framework for FFT

Objective
𝑝 𝑥 = 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎𝑑−1 𝑥 𝑑−1
𝑟 𝑥 = 𝑝 𝑥 ⋅ 𝑞(𝑥)
𝑞 𝑥 = 𝑏0 + 𝑏1 𝑥 + ⋯ + 𝑏𝑑−1 𝑥 𝑑−1 = 𝑐0 + 𝑐1 𝑥 + ⋯ + 𝑐2𝑑−2 𝑥 2𝑑−2

Interpolation Step Recovery Step

(FFT) (Inverse FFT)

𝑥0 , 𝑝 𝑥0 , 𝑥1 , 𝑝 𝑥1 , … , 𝑥2𝑑−2 , 𝑝 𝑥2𝑑−2
𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2
𝑥0 , 𝑞 𝑥0 , 𝑥1 , 𝑞 𝑥1 , … , 𝑥2𝑑−2 , 𝑞 𝑥2𝑑−2
Multiplication
𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
Step 1: Interpolation
Interpolation Step (FFT)
Interpolation Step

▪ Interpolation Step (FFT):

– Choose 2𝑑 − 1 distinct numbers 𝑥0 , 𝑥1 , … , 𝑥2𝑑−2 , and
– Compute the values of 𝑝(𝑥0 ), 𝑝(𝑥1 ), … , 𝑝(𝑥2𝑑−2 ), 𝑞(𝑥0 ), 𝑞(𝑥1 ), … , 𝑞(𝑥2𝑑−2 )

▪ Computing each 𝑝(𝑥𝑖 ) or 𝑞(𝑥𝑖 ) requires 𝑂(𝑑) time.

– assume we can do 𝑥 𝑑 fast.

▪ We need to compute 4𝑑 − 2 of them.

▪ Overall time complexity: 𝑂 𝑑2 .
– Even assume we can calculate 𝑎𝑖𝑑 fast.

▪ Can we do faster by divide and conquer?

Some Notations

▪ Let 𝐷 = 2𝑑 − 1.
▪ Assume 𝐷 is an integer power of 2.
– We want to divide and conquer!

▪ We can solve 𝑥 𝑑 in 𝑂 1 ! Seems impossible, but it

dose not matter.

▪ Interpolation Step (FFT):

– Choose 𝐷 − 1 distinct numbers 𝑥0 , 𝑥1 , … , 𝑥𝐷−1 , and
– compute the values of 𝑝(𝑥0 ), 𝑝(𝑥1 ), … , 𝑝(𝑥𝐷−1 ), 𝑞(𝑥0 ), 𝑞(𝑥1 ), … , 𝑞(𝑥𝐷−1 )
Can we calculate 𝑝 𝑥𝑖
faster?
Divide and Conquer: Computing 𝑝(𝑥𝑖 )

𝐷
▪ “Left-right decomposition”: 𝑝 𝑥𝑖 = 𝑝1 𝑥𝑖 + 𝑝2 𝑥𝑖 ⋅ 𝑥𝑖 2

𝑎0 𝑎1 𝑥𝑖 … 𝐷/2−1
𝑎𝐷/2−1 𝑥𝑖
𝐷/2
𝑎𝐷/2 𝑥𝑖
𝐷/2+1
𝑎𝐷/2+1 𝑥𝑖 … 𝑎𝐷−1 𝑥𝑖𝐷−1

𝑝1 (𝑥𝑖 ) 𝑝2 (𝑥𝑖 )
Divide and Conquer: Computing 𝑝(𝑥𝑖 )

▪ Compute 𝑝1 𝑥𝑖 and 𝑝2 𝑥𝑖 recursively.

𝐷
▪ Time complexity: 𝑇 𝐷 = 2𝑇 + 𝑂 1 ⟹ 𝑇 𝐷 = 𝑂(𝐷)
2

▪ No faster than direct computation!

Divide and Conquer: Computing different 𝑥𝑖

▪ Divide among different 𝑥𝑖 …

𝑝(𝑥0 )
𝑝(𝑥1 )
…
𝑝(𝑥𝐷/2−1 )
𝑝(𝑥𝐷/2 )
𝑝(𝑥𝐷/2+1 )
…
𝑝(𝑥𝐷−1 )
Lessons we learned

▪ Computing each 𝑝(𝑥𝑖 ) requires 𝑂(𝐷) time.

– It seems very hard to improve!
▪ We need to choose 𝑥0 , 𝑥1 , … , 𝑥𝐷−1 in a clever way so
that, for example, 𝑝(𝑥1 ) and 𝑝(𝑥2 ) can be
computed together!
▪ Consider the example 𝑝 1 and 𝑝(−1).
An Idea to Compute 𝑝(𝑥1 ) and 𝑝(𝑥2 ) Together

▪ Even-Odd Decomposition:
𝑝 𝑥 = 𝑝𝑒 𝑥 2 + 𝑥 ⋅ 𝑝𝑜 𝑥 2 ,
where
𝐷−2
2
𝑝𝑒 𝑥 = 𝑎0 + 𝑎2 𝑥 + 𝑎4 𝑥 + ⋯ + 𝑎𝐷−2 𝑥 2
𝐷−2
𝑝𝑜 𝑥 = 𝑎1 + 𝑎3 𝑥 2
+ 𝑎5 𝑥 + ⋯ + 𝑎𝐷−1 𝑥 2
▪ Choose 𝑥1 and 𝑥2 such that 𝑥1 = −𝑥2 . We have
𝑝𝑒 𝑥12 = 𝑝𝑒 𝑥22 and 𝑝𝑜 𝑥12 = 𝑝𝑜 𝑥22
An Idea to Compute 𝑝(𝑥1 ) and 𝑝(𝑥2 ) Together

𝑝 𝑥1 = 𝑝𝑒 𝑥12 + 𝑥1 ⋅ 𝑝𝑜 𝑥12

𝑝 𝑥2 = 𝑝𝑒 𝑥22 + 𝑥2 ⋅ 𝑝𝑜 𝑥22

𝐷
Two size-𝐷 computations → four two size- computations, great!
2
 A Divide and Conquer Attempt

▪ Choose 𝑥0 , 𝑥1 , … , 𝑥𝐷−1 such that 𝑥0 = −𝑥1 , 𝑥2 = −𝑥3 , …, 𝑥𝐷−2 = −𝑥𝐷−1 .

▪ Divide:
– 𝑝𝑒 𝑥02 , 𝑝𝑒 𝑥22 , … , 𝑝𝑒 𝑥𝐷−2
2

– 𝑝𝑜 𝑥02 , 𝑝𝑜 𝑥22 , … , 𝑝𝑜 𝑥𝐷−2

2

▪ Combine: Compute 𝑝 𝑥𝑖 = 𝑝𝑒 𝑥𝑖2 + 𝑥𝑖 ⋅ 𝑝𝑜 𝑥𝑖2 .

▪ Time Complexity
𝐷
▪ 𝑇 𝐷 = 2𝑇 + 𝑂 𝐷 ⟹ 𝑇 𝐷 = 𝑂 𝐷log𝐷
2
– 𝑇 𝐷 : compute 𝐷 𝑝(𝑥) where the degree of 𝑝(𝑥) is 𝐷.
 What Happens?

𝑝(𝑥0 ) 𝑝𝑜 (𝑥02 )
𝑝(𝑥1 ) 𝑝𝑜 (𝑥22 )
… …
𝑝(𝑥𝐷/2−1 ) 2
𝑝𝑜 (𝑥𝐷−2 )
𝑝(𝑥𝐷/2 ) 𝑂 𝐷 𝑝𝑒 (𝑥02 )
𝑝(𝑥𝐷/2+1 ) 𝑝𝑒 (𝑥12 )
… …
2
𝑝(𝑥𝐷−1 ) 𝑝𝑒 (𝑥𝐷−2 )

▪ Time Complexity
𝐷
▪ 𝑇 𝐷 = 2𝑇 + 𝑂 𝐷 ⟹ 𝑇 𝐷 = 𝑂 𝐷log𝐷
2
Are We Done?
 Are We Done?

▪ Choose 𝑥0 , 𝑥1 , … , 𝑥𝐷−1 such that 𝑥0 = −𝑥1 , 𝑥2 = −𝑥3 , …, 𝑥𝐷−2 = −𝑥𝐷−1 .

▪ Divide:
– 𝑝𝑒 𝑥02 , 𝑝𝑒 𝑥22 , … , 𝑝𝑒 𝑥𝐷−2
2
How to do it recursively?
– 𝑝𝑜 𝑥02 , 𝑝𝑜 𝑥22 , … , 𝑝𝑜 𝑥𝐷−2
2

▪ Combine: Compute 𝑝 𝑥𝑖 = 𝑝𝑒 𝑥𝑖2 + 𝑥𝑖 ⋅ 𝑝𝑜 𝑥𝑖2 .

▪ In the second step:
– 𝑥02 , 𝑥12 … … are all positive!
– How to make 𝑥02 = −𝑥12 ?
Why it fails?

𝑥0 𝑥1 = −𝑥0 𝑥2 𝑥3 = −𝑥2 𝑥4 𝑥5 = −𝑥4 𝑥6 𝑥7 = −𝑥6

I can do it!

𝑥02 𝑥22 𝑥42 𝑥62

How to make
𝑥02 = −𝑥22 .

? ?
Complex Numbers

▪ 𝑧 = 𝑎 + 𝑏𝑖
– 𝑎: real part
– 𝑏: imaginary part
– 𝑖 = −1: imaginary unit

▪ Polar form: 𝑧 = 𝑟 cos 𝜃 + 𝑖 sin 𝜃

– 𝑟: the length of the 2-dimensional vector 𝑎, 𝑏
– 𝜃: the angle between the vector 𝑎, 𝑏 and the 𝑥-axis (the real axis)

▪ Euler’s formula: 𝑧 = 𝑟 cos 𝜃 + 𝑖 sin 𝜃 = 𝑟 ⋅ 𝑒 𝜃𝑖

Squares and Square Roots of Unit Length
Complex Numbers

▪ The square of 𝑒 𝜃𝑖 is 𝑒 2𝜃𝑖 : we have just rotated 𝑒 𝜃𝑖 by an

angle 𝜃.
▪ Two complex numbers of unit length opposite to each
other have the same square:
𝜃+𝜋 𝑖 2 2𝜃𝑖 2𝜋𝑖 2𝜃𝑖 𝜃𝑖 2
𝑒 =𝑒 ⋅𝑒 =𝑒 = 𝑒
𝜃 𝜃
𝑖 +𝜋 𝑖
▪ The square roots of 𝑒 𝜃𝑖 are 𝑒 2 and 𝑒 2
Example for 𝐷 = 8
𝜋 𝜋 3𝜋
𝜔0 = 1, 𝜔1 = 𝑒 4𝑖 , 𝜔2 = 𝑒 2𝑖 , 𝜔3 = 𝑒4𝑖
5𝜋 3𝜋 7𝜋
𝜔4 = 𝑒 𝜋𝑖 , 𝜔5 = 𝑒 4 𝑖, 𝜔6 = 𝑒 2 𝑖, 𝜔7 = 𝑒4𝑖
Example for 𝐷 = 8
How to represent the 𝐷 points

▪ Fix 𝐷, we know the 𝐷 numbers are 𝜔0 , 𝜔1 , 𝜔2 … , 𝜔𝐷−1 .

2𝜋
𝑖
– 𝜔=𝑒 𝐷

▪ We only need one parameter 𝜔 to represent the 𝐷

numbers!
▪ What about the next level numbers?
▪ They are 𝜔0 , 𝜔2 , 𝜔4 … , 𝜔𝐷−2 .
▪ We can use 𝜔2 to represent the next level numbers!
Interpolation: Putting Together

Algorithm 1: Fast Fourier Transform

2𝜋
𝑖
FFT(𝑝, 𝜔): // 𝑝 is a polynomial of degree 𝐷 − 1 and 𝜔 = 𝑒 𝐷

1. if 𝜔 = 1, return (𝑝(1));
𝐷−2
2
2. 𝑝𝑒 𝑥 = 𝑎0 + 𝑎2 𝑥 + 𝑎4 𝑥 + ⋯ + 𝑎𝐷−2 𝑥 2

𝐷−2
2
3. 𝑝𝑜 𝑥 = 𝑎1 + 𝑎3 𝑥 + 𝑎5 𝑥 + ⋯ + 𝑎𝐷−1 𝑥 2

4. 𝑝𝑒 𝜔0 , 𝑝𝑒 𝜔2 , … , 𝑝𝑒 𝜔𝐷−2 ← FFT 𝑝𝑒 , 𝜔2 ;

5. 𝑝𝑜 𝜔0 , 𝑝𝑜 𝜔2 , … , 𝑝𝑜 𝜔𝐷−2 ← FFT 𝑝𝑜 , 𝜔2 ;
6. for 𝑡 = 0,1, … , 𝐷 − 1:
7. 𝑝 𝜔𝑡 = 𝑝𝑒 𝜔2𝑡 + 𝜔𝑡 ⋅ 𝑝𝑜 𝜔2𝑡
8. return 𝑝 𝜔0 , 𝑝 𝜔1 , … , 𝑝 𝜔𝐷−1 ;
Time Complexity for Interpolation Step

▪ Let 𝑇(𝐷) be the time complexity for computing FFT(𝑝, 𝜔),

where 𝑝 has degree 𝐷 − 1.
𝐷
▪ We have 𝑇 𝐷 = 2𝑇 + 𝑂(𝐷) = 𝑂 𝐷 log 𝐷 .
2

▪ Interpolation step requires 𝑇 𝐷 = 𝑂 𝑑 log 𝑑 time.

 Framework for FFT

Objective
𝑝 𝑥 = 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎𝑑−1 𝑥 𝑑−1
𝑟 𝑥 = 𝑝 𝑥 ⋅ 𝑞(𝑥)
𝑞 𝑥 = 𝑏0 + 𝑏1 𝑥 + ⋯ + 𝑏𝑑−1 𝑥 𝑑−1 = 𝑐0 + 𝑐1 𝑥 + ⋯ + 𝑐2𝑑−2 𝑥 2𝑑−2

Interpolation Step Recovery Step

(FFT) (Inverse FFT)
𝑂(𝑑 log 𝑑)
𝑥0 , 𝑝 𝑥0 , 𝑥1 , 𝑝 𝑥1 , … , 𝑥2𝑑−2 , 𝑝 𝑥2𝑑−2
𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2
𝑥0 , 𝑞 𝑥0 , 𝑥1 , 𝑞 𝑥1 , … , 𝑥2𝑑−2 , 𝑞 𝑥2𝑑−2
Multiplication
𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
Step 2: Multiplication
Multiplication Step:
For each 𝑖 = 0,1, … , 2𝑑 − 2, compute 𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
Obtain interpolation for 𝑟(𝑥): 𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2
It’s easy! Just compute it one-by-one…

▪ For each 𝑖 = 0, 1, … , 2𝑑 − 2, compute 𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )

▪ Time complexity: 𝑂 𝑑
 Framework for FFT

Objective
𝑝 𝑥 = 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎𝑑−1 𝑥 𝑑−1
𝑟 𝑥 = 𝑝 𝑥 ⋅ 𝑞(𝑥)
𝑞 𝑥 = 𝑏0 + 𝑏1 𝑥 + ⋯ + 𝑏𝑑−1 𝑥 𝑑−1 = 𝑐0 + 𝑐1 𝑥 + ⋯ + 𝑐2𝑑−2 𝑥 2𝑑−2

Interpolation Step Recovery Step

(FFT) (Inverse FFT)
𝑂(𝑑 log 𝑑)
𝑥0 , 𝑝 𝑥0 , 𝑥1 , 𝑝 𝑥1 , … , 𝑥2𝑑−2 , 𝑝 𝑥2𝑑−2 𝑂(𝑑)
𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2
𝑥0 , 𝑞 𝑥0 , 𝑥1 , 𝑞 𝑥1 , … , 𝑥2𝑑−2 , 𝑞 𝑥2𝑑−2
Multiplication
𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
Step 3: Recovery
Recovery Step (inverse FFT):
Recover (𝑐0 , 𝑐1 , … , 𝑐2𝑑−2 ), the polynomial 𝑟 𝑥 = σ2𝑑−2
𝑖=0 𝑐𝑖 𝑥 , from the
𝑖
interpolation obtained in the previous step.
We Have Interpolation of 𝑟(𝑥) Now…

▪ We have 1, 𝑟 1 , 𝜔, 𝑟 𝜔 , 𝜔2 , 𝑟 𝜔2 , … , 𝜔𝐷−1 , 𝑟 𝜔𝐷−1 ,

2𝜋
𝑖
where 𝜔 = 𝑒 𝐷 .
𝑟(1) 1 1 1 ⋯ 1 𝑐0
𝑟(𝜔) 1 𝜔 𝜔2 ⋯ 𝜔𝐷−1 𝑐1
𝑟(𝜔2 ) = 1 𝜔2 𝜔4 ⋯ 𝜔 2(𝐷−1) ⋅ 𝑐2
⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝑟 𝜔𝐷−1 1 𝜔𝐷−1 𝜔 2(𝐷−1) ⋯ 𝜔 (𝐷−1)(𝐷−1) 𝑐𝐷−1

What we want…
We Have Interpolation of 𝑟(𝑥) Now…

▪ We have 1, 𝑟 1 , 𝜔, 𝑟 𝜔 , 𝜔2 , 𝑟 𝜔2 , … , 𝜔𝐷−1 , 𝑟 𝜔𝐷−1 ,

2𝜋
𝑖
where 𝜔 = 𝑒 .
𝐷
1 1 1 ⋯ 1
1 𝜔 𝜔2 ⋯ 𝜔𝐷−1
𝐴= 1 𝜔2 𝜔4 ⋯ 𝜔 2(𝐷−1)
⋮ ⋮ ⋮ ⋱ ⋮
1 𝜔𝐷−1 𝜔 2(𝐷−1) ⋯ 𝜔 (𝐷−1)(𝐷−1)
𝑟(1) 𝑐0
𝑟(𝜔) 𝑐1
𝐴−1 𝑟(𝜔2 ) = 𝑐2
⋮ ⋮
𝑟 𝜔𝐷−1 𝑐𝐷−1
Complex Matrices Recap

▪ The complex conjugate of 𝑧 = 𝑎 + 𝑏𝑖 is 𝑧 = 𝑎 − 𝑏𝑖.

▪ Given two complex vectors 𝐚, 𝐛 ∈ ℂ𝑛 , their inner product is
𝑛
𝐚, 𝐛 = ෍ 𝑎𝑗 ⋅ 𝑏𝑗
𝑗=1

▪ 𝐚, 𝐛 are orthogonal if 𝐚, 𝐛 = 0; 𝐚, 𝐛 are orthonormal if 𝐚, 𝐛 = 0 and

𝐚, 𝐚 = 𝐛, 𝐛 = 1.
▪ A square matrix 𝐴 is an orthonormal (unitary) matrix if every pair of
its columns is orthonormal.
▪ Conjugate transpose of 𝐴, denoted by 𝐴∗ , is defined as 𝐴∗ 𝑖,𝑗 = 𝐴𝑗,𝑖 .
▪ If 𝐴 is an orthonormal, then 𝐴 is invertible and 𝐴−1 = 𝐴∗ .
Complex Matrices Recap

▪ A square matrix 𝐴 is an orthonormal (unitary) matrix if every pair of

its columns is orthonormal.
▪ Conjugate transpose of 𝐴, denoted by 𝐴∗ , is defined as 𝐴∗ 𝑖,𝑗 = 𝐴𝑗,𝑖 .
▪ If 𝐴 is an orthonormal, then 𝐴 is invertible and 𝐴−1 = 𝐴∗ .

Example: 𝐴∗ = 𝑎ത 𝑏ത 𝐴 = 𝑎 𝑐
𝑐ҧ 𝑑ҧ 𝑏 𝑑

We at least have a method to calculate 𝑨−𝟏 !

Let’s come back…

▪ We have 1, 𝑟 1 , 𝜔, 𝑟 𝜔 , 𝜔2 , 𝑟 𝜔2 , … , 𝜔𝐷−1 , 𝑟 𝜔𝐷−1 ,

2𝜋
𝑖
where 𝜔 = 𝑒 𝐷 .
𝑟(1) 1 1 1 ⋯ 1 𝑐0
𝑟(𝜔) 1 𝜔 𝜔2 ⋯ 𝜔𝐷−1 𝑐1
𝑟(𝜔2 ) = 1 𝜔2 𝜔4 ⋯ 𝜔 2(𝐷−1) ⋅ 𝑐2
⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝑟 𝜔𝐷−1 1 𝜔𝐷−1 𝜔 2(𝐷−1) ⋯ 𝜔 (𝐷−1)(𝐷−1) 𝑐𝐷−1
𝐴(𝜔)
Is 𝐴 orthonormal (unitary)?
Two Different Columns

▪ We have 1, 𝑟 1 , 𝜔, 𝑟 𝜔 , 𝜔2 , 𝑟 𝜔2 , … , 𝜔𝐷−1 , 𝑟 𝜔𝐷−1 ,

2𝜋
where 𝜔 = 𝑒 𝐷 𝑖 . 𝑐𝑖 𝑐𝑗
𝑟(1) 1 1 1 ⋯ 1 𝑐0
𝑟(𝜔) 1 𝜔 𝜔2 ⋯ 𝜔𝐷−1 𝑐1
𝑟(𝜔2 ) = 1 𝜔2 𝜔4 ⋯ 𝜔 2(𝐷−1) ⋅ 𝑐2
⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝑟 𝜔𝐷−1 1 𝜔𝐷−1 𝜔 2(𝐷−1) ⋯ 𝜔 (𝐷−1)(𝐷−1) 𝑐𝐷−1
𝜔𝜔
ഥ =1

▪ 𝐜𝑖 , 𝐜𝑗 = σ𝐷
𝑘=1 𝜔
(𝑘−1)(𝑖−1) 𝜔 (𝑘−1)(𝑗−1)

1−𝜔 𝑗−𝑖 𝐷 𝜔𝐷 = 𝑒 2𝜋𝑖 = 1

▪ = σ𝐷
𝑘=1 𝜔 (𝑘−1)(𝑗−𝑖) = =0
1−𝜔𝑗−𝑖
The Same Column

▪ We have 1, 𝑟 1 , 𝜔, 𝑟 𝜔 , 𝜔2 , 𝑟 𝜔2 , … , 𝜔𝐷−1 , 𝑟 𝜔𝐷−1 ,

2𝜋
where 𝜔 = 𝑒 𝐷 𝑖 . 𝑐𝑖
𝑟(1) 1 1 1 ⋯ 1 𝑐0
𝑟(𝜔) 1 𝜔 𝜔2 ⋯ 𝜔𝐷−1 𝑐1
𝑟(𝜔2 ) = 1 𝜔2 𝜔4 ⋯ 𝜔 2(𝐷−1) ⋅ 𝑐2
⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝑟 𝜔𝐷−1 1 𝜔𝐷−1 𝜔 2(𝐷−1) ⋯ 𝜔 (𝐷−1)(𝐷−1) 𝑐𝐷−1

▪ 𝐜 𝑖 , 𝐜 𝑖 = σ𝐷
𝑘=1 𝜔
(𝑘−1)(𝑖−1) 𝜔 (𝑘−1)(𝑖−1) = 𝐷

𝜔𝜔
ഥ =1
𝐴 is not orthonormal.
But we can scale it!
1
𝐴(𝜔) is orthonormal!
𝐷
 2𝜋
1 𝑖
Proposition. 𝐴(𝜔) is orthonormal for 𝜔 = 𝑒 𝐷 .
𝐷

Proof.
1
▪ Let 𝐜𝑖 , 𝐜𝑗 be two arbitrary columns of 𝐴(𝜔).
𝐷 𝐷 𝐷
1 1
𝐜𝑖 , 𝐜𝑗 = ෍ ⋅ 𝜔 𝑘−1 𝑖−1 ⋅ 𝜔 (𝑘−1)(𝑗−1) = ෍ 𝜔 (𝑘−1)(𝑗−𝑖)
𝐷 𝐷
𝑘=1 𝑘=1
1
▪ If 𝑖 = 𝑗, we have 𝐜𝑖 , 𝐜𝑗 = σ𝐷 0
𝑘=1 𝜔 = 1;
𝐷
▪ If 𝑖 ≠ 𝑗, then 𝜔𝐷 = 𝑒 2𝜋𝑖 = 1
𝐷
1 1 1 − 𝜔 𝑗−𝑖 𝐷
𝐜𝑖 , 𝐜𝑗 = ෍ 𝜔 (𝑘−1)(𝑗−𝑖) = 𝑗−𝑖
=0
𝐷 𝐷 1−𝜔
𝑘=1
1
▪ Thus, 𝐴(𝜔) is orthonormal.
𝐷
 Inverting 𝐴(𝜔)…

▪ Theorem. If 𝐴 is an orthonormal, then 𝐴 is invertible and 𝐴−1 = 𝐴∗ .

2𝜋
1 𝑖
▪ Proposition. 𝐴(𝜔) is orthonormal for 𝜔 = 𝑒 𝐷 .
𝐷

▪ We have
−1 −1 ∗
−1
1 1 1 1 1 1 ∗
𝐴 𝜔 = 𝐷⋅ ⋅𝐴 𝜔 = ⋅𝐴 𝜔 = ⋅𝐴 𝜔 = 𝐴 𝜔
𝐷 𝐷 𝐷 𝐷 𝐷 𝐷
▪ Therefore,
−1
1 1 1 −1
𝐴 𝜔 𝑖,𝑗 = 𝐴 𝜔 = ⋅ 𝜔− 𝑖−1 𝑗−1 = 𝜔 𝑖−1 𝑗−1 ,
𝐷 𝑗,𝑖 𝐷 𝐷
which implies
−1
1
𝐴 𝜔 = ⋅ 𝐴 𝜔−1 .
𝐷
After blablabla math parts
 1
Putting 𝐴 𝜔 −1
= ⋅ 𝐴 𝜔−1 back
𝐷

𝒄𝟎 1 1 1 ⋯ 1 𝑟(1)
𝒄𝟏 1 𝜔 −1 𝜔−2 ⋯ 𝜔 −(𝐷−1) 𝑟(𝜔)
1
𝒄𝟐 = ⋅ 1 𝜔−2 𝜔−4 ⋯ 𝜔 −2(𝐷−1) ⋅ 𝑟(𝜔2 )
⋮ 𝐷
⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝒄𝑫−𝟏 1 𝜔 −(𝐷−1) 𝜔 −2(𝐷−1) ⋯ 𝜔 −(𝐷−1)(𝐷−1) 𝑟 𝜔𝐷−1
What we want…
▪ Naïve way also need 𝑂 𝐷2 times!
▪ How to improve it? This is 𝐴−1 !
 1
Putting 𝐴 𝜔 −1
= ⋅ 𝐴 𝜔−1 back
𝐷

𝑐0 1 1 1 ⋯ 1 𝑟(1)
𝑐1 1 𝜔 −1 𝜔−2 ⋯ 𝜔 −(𝐷−1) 𝑟(𝜔)
1
𝑐2 = ⋅ 1 𝜔−2 𝜔−4 ⋯ 𝜔 −2(𝐷−1) ⋅ 𝑟(𝜔2 )
⋮ 𝐷
⋮ ⋮ ⋮ ⋱ ⋮ ⋮
𝑐𝐷−1 1 𝜔 −(𝐷−1) 𝜔 −2(𝐷−1) ⋯ 𝜔 −(𝐷−1)(𝐷−1) 𝑟 𝜔𝐷−1

▪ This is very similar to the first step!

▪ Let 𝑠 𝑥 = 𝑟 1 + 𝑟 𝜔 ⋅ 𝑥 + 𝑟 𝜔2 ⋅ 𝑥 2 … + 𝑟 𝜔𝐷−1 ⋅ 𝑥 𝐷−1
Problem

▪ In the first step.

▪ We chose 𝐷 good numbers 𝜔0 , 𝜔1 … 𝜔𝐷−1 to make our
algorithm quick!
▪ But in this step.
▪ We must choose 𝜔0 , 𝜔−1 , 𝜔−2 … 𝜔 −(𝐷−1)
▪ Are they still good?
Let us recall when
FFT(𝑠, 𝜔 ) is good?
−1
Is 𝜔−1 , 𝜔−2 , … , 𝜔 − 𝐷−1
good?

▪ 𝜔−1 , 𝜔−2 , … , 𝜔 − 𝐷−1 is just the same as 𝜔1 , 𝜔2 , … , 𝜔 𝐷−1

with a clockwise orientation!

▪ Yes, we can just apply FFT(𝑠, 𝜔−1 )!
 Framework for FFT

Objective
𝑝 𝑥 = 𝑎0 + 𝑎1 𝑥 + ⋯ + 𝑎𝑑−1 𝑥 𝑑−1
𝑟 𝑥 = 𝑝 𝑥 ⋅ 𝑞(𝑥)
𝑞 𝑥 = 𝑏0 + 𝑏1 𝑥 + ⋯ + 𝑏𝑑−1 𝑥 𝑑−1 = 𝑐0 + 𝑐1 𝑥 + ⋯ + 𝑐2𝑑−2 𝑥 2𝑑−2

Interpolation Step Recovery Step

𝑂(𝑑 log 𝑑)
(FFT) (Inverse FFT)
𝑂(𝑑 log 𝑑)
𝑥0 , 𝑝 𝑥0 , 𝑥1 , 𝑝 𝑥1 , … , 𝑥2𝑑−2 , 𝑝 𝑥2𝑑−2 𝑂(𝑑)
𝑥0 , 𝑟 𝑥0 , 𝑥1 , 𝑟 𝑥1 , … , 𝑥2𝑑−2 , 𝑟 𝑥2𝑑−2
𝑥0 , 𝑞 𝑥0 , 𝑥1 , 𝑞 𝑥1 , … , 𝑥2𝑑−2 , 𝑞 𝑥2𝑑−2
Multiplication
𝑟 𝑥𝑖 = 𝑝 𝑥𝑖 𝑞(𝑥𝑖 )
Putting 3 Steps Together
Putting Three Steps Together

Algorithm 2: Polynomial multiplication by FFT

Multiply(𝑝, 𝑞): // 𝑝, 𝑞 are two polynomials with degrees at most 𝑑
𝐷
1. let 𝐷 be the smallest integer power of 2 such that 𝑑 ≤ ;
2
2𝜋
𝑖
2. let 𝜔 = 𝑒 𝐷 ;
3. 𝑝0 , 𝑝1 , … , 𝑝𝐷−1 ← FFT(𝑝, 𝜔); // where 𝑝𝑖 = 𝑝 𝜔𝑖
4. 𝑞0 , 𝑞1 , … , 𝑞𝐷−1 ← FFT(𝑞, 𝜔); // where 𝑞𝑖 = 𝑞 𝜔𝑖
5. for each 𝑡 = 0,1, … , 𝐷 − 1: compute 𝑟𝑡 ← 𝑝𝑡 ⋅ 𝑞𝑡
6. let 𝑠 𝑥 = σ𝐷−1
𝑡=0 𝑟𝑡 𝑥
𝑡

7. 𝑐0 , 𝑐1 , … , 𝑐𝐷−1 ← FFT(𝑠, 𝜔−1 );

𝑐
8. let 𝑟 𝑥 = σ𝐷−1
𝑡=0
𝑡 𝑡
𝑥 ;
𝐷
9. return 𝑟;
Overall Time Complexity

𝑂 𝑑 log 𝑑 + 𝑂 𝑑 + 𝑂 𝑑 log 𝑑 = 𝑂 𝑑 log 𝑑