The Fast Fourier Transform

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

FFT

Outline and Reading

Polynomial Multiplication Problem Primitive Roots of Unity (10.4.1) The Discrete Fourier Transform (10.4.2) The FFT Algorithm (10.4.3) Integer Multiplication (10.4.4) Java FFT Integer Multiplication (10.5)

FFT

Polynomials
Polynomial:

p( x ) 5 2 x 8 x 2 3x 3 4 x 4

In general,

p ( x ) ai x
i 0

n 1

or p ( x ) a0 a1 x a2 x an 1 x
2
FFT

n 1

Polynomial Evaluation
Horners Rule:

Given coefficients (a0,a1,a2,,an-1), defining polynomial n 1 i i i 0 Given x, we can evaluate p(x) in O(n) time using the equation

p( x ) a x

p( x ) a0 x(a1 x(a2 x(an 2 xan 1 ) ))

Eval(A,x):

[Where A=(a0,a1,a2,,an-1)]

If n=1, then return a0 Else,

Let A=(a1,a2,,an-1) return a0+x*Eval(A,x)
FFT 4

[assume this can be done in constant time]

Polynomial Multiplication Problem

Given coefficients (a0,a1,a2,,an-1) and (b0,b1,b2,,bn-1) defining two polynomials, p() and q(), and number x, compute p(x)q(x). Horners rule doesnt help, since

p ( x ) q( x ) ci x
where

n 1 i 0

ci a j bi j
j 0

A straightforward evaluation would take O(n2) time. The magical FFT will do it in O(n log n) time.
FFT 5

Polynomial Interpolation & Polynomial Multiplication

Given a set of n points in the plane with distinct x-coordinates, there is exactly one (n-1)-degree polynomial going through all these points. Alternate approach to computing p(x)q(x):

Calculate p() on 2n x-values, x0,x1,,x2n-1. Calculate q() on the same 2n x values. Find the (2n-1)-degree polynomial that goes through the points {(x0,p(x0)q(x0)), (x1,p(x1)q(x1)), , (x2n-1,p(x2n-1)q(x2n-1))}.

Unfortunately, a straightforward evaluation would still take O(n2) time, as we would need to apply an O(n)-time Horners Rule evaluation to 2n different points. The magical FFT will do it in O(n log n) time, by picking 2n points that are easy to evaluate
FFT 6

Primitive Roots of Unity

A number w is a primitive n-th root of unity, for n>1, if

wn = 1 The numbers 1, w, w2, , wn-1 are all distinct Z*11:
x 1 2 3 4 5 6 7 8 9 10 x^2 1 4 9 5 3 3 5 9 4 1 x^3 1 8 5 9 4 7 2 6 3 10 x^4 1 5 4 3 9 9 3 4 5 1 x^5 1 10 1 1 1 10 10 10 1 10 x^6 1 9 3 4 5 5 4 3 9 1 x^7 1 7 9 5 3 8 6 2 4 10 x^8 1 3 5 9 4 4 9 5 3 1 x^9 1 6 4 3 9 2 8 7 5 10 x^10 1 1 1 1 1 1 1 1 1 1

Example 1:

2, 6, 7, 8 are 10-th roots of unity in Z*11 22=4, 62=3, 72=5, 82=9 are 5-th roots of unity in Z*11 2-1=6, 3-1=4, 4-1=3, 5-1=9, 6-1=2, 7-1=8, 8-1=7, 9-1=5

Example 2: The complex number e2pi/n is a primitive n-th root of unity, where i 1 FFT 7

Properties of Primitive Roots of Unity

Inverse Property: If w is a primitive root of unity, then w -1=wn-1

Proof: wwn-1=wn=1 Proof:

Cancellation Property: For non-zero -n<k<n,

w
j 0

n 1

kj

(w k )n 1 (w n )k 1 (1)k 1 11 w k k 0 k k w 1 w 1 w 1 w 1 j 0
n 1 kj

Reduction Property: If w is a primitve (2n)-th root of unity, then w2 is a primitive n-th root of unity.

Proof: If 1,w,w2,,w2n-1 are all distinct, so are 1,w2,(w2)2,,(w2)n-1

Reflective Property: If n is even, then wn/2 = -1.

Proof: By the cancellation property, for k=n/2:

0 w ( n / 2 ) j w 0 w n / 2 w 0 w n / 2 w 0 w n / 2 ( n / 2)(1 w n / 2 )
j 0

n 1

Corollary: wk+n/2= -wk.

FFT

The Discrete Fourier Transform

Given coefficients (a0,a1,a2,,an-1) for an (n-1)-degree polynomial p(x) The Discrete Fourier Transform is to evaluate p at the values

1,w,w2,,wn-1 We produce (y0,y1,y2,,yn-1), where yj=p(wj) n 1 That is, ij

y j aiw
i 0

Matrix form: y=Fa, where F[i,j]=wij.

The Inverse Discrete Fourier Transform recovers the coefficients of an (n-1)-degree polynomial given its values at 1,w,w2,,wn-1

Matrix form: a=F -1y, where F -1[i,j]=w-ij/n.

FFT

Correctness of the inverse DFT

The DFT and inverse DFT really are inverse operations Proof: Let A=F -1F. We want to show that A=I, where

1 n 1 ki kj A[i, j ] w w n k 0
If i=j, then

1 n 1 ki ki 1 n 1 0 1 A[i, i ] w w w n 1 n k 0 n k 0 n
If i and j are different, then

1 n 1 ( j i ) k A[i, j ] w 0 n k 0

(by Cancellati on Property)

FFT

10

Convolution
The DFT and the inverse DFT can be used to multiply two polynomials

[a0,a1,a2,...,an-1] Pad with n 0's [a0,a1,a2,...,an-1,0,0,...,0] DFT [y0,y1,y2,...,y2n-1]

Component Multiply

[b0,b1,b2,...,bn-1] Pad with n 0's [b0,b1,b2,...,bn-1,0,0,...,0] DFT [z0,z1,z2,...,z2n-1]

So we can get the coefficients of the product polynomial quickly if we can compute the DFT (and its inverse) quickly

[y0z0,y1z1,...,y2n-1z2n-1]

inverse DFT

[c0,c1,c2,...,c2n-1]

FFT

(Convolution)

11

The Fast Fourier Transform

The FFT is an efficient algorithm for computing the DFT The FFT is based on the divide-and-conquer paradigm:

If n is even, we can divide a polynomial

into two polynomials

and we can write

FFT

12

The FFT Algorithm

The running time is O(n log n). [inverse FFT is similar]

FFT 13

Multiplying Big Integers

Given N-bit integers I and J, compute IJ. Assume: we can multiply words of O(log N) bits in constant time. Setup: Find a prime p=cn+1 that can be represented in one word, and set m=(log p)/3, so that we can view I and J as n-length vectors of m-bit words. Finding a primitive root of unity.

Find a generator x of Z*p. Then w=xc is a primitive n-th root of unity in Z*p (arithmetic is mod p)

Apply convolution and FFT algorithm to compute the convolution C of the vector representations of I and J. n 1 Then compute K ci 2 mi
i 0

K is a vector representing IJ, and takes O(n log n) time to compute.

FFT 14

Java Example: Multiplying Big Integers

Setup: Define BigInt class, and include essential parameters, including the prime, P, and primitive root of unity, OMEGA.

10;

FFT

15

Java Integer Multiply Method

Use convolution to multiply two big integers, this and val:

FFT

16

Java FFT in Z*p

FFT

17

Support Methods for Java FFT in Z*p

FFT

18

Non-recursive FFT
There is also a non-recursive version of the FFT

Performs the FFT in place Precomputes all roots of unity Performs a cumulative collection of shuffles on A and on B prior to the FFT, which amounts to assigning the value at index i to the index bit-reverse(i).

The code is a bit more complex, but the running time is faster by a constant, due to improved overhead

FFT

19

Experimental Results
Log-log scale shows traditional multiply runs in O(n2) time, while FFT versions are almost linear

FFT

20