CSE 317: Design and Analysis of

Algorithms

Shahid Hussain
Week 5: September 16, 18: Fall 2024

1
Divide and Conquer Algorithms
Maximum Subarray Sum

• For a given array A[1..n] of n numbers, the maximum

subarray sum, denoted as M is the largest sum of
contiguous subarray of A[i..j] where 1 ≤ i ≤ j ≤ n, i.e.,
j
X
M(A) = A[x]
x=i

• For example, for A = [−2, 3, 4, −1, 5, −3, 2, −1], M(A) = 11

for the subarray [3, 4, −1, 5]

2
Maximum Subarray Sum: Trivial Solution

Algorithm: bruteforce-max-subarray-sum
Input: An array A[1..n] of n numbers
Output: The maximum subarray sum M(A)
1. M = −∞
2. for i = 1 to n
3. for j = i to n
4. s=0
5. for k = i to j
6. s = s + A[k]
7. if s > M then M = s
8. return M

The time complexity of above algorithms is Θ(n3 )

3
Maximum Subarray Sum

• Let us design a divide and conquer algorithm

• We divide the array into two almost equal parts and find
the maximum subarray sum in both these parts recursively
• At this point we have three cases:
1. The maximum subarray sum is in the left part
2. The maximum subarray sum is in the right part
3. The maximum subarray sum crossovers the middle point
• The algorithm returns the maximum of these three cases
• The crossover sum can be computed in O(n) time by
starting from the middle point and going left and right to
find the maximum sum
• If T (n) represents the time complexity of the algorithm
then we have the following recurrence:
n
T (n) = 2T + O(n)
2
4
Maximum Subarray Sum: Maximum Crossing Sum

Input: Array A[l..h] of numbers

Output: The max. subarray sum

1. if l = h then return A[l]

2. m = ⌊(l + h)/2⌋
3. ML = dc-max-subarray-sum(A[l..m])
4. ML = dc-max-subarray-sum(A[m + 1..h])
5. ML = max-crossing-sum(A[l..h])
6. return max{ML , MR , MC }

5
Maximum Subarray Sum: Divide and Conquer

Algorithm: max-crossing-sum
Input: An array A[l..h] of numbers
Output: The max crossing sum

1. m = ⌊(l + h)/2⌋, s = 0, left sum = −∞

2. for i = m downto l
3. s = s + A[i]
4. if s > left sum then left sum = s
5. s = 0, right sum = −∞
6. for i = m + 1 to h
7. s = s + A[i]
8. if s > right sum then right sums
9. return left sum + right sum
6
Closest-Pair Problem

• Given a set of n points

P = {p1 = (x1 , y1 ), . . . , pn = (xn , yn )}
• The closest-pair problem asks to find the pair of points pi
and pj that are closest to each other i.e., d(pi , pj ) is
minimum, where
q
d(pi , pj ) = (xi − xj )2 + (yi − yj )2

• A brute force algorithm will find the distance between all

pairs of points and return the minimum distance
• This algorithm has a time complexity of O(n2 ) as there are
n(n − 1)/2 different pairs of points
• We will see a divide and conquer algorithm that solves this
problem in O(n log n) time
7
Closest-Pair Problem: Divide and Conquer

• The first step is to sort the points according to their

x-coordinates
• We divide the points into two almost equal parts PL and
PR and find the closest pair in the left and right parts
recursively
• Let l be the vertical line that separates PL and PR
• Let δL and δR be the minimum distance between two
points in PL and PR , respectively
• Let δ ′ be the minimum distance between a point in PL and
a point in PR
• The closest pair is either in PL or PR or it crosses the line
l, therefore the closest pair is the minimum of δL , δR and δ ′

8
Closest-Pair Problem: Divide and Conquer

• We set δ = min{δL , δR }
• If there is a pair of points pi ∈ PL and
pj ∈ PR such that d(pi , pj ) < δ then this
is the closest pair
δ δ

δ
• If the closest pair consists of some point
in PL and PR then we need to check
only those points that are within δ
distance from the line l (both sides)
• We can now run a linear scan to find
the closest pair in this region from top
to bottom (see the figure)

9
Closest-Pair Problem: Divide and Conquer

• Let T (n) be the time complexity of the algorithm

• We have the following recurrence:
n
T (n) = 2T + O(n) = O(n log n)
2

10
Fast Fourier Transform
Multiplying polynomials

• Let A(x) = a0 + a1 x + · · · + an−1 xn−1 and

B(x) = b0 + b1 x + · · · + bn−1 xn−1 be two n − 1 degree
polynomials
• Then the product: C(x) = A(x) · B(x) is a 2n − n degree
polynomial: defined as

C(x) = c0 + c1 x + · · · + c2n−2 x2n−2

• The polynomial C(x) can be computed in O(n2 ) time

• For example:

(1 + 2x + 3x2 ) · (2 + x + 4x2 ) = 2 + 5x + 12x2 + 11x3 + 12x4

11
Representating Polynomials

400

60
10
200

0 40
0

−10
20
−200

−20

0 −400

−30
−6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6

y = 4x − 7 y = 2x2 − 3x − 2 y = 3x3 − 2x2 + x − 5

• We need n + 1 distinct points to determine any n degree

polynomial
• We can specify any n degree polynomial
A(x) = a0 + a1 x + · · · + an xn by either one of the following:
• Its coefficients: a0 , a1 , . . . , an
• The values: A(x0 ), A(x1 ), . . . , A(xn )

12
Polynomial Multiplication

• For two n degree polynomials A(x) and B(x) represented

using their coefficients a0 , . . . an and b0 , . . . bn
• The polynomial C(x) = A(x) · B(x) can easily be computed
using A(x) and B(x) as following:

13
Polynomial Multiplication

• For two n degree polynomials A(x) and B(x) represented

using their coefficients a0 , . . . an and b0 , . . . bn
• The polynomial C(x) = A(x) · B(x) can easily be computed
using A(x) and B(x) as following:

Algorithm: PolynomialMultiplication (fft)

Input: Coefficients of A(x) and B(x)
Output: The product C(x) = A(x) · B(x)

1. Pick some k ≥ 2n + 1 points x0 , . . . , xk−1

2. Compute A(x0 ), . . . , A(xk−1 ) and B(x0 ), . . . , B(xk−1 )
3. Compute C(xj ) = A(xj ) · B(xj ) for all j = 0, . . . k − 1
4. Recover C(x) = c0 + c1 x + · · · + c2n x2n
13
Polynomial Multiplication

• How to pick these points? randomly?

• One idea is to use positive-negative pairs i.e,

±x0 , ±x1 , . . . ± xn/2−1

This will allow us to reuse some parts of computation e.g.,

A(xi ) and A(−xi ) will be same for all even powers of xi
• For example: let A(x) be

3+4x+6x2 +2x3 +x4 +10x5 = (3+6x2 +x4 )+x(4+2x2 +10x4 )

• We see that:
A(x) = Ae (x2 ) + xAo (x2 )
where Ae (·) and Ao (·) are polynomials of degree ≤ n/2 − 1
of even- and odd-numbered coefficients, respectively
14
Polynomial Multiplication

• Assuming n is even and the ± pairs of points ±xi

A(xi ) = Ae (x2i ) + xi Ao (x2i )

A(−xi ) = Ae (x2i ) − xi Ao (x2i )

• So, evaluating A(x) at n paired points ±x0 , . . . , ±xn/2−1

reduces to evaluating Ae (x) and Ao (x) at just n/2 points
x20 , . . . , x2n/2−1
• Therefore, applying it recursively, the time complexity is
now as T (n) = 2T (n/2) + O(n) = O(n log n)

15
Polynomial Multiplication

• Assuming n is even and the ± pairs of points ±xi

A(xi ) = Ae (x2i ) + xi Ao (x2i )

A(−xi ) = Ae (x2i ) − xi Ao (x2i )

• So, evaluating A(x) at n paired points ±x0 , . . . , ±xn/2−1

reduces to evaluating Ae (x) and Ao (x) at just n/2 points
x20 , . . . , x2n/2−1
• Therefore, applying it recursively, the time complexity is
now as T (n) = 2T (n/2) + O(n) = O(n log n)
• Unfortunately, this only works for the top-level recursion,
we need n/2 evaluation points x20 , . . . , x2n/2−1 to be
themselves plus-minus pairs, which is impossible, unless

15
Polynomial Multiplication

• Assuming n is even and the ± pairs of points ±xi

A(xi ) = Ae (x2i ) + xi Ao (x2i )

A(−xi ) = Ae (x2i ) − xi Ao (x2i )

• So, evaluating A(x) at n paired points ±x0 , . . . , ±xn/2−1

reduces to evaluating Ae (x) and Ao (x) at just n/2 points
x20 , . . . , x2n/2−1
• Therefore, applying it recursively, the time complexity is
now as T (n) = 2T (n/2) + O(n) = O(n log n)
• Unfortunately, this only works for the top-level recursion,
we need n/2 evaluation points x20 , . . . , x2n/2−1 to be
themselves plus-minus pairs, which is impossible, unless
• We use complex numbers
15
Complex Numbers and Roots of Unity

• Let z = a + bi a complex number

• Let x8 = 1 gives us 8 roots of unity, namely w0,1 , . . . , ω7,8
(w0,8 = 1)

ω2,8
ω3,8 ω1,8
Imaginary
ω4,8 θ
b z = a + bi ω0,8

θ ω5,8 ω7,8
Real
a ω6,8

• For xn = 1 we get w0,n = 1, ω1,n , ω2,n , . . . , ωn−1,n

• Here ωi,n = e2πi/n for i = 0, . . . , n − 1

16
Fast Fourier Transform

• Let n = 2k for some k > 0

• We represent:

A(x) = Ae (x2 ) + xAo (x2 )

B(x) = Be (x2 ) + xBo (x2 )

• We compute Ae (x2 ), Ao (x2 ), Be (x2 ) and Bo (x2 ) at:

(ωj,2n )2 for j = 0, . . . , 2n − 1

• Note that: (ωj,2n )2 = e2πji/n = e2πjli/n where l = j mod n.

17
Fast Fourier Transform

• Let L(n) be the number of operation of multiplication and

addition required to evaluate a polynomial A of degree
n − 1 on (2n)-th roots of unity (Steps 1. and 2. of fft
algorithm)
• Then: L(n) = 2L(n/2) + 4n = O(n log n) for some n = 2k
• Let us consider the Step 3. of fft algorithm to recover the
coefficients of C(x), we know:
2n−1
X
C(x) = cm xm
m=0

• From values:

d0 = C(ω0,2n ), . . . , d2n−1 = C(ω2n−1,2n )

18
Fast Fourier Transform

• Let use define a new polynomial D(x) as:

n−1
X
D(x) = dm xm
m=0

• The value of D(x) at x = ωj,2n is:

n−1
X n−1
X
D(ωj,2n ) = dm (ωj,2n )m = C(ωj,2n )(ωj,2n )m
m=0 m=0
n−1 2n−1
!
X X
l
= cl (ωm,2n ) (ωj,2n )m
m=0 l=0
2n−1 n−1
!
X X
= cl (ωj,2n )m (ωm,2n )l
l=0 m=0

19
Fast Fourier Transform

2
• We know ωj,2n = e2πji/2n
• Therefore,
2n−1 2n−1
!
X X
2πi(lm+jl)/2n
D(ωj,2n ) = cl e
l=0 m=0
2n−1 2n−1
!
X X
m
= cl (ωl+j,2n )
l=0 m=0

• Let us consider the sum:


2n−1 2n if l + j = 0
X mod 2n
(ωl+j,2n )m =
0 otherwise
m=0

• Therefore, D(ωj,2n ) = 2nc2n−j and c2n−j = D(ωj,2n )/2n

20
Fast Fourier Transform

• So, if we evaluate D and (2n)-th roots of unity:

ω2n,2n = ω0 , ω2n−1.2m , . . . , ω1,2n

• We get: c0 2n, c1 2n, . . . , c2n−1 2n

• These evaluations can be computed in O(n log n) operations
of multiplications and additions using divide-and-conquer
approach developed for the Step 1. of fft
• Thus, fft requires O(n log n) operations

21
Fast Fourier Transform

Algorithm: recursive-fft
Input: Coefficients of A(x) = ⟨a0 , a1 , . . . , an−1 ⟩
Output: The discrete FFT of of A(x)

1. if n = 1 then return a0
2. ωn = e2πi/n , ω = 1
3. Ae = ⟨a0 , a2 , . . . , an−2 ⟩, Ao = ⟨a1 , a3 , . . . , an−1 ⟩
4. Ye = recursive-fft(Ae ), Yo = recursive-fft(Ao )
5. for k = 0 to n/2 − 1
6. yk = Ye [k] + ωYo [k]
7. yk+n/2 = Ye [k] − ωYo [k]
8. ω = ω · ωn
9. return Y = ⟨y0 , y1 , . . . , yn−1 ⟩
22
Fast Fourier Transform

···
    
y0 1 1 1 1 a0

 y1  
  1 ω1,n ω2,n ··· ωn−1,n 
 a1 

y2 1 ω2,n ω4,n ··· ω2(n−1),n a2
    
 =  
.. .. .. .. .. .. ..
    
    
 .   . . . . .  . 
yn−1 1 ωn−1,n ω2(n−1),n ··· ω(n−1)(n−1),n an−1

23