CS 341: Algorithms

Lec 03: Divide and Conquer

Armin Jamshidpey Collin Roberts

Based on lecture notes by Éric Schost and many previous CS 341 instructors

David R. Cheriton School of Computer Science, University of Waterloo

Winter 2025

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 1 / 19
Divide-and-Conquer

A general algorithmic paradigm (strategy):

Divide: Split a problem into several subproblems.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 2 / 19
Divide-and-Conquer

A general algorithmic paradigm (strategy):

Divide: Split a problem into several subproblems.

Conquer: Solve the subproblems (recursively) applying the

same algorithm.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 2 / 19
Divide-and-Conquer

A general algorithmic paradigm (strategy):

Divide: Split a problem into several subproblems.

Conquer: Solve the subproblems (recursively) applying the

same algorithm.

Combine: Use subproblem results to derive a final result

for the original problem.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 2 / 19
Divide-and-Conquer

A general algorithmic paradigm (strategy):

Divide: Split a problem into several subproblems.

Conquer: Solve the subproblems (recursively) applying the

same algorithm.

Combine: Use subproblem results to derive a final result

for the original problem.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 2 / 19
When can we use Divide and Conquer?

Original problem is easily decomposable into subproblems

(we do not want to see “overlap” in the subproblems).

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 3 / 19
When can we use Divide and Conquer?

Original problem is easily decomposable into subproblems

(we do not want to see “overlap” in the subproblems).

Combining solutions is not too costly.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 3 / 19
When can we use Divide and Conquer?

Original problem is easily decomposable into subproblems

(we do not want to see “overlap” in the subproblems).

Combining solutions is not too costly.

Subproblems are not overly unbalanced.

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 3 / 19
Counting inversions
Collaborative filtering:
matches users preference (movies, music, ...)
determine users with similar tastes
recommends new things to users based on preferences of
similar users

Padlet
The basis of collaborative filtering is ...
https://padlet.com/arminjamshidpey/CS341

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 4 / 19
Counting inversions

Goal: given an unsorted array A[1..n], find the number of

inversions in it.
Def: (i, j) is an inversion if i < j and A[i] > A[j]

Example: with A = [1, 5, 2, 6, 3, 8, 7, 4], we get

(2, 3), (2, 5), (2, 8), (4, 5), (4, 8), (6, 7), (6, 8), (7, 8)

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 5 / 19
Counting inversions

Goal: given an unsorted array A[1..n], find the number of

inversions in it.
Def: (i, j) is an inversion if i < j and A[i] > A[j]

Example: with A = [1, 5, 2, 6, 3, 8, 7, 4], we get

(2, 3), (2, 5), (2, 8), (4, 5), (4, 8), (6, 7), (6, 8), (7, 8)

Remark: we show the indices where inversions occur

Remark: easy algorithm (two nested loops) in Θ(n2 )

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 5 / 19
Toward a divide-and-conquer algorithm

Idea (for n a power of two)

cℓ = number of inversions in A[1..n/2]
cr = number of inversions in A[n/2 + 1..n]
ct = number of transverse inversions with i ≤ n/2 and
j > n/2
return cℓ + cr + ct

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 6 / 19
Toward a divide-and-conquer algorithm

Idea (for n a power of two)

cℓ = number of inversions in A[1..n/2]
cr = number of inversions in A[n/2 + 1..n]
ct = number of transverse inversions with i ≤ n/2 and
j > n/2
return cℓ + cr + ct

Example: with A = [1, 5, 2, 6, 3, 8, 7, 4]

cℓ = 1 (2, 3)
cr = 3 (6, 7), (6, 8), (7, 8)
ct = 4 (2, 5), (2, 8), (4, 5), (4, 8)

cℓ and cr done recursively. What about ct ?

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 6 / 19
Transverse inversions

Goal: how many pairs (i, j) with i ≤ n/2, j > n/2, A[i] > A[j]?

Remark: this number does not change if both sides are sorted
So assume that we sort left and right after the recursive calls.

Example: starting from [1, 5, 2, 6, 3, 8, 7, 4], we get

[1, 2, 5, 6, 3, 4, 7, 8]

ct = #i’s greater than 3 + #i’s greater than 4 +

#i’s greater than 7 + #i’s greater than 8

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 7 / 19
Option 1
Algorithm: take each i ≤ n/2 and binary-search its position in
the right-hand side.
this is O(log(n)) per i, so total O(n log(n))
plus another O(n log(n)) for sorting left and right
recurrence: T (n) ≤ 2T (n/2) + O(n log(n))
gives T (n) = O(n log2 (n))

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 8 / 19
Option 1
Algorithm: take each i ≤ n/2 and binary-search its position in
the right-hand side.
this is O(log(n)) per i, so total O(n log(n))
plus another O(n log(n)) for sorting left and right
recurrence: T (n) ≤ 2T (n/2) + O(n log(n))
gives T (n) = O(n log2 (n))

Sketchy proof:
T (n) ≤ 2T (n/2) + n log(n)
≤ 4T (n/4) + n log(n/2) + n log(n)
≤ 8T (n/8) + n log(n/4) + n log(n/2) + n log(n)
≤ · · · ≤ n(log(n) + log(n/2) + · · · + log(2))
≤ n log2 (n)

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 8 / 19
Option 2: enhance mergesort

Idea: find ct during merge.

Merge(A[1..n]) (both halves of A assumed sorted)

1. copy A into a new array S
2. i = 1; j = n/2 + 1;
3. for (k ← 1; k ≤ n; k++) do
4. if (i > n/2) A[k] ← S[j++]
5. else if (j > n) A[k] ← S[i++]
6. else if (S[i] < S[j]) A[k] ← S[i++]
7. else A[k] ← S[j++]

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 9 / 19
 Merge(A[1..n]) (both halves of A assumed sorted)
1. copy A into a new array S; c = 0
2. i = 1; j = n/2 + 1;
3. for (k ← 1; k ≤ n; k++) do
4. if (i > n/2) A[k] ← S[j++]
5. else if (j > n) A[k] ← S[i++]; c = c + n/2
6. else if (S[i] < S[j]) A[k] ← S[i++]; c = c + j − (n/2 + 1)
7. else A[k] ← S[j++]

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 10 / 19
 Merge(A[1..n]) (both halves of A assumed sorted)
1. copy A into a new array S; c = 0
2. i = 1; j = n/2 + 1;
3. for (k ← 1; k ≤ n; k++) do
4. if (i > n/2) A[k] ← S[j++]
5. else if (j > n) A[k] ← S[i++]; c = c + n/2
6. else if (S[i] < S[j]) A[k] ← S[i++]; c = c + j − (n/2 + 1)
7. else A[k] ← S[j++]

Example: with [1, 2, 5, 6, 3, 4, 7, 8]

when we insert 1 back into A, j = 5 so c = c + 0
when we insert 2 back into A, j = 5 so c = c + 0
when we insert 5 back into A, j = 7 so c = c + 2
when we insert 6 back into A, j = 7 so c = c + 2
Enhanced merge is still Θ(n) so total remains Θ(n log(n)).

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 10 / 19
Multiplying polynomials

Goal: given F = f0 + · · · + fn−1 xn−1 and

G = g0 + · · · + gn−1 xn−1 , compute

H = F G = f0 g0 + (f0 g1 + f1 g0 )x + · · · + fn−1 gn−1 x2n−2

1. for i = 0, . . . , n − 1 do
2. for j = 0, . . . , n − 1 do
3. hi+j = hi+j + fi gj

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 11 / 19
Divide-and-conquer
Idea: write F = F0 + F1 xn/2 , G = G0 + G1 xn/2 . Then

H = F0 G0 + (F0 G1 + F1 G0 )xn/2 + F1 G1 xn

Analysis:
4 recursive calls in size n/2
Θ(n): additions to compute F0 G1 +F1 G0 and etc.

Recurrence: T (n) = 4T (n/2) + Θ(n)

a = 4, b = 2, y = 1 so T (n) = Θ(n2 )
Not better than the naive algorithm. We do the same
operations.
Padlet
Use only one multiplication to write F0 G1 + F1 G0 in terms of
F0 , F1 , G0 , G1 , F0 G0 , F1 G1 (assume F0 G0 , F1 G1 are given).
https://padlet.com/arminjamshidpey/CS341
A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 12 / 19
Karatsuba’s algorithm
Idea: use the identity

(F0 + F1 xn/2 )(G0 + G1 xn/2 ) =

F0 G0 + ((F0 + F1 )(G0 + G1 ) − F0 G0 − F1 G1 )xn/2 + F1 G1 xn

Analysis:
3 recursive calls in size n/2
Θ(n) additions to compute F0 + F1 and G0 + G1
multiplications by xn/2 and xn are free
Θ(n) additions and subtractions to combine the results

Recurrence: T (n) = 3T (n/2) + Θ(n)

a = 3, b = 2, c = 1 so T (n) = Θ(nlog2 (3) )
log2 (3) = 1.58 . . .

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 13 / 19
Karatsuba’s algorithm
Idea: use the identity

(F0 + F1 xn/2 )(G0 + G1 xn/2 ) =

F0 G0 + ((F0 + F1 )(G0 + G1 ) − F0 G0 − F1 G1 )xn/2 + F1 G1 xn

Analysis:
3 recursive calls in size n/2
Θ(n) additions to compute F0 + F1 and G0 + G1
multiplications by xn/2 and xn are free
Θ(n) additions and subtractions to combine the results

Recurrence: T (n) = 3T (n/2) + Θ(n)

a = 3, b = 2, c = 1 so T (n) = Θ(nlog2 (3) )
log2 (3) = 1.58 . . .

Remark: key idea = a formula for degree-1 polymomials that

does 3 multiplications
A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 13 / 19
Toom-Cook and FFT
Toom-Cook:
a family of algorithms based on similar expressions as
Karatsuba
for k ≥ 2, 2k − 1 recursive calls in size n/k
so T (n) = Θ(nlogk (2k−1) )
gets as close to exponent 1 as we want (but very slowly)

FFT:
if we use complex coefficients, FFT can be used to multiply
polynomials
FFT follows the same recurrence as merge sort,
T (n) = 2T (n/2) + Θ(n)
so we can multiply polynomials in Θ(n log(n)) ops over C

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 14 / 19
Multiplying matrices

Goal: given A = [ai,j ]1≤i,j≤n and B = [bj,k ]1≤j,k≤n compute

C = AB

Remark: input and output size Θ(n2 ), easy algorithm in Θ(n3 )

1. for i = 1, . . . , n do
2. for j = 1, . . . , n do
3. for k = 1, . . . , n do
4. ci,k = ci,k + ai,j bj,k

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 15 / 19
Divide-and-conquer
Setup: write
   
A1,1 A1,2 B1,1 B1,2
A= B=
A2,1 A2,2 B2,1 B2,2

with all Ai,k , Bi,j of size n/2 × n/2. Then

 
A1,1 B1,1 + A1,2 B2,1 A1,1 B1,2 + A1,2 B2,2
C=
A2,1 B1,1 + A2,2 B2,1 A2,1 B1,2 + A2,2 B2,2

Naively: 8 recursive calls in size n/2 + Θ(n2 ) additions =⇒

T (n) = Θ(n3 )

Padlet
Can we do better than 8 recursive calls?
https://padlet.com/arminjamshidpey/CS341

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 16 / 19
Strassen’s algorithm
Compute
Q1 = (A1,1 − A1,2 )B2,2
Q2 = (A2,1 − A2,2 )B1,1
C1,1 = Q1 − Q3 − Q5 + Q7
Q3 = A2,2 (B1,1 + B2,1 )
C1,2 = Q4 − Q1
Q4 = A1,1 (B1,2 + B2,2 ) and
C2,1 = Q2 + Q3
Q5 = (A1,1 + A2,2 )(B2,2 − B1,1 )
C2,2 = −Q2 − Q4 + Q5 + Q6
Q6 = (A1,1 + A2,1 )(B1,1 + B1,2 )
Q7 = (A1,2 + A2,2 )(B2,1 + B2,2 )

Analysis: 7 recursive calls in size n/2 + Θ(n2 ) additions =⇒

T (n) = Θ(nlog2 (7) )
log2 (7) = 2.80 . . .

Padlet
Can we multiply two 2 × 2 matrices with less than 7
multiplications?
https://padlet.com/arminjamshidpey/CS341

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 17 / 19
What this means

Direct generalization
an algorithm that does k multiplications for matrices of
size ℓ gives T (n) ∈ Θ(nlogℓ (k) )

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 18 / 19
What this means

Direct generalization
an algorithm that does k multiplications for matrices of
size ℓ gives T (n) ∈ Θ(nlogℓ (k) )
Going beyond
an algorithm that does k multiplications for matrices of
size ℓ, m by m, p gives T (n) ∈ Θ(n3 logℓmp (k) )

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 18 / 19
What this means

Direct generalization
an algorithm that does k multiplications for matrices of
size ℓ gives T (n) ∈ Θ(nlogℓ (k) )
Going beyond
an algorithm that does k multiplications for matrices of
size ℓ, m by m, p gives T (n) ∈ Θ(n3 logℓmp (k) )
Best exponent to date (using more than just
divide-and-conquer)
O(n2.37188 ), improves from previous record O(n2.37286 )
galactic algorithms

A. Jamshidpey, C. Roberts (UW) Lec 03: Divide and Conquer Winter 2025 18 / 19