CSE 417

Algorithms:
Divide and Conquer

Larry Ruzzo

Thanks to Richard Anderson, Paul Beame, Kevin Wayne for some slides
1

 algorithm design paradigms: divide and conquer
Outline:

General Idea

Review of Merge Sort

Why does it work?

Importance of balance

Importance of super-linear growth

Some interesting applications

Closest points

Integer Multiplication

Finding & Solving Recurrences

2

 algorithm design techniques
Divide & Conquer

Reduce problem to one or more sub-problems of
the same type

Typically, each sub-problem is at most a constant
fraction of the size of the original problem

Subproblems typically disjoint

Often gives significant, usually polynomial, speedup

Examples:

Binary Search, Mergesort, Quicksort (roughly),
Strassen’s Algorithm, integer multiplication, powering,
FFT, …

3

Motivating Example:
Mergesort

4

 merge sort


A U C
MS(A: array[1..n]) returns array[1..n] {

If(n=1) return A;

New U:array[1:n/2] = MS(A[1..n/2]);

New L:array[1:n/2] = MS(A[n/2+1..n]);

Return(Merge(U,L));

}



Merge(U,L: array[1..n]) {
L

New C: array[1..2n];

a=1; b=1;

split sort merge

For i = 1 to 2n


“C[i] = smaller of U[a], L[b] and correspondingly a++ or b++,


while being careful about running past end of either”;

Return C;

}

5

 divide & conquer – the key idea
Why does it work? Suppose we’ve already
invented DumbSort, taking time n2

Try Just One Level of divide & conquer:

DumbSort(first n/2 elements)

DumbSort(last n/2 elements)

Merge results

D&C in a

Time: 2 (n/2)2 +n= n2/2 +n≪ n2
nutshell

Almost twice as fast!

6

 d&c approach, cont.
Moral 1: “two halves are better than a whole”


Two problems of half size are better than one full-size
problem, even given O(n) overhead of recombining, since
the base algorithm has super-linear complexity.



Moral 2: “If a little's good, then more's better”


Two levels of D&C would be almost 4 times faster, 3 levels
almost 8, etc., even though overhead is growing. 
Best is usually full recursion down to some small constant
size (balancing "work" vs "overhead").

In the limit: you’ve just rediscovered mergesort!

7

 d&c approach, cont.
Moral 3: unbalanced division good, but less so:

(.1n)2 + (.9n)2 + n = .82n2 + n

The 18% savings compounds significantly if you carry recursion to
more levels, actually giving O(nlogn), but with a bigger constant.
So worth doing if you can’t get 50-50 split, but balanced is better
if you can.

This is intuitively why Quicksort with random splitter is good –
badly unbalanced splits are rare, and not instantly fatal.

Moral 4: but consistent, completely

unbalanced division doesn’t help much:

(1)2 + (n-1)2 + n = n2 - n + 2

Little improvement here.

8

 mergesort (review)
Mergesort: (recursively) sort 2 half-lists, then
merge results.



T(n) = 2T(n/2)+cn, n≥2

Log n levels

O(n)
T(1) = 0
work
per
Solution: Θ(n log n) level

(details later)

9

A Divide & Conquer Example:
Closest Pair of Points

10

 closest pair of points: non-geometric version
Given n points and arbitrary distances between them,
find the closest pair. (E.g., think of distance as airfare
– definitely not Euclidean distance!)






(… and all the rest of the (2

n) edges…)

Must look at all n choose 2 pairwise distances, else 
any one you didn’t check might be the shortest.

Also true for Euclidean distance in 1-2 dimensions?
11

 closest pair of points: 1 dimensional version
Given n points on the real line, find the closest pair







Closest pair is adjacent in ordered list

Time O(n log n) to sort, if needed

Plus O(n) to scan adjacent pairs

Key point: do not need to calc distances between all
pairs: exploit geometry + ordering

12

 closest pair of points: 2 dimensional version
Closest pair. Given n points in the plane, find a pair with smallest
Euclidean distance between them.



Fundamental geometric primitive.

Graphics, computer vision, geographic information systems, molecular
modeling, air traffic control.

Special case of nearest neighbor, Euclidean MST, Voronoi.


fast closest pair inspired fast algorithms for these problems



Brute force. Check all pairs of points p and q with Θ(n2) comparisons.



1-D version. O(n log n) easy if points are on a line.



Assumption. No two points have same x coordinate.

Just to simplify presentation

13

 closest pair of points. 2d, Euclidean distance: 1st try
Divide. Sub-divide region into 4 quadrants.

14

 closest pair of points: 1st try
Divide. Sub-divide region into 4 quadrants.

Obstacle. Impossible to ensure n/4 points in
each piece, so the “balanced subdivision”
goal may be elusive/problematic.

15

 closest pair of points
Algorithm.

Divide: draw vertical line L with ≈ n/2 points on each side.

16

 closest pair of points
Algorithm.

Divide: draw vertical line L with ≈ n/2 points on each side.

Conquer: find closest pair on each side, recursively.

21

12

17

 closest pair of points
Algorithm.

Divide: draw vertical line L with ≈ n/2 points on each side.

Conquer: find closest pair on each side, recursively.

Combine: find closest pair with one point in each side.

Return best of 3 solutions.
seems 
like 

Θ(n ) ?
2

5

21

12

18

 closest pair of points
Find closest pair with one point in each side,
assuming distance < δ.

21

δ = min(12, 21)

12

19

 closest pair of points
Find closest pair with one point in each side,
assuming distance < δ.

Observation: suffices to consider points within δ of line L.

21

δ = min(12, 21)

12

20

δ

 closest pair of points
Find closest pair with one point in each side,
assuming distance < δ.

Observation: suffices to consider points within δ of line L.

Almost the one-D problem again: Sort points in 2δ-strip by
their y coordinate.

L

7

6

5

4
21

δ = min(12, 21)

12
3

2

1
21

δ

 closest pair of points
Find closest pair with one point in each side,
assuming distance < δ.

Observation: suffices to consider points within δ of line L.

Almost the one-D problem again: Sort points in 2δ-strip by
their y coordinate. Only check pts within 8 in sorted list!

L

7

6

5

4
21

δ = min(12, 21)

12
3

2

1
22

δ

 closest pair of points
Def. Let si have the ith smallest
y-coordinate among points  39

j
in the 2δ-width-strip.

Claim. If |i – j| > 8, then the  31

distance between si and sj  30

is > δ.

δ/2

29

28

δ/2

Pf: No two points lie in the 
i 27

same δ/2-by-δ/2 square: 

" δ % " δ %
2 2
2
26

$ ' +$ ' = δ ≈ 0.7δ < δ 25

# 2& # 2& 2
δ
δ


so ≤ 8 points within +δ of y(si).
23

 closest pair algorithm
Closest-Pair(p1, …, pn) {
if(n <= ??) return ??

Compute separation line L such that half the points

are on one side and half on the other side.

δ1 = Closest-Pair(left half)
δ2 = Closest-Pair(right half)
δ = min(δ1, δ2)

Delete all points further than δ from separation line L

Sort remaining points p[1]…p[m] by y-coordinate.

for i = 1..m
k = 1
while i+k <= m && p[i+k].y < p[i].y + δ
δ = min(δ, distance between p[i] and p[i+k]);
k++;

return δ.
}
24

 closest pair of points: analysis
Analysis, I: Let D(n) be the number of pairwise distance
calculations in the Closest-Pair Algorithm when run on n ≥ 1
points




# 0 n = 1&
D(n) ≤ $2D n /2 + 7n n > 1' ⇒ D(n) = O(n log n)
% ( ) (




€
BUT – that’s only the number of distance calculations



What if we counted comparisons?

25

 closest pair of points: analysis
Analysis, II: Let C(n) be the number of comparisons between
coordinates/distances in the Closest-Pair Algorithm when run
on n ≥ 1 points


"$ 0 n = 1 &$ 2

C(n) ≤ #
2C ( n / 2 ) + kn log n n > 1
' ⇒ C(n) = O(n log n)
$% $(


for some constant k


Q. Can we achieve O(n log n)?



A. Yes. Don't sort points from scratch each time.

Sort by x at top level only.

Each recursive call returns δ and list of all points sorted by y

Sort by merging two pre-sorted lists.

T(n) ≤ 2T ( n /2) + O(n) ⇒ T(n) = O(n log n) 26

 is it worth the effort?
Code is longer & more complex

O(n log n) vs O(n2) may hide 10x in constant?



How many points?

Speedup:
n

n2 / (10 n log2 n)

10
0
.3

100
1
.5

1,000
10

10,000
75

100,000
602

1,000,000
5,017

10,000,000
43,004

27

Going From Code to Recurrence

28

 going from code to recurrence
Carefully define what you’re counting, and write it
down!

“Let C(n) be the number of comparisons between sort keys
used by MergeSort when sorting a list of length n ≥ 1”

In code, clearly separate base case from recursive case,
highlight recursive calls, and operations being counted.

Write Recurrence(s)

29

 merge sort

Base Case

MS(A: array[1..n]) returns array[1..n] {


If(n=1) return A;

New L:array[1:n/2] = MS(A[1..n/2]);
Recursive
New R:array[1:n/2] = MS(A[n/2+1..n]);
calls

Return(Merge(L,R));

}

One
Merge(A,B: array[1..n]) {
Recursive

New C: array[1..2n];

Level

a=1; b=1;

For i = 1 to 2n {
Operations


C[i] = “smaller of A[a], B[b] and a++ or b++”;

being

Return C;

}
counted

30

 the recurrence





Base case





#0 if n = 1
C(n) = $



%2C(n /2) + (n −1) if n > 1

One compare per

Recursive calls
element added to
merged list, except
Total time: proportional to C(n)
the last.

(loops, copying data, parameter passing, etc.)

31

 going from code to recurrence
Carefully define what you’re counting, and write it
down!

“Let D(n) be the number of pairwise distance calculations
in the Closest-Pair Algorithm when run on n ≥ 1 points”

In code, clearly separate base case from recursive case,
highlight recursive calls, and operations being counted.

Write Recurrence(s)

32

 Basic operations:
closest pair algorithm
distance calcs

Closest-Pair(p1, …, pn) {
if(n <= 1) return ∞ Base Case
0

Compute separation line L such that half the points
are on one side and half on the other side.

δ1 = Closest-Pair(left half)
Recursive calls (2)
2D(n / 2)

δ2 = Closest-Pair(right half)
δ = min(δ1, δ2)

Delete all points further than δ from separation line L

One 
recursive 
Sort remaining points p[1]…p[m] by y-coordinate.
level

for i = 1..m Basic operations at

this recursive level

k = 1
while i+k <= m && p[i+k].y < p[i].y + δ
7n

δ = min(δ, distance between p[i] and p[i+k]);
k++;

return δ.
}
33

 closest pair of points: analysis
Analysis, I: Let D(n) be the number of pairwise distance
calculations in the Closest-Pair Algorithm when run on n ≥ 1
points




# 0 n = 1&
D(n) ≤ $2D n /2 + 7n n > 1' ⇒ D(n) = O(n log n)
% ( ) (




€
BUT – that’s only the number of distance calculations



What if we counted comparisons?

34

 going from code to recurrence
Carefully define what you’re counting, and write it
down!

“Let D(n) be the number of comparisons between
coordinates/distances in the Closest-Pair Algorithm 
when run on n ≥ 1 points”

In code, clearly separate base case from recursive case,
highlight recursive calls, and operations being counted.

Write Recurrence(s)

35

 Basic operations:
Base Case
closest pair algorithm
comparisons

Closest-Pair(p1, …, pn) {
Recursive calls (2)

if(n <= 1) return ∞ 0

Compute separation line L such that half the points
k1n log n

are on one side and half on the other side.

δ1 = Closest-Pair(left half)
2C(n / 2)

δ2 = Closest-Pair(right half)
δ = min(δ1, δ2) 1

Delete all points further than δ from separation line L k2n

Sort remaining points p[1]…p[m] by y-coordinate. k3n log n

for i = 1..m Basic operations at

this recursive level

k = 1
while i+k <= m && p[i+k].y < p[i].y + δ 8n

δ = min(δ, distance between p[i] and p[i+k]); 7n

k++;
One 
return δ.
recursive 
}
36
level

 closest pair of points: analysis
Analysis, II: Let C(n) be the number of comparisons of
coordinates/distances in the Closest-Pair Algorithm 
when run on n ≥ 1 points


"
$ 0 n = 1 &$ 2

C(n) ≤ # 2C n / 2 + k n logn +1 n > 1 ' ⇒ C(n) = O(n log n)
$% ( ) 4 $(


for some k 4 ≤ k1 + k 2 + k 3 +15


Q. Can we achieve time O(n log n)?



A. Yes. Don't sort points from scratch each time.

Sort by x at top level only.

Each recursive call returns δ and list of all points sorted by y

Sort by merging two pre-sorted lists.

T(n) ≤ 2T ( n /2) + O(n) ⇒ T(n) = O(n log n) 37

Integer Multiplication

38

 integer arithmetic
Add. Given two n-bit  1
1
1
1
1
1
0
1

integers a and b,  1

1
0
1
0
1
0
1

compute a + b.
Add
+
0
1
1
1
1
1
0
1



1
0
1
0
1
0
0
1
0

O(n) bit operations.

1
1
0
1
0
1
0
1


*
0
1
1
1
1
1
0
1

Multiply. Given two n-digit  1

1
0
1
0
1
0
1

0
0
0
0
0
0
0
0

Multiply

integers a and b,  1
1
0
1
0
1
0
1

compute a × b. 1
1
0
1
0
1
0
1

1
1
0
1
0
1
0
1

The “grade school” method:
1
1
0
1
0
1
0
1

1
1
0
1
0
1
0
1

Θ(n2) bit operations.
0
0
0
0
0
0
0
0

0
1
1
0
1
0
0
0
0
0
0
0
0
039

0


1

 integer arithmetic
Add. Given two n-bit  1
1
1
1
1
1

Carries

0
1

integers a and b,  1

1
0
1
0
1
0
1

compute a + b.
Add
+
0
1
1
1
1
1
0
1



1
0
1
0
1
0
0
1
0

O(n) bit operations.

1
1
0
1
0
1
0
1


*
0
1
1
1
1
1
0
1

Multiply. Given two n-bit  1

1
0
1
0
1
0
1

0
0
0
0
0
0
0
0

Multiply

integers a and b,  1
1
0
1
0
1
0
1

compute a × b. 1
1
0
1
0
1
0
1

1
1
0
1
0
1
0
1

The “grade school” method:
1
1
0
1
0
1
0
1

1
1
0
1
0
1
0
1

Θ(n2) bit operations.
0
0
0
0
0
0
0
0

0
1
1
0
1
0
0
0
0
0
0
0
0
040

0


1

 divide & conquer multiplication: warmup
To multiply two 2-digit integers:

Multiply four 1-digit integers.

Add, shift some 2-digit integers to obtain result.


x = 10⋅ x1 + x 0
4 5
y1 y0


y = 10⋅ y1 + y 0
3

2
x1 x0

xy = (10⋅ x1 + x 0 ) (10⋅ y1 + y 0 )

= 100 ⋅ x y + 10⋅ x y + x y + x y 0
x0⋅y0

1 1 ( 1 0 0 1) 0 0 1

0 8

x0⋅y1

Same idea works for long integers –

1 5

x1⋅y0

1 2
€ can split them into 4 half-sized ints

x1⋅y1

1 4 4 0

(“10” becomes “10 ”, k = length/2)

k

41

 divide & conquer multiplication: warmup
To multiply two n-bit integers:

Multiply four ½n-bit integers.

Shift/add four n-bit integers to obtain result.



x = 2 n / 2 ⋅ x1 +


y = 2 n / 2 ⋅ y1 +
x0
y0
1
1
0
1
0
1
0
1
y1 y0

*
0
1
1
1
1
1
0
1
x1 x0

xy
= (2 n / 2 ⋅ x1 + x 0 ) (2 n / 2 ⋅ y1 + y 0 ) 0
1
0
0
0
0
0
1
x0⋅y0

= 2 n ⋅ x1 y1 + 2 n / 2 ⋅ ( x1 y 0 + x 0 y1 ) + x 0 y 0

1
0
1
0
1
0
0
1
x0⋅y1

0
0
1
0
0
0
1
1


0
1
0
1
1
0
1
1

x1⋅y0

T(n) = 4T (n /2 ) + Θ(n) ⇒ T(n) = Θ(n 2 ) x1⋅y1



 
recursive calls add, shift 0
1
1
0
1
0
0
0
0
0
0
0
0
0
0
1

€ assumes n is a power of 2

42

 key trick: 2 multiplies for the price of 1:

x = 2 n / 2 ⋅ x1 + x0
y = 2 n / 2 ⋅ y1 + y0 Well, ok, 4 for 3 is
xy = (2 n / 2 ⋅ x1 + x 0 ) (2 n / 2 ⋅ y1 + y 0 ) more accurate…

= 2 n ⋅ x1 y1 + 2 n / 2 ⋅ ( x1 y 0 + x 0 y1 ) + x 0 y 0

α = x1 + x 0
β = y1 + y 0
αβ = ( x1 + x 0 ) ( y1 + y 0 )
= x1 y1 + ( x1 y 0 + x 0 y1 ) + x 0 y 0
( x1 y 0 + x 0 y1 ) = αβ − x1 y1 − x 0 y 0
43

€
 Karatsuba multiplication
To multiply two n-bit integers:

Add two pairs of ½n bit integers.

Multiply three pairs of ½n-bit integers.

Add, subtract, and shift n-bit integers to obtain result.




x = 2 n / 2 ⋅ x1
+ x0
y = 2 n / 2 ⋅ y1
+ y0

xy = 2 n ⋅ x1 y1
+ 2 n / 2 ⋅ ( x1 y0 + x0 y1 ) + x0 y0

= 2 n ⋅ x1 y1
+ 2 n / 2 ⋅ ( (x1 + x0 ) (y1 + y0 ) − x1 y1 − x0 y0 ) + x0 y0

A
B
A
C
C

Theorem.
€
[Karatsuba-Ofman, 1962] Can multiply two n-digit
integers in O(n1.585) bit operations.


T(n) ≤ T ( #n /2$ ) + T ( %n /2& ) + T ( 1+ %n /2& ) + Θ(n)
 
recursive calls add, subtract, shift

Sloppy version : T(n) ≤ 3T(n /2) + O(n)

log 2 3
⇒ T(n) = O(n ) = O(n1.585 )
44

€
 Karatsuba multiplication
Theorem. [Karatsuba-Ofman, 1962] Can multiply two n-digit
integers in O(n1.585) bit operations.



T(n) ≤ T ( #n /2$ ) + T ( %n /2& ) + T ( 1+ %n /2& ) + Θ(n)

 
add, subtract, shift
recursive calls

Sloppy version : T(n) ≤ 3T(n /2) + O(n)
log 2 3
Best to solve it directly )(but messy). Instead, it nearly always
1.585
⇒ T(n) = O(n = O(n )

suffices toT(n)
solve
≤ T ( a#nsimpler
/2$ ) + T ( %recurrence:


n /2& ) + T ( 1+ %n /2& ) + Θ(n)


€ recursive calls add, subtract, shift

Sloppy version : T(n) ≤ 3T(n /2) + O(n)

⇒ T(n) = O(n
log 2 3
) = O(n ) 1.585

T(n) ≤ T ( #n /2$ ) + T ( %n /2& ) + T ( 1+ %n /2& ) + Θ(n)


Intuition: If T(n) = n , then T(n+1) = n +kn +… = O(nk)

   k          
recursive calls
k   k-1
add, subtract, shift

€ Sloppy version : T(n) ≤ 3T(n /2) + O(n)
log 2 3 1.585
⇒ T(n) = O(n ) = O(n )


(Proof later.)

€ 45

 multiplication – the bottom line
Naïve:

Θ(n2)

Karatsuba:
Θ(n1.59…)

Amusing exercise: generalize Karatsuba to do 5 size 
n/3 subproblems → Θ(n1.46…)

Best known:
Θ(n log n loglog n)

"Fast Fourier Transform"

but mostly unused in practice (unless you need really big
numbers - a billion digits of π, say)

High precision arithmetic IS important for crypto

46

 Recurrences

Above: Where they come 

from, how to find them


Next: how to solve them

47

 mergesort (review)
Mergesort: (recursively) sort 2 half-lists, then
merge results.



T(n) = 2T(n/2)+cn, n≥2

Log n levels

O(n)
T(1) = 0
work
per
Solution: Θ(n log n) level

(details later)

now!

48

 Solve: T(1) = c
T(n) = 2 T(n/2) + cn

Level Num

Level

Num
SSize

ize
Work

Work

0

0 1 = 2
0

1=2n
0
n cn

1

cn
2 = 21
n/2
2cn/2

2

1
2 =2
4 = 2
2 1
nn/4

/2
2 c 4cn/4

n/2

…

2
4
…
=2 2
n…

/4
4 c n/4
…

i

… 2
i
…
…n/2i


…2
i c n/2i

…

i …

2
n…

/2i
2i c n/2
i i

…


… 2k-1
…
k-1



n/2
… k-1


…
2k-1
c n/2k-1

n = 2k ; k = log2n

k-1 2
k2k-1
k



n/2
n/2 k-1
k = 1

2k-12kcT(1)

n/2k-1



Total Work: c n (1+log2n)
(add last col)
49

 Solve: T(1) = c
T(n) = 4 T(n/2) + cn

Level Num

Level

Num
SSize

ize
Work
Work

0

0 1 =
14=4 0
0
n n

cn
cn

1

1 4 =
44=4 1
1

nn/2

/2
4 c 4cn/2

n/2

.
.
2

2 16
1=6=4 42
2
nn/4

/4
16 16cn/4

c n/4

.

… …

…

…
… …


…
…

.

.
i

i 4i
4i
nn/2
/2ii

4i c4i n/2 i
i

c n/2
.
.
.

.

… …

…

…
… …


…
…


k-1 4k-1
k-1

4

k-1
n/2
n/2k-1
k-1

44k-1 c n/2k-1


k-1 c n/2k-1

n = 2k ; k = log2n
k

k 4k
4

k n/2
n/2=k=1
k 1

4k T(1)

4k T(1)


k
4k = (22)k=
Total Work: T(n) =
∑i =0
i i 2
4 cn / 2 = O(n ) 50

(2k)2 = n2

 Solve: T(1) = c
T(n) = 3 T(n/2) + cn

LevelNum

Level

Num
Size
Size

WWork

ork

0

0 1 =
310
=30
n n

cn
cn

1

1 3 =
331
=31 n/2


n/2
3 3cn/2

c n/2

2

2 9 =
392
=32 n/4


n/4
9 9cn/4

c n/4

…

… …

…
…

…

……



i

i 3i

3i
n/2
n/2 i
i

33i icc n/2
n/2ii


.

.
.
.
.
.
…

… …

…
…

…

……



.
.
.
k-1

k-1 3k-1


3k-1 n/2

nk-1
/2
k-1
33k-1k-1c cn/2 n/2
k-1
k-1

n = 2k ; k = log2n
k

k 3k

3k n/2
kn= /21
k=1
3k3kT(1)

T(1)


i i
k
∑i =0 3 cn / 2
Total Work: T(n) =
51

 a useful identity
Theorem: for x ≠ 1,

1 + x + x2 + x3 + … + xk = (xk+1-1)/(x-1)

proof:

y
= 1 + x + x2 + x3 + … + xk

xy
= x + x2 + x3 + … + xk + xk+1

xy-y
= xk+1 - 1

y(x-1)
= xk+1 - 1

y
= (xk+1-1)/(x-1)

52



Solve: T(1) = c
T(n) = 3 T(n/2) + cn (cont.)

k
T ( n ) = ∑i=0 3 cn / 2
i i

k
= cn∑ i
3 /2 i
i=0
k i
= cn∑ 3
( )
i=0 2
k+1

= cn
( )3
2 −1
(2)
3 −1

53

Solve: T(1) = c
T(n) = 3 T(n/2) + cn (cont.)

k+1

cn
( 3
2) −1
= 2cn (( 2 ) −1)
3
k+1

(2)
3 −1

k+1
< 2cn ( 23 )
k
= 3cn ( 2 )
3

k
3
= 3cn k
2 54

Solve: T(1) = c
T(n) = 3 T(n/2) + cn (cont.)

k log 2 n
3 3
3cn k = 3cn log2 n
2 2
3 log 2 n
a log b n
= 3cn
n log b a log b n
= (b )
= 3c3log2 n
log b n log b a
= 3c ( n log2 3 ) = (b )
= O ( n1.585... ) = n log b a
55

 divide and conquer – master recurrence
T(n) = aT(n/b)+cnk for n > b then



a > bk ⇒ T(n) = Θ(nlog
b a )
[many subprobs → leaves dominate]



a < bk ⇒ T(n) = Θ(nk)

[few subprobs → top level dominates]



a = bk ⇒ T(n) = Θ (nk log n)
[balanced → all log n levels contribute]



Fine print: 
T(1) = d; a ≥ 1; b > 1; c, d, k ≥ 0; n = bt for some t > 0; 
a, b, k, t integers. True even if it is ⎡n/b⎤ instead of n/b.

56

 master recurrence: proof sketch
Expand recurrence as in earlier examples, to get


T(n) = nh ( d + c S ) 


logb n
where  h = logb(a) (and nh = number of tree leaves) and S = ∑ x j,
j=1
where x = bk/a.

If c = 0 the sum S is irrelevant, and T(n) = O(nh): all work happens in the
base cases, of which there are nh, one for each leaf in the recursion tree.

If c > 0, then the sum matters, and splits into 3 cases (like previous slide):  

if x < 1, then S < x/(1-x) = O(1).  
[S is the first log n terms of the

infinite series with that sum.]  

if x = 1, then S = logb(n) = O(log n).   
[All terms in the sum are 1 and

there are that many terms.]  

if x > 1, then S = x • (x1+logb(n)-1)/(x-1).
[And after some algebra, 

nh * S = O(nk).]

57

Another Example:
Exponentiation

58

 another d&c example: fast exponentiation
Power(a,n)

Input: integer n and number a

Output: an

Obvious algorithm

n-1 multiplications

Observation:

if n is even, n = 2m, then an = am• am

59

 divide & conquer algorithm
Power(a,n)


if n = 0 then return(1)


if n = 1 then return(a)
x ← Power(a,⎣n/2⎦)
x ← x•x


if n is odd then



x ← a•x


return(x)

60

 analysis
Let M(n) be number of multiplies

Worst-case  &( 0 n ≤1
M (n) = '
recurrence:
() M ("#n / 2$%) + 2 n > 1
By master theorem

M(n) = O(log n)
(a=1, b=2, k=0)

More precise analysis:

M(n) = ⎣log2n⎦ + (# of 1’s in n’s binary representation) - 1

Time is O(M(n)) if numbers < word size, else also
depends on length, multiply algorithm

61

 a practical application - RSA
Instead of an want an mod N

ai+j mod N = ((ai mod N) • (aj mod N)) mod N

same algorithm applies with each x • y replaced by

((x mod N) • (y mod N)) mod N



In RSA cryptosystem (widely used for security)

need an mod N where a, n, N each typically have 1024 bits

Power: at most 2048 multiplies of 1024 bit numbers

relatively easy for modern machines

Naive algorithm: 21024 multiplies

62

 d & c summary
Idea:

“Two halves are better than a whole”

if the base algorithm has super-linear complexity.

“If a little's good, then more's better”

repeat above, recursively

Analysis: recursion tree or Master Recurrence

Applications: Many.

Binary Search, Merge Sort, (Quicksort), Closest
Points, Integer Multiply, Exponentiation,…

63