2004 Goodrich, Tamassia

Divide-and-Conquer 1
Divide-and-Conquer
7 2 | 9 4 2 4 7 9
7 | 2 2 7 9 | 4 4 9
7 7 2 2 9 9 4 4
2004 Goodrich, Tamassia
Divide-and-Conquer 2
Divide-and-Conquer
Divide-and conquer is a
general algorithm design
paradigm:
Divide: divide the input data S in
two or more disjoint subsets S
1
,
S
2
,
Recur: solve the subproblems
recursively
Conquer: combine the solutions
for S
1
, S
2
, , into a solution for S
The base case for the
recursion are subproblems of
constant size
Analysis can be done using
recurrence equations
2004 Goodrich, Tamassia
Divide-and-Conquer 3
Merge-Sort Review
Merge-sort on an input
sequence S with n
elements consists of
three steps:
Divide: partition S into
two sequences S
1
and S
2

of about n/2 elements
each
Recur: recursively sort S
1

and S
2
Conquer: merge S
1
and
S
2
into a unique sorted
sequence
Algorithm mergeSort(S, C)
Input sequence S with n
elements, comparator C
Output sequence S sorted
according to C
if S.size() > 1
(S
1
, S
2
) partition(S, n/2)
mergeSort(S
1
, C)
mergeSort(S
2
, C)
S merge(S
1
, S
2
)
2004 Goodrich, Tamassia
Divide-and-Conquer 4
Recurrence Equation
Analysis
The conquer step of merge-sort consists of merging two sorted
sequences, each with n/2 elements and implemented by means of
a doubly linked list, takes at most bn steps, for some constant b.
Likewise, the basis case (n < 2) will take at b most steps.
Therefore, if we let T(n) denote the running time of merge-sort:





We can therefore analyze the running time of merge-sort by
finding a closed form solution to the above equation.
That is, a solution that has T(n) only on the left-hand side.

> +
<
=
2 if ) 2 / ( 2
2 if
) (
n bn n T
n b
n T
2004 Goodrich, Tamassia
Divide-and-Conquer 5
Iterative Substitution
In the iterative substitution, or plug-and-chug, technique, we
iteratively apply the recurrence equation to itself and see if we can
find a pattern:








Note that base, T(n)=b, case occurs when 2
i
=n. That is, i = log n.
So,

Thus, T(n) is O(n log n).
ibn n T
bn n T
bn n T
bn n T
bn n b n T
bn n T n T
i i
+ =
=
+ =
+ =
+ =
+ + =
+ =
) 2 / ( 2
...
4 ) 2 / ( 2
3 ) 2 / ( 2
2 ) 2 / ( 2
)) 2 / ( )) 2 / ( 2 ( 2
) 2 / ( 2 ) (
4 4
3 3
2 2
2
n bn bn n T log ) ( + =
2004 Goodrich, Tamassia
Divide-and-Conquer 6
The Recursion Tree
Draw the recursion tree for the recurrence relation and look for a
pattern:
depth Ts size
0 1 n
1 2 n/2

i 2
i
n/2
i

> +
<
=
2 if ) 2 / ( 2
2 if
) (
n bn n T
n b
n T
time
bn
bn
bn

Total time = bn + bn log n
(last level plus all previous levels)
2004 Goodrich, Tamassia
Divide-and-Conquer 7
Guess-and-Test Method
In the guess-and-test method, we guess a closed form solution
and then try to prove it is true by induction:


Guess: T(n) < cn log n.






Wrong: we cannot make this last line be less than cn log n








n bn cn n cn
n bn n cn
n bn n n c
n bn n T n T
log log
log ) 2 log (log
log )) 2 / log( ) 2 / ( ( 2
log ) 2 / ( 2 ) (
+ =
+ =
+ =
+ =

> +
<
=
2 if log ) 2 / ( 2
2 if
) (
n n bn n T
n b
n T
2004 Goodrich, Tamassia
Divide-and-Conquer 8
Guess-and-Test Method,
Part 2
Recall the recurrence equation:


Guess #2: T(n) < cn log
2
n.






if c > b.
So, T(n) is O(n log
2
n).
In general, to use this method, you need to have a good guess
and you need to be good at induction proofs.
n cn
n bn cn n cn n cn
n bn n cn
n bn n n c
n bn n T n T
2
2
2
2
log
log log 2 log
log ) 2 log (log
log )) 2 / ( log ) 2 / ( ( 2
log ) 2 / ( 2 ) (
s
+ + =
+ =
+ =
+ =

> +
<
=
2 if log ) 2 / ( 2
2 if
) (
n n bn n T
n b
n T
2004 Goodrich, Tamassia
Divide-and-Conquer 9
Master Method (Appendix)
Many divide-and-conquer recurrence equations have
the form:



The Master Theorem:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
2004 Goodrich, Tamassia
Divide-and-Conquer 10
Master Method, Example 1
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
n n T n T + = ) 2 / ( 4 ) (
Solution: log
b
a=2, so case 1 says T(n) is O(n
2
).
2004 Goodrich, Tamassia
Divide-and-Conquer 11
Master Method, Example 2
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
n n n T n T log ) 2 / ( 2 ) ( + =
Solution: log
b
a=1, so case 2 says T(n) is O(n log
2
n).
2004 Goodrich, Tamassia
Divide-and-Conquer 12
Master Method, Example 3
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
n n n T n T log ) 3 / ( ) ( + =
Solution: log
b
a=0, so case 3 says T(n) is O(n log

n).
2004 Goodrich, Tamassia
Divide-and-Conquer 13
Master Method, Example 4
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
2
) 2 / ( 8 ) ( n n T n T + =
Solution: log
b
a=3, so case 1 says T(n) is O(n
3
).
2004 Goodrich, Tamassia
Divide-and-Conquer 14
Master Method, Example 5
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
3
) 3 / ( 9 ) ( n n T n T + =
Solution: log
b
a=2, so case 3 says T(n) is O(n
3
).
2004 Goodrich, Tamassia
Divide-and-Conquer 15
Master Method, Example 6
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
1 ) 2 / ( ) ( + = n T n T
Solution: log
b
a=0, so case 2 says T(n) is O(log n).
(binary search)
2004 Goodrich, Tamassia
Divide-and-Conquer 16
Master Method, Example 7
The form:

The Master Theorem:




Example:

> +
<
=
d n n f b n aT
d n c
n T
if ) ( ) / (
if
) (
. 1 some for ) ( ) / ( provided
)), ( ( is ) ( then ), ( is ) ( if 3.
) log ( is ) ( then ), log ( is ) ( if 2.
) ( is ) ( then ), ( is ) ( if 1.
log
1 log log
log log
< s
O O
O O
O
+
+

o o
c
c
n f b n af
n f n T n n f
n n n T n n n f
n n T n O n f
a
k a k a
a a
b
b b
b b
n n T n T log ) 2 / ( 2 ) ( + =
Solution: log
b
a=1, so case 1 says T(n) is O(n).
(heap construction)
2004 Goodrich, Tamassia
Divide-and-Conquer 17
Iterative Proof of the
Master Theorem
Using iterative substitution, let us see if we can find a pattern:










We then distinguish the three cases as
The first term is dominant
Each part of the summation is equally dominant
The summation is a geometric series

=
+ =
+ =
=
+ + + =
+ + =
+ + =
+ =
1 ) (log
0
log
1 ) (log
0
log
2 2 3 3
2 2
2
) / ( ) 1 (
) / ( ) 1 (
. . .
) ( ) / ( ) / ( ) / (
) ( ) / ( ) / (
)) / ( )) / ( (
) ( ) / ( ) (
n
i
i i a
n
i
i i n
b
b
b
b
b n f a T n
b n f a T a
n f b n af b n f a b n T a
n f b n af b n T a
bn b n f b n aT a
n f b n aT n T
2004 Goodrich, Tamassia
Divide-and-Conquer 18
Integer Multiplication
Algorithm: Multiply two n-bit integers I and J.
Divide step: Split I and J into high-order and low-order bits



We can then define I*J by multiplying the parts and adding:



So, T(n) = 4T(n/2) + n, which implies T(n) is O(n
2
).
But that is no better than the algorithm we learned in grade
school.
l
n
h
l
n
h
J J J
I I I
+ =
+ =
2 /
2 /
2
2
l l
n
h l
n
l h
n
h h
l
n
h l
n
h
J I J I J I J I
J J I I J I
+ + + =
+ + =
2 / 2 /
2 / 2 /
2 2 2
) 2 ( * ) 2 ( *
2004 Goodrich, Tamassia
Divide-and-Conquer 19
An Improved Integer
Multiplication Algorithm
Algorithm: Multiply two n-bit integers I and J.
Divide step: Split I and J into high-order and low-order bits


Observe that there is a different way to multiply parts:





So, T(n) = 3T(n/2) + n, which implies T(n) is O(n
log
2
3
), by
the Master Theorem.
Thus, T(n) is O(n
1.585
).
l
n
h
l
n
h
J J J
I I I
+ =
+ =
2 /
2 /
2
2
l l
n
h l l h
n
h h
l l
n
l l h h h l h h l l l h
n
h h
l l
n
l l h h h l l h
n
h h
J I J I J I J I
J I J I J I J I J I J I J I J I
J I J I J I J J I I J I J I
+ + + =
+ + + + + =
+ + + + =
2 /
2 /
2 /
2 ) ( 2
2 ] ) [( 2
2 ] ) )( [( 2 *