Theory of Algorithms:

Divide and Conquer

Objectives
To introduce the divide-and-conquer mind
set
To show a variety of divide-and-conquer
solutions:
Merge Sort
Quick Sort
Strassens Matrix Multiplication
Convex Hull by Divide-and-Conquer
To discuss the strengths and weaknesses of a
divide-and-conquer strategy
Divide-and-Conquer
Best known algorithm design strategy:
1. Divide instance of problem into two or more
smaller instances
2. Solve smaller instances recursively
3. Obtain solution to original (larger) instance by
combining these solutions
Silly Example (Addition):
a
0
+ . + a
n-1
= (a
0
+ + a
n/2-1
) + (a
n/2
+ a
n-1
)
Divide-and-Conquer
Recurrence Templates apply
General divide and conquer recurrence:
T(n) = aT(n/b) + f (n), a ! 1, b ! 2
Efficiency analysis of d&c algorithms
simplified by Master Theorem:
If f(n) !(n
d
) where d>= 0 then:
T(n) !(n
d
) if a < b
d
T(n) !(n
d
log n) if a = b
d
T(n) !(n
log
b
a
) if a > b
d
Divide-and-Conquer
Illustrated
SUBPROBLEM 2
OF SIZE n/2
SUBPROBLEM 1
OF SIZE n/2
A SOLUTION TO
SUBPROBLEM 1
A SOLUTION TO THE
ORIGINAL PROBLEM
A SOLUTION TO
SUBPROBLEM 2
A PROBLEM OF
SIZE n
Perfect example:
Mergesort
Algorithm:
1. Split A[1..n] in half and put copy of each half into arrays B
[1.. n/2 ] and C[1.. n/2 ]
2. Recursively MergeSort arrays B and C
3. Merge sorted arrays B and C into array A
Merging:
REPEAT until no elements remain in one of B or C
1. Compare 1st elements in the rest of B and C
2. Copy smaller into A, incrementing index of corresponding
array
3. Once all elements in one of B or C are processed, copy
the remaining unprocessed elements from the other
array into A
7
Mergesort Example
7 2 1 6 4
7 2 1 6 4
7 2 1 6 4
7 2
2 7
1 2 7
4 6
1 2 4 6 7
Efficiency of Mergesort
Recurrence:
C(n) = 2 C(n/2) + C
merge
(n) for n > 1, C(1) = 0
C
merge
(n) = n - 1 in the worst case
All cases have same efficiency: "(n log n)
Number of comparisons is close to
theoretical minimum for comparison-based
sorting:
log n! ! n lg n - 1.44 n
Improving Efficiency of
Mergesort
Principal drawback is space requirement: "
( n ) (NOT in-place)
CAN be done in place, but with large
multiplicative constant
Can be implemented without
recursion (bottom-up)
Quicksort
Select a pivot (partitioning element)
Rearrange the list into two sublists:
All elements positioned before the pivot are " the pivot
Those positioned after the pivot are > the pivot
Requires a pivoting algorithm
Exchange the pivot with the last element in the first
sublist
The pivot is now in its final position
QuickSort the two sublists
p
A[i]"p A[i]>p
The Partition Algorithm
Quicksort Example
5 3 1 9 8 2 7
5 3 1 9 8 2 7
i j
i j
5 3 1 2 8 9 7
j i
2 3 1 5 8 9 7
2 3 1 8 9 7
i j
2 1 3
5 3 1 2 8 9 7
i j
i j
2 1 3
j i
1 2 3
Recursive Call
Quicksort (2 3 1) &
Quicksort (8 9 7)
Recursive Call
Quicksort (1) &
Quicksort (3)
i j
8 7 9
i j
8 7 9
j i
7 8 9
Recursive Call
Quicksort (7) &
Quicksort (9)
Worst Case Efficiency of
Quicksort
In the worst case all splits are completely
skewed
For instance, an already sorted list!
One subarray is empty, other reduced by
only one:
Make n+1 comparisons
Exchange pivot with itself
Quicksort left = , right = A[1..n-1]
C
worst
= (n+1) + n + + 3 = (n+1)(n+2)/2 - 3 = "(n
2
)
p
A[i]"p A[i]>p
General Efficiency of
Quicksort
Efficiency Cases:
Best: split in the middle "( n log n)
Worst: sorted array! "( n
2
)
Average: random arrays "( n log n)
Improvements (in combination 20-25%
faster):
Better pivot selection: median of three
partitioning avoids worst case in sorted files
Switch to Insertion Sort on small subfiles
Elimination of recursion
Considered the method of choice for
internal sorting for large files (n # 10000)
Binary Search
O(logn) algorithms for searching sorted
arrays
Atypical example - solves just one
problem of half the size on each ofi its
iterations
16
Objectives
To introduce the divide-and-conquer mind
set
To show a variety of divide-and-conquer
solutions:
Merge Sort
Quick Sort
Strassens Matrix Multiplication
Convex Hull by Divide-and-Conquer
To discuss the strengths and weaknesses of a
divide-and-conquer strategy
Strassens Matrix
Multiplication
Strassen observed [1969] that the product of two
matrices can be computed as follows:
where:
m1=(a00+a11) * (b00+b11)
m2=(a10+a11) * b00
m3=a00 * (b01-b11)
m4=a11 * (b10-b00)
m5=(a00+a01) * b11
m6= (a10-a00) * (b00+b01)
m7= (a01-a11) * (b10+b11)
C
00
C
01
C
10
C
11
A
00
A
01
A
10
A
11
B
00
B
01
B
10
B
11
M
1
+ M
4
-M
5
+M
7
M
3
+ M
5
M
2
+ M
4
M
1
+ M
3
- M
2
+ M
6
=
= *
Strassens Matrix
Multiplication
Op Count:
Each of M
1, ,
M
7
requires 1 mult and 1 or 2 add/sub
Total = 7 mul and 18 add/sub
Compared with brute force which requires 8 mults and 4
add/sub. But is asymptotic behaviour is NB
n/2 # n/2 submatrices are computed recursively
by the same method
Efficiency of Strassens
Algorithm
If n is not a power of 2, matrices can be
padded with zeros
Number of multiplications:
M(n) = 7 M(n/2) for n > 1, M(1) = 1
Set n = 2
k
, apply backward substitution
M(2
k
) = 7M(2
k-1
) = 7 [7 M(2
k-2
)] = 7
2
M(2
k-2
) = =
7
k
M(n) = 7
log n
= n
log 7
! n
2.807
Number of additions: A(n) $ "(n
log 7
)
Other algorithms closer to the lower limit of
n
2
multiplications, but are even more
complicated
QuickHull Algorithm
1. Sort points by increasing x-coordinate values
2. Identify leftmost and rightmost extreme points P
1

and P
2
(part of hull)
3. Compute upper hull:
Find point P
max
that is farthest away from line P
1
P
2
Quickhull the points to the left of line P
1
P
max
Quickhull the points to the left of line P
max
P
2
4. Similarly compute lower hull
QuickHull Algorithm
1. Sort points by increasing x-coordinate values
2. Identify leftmost and rightmost extreme points P
1

and P
2
(part of hull)
3. Compute upper hull:
Find point P
max
that is farthest away from line P
1
P
2
Quickhull the points to the left of line P
1
P
max
Quickhull the points to the left of line P
max
P
2
4. Similarly compute lower hull
P
1
P
max
P
2
L
L
Finding the Furthest Point
Given three points in the plane p
1
, p
2
, p
3

Area of Triangle = % p
1
p
2
p
3
= 1/2 ! D !
D =
! D ! = x
1
y
2
+ x
3
y
1
+ x
2
y
3
- x
3
y
2
- x
2
y
1
- x
1
y
3
Properties of ! D ! :
Positive iff p
3
is to the left of p
1
p
2
Correlates with distance of p
3
from p
1
p
2
x
1
y
1
1
x
2
y
2
1
x
3
y
3
1
Efficiency of Quickhull
Finding point farthest away from line P
1
P
2
is
linear in the number of points
This gives same efficiency as quicksort:
Worst case: "(n
2
)
Average case: "(n log n)
If an initial sort is required, this can be
accomplished in "( n log n) no increase in
asymptotic efficiency class
Alternative Divide-and-Conquer Convex
Hull:
Grahams scan and DCHull
Worst-case efficiency "( n log n)
Strengths and
Weaknesses of
!
Strengths:
Generally improves on Brute Force by one base
efficiency class
Easy to analyse using the Recurrence Templates
Ideally suited for parallel computations
"
Weaknesses:
Often requires recursion, which introduces
overheads
Can be inapplicable and inferior to simpler
algorithmic solutions