Divide-and-Conquer

 Divide-and conquer is a general

algorithm design paradigm:
 Divide the input data S in two or
more disjoint subsets S1, S2, …
 Conquer the subproblems by
solving them recursively
 base case for the recursion: If
the subproblems are small
enough just solve them
 Combine the solutions for S1, S2,
…, into a solution for S
 Analysis can be done using
recurrence equations

Divide-and-Conquer 1
Divide-and-Conquer Technique
(cont.)
a problem of size n
(instance)

subproblem 1 subproblem 2
of size n/2 of size n/2

a solution to a solution to
subproblem 1 subproblem 2

a solution to It generally leads to a

the original problem recursive algorithm!
Drawing the Recursion Tree
With: T (n)   b if n  2

2T (n / 2)  dn if n  2

b bn

bn + dn logn

Divide-and-Conquer 4
Merge-Sort
 Merge-sort is a sorting
algorithm based on the divide-
and-conquer paradigm
 It has O(n log n) running time

Divide-and-Conquer
5
Merge-Sort
 Merge-sort on
an input
sequence S with
n elements
consists of three
steps:
 Divide
 Recur
 Conquer

Divide-and-Conquer 6
Merging Two Sorted Sequences

Divide-and-Conquer 7
Merging (cont.)

X: Y:
3 10 23 54 1 5 25 75

Result:
Merging (cont.)

X: Y:
3 10 23 54 5 25 75

Result:
1
Merging (cont.)

X: Y:
10 23 54 5 25 75

Result:
1 3
Merging (cont.)

X: Y:
10 23 54 25 75

Result:
1 3 5
Merging (cont.)

X: Y:
23 54 25 75

Result:
1 3 5 10
Merging (cont.)

X: Y:
54 25 75

Result:
1 3 5 10 23
Merging (cont.)

X: Y:
54 75

Result:
1 3 5 10 23 25
Merging (cont.)

X: Y:
75

Result:
1 3 5 10 23 25 54
Merging (cont.)

X: Y:

Result:
1 3 5 10 23 25 54 75
Merge-Sort Tree
 An execution of merge-sort is depicted by a binary tree
 each node represents a recursive call of merge-sort and stores
 unsorted sequence before the execution and its partition
 sorted sequence at the end of the execution
 the root is the initial call
 the leaves are calls on subsequences of size 0 or 1

7 29 4  2 4 7 9

72  2 7 94  4 9

77 22 99 44

Divide-and-Conquer 17
Execution Example
 Partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 2 9 4  2 4 7 9 3 8 6 1  1 3 8 6

7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 18
Execution Example (cont.)
 Recursive call, partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 19
Execution Example (cont.)
 Recursive call, partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 20
Execution Example (cont.)
 Recursive call, base case

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 21
Execution Example (cont.)
 Recursive call, base case

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 22
Execution Example (cont.)
 Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 23
Execution Example (cont.)
 Recursive call, …, base case, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 24
Execution Example (cont.)
 Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 25
Execution Example (cont.)
 Recursive call, …, merge, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 26
Execution Example (cont.)
 Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

Divide-and-Conquer 27
Merge Sort Example
99 6 86 15 58 35 86 4 0
Merge Sort Example
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
Merge Sort Example
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
Merge Sort Example
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0
Merge Sort Example
99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

4 0
Merge Sort Example

99 6 86 15 58 35 86 0 4

Merge
4 0
Merge Sort Example

6 99 15 86 58 35 0 4 86

99 6 86 15 58 35 86 0 4

Merge
Merge Sort Example

6 15 86 99 0 4 35 58 86

6 99 15 86 58 35 0 4 86

Merge
Merge Sort Example
0 4 6 15 35 58 86 86 99

6 15 86 99 0 4 35 58 86

Merge
Merge Sort Example
0 4 6 15 35 58 86 86 99
Analysis

of Merge-Sort
The height h of the merge-sort tree is O(log n)
 at each recursive call we divide in half the sequence,
 The overall amount or work done at the nodes of depth i is O(n)
 we partition and merge 2i sequences of size n/2i
 we make 2i1 recursive calls
 Thus, the total running time of merge-sort is O(n log n)

depth #seqs size

0 1 n

1 2 n/2

i 2i n/2i

… … …

Divide-and-Conquer 38
Recurrence Equation Analysis
 The conquer step of merge-sort consists of merging two sorted
sequences, each with n/2 elements and implemented by means of a
doubly linked list, takes at most bn steps, for some constant b.
 Likewise, the basis case (n < 2) will take at b most steps.
 Therefore, if we let T n denote the running time of merge-sort:

 b if n  2
T (n)  
2T (n / 2)  bn if n  2
 We can therefore analyze the running time of merge-sort by finding a
closed form solution to the above equation.
 That is, a solution that has T n only on the left-hand side.

Divide-and-Conquer 39
Iterative Substitution
 In the iterative substitution, or “plug-and-chug,” technique, we
iteratively apply the recurrence equation to itself and see if we can
find a pattern: T ( n )  2T (n / 2)  bn
 2(2T ( n / 22 ))  b( n / 2))  bn
 22 T (n / 22 )  2bn
 23 T (n / 23 )  3bn
 24 T (n / 24 )  4bn
 ...
 2i T (n / 2i )  ibn
 Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.
 So,
T (n)  bn  bn log n
 Thus, T(n) is O(n log n).
Divide-and-Conquer 40
Quick Sort
Partition(A, p, r)
Quicksort(A, p, r) x, i, j := A[p], p , r + 1;
if p < r then while true do
q := Partition(A, p, r); repeat j := j – 1 until A[j]  x;
Quicksort(A, p, q); repeat i := i + 1 until A[i]  x;
Quicksort(A, q + 1, r) if i < j then
fi A[i]  A[j]
else
A[p..r] A[p] := A[j]; A[j] := x; return j;
fi
5 od

A[p..q] A[q+1..r]
Partition

5 5
Quicksort - 41