Divide-and-Conquer

• Divide the problem into a number of sub-problems

– Similar sub-problems of smaller size

• Conquer the sub-problems

– Solve the sub-problems recursively

– Sub-problem size small enough  solve the problems in

straightforward manner

• Combine the solutions of the sub-problems

– Obtain the solution for the original problem

1
Divide-and-Conquer Technique (cont.)

a problem of size n

subproblem 1 subproblem 2
of size n/2 Divide of size n/2

a solution to a solution to
subproblem 1 subproblem 2

a solution to
the original problem
 A General Template
//S is a large problem with input size of n
Algorithm divide_and_conquer(S)
if (S is small enough to handle)
solve it //base case: conquer
else
split S into two (equally-sized) subproblems S1 and S2
divide_and_conquer(S1)
divide_and_conquer(S2)
combine solutions to S1 and S2
endif
End
 General Divide-and-Conquer Recurrence
T(n) = aT(n/b) + f (n)
where f(n)  (nd), d  0, f(n) accounts for the time spent
on dividing the problem into smaller ones and combining
their solutions
Master Theorem: If a < bd, T(n)  (nd)
If a = bd, T(n)  (nd log n)
If a > bd, T(n)  (nlog b a )
Note: The same results hold with O instead of .

Examples: T(n) = 4T(n/2) + n  T(n)  ?

T(n) = 4T(n/2) + n2  T(n)  ?
T(n) = 4T(n/2) + n3  T(n)  ?
 Divide-and-Conquer Examples

• Binary Search

• Sorting: mergesort and quicksort

• Maximum subarray pronblem

• Closest-pair problem

• Matrix multiplication: Strassen’s algorithm

(reading)
 Binary search
Search a sorted array for a given item, x
 If x == middle array element, return true
 Else, BinarySearch lower (x<mid) or upper (x>mid) sub-array
 1 subproblem, half as large
• e.g. 2 4 5 6 7 8 9



search 7: needs 3 comparisons
• time: O(log n)
• The binary search can be used only if the elements are
sorted and stored in an array.
4 -6
 Algorithm binary-search

Input: A sorted sequence of n elements stored in an

array.
Output: The position of x (to be searched).
Step 1: If only one element remains in the array, solve it
directly.
Step 2: Compare x with the middle element of the array.
Step 2.1: If x = middle element, then output it and stop.
Step 2.2: If x < middle element, then recursively solve the
problem with x and the left half array.
Step 2.3: If x > middle element, then recursively solve the
problem with x and the right half array.

4 -7
Algorithm BinSearch(a, low, high, x)

// a[]: sorted sequence in nondecreasing order

// low, high: the bounds for searching in a []
// x: the element to be searched
// If x = a[j], for some j, then return j else return –1
if (low > high) then return –1 // invalid range
if (low = high) then // if small P
if (x == a[i]) then return i
else return -1
else // divide P into two smaller subproblems
mid = (low + high) / 2
if (x == a[mid]) then return mid
else if (x < a[mid]) then
return BinSearch(a, low, mid-1, x)
else return BinSearch(a, mid+1, high, x)
4 -8
 Example: Binary Search

Search a sorted array for a given item, x

 If x == middle array element, return true
 Else, BinarySearch lower (x<mid) or upper (x>mid) sub-array
 1 subproblem, half as large

Equation:
Base Subproblem size
case T(1)  b
T(n)  1 T(n/2) + c for n>1 Work dividing
# subproblems and combining
9
Divide-and-Conquer
 Binary Search Solution

Equation:

T(1)  b
T(n)  T(n/2) + c for n>1

Solution: (finding the closed form solution)

T(n)  T(n/2) + c
 T(n/4) + c + c
 T(n/8) + c + c + c Iterative
 T(n/2k) + kc substitution
 T(1) + c log n where k = log n method
 b + c log n = O(log n)

10
Divide-and-Conquer
 Recursion Tree for
Binary Search

Problem size Cost per

stage
n O(1)

n/2 O(1)

n/4 O(1) log n

… …

1 O(1)

Θ(log n)

11
Divide-and-Conquer
 Merge Sort

7 29 4  2 4 7 9

72  2 7 94  4 9

77 22 99 44

Divide-and-Conquer 12
 Merge Sort Approach
• To sort an array A[F. . L]:
• Divide
– Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
• Conquer
– Sort the subsequences recursively using merge sort
– When the size of the sequences is 1 there is nothing
more to do
• Combine
– Merge the two sorted subsequences
13
 Merge Sort
F M L
1 2 3 4 5 6 7 8

Alg.: MERGE-SORT(A, F, L) 5 2 4 7 1 3 2 6

if (F < L) Check for base case

{ then M ← (F + L)/2 Divide

MERGE-SORT(A, F, M) Conquer

MERGE-SORT(A, M + 1, L) Conquer

MERGE(A, F, M, L) Combine
• }

• Initial call: MERGE-SORT(A, 1, n)

14
 Example – n Power of 2
1 2 3 4 5 6 7 8

Divide 5 2 4 7 1 3 2 6 q=4

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

15
 Example – n Power of 2
1 2 3 4 5 6 7 8

Conquer 1 2 2 3 4 5 6 7
and
Merge 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6

1 2 3 4 5 6 7 8

2 5 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

16
 Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6 q=6
Divide
1 2 3 4 5 6 7 8 9 10 11

q=3 4 7 2 6 1 4 7 3 5 2 6 q=9

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

17
 Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11

Conquer 1 2 2 3 4 4 5 6 6 7 7
and
Merge
1 2 3 4 5 6 7 8 9 10 11

1 2 4 4 6 7 2 3 5 6 7

1 2 3 4 5 6 7 8 9 10 11

2 4 7 1 4 6 3 5 7 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 1 6 4 3 7 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

18
 Execution Example

• Partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 2 9 4  2 4 7 9 3 8 6 1  1 3 8 6

7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

19
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

20
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, partition

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

21
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, base case

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

22
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, base case

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

23
Divide-and-Conquer
 Execution Example (cont.)

• Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

24
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, …, base case, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 94 3 8  3 8 6 1  1 6

77 22 33 88 66 11

25
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, …, base case, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 94 3 8  3 8 6 1  1 6

77 22 99 33 88 66 11

26
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, …, base case, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 94 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

27
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, …, base case, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

28
Divide-and-Conquer
 Execution Example (cont.)

• Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 8 6

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

29
Divide-and-Conquer
 Execution Example (cont.)

• Recursive call, …, merge, merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

30
Divide-and-Conquer
 Execution Example (cont.)

• Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

31
Divide-and-Conquer
 Merging
F M L
1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6

• Input: Array A and indices F, M, L such that

F≤M<L
– Subarrays A[F . . M] and A[M + 1 . . L] are sorted
• Output: One single sorted subarray A[F . . L]

32
 Merging
F M L
• Idea for merging: 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6
– Two piles of sorted cards
• Choose the smaller of the two top cards
• Remove it and place it in the output pile

– Repeat the process until one pile is empty

– Take the remaining input pile and place it face-down
onto the output pile
A1 A[p, q]
A[p, r]

A2 A[q+1, r]

33
Example: MERGE(A, 9, 12, 16)
p q r

34
Example: MERGE(A, 9, 12, 16)

35
Example (cont.)

36
Example (cont.)

37
Example (cont.)

Done!

38
 Merge - Pseudocode
p q r
Alg.: MERGE(A, p, q, r) 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6
1. Compute n1 and n2
2. Copy the first n1 elements into n1 n2
L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1]
3. L[n1 + 1] ← ; R[n2 + 1] ←  p q

4. i ← 1; j ← 1 L 2 4 5 7 
5. for k ← p to r q+1 r

R 1 2 3 6 
6. do if L[ i ] ≤ R[ j ]
7. then A[k] ← L[ i ]
8. i ←i + 1
9. else A[k] ← R[ j ]
10. j←j+1 39
 Running Time of Merge
(assume last for loop)
• Initialization (copying into temporary arrays):
 (n1 + n2) = (n)
• Adding the elements to the final array:
- n iterations, each taking constant time  (n)
• Total time for Merge:
 (n)

40
 Analyzing Divide-and Conquer
Algorithms
• The recurrence is based on the three steps of
the paradigm:
– T(n) – running time on a problem of size n
– Divide the problem into a subproblems, each of size
n/b: takes D(n)
– Conquer (solve) the subproblems aT(n/b)
– Combine the solutions C(n)

(1) if n ≤ c
T(n) = aT(n/b) + D(n) + C(n) otherwise

41
 MERGE-SORT Running Time
• Divide:
– compute q as the average of p and r: D(n) = (1)
• Conquer:
– recursively solve 2 subproblems, each of size n/2
 2T (n/2)
• Combine:
– MERGE on an n-element subarray takes (n) time
 C(n) = (n)
(1) if n =1
T(n) = 2T(n/2) + (n) if n > 1

42
 Solve the Recurrence
T(n) = c if n = 1
2T(n/2) + cn if n > 1

Use Master’s Theorem:

Compare n with f(n) = cn

Case 2: T(n) = Θ(nlgn)

43
 Merge Sort - Discussion
• Running time insensitive of the input

• Advantages:
– Guaranteed to run in (nlgn)

• Disadvantage
– Requires extra space N

44
 Sorting Challenge 1
Problem: Sort a file of huge records with large
size keys
Example application: Reorganize your MP-3 files

Which method to use?

A. merge sort, guaranteed to run in time NlgN
B. selection sort
C. bubble sort
D. insertion sort

45
 Sorting Files with Huge Records and
Small Keys

• Insertion sort or bubble sort?

– NO, too many exchanges

• Selection sort?

– YES, it takes linear time for exchanges

• Merge sort or custom method?

– Probably not: selection sort simpler, does less swaps

46
 Sorting Challenge 2
Problem: Sort a huge randomly-ordered file of
small records
Application: Process transaction record for a
phone company

Which sorting method to use?

A. Bubble sort
B. Selection sort
C. Mergesort guaranteed to run in time NlgN
D. Insertion sort

47
Sorting Huge, Randomly - Ordered Files

• Selection sort?
– NO, always takes quadratic time

• Bubble sort?
– NO, quadratic time for randomly-ordered keys

• Insertion sort?
– NO, quadratic time for randomly-ordered keys

• Mergesort?
– YES, it is designed for this problem

48
 Sorting Challenge 3
Problem: sort a file that is already almost in
order
Applications:
– Re-sort a huge database after a few changes
– Doublecheck that someone else sorted a file
Which sorting method to use?
A. Mergesort, guaranteed to run in time NlgN
B. Selection sort
C. Bubble sort
D. Insertion sort

49
 Sorting Files That are Almost in Order
• Selection sort?
– NO, always takes quadratic time
• Bubble sort?
– NO, bad for some definitions of “almost in order”
– Ex: B C D E F G H I J K L M N O P Q R S T U V W X Y Z A
• Insertion sort?
– YES, takes linear time for most definitions of “almost
in order”
• Mergesort or custom method?
– Probably not: insertion sort simpler and faster

50