Divide & Conquer

Divide and Conquer Algorithms

-
Example
- Binary Search
- Merge Sort
- Quick Sort
- Counting
- Closest Pair of Points

2
 Divide and Conquer

Divide the problem into a number of subproblems


There must be base case (to stop recursion).

Conquer (solve) each subproblem recursively

Combine (merge) solutions to subproblems into a solution to the

original problem

3
 Divide-and-Conquer

Most common usage.


Break up problem of size n into two equal parts of size ½n.

Solve two parts recursively.

Combine two solutions into overall solution in linear time.

Consequence.

Brute force: n2.

Divide-and-conquer: n log n.

4
 Quicksort

10/15/20 5
 Quick Sort

• It is a comparison sort where elements are sorted in place.

• This sort uses the divide-and-conquer algorithm for sorting an
array of n elements, S[1,2,…..n].

10/15/20
 Quicksort Algorithm

Given an array of n elements (e.g., integers):

• If array only contains one element, return
• Else
– pick one element to use as pivot.
– Partition elements into two sub-arrays:
• Elements less than or equal to pivot
• Elements greater than pivot
– Quicksort two sub-arrays
– Return results

10/15/20
Pivot Selection

1. Use the first element as pivot. If the input is random, it is

acceptable. But if
the input is presorted or in reverse order, pivot provides a poor
partition.
2. Choose the pivot randomly. Although a random pivot is very
unlikely provide poor partition but it is an expensive approach
and does not reduce the average running time of rest of the
algorithm.
3. Choose the median of three numbers as pivot. The best choice
of pivot would be the median of array. Unfortunately, it is
relatively hard to calculate and would slow down quicksort
considerably.

10/15/20
Quicksort Algorithm

Quicksort(A , Left, Right) Partition(A, Left, Right)

1. If Left < Right 1. x = A[Right]
2. then q = Partition(A, Left, 2. i = Left – 1
Right)
3. For j = Left to Right – 1
3. Quicksort(A, Left, q-1)
4. do if A[j] <= x
4. Quicksort(A, q+1, Right)
5. then i = i + 1
5. END
6. Swap(A[i], A[j])
7. Swap(A[i + 1], A[Right])
8. Return (i + 1)

10/15/20
Example:

13 81 10 43 31 75 0 92

Select pivot
13 81 10 43 31 75 0 92

Partition

13 10 0 31 81 43 75 92

Quicksort

0 10 13 31 43 75 81 92

Sorted List
0 10 13 31 43 75 81 92

10

10/15/20
 Example

We are given array of n integers to

sort:
20 80 70 80 60 50 7 30 40

11

10/15/20
 Pick Pivot Element

There are a number of ways to pick the pivot

element. In this example, we will use the last
element in the array:
20 80 70 10 60 50 7 30 40

12

10/15/20
 Partitioning Array

Given a pivot, partition the elements of the

array such that the resulting array consists
of:
1. One sub-array that contains elements >= pivot
2. Another sub-array that contains elements < pivot

The sub-arrays are stored in the original data

array.

Partitioning loops through, swapping elements

below/above pivot.

13

10/15/20
pivot_index = 0 20 80 70 10 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

14

10/15/20
pivot_index = 0 20 80 70 10 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

15

10/15/20
pivot_index = 0 20 80 70 10 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

16

10/15/20
pivot_index = 0 20 80 70 10 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

17

10/15/20
pivot_index = 0 20 80 70 10 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

18

10/15/20
pivot_index = 0 20 10 70 80 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

19

10/15/20
pivot_index = 0 20 10 70 80 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

20

10/15/20
pivot_index = 0 20 10 70 80 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

21

10/15/20
pivot_index = 8 20 10 70 80 60 50 7 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

22

10/15/20
pivot_index = 0 20 10 7 80 60 50 70 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

23

10/15/20
pivot_index = 0 20 10 7 80 60 50 70 30 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

24

10/15/20
pivot_index = 0 20 10 7 30 60 50 70 80 40

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j

25

10/15/20
pivot_index = 0 20 10 7 30 40 50 70 80 60

[0] [1] [2] [3] [4] [5] [6] [7] [8]

26

10/15/20
 Partition Result

20 10 7 30 40 50 70 80 60

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

27

10/15/20
 Recursion: Quicksort Sub-arrays

20 10 7 30 40 50 70 80 60

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

28

10/15/20
Complexity of Quick Sort

1. Worst Case:
The worst case occurs if the input is presorted or in reverse order
and if the 1st or the last element is chosen as pivot. In this case
pivot provides a poor partition because all elements go into one
partition and the other partition is empty. The worst case running
time is 0(N2).

2. Average Case:
The average running time of Quick sort is obtained by averaging
the runtime of all possible subproblems’ sizes. That means it
occurs from the fact that on average each reduction step of the
algorithm produces two subsets. So, the average running time is
0(NlogN).

3. Best Case:
The best case occurs when the partition always splits into two
subsets with equal number of elements. So, the running time is 29

0(NlogN). 10/15/20
 Mergesort

10/15/20 30
Merge Sort
• Merge sort is a comparison sorting technique.
• This technique follows the divide-and-conquer approach.
• It maintains the following 3 steps:
1. Divide: Divide N-element sequence to be sorted into two
subsequences of
about N/2 elements each and sort the two subsequences
recursively .

2. Conquer: Merge the two sorted subsequences to produce the sorted

result.

• Merge sort uses the “merge” step to combine two sorted sublists to create
one single
sorted list.
• Suppose A is a sorted list with R elements and B is another sorted list with
S elements.
After merging there is only a single sorted list C with N=R+S elements.
31

10/15/20
Mergesort Algorithm

Merge(A, Left, Center, Right)

Merge-Sort(A, Left, Right)
(1) n1 = Center – Left + 1; n2 = Right – Center
(1) If Left < Right do steps 2 to 5
(2) Set Center = (Left+Right)/2 (2) Create arrays L[1…n1] and R[1…n2]

(3) Merge-Sort (A, Left, Center) (3) For i = 1 to n1

(4) do L[i] = A[Left + i -1]
(4) Merge-Sort (A,Center+1,
Right) (5) For j = 1 to n2
(6) do R[j] = A[Center + j]
(5) Merge(A, Left, Center, Right)
(7) L[n1 +1]=inf; R[n2+1]=inf;
(6) End
(8) i=1;j=1
(9) For k = Left to Right
(10) do if L[i] <= R[j]
(11) then A[k] = L[i]
(12) i=i+1
(13) else A[k] = R[j]
(14) j=j+1
(16) End

32

10/15/20
Example:

Suppose Array A = 5, 2, 4, 7, 1, 3, 2, 6. Sort the array using Mergesort.

5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

2 5 1 4 3 7 2 6

1 2 4 5 2 3 6 7

1 2 2 3 4 5 6 7

33

10/15/20
Example:
Suppose Array A = 38, 27, 43, 3, 9, 82 and 10. Sort the array using Mergesort.

10/15/20
34
 Merging

Merging. Combine two pre-sorted lists into a sorted whole.

How to merge efficiently?


Linear number of comparisons.

Use temporary array.

A G L O R H I M S T

A G H I

35
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A auxiliary array

36
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G auxiliary array

37
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H auxiliary array

38
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H I auxiliary array

39
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H I L auxiliary array

40
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H I L M auxiliary array

41
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H I L M O auxiliary array

42
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

smallest smallest

A G L O R H I M S T

A G H I L M O R auxiliary array

43
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

first half
exhausted smallest

A G L O R H I M S T

A G H I L M O R S auxiliary array

44
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

first half
exhausted smallest

A G L O R H I M S T

A G H I L M O R S T auxiliary array

45
 Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

first half second half

exhausted exhausted

A G L O R H I M S T

A G H I L M O R S T auxiliary array

46
 A Useful Recurrence Relation

Def. T(n) = number of comparisons to mergesort an input of size n.

Mergesort recurrence.

Solution. T(n) = O(n log2 n).

Assorted proofs. We describe several ways to prove this recurrence.

Initially we assume n is a power of 2 and replace  with =.

47
 Proof by Recursion Tree

T(n) n

T(n/2) T(n/2) 2(n/2)

T(n/4) T(n/4) T(n/4) T(n/4) 4(n/4)

log2n
...
T(n / 2k) 2k (n / 2k)

...

T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) n/2 (2)

n log2n
48
 Proof by Telescoping

Claim. If T(n) satisfies this recurrence, then T(n) = n log 2 n.

assumes n is a power of 2

Pf. For n > 1: T(n) 2T(n /2)

 1
n n
T(n /2)
 1
n /2
T(n / 4)
 1 1
n/4

T(n /n)
  1  1
n /n  
log2 n
 log2 n


49
 Proof by Induction

Claim. If T(n) satisfies this recurrence, then T(n) = n log 2 n.

assumes n is a power of 2

Pf. (by induction on n)


Base case: n = 1.

Inductive hypothesis: T(n) = n log2 n.

Goal: show that T(2n) = 2n log2 (2n).

50