Chapter two

DIVIDE AND CONQURE

1
 OUTLINE
Divide and Conquer
• The General Method
• Binary Search
• Merge sort
• Selection sort
• Quick sort
• Finding Maximum and Minimum Value

2
 DIVIDE AND CONQURE
• Divide and Conquer algorithms consist of two parts:

1. break a problem into smaller pieces and

2. solve the smaller sub-problems

Divide: The problem is divided into smaller sub-problems. Smaller

problems are solved recursively.

Conquer: The solution to the original problem is then formed from the
solutions to the sub problems.

Combine: Take the solutions to the sub problems and combine these
solutions into a solution for the original problem

• Examples: Binary Search, Merge sort, Quicksort, and Selection Sort……

3
Divide and conquer Technique
a problem of size n

subproblem 1 subproblem 2
of size n/2 of size n/2

a solution to a solution to
subproblem 1 subproblem 2

a solution to
the original problem

It general leads to a recursive

algorithm!
4
A general divide and conquer algorithm
Step 1: If the problem size is small, solve this problem directly; otherwise, split
the original problem into 2 sub problems with equal sizes.

Step 2: Recursively solve these 2 sub-problems by applying this algorithm.

Step 3: Merge(Combined) the solutions of the 2 sub-problems into a solution of

the original problem.

• Time complexity:  2T(n/2)+S(n)+M(n) , n  c

T(n)= 
 b ,n<c

where S(n) : time for splitting and M(n) : time for merging

b : a constant and c : a constant

5
 Binary search
The binary search is one of searching mechanism which
search a given element by using the divide -and-conquer
strategy to search through an array.
The array must be sorted
• the “zeroing in” strategy for looking up a word in the dictionary
won’t work it the
words are not in alphabetical order
• Binary search will not work unless the array is sorted
Very efficient algorithm for searching in sorted array:
K
vs
A[0] . . . A[m] . . . A[n-1]
If K = A[m], stop (successful search); otherwise,
continue searching by the same method in A[0..m-1] if
K < A[m] and in A[m+1..n-1] if K > A[m] 6
 Binary search Algorithm
Input: A sorted sequence of n elements stored in an array.

Output: The position of x (to be searched).

Step 1: If only one element remains in the array, solve it directly.

Step 2: Compare x with the middle element of the array.

Step 2.1: If x = middle element, then output it and stop.

Step 2.2: If x < middle element, then recursively solve the problem with x and
the left half array.

Step 2.3: If x > middle element, then recursively solve the problem with x and
the right half array.

The algorithm searches for an element x in an ascending array of elements

7
 CONT…
Algorithm body:
index:=0, bot:=1, top:=n
while (top ≥ bot and index=0)
mid :=  (bot + top) / 2
if a[mid] = x
then index := mid
if a[mid] > x
then top := mid-1
else bot := mid+1
end while
Output: index (If index has the value 0 then x is not in the array;
otherwise, index gives the index of the array where x is
located.) 8
 Example of Binary Search
To search a list of n items, first look at the item in location n/2

• then search either the region from 0 to n/2-1 or the region

from n/2+1 to n-1
Binary search. Given value and sorted array a[], find index I such that
a[i] = value, or report that no such index exists.

Invariant. Algorithm maintains a[bot]  value  a[top].

Example. Binary search for 33.

6 13 14 25 33 43 51 53 72 84 92 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13

bot Mid = low + top 9

top /2
 Run time of Binary search
• At each iteration, the length of the new sub array to be searched is
approximately half of the previous one.

• If n = 2k+m, where 0  m < 2k,

then n can be split approximately in half k times.

• Since 2k  n < 2k+1, then k = log2n (by proposition 1)

• Thus, the number of iterations of the while loop

in a worst-case execution of the algorithm is log2n+1.

• The number of operations in each loop is constant (doesn’t increase with

n).
10
 Merge Sort
Merge sort is one of the sorting mechanisms which sort a given
data(element) in ascending or descending order. By using divide
and conquer strategy and divided in to two equal or almost equal
parts.

Merge-sort on an input sequence S with n elements consists of

three steps:

• Divide: Divide the n-element sequence to be sorted into two

subsequences of n/2 elements each

• Conquer: Sort the two subsequences recursively using merge

11
 Merge Sort Algorithm
Input: A set S of n elements.
Output: The sorted sequence of the inputs in non decreasing
order.
Step 1: If |S|2, solve it directly.
Step 2: Recursively apply this algorithm to solve the left half
part and right half part of S, and the results are stored in S 1

and S2, respectively.

Step 3: Perform the two-way merge scheme on S1 and S2.

12
 Merge Sort Example
8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 22 7 9 4 5 1
0 1 22 3 4 5 6 7

3 8 2 7 4 9 1 5
0 1 2 3 4 5 4 5

2 3 7 8 1 4 5 9
0 1 2 3 4
4 5 6 77

1 2 3 4 5 7 8 9 13
0 1 2 3 4 5 6 7
 Merge Sort Algorithm
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers

MergeSort (A, p, r) // sort A[p..r] by divide

& conquer
1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r) // merges A[p..q]
with A[q+1..r]
Initial Call: MergeSort(A, 1, n)
14
 Merge Algorithm
Merge(A, p, q, r)
1 n1  q – p + 1
Input: Array containing sorted
2 n2  r – q sub-arrays A[p..q] and A[q+1..r].
3 for i  1 to n1
Output: Merged sorted sub-
4 do L[i]  A[p + i – 1]
array in A[p..r].
5 for j  1 to n2
6 do R[j]  A[q + j]
7 L[n1+1]  
8 R[n2+1]  
9 i1
10 j  1
Sentinels, to avoid having to
11 for k p to r
check if either sub array is
12 do if L[i]  R[j]
fully copied at each step.
13 then A[k]  L[i]
14 ii+1
15 else A[k]  R[j]
16 jj+1 15
 Example of Merge

A … 61 86 26
8 32
9 1
26 9
32 42 43 …

kk k k k k k k k

L 6 8 26 32 
R 1 9 42 43 

i i i i i j j j j j

16
 Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.

cn cn

cn/2 cn/2 cn

lg n

cn/4 cn/4 cn/4 cn/4 cn

c c c c c c cn
Total : cnlgn+cn

17
 Run Time of Merge Sort
• Running time T(n) of Merge Sort:

• Divide: computing the middle takes (1)

• Conquer: solving 2 sub problems takes 2T (n/2)

• Combine: merging n elements takes (n)

• Total:

T (n) = (1) if n = 1
T (n) = 2T (n/2) + (n) if n > 1
 T (n) = O(n log n)
18
 Selection Sort
Selection sort is one of the sorting mechanisms which sort a
given data (elements), in ascending or descending order. By
using divide and conquer strategy and

Depend on the selection element. And swap with the

smallest element.

The Selection Sort searches (linear search) all of the

elements in a list until it finds the smallest element.

19
 Selection Sort Algorithm
For each index position i
• It “swaps” this with the first element in the list.
• Next it finds the smallest of the remaining elements, and
• “swaps” it with the second element.
• Repeat this process until you compare only the last two
elements in the list

20
 Selection Sort Algorithm
Selection Sort( int[] number) {
Input: Array containing sorted sub-
int min-Index, A.length, arrays number[ ].
temp;
Output: selection sorted sub-array
for (int i=0; i<=size-2; i+
+) { in order number[ ].
min-Index = i; We start by searching for the
for (int j=i+1; j<=size-1; j+
+) { smallest element in the List.
if(number[j]<number[min-
Index]) { Swapping keeps it in this
min-Index=j; position.
}}
temp= number[i];
number[i]= number[min-
The last two elements are
Index]; in order, so no swap is
number[min-Index] = temp; 21
 Example

7 15 34
20 13 15 34
15 34
20
13 20
7
0 1 2 3 4

 Selection Sort actually employs three things we should

know how to do.
• Traverse through an array
• Find the minimum of a set of array elements
• Swap positions of two array element
Running time of Selection Sort Algorithm
Selection Sort( int[] number) {
T(n): Here we have two for loop
int minIndex, size, temp;
that means nested for loop so the
size = number.length;
for (int i=0; i<=size-2; i+ first loop excite from i = 0 to n
+) { so .
minIndex = i;
for (int j=i+1; j<=size-1; T(n) = O(n)
j++) {
if (number[j] < The second loop also excite
number[minIndex]) { from I + 1 to n so
minIndex=j;
}} T(n) = O(n)
temp= number[i];
number[i]= So the run time of Selection sort
number[minIndex]; is
number[minIndex] = temp;
}} T(n) = n * n = n2 23
 Quick Sort
Quick sort is one of the sorting mechanisms which sort a given
data (elements), in ascending or descending order. By using
divide and conquer strategy and divided in to two
depend on the partition element. the heart of Quick sort is
Partitioning.
The main idea is to partition the array into two regions:
• small items are moved to the left side of the array
• large items are moved to the right side
After partitioning, repeat the sort on the left and
right sides
• each region is a sub-problem, a smaller version of the
original problem
A common technique: use the first/last item in the 24
 CONT…
• Quick sort Here is the three-step divide-and-conquer process for sorting
a typical sub-array A[p …..r ]

• Divide: Partition (rearrange) the array A[p .. r] into two (possibly empty)
sub arrays A[p .. q - 1 and A[q + 1 ..r] such that each element of A[p ...
q - 1] is less than or equal to A[q] which is, in turn, less than or equal to
each element of A[q + 1.. r . Compute the index q as part of this
partitioning procedure.

• Conquer: Sort the two sub-arrays A[p .. q – 1] and A[q + 1.. r] by

recursive calls to quicksort.

• Combine: Because the sub-arrays are already sorted, no work is needed

25
 Algorithm for Quick Sort
Input: A set S of n elements.

Output: The sorted sequence of the inputs in non decreasing

order.

Step 1: If |S|2, solve it directly.

Step 2: (Partition step) Use a pivot to scan all elements in S.

Put the smaller elements in S1, and the larger elements in S2.

Step 3: Recursively solve S1 and S2.

26
 Example
Pivot 24
7

0 331 12
1 93
3 2
9 84
8 27
2 44
9 8 79
7
0 1 2 3 4 5 6 7

Note : If A[j] < Pivot = True Move i to next element

Else = swap A[j] with A[i]
Pivot Might be whether the first or the last index of an array
 Quick Sort Algorithm
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers

QUICKSORT.(A, p, r)
1 if p < r
2 q = PA RTITIO N (A, p, r)
3 QUICKSORT (A, p, q – 1)
4 QUICKSORT (A, q + 1, r)

To sort an entire array A, the initial call is QUICKSORT (A, 1, A. length)

 Partition Algorithm
Partitioning the array
• The key to the algorithm is the Input: Array containing sorted
PA RTITIO N procedure, which sub-arrays A[p..r].
rearranges the sub array A[p.. r]
in place. Output: Quick sorted sub-array
in A[p…..r].
PARTITIO N (A, p, r )
1 x = A[r]
2 i=p-1
3 for j = p to r - 1
4 if A[j] ≤ x
5 i=i+1
6 exchange A[i] with A[j]
7 exchange A[I + 1] with A[r]
8 return i + 1
Correctness of Quick sort
Partitioning the array
Loop Invariant for the for loop
• The key to the algorithm is the
by the procedure PARTITION on a sub array
PA RTITIO N procedure, which
A[p…..r] The values in A[p … i] are all less than
rearranges the sub array A[p.. r]
or equal to x, the values in A[i +1…j -1] are all
in place.
greater than x, and A[r]= x. The sub array
PA RTITIO N (A, p, r ) A[j ..r – 1] can take on any values.
1 x = A[r]
2 i=p-1
3 for j = p to r - 1
4 if A[j] ≤ x Initialization:
5 i=i+1 •Prior to the first iteration of the loop, I = p – 1
6 exchange A[i] with A[j] and j = p Because no values lie between p and I
7 exchange A[I + 1] with A[r] and no values lie between i +1 and j -1,
8 return i + 1 • the first two conditions of the loop invariant
are trivially satisfied. The assignment in line 1
Satisfies the third condition.
Correctness of Quick sort
Partitioning the array Maintenance:
• The key to the algorithm is the • when A[j] > x; the only action in the
PA RTITIO N procedure, which loop is to increment j . After j is
rearranges the sub array A[p.. r] incremented, condition 2 holds for A[j
in place. -1] and all other entries remain
PA RTITIO N (A, p, r ) unchanged.
1 x = A[r]
2 i=p-1
3 for j = p to r - 1
Termination:
4 if A[j] ≤ x • At termination, j = r. Therefore, every
5 i=i+1 entry in the array is in one of the three sets
6 exchange A[i] with A[j] described by the invariant, and we have
partitioned the values in the array into three sets:
7 exchange A[I + 1] with A[r] • those less than or equal to x, those greater
8 return i + 1 • than x, and a singleton set containing x.

Comp 122
Recursion Tree for Quick Sort
 Continue expanding until the problem size reduces to 1.
cn cn

1/10 n cn
9/10 n

lg n

1/100 n 9/100 n 9/100 n 81/100 n cn

c c c c c c cn
Total : O(n log n)
 Run Time of Quick Sort
• Running time T(n) of Quick Sort:
• Divide: computing the middle takes (1)
• Conquer: solving 2 sub problems takes T(n-1)
• Combine: merging n elements takes (n)
• Total:
• Best Case T(n) = (1) if n = 1
T(n) = + (1) +2(T/n) + (n) if n > 1
 T(n) = O(n log n)
• Worst Case T(n) = (1) if n = 1

T(n) = + (0) +(n-1) + (n) if n > 1

• if the element is sorted whether ascending or descending order 
T(n) = O(n2) 33
 Find Maximum and minimum value
• It is one of the sorting mechanisms which sort a given data (elements), in
ascending or descending order. By using divide and conquer strategy we
can identify which one is Maximum or minimum value in the given array.

and divided in to two equal or almost equal parts to compare each

elements.

• Divide: Divide the n-element sequence to be sorted into two

subsequences of n/2 elements each

• Conquer: find the maximum or minimum value from the two

subsequences recursively.

• Combine: the two subsequences to produce the maximum or minimum

34
 Find Max and Min Algorithm
Input: A set S of n elements.
Output: The sorted sequence of the inputs in non decreasing
order.
Step 1: If |S|2, solve it directly.
Step 2: Recursively apply this algorithm to solve the left half
part and right half part of S, and the results are stored in S 1

and S2, respectively.

Step 3: Perform the two-way finding max or min scheme on

35
 Max and Min Algorithm
Max(A, p, q, r)
3 L[i]  A[p + i – 1]
4 R[j]  A[q + j]
5 if L[i] ≥ R[j]
6 then Input: Array containing sorted
7 A[k]  L[i] sub-arrays A[p..q] and A[q+1..r].
8 else
9 A[k]  R[j] Output: Maximum or min value
Min(A, p, q, r)
10 L[i]  A[p + i – 1]
11 R[j]  A[q + j]
12 if L[i]  R[j]
13 then
14 A[k]  L[i]
15 else
16 A[k]  R[j]

36
 Find Maximum Value
8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 22 7 9 4 5 1
0 1 22 3 4 5 6 7

8 7 9 5

8 9

9 37
 Finding Minimum Value
8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 2 7 9 4 5 1
0 1 2 3 4 5 6 7

8 3 22 7 9 4 5 1
0 1 22 3 4 5 6 7

3 2 4 1

2 1

1 38
 Run Time of finding Max or Min
• Running time T(n) of Merge Sort:

• Divide: computing the middle takes (1)

• Conquer: solving 2 sub problems takes 2T(n/2)

• Combine: merging n elements takes (1)

• Total:
T(n) = (1) if n = 1

T(n) = 2T(n/2) + (1) if n > 1

 T(n) = O( log n)
39
 Summery of sorting
Algorithm Worst case Best case Notes

slow AND in-place

selection-sort O(n2) O(n)
for small data sets (< 1K)

slow AND in-place

insertion-sort O(n2) O(n)
for small data sets (< 1K)

fast AND in-place

heap-sort O(n log n) O(n log n)
for large data sets (1K — 1M)

fast
merge-sort O(n log n) O(n log n) sequential data access
for huge data sets (> 1M)

fast
O(n2) O(n log n)
Quick-Sort sequential data access
for huge data sets (> 1M)
40
 D! !
E N

41