Divide-and-Conquer

• Divide-and-conquer is the best-known general algorithm design technique.

• General Working Plan:

• A problem is divided into several subproblems of the same type, ideally

of about equal size.

• The subproblems are solved

• If necessary, the solutions to the subproblems are combined to get a

solution to the original problem.

Divide-and-Conquer
 problem of computing the sum of n numbers.

• Consider the problem of computing the sum of n numbers.

• If n > 1, : Divide the problem into two instances of the same problem:
(i) To compute the sum of the first [n/2] numbers

(II) To compute the sum of the remaining n/2 numbers.

• if n = 1, return a0 as the answer

• Once each of these two sums is computed by applying the same method
recursively, we can add their values to get the sum in question:
 General Divide-and-Conquer recurrence
 A problem’s instance of size n is divided into two instances of size n/2

 An instance of size n can be divided into b instances of size n/b, with

a of them needing to be solved

 a and b are constants; a ≥ 1 and b > 1.

 Assuming that size n is a power of b to simplify the analysis,

recurrence for the running time T(n) is:

T(n) = aT(n/b) + f(n) --- (5.1)

f (n) : Function that accounts for the order of growth of its solution T (n)
the time spent on dividing an depends on the values of the constants a
instance of size n into and b and the order of growth of the
instances of size n/b and function f (n).
combining their solutions
 Master Theorem
• If
The recurrence for the number of additions A(n) made by the divide-and-
conquer sum-computation algorithm on inputs of size
 Merge Sort Algorithm:
 Merge sort is a perfect example of a successful application of the divide-and-conquer
technique.

 Split array A[1..n] in two and make copies of each half in arrays B[1.. n/2 ] and C[1.. n/2 ]

 Sort arrays B and C

 Merge sorted arrays B and C into array A as follows:

 Repeat the following until no elements remain in one of the arrays:

 compare the first elements in the remaining unprocessed portions of the arrays

 copy the smaller of the two into A, while incrementing the index indicating the
unprocessed portion of that array

 Once all elements in one of the arrays are processed, copy the remaining
unprocessed elements from the other array into A.
 Algorithm: Merge (B [0…p-1], C [0...q-1], A [0...p+q-1])
//Merges two sorted arrays into one sorted array.
//Input: Arrays B [0…p-1] and C [0...q-1] both sorted.
Algorithm: MergeSort (A [0...n-1]) //Output: Sorted array A [0...p+q-1] of the elements of B
//This algorithm sorts array A [0...n-1] by recursive mergesort. and C.
//Input: An array A [0...n-1] of orderable elements.
//Output: Array A [0...n-1] sorted in non-decreasing order
{
{
i ← 0; j←0; k←0
if n>1 {
while i<p and j<q do
Copy A [0…└n/2┘-1] to B [0…└n/2┘-1] {
Copy A [└n/2┘… n-1] to C [0…┌n/2┐-1] if B[i] <= C[j]
A[k] ← B[i]; i← i+1
MergeSort (B [0…└n/2┘-1])
else
MergeSort (C [0…┌n/2┐-1]) A[k] ← C[j]; j← j+1
Merge (B, C, A) k ← k+1
} }
if i=p
}
copy C [j…q-1] to A[k…p+q-1]
else
copy B [i…p-1] to A[k…p+q-1]
}
 Steps for Merge Sort
• Step 1: Create duplicate copies of sub-arrays to be sorted

• Step 2: Maintain current index of sub-arrays and main

array

• Step 3: Until we reach the end of either L or M, pick larger

among elements L and M and place them in the correct
position at A[p..r]

• Step 4: When we run out of elements in either L or M, pick

up the remaining elements and put in A[p..r]
Have we reached the end of any of the arrays?

No:
 Compare current elements of both arrays

 Copy smaller element into sorted array

 Move pointer of element containing smaller element

Yes:
 Copy all remaining elements of non-empty array
 Efficiency of Mergesort
Assuming that n is a power of 2,
The recurrence relation for the number of key comparisons C(n) is

the number of key comparisons performed during

Analyse
the merging stage
• At each step, exactly one comparison is made, after
which the total number of elements in the two arrays still
needing to be processed is reduced by 1.

• In the worst case, neither of the two arrays becomes

empty before the other one contains just one element

• e.g., smaller elements may come from the alternating

arrays
Therefore, for the worst case,

the recurrence is

Hence, according to the Master Theorem

it is easy to find the exact solution to the worst-case recurrence for n = 2𝑘 :

• The number of key comparisons made by mergesort in the worst case comes very

close to the theoretical 𝑚𝑖𝑛𝑖𝑚𝑢𝑚2 that any general comparison-based sorting algorithm

can have.

• For large n, the number of comparisons made by this algorithm in the average case

turns out to be about 0.25n less

• Hence is also in Θ(n log n).

• Advantage of mergesort over quicksort and heapsort is its stability

• Disadvantage of mergesort is the linear amount of extra storage the algorithm requires.
• There are two main ideas leading to several variations of mergesort.

• First, the algorithm can be implemented bottom up by merging pairs

of the array’s elements, then merging the sorted pairs, and so on.

• This avoids the time and space overhead of using a stack to handle
recursive calls.

• Second, we can divide a list to be sorted in more than two parts, sort
each recursively, and then merge them together.

• This scheme, which is particularly useful for sorting files residing on

secondary memory devices, is called multiway mergesort
 Quicksort
• It divides its input elements according to their position in the array,
quicksort divides them according to their value

• In a partition the array all the elements to the left of some element
A[s] are less than or equal to A[s],

• and all the elements to the right of A[s] are greater than or equal to
it: (A[s] - Pivot element)
 Quicksort
Step 1 − Choose the highest index value has pivot

Step 2 − Take two variables to point left and right of the list excluding pivot S

Step 3 − left points to the low index

Step 4 − right points to the high

Step 5 − while value at left is less than pivot move right

Step 6 − while value at right is greater than pivot move left

Step 7 − if both step 5 and step 6 does not match swap left and right

Step 8 − if left ≥ right, the point where they met is new pivot
 Quicksort
Step 1 - Consider the first element of the list as pivot
(i.e., Element at first position in the list).
Step 2 - Define two variables i and j.
Set i and j to first and last elements of the list respectively.
Step 3 - Increment i until list[i] > pivot then stop.
Step 4 - Decrement j until list[j] < pivot then stop.
Step 5 - If i < j then exchange list[i] and list[j].
Step 6 - Repeat steps 3,4 & 5 until i > j.
Step 7 - Exchange the pivot element with list[j] element.
 Quicksort
• After a partition is achieved, A[s] will be in its final position in the
sorted array, and continue sorting the two subarrays to the left and
to the right of A[s] independently

• In mergesort: there, the division of the problem into two

subproblems is immediate and the entire work happens in
combining their solutions;

• In quick sort , the entire work happens in the division stage, with no
work required to combine the solutions to the subproblems
ALGORITHM Quicksort
ALGORITHM HoarePartition(A[l..r])
 Quicksort
• Partition A[0..n − 1]

• Its subarray A[l..r](0 ≤ l <r ≤ n − 1) by the more sophisticated method

suggested by C.A.R. Hoare, the prominent British computer
scientist who invented quicksort

• Select a pivot—an element with respect to whose value we are

going to divide(Separate) the subarray.

• There are several different strategies for selecting a pivot

• The simplest strategy of selecting the pivot is , taking subarray’s

first element: p = A[l].
 Quicksort

• Scan the subarray from both ends, comparing the subarray’s elements to
the pivot.

• The left-to-right scan, denoted below by index pointer i, starts with the
second element.

• Since we want elements smaller than the pivot to be in the left part of the
subarray, this scan skips over elements that are smaller than the pivot and
stops upon encountering the first element greater than or equal to the pivot.

• The right-to-left scan, denoted below by index pointer j, starts with the last
element of the subarray.

• Since we want elements larger than the pivot to be in the right part of the
subarray, this scan skips over elements that are larger than the pivot and
stops on encountering the first element smaller than or equal to the pivot.
 Quicksort

• After both scans stop, three situations may arise, depending on

whether or not the scanning indices have crossed.

• If scanning indices i and j have not crossed, i.e., i < j, we simply

exchange A[i] and A[j ] and resume the scans by incrementing i and
decrementing j, respectively:
 Quicksort
• If the scanning indices have crossed over, i.e., i > j, we will have
partitioned the subarray after exchanging the pivot with A[j ]:

Finally, if the scanning indices stop while pointing to the same element,
i.e., i = j, the value they are pointing to must be equal to p (why?).
Thus, we have the subarray partitioned, with the split position s = i = j :

We can combine the last case with the case of crossed-over indices
(i > j ) by exchanging the pivot with A[j ] whenever i ≥ j .
 Quicksort

• Index i can go out of the subarray’s bounds in this pseudocode.

• Rather than checking for this possibility every time index i is

incremented, we can append to array A[0..n − 1] a “sentinel” that
would prevent index i from advancing beyond position n.

• Discussion of quicksort’s efficiency by noting that the number of key

comparisons made before a partition is achieved is n + 1 if the
scanning indices cross over and n if they coincide
 Quicksort

• In the worst case, all the splits will be skewed to the extreme: one of the two

subarrays will be empty, and the size of the other will be just 1 less than the

size of the subarray being partitioned.

• This unfortunate situation will happen, in particular, for increasing arrays,

i.e., for inputs for which the problem is already solved!

• Indeed, if A[0..n − 1] is a strictly increasing array and we use A[0] as the

pivot, the left-to-right scan will stop on A[1] while the right-to-left scan will go

all the way to reach A[0], indicating the split at position 0

 Quicksort
• Because of quicksort’s importance, there have been persistent efforts over

the years to refine the basic algorithm.

• Among several improvements discovered by researchers are: better pivot

selection methods such as randomized quicksort that uses a random element

or the median-of-three method that uses the median of the leftmost,

rightmost, and the middle element of the array switching to insertion sort on

very small subarrays (between 5 and 15 elements for most computer

systems) or not sorting small subarrays at all and finishing the algorithm with

insertion sort applied to the entire nearly sorted array modifications of the

partitioning algorithm such as the three-way partition into segments smaller

than, equal to, and larger than the pivot

QUICKSORT (array A, int m, int n)
1 if (n > m)
2 then
3 i ← a random index from [m,n]
4 swap A [i] with A[m]
5 o ← PARTITION (A, m, n)
6 QUICKSORT (A, m, o - 1)
7 QUICKSORT (A, o + 1, n)
PARTITION (array A, int m, int n)
1 x ← A[m]
2o←m
3 for p ← m + 1 to n
4 do if (A[p] < x)
5 then o ← o + 1
6 swap A[o] with A[p]
7 swap A[m] with A[o]
8 return o
 Binary Tree Traversals and Related
Properties

• A binary tree T is defined as a finite set of nodes

that is either empty or consists of a root and two
disjoint binary trees TL and TR called,
respectively, the left and right subtree of the root.

• A binary tree is a special case of an ordered tree

Standard representation of a binary tree
• Problems about binary trees can be solved by applying the

divide-and-conquer technique.

• Consider a recursive algorithm for computing the height of

a binary tree.

• Height is defined as the length of the longest path from the

root to a leaf.

• It can be computed as the maximum of the heights of the

root’s left and right subtrees plus 1.

• It is convenient to define the height of the empty tree as −1.

Recursive algorithm
• Measure the problem’s instance size by the
number of nodes n(T ) in a given binary tree T .

• The number of comparisons made to compute

the maximum of two numbers and the number of
additions A(n(T )) made by the algorithm are the
same.

• Recurrence relation for A(n(T )):

• Addition is not the most frequently executed operation of this
algorithm.

• For the empty tree, the comparison T = ∅ is executed once but there
are no additions, and for a single-node tree, the comparison and
addition numbers are 3 and 1, respectively.

• It helps in the analysis of tree algorithms to draw the tree’s extension

by replacing the empty subtrees by special nodes.

• The extra nodes are called external; the original nodes (shown by
little circles) are called internal.

• By definition, the extension of the empty binary tree is a single

external node.
• Height algorithm makes exactly one addition for every
internal node of the extended tree, and it makes one
comparison to check whether the tree is empty for every
internal and external node.

• Therefore, to ascertain the algorithm’s efficiency, the

number of external nodes of an extended binary tree
with n internal nodes required.

• The number of external nodes x is always 1 more than

the number of internal nodes n:
• To prove equality, consider the total number of nodes,
both internal and external.

• Every node, except the root, is one of the two children of

an internal node,

• Equation which implies equality is

• Equality also applies to any nonempty full binary tree, in
which, by definition, every node has either zero or two
children: for a full binary tree, n and x denote the
numbers of parental nodes and leaves, respectively.

• In Algorithm Height, the number of comparisons to check

whether the tree is empty is

and the number of additions is

• Divide-and-conquer algorithms for binary trees are

preorder, inorder, and postorder.

• All three traversals visit nodes of a binary tree recursively,

i.e., by visiting the tree’s root and its left and right subtrees.

• In the preorder traversal, the root is visited before the left

and right subtrees are visited (in that order).

• In the inorder traversal, the root is visited after visiting its

left subtree but before visiting the right subtree.

• In the postorder traversal, the root is visited after visiting

the left and right subtrees (in that order)

• Not all questions about binary trees require traversals of

both left and right subtrees.

• For example, the search and insert operations for a

binary search tree require processing only one of the two

subtrees.

• It is an example of the variable-size-decrease technique

Binary Tree and its Traversals
Binary Tree and its Traversals
 Multiplication of Large Integers and
Strassen’s Matrix Multiplication
• Two algorithms for seemingly straight forward
tasks:
(i) Multiplying two integers
(ii) Multiplying two square matrices
• Both achieve a better asymptotic efficiency by
ingenious application of the divide-and
conquer technique
 Multiplication of Large Integers

• Modern cryptography, require manipulation of integers that are

over 100 decimal digits long.

• Integers are too long to fit in a single word of a modern

computer, they require special treatment.

• This practical need supports investigations of algorithms for

efficient manipulation of large integers.

• Outline an algorithm for multiplying such numbers.

• compute the middle term with just one digit
multiplication by taking advantage of the
products 2 ∗ 1 and 3 ∗ 4 that need to be
computed anyway:
• 2 ∗ 4 + 3 ∗ 1 = (2 + 3) ∗ (1 + 4) − 2 ∗ 1 − 3 ∗ 4.
• Of course, there is nothing special about the
numbers we just multiplied.
• For any pair of two-digit numbers
• Obviously, if we use the conventional pen-and-pencil algorithm for
multiplying two n-digit integers, each of the n digits of the first number is
multiplied by each of the n digits of the second number for the total of 𝑛2
digit multiplications.

• (If one of the numbers has fewer digits than the other, we can pad the
shorter number with leading zeros to equalize their lengths.)

• Though it might appear that it would be impossible to design an algorithm

with fewer than 𝑛2 digit multiplications, this turns out not to be the case.

• The miracle of divide-and-conquer comes to the rescue to accomplish this

feat.
 To demonstrate the basic idea of the algorithm, let us
start with a case of two-digit integers, say, 23 and 14.

These numbers can be represented as follows:

 23 = 2.101 +3. 100 and 14=1.101 + 4* 100

 23 *14 = (2.𝟏𝟎𝟏 +3. 𝟏𝟎𝟎 ) *(1.𝟏𝟎𝟏 + 4* 𝟏𝟎𝟎 )

 = (2*1) 102 + (2*4 + 3*1) 101 + (3*4) 100

 The last formula yields the correct answer of 322

For any pair of two-digit numbers

a = 𝑎1 𝑎0 and b = 𝑏1 𝑏0 ,

their product c can be computed by the formula

c = a ∗ b = 𝑐2 102 + 𝑐1 101 + 𝑐0

where 𝒄𝟐 = 𝒂𝟏 ∗ 𝒃𝟏 is the product of their first digits,

𝒄𝟎 = 𝒂𝟎 ∗𝒃𝟎 is the product of their second digits,

𝒄𝟏 = (𝒂𝟏 + 𝒂𝟎 ) ∗ (𝒃𝟏 + 𝒃𝟎 ) − (𝒄𝟐 + 𝒄𝟎 ) is the product

of the sum of the a’s digits and the sum of the b’s
• Multiplying two n-digit integers a and b where n is a positive even number.

• Divide both numbers in the middle to take advantage of the divide-and-conquer

technique.

• First half of the a’s digits by 𝒂𝟏 and the second half by 𝒂𝟎 ;

• For b, the notations are 𝒃𝟏 and 𝒃𝟎 , respectively.

• In these notations, a = 𝒂𝟏 𝒂𝟎 implies that

a = 𝒂𝟏 𝟏𝟎𝒏/𝟐 + 𝒂𝟎 and

b = 𝒃𝟏 𝒃𝟎 implies that

b = 𝒃𝟏 𝟏𝟎𝒏/𝟐 + 𝒃𝟎 .

• Therefore, taking advantage of the same trick we used for two-digit numbers,

we get

• 𝒂 = 𝒂𝟏 𝒂𝟎 and b = 𝒃𝟏 𝒃𝟎 ,
c= ɑ*b= ( 𝑎1 10𝑛/2 + 𝑎0 ) * (𝑏1 10𝑛/2 + 𝑏0 )
= (𝑎1 ∗ 𝑏1 )10𝑛 + (𝑎1 *𝑏0 + 𝑎0 * 𝑏1 ) 10𝑛/2 + (𝑎0 *𝑏0 )

= 𝑐2 10𝑛 + 𝑐1 10𝑛/2 + 𝑐0

where 𝑐2 = 𝑎1 ∗ 𝑏1 is the product of their first digits,

𝑐0 = 𝑎0 ∗ 𝑏0 is the product of their second digits,
𝑐1 = (𝑎1 + 𝑎0 ) ∗ (𝑏1 + 𝑏0 ) − (𝑐2 + 𝑐0 )

is the product of the sum of the a’s haves and the sum of the b’s
haves minus the sum of 𝑐2 and 𝑐0 .
• If n/2 is even, we can apply the same method for computing the products
c2, c0, and c1.

• Thus, if n is a power of 2, we have a recursive algorithm for computing the

product of two n-digit integers. In its pure form, the recursion is stopped
when n becomes 1.

• It can also be stopped when we deem n small enough to multiply the

numbers of that size directly

• Since multiplication of n-digit numbers requires three multiplications of n/2-

digit numbers, the recurrence for the number of multiplications M(n) is

• M(n) = 3M(n/2) for n > 1, M(1) = 1

• Let A(n) be the number of digit additions and
subtractions executed by the above algorithm in
multiplying two n-digit decimal integers.
• Besides 3A(n/2) of these operations needed to
compute the three products of n/2-digit numbers, the
above formulas require five additions and one
subtraction.
• Hence, we have the recurrence
• A(n) = 3A(n/2) + cn for n > 1, A(1) = 1
• Since this savings in the number of multiplications was achieved at
the expense of making extra additions, we must check the number
of additions A(n) made by Strassen’s algorithm.

• To multiply two matrices of order n > 1, the algorithm needs to

multiply seven matrices of order n/2 and make 18 additions/
subtractions of matrices of size n/2; when n = 1, no additions are
made since two numbers are simply multiplied. These observations
yield the following recurrence relation:
• According to the Master Theorem,
.
• In other words, the number of additions has the same
order of growth as the number of multiplications.
• This puts Strassen’s algorithm in which is a
better efficiency class than of the brute-force
method.