DIVIDE AND CONQUER

I. General Method
• Given a function to compute on inputs the divide-and-
conquer strategy, splits the inputs into distinct subsets,
1 < ≤ , yielding subproblems.
• These sub problems must be solved, and then a method
must be found to combine sub solutions into a solution of
the whole.
• If the sub problems are still relatively large, then the divide-
and-conquer strategy can possibly be reapplied.
• Often the sub problems resulting from a divide-and-conquer
design are of the same type as the original problem. Hence,
reapplication of the divide-and-conquer principle is naturally
expressed by a recursive algorithm.
• Now smaller and smaller sub problems of the same kind are
generated until eventually sub problems that are small
enough to be solved without splitting are produced.
• The control abstraction for the general method of divide-
and-conquer strategy is given in Algorithm 3.1.
• is a Boolean-valued function that determines whether
the input size is small enough that the answer can be
computed without splitting. If this is so, the function is
invoked. Otherwise the problem is divided into smaller sub
problems. These sub problems , ,…, are solved by
recursive applications of . is a function that
determines the solution to using the solutions to the sub
problems.
 • If the size of is and the sizes of the sub problems are
, , … , , respectively, then the computing time of is
described by the recurrence relation:
= (
+ + ⋯+ " + # $ℎ &'
where is the time for on any input of size and
is the time to compute the answer directly for small
inputs. The function # is the time for dividing and
combining the solutions to the sub problems.
• For divide-and-conquer based algorithms that produce sub
problems of the same type as the original problem, we describe
such algorithms using recursion.
• The complexity of many divide-and-conquer algorithms is
given by recurrences of the form
1 =1
=) (
* ++# >1

where and are known constants.

 BINARY SEARCH
• Let - , 1 ≤ ≤ , be a list of elements that are sorted in non
decreasing order. Consider the problem of determining whether a
given element . is present in the list. If . is present, we are to
determine a value / such that 0 = . If . is not in the list, then /
is to be set to zero.
• Let = , - , … , 1 , . denote an arbitrary instance of this search
problem ( is the number of elements in the list, - , … , 1 is the
list of elements, and . is the element searched for).
• Divide-and-conquer can be used to solve this problem. Let
be true if = 1. In this case, ) will take the value if
. = - , otherwise it will take the value 0. Then 1 = 2 1 .
• If has more than one element, it can be divided (or reduced)
into a new sub problem as follows. Pick an index 4 (in the
range 5 , 6 compare . with 7 . There are three possibilities:

i. . = 7 : In this case the problem is immediately solved.

ii. . < 7 : In this case . has to be searched for only in the sub
list - , -8 , … , 79 . Therefore, P reduces to
4, - , -8 , … , 79 , . .
iii. . > 7 : In this case the sub list to be searched is 78 , … , 1 .
reduces to − 4, 78 , … , 1, . .
The analysis just done applies to any sorted sequence containing 9 elements.
But the result we would prefer is a formula for n elements. A good way to derive
such a formula plus a better way to understand the algorithm is to consider
the sequence of values for mid that are produced by BinSearch for all possible
values of x. These values are nicely described using a binary decision tree in
which the value in each node is the value of mid.

• The first comparison is . with 576. If . < 576, then the next comparison
is with 536; similarly, if . > 576, then the next comparison is with 5116.
• Each path through the tree represents a sequence of comparisons in the
binary search method.
• If . is the present, then the algorithm will end at one of the circular
nodes that lists the index into the array where . was found.
• If . is not present, the algorithm will terminate at one of the square
nodes.
• Circular nodes are called internal nodes and square nodes are called
external nodes.
• If 2 9 ≤ < 2 , then all circular nodes are at levels 1,2, … , whereas all
square nodes are at levels and + 1.
• The number of element comparisons needed to terminate at a circular
node on level is whereas the number of element comparisons needed
to terminate at a square node at level is − 1.
In conclusion, the time complexity of binary search for successful
searches is ? log , and the time complexity of binary search for
unsuccessful searches is 2 log .
 MERGE SORT

• Given a sequence of elements 516, … , 5 6, the general idea

is to imagine them split into two sets 516, 6 … , 5H /2J and
E
CD + 1FG , … , 5 66.
• Eat set is individually sorted, and the resulted sorted
sequences are merged to produce a single sorted sequence of
elements.
 Figure 3.3 Tree of Calls of MergeSort(1,10) that represents the sequence of recursive
calls that are produced by MergeSort when it is applied to ten elements.

Figure 3.4 Tree representing the calls to procedure Merge by MergeSort.

The time for the merging operation is proportional to , and the
computing time for merge sort can be described by the recurrence
relation.
=1
= K2 (
* ++L >1
2

where and L are constants. When is a power of 2, = 2 , we can

solve this equation by successive substitutions:
L
= 2 *2 * + + + + L
4 2
= 4 * + + 2L
4
L
= 4 *2 * + + + + 2L
8 4
⋮
=2 1 + L
= + L log

It is easy to see that if 2 < ≤2 8

, then ≤ 2 8
. Therefore

=? log
 QUICKSORT

• In merge sort, the file 51: 6 was divided at its midpoint into
sub arrays which were independently sorted and later merged.
• In quicksort, the division into two sub arrays is made so that
the sorted sub arrays do not need to be merged later.
• This is accomplished by rearranging the elements in
51: 6 such that 5 6 ≤ 5/6 for all between 1 and and all
/ between + 1 and for some , 1 ≤ ≤ . Thus, the
elements in 51: 6 and 5 + 1: 6 can be independently sorted.
No merge is needed.
• The rearrangement of the elements is accomplished by picking
some element of 5 6, say $ = 5 6, and then reordering the other
elements so that all elements appearing before $ in 51: 6 are
less than or equal to $ and all elements appearing after $ are
greater than or equal to $. This rearranging is referred to as
partitioning.
In analyzing, Quicksort, we count only the number of element
comparisons .

• Let us obtain the worst-case value QR S of Q S .

o The number of element comparisons in each call of
&$ $ , , T is at most T − + 1.
o Let & be the total number of elements in all the calls to
&$ $ at any level of recursion.
o At level one, only one call to &$ $ , 1, + 1 is made
and & = ;
o At level two at most two calls are made and & = − 1; and
so on.
o At each level of recursion, ? & element comparisons are
made by &$ $ .
o At each level, & is at least one less than the & at the
previous level as the partitioning elements of the previous
level are eliminated.
o Hence U is the sum on & as & varies from 2 to .
o U = 2 + 3 + 4 + ⋯+ = ?
 Average Case Time Complexity of Quick Sort QV S
WX
Assuming the partition element has an equal probability of being smallest
element.

Y = +1 + ∑ [ [E Y −1 + Y −
E
(1)

The number of element comparisons required by &$ $ on its first call is

+ 1. Note that Y 0 = Y 1 = 0.

Multiplying (1) by , we obtain

Y = + 1 + 25 Y 0 + Y 1 + ⋯+ Y −1 6 (2)

Replacing by − 1 in (2) gives,

−1 Y −1 = − 1 + 25 Y 0 + Y 1 + ⋯+ Y −2 6 (3)

Subtracting (3) from (2), we obtain

Y − −1 Y −1 =2 +2 Y −1 (4)

or

]^ E ]^ E9
= +
E8 E E8
(5)

Repeatedly using (5) to substitute for Y −1 , Y − 2 , …, we get

_^ ` _ `db e
c ^ 8
`ab ` `ab
_^ `de e e
c 8 8
`db ` `ab
_^ `df e e e
c 8 8 8 (6)
`de `db ` `ab
⋮
_ b b
c ^ 8 ∑fhgh`ab
e g
b
c ∑fhgh`ab
g

Since
E8
2 ∑i[ [E8 ≤j .= l +1 − l2
k

Finally, Y ≤2 +1 5 l +2 − l 26 = m S nop S
 NAÏVE MATRIX MULTIPLICATION

Let and q be two × matrices. The product matrix = q is also

an × matrix whose , /WX element is formed by taking the
elements in the WX row of and /WX column of q and multiplying
them to get

,/ = s , q ,/
[ [E

for all and / between 1 and . To compute , / using this formula,

we need multiplications. As the matrix has elements, the time
for the resulting matrix multiplication algorithm, which we refer to
as the conventional method is 2 i .
Recursive Algorithm for Matrix Multiplication
The divide-and-conquer strategy suggests another way to compute the product
of two × matrices.

For simplicity we assume that is a power of 2. In case is not a power of 2,

then enough rows and `columns of zeros can be added to both A and B so that
the resulting dimensions are a power of 2.

Imagine that A and B are each partitioned into four square sub matrices, each
E E
sub matrix having dimensions × .

Then the product AB can be computed by using the following formula:

To compute AB using the above formula, we need to perform eight

E E E E
multiplications of × matrices and four additions of × matrices. Since two
E E
× matrices can be added in time L for some constant c, the overall
computing time can be given by the following recurrence equation:

≤2
=) (
8 * ++L >2
2

The above recurrence equation when solved gives, t S = m Su .

 Strassen’s Matrix Multipliation

Since matrix multiplications are more expensive than matrix

additions ? i versus ? , we can attempt to reformulate the
equations for -0 so as to have fewer multiplications and possible
more additions.

Volker Strassen has discovered a way to compute -0 ′ using only 7

multiplications and 18 additions or subtractions as follows:

The resulting recurrence relation for is

≤2
=) (
7 * ++ >2
2

Working with this formula we get,