General method:

 Given a function to compute on ‘n’ inputs the divide-and-conquer strategy suggests

splitting the inputs into ‘k’ distinct subsets, 1<k<=n, yielding ‘k’ sub problems.

 These sub problems must be solved, and then a method must be found to combine sub
solutions into a solution of the whole.

 If the sub problems are still relatively large, then the divide-and-conquer strategy can
possibly be reapplied.

 Often the sub problems resulting from a divide-and-conquer design are of the same
type as the original problem.

 For those cases the re application of the divide-and-conquer principle is naturally

expressed by a recursive algorithm.

 D And C(Algorithm) is initially invoked as D and C(P), where ‘p’ is the problem to be
solved.

 Small(P) is a Boolean-valued function that determines whether the i/p size is small
enough that the answer can be computed without splitting.

 If this so, the function ‘S’ is invoked.

 Otherwise, the problem P is divided into smaller sub problems.

 These sub problems P1, P2 …Pk are solved by recursive application of D And C.

 Combine is a function that determines the solution to p using the solutions to the ‘k’
sub problems.

 If the size of ‘p’ is n and the sizes of the ‘k’ sub problems are n1, n2 ….nk,
respectively, then the computing time of D And C is described by the recurrence
relation.

T(n)= { g(n) n small

T(n1)+T(n2)+……………+T(nk)+f(n); otherwise.

Where T(n)  is the time for D And C on any I/p of size ‘n’.
g(n)  is the time of compute the answer directly for small I/ps.
f(n)  is the time for dividing P & combining the solution to sub problems.

1. Algorithm D And C(P)

2. {
3. if small(P) then return S(P);
4. else
5. {
6. divide P into smaller instances
P1, P2… Pk, k>=1;
7. Apply D And C to each of these sub problems;
8. return combine (D And C(P1), D And C(P2),…….,D And C(Pk));
9. }
10. }

 The complexity of many divide-and-conquer algorithms is given by recurrences

of the form
T(n) = { T(1) n=1
AT(n/b)+f(n) n>1
 Where a & b are known constants.
 We assume that T(1) is known & ‘n’ is a power of b(i.e., n=bk)
 One of the methods for solving any such recurrence relation is called the substitution
method.
 This method repeatedly makes substitution for each occurrence of the function. T is
the Right-hand side until all such occurrences disappear.

Example:
1) Consider the case in which a=2 and b=2. Let T(1)=2 & f(n)=n.
We have,
T(n) = 2T(n/2)+n
= 2[2T(n/2/2)+n/2]+n
= [4T(n/4)+n]+n
= 4T(n/4)+2n
= 4[2T(n/4/2)+n/4]+2n
= 4[2T(n/8)+n/4]+2n
= 8T(n/8)+n+2n
= 8T(n/8)+3n
*
*
*

 In general, we see that T(n)=2iT(n/2i )+in., for any log n >=I>=1.

 T(n) =2log n T(n/2log n) + n log n

Corresponding to the choice of i=log n

 Thus, T(n) = 2log n T(n/2log n) + n log n

= n. T(n/n) + n log n
= n. T(1) + n log n [since, log 1=0, 20=1]
= 2n + n log n

BINARY SEARCH

1. Algorithm Bin search(a,n,x)

2. // Given an array a[1:n] of elements in non-decreasing
3. //order, n>=0,determine whether ‘x’ is present and
4. // if so, return ‘j’ such that x=a[j]; else return 0.
5. {
6. low:=1; high:=n;
7. while (low<=high) do
 8. {
9. mid:=[(low+high)/2];
10. if (x<a[mid]) then high;
11. else if(x>a[mid]) then
low=mid+1;
12. else return mid;
13. }
14. return 0;
15. }

 Algorithm, describes this binary search method, where Binsrch has 4I/ps a[], I , l & x.
 It is initially invoked as Binsrch(a,1,n,x)
 A non-recursive version of Binsrch is given below.
 This Bin search has 3 i/ps a, n, & x.
 The while loop continues processing as long as there are more elements left to check.
 At the conclusion of the procedure 0 is returned if x is not present, or ‘j’ is returned,
such that a[j]=x.
 We observe that low & high are integer Variables such that each time through the loop
either x is found or low is increased by at least one or high is decreased at least one.

 Thus we have 2 sequences of integers approaching each other and eventually low
becomes > than high & causes termination in a finite no. of steps if ‘x’ is not present.

Example:
1) Let us select the 14 entries.
-15,-6,0,7,9,23,54,82,101,112,125,131,142,151.
 Place them in a[1:14], and simulate the steps Bin search goes through as it searches for
different values of ‘x’.
 Only the variables, low, high & mid need to be traced as we simulate the algorithm.
 We try the following values for x: 151, -14 and 9.
for 2 successful searches & 1 unsuccessful search.

 Table. Shows the traces of Bin search on these 3 steps.

X=151 low high mid

1 14 7
8 14 11
12 14 13
14 14 14
Found

x=-14 low high mid

1 14 7
1 6 3
1 2 1
2 2 2
2 1 Not found

x=9 low high mid

1 14 7
1 6 3
 4 6 5
Found

Theorem: Algorithm Bin search (a, n, x) works correctly.

Proof:
We assume that all statements work as expected and that comparisons such as x>a[mid] are
appropriately carried out.

 Initially low =1, high= n, n>=0, and a[1]<=a[2]<=……..<=a[n].

 If n=0, the while loop is not entered and is returned.

 Otherwise we observe that each time thro’ the loop the possible elements to be
checked of or equality with x and a[low], a[low+1],……..,a[mid],……a[high].
 If x=a[mid], then the algorithm terminates successfully.
 Otherwise, the range is narrowed by either increasing low to (mid+1) or decreasing
high to (mid-1).
 Clearly, this narrowing of the range does not affect the outcome of the search.
 If low becomes > than high, then ‘x’ is not present & hence the loop is exited.

MERGE SORT

 As another example divide-and-conquer, we investigate a sorting algorithm that has

the nice property that is the worst case its complexity is O(n log n)
 This algorithm is called merge sort
 We assume throughout that the elements are to be sorted in non-decreasing order.
 Given a sequence of ‘n’ elements a[1],…,a[n] the general idea is to imagine then split
into 2 sets a[1],…..,a[n/2] and a[[n/2]+1],….a[n].
 Each set is individually sorted, and the resulting sorted sequences are merged to
produce a single sorted sequence of ‘n’ elements.
 Thus, we have another ideal example of the divide-and-conquer strategy in which the
splitting is into 2 equal-sized sets & the combining operation is the merging of 2 sorted
sets into one.

Algorithm For Merge Sort:

1. Algorithm MergeSort(low,high)
2. //a[low:high] is a global array to be sorted
3. //Small(P) is true if there is only one element
4. //to sort. In this case the list is already sorted.
5. {
6. if (low<high) then //if there are more than one element
7. {
8. //Divide P into subproblems
9. //find where to split the set
10. mid = [(low+high)/2];
11. //solve the subproblems.
12. mergesort (low,mid);
13. mergesort(mid+1,high);
14. //combine the solutions .
 15. merge(low,mid,high);
16. }
17. }

Algorithm: Merging 2 sorted sub arrays using auxiliary storage.

1. Algorithm merge(low,mid,high)
2. //a[low:high] is a global array containing
3. //two sorted subsets in a[low:mid]
4. //and in a[mid+1:high].The goal is to merge these 2 sets into
5. //a single set residing in a[low:high].b[] is an auxiliary global array.
6. {
7. h=low; I=low; j=mid+1;
8. while ((h<=mid) and (j<=high)) do
9. {
10. if (a[h]<=a[j]) then
11. {
12. b[I]=a[h];
13. h = h+1;
14. }
15. else
16. {
17. b[I]= a[j];
18. j=j+1;
19. }
20. I=I+1;
21. }
22. if (h>mid) then
23. for k=j to high do
24. {
25. b[I]=a[k];
26. I=I+1;
27. }
28. else
29. for k=h to mid do
30. {
31. b[I]=a[k];
32. I=I+1;
33. }
34. for k=low to high do a[k] = b[k];
35. }

 Consider the array of 10 elements a[1:10] =(310, 285, 179, 652, 351, 423, 861, 254,
450, 520)

 Algorithm Mergesort begins by splitting a[] into 2 sub arrays each of size five (a[1:5]
and a[6:10]).
 The elements in a[1:5] are then split into 2 sub arrays of size 3 (a[1:3] ) and 2(a[4:5])
 Then the items in a a[1:3] are split into sub arrays of size 2 a[1:2] & one(a[3:3])
 The 2 values in a[1:2} are split to find time into one-element sub arrays, and now the
merging begins.
 (310| 285| 179| 652, 351| 423, 861, 254, 450, 520)

 Where vertical bars indicate the boundaries of sub arrays.

Elements a[1] and a[2] are merged to yield,

(285, 310|179|652, 351| 423, 861, 254, 450, 520)

 Then a[3] is merged with a[1:2] and

(179, 285, 310| 652, 351| 423, 861, 254, 450, 520)

 Next, elements a[4] & a[5] are merged.

(179, 285, 310| 351, 652 | 423, 861, 254, 450, 520)

 And then a[1:3] & a[4:5]

(179, 285, 310, 351, 652| 423, 861, 254, 450, 520)

 Repeated recursive calls are invoked producing the following sub arrays.
(179, 285, 310, 351, 652| 423| 861| 254| 450, 520)

 Elements a[6] &a[7] are merged.

Then a[8] is merged with a[6:7]

(179, 285, 310, 351, 652| 254,423, 861| 450, 520)

 Next a[9] &a[10] are merged, and then a[6:8] & a[9:10]
(179, 285, 310, 351, 652| 254, 423, 450, 520, 861 )

 At this point there are 2 sorted sub arrays & the final merge produces the fully
sorted result.
(179, 254, 285, 310, 351, 423, 450, 520, 652, 861)

 If the time for the merging operations is proportional to ‘n’, then the computing time
for merge sort is described by the recurrence relation.

T(n) = { a n=1,’a’ a constant

2T(n/2)+cn n>1,’c’ a constant.

 When ‘n’ is a power of 2, n= 2k, we can solve this equation by successive substitution.

T(n) =2(2T(n/4) +cn/2) +cn

= 4T(n/4)+2cn
= 4(2T(n/8)+cn/4)+2cn
*
*
= 2k T(1)+kCn.
= an + cn log n.

 It is easy to see that if sk<n<=2k+1, then T(n)<=T(2k+1). Therefore,

T(n)=O(n log n)
 QUICK SORT

 The divide-and-conquer approach can be used to arrive at an efficient sorting method

different from merge sort.

 In merge sort, the file a[1:n] was divided at its midpoint into sub arrays which were
independently sorted & later merged.

 In Quick sort, the division into 2 sub arrays is made so that the sorted sub arrays do not
need to be merged later.

 This is accomplished by rearranging the elements in a[1:n] such that a[I]<=a[j] for all I
between 1 & n and all j between (m+1) & n for some m, 1<=m<=n.

 Thus the elements in a[1:m] & a[m+1:n] can be independently sorted.

 No merge is needed. This rearranging is referred to as partitioning.

 Function partition of Algorithm accomplishes an in-place partitioning of the elements

of a[m:p-1]

 It is assumed that a[p]>=a[m] and that a[m] is the partitioning element. If m=1 & p-
1=n, then a[n+1] must be defined and must be greater than or equal to all elements in
a[1:n]

 The assumption that a[m] is the partition element is merely for convenience, other
choices for the partitioning element than the first item in the set are better in practice.

 The function interchange (a,i,j) exchanges a[i] with a[j].

Algorithm: Partition the array a[m:p-1] about a[m]

1. Algorithm Partition(a,m,p)
2. //within a[m],a[m+1],…..,a[p-1] the elements
3. // are rearranged in such a manner that if
4. //initially t=a[m],then after completion
5. //a[q]=t for some q between m and
6. //p-1,a[k]<=t for m<=k<q, and
7. //a[k]>=t for q<k<p. q is returned
8. //Set a[p]=infinite.
9. {
10. v=a[m]; i=m; j=p;
11. repeat
12. {
13. repeat
14. i=i+1;
15. until(a[i]>=v);
16. repeat
17. j=j-1;
18. until(a[j]<=v);
 19. if (i<j) then interchange(a,i.j);
20. }until(i>=j);
21. a[m]=a[j]; a[j]=v;
22. return j;
23. }

1. Algorithm Interchange(a,i,j)
2. //Exchange a[i] with a[j]
3. {
4. p=a[i];
5. a[i]=a[j];
6. a[j]=p;
7. }

Algorithm: Sorting by Partitioning

1. Algorithm Quicksort(p,q)
2. //Sort the elements a[p],….a[q] which resides
3. //is the global array a[1:n] into ascending
4. //order; a[n+1] is considered to be defined
5. // and must be >= all the elements in a[1:n]
6. {
7. if(p<q) then // If there are more than one element
8. {
9. // divide p into 2 subproblems
10. j=partition(a,p,q+1);
11. //’j’ is the position of the partitioning element.
12. //solve the subproblems.
13. quicksort(p,j-1);
14. quicksort(j+1,q);
15. //There is no need for combining solution.
16. }
17. }

Divide and Conquer (DAC) approach has three steps at each level of recursion:

1. Divide the problem into number of smaller units called sub-problems.

2. Conquer (Solve) the sub-problems recursively.

3. Combine the solutions of all the sub-problems into a solution for the original
problem.
Maximum and Minimum:

1. Let us consider simple problem that can be solved by the divide-and conquer
technique.

2. The problem is to find the maximum and minimum value in a set of ‘n’ elements.
 3. By comparing numbers of elements, the time complexity of this algorithm can be
analyzed.

4. Hence, the time is determined mainly by the total cost of the element comparison.

Explanation:
a. Straight MaxMin requires 2(n-1) element comparisons in the best, average & worst cases.
b. By realizing the comparison of a [i]max is false, improvement in a algorithm can be done.
c. Hence we can replace the contents of the for loop by, If (a [i]> Max) then Max = a [i]; Else if
(a [i]< 2(n-1)
d. On the average a[i] is > max half the time, and so, the avg. no. of comparison is 3n/2-1.
A Divide and Conquer Algorithm for this problem would proceed as follows:
a. Let P = (n, a [i],……,a [j]) denote an arbitrary instance of the problem.
b. Here ‘n’ is the no. of elements in the list (a [i],….,a[j]) and we are interested in finding the
maximum and minimum of the list.
c. If the list has more than 2 elements, P has to be divided into smaller instances.
d. For example, we might divide ‘P’ into the 2 instances, P1=([n/2],a[1],……..a[n/2]) & P2=
( n-[n/2], a[[n/2]+1],….., a[n]) After having divided ‘P’ into 2 smaller sub problems, we can
solve them by recursively invoking the same divide-and-conquer algorithm.

Algorithm:
Example:

A 1 2 3 4 5 6 7 8 9

Values 22 13 -5 -8 15 60 17 31 47

Tree Diagram:
i. As shown in figure 4, in this Algorithm, each node has 4 items of information: i, j, max &
min.
ii. In figure 4, root node contains 1 & 9 as the values of i& j corresponding to the initial call
to MaxMin.
iii. This execution produces 2 new calls to MaxMin, where i& j have the values 1, 5 & 6, 9
respectively & thus split the set into 2 subsets of approximately the same size.
iv. Maximum depth of recursion is 4.
Complexity:
If T(n) represents this no., then the resulting recurrence relations is
T (n)=T([n/2]+T[n/2]+2 n>2
1 n=2
1 n=1
When ‘n’ is a power of 2, n=2k for some positive integer ‘k’, then
T (n) = 2T(n/2) +2
= 2(2T(n/4)+2)+2
= 4T(n/4)+4+2
*
*
= 2k-1 T (2) + Σ 1 ≤ I ≤ k-1 ≤ 2i
= 2k-1+ 2k - 2
T(n) = (3n/2) – 2
Note that (3n/2) - 2 is the best-average and worst-case no. of comparisons when ‘n’ is a
power of 2.

Brute Force Approach:

Brute Force
Brute force is a straightforward approach to solving a problem, usually directly based on the
problem’s statement and definitions of the concepts involved. If there are n items to choose
from, then there will be 2n possible combinations of items for the knapsack. An
item is either chosen or not chosen. A bit string of 0’s and 1’s is generated which is of length
n. If the ith symbol of a bit string is 0, then the ith item is not chosen and if it is 1, the ith item is
chosen.

As a simple example, consider searching through a sorted list of items for some target. Brute
force would simply start at the first item, see if it is the target, and if not sequentially move to
the next until we either find the target or hit the end of the list. For small lists this is no
problem (and would actually be the preferred solution), but for extremely large lists we could
use more efficient techniques

ALGORITHM BruteForce (Weights [1 … N], Values [1 … N], A[1…N])

//Finds the best possible combination of items for the KP

//Input: Array Weights contains the weights of all items Array Values contains the values of
all items

Array A initialized with 0s is used to generate the bit strings

//Output: Best possible combination of items in the knapsack bestChoice [1 .. N]

for i = 1 to 2n do

j ← n tempWeight ← 0

tempValue ← 0

while ( A[j] != 0 and j > 0)

A[j] ← 0 j ← j – 1

A[j] ← 1

for k ← 1 to n do

if (A[k] = 1) then

tempWeight ← tempWeight + Weights[k] tempValue ← tempValue + Values[k]

if ((tempValue > bestValue) AND (tempWeight ≤ Capacity)) then bestValue ← tempValue

bestWeight ← tempWeight
bestChoice ← A return bestChoice

Complexity
2n 1 n 2n
Σ [Σ+ Σ]= Σ[{1+..+1}(n times) +{1+..+1}(n times)]

i=1 j=n k=1 i=1

= (2n)* [1+1+1…..+1] (2n times)

= O(2n*2n)

= O(n*2n)

Knapsack Problem

Given n items:
weights: w1 w2 … wn
values: v1 v2 … vn
A knapsack of capacity W
Find the most valuable subset of the items that fit into the knapsack (sum of weights ≤ W)

Example:
item weight value Knapsack capacity W=16
1 2 $20
2 5 $30
3 10 $50
4 5 $10

Subset Total weight Total value

{1} 2 $20
{2} 5 $30
{3} 10 $50
{4} 5 $10
{1,2} 7 $50
{1,3} 12 $70
{1,4} 7 $30
{2,3} 15 $80
{2,4} 10 $40
 {3,4} 15 $60
{1,2,3} 17 not feasible
{1,2,4} 12 $60
{1,3,4} 17 not feasible
{2,3,4} 20 not feasible
{1,2,3,4} 22 not feasible

Therefore, the complexity of the Brute Force algorithm is O (n2n). Since the complexity of
this algorithm grows exponentially, it can only be used for small instances of the KP.
Otherwise, it does not require much programming effort in order to be implemented.

Besides the memory used to store the values and weights of all items, this algorithm requires a
two one dimensional arrays (A[] and bestChoice[]).

What is the Travelling Salesman Problem (TSP)?

Travelling Salesman Problem (TSP) is a classic combinatorics problem of theoretical
computer science. The problem asks to find the shortest path in a graph with the condition of
visiting all the nodes only one time and returning to the origin city.
The problem statement gives a list of cities along with the distances between each city.
Objective: To start from the origin city, visit other cities only once, and return to the original
city again. Our target is to find the shortest possible path to complete the round-trip route.

Example of TSP
Here a graph is given where 1, 2, 3, and 4 represent the cities, and the weight associated with
every edge represents the distance between those cities.

The goal is to find the shortest possible path for the tour that starts from the origin city,
traverses the graph while only visiting the other cities or nodes once, and returns to the origin
city.
For the above graph, the optimal route is to follow the minimum cost path: 1-2-4-3-1. And this
shortest route would cost 10+25+30+15 =80

Different Solutions to Travelling Salesman Problem

Travelling Salesman Problem (TSP) is classified as a NP-hard problem due to having no
polynomial time algorithm. The complexity increases exponentially by increasing the number
of cities.
There are multiple ways to solve the traveling salesman problem (tsp). Some popular solutions
are:
The brute force approach is the naive method for solving traveling salesman problems. In
this approach, we first calculate all possible paths and then compare them. The number of
paths in a graph consisting of n cities is n! It is computationally very expensive to solve the
traveling salesman problem in this brute force approach.

Some Extra points to be noted:

Travelling Salesman Problem (TSP) : Given a set of cities and distances between every pair of
cities, the problem is to find the shortest possible route that visits every city exactly once and
returns to the starting point.
Note the difference between Hamiltonian Cycle and TSP. The Hamiltonian cycle problem is
to find if there exists a tour that visits every city exactly once. Here we know that Hamiltonian
Tour exists (because the graph is complete) and in fact, many such tours exist, the problem is
to find a minimum weight Hamiltonian Cycle.
For example, consider the graph shown in the figure on the right side. A TSP tour in the graph
is 1-2-4-3-1. The cost of the tour is 10+25+30+15 which is 80.
The problem is a famous NP-hard problem. There is no polynomial-time known solution for
this problem.

Traveling Salesman Problem

Find shortest Hamiltonian circuit in a weighted connected graph.

Tour Cost .
a→b→c→d→a 2+3+7+5 = 17
 a→b→d→c→a 2+4+7+8 = 21
a→c→b→d→a 8+3+4+5 = 20
a→c→d→b→a 8+7+4+2 = 21
a→d→b→c→a 5+4+3+8 = 20
a→d→c→b→a 5+7+3+2 = 17

The Convex Hull Problem

In this problem, we want to compute the convex hull of a set of points? What does this
mean?

 Formally: It is the smallest convex set containing the points. A convex set is one in
which if we connect any two points in the set, the line segment connecting these
points must also be in the set.
 Informally: It is a rubber band wrapped around the "outside" points.

Here is a picture from:

http://www.cs.princeton.edu/~ah/alg_anim/version1/ConvexHull.html
It is an applet so you can play with it to see what a convex hull is if you like.

Theorem: The convex hull of any set S of n>2 points (not all collinear) is a convex
polygon with the vertices at some of the points of S.

How could you write a brute-force algorithm to find the convex hull?

In addition to the theorem, also note that a line segment connecting two points P1 and P2 is
a part of the convex hull’s boundary if and only if all the other points in the set lie on the
same side of the line drawn through these points. With a little geometry:

(x2,y2
)

(x1,y1
 Line defined by a=(y2-y1), b=(x1-x2),
c=(x1y2 – x2y1)
For all points above the line, ax + by > c, while for all points below the line, ax + by < c.
Using these formulas, we can determine if two points are on the boundary to the convex
hull.
 DIVIDE AND CONQUER UNIT - II

High level pseudocode for the algorithm then becomes:

for each point Pi

for each point Pj where Pj  Pi
Compute the line segment for Pi and Pj
for every other point Pk where Pk  Pi and Pk  Pj
If each Pk is on one side of the line segment, label Pi
and Pj in the convex hull

R SATHYA PRAKASH CSED MVSREC Page 18