15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

UNIT V
BRANCH BOUND AND RANDOMIZED ALGORITHM
Branch and bound: 0/1 Knapsack - Travelling Salesman Problem. Randomized algorithm: Hiring
Problem - Matrix Chain Multiplication - Randomized Quick Sort. Introduction to PN problems -
Introduction to NP problems - NP Complete.

INTRODUCTION
General method:
The design technique known as branch and bound is very similar to backtracking (seen in
unit 4) in that it searches a tree model of the solution space and is applicable to a wide variety of
discrete combinatorial problems.
Each node in the combinatorial tree generated in the last Unit defines a problem state. All
paths from the root to other nodes define the state space of the problem.
Solution states are those problem states 's' for which the path from the root to 's' defines a tuple
in the solution space. The leaf nodes in the combinatorial tree are the solution states.
Answer states are those solution states 's' for which the path from the root to 's' defines a tuple
that is a member of the set of solutions (i.e,it satisfies the implicit constraints) of the problem.
The tree organization of the solution space is referred to as the state space tree.
A node which has been generated and all of whose children have not yet been generated is
called alive node
The live node whose children are currently being generated is called the E-node (node being
expanded).
A dead nodeis a generated node, which is not to be expanded further or all of whose children
have been generated.
Bounding functions are used to kill live nodes without generating all their children.
Depth first node generation with bounding function is called backtracking. State generation
methods in which the E-node remains the E-node until it is dead lead to branch-and-bound
method.
The term branch-and-bound refers to all state space search methods in which all children of
the E-node are generated before any other live node can become the E-node.
In branch-and-bound terminology breadth first search(BFS)- like state space search will be
called FIFO (First In First Output) search as the list of live nodes is a first -in-first -out list(or
queue).

Mrs. Selva Mary. G Page 1

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

A D-search (depth search) state space search will be called LIFO (Last In First Out) search, as
the list of live nodes is a last-in-first-out list (or stack).
Bounding functions are used to help avoid the generation of sub trees that do not contain an
answer node.
The branch-and-bound algorithms search a tree model of the solution space to get the
solution. However, this type of algorithms is oriented more toward optimization. An algorithm of
this type specifies a real -valued cost function for each of the nodes that appear in the search
tree.
Usually, the goal here is to find a configuration for which the cost function is minimized. The
branch-and-bound algorithms are rarely simple. They tend to be quite complicated in many
cases.

0/1 KNAPSACK PROBLEM

Consider the instance: M = 15, n = 4, (P1, P2, P3, P4) = (10, 10, 12, 18) and (w1, w2, w3, w4) = ( 2,
4, 6, 9).
0/1 knapsack problem can be solved by using branch and bound technique. In this problem we
will calculate lower bound and upper bound for each node.
Place first item in knapsack. Remaining weight of knapsack is 15 – 2 = 13. Place next item w2 in
knapsack and the remaining weight of knapsack is 13 – 4 = 9. Place next item w3 in knapsack
then the remaining weight of knapsack is 9 – 6 = 3. No fractions are allowed in calculation of
upper bound so w4 cannot be placed in knapsack.
Profit = P1 + P2 + P3 = 10 + 10 + 12
So, Upper bound = 32
To calculate lower bound we can place w4 in knapsack since fractions are allowed in calculation
of lower bound.
Lower bound = 10 + 10 + 12 + ( × 18) = 32 + 6 = 38

Knapsack problem is maximization problem but branch and bound technique is applicable for
only minimization problems. In order to convert maximization problem into minimization
problem we have to take negative sign for upper bound and lower bound.
Therefore,
Upper bound (U) = -32
Lower bound (L) = -38

Mrs. Selva Mary. G Page 2

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

We choose the path, which has minimum difference of upper bound and lower bound. If the
difference is equal, then we choose the path by comparing upper bounds and we discard node
with maximum upper bound.

Now we will calculate upper bound and lower bound for nodes 2, 3. For node 2, x1= 1, means we
should place first item in the knapsack.
= 10 + 10 + 12 = 32, − 32
= 10 + 10 + 12 + × 18 = 32 + 6 = 38, − 38

For node 3, x1 = 0, means we should not place first item in the knapsack.
= 10 + 12 = 22, − 22

= 10 + 12 + × 18 = 10 + 12 + 10 = 32, − 32

Next, we will calculate difference of upper bound and lower bound for nodes 2, 3
For node 2, U – L = -32 + 38 = 6
For node 3, U – L = -22 + 32 = 10
Choose node 2, since it has minimum difference value of 6.

Now we will calculate lower bound and upper bound of node 4 and 5. Calculate difference of
lower and upper bound of nodes 4 and 5.
For node 4, U – L = -32 + 38 = 6
For node 5, U – L = -22 + 36 = 14
Choose node 4, since it has minimum difference value of 6.

Mrs. Selva Mary. G Page 3

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Now we will calculate lower bound and upper bound of node 8 and 9. Calculate difference of
lower and upper bound of nodes 8 and 9.
For node 6, U – L = -32 + 38 = 6
For node 7, U – L = -38 + 38 = 0
Choose node 7, since it is minimum difference value of 0.

Now we will calculate lower bound and upper bound of node 8 and 9. Calculate difference of
lower and upper bound of nodes 8 and 9.
For node 8, U – L = -38 + 38 = 0
For node 9, U – L = -20 + 20 = 0
Here the difference is same, so compare upper bounds of nodes 8 and 9. Discard the node, which
has maximum upper bound. Choose node 8, discard node 9 since, it has maximum upper bound.
Consider the path from 1 2478
X1 = 1
X2 = 1

Mrs. Selva Mary. G Page 4

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

X3 = 0
X4 = 1
The solution for 0/1 Knapsack problem is (x1, x2, x3, x4) = (1, 1, 0, 1) Maximum profit is:

= 10 × 1 + 10 × 1 + 12 × 0 + 18 × 1

= 10 + 10 + 18 = 38.

TRAVELLING SALESMAN PROBLEM

It is algorithmic procedures similar to backtracking in which a new branch is chosen and

is there (bound there) until new branch is choosing for advancing. This technique is
implemented in the traveling salesman problem [TSP] which are asymmetric (Cij <>Cij) where
this technique is an effective procedure.
Steps involved in this procedure are as follows:
STEP 0: Generate cost matrix C [for the given graph g]
STEP 1: [ROW REDUCTION] For all rows do step 2
STEP 2: Find least cost in a row and negate it with rest of the elements
STEP 3: [COLUMN REDUCTION] Use cost matrix- Row reduced one for all columns do STEP 4.
STEP 4: Find least cost in a column and negate it with rest of the elements
STEP 5: Preserve cost matrix C [which row reduced first and then column reduced]
for the ith time.
STEP 6: Enlist all edges (i, j) having cost = 0.
STEP 7: Calculate effective cost of the edges.  (i, j)=least cost in the i th row excluding (i,

j) + least cost in the j th column excluding (i, j).

STEP 8: Compare all effective cost and pick up the largest l. If two or more have same cost
then arbitrarily choose any one among them
STEP 9: Delete (i, j) means delete ith row and jth column change (j, i) value to infinity. (Used
to avoid infinite loop formation) If (i,j) not present, leave it.
STEP 10: Repeat step 1 to step 9 until the resultant cost matrix having order of 2*2 and
reduce it. (Both R.R and C.C)
STEP 11: Use preserved cost matrix Cn, Cn-1… C1. Choose an edge [i, j] having value =0, at
the first time for a preserved matrix and leave that matrix.

Mrs. Selva Mary. G Page 5

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

STEP 12: Use result obtained in Step 11 to generate a complete tour.

EXAMPLE: Given graph G
MATRIX:

1 2 3 4 5
1  25 40 31 27
2 5  17 30 25
3 19 15  6 1
4 9 50 24  6
5 22 8 7 10 

PHASE I

STEP 1: Row Reduction C

C1 [ROW REDUCTION:

1 2 3 4 5
1  0 15 6 2
2 0  12 25 20
3 18 14  5 0
4 3 44 18  0
5 15 1 0 3 

STEP 3:
C1 [Column Reduction]

1 2 3 4 5
1  0 15 3 2
2 0  12 22 20
3 18 14  2 0
4 3 44 18  0
5 15 1 0 0 
STEP 5:
Preserve the above in C1,
1 2 3 4 5
1  0 15 3 2
2 0  12 22 20
3 18 14  2 0
4 3 44 18  0
5 15 1 0 0 
STEP 6:

Mrs. Selva Mary. G Page 6

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

L= (1,2),(2,1),(3,5),(4,5),(5,3),(5,4)
STEP 7:
Calculation of effective cost [E.C]
(1,2) = 2+1 =3
(2,1) = 12+3 = 15
(3,5) = 2+0 =2
(4,5) = 3+0 = 3
(5,3) = 0+12 = 12
(5,4) = 0+2 = 2

STEP 8:
L having edge (2,1) is the largest.

STEP 9: Delete (2,1) from C1 and make change in it as (1,2)  if exists.

Now Cost Matrix =

2 3 4 5
1  15 3 2
3 14  2 0
4 44 18  0
5 1 0 0 

STEP 10: The Cost matrix  2 x 2.

Therefore, go to step 1.

PHASE II:

STEP1: C2(R, R)

2 3 4 5
1  13 1 0
3 14  2 0
4 44 18  0
5 1 0 0 
STEP 3: C2 (C, R)

2 3 4 5
1  13 1 0
3 13  2 0
4 43 18  0
5 0 0 0 

STEP 5: Preserve the above in C2

Mrs. Selva Mary. G Page 7

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

C2 =

2 3 4 5
1  13 1 0
3 13  2 0
4 43 18  0
5 0 0 0 

STEP 6:

L= {(1,5), (3,5), (4,5), (5,2), (5,3), (5,4)}

STEP 7: calculation of E.C.

(1,5) = 1+0 =1
(3,5) = 2+0 =2
(4,5) = 18+0 =18
(5,2) = 0+13 =13
(5,3) = 0+13 =13
(5,4) = 0+1 =1

STEP 8: L having an edge (4,5) is the largest.

STEP 9: Delete (4,5) from C2 and make change in it as (5,4) = 

if exists.

Now, cost matrix

2 3 4
1  13 1
3 13  2
5 0 0 

STEP 10: THE cost matrix  2x2 hence go to step 1

PHASE III:

STEP 1: C3 (R, R)

2 3 4
1  12 0
3 11  0
5 0 0 

STEP 3: C3 (C, R)

Mrs. Selva Mary. G Page 8

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

2 3 4
1  12 0
3 11  0
5 0 0 

STEP 5: preserve the above in C3

STEP 6: L={(1,4), (3,4), (5,2), (5,3)}

STEP 7: calculation of E.C

(1,4)=12+0=12
(3,4)=11+0=11
(5,2)=0+11=11
(5,3)=0+12=12

STEP 8: Here we are having two edges (1,4) and (5,3) with cost = 12. Hence arbitrarily choose
(1,4)

STEP 9: Delete (i,j) (1,4) and make change in it (4,1) =  if exists.

Now cost matrix is

2 3
3 11 
5 0 0

STEP 10: We have got 2x2 matrix

C4 (RR)=

2 3
3 0 
5 0 0

C4 (C, R) =

2 3
3 0 
5 0 0

Therefore,C4 =

2 3
3 0 
5 0 0

Mrs. Selva Mary. G Page 9

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

STEP 11: LIST C1, C2, C3 AND C4

C4
2 3
3 0 
5 0 0
C3
2 3 4
1
1  0
2
1
3  0
1
5 0 0 

C2 =
2 3 4 5
1  13 1 0
3 13  2 0
4 43 18  0
5 0 0 0 

C1 =

1 2 3 4 5
1  0 15 3 2
2 0  12 22 20
3 18 14  2 0
4 3 44 18  0
5 15 1 0 0 

STEP 12:
i) Use C4 =

2 3
3 0 
5 0 0

Pick up an edge (I, j) =0 having least index

Here (3,2) =0

Hence, T (3,2)

Mrs. Selva Mary. G Page 10

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Use C3 =
2 3 4
1  12 0
3 11  0
5 0 0 

Pick up an edge (i, j) =0 having least index

Here (1,4) =0

Hence, T(3,2), (1,4)

Use C2=
2 3 4 5
1  13 1 0
3 13  2 0
4 43 18  0
5 0 0 0 

Pick up an edge (i, j) with least cost index.

Here (1,5)  not possible because already chosen index i (i=j)

(3,5)  not possible as already chosen index.
(4,5) 0

Hence, T (3,2), (1,4), (4,5)

Use C1 =
1 2 3 4 5
1  0 15 3 2
2 0  12 22 20
3 18 14  2 0
4 3 44 18  0
5 15 1 0 0 

Pick up an edge (i, j) with least index

(1,2)  Not possible

(2,1)  Choose it
HENCE T (3,2), (1,4), (4,5), (2,1)

SOLUTION:

From the above list

3—2—1—4—5

Mrs. Selva Mary. G Page 11

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

This result now, we have to return to the same city where we started (Here 3).

Final result:
3—2—1—4—5—3

Cost is 15+15+31+6+7=64

Mrs. Selva Mary. G Page 12

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 13

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 14

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 15

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 16

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 17

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 18

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 19

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 20

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 21

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Mrs. Selva Mary. G Page 22

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Difference between Backtracking and Branch and Bound

Back tracking Branch and Bound

It is used to find all possible solutions available to It is used to solve optimization problem
the problem
It traverse tree by DFS(Depth First Search). It may traverse the tree in any manner, DFS or BFS
It realizes that it has made a bad choice & undoes It realizes that it already has a better optimal
the last choice by backing up solution that the pre-solution leads to so it
abandons that pre-solution.
It search the state space tree until it found a It completely searches the state space tree to get
solution optimal solution
It involves feasibility function It involves bounding function
It is more often not applied to non-optimization. It can be applied only to optimization problems.

Mrs. Selva Mary. G Page 23

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

RANDOMIZED ALGORITHMS

An algorithm that uses random numbers to decide what to do next anywhere in its logic is called
Randomized Algorithm. For example, in Randomized Quick Sort, we use random number to pick the next
pivot (or we randomly shuffle the array).

Classification of Randomized algorithms

Las Vegas: These algorithms always produce correct or optimum result. Time complexity of these
algorithms is based on a random value and time complexity is evaluated as expected value. For example,
Randomized QuickSort always sorts an input array and expected worst case time complexity of QuickSort
is O(nLogn).

Monte Carlo: Produce correct or optimum result with some probability. These algorithms have
deterministic running time and it is generally easier to find out worst case time complexity. For example
implementation of Karger’s Algorithm produces minimum cut with probability greater than or equal to
1/n2 (n is number of vertices) and has worst case time complexity as O(E). Another example is Fermet
Method for Primality Testing.

Example to Understand Classification:

Consider a binary array where exactly half elements are 0 and half are 1. The task is to find index of any 1.

A Las Vegas algorithm for this task is to keep picking a random element until we find a 1. A Monte Carlo
algorithm for the same is to keep picking a random element until we either find 1 or we have tried
maximum allowed times say k.

The Las Vegas algorithm always finds an index of 1, but time complexity is determined as expect value.
The expected number of trials before success is 2, therefore expected time complexity is O(1).
The Monte Carlo Algorithm finds a 1 with probability [1 – (1/2)k]. Time complexity of Monte Carlo is O(k)
which is deterministic

Applications and Scope:

 Consider a tool that basically does sorting. Let the tool be used by many users and there are few
users who always use tool for already sorted array. If the tool uses simple (not randomized)
QuickSort, then those few users are always going to face worst case situation. On the other hand if
the tool uses Randomized QuickSort, then there is no user that always gets worst case. Everybody
gets expected O(n Log n) time.

 Randomized algorithms have huge applications in Cryptography.

 Load Balancing.

Mrs. Selva Mary. G Page 24

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

 Number-Theoretic Applications: Primality Testing

 Data Structures: Hashing, Sorting, Searching, Order Statistics and Computational Geometry.

 Algebraic identities: Polynomial and matrix identity verification. Interactive proof systems.

 Mathematical programming: Faster algorithms for linear programming, Rounding linear program
solutions to integer program solutions

 Graph algorithms: Minimum spanning trees, shortest paths, minimum cuts.

 Counting and enumeration: Matrix permanent Counting combinatorial structures.

 Parallel and distributed computing: Deadlock avoidance distributed consensus.

 Probabilistic existence proofs: Show that a combinatorial object arises with non-zero probability
among objects drawn from a suitable probability space.

 Derandomization: First devise a randomized algorithm then argue that it can be derandomized to
yield a deterministic algorithm.

HIRING PROBLEM
We will now begin our investigation of randomized algorithms with a toy problem:

• You want to hire an office assistant from an employment agency.

• You want to interview candidates and determine if they are better than the current assistant
and if so replace the current assistant.
• You are going to eventually interview every candidate from a pool of n candidates.
• You want to always have the best person for this job, so you will replace an assistant with a
better one as soon as you are done the interview.
• However, there is a cost to fire and then hire someone.
• You want to know the expected price of following this strategy until all n candidates have
been interviewed.

Hire-Assistant(n)

1 best ← 0 candidate 0 is a least-qualified dummy candidate

2 for i ← 1 to n

3 do interview candidate i in random permutation

4 if candidate i is better than candidate best

5 then best ← i

6 hire candidate i

Mrs. Selva Mary. G Page 25

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Total Cost and Cost of Hiring

• Interviewing has a low cost .
• Hiring has a high cost .
• Let n be the number of candidates to be interviewed andlet m be the number of people hired.
• The total cost then goes as ( ∗ + ∗ )
• The number of candidates is fixed so the part of thealgorithm we want to focus on is the
∗ term.
• This term governs the cost of hiring.

Worst-case Analysis
• In the worst case, everyone we interview turns out to be better than the person we currently
have.
• In this case, the hiring cost for the algorithm will be O(n*ch).
• This bad situation presumably doesn’t typically happen so it is interesting to ask what
happens in the average case.

Probabilistic analysis

• Probabilistic analysis is the use of probability to analyze problems.

• One important issue is what is the distribution of inputs to the problem.
• For instance , we could assume all orderings of candidates are equally likely.
• That is, we consider all functions rank: [0..n] --> [0..n] where rank[i] is supposed to be the ith
candidate that we interview. So <rank(1), rank(2), …rank(n)> should be a permutation of of
<1,…,n>
• There are n! many such permutations and we want each to be equally likely.
• If this is the case, the ranks form a uniform random permutation.

Randomized algorithms

• In order to use probabilistic analysis, we need to know something about the distribution of
the inputs.
• Unfortunately, often little is known about this distribution.
• We can nevertheless use probability and analysis as a tool for algorithm design by having the
algorithm we run do some kind of randomization of the inputs.
• This could be done with a random number generator. i.e.,
• We could assume we have primitive function Random(a,b) which returns an integer between
integers a and b inclusive with equally likelihood.
• Algorithms which make use of such a generator are called randomized algorithms.
• In our hiring example we could try to use such a generator to create a random permutation of
the input and then run the hiring algorithm on that.

Mrs. Selva Mary. G Page 26

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Analysis of the Hiring Problem

• Let Xi be the indicator random variable which is 1 if candidate i is hired and 0 otherwise.

• Let
• By our lemma E[Xi] = Pr{candidate i is hired}
• Candidate i will be hired if i is better than each of candidates 1 through i-1.
• As each candidate arrives in random order, any one of the first candidate i is equally likely to
be the best candidate so far. So E[Xi] =1/i.

More analysis of hiring problem

Lemma Assume that the candidates are presented in random order, then algorithmHire-Assistant has a
hiring cost of O(chln n)

Proof. From before hiring cost is O(m*ch)where m is the number of candidate hired.From the lemma this
is O(ln n).

RANDOMIZED_QUICKSORT
In the randomized version of Quick sort we impose a distribution on input. This does not improve the
worst-case running time independent of the input ordering.In this version we choose a random key for
the pivot. Assume that procedure Random (a, b) returns a random integer in the range [a, b); there are b-
a+1 integers in the range and procedure is equally likely to return one of them. The new partition
procedure, simply implemented the swap before actually partitioning.

RANDOMIZED_PARTITION (A, p, r)

i ← RANDOM (p, r)

Exchange A[p]↔ A[i]

return PARTITION (A, p, r)

Now randomized quick sort call the above procedure in place of PARTITION

RANDOMIZED_QUICKSORT (A, p, r)

If p < r then

q ← RANDOMIZED_PARTITION (A, p, r)

RANDOMIZED_QUICKSORT (A, p, q)

RANDOMIZED_QUICKSORT (A, q+1, r)

Mrs. Selva Mary. G Page 27

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Like other randomized algorithms, RANDOMIZED_QUICKSORT has the property that no particular input
elicits its worst-case behavior; the behavior of algorithm only depends on the random-number generator.
Even intentionally, we cannot produce a bad input for RANDOMIZED_QUICKSORT unless we can predict
generator will produce next.

For example, consider below a randomized version of QuickSort.

A Central Pivot is a pivot that divides the array in such a way that one side has at-least 1/4 elements.

// Sorts an array arr[low..high]

randQuickSort(arr[], low, high)

1. If low >= high, then EXIT.

2. While pivot 'x' is not a Central Pivot.

(i) Choose uniformly at random a number from [low..high].

Let the randomly picked number number be x.

(ii) Count elements in arr[low..high] that are smaller

than arr[x]. Let this count be sc.

(iii) Count elements in arr[low..high] that are greater

than arr[x]. Let this count be gc.

(iv) Let n = (high-low+1). If sc >= n/4 and

gc >= n/4, then x is a central pivot.

3. Partition arr[low..high] around the pivot x.

4. // Recur for smaller elements

randQuickSort(arr, low, sc-1)

5. // Recur for greater elements

randQuickSort(arr, high-gc+1, high)

The important thing in our analysis is, time taken by step 2 is O(n).

How many times while loop runs before finding a central pivot?

The probability that the randomly chosen element is central pivot is 1/2. Therefore, expected number of
times the while loop runs is 2 Thus, the expected time complexity of step 2 is O(n).

What is overall Time Complexity in Worst Case?

In worst case, each partition divides array such that one side has n/4 elements and other side has 3n/4
elements. The worst case height of recursion tree is Log 3/4 n which is O(Log n).

T(n) < T(n/4) + T(3n/4) + O(n)

Mrs. Selva Mary. G Page 28

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

T(n) < 2T(3n/4) + O(n)

Solution of above recurrence is O(n Log n)

Note that the above randomized algorithm is not the best way to implement randomized Quick Sort. The
idea here is to simplify the analysis as it is simple to analyze.

Typically, randomized Quick Sort is implemented by randomly picking a pivot (no loop). Or by shuffling
array elements. Expected worst case time complexity of this algorithm is also O(n Log n)

NP PROBLEMS AND NP COMPLETE

Status of NP Complete problems is another failure story, NP complete problems are problems whose
status is unknown. No polynomial time algorithm has yet been discovered for any NP complete problem,
nor has anybody yet been able to prove that no polynomial-time algorithm exist for any of them. The
interesting part is, if any one of the NP complete problems can be solved in polynomial time, then all of
them can be solved.

What are NP, P, NP-complete and NP-Hard problems?

P is set of problems that can be solved by a deterministic Turing machine in Polynomial time.

NP is set of decision problems that can be solved by a Non-deterministic Turing Machine in Polynomial
time. P is subset of NP (any problem that can be solved by deterministic machine in polynomial time can
also be solved by non-deterministic machine in polynomial time).

Informally, NP is set of decision problems which can be solved by a polynomial time via a “Lucky
Algorithm”, a magical algorithm that always makes a right guess among the given set of choices.

NP-complete problems are the hardest problems in NP set. A decision problem L is NP-complete if:

1. L is in NP (Any given solution for NP-complete problems can be verified quickly, but there is no
efficient known solution).
2. Every problem in NP is reducible to L in polynomial time (Reduction is defined below).

A problem is NP-Hard if it follows property 2 mentioned above, doesn’t need to follow property 1.
Therefore, NP-Complete set is also a subset of NP-Hard set.

Mrs. Selva Mary. G Page 29

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Decision vs Optimization Problems

NP-completeness applies to the realm of decision problems. It was set up this way because it’s easier to
compare the difficulty of decision problems than that of optimization problems. In reality, though, being
able to solve a decision problem in polynomial time will often permit us to solve the corresponding
optimization problem in polynomial time (using a polynomial number of calls to the decision problem).
So, discussing the difficulty of decision problems is often really equivalent to discussing the difficulty of
optimization problems.

For example, consider the vertex cover problem (Given a graph, find out the minimum sized vertex set
that covers all edges). It is an optimization problem. Corresponding decision problem is, given undirected
graph G and k, is there a vertex cover of size k?

What is Reduction?
Let L1 and L2be two decision problems. Suppose algorithm A2solves L2. That is, if y is an input for L2 then
algorithm A2will answer Yes or No depending upon whether y belongs toL2 or not.
The idea is to find a transformation from L1 to L2so that the algorithm A2 can be part of an algorithm A1 to
solve L1.

Learning reduction in general is very important. For example, if we have library functions to solve certain
problem and if we can reduce a new problem to one of the solved problems, we save a lot of time.
Consider the example of a problem where we have to find minimum product path in a given directed
graph where product of path is multiplication of weights of edges along the path. If we have code for

Mrs. Selva Mary. G Page 30

15CS204J- ALGORITHM DESIGN AND ANALYSIS UNIT-V

Dijkstra’s algorithm to find shortest path, we can take log of all weights and use Dijkstra’s algorithm to
find the minimum product path rather than writing a fresh code for this new problem.

How to prove that a given problem is NP complete?

From the definition of NP-complete, it appears impossible to prove that a problem L is NP-Complete. By
definition, it requires us to that show every problem in NP is polynomial time reducible to L. Fortunately,
there is an alternate way to prove it. The idea is to take a known NP-Complete problem and reduce it to
L. If polynomial time reduction is possible, we can prove that L is NP-Complete by transitivity of
reduction (If a NP-Complete problem is reducible to L in polynomial time, then all problems are reducible
to L in polynomial time).

What was the first problem proved as NP-Complete?

There must be some first NP-Complete problem proved by definition of NP-Complete problems. SAT
(Boolean satisfiability problem) is the first NP-Complete problem proved by Cook

It is always useful to know about NP-Completeness even for engineers. Suppose you are asked to write an
efficient algorithm to solve an extremely important problem for your company. After a lot of thinking, you
can only come up exponential time approach which is impractical. If you don’t know about NP-
Completeness, you can only say that I could not come with an efficient algorithm. If you know about NP-
Completeness and prove that the problem as NP-complete, you can proudly say that the polynomial time
solution is unlikely to exist. If there is a polynomial time solution possible, then that solution solves a big
problem of computer science many scientists have been trying for years.

Mrs. Selva Mary. G Page 31