Design and Analysis of Algorithms

CSE III year II sem (R18)

UNIT V

Shumama Ansa
 BRANCH & BOUND –
GENERAL METHOD
• Branch & Bound (B & B) is general algorithm (or Systematic method) for
finding optimal solution of various optimization problems, especially in
discrete and combinatorial optimization.
• The differences are that the B&B method
– Does not limit us to any particular way of traversing the tree.
– It is used only for optimization problem
– It is applicable to a wide variety of discrete combinatorial problem.
• The term B&B refers to all state space search methods in which all children
of the “E-node” are generated before any other “live node” can become the
“E-node”
– Live node is a node that has been generated but whose children have
not yet been generated.
– E-nodeis a live node whose children are currently being explored.
• Dead node is a generated node that is not to be expanded or explored
any further. All children of a dead node have already been expanded.
FIFOBB

• FIFO Branch & Bound is a BFS.

• In this, children of E-Node (or Live nodes) are inserted in a queue.

LIFO BB

• LIFO Branch & Bound is a DFS.

• In this, children of E-Node (or Live nodes) are inserted in a
stack.
 LEAST COST BB

• The selection rule for the next E-node in FIFO or LIFO branch and bound
is sometimes “blind”. i.e., the selection rule does not give any preference to
a node that has a very good chance of getting the search to an answer node
quickly.
• The search for an answer node can often be speeded by using an
“intelligent” ranking function. It is also called an approximate cost function
“Ĉ”.
• Expanded node (E-node) is the live node with the best Ĉ value.
• Branching: A set of solutions, which is represented by a node, can be
partitioned into mutually (jointly or commonly) exclusive (special) sets.
Each subset in the partition is represented by a child of the original node.
• Lower bounding: An algorithm is available for calculating a lower bound
on the cost of any solution in a given subset.
•In both FIFO and LIFO Branch and Bound the
selection rules for the next E-node in rigid and blind.
•The selection rule for the next E-node does not give
any preferences to a node that has a very good chance
of getting the search to an answer node quickly.
•In this we will use ranking function or cost function.
•We generate the children of E-node, among these live
nodes; we select a node which has minimum cost.
•By using ranking function we will calculate the cost of
each node.
 BRANCH & BOUND –
APPLICATIONS

• TRAVELLING SALES PERSON PROBLEM

• 0/1 KNAPSACK PROBLEM
• LCBB
• FIFOBB
 0/1 KNAPSACK PROBLEM (LCBB)

•There are n objects given and capacity of knapsack is M.

•Select some objects to fill the knapsack in such a way that it
should not exceed the capacity of Knapsack and maximum profit
can be earned.
•The Knapsack problem is maximization problem. It means we
will always seek for maximum p1x1 (where p1 represents profit
of object x1).
•A branch bound technique is used to find solution to the
knapsack problem.
•But we cannot directly apply the branch and bound technique to
the knapsack problem.
•Because the branch bound deals only the minimization
problems.
•We modify the knapsack problem to the minimization problem.
• Example: Consider the instance M=15, n=4,
(p1, p2, p3, p4) = 10, 10, 12, 18 and (w1, w2,
w3, w4)=(2, 4, 6, 9).
• knapsack problem can be solved by using
branch and bound technique. In this problem
we will calculate lower bound and upper
bound for each node.
• No fraction are allowed in calculation of upper
bound so w4, cannot be placed in knapsack.
• Profit= p1+p2+ p3=10+10+12
• So, Upper bound=32
• To calculate Lower bound/ cost we can place
w4 in knapsack since fractions are allowed in
calculation of lower bound.
• Lower bound=10+10+12+ (3/9*18)=32+6=38
• In order to convert maximization problem into
minimization problem we have to take
negative sign for upper bound and lower
bound.
• Therefore, upper bound (U) =-32
• Lower bound (L)=-38
• For node 2, x1=1, means we should place first
item in the knapsack.
• U=10+10+12=32, make it as -32
• L=10+10+12+ (3/9*18) = 32+6=38, we make it as
-38
• For node 3, x1=0, means we should not place first
item in the knapsack.
• U=10+12=22, make it as -22
• L=10+12+ (5/9*18)
= 10+12+10
=32, we make it as -32
Example: Consider the instance M=15, n=4, (p1, p2, p3,
p4) = 10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6, 9).
 Example: Consider the instance M=15, n=4, (p1, p2, p3, p4) =
10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6, 9).

• Next we will calculate difference of upper

bound and lower bound for nodes 2, 3
• For node 2, U-L=-32+38=6
• For node 3, U-L=-22+32=10
• Choose node 2, since it has minimum
difference value of 6.
Example: Consider the instance M=15, n=4, (p1, p2, p3, p4) =
10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6, 9).
• Now we will calculate lower bound and upper
bound of node 4 and 5. Calculate difference of
lower and upper bound of nodes 4 and 5.
• For node 4, U-L=-32+38=6
• For node 5, U-L=-22+36=14
• Choose node 4, since it has minimum
difference value of 6
Example: Consider the instance M=15, n=4, (p1, p2, p3, p4) =
10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6, 9).
• Now we will calculate lower bound and upper
bound of node 6 and 7. Calculate difference of
lower and upper bound of nodes 6 and 7.
• For node 6, U-L=-32+38=6
• For node 7, U-L=-38+38=0
• Choose node 7, since it has minimum
difference value of 0.
Example: Consider the instance M=15, n=4, (p1, p2, p3, p4) = 10, 10, 12, 18 and (w1, w2, w3, w4)=(2, 4, 6,
• Now we will calculate lower bound and upper
bound of node 8 and 9. Calculate difference of
lower and upper bound of nodes 8 and 9.
• For node 8, U-L=-38+38=0
• For node 9, U-L=-20+20=0
• Here, the difference is same, so compare upper
bounds of nodes 8 and 9. Discard the node,
which has maximum upper bound. Choose node
8, discard node 9 since, it has maximum upper
bound
• Consider the path from 1=>2=>4=>7=>8
• X1=1
• X2=1
• X3=0
• X4=1
• The solution for 0/1 knapsack problem is ( x1, x2,
x3, x4)=(1, 1, 0, 1)
• Maximum profit is:
Σpixi=10*1+10*1+12*0+18*1 = 10+10+18=38.
TRAVELLING SALES PERSON PROBLEM
• Let G = (V', E) be a directed graph defining an
instance of the traveling salesperson problem.
• Let Cij equal the cost of edge (i, j), Cij = ∞ if (i,
j) != E, and let IVI = n, without loss of
generality, we can assume that every tour
starts and ends at vertex 1.
Procedure for solving travelling sales person problem
Reduce the given cost matrix. A matrix is reduced if every
row and column is reduced. This can be done as follows:
Row Reduction:
a) Take the minimum element from first row, subtract it from
all elements of first row, next take minimum element from
the second row and subtract it from second row. Similarly
apply the same procedure for all rows.
b) Find the sum of elements, which were subtracted from
rows.
c) Apply column reductions for the matrix obtained after row
reduction.
• Column Reduction:
• d) Take the minimum element from first column, subtract it from all
elements of first column, next take minimum element from the
second column and subtract it from second column. Similarly apply
the same procedure for all columns.
• e) Find the sum of elements, which were subtracted from columns.
• f) Obtain the cumulative sum of row wise reduction and column
wise reduction.

Cumulative reduced sum=Row wise reduction sum + Column wise

reduction sum.
Associate the cumulative reduced sum to the starting state as lower
bound and α as upper bound.
Calculate the reduced cost matrix for every node.
a) If path (i,j) is considered then change all entries
in row i and column j of A to α.
b) Set A(j,1) to α.
c) Apply row reduction and column reduction
except for rows and columns containing only α.
Let r is the total amount subtracted to reduce
the matrix.
d) Find ĉ(S)= ĉ(R)+A(i,j)+r.
Repeat step 2 until all nodes are visited.
• Example: Find the LC branch and bound solution for
the travelling sales person problem whose cost
matrix is as follows.

∞ 20 30 10 11
15 ∞ 16 4 2
3 5 ∞ 2 4
19 6 18 ∞ 3
16 4 7 16 ∞
 • Substract minimum element from every row

∞ 20 30 10 11 ∞ 10 20 0 1
15 ∞ 16 4 2 13 ∞ 14 2 0
3 5 ∞ 2 4 1 3 ∞ 0 2
19 6 18 ∞ 3 16 3 15 ∞ 0
16 4 7 16 ∞ 12 0 3 12 ∞
 • Subtract minimum element from every
column
M1
∞ 10 20 0 1 ∞ 10 17 0 1 10
13 ∞ 14 2 0 12 ∞ 11 2 0 2
1 3 ∞ 0 2 0 3 ∞ 0 2 2
16 3 15 ∞ 0 15 3 12 ∞ 0 3
12 0 3 12 ∞ 11 0 0 12 ∞ 4
1 0 3 0 0 25
This is the cost of a root i.e. node 1, because this is the initially reduced
cost matrix.
The lower bound for node is 25 and upper bound is ∞.
 Node2
• 1st row and 2nd column all elements and (2,1) = ∞
• Updated matrix
M2
Matrix is in reduced form ∞ ∞ ∞ ∞ ∞ 0
Cost of node 2 =
∞ ∞ 11 2 0 0
reduced cost + cost[1,2] + r^
= 25+10+0 0 ∞ ∞ 0 2 0
= 35 15 ∞ 12 ∞ 0 0
11 ∞ 0 12 ∞ 0
0 0 0 0 0 0
 ∞ 10 17 0 1
Node3 12 ∞ 11 2 0
• 1st row and 3rd column all elements 0 3 ∞ 0 2
• and (3,1) = ∞ Step 1 15 3 12 ∞ 0
• Updated matrix 11 0 0 12 ∞
Step 2 M3
Step 3
∞ ∞ ∞ ∞ ∞ 0
12 ∞ ∞ 2 0 0 ∞ ∞ ∞ ∞ ∞
∞ 3 ∞ 0 2 0 1 ∞ ∞ 2 0
15 3 ∞ ∞ 0 0 ∞ 3 ∞ 0 2
11 0 ∞ 12 ∞ 0 4 3 ∞ ∞ 0
11 0 0 0 0 11 0 0 ∞ 12 ∞

Substract 11 from column 1 Cost = 17 + 11+ 25 = 53

 ∞ 10 17 0 1
Node4 12 ∞ 11 2 0
• 1st row and 4th column all elements 0 3 ∞ 0 2
and (4,1) = ∞ 15 3 12 ∞ 0
• Updated matrix 11M40 0 12 ∞
M4
∞ ∞ ∞ ∞ ∞
12 ∞ 11 ∞ 0
0 3 ∞ ∞ 2
∞ 3 12 ∞ 0
11 0 0 ∞ ∞

Cost = 25+0+0 = 25
 ∞ ∞ ∞ ∞ ∞ 0
Node5 Step 2
12 ∞ 11 2 ∞ 2
• 1st row and 5th column all elements and
(5,1) = ∞ M1 Step 1 0 3 ∞ 0 ∞ 0
∞ 10 17 0 1 15 3 12 ∞ ∞ 3
12 ∞ 11 2 0 ∞ 0 0 12 ∞ 0
0 3 ∞ 0 2
Step 3
15 3 12 ∞ 0 ∞ ∞ ∞ ∞ ∞ 0
11 0 0 12 ∞ 10 ∞ 9 0 ∞ 2
Step 4 M5
0 3 ∞ 0 ∞ 0
∞ ∞ ∞ ∞ ∞
12 3 9 ∞ ∞ 3
10 ∞ 9 0 ∞
∞ 0 0 12 ∞ 0
0 3 ∞ 0 ∞
0 0 0 0 0 5
12 0 9 ∞ ∞
∞ 0 0 12 ∞ Cost = 25+ 1 + 5 = 31
• Among node 2, node 3, node4, node 5 the
least cost is obtained for node 4
• So, now explore node 4
Use M4, obtain M6 for Node6 ∞ ∞ ∞ ∞ ∞ 0
(vertex 4 to vertex 2)
4th row and 2nd column all elements ∞
Step 2 ∞ ∞ 11 ∞ 0 0
M4 Step 1 0 ∞ ∞ ∞ 2 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
12 ∞ 11 ∞ 0
11 ∞ 0 ∞ ∞ 0
0 3 ∞ ∞ 2
Step 3
∞ 3 12 ∞ 0 ∞ ∞ ∞ ∞ ∞ 0
11 0 0 ∞ ∞ ∞ ∞ 11 ∞ 0 0
M6 Step 4
0 ∞ ∞ ∞ 2 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ 11 ∞ 0
11 ∞ 0 ∞ ∞ 0
0 ∞ ∞ ∞ 2
0 0 0 0 0 0
∞ ∞ ∞ ∞ ∞
11 ∞ 0 ∞ ∞ Cost = 25+ 3 + 0 = 28
Use M4, obtain M7 for Node7 ∞ ∞ ∞ ∞ ∞ 0
(vertex
4th
4 tord vertex 3) 12 ∞ ∞ ∞ 0 0
row and 3 column all elements ∞
M4 Step 1 Step 2 ∞ 3 ∞ ∞ 2 2
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
12 ∞ 11 ∞ 0
11 0 ∞ ∞ ∞ 0
0 3 ∞ ∞ 2
∞ 3 12 ∞ 0 Step 3 ∞ ∞ ∞ ∞ ∞ 0
11 0 0 ∞ ∞ 12∞ ∞ ∞ 0 0
M7 Step 4
∞ 1 ∞ ∞ 0 2
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
1 ∞ ∞ ∞ 0
11 0 ∞ ∞ ∞ 0
∞ 1 ∞ ∞ 0
11 0 0 0 0 13
∞ ∞ ∞ ∞ ∞
Cost = 25+ 13 + 12 = 50
0 0 ∞ ∞ ∞
Use M4, obtain M8 for Node8 ∞ ∞ ∞ ∞ ∞ 0
(vertex
4th
4 toth vertex 5) 12 ∞ 11 ∞ ∞ 11
row and 5 column all elements ∞
Step 1
Step 2 0 3 ∞ ∞ ∞ 0
M4
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
12 ∞ 11 ∞ 0
∞ 0 0 ∞ ∞ 0
0 3 ∞ ∞ 2
∞ 3 12 ∞ 0
Step 3 ∞ ∞ ∞ ∞ ∞ 0
11 0 0 ∞ ∞ 1 ∞ 0 ∞ ∞ 11
M8 Step 4
0 3 ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
1 ∞ 0 ∞ ∞
∞ 0 0 ∞ ∞ 0
0 3 ∞ ∞ ∞
0 0 0 0 0 11
∞ ∞ ∞ ∞ ∞
∞ 0 0 ∞ ∞ Cost = 25+ 0 + 11 = 36
 1

2 3 4 5

7 8
6
(3) (5)
(2)

• Node 6 has least cost, so explore node 6

Use M6, obtain M9 for Node9 ∞ ∞ ∞ ∞ ∞ 0
(vertex
2nd
2rdto vertex 3) ∞ ∞ ∞ ∞ ∞ 0
row and 3 column all elements ∞
Step 2
M6 Step 1 ∞ ∞ ∞ ∞ 2 2
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
∞ ∞ 11 ∞ 0 11 ∞ ∞ ∞ ∞ 11
0 ∞ ∞ ∞ 2
∞ ∞ ∞ ∞ ∞ Step 3 ∞ ∞ ∞ ∞ ∞ 0
11 ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
M9 Step 4
∞ ∞ ∞ ∞ 0 2
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
0 ∞ ∞ ∞ ∞ 11
∞ ∞ ∞ ∞ 0
0 0 0 0 0 13
∞ ∞ ∞ ∞ ∞
Cost = 28+ 11+13 = 52
0 ∞ ∞ ∞ ∞
Use M6, obtain M10 for ∞ ∞ ∞ ∞ ∞ 0
Node9 (vertex 2 to vertex 5) ∞ ∞ ∞ ∞ ∞ 0
2nd row and 5th column all elements ∞ and [5,1] = ∞
Step 1
M6 0 ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
Step 2 ∞ ∞ ∞ ∞ ∞ 0
∞ ∞ 11 ∞ 0
∞ ∞ 0 ∞ ∞ 0
0 ∞ ∞ ∞ 2
∞ ∞ ∞ ∞ ∞ Step 3 ∞ ∞ ∞ ∞ ∞ 0
11 ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
M10 Step 4
0 ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
∞ ∞ 0 ∞ ∞ 0
0 ∞ ∞ ∞ ∞
0 0 0 0 0 0
∞ ∞ ∞ ∞ ∞
∞ ∞ 0 ∞ ∞ Cost = 28+ 0+0 = 28
• Node 10 has least cost,
• Explore node 10
Use M10, obtain M11 for ∞ ∞ ∞ ∞ ∞ 0
Node11
5th
(vertex
rd
5 to vertex 3) ∞ ∞ ∞ ∞ ∞ 0
row and 3 column all elements ∞
M10 ∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
0 ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
∞ ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ 0
M11
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞
0 0 0 0 0 0
∞ ∞ ∞ ∞ ∞
∞ ∞ ∞ ∞ ∞ Cost = 28+ 0+0 = 28
• 1-4-2-5-3-1
NP-Hard and NP-Complete problems:
Basic concepts

• Many problems are solved where the objective is to

maximize or minimize some values
• In other problems we try to find whether there is a
solution or not.
• Hence, the problems can be categorized as follows.
Optimization Problem

• Optimization problems are those for which the objective is to maximize or

minimize some values.

• For example, Finding the minimum number of colors needed to color a

given graph.

• Finding the shortest path between two vertices in a graph.

Decision Problem
• There are many problems for which the answer
is a Yes or a No.
• These types of problems are known as
decision problems.
• For example, Whether a given graph can be
colored by only 4-colors.
• checking a graph is Hamiltonian or not is a
decision problem.
P-Class
• The class P consists of those problems that
are solvable in polynomial time, i.e. these
problems can be solved in time O(nk) in worst-
case, where k is constant.
• These problems are called tractable, while
others are called intractable or super
polynomial.
• Formally, an algorithm is polynomial time
algorithm, if there exists a polynomial p(n)
such that the algorithm can solve any instance
of size n in a time O(p(n)).
NP-Class
• The class NP consists of those problems that
are verifiable in polynomial time.
• NP is the class of decision problems for
which it is easy to check the correctness of a
claimed answer, with the aid of a little extra
information.
• Hence, we aren’t asking for a way to find a
solution, but only to verify that an alleged
solution really is correct.
• Every problem in this class can be solved in
exponential time.
P versus NP
• Every decision problem that is solvable by
a deterministic polynomial time algorithm
is also solvable by a polynomial time non-
deterministic algorithm.
• All problems in P can be solved with
polynomial time algorithms, whereas all
problems in NP - P are intractable.
 Non-deterministic algorithms
Nondeterministic Algorithms:
• Algorithms with the property that the result of every operation is uniquely
defined are termed as deterministic algorithms.
• Such algorithms agree with the way programs are executed on a computer.
• Algorithms which contain operations whose outcomes are not uniquely
defined but are limited to specified set of possibilities. Such algorithms are
called nondeterministic algorithms.
• The machine executing such operations is allowed to choose any one of
these outcomes subject to a termination condition to be defined later.
• To specify nondeterministic algorithms, there are 3 new functions.
– Choice(S) arbitrarily chooses one of the elements of sets S
– Failure () Signals an Unsuccessful completion
– Success () Signals a successful completion.
Example for Non Deterministic algorithms:
Algorithm Search(x)
{
//Problem is to search an element x
//output J, such that A[J]=x; or J=0 if x is not in A J:=Choice(1,n);
if( A[J]:=x) then
{
Write(J);
Success();
}
else
{
write(0);
failure();
}
Whenever there is a set of choices that leads to a successful completion then one such set of choices is always
made and the algorithm terminates.
A Nondeterministic algorithm terminates unsuccessfully if and only if (iff) there exists no set of choices leading
to a successful signal.
 NP-Hard & NP-Complete classes:
•If an NP-hard problem can be solved in polynomial time,
then all NP- complete problems can be solved in
polynomial time.
• All NP-complete problems are NP-hard, but all NP- hard
problems are not NP-complete.
• The class of NP-hard problems is very rich in the sense
that it contains many problems from a wide variety of
disciplines.
• P: The class of problems which can be solved by a
deterministic polynomial algorithm.
• NP: The class of decision problem which can be solved by a
non- deterministic polynomial algorithm.
• NP-hard: The class of problems to which every NP problem
reduces
• NP-complete (NPC): the class of problems which are NP-hard
and belong to NP.
 Cook's theorem
• The most famous unsolvable problems in Computer Science is
Whether P=NP or P≠NP
• In considering this problem, s.cook formulated the following
question.
• If there any single problem in NP, such that if we showed it to
be in ‘P’ then that would imply that P=NP.
• Cook answered this question with Theorem: Satisfiability is in
P if and only if (iff) P=NP
• Notation of Reducibility
• Let L1 and L2 be problems, Problem L1 reduces to L2 (written L1 α L2) iff
there is a way to solve L1 by a deterministic polynomial time algorithm
using a deterministic algorithm that solves L2 in polynomial time
• This implies that, if we have a polynomial time algorithm for L2, Then we
can solve L1 in polynomial time.
• Here α is a transitive relation i.e., L1 α L2and L2 α L3then L1 α L3
• A problem L is NP-Hard if and only if (iff) satisfiability reduces to L
ie.,Statisfiability α L
• A problem L is NP-Complete if and only if (iff) L is NP-Hard and L Є NP
• Commonly believed relationship among P, NP, NP-Complete and NP-Hard
• Most natural problems in NP are either in P or NP-complete.