DAA Unit 7 Branch and Bound
DAA Unit 7 Branch and Bound
Introduction:
The term Branch and Bound refers to all state space search
methods in which all children of the E-Node are generated
before any other live node can become the E-Node.
We have already seen two graph search strategies, BFS and DFS
in which the exploration of a new node cannot begin until the
node currently being explored is fully explored.
In Branch and Bound BFS like space search will be called FIFO
search as the list of live nodes is a first in first out list
(or queue).
Ex) let us see how a FIFO branch and bound algorithm would
search the state space tree for the 4 queens problem. We have
already solved the problem using backtracking as under.
State space: All paths from the root to other nodes define the
state space of the problem.
Live Node: A node which has been generated and all of whose
children have not yet been generated is called a live node.
Fig 2) Portion of the tree of fig (1) that is generated during Backtracking.
At the time the answer node, i.e. node 31 is reached, the only
live nodes remaining are nodes 38 and 54.
Fig 3) Portion of the tree in fig 1) that is generated by a FIFO Branch and Bound search.
Again delete an element from Q. generate children of node 5, i.e. nodes 10 and 11 are
generated and killed by bounding function, last node in the queue is 6.
6
The child of node 6 is 12 and it satisfies the conditions of the problem which is the answer
node, so search terminates.
10 top
5
2
1
stack
Fig 5) State space tree
For LIFO branch and bound we use the data structure stack. Initially stack is empty.
The order is 1,2,5,10,11,6,12
In this we use a ranking function or cost function, which is denoted by cˆ( x) . We generate the
children of the E-node, among these live nodes, we select a node which has minimum cost.
By using the ranking function we will calculate the cost of each node.
Note that the ranking function cˆ( x) or cost function which depends on the problem.
Generate children of node 1, the children are 2,3,and 4. By using the ranking function we
calculate the cost of 2,3 and 4 nodes as cˆ(2) = 2 , cˆ(3) = 3 , cˆ(4) = 4 .Now we select a node
which has minimum cost i.e. node 2. For node 2 the children are 5 and 6. cˆ(5) = 4 , cˆ(6) = 1 we
select node 6 as cost is less. Generate children of 6 i.e. 12 and 13. We select node 12 since its
cost is 1 i.e. minimum. More over node 12 is the answer node. So we terminate search
process.
Applications
Travelling salesperson problem :: LCBB Solution
0/1 Knapsack problem :: LCBB solution
0/1 Knapsack problem :: FIFOBB solution
Figure: State space Tree for the travelling salesperson problem with n=4 and i0 = i4 = 1 i.e. the
graph contains 4 nodes and start and end nodes are node 1.
Now one of nodes 2 or 3 or 4 becomes E-node depending on the cost. For each node a cost
value will be calculated. Depending on the cost the next E-node is selected. The process is
continued.
Let G=(V,E) be a directed graph defining an instance of the travelling sales person problem.
Let Cij be the cost of the edge (i,j) and cij= ∞ if (i,j) ∉ E(G) and let V = n .
Assume that every tour starts and ends at vertex 1.
To use least cost branch and bound to search the travelling sales person state space tree, we
must define a cost function c(x) and two other functions cˆ( x) and uˆ ( x) such that
cˆ( x ) ≤ c ( x ) ≤ uˆ ( x ) .
Cost matrix:
1 2 3 4 5
1 ∞ 20 30 10 11
2 15 ∞ 16 4 2
3 3 5 ∞ 2 4
4 19 6 18 ∞ 3
5 16 4 7 16 ∞
Solution
A row is said to be reduced iff it contains at least one 0 and all remaining entries are non-
negative.
A column is said to be reduced iff it contains at least one 0 and all remaining entries are non-
negative.
A matrix is reduced iff every row and column is reduced.
Row Reduction :
∞ 20 30 10 11 10 ∞ 10 20 0 1
15 ∞ 16 4 2 2 13 ∞ 14 2 0
3 5 ∞ 2 4 2 ⇒ 1 2 ∞ 0 2
19 6 18 ∞ 3 3 16 3 15 ∞ 0
16 4 7 16 ∞ 4 12 0 3 12 ∞
Column Reduction:
∞ 10 20 0 1 ∞ 10 17 0 1
13 ∞ 14 2 0 12 ∞ 11 2 0
1 2 ∞ 0 2 ⇒ 0 2 ∞ 0 2
16 3 15 ∞ 0 15 3 12 ∞ 0
12 0 3 12 ∞ 11 0 0 12 ∞
1 − 3 − −
Total amount subtracted = r + c = 21 + 4 = 25
cˆ(1) = 25
25 will be the minimum cost to travel from node 1 as source and destination and nodes
2,3,4,5 as intermediate nodes. So node 1 is root node and it is E-node. As there is a possibility
of selecting path in either of nodes 2 or 3 or 4 or 5 it generates node 2, node 3 , node 4 and
node 5 as live nodes.
Consider the path (1,2) node 2: The reduced cost matrix may be obtained as follows.
1) Change all entries in row i and column j of A to ∞ . This prevents any more edges leaving
vertex i or entering vertex j.
2) Set A(j,1) to ∞ . This prevents the use of edge <j,1>.
3) Reduce all rows and columns in the resulting matrix except for rows and columns
containing only ∞ .
Change all entries of first row and second column of reduced matrix to ∞ .
Assign A[2,1]= ∞ .
Reduce row and reduce column
We get the following reduced cost matrix:
∞ 10 17 0 1 ∞ ∞ ∞ ∞ ∞
12 ∞ 11 2 0 ∞ ∞ 11 2 0
A = 0 2 ∞ 0 2 ⇒ 0 ∞ ∞ 0 2
15 3 12 ∞ 0 15 ∞ 12 ∞ 0
11 0 0 12 ∞ 11 ∞ 0 12 ∞
The reduced cost matrix after applying row reduction and column reduction i.e. ⇒ r = 0
cˆ(2) = cˆ(1) + A(1, 2) + r = 25 + 10 + 0 = 35
Consider the path (1,3) node 3: Change all entries of first row and third column of reduced
matrix to ∞ and set A(3,1) to ∞ .
∞ 10 17 0 1 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
12 ∞ 11 2 0 12 ∞ ∞ 2 0 1 ∞ ∞ 2 0
A= 0 2 ∞ 0 2 ⇒ ∞ 3 ∞ 0 2 ⇒ ∞ 3 ∞ 0 2
15 3 12 ∞ 0 15 3 ∞ ∞ 0 4 3 ∞ ∞ 0
11 0 0 12 ∞ 11 0 ∞ 12 ∞ 0 0 ∞ 12 ∞
11
Applying row reduction and column reduction. ⇒ r = 11 +0=11
cˆ(3) = cˆ(1) + A(1, 3) + r = 25 + 17 + 11 = 53
Consider the path (1,4) node 4 : Changing all entries of first row and fourth column to
∞ and set A(4,1) to ∞ .
∞ 10 17 0 1 ∞ ∞ ∞ ∞ ∞
12 ∞ 11 2 0 12 ∞ 11 ∞ 0
A= 0 2 ∞ 0 2 ⇒ 0 3 ∞ ∞ 2
15 3 12 ∞ 0 ∞ 3 12 ∞ 0
11 0 0 12 ∞ 11 0 0 ∞ ∞
Consider the path (1,5) node 5: Changing all entries of first row and fifth column to ∞ and
set A(5,1) to ∞ . Reduce row and columns.
∞ 10 17 0 1 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
12 ∞ 11 2 0 12 ∞ 11 2 ∞ 2 10 ∞ 9 0 ∞
A= 0 2 ∞ 0 2 ⇒ 0 3 ∞ 0 ∞ ⇒ 0 3 ∞ 0 ∞
15 3 12 ∞ 0 15 3 12 ∞ ∞ 3 12 0 9 ∞ ∞
11 0 0 12 ∞ ∞ 0 0 12 ∞ ∞ 0 0 12 ∞
5
Applying row reduction and column reduction, ⇒ r = 5
cˆ(5) = cˆ(1) + A(1,5) + r = 25 + 1 + 5 = 31
The matrix obtained for path (1,4) is considered as reduced cost matrix.
∞ ∞ ∞ ∞ ∞
12 ∞ 11 ∞ 0
A = 0 3 ∞ ∞ 2
∞ 3 12 ∞ 0
11 0 0 ∞ ∞
Now node 4 becomes E-node. Its children 6,7, and 8 are generated. The cost values are
calculated for nodes 6,7 and 8.
Consider the path(1,4,3) node 7: Change all entries of fourth row and third column of
reduced matrix A to ∞ and set A(3,1) to ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
12 ∞ 11 ∞ 0 12 ∞ ∞ ∞ 0 1 ∞ ∞ ∞ 0
A = 0 3 ∞ ∞ 2 ⇒ ∞ 3 ∞ ∞ 22 ⇒ ∞ 1 ∞ ∞ 0
∞ 3 12 ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
11 0 0 ∞ ∞ 11 ∞ 0 ∞ ∞ 0 0 ∞ ∞ ∞
11
Consider the path(1,4,5) node 8: Change all entries of fourth row and fifth column of
reduced matrix A to ∞ and set A(5,1) to ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
12 ∞ 11 ∞ 0 12 ∞ 11 ∞ ∞ 11 1 ∞ 0 ∞ ∞
A = 0 3 ∞ ∞ 2 ⇒ 0 3 ∞ ∞ ∞ ⇒ 0 3 ∞ ∞ ∞
∞ 3 12 ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
11 0 0 ∞ ∞ ∞ 0 0 ∞ ∞ ∞ 0 0 ∞ ∞
Consider the path(1,4,2,3) node 9: Change all entries of second row and third column to
∞ and set A(3,1) to ∞
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
∞ ∞ 11 ∞ 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
A = 0 ∞ ∞ ∞ 2 ⇒ ∞ ∞ ∞ ∞ 2 2 ⇒ ∞ ∞ ∞ ∞ 0
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
11 ∞ 0 ∞ ∞ 11 ∞ ∞ ∞ ∞ 0 ∞ ∞ ∞ ∞
11
A branch and bound technique is used to find solution to the knapsack problem. But we
cannot directly apply the branch and bound technique to the knapsack problem. Because the
branch and bound deals with only minimization problems. We modify the knapsack problem
to the minimization problem. The modified problem is
Min z = − p1 x1 − p2 x2 − ..... − pn xn
w1 x1 + w2 x2 + ..... + wn xn ≤ m
Xi=0 or 1.
Let cˆ( x) and uˆ( x) are the two cost functions such that cˆ( x) ≤ c( x) ≤ uˆ ( x) satisfying the
n
requirements where c( x) = − ∑ pi xi . The c(x) is the cost function for answer node x, which
i =1
lies between two functions called lower and upper bounds for the cost function c(x). The
search begins at the root node. Initially we compute the lower and upper bound at root node
called cˆ(1) and uˆ(1) . Consider the first variable x1 to take a decision. The x1 takes values
either 0 or 1. Compute the lower and upper bounds in each case of variable. These are the
nodes at the first level. Select the node whose cost is minimum i.e.
c( x) = min{c(lchild ( x)), c(rchild ( x))}
c( x) = min{cˆ(2)), cˆ(3))}
The problem can be solved by making a sequence of decisions on the variables x1,x2,…,xn
level wise. A decision on the variable xi involves determining which of the values 0 or 1 is to
be assigned, to it by defining c(x) recursively.
The path from root to the leaf node whose height is maximum is selected and is the solution
space for the knapsack problem.
cp - current profit
cw- current weight
k – k number of decisions
m - Capacity of knapsack
Algorithm Ubound(cp,cw,k,m)
{
b := cp;
c := cw;
for i:= k+1 to n do
{
if ( c + w[i] <= m ) then
{
c := c + w[i];
b := b - p[i];
}
}
return b;
}
Function U(.) for Knapsack problem
Node 1 Ubound(0,0,0,15)
i=1 i=2 i=3 i=4
c=2 c=6 c=12
b=-10 b=-20 b=-32
U(1)=-32
cˆ(1) = -32+3/9 x 18= -38
To calculate lower bound we allow fractions
Node2 Ubound(-10,2,1,15) x1=1
i=2 i=3 i=4
c=6 c=12
b= -20 b= -32
U(2)=-32
cˆ(2) = -32+3/9 x 18= -38
To calculate lower bound we allow fractions
Node 3 Ubound(0,0,1,15)
i=2 i=3 i =4
c=4 c = 10
b= -10 b = -22
U(3)=-22
cˆ(3) = -22+5/9 x 18= -32
To calculate lower bound we allow fractions
Node 4 Ubound(-20,6,2,15)
i=3 i=4
c=12
b=-32
U(4)=-32
cˆ(4) = -32+3/9 x 18= -38
To calculate lower bound we allow fractions
U(6)=-32
cˆ(6) = -32+3/9 x 18= -38
To calculate lower bound we allow fractions
Node 7 Ubound(-20,6,3,15)
i=4
c=15
b=-38
U(7)=-38
cˆ(7) = -38
Node 8 Ubound(-38,15,4,15)
U(8)=-38
cˆ(8) = -38
Node 9 Ubound(-20,6,4,15)
U(1)=-20
cˆ(1) = -20
p1x1+p2x2+p3x3+p4x4
10x1+10=1+12x0+18x1
=38
We need to consider all n items.
The tuple size is n
Now let us trace through the FIFOBB algorithm using the same knapsack instance. The
problem is to find most valuable subset of the items that fit in the knapsack.
Initially the root node i.e. node 1 be the E-Node. The Queue of Live nodes is empty.
Since this is not the solution Upper is initialized to u(1) = -32.
We assume the children are generated from left to right. Nodes 2 and 3 are generated and are
added to the queue.
2 3
Node 2 becomes the next E Node. Its children nodes are generated. Node 4 and 5 and are
added to Queue.
3 4 5
Node 3 the next e-Node, is expanded. Its Children nodes are generated. Node 6 and 7 gets
added to Queue.
4 5 6
Node 4 is expanded next. Nodes 8 and 9 are generated and are added to queue.
5 6 8 9
Node 9 is expanded Next. When node 12 is generated, upper and answer are updated to -38
and 12 respectively. Node 12 joins the queue of live nodes.
12
Node 13 is killed before it can get onto the queue of live nodes as cˆ (13) > upper.
The only remaining live node is node 12. It has no children and the search terminates.
The value of upper and the path from node 12 to the root is the output additional information
is needed to determine the xi values on this path.
Solution:
Convert the profits to –ve. , (p1,p2,p3,p4,p5)=(-10,-15,-6,-8,-4).
Calculate the lower bound and upper bound for each node.
Place the first item in knapsack. i.e. weight =4. Remaining weight is 12-4=8
Place second item i.e. with weight 6. Remaining weight = 8-6=2.
Since fractions are not allowed in calculation of upper bound, so we cannot place the third
and fourth items. place fifth item .
To calculate lower bound, place third item in bag considering fractions are allowed
2
∴Lower bound = −10 − 15 − × 6 = −29
3
∴ uˆ (1) = −29, cˆ(1) = −29.
For node 2, x1=1 means, we should place first item in the bag.
2
cˆ(2) = −10 − 15 − × 6 = −29
3
uˆ(2) = −10 − 15 − 4 = −29