Department of AIML & CSM

DESIGN AND ANALYSIS OF ALGORITHMS

STUDY MATERIAL

Module 5
B.Tech II Year – II Sem

PREPARED & COMPILED BY

Mrs.S.AshaJyothi
Assistant Professor,
Department of AI&ML
JBIET
Bhaskar Nagar, Yenkapally (V), Moinabad (M),
Ranga Reddy,District Hyderabad., Moinabad,
Hyderabad 500075Telangana,
India.
 MODULE 5
BRANCH AND BOUND

General Method, Applications: Travelling Sales Person Problem, 0/1 Knapsack Problem, LC
Branch and Bound Solution, FIFO Branch and Bound Solution.
NP-Hard and NP-Complete Problems: Basic Concepts, Non-Deterministic Algorithms, NP-Hard and
NP-Complete Classes, Cook’s Theorem.

Branch and bound

What is Branch and bound?

Branch and bound is one of the techniques used for problem solving. It is similar to the backtracking since
it also uses the state space tree. It is used for solving the optimization problems and minimization problems.
If we have given a maximization problem then we can convert it using the Branch and bound technique by
simply converting the problem into a maximization problem.

Let's understand through an example.

Jobs = {j1, j2, j3, j4}

P = {10, 5, 8, 3}

d = {1, 2, 1, 2}

The above are jobs, problems and problems given. We can write the solutions in two ways which are given
below:

Suppose we want to perform the jobs j1 and j2 then the solution can be represented in two ways:

The first way of representing the solutions is the subsets of jobs.

S1 = {j1, j4}

The second way of representing the solution is that first job is done, second and third jobs are not done, and
fourth job is done.

S2 = {1, 0, 0, 1}

The solution s1 is the variable-size solution while the solution s2 is the fixed-size solution.

First, we will see the subset method where we will see the variable size.

First method:
In this case, we first consider the first job, then second job, then third job and finally we consider the last
job.

As we can observe in the above figure that the breadth first search is performed but not the depth first
search. Here we move breadth wise for exploring the solutions. In backtracking, we go depth-wise whereas
in branch and bound, we go breadth wise.

Now one level is completed. Once I take first job, then we can consider either j2, j3 or j4. If we follow the
route then it says that we are doing jobs j1 and j4 so we will not consider jobs j2 and j3.

Now we will consider the node 3. In this case, we are doing job j2 so we can consider either job j3 or j4.
Here, we have discarded the job j1.
Now we will expand the node 4. Since here we are doing job j3 so we will consider only job j4.

Now we will expand node 6, and here we will consider the jobs j3 and j4.

Now we will expand node 7 and here we will consider job j4.
Now we will expand node 9, and here we will consider job j4.

The last node, i.e., node 12 which is left to be expanded. Here, we consider job j4.

The above is the state space tree for the solution s1 = {j1, j4}

Second method:

We will see another way to solve the problem to achieve the solution s1.

First, we consider the node 1 shown as below:

Now, we will expand the node 1. After expansion, the state space tree would be appeared as:

On each expansion, the node will be pushed into the stack shown as below:
Now the expansion would be based on the node that appears on the top of the stack. Since the node 5
appears on the top of the stack, so we will expand the node 5. We will pop out the node 5 from the stack.
Since the node 5 is in the last job, i.e., j4 so there is no further scope of expansion.

The next node that appears on the top of the stack is node 4. Pop out the node 4 and expand. On expansion,
job j4 will be considered and node 6 will be added into the stack shown as below:
The next node is 6 which is to be expanded. Pop out the node 6 and expand. Since the node 6 is in the last
job, i.e., j4 so there is no further scope of expansion.

The next node to be expanded is node 3. Since the node 3 works on the job j2 so node 3 will be expanded
to two nodes, i.e., 7 and 8 working on jobs 3 and 4 respectively. The nodes 7 and 8 will be pushed into the
stack shown as below:

The next node that appears on the top of the stack is node 8. Pop out the node 8 and expand. Since the node
8 works on the job j4 so there is no further scope for the expansion.
The next node that appears on the top of the stack is node 7. Pop out the node 7 and expand. Since the node
7 works on the job j3 so node 7 will be further expanded to node 9 that works on the job j4 as shown as
below and the node 9 will be pushed into the stack.

The next node that appears on the top of the stack is node 9. Since the node 9 works on the job 4 so there
is no further scope for the expansion.
The next node that appears on the top of the stack is node 2. Since the node 2 works on the job j1 so it
means that the node 2 can be further expanded. It can be expanded upto three nodes named as 10, 11, 12
working on jobs j2, j3, and j4 respectively. There newly nodes will be pushed into the stack shown as below:

In the above method, we explored all the nodes using the stack that follows the LIFO principle.

Third method

There is one more method that can be used to find the solution and that method is Least cost branch and
bound. In this technique, nodes are explored based on the cost of the node. The cost of the node can be
defined using the problem and with the help of the given problem, we can define the cost function. Once
the cost function is defined, we can define the cost of the node.

Let's first consider the node 1 having cost infinity shown as below:

Now we will expand the node 1. The node 1 will be expanded into four nodes named as 2, 3, 4 and 5 shown
as below:

Let's assume that cost of the nodes 2, 3, 4, and 5 are 25, 12, 19 and 30 respectively.
Since it is the least cost branch n bound, so we will explore the node which is having the least cost. In the
above figure, we can observe that the node with a minimum cost is node 3. So, we will explore the node 3
having cost 12.

Since the node 3 works on the job j2 so it will be expanded into two nodes named as 6 and 7 shown as
below:

The node 6 works on job j3 while the node 7 works on job j4. The cost of the node 6 is 8 and the cost of
the node 7 is 7. Now we have to select the node which is having the minimum cost. The node 7 has the
minimum cost so we will explore the node 7. Since the node 7 already works on the job j4 so there is no
further scope for the expansion.

Traveling Salesperson problem using branch and

bound
Given the vertices, the problem here is that we have to travel each vertex exactly once and reach back to
the starting point. Consider the below graph:
As we can observe in the above graph that there are 5 vertices given in the graph. We have to find the
shortest path that goes through all the vertices once and returns back to the starting vertex. We mainly
consider the starting vertex as 1, then traverse through the vertices 2, 3, 4, and 5, and finally return to vertex
1.

The adjacent matrix of the problem is given below:

Now we look at how this problem can be solved using the branch n bound.

Let's first understand the approach then we solve the above problem.

The graph is given below, which has four vertices:

Suppose we start travelling from vertex 1 and return back to vertex 1. There are various ways to travel
through all the vertices and returns to vertex 1. We require some tools that can be used to minimize the
overall cost. To solve this problem, we make a state space tree. From the starting vertex 1, we can go to
either vertices 2, 3, or 4, as shown in the below diagram.
From vertex 2, we can go either to vertex 3 or 4. If we consider vertex 3, we move to the remaining vertex,
i.e., 4. If we consider the vertex 4 shown in the below diagram:

From vertex 3, we can go to the remaining vertices, i.e., 2 or 4. If we consider the vertex 2, then we move
to remaining vertex 4, and if we consider the vertex 4 then we move to the remaining vertex, i.e., 3 shown
in the below diagram:

From vertex 4, we can go to the remaining vertices, i.e., 2 or 3. If we consider vertex 2, then we move to
the remaining vertex, i.e., 3, and if we consider the vertex 3, then we move to the remaining vertex, i.e., 2
shown in the below diagram:
The above is the complete state space tree. The state space tree shows all the possibilities. Backtracking
and branch n bound both use the state space tree, but their approach to solve the problem is different. Branch
n bound is a better approach than backtracking as it is more efficient. In order to solve the problem using
branch n bound, we use a level order. First, we will observe in which order, the nodes are generated. While
creating the node, we will calculate the cost of the node simultaneously. If we find the cost of any node
greater than the upper bound, we will remove that node. So, in this case, we will generate only useful nodes
but not all the nodes.

Let's consider the above problem.

As we can observe in the above adjacent matrix that 10 is the minimum value in the first row, 2 is the
minimum value in the second row, 2 is the minimum value in the third row, 3 is the minimum value in the
third row, 3 is the minimum value in the fourth row, and 4 is the minimum value in the fifth row.

Now, we will reduce the matrix. We will subtract the minimum value with all the elements of a row. First,
we evaluate the first row. Let's assume two variables, i.e., i and j, where 'i' represents the rows, and 'j'
represents the columns.
When i = 0, j =0

M[0][0] = ∞-10= ∞

When i = 0, j = 1

M[0][1] = 20 - 10 = 10

When i = 0, j = 2

M[0][2] = 30 - 10 = 20

When i = 0, j = 3

M[0][3] = 10 - 10 = 0

When i = 0, j = 4

M[0][4] = 11 - 10 = 1

The matrix is shown below after the evaluation of the first row:

Consider the second row.

When i = 1, j =0

M[1][0] = 15-2= 13

When i = 1, j = 1

M[1][1] = ∞ - 2= ∞

When i = 1, j = 2

M[1][2] = 16 - 2 = 14
When i = 1, j = 3

M[1][3] = 4 - 2 = 2

When i = 1, j = 4

M[1][4] = 2 - 2 = 0

The matrix is shown below after the evaluation of the second row:

Consider the third row:

When i = 2, j =0

M[2][0] = 3-2= 1

When i = 2, j = 1

M[2][1] = 5 - 2= 3

When i = 2, j = 2

M[2][2] = ∞ - 2 = ∞

When i = 2, j = 3

M[2][3] = 2 - 2 = 0

When i = 2, j = 4

M[2][4] = 4 - 2 = 2

The matrix is shown below after the evaluation of the third row:
Consider the fourth row:

When i = 3, j =0

M[3][0] = 19-3= 16

When i = 3, j = 1

M[3][1] = 6 - 3= 3

When i = 3, j = 2

M[3][2] = 18 - 3 = 15

When i = 3, j = 3

M[3][3] = ∞ - 3 = ∞

When i = 3, j = 4

M[3][4] = 3 - 3 = 0

The matrix is shown below after the evaluation of the fourth row:

Consider the fifth row:

When i = 4, j =0

M[4][0] = 16-4= 12

When i = 4, j = 1

M[4][1] = 4 - 4= 0

When i = 4, j = 2

M[4][2] = 7 - 4 = 3
When i = 4, j = 3

M[4][3] = 16 - 4 = 12

When i = 4, j = 4

M[4][4] = ∞ - 4 = ∞

The matrix is shown below after the evaluation of the fifth row:

The above matrix is the reduced matrix with respect to the rows.

Now we reduce the matrix with respect to the columns. Before reducing the matrix, we first find the
minimum value of all the columns. The minimum value of first column is 1, the minimum value of the
second column is 0, the minimum value of the third column is 3, the minimum value of the fourth column
is 0, and the minimum value of the fifth column is 0, as shown in the below matrix:

Now we reduce the matrix.

Consider the first column.

When i = 0, j =0

M[0][0] = ∞-1= ∞

When i = 1, j = 0

M[1][0] = 13 - 1= 12

When i = 2, j = 0

M[2][0] = 1 - 1 = 0

When i = 3, j = 0

M[3][0] = 16 - 1 = 15
When i = 4, j = 0

M[4][0] = 12 - 1 = 11

The matrix is shown below after the evaluation of the first column:

Since the minimum value of the first and the third columns is non-zero, we will evaluate only first and third
columns. We have evaluated the first column. Now we will evaluate the third column.

Consider the third column.

When i = 0, j =2

M[0][2] = 20-3= 17

When i = 1, j = 2

M[1][2] = 13 - 1= 12

When i = 2, j = 2

M[2][2] = 1 - 1 = 0

When i = 3, j = 2

M[3][2] = 16 - 1 = 15

When i = 4, j = 2

M[4][2] = 12 - 1 = 11

The matrix is shown below after the evaluation of the third column:
The above is the reduced matrix. The minimum value of rows is 21, and the columns is 4. Therefore, the
total minimum value is (21 + 4) equals to 25.

Let's understand that how to solve this problem using branch and bound with the help of a state-
space tree.

To make a state-space tree, first, we consider node 1. From node 1, we can go either to nodes 2, 3, 4, or 5
as shown in the below image. The cost of node 1 would be the cost which we achieved in the above-reduced
matrix, i.e.., 25. Here, we will also maintain the upper bound. Initially, the upper bound would-be infinity.

Now, consider node 2. It means that we are moving from node 1 to node 2. Make the first row and second
column as infinity shown in the below matrix:

Once we move from node 1 to node 2, we cannot move back to node 1. Therefore, we have to make 2 to 1
as infinity shown in the below matrix:
Since each row and column contains atleast one zero value; therefore, we can say that above matrix has
been reduced. The cost of reduction of node 2 is c(1, 2) + r + r` = 10 + 25 + 0 = 35.

Now we will find the minimum value of each column of the new reduced matrix. The minimum value of
the first column is 11 and the minimum value of other three columns is 0.

Now, consider the node 3. It means that we are moving from the node 1 to node 3. Make the first row and
third column as infinity shown in the below matrix:

Once we move from the node 1 to node 3, we cannot move back to the node 1. Therefore, we have to make
3 to 1 as infinity shown in the below matrix:

Since each row and column contains atleast one zero value; therefore, we can say that above matrix has
been reduced. The cost of reduction of node 3 is c(1, 3) + r + r` = 17 + 25 + 11= 53.

Now, consider the node 4. It means that we are moving from the node 1 to node 4. Make the first row and
forth column as infinity shown in the below matrix:
Once we move from the node 1 to node 4, we cannot move back to the node 1. Therefore, we have to make
4 to 1 as infinity shown in the below matrix:

Since each row and column contains atleast one zero value; therefore, we can say that above matrix has
been reduced. The cost of reduction of node 4 is c(1, 4) + r + r` = 0 + 25 + 0 = 25.

Now, consider the node 5. It means that we are moving from the node 1 to node 5. Make the first row and
fifth column as infinity shown in the below matrix:

Once we move from the node 1 to node 5, we cannot move back to the node 1. Therefore, we have to make
5 to 1 as infinity shown in the below matrix:

Since each row and column contains atleast one zero value; therefore, we can say that above matrix has
been reduced. In this case, second and third rows are non-zero. Therefore, we have to first find the minimum
values of both the rows. The minimum value of second row is 2; therefore, we subtract 2 from all the
elements of the second row. The elements of second row would be:

A[1][0] = 12-2 = 10

A[1][1] = ∞

A[1][2] = 11 - 2 = 9

A[1][3] = 2 - 2 = 0

A[1][4] = ∞ - 2 = ∞

As we can observe now that the second row contains one zero value.

The cost of reduction of node 5 is c(1, 5) + r + r` = 1 + 25 + 5 = 31

Since the node 4 has the minimum cost, i.e., 25. So we will explore the node 4 first. From the vertex 4, we
can go either to the vertex 2, 3 or 5 as shown in the below image:

Now we have to calculate the cost of the path from the vertex 4 to 2, vertex 4 to 3, and vertex 4 to 5. Here,
we will use the matrix of node 4 to find the cost of all the nodes.
First, we consider the path from the vertex 4 to the vertex 2. We make fourth row as ∞ and second column
as ∞. Since we cannot move back from 2 to 1, so 1 to 2 is also infinity as shown in the below matrix:

Since all the rows and columns have atleast one zero value. Therefore, we can say that this matrix is already
reduced. So, there would be no reduction cost. The cost of reduction of node 2 is c(4, 2) + r + r` = 3 + 25 +
0 = 28

Now we have to calculate the cost of the path from the vertex 4 to the vertex 3. We make fourth row and
third column as infinity as shown in the below matrix. Since we cannot move from the vertex 3 to 1, so we
make 3 to 1 as infinity shown in the below matrix:

Now we will check whether each row and column contain atleast one zero value or not. First, we observe
all the rows. Since the third row does not have a zero value, so we first find the minimum value of the third
row. The minimum value of the third row is 2, so we subtract 2 from all the elements of the third row. The
elements of third row would be:

A[2][0] = ∞ - 2 = ∞

A[2][1] = 3 - 2 = 1

A[2][2] = ∞ - 2 = ∞

A[2][3] = ∞ - 2 = ∞

A[2][4] = 2 - 2 = 0

As we can observe now that the third row contains one zero value.

The first column does not contain the zero value. The minimum value of the first column is 11. We subtract
11 from all the elements of the first column. The elements of first column would be:

A[0][0] = ∞ - 11 = ∞

A[1][0] = 12 - 11 = 1

A[2][0] = ∞ - 11= ∞

A[3][0] = ∞ - 11= ∞

A[4][0] = 11 - 11 = 0

As we can observe now that the first column contains one zero value. The total minimum cost is 11 +2
equals to 13. The cost of reduction of node 3 is c(4, 3) + r + r` = 12 + 25 + 13 = 50.

Now we will calculate the cost of the path from the vertex 4 to 5. We make fourth row and fifth column as
infinity. Since we cannot move back from the node 5 to 1, so we also make 1 to 5 as infinity shown in the
below matrix:

Now we will check whether each row and column contain atleast one zero value or not. First, we observe
all the rows. The second row does not contain the zero value, so we find the minimum value of the second
row. The minimum value is 11 so we subtract 11 from all the elements of the second row. The elements of
second row would be:
A[1][0] = 12 - 11 = 1

A[1][1] = ∞ - 11 = ∞

A[1][2] = 11 - 11 = 0

A[1][3] = ∞ - 11 = ∞

A[1][4] = ∞ - 11 = ∞

As we can observe now that the second row contains one zero value. The cost of reduction of node 5 is c(4,
5) + r + r` = 0 + 25 + 11 = 36.

Now we will compare the cost of all the leaf nodes. The node with a cost 28 is minimum so we will explore
this node. The node with a cost 28 can be further expanded to the nodes 3 and 5 as shown in the below
figure:

Now we have to calculate the cost of both the nodes, i.e., 3 and 5. First we consider the path from node 2
to node 3. Consider the matrix of node 2 which is shown below:

We make second row and third column as infinity. Also, we cannot move back from the node 3 to node 1
so we make 3 to 1 as infinity as shown in the below matrix:

Now we will check whether any row contains zero value or not. Since third row does not contain any zero
value so we will find the minimum value of the third row. The minimum value of the third row is 2 so we
subtract 2 from all the elements of the third row. The elements of third row would be:

A[2][0] = ∞ - 2 = ∞

A[2][1] = ∞ - 2 = ∞

A[2][2] = ∞ - 2 = ∞

A[2][3] = ∞ - 2 = ∞

A[2][4] = 2 - 2 = 0
Since fifth row does not contain any zero value so we will find the minimum value of the fifth row. The
minimum value of the fifth row is 11 so we subtract 11 from all the elements of the fifth row.

A[4][0] = 11 - 11 = 0

A[4][1] = ∞ - 11 = ∞

A[4][2] = ∞ - 11 = ∞

A[4][3] = ∞ - 11 = ∞

A[4][4] = ∞ - 11 = ∞

The total minimum cost is (11 + 2) equals to 13. The cost of reduction of node 3 is c(2, 3) + r + r` = 11 +
28 + 13 = 52.

Consider the path from node 2 to node 5. Make the fourth row and third column as infinity. Since we cannot
move back from the node 5 to node 1 so make 1 to 5 also as infinity as shown in the below matrix:

Now we will check whether any row contains zero value or not. Since every row and column contains a
zero value; therefore, the above matrix is the reduced matrix.

The cost of reduction of node 5 is c(2, 5) + r + r` = 0 + 28 + 0 = 28

Now we will find out the leaf node with a minimum cost. The node 5 with a cost 28 is minimum so we
select node 5 for the further exploration. The node 5 can be further expanded to the node 3 as shown in the
below figure:

Here, we will use the matrix of node 5 having cost 28 as shown below:

Consider the path from node 5 to node 3. Make the fifth row and third column as infinity. Since we cannot
move back from the node 3 to node 1 so make 1 to 5 also as infinity as shown in the below matrix:

Now we will check whether any row contains zero value or not. Since every row and column contains a
zero value; therefore, the above matrix is the reduced matrix.

The cost of reduction of node 3 is c(5, 3) + r + r` = 0 + 28 + 0 = 28.

Finally, we traverse all the nodes. The upper value is updated from infinity to 28. We will check whether
any leaf node has a value less than 28. Since no leaf node contains the value less than 28 so we discard all
the leaf nodes from the tree as shown below:

The path of the tour would be 1->4->2->5->3.

0/1 Knapsack using Branch and Bound

Branch and bound is an algorithm design paradigm which is generally used for solving combinatorial
optimization problems. These problems typically exponential in terms of time complexity and may
require exploring all possible permutations in worst case. Branch and Bound solve these problems
relatively quickly.
Let us consider below 0/1 Knapsack problem to understand Branch and Bound. Given two integer
arrays val[0..n-1] and wt[0..n-1] that represent values and weights associated with n items
respectively.
Find out the maximum value subset of val[] such that sum of the weights of this subset is smaller than
or equal to Knapsack capacity W. Let us explore all approaches for this problem.
1. A Greedy approach is to pick the items in decreasing order of value per unit weight. The
Greedy approach works only for fractional knapsack problem and may not produce correct
result for 0/1 knapsack.
2. We can use Dynamic Programming (DP) for 0/1 Knapsack problem. In DP, we use a 2D
table of size n x W. The DP Solution doesn’t work if item weights are not integers.
3. Since DP solution doesn’t always work, a solution is to use Brute Force. With n items, there
are 2n solutions to be generated, check each to see if they satisfy the constraint, save
maximum solution that satisfies constraint. This solution can be expressed as tree.



4. We can use Backtracking to optimize the Brute Force solution. In the tree representation,
we can do DFS of tree. If we reach a point where a solution no longer is feasible, there is no
need to continue exploring. In the given example, backtracking would be much more
effective if we had even more items or a smaller knapsack capacity.
 
Branch and BoundThe backtracking based solution works better than brute force by ignoring
infeasible solutions. We can do better (than backtracking) if we know a bound on best possible solution
subtree rooted with every node. If the best in subtree is worse than current best, we can simply ignore
this node and its subtrees. So we compute bound (best solution) for every node and compare the bound
with current best solution before exploring the node. Example bounds used in below diagram
are, A down can give $315, B down can $275, C down can $225, D down can $125 and E down can
$30. In the next article, we have discussed the process to get these bounds.
Branch and bound is very useful technique for searching a solution but in worst case, we need to fully
calculate the entire tree. At best, we only need to fully calculate one path through the tree and prune the
rest of it.

Pseudo code

function knapsack(items, max_weight):

best_value = 0
queue = [{items: [], value: 0, weight: 0}]
while queue is not empty:
node = queue.pop()
if node is a leaf node:
update best_value if necessary
else:
for each remaining item:
child = create child node for item
if child is promising:
queue.append(child)
return best_value
function is_promising(node, max_weight, best_value):
if node.weight > max_weight:
return False
if node.value + bound(node.items) < best_value:
return False
return True

Time Complexity: O(N), as only one path through the tree will have to be traversed in the best case and
its worst time complexity is still given as O(2^N) .
 Least Cost Branch and Bound
The branch and bound can be solved using FIFO, LIFO, and least count strategies. The selection rule used
in the FIFO or LIFO linked list can sometimes be erroneous, as it does not give preference to certain nodes,
and hence may not find the best response as quickly. To overcome this drawback we use the least cost
method, which is also considered to be an intelligent method as it uses an intelligent ranking method. It is
also referred to as the approximate cost function “C”.

The least-cost method of branch and bound selects the next node based on the Heuristic Cost Function, and
it picks the one with the least count, therefore it is one of the best methods. In the 0/1 knapsack problem,
we need to maximize the total value, but we cannot directly use the least count branch and bound method
to solve this.

Therefore, we convert this into a minimization problem by taking the negative of the given values.

The steps to solve the 0/1 Knapsack problem is as follows:

1. We need to sort the items based on the value/ weight ratio.

2. A dummy node is inserted in the priority queue.
3. Then we follow the following steps until the priority queue is empty:

 The peak element from the priority queue is extracted and inserted into the current node.
 When the upper bound of the current node is less than the minimum lower bound then we proceed
further with the next element.
 We do not consider the elements with upper bounds more than the minimum lower bound as the
upper bound stores the best value, and if the best value is itself not optimal than the minimum lower
bound then exploring further is of no use.

1. Then update the path array.

2. We check the node level of the current node, if the lower node level of the current node is less than
the final lower bound then we update the final path and final lower bound. If it is not less than the
final lower bound then we continue with the next element.
3. The lower and upper bounds of the right child of the current node are calculated.
4. Then we calculate the lower and upper bounds of the left child of the current node if the current
item can be inserted in a knapsack.
5. We finally update the minimum lower bound and insert the children, in case the upper bound is less
than the minimum lower bound.

FIFO Branch and Bound solution

FIFO Branch and Bound solution is one of the methods of branch and bound.
Branch and Bound is the state space search method where all the children E-node that is
generated before the live node, becomes an E- node.
FIFO branch and bound search is the one that follows the BFS like method. It does so as the list
follows first in and first out.
Some key terms to keep in mind while proceeding further:
What is a live node?
A live node is the one that has been generated but its children are not yet generated.

What is an E node?
An E node is the one that is being explored at the moment.

What is a dead node?

A dead node is one that is not being generated or explored any further. The children of the dead
node have already been expanded to their full capacity.

To proceed further with the FIFO branch and bound we use a queue.

To begin with, we keep the queue empty. Then we assume a node 1.

In FIFO search the E node is assumed as node 1. After that, we need to generate children of Node 1. The
children are the live nodes, and we place them in the queue accordingly. Then we delete node 2 from the
queue and generate children of node 2.

Next, we delete another element from the queue and assume that as the E node. We then generate all its
children.

In the same process, we delete the next element and generate their children. We keep repeating the process
till the queue is covered and we find a node that satisfies the conditions of the problem. When we have
found the answer node, the process terminates.

Let us understand it through an example:

We start by taking an empty queue:

In the given diagram we have assumed that node 12 is the answer node.

As mentioned we start by assuming node 1 as the E node and generate all its children.

2 3 4

Then we delete node 1 from the queue, take node 2 as the E node, and generate all its children:

3 4 5 6

We delete the next element and generate children for the next node, assuming it to be the E node.

4 5 6

We repeat the same process. The generated children of 4, that is node 9 are killed by the boundary function.
 5 6

Similarly, we proceed further, but the children of nodes 5, meaning the nodes 10, and 11 are also generated
and killed by boundary function. The last standing node in the queue is 6, the children of node 6 are 12, and
it satisfies all the conditions for the problem, therefore it is the answer node.

Basic concept of NP Hard & NP Complete

REFER PPT SHARED IN GROUP

Non-deterministic algorithms
What is a non-deterministic algorithm?

A deterministic signal gives only a single output for the same input even when it is on different
runs. On the contrary, a non-deterministic algorithm gives different outputs, as it travels through
different routes.
In a situation where finding an exact solution is expensive and difficult using deterministic
algorithms, non-deterministic algorithms are used. They are helpful when it comes to finding
approximate solutions.
A non-deterministic algorithm can run on a deterministic computer with multiple parallel
processors, and usually takes two phases and output steps. The first phase is the guessing phase,
and the second is the verifying phase. In the first phase, we make use of arbitrary characters to
run the problem, and in verifying phase, it returns true or false values for the chosen string. An
example that follows the concept of a non-deterministic algorithm is the problem of P vs NP in
computing theory. They are used in solving problems that allow multiple outcomes. Every output
returned is valid, irrespective of their difference in choices during the running process.
Another example of the implementation of the non-deterministic algorithm is the mathematical
use in computational models like a non-deterministic finite automaton.
While looking at the complex computational theory, we notice that the non-deterministic have the
capability to allow multiple continuations at every step.
It is also widely used for game AI. It is used by the game engine to make non-player characters or
referred to as NPC to learn the behavior of the game, such as the tactics and the pattern. This
feature is very useful in combat games where an element of unpredictability can be added to make
the game more interesting. This is done so that the scenarios are not as repetitive and the players
do not get bored of it. This keeps the players hooked and hence increases the game-play life.
 What are the major differences betwe en deterministic and non-deterministic
algorithms?
Deterministic algorithm Non-deterministic algorithm
 In a deterministic algorithm, the  In a non-deterministic algorithm, the computer
computer produces the same output may produce different outputs for the same input
while going through different steps. while going through different runs.
 The non-deterministic algorithm cannot solve
 The deterministic algorithm can problems in polynomial time, and cannot
determine the next step. determine the next step.
 The output has a certain degree of randomness to
 The output is not random. it.
 In a deterministic algorithm, the next  In a non-deterministic algorithm, the next step
step can be determined. cannot be determined.

Non-deterministic algorithms do not have any s theytandard programming language operators, neither
are part of any standard programming lang uage.

NP-Hard and NP-Comp lete classes

A problem is considered to be in NPC, that is NP-complete class, if it is a part of NP, and is as tough as
any other problem in NP. And, a problem is considered to be NP-hard if all the problems in NP can be
reduced in polynomial time. If a problem can be solved in polynomial time and is in NP, it means they
have a polynomial time complexity.
These types of problems are referred to as NP-complete problems.
The NP-complete phenomenon is important for theo retical and practical purposes.

How to define NP-completeness?

To be NP-complete it must fulfill the following crit eria:
 The language must be in NP.
 It must be polynomial-time reducible.
If the language fails to satisfy the first criteria but fulfills the second one it is considered to be NP-hard.
The NP-hard problem cannot be solved in polynomial time. But, if a problem is proved to be NPC, we do
not focus on finding the most efficient algorithm forit, instead, we focus on designing an approximation
algorithm.
Some examples of NP-complete problems are as fol lows:
 To determine if a graph has a Hamiltonian c ycle.
 To determine if a boolean formula is satisfac tory.
Some examples of NP-hard problems are as follows :
 The traveling salesperson problem or TSP.
 Set cover problem
 The circuit satisfiability problem.
 Vertex cover problem.
Why do we consider the traveling salesperson problem is NP-complete?
To understand why we consider TSP as NP-complete we must first know what the traveling salesperson
problem is.
In the traveling salesperson problem, we have a salesperson who has a list of cities to visit, exactly once,
and must end at the source where he started his journey from.
Proof:
For TSP to be NP-complete it must fulfill the criteria we discussed above, so we start by proving that TSP
belongs to NP. In the TSP problem we check if the tour of the listed cities contains each vertex once.
Then we calculate the total cost of the edges of the tour. In the end, we decide which tour has the least
cost and can be completed in polynomial time. Therefore, we can conclude that TSP belongs to NP.
The second criterion is to prove that TSP is NP-hard, we do this by showing that the Hamiltonian cycle is
less than or equal to TSP, as we already know that the Hamiltonian cycle is NP-Hard.
We assume a graph, that has G= (V, E).
Let G be the graph for the Hamiltonian cycle.
We construct an instance of TSP, the graph for which is denoted by: G’= (V’, E’).
Let us suppose the Hamiltonian cycle h, exists in G. It implies that the cost of each edge is zero, in G’,
and each edge belongs to E. Therefore, h also has a cost of 0 in G’. So if the graph of the Hamiltonian
cycle G has a cost of 0, then graph G’for TSP, also has a total tour cost of zero. And vice versa.

Hence, we can conclude that TSP is NP-hard, and hence NP-complete.

Cook’s Theorem
Following are the four theorems by Stephen Cook −

Theorem-1
If a set S of strings is accepted by some non-deterministic Turing machine within polynomial time,
then S is P-reducible to {DNF tautologies}.

Theorem-2
The following sets are P-reducible to each other in pairs (and hence each has the same polynomial degree
of difficulty): {tautologies}, {DNF tautologies}, D3, {sub-graph pairs}.

Theorem-3
 For any TQ(k) of type Q, TQ(k)k√(logk)2 is unbounded

 There is a TQ(k) of type Q such that TQ(k)⩽2k(logk)2

 Theorem-4
If the set S of strings is accepted by a non-deterministic machine within time T(n) = 2n, and if TQ(k) is an
honest (i.e. real-time countable) function of type Q, then there is a constant K, so S can be recognized by
a deterministic machine within time TQ(K8n).

 First, he emphasized the significance of polynomial time reducibility. It means that if we have a
polynomial time reduction from one problem to another, this ensures that any polynomial time
algorithm from the second problem can be converted into a corresponding polynomial time
algorithm for the first problem.
 Second, he focused attention on the class NP of decision problems that can be solved in
polynomial time by a non-deterministic computer. Most of the intractable problems belong to this
class, NP.
 Third, he proved that one particular problem in NP has the property that every other problem in
NP can be polynomially reduced to it. If the satisfiability problem can be solved with a
polynomial time algorithm, then every problem in NP can also be solved in polynomial time. If
any problem in NP is intractable, then satisfiability problem must be intractable. Thus,
satisfiability problem is the hardest problem in NP.
 Fourth, Cook suggested that other problems in NP might share with the satisfiability problem this
property of being the hardest member of NP.

***************************************************