Sanjivani Rural Education Society’s

Sanjivani College of Engineering, Kopargaon-423 603

(An Autonomous Institute, Affiliated to Savitribai Phule Pune University, Pune)
NAAC ‘A’ Grade Accredited, ISO 9001:2015 Certified

Department of Computer Engineering

(NBA Accredited)

Subject- Design and Analysis of Algorithms (DAA) [CO 301)]

Unit 4:- Backtracking and Branch -and-Bound

Prof. Abhijit S. Bodhe

Assistant Professor
Department of Computer Engineering
E-mail : [email protected]
Contact No: 7709 340 570
 Unit 4:-Backtracking and Branch -and-Bound
• Backtracking: Principle, Control Abstraction,
• Time Complexity Analysis,
• 8-Queen Problem.
• Case Study: Sum of Subsets Problem.
• Application of BT: Sudoku Solver App

• Branch-and-Bound: Principle, Control Abstraction,

• Time Complexity Analysis, Knapsack Problem.
• Case Study :- Traveling Salesperson Problem,
• Application: Airline Crew Scheduling problem
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 2
 Backtracking
• Backtracking is a general algorithm for finding all (or some)
solutions to some computational problems, that incrementally builds
candidates to the solutions, and abandons each partial candidate
(“backtracks”) as soon as it determines that the candidate cannot
possibly be completed to a valid solution.
• It uses recursive calling to find the solution by building a solution step
by step increasing values with time. It removes the solutions that
doesn't give rise to the solution of the problem based on the constraints
given to solve the problem.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 3

 Backtracking
• Backtracking algorithm is applied to some specific types of problems,
1. Decision problem used to find a feasible solution of the problem.
2. Optimization problem used to find the best solution that can be
applied.
3. Enumeration problem used to find the set of all feasible solutions of
the problem,

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 4

 Backtracking
• In backtracking problem, the algorithm tries to find a sequence path to
the solution which has some small checkpoints from where the
problem can backtrack if no feasible solution is found for the problem.
• It uses recursive calling to find a solution set by building a solution
step by step, increasing levels with time. In order to find these
solutions, a search tree named state-space tree is used. In a state-space
tree, each branch is a variable, and each level represents a solution.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 5

 Backtracking
• A backtracking algorithm uses the depth-first search method.
When it starts exploring the solutions, a bounding function is applied
so that the algorithm can check if the so-far built solution satisfies the
constraints.
• If it does, it continues searching. If it doesn’t, the branch would be
eliminated, and the algorithm goes back to the level before.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 6

 Control Abstraction

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 7

Generalized Algorithm

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 8

 Application:- 8-queen problem
• Eight Queens Puzzle The eight queens puzzle is the problem of
placing eight chess queens on an 8×8 chessboard so that no two
queens threaten each other.
• Thus, a solution requires that no two queens share the same row,
column, or diagonal.
• The eight queens puzzle has 92 distinct solutions. If solutions that
differ only by the symmetry operations of rotation and reflection of the
board are counted as one, the puzzle has 12 Unique solutions.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 9

 Control Abstraction/Design
1) Start in the leftmost column b) If placing queen in [row, column] leads to a
2) If all queens are placed solution then return true.
return true c) If placing queen doesn't lead to a solution
3) Try all rows in the current column. Do then unmark this [row, column] (Backtrack)
following and go to step (a) to try other rows.
for every tried row.
3) If all rows have been tried and nothing worked,
a) If the queen can be placed safely in this row return false to trigger backtracking.
then mark this [row, column] as part of the
solution and recursively check if placing
queen here leads to a solution.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE,

10
 Control Abstraction for 8 queen
• Create a function backtrack that simply places the queens on corresponding rows and
columns and marks them visited.
• The working of backtrack is as follows:
• If the current row is equal to 8, then a solution has been found. Therefore, add this to
the answer.
• Traverse through the columns of the current row. At each column, try to place the queen
in (row, col):
• Calculate the diagonal and anti-diagonal which the current square belongs to. If it is unvisited, place
the queen in the (row, col).
• Skip this column, if the queen cannot be visited.
• If the queen has been placed successfully, call the backtrack function of row + 1.
• Since, all paths have now been explored, clear the values of the queens placed so far and the visiting
arrays, so that next distinct solution can be found.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 11

 Time Complexity
• O(N!), where N is the size of the board.
• backtrack function explores all paths possible by placing the queens on
the rows and the columns. It keeps a visiting set to keep track of the
rows and columns which have been already used.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 12

Solutions-Exact enumeration

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 13

 Case Study:- Sum of subsets problem
• The SUBSET-SUM problem involves determining whether or not a subset
from a list of integers can sum to a target value.
OR
• Given an array of integers and a sum, the task is to have all subsets of given
array with sum equal to the given sum.
OR
• Given a set of positive integers, and a value sum, determine that the sum of
the subset of a given set is equal to the given sum.
• https://codecrucks.com/sum-of-subsets-how-to-solve-using-backtracking/

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 14

 Sum of subsets problem statement
• Example:-Consider the sum-of-subset problem, n = 4, Sum = 13, and
w1 = 3, w2 = 4, w3 = 5 and w4 = 6. Find a solution to the problem using
backtracking. Show the state-space tree leading to the solution.

• Solution:-The correct combination to get the sum of M = 13 for given

W = {3, 4, 5, 6} is [3, 4, 6]. The solution vector for [3, 4, 6] would be
X = [1, 1, 0, 1] because element 5 is not chosen, so X[3] = 0.
• The solution is often represented using the solution vector

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 15

State Space Tree:-Solution

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 16

 Control Abstraction-Sum of SubSet
1.Start with an empty set
2.Add the next element from the list to the set
3.If the subset is having sum M, then stop with that subset as solution.
4.If the subset is not feasible or if we have reached the end of the set,
then backtrack through the subset until we find the most suitable
value.
5.If the subset is feasible (sum of seubset < M) then go to step 2.
6.If we have visited all the elements without finding a suitable subset
and if no backtracking is possible then stop without solution.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 17

Example:- M = 35 & w = {5, 7, 10, 12, 15, 18, 20}

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 18

 Control Abstraction /Algorithm
SUB_SET_PROBLEM (i, sum, W, remSum)
// Description : Solve sub of subset problem using backtracking
// Inputs :
1. W: Number for which subset is to be computed
2. i: Item index
3. sum : Sum of integers selected so far
4. remSum : Size of remaining problem i.e. (W – sum)
// Output : Solution tuple X

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 19

 Control Abstraction /Algorithm/Programing Logic

If FEASIBLE_SUB_SET(i) == 1
{ If (sum == W) then print X[1…i] end }
Else
{ X[i + 1] ← 1
SUB_SET_PROBLEM(i + 1, sum + w[i] + 1, W, remSum – w[i] + 1 )
X[i + 1] ← 0 // Exclude the ith item
SUB_SET_PROBLEM(i + 1, sum, W, remSum – w[i] + 1 ) end
Function FEASIBLE_SUB_SET(i)
If (sum + remSum ≥ W) AND (sum == W) or (sum + w[i] + 1 ≤ W) then
return 0
End
return 1
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 20
 Complexity Analysis
• To derive the complexity of sum of the subset problem, In the state-
space tree, at level i, the tree has 2i nodes.
• So, given n items, the total number of nodes in the tree would be 1 + 2
+ 22 + 23 + .. 2n.
• T(n) = 1 + 2 + 22 + 23 + .. 2n = 2n+1 – 1 = O(2n)
• Thus, sum of sub set problem runs in exponential order.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 21

 Application:- Sudoku Solver App
• https://www.youtube.com/watch?v=rctRBFE7wmg&ab_channel=SaiA
nishMalla

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 22

Difference –BT & B&B

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 23

 Branch-and-Bound (B&B): Principle
• Differ form backtracking, B&B is to cut off a branch of the problem’s state-
space tree as soon as we can deduce that, it cannot lead to a solution.
• An optimal solution is a feasible solution with the best value of the objective
function. (Eg. The most valuable subset of items that fit the knapsack).
• We terminate a search path at the current node in a state-space tree of a B&B
algorithm for any one of the following three reasons:
1. The value of the node is not better than the value of the best solution seen so far.
2. The node represents no feasible solutions because the constraints of the problem are
already violated.
3. The subset of feasible solutions represented by the node consists of a single point

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 24

 B&B SPT Terminology
• Branch and Bound refers to all state space search methods in which all
children of the E-node are generated before any other live node
becomes the E-node.
• Live node is a node that has been generated but whose children have
not yet been generated.
• E-node is a live node whose children are currently being explored. In
other words, an E-node is a node currently being expanded.
• Dead node is a generated node that is not to be expanded or explored
any further. All children of a dead node have already been expanded.
• Branch and Bound is the generalization of both tree traversal
strategies, BFS and D- search.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 25

 Control Abstraction: Bounding function
• A good bounding function helps to prune (cut) efficiently the tree, leading to
a faster exploration of the solution space.
• A branch and bound method searches a state space tree using any search
mechanism (DFS/BFS) in which all the children of the E-node are generated
before another node becomes the E-node.
• We assume that each answer node x has a cost c(x) associated with it and
that a minimum-cost answer node is to be found.
• Three common search strategies are FIFO(DFS), LIFO(BFS), and LC(Least
Cost). The three search methods differ only in the selection rule used to
obtain the next E-node.
• LC search terminates only when either an answer node is found or the entire
state space tree has been generated and searched.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 26
 Control abstraction of Branch & Bound: LC method
Algorithm LCSearch(t)
{ //Search t for an answer node
if *t is an answer node then output *t and return; E := t; //E-node.
initialize the list of live nodes to be empty; repeat
{ for each child x of E do
if x is an answer node then output the path from x to t and return; Add (x);
//x is a new live node.
(x à parent) := E; // pointer for path to root }
if there are no more live nodes then
{ write (“No answer node”); return; }
E := Least();
} until (false);
}
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 27
 Application 1:- 0/1 knapsack problem

• Given n positive weights wi, n positive profits pi,

a positive number m that is the knapsack capacity,

The problem calls for choosing a subset of the weights

such that:

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 28

 Control Abstraction 0/1 knapsack problem: using LC method

• we will calculate lower bound and upper bound for each node in State Space
Tree (SPT).
• Knapsack problem is maximization problem, but branch and bound technique
is applicable for only minimization problems. In order to convert
maximization problem into minimization problem we have to take negative
sign for upper bound and lower bound.
• We choose the path in SPT, which has minimum difference of upper bound
and lower bound. If the difference is equal then we choose the path by
comparing upper bounds and we discard node with maximum upper bound.
• fractions are allowed in calculation of lower bound.(L)
• No fractions are allowed in calculation of upper bound(u)

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 29

 Steps:- 0/1 Knapsack BnB LC Method
1. Derive state space tree.
2. Compute lower bound hat{c}(.) hatc(.) and upper bound u(.)u(.) for
each node in state space tree.
3. If lower bound is greater than upper bound than kill that node.
4. Else select node with minimum lower bound as E-node.
5. Repeat step 3 and 4 until all nodes are examined.
6. The node with minimum lower bound value hat{c}(.)hatc(.) is the
answer node. Trace the path from leaf to root in the backward
direction to find the solution tuple.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 30

 Problem Instance : 0/1 Knapsack
• Consider the instance: M = 15, n = 4, (P1, P2, P3, P4) = (10, 10, 12,
18) and (w1, w2, w3, w4) = ( 2, 4, 6, 9).
Solve using branch and bound method draw state space tree wit h all
possible nodes, Use LC method.

Solution PPT

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 31

 Application 2:- TSP
• Problem: A salesman spends his time in visiting cities cyclically. In his tour he visits
each city just once, and finishes up where he started. In which order should he visit to
minimize the distance traveled.
• is defined as “given n cities and distance between each pair of cities, find out the path
which visits each city exactly once and come back to starting city, with the constraint of
minimizing the travelling distance.”
• The problem an be represented using a graph. Let G=(V,E) be a directed graph with cost
cij, cij >0 for all <i j> E and cij = ∞ for all <j,t> E. |V| = n and n>1. We know that tour of
a graph includes all vertices in V and cost of tour is the sum of the cost of all edges on
the tour.
• Hence the traveling salesman problem is to minimize the cost.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 32

 Example
• Find the solution of following travelling salesman problem using
branch and bound method.
• https://codecrucks.com/travelling-salesman-problem-solved-using-branch-and-
bound/

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 33

• Refer Solution Here:-
• https://codecrucks.com/travelling-salesman-problem-solved-using-
branch-and-bound/

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 34

 TSP using BnB- Control Abstraction
1. Initially, graph is represented by cost matrix C, where,
Cij = cost of edge, if there is a direct path from city i to city j
Cij = ∞, if there is no direct path from city i to city j.
2. Convert cost matrix to reduced matrix by subtracting minimum values from appropriate
rows and columns, such that each row and column contains at least one zero entry.
3. Find cost of reduced matrix. Cost is given by summation of subtracted amount from the
cost matrix to convert it in to reduce matrix.
4. Prepare state space tree for the reduce matrix

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 35

 TSP using BnB- Control Abstraction
5. Find least cost valued node A (i.e. E-node), by computing reduced cost
node matrix with every remaining node.
6. If <i, j> edge is to be included, then do following :
(a) Set all values in row i and all values in column j of A to ∞
(b) Set A[j, 1] = ∞
(c) Reduce A again, except rows and columns having all ∞ entries.
7. Compute the cost of newly created reduced matrix as,
Cost = L + Cost(i, j) + r
Where, L is cost of original reduced cost matrix and r is A[i, j].
8. If all nodes are not visited then go to step 4.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 36
Application: Airline Crew Scheduling problem

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 37

 Unit 5:- Complexity Theory
• Polynomial and non-polynomial problems,
• deterministic and non-deterministic algorithms,
• P class & NP class problems
• NP complete problems- vertex cover and 3-SAT Problem
• NP-Hard Problems: Clique problem.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 38