Module 5

Backtracking
The General Method

Sudeep Manohar
Department of Information Science and Engineering
JNNCE, Shivamogga
 Backtracking – The general method
• The name backtrack was first coined by D H Lehmer in 1950
• The principal idea is to construct solutions one component at a time and evaluate
such partially constructed candidates as follows

• If a partially constructed solution can be developed further without violating the problem’s
constraints, it is done by taking the first remaining option for the next component.

• If there is no legitimate option for the next component, no alternatives for any remaining
component need to be considered. In this case the algorithm backtracks to replace the last
component of the partially constructed solution with its next option

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• A tree can be constructed of choices being made called as the state-space tree
• Its root represent an initial state before the search for a solution begins
• The nodes of the first level in the tree represent the choices made for the first
component of a solution, the second level for the second component of a solution
and so on
• The node in a state-space tree is said to be promising if it corresponds to a partially
constructed solution that may still lead to a complete solution; otherwise it is called
nonpromising
• Leaves represents either nonpromising dead ends or complete solutions found by the
algorithm

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 State space tree

Initial state
First component of the solution
Promising node

Non promising node/Dead end

X X X X X X X Second component of solution
Soln. Soln.
1 2 Third component of solution

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

n-Queen’s problem
• The problem is to place n queens on an n-by-n
chessboard so that no two queens attack each other
by being in the same row or in the same column or
on the same diagonal

• For n=1, no. of queen is 1 and no other choice to

place the queen expect that one cell. For n=2 and
n=3 there are no. solutions

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

n=1 n=2 n=3
No solution No solution

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

4 queens problem

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
5 Queens problem

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Subset-Sum Problem
• Find the subset of a given set S = {s1, …… sn} of n positive integers whose sum is
equal to a given positive integer d

• For example, for S = {1, 2, 5, 6, 8} and d = 9, there are two solutions {1, 2, 6} and
{1, 8}

• First sort the elements of the set in increasing order

• The state-space tree is a binary tree, with left and right child representing inclusion
and exclusion of s1 in a set that is desired

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

S = {3, 5, 6, 7}
d = 15

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Todo
S = {2, 1, 5, 8, 6} and d = 9

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Graph Coloring
• If G is a graph and m is a positive integer, this problem finds whether the nodes of
G can be colored in such a way that no two adjacent nodes have the same color yet
only m colors are used
• This is termed the m-colorability decision problem
• If d is the degree of the given graph, then it can be colored with d + 1 color
• The m-colorability optimization problem finds the smallest integer m for which the
graph G can be colored
• This integer is referred to as the chromatic number of the graph

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
• A graph is said to be planar if it can be drawn in a plane in such a way that
no two edges cross each other
• An planar graph can be colored using only 4 colors

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

State space tree for n=3 and m=3

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

State space tree for m=3

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
• Consider m=3

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Hamiltonian Cycle
• Let G=(V, E) be a connected graph with n vertices
• A Hamiltonian cycle is a round trip path along n edges of G that visits every vertex
once and returns to its starting position

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• Consider a solution vector x where xi represents the ith visited vertex of the
cycle
• We have to determine possible vertex for xk if x1…… xk-1 have already been
chosen
• Consider x1=1, the starting vertex of the cycle
• xk can be any vertex v that is distinct from x1, x2, ……xk-1 and v is connected
by an edge to xk-1
• xn can only be the one remaining vertex and it must be connected to both
xn-1 and x1

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
x[ ] [1] [2] [3] [4] [5] [6] [7] [8]
1 2 8 7 6 5 4 3
x1 xk-1 xk
x1 xn-1 xn

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 a

b
c f

d e e
c
e d f
Dead end Dead end d
f
Dead end a Solution

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
Write the state space tree and find the Hamiltonian cycle

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Branch and Bound
• Branch and Bound technique is applied for optimization problems
• This technique requires that for every node in the state space tree, there should be a
bound on the best value of the objective function
• The node is non promising if the bound value is not better that the best solution
seen so far

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Assignment Problem
• Problem is to assign n people to n jobs so that the total cost of the assignment is as
small as possible (minimization problem)
• C is the cost matrix

• First find the lower bound on the cost of an optimal selection

• Lower bound = 2 + 3 + 1 + 4 = 10

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
3 5 4 2
9 7 6 8
C=
1 4 2 7
8 6 4 5

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Travelling Salesperson problem
• Find the minimum cost Hamiltonian cycle for a given graph
• This is a minimization problem
• Hence lower bound has to be considered
• For each city i, find the sum si of the distances from city i to two nearest cities
• Compute the sum s of all cities and divide result by 2
• And if all distances are integers, round up the result to the next nearest integer
• Bounding function: Lowerbound lb = s/2

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• Tours starts with a
• Generate tours in which b is visited before c (Constraint specified)
• After visiting n-1 cities, there is no other way than to visit the remaining city
and to return to the starting city

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Knapsack Problem
• Given n items of known weights wi and values vi, i = 1, 2, . . . n and a knapsack of
capacity W, find the most valuable subset of the items that fit in the knapsack
• First, order the items of the a given instance in descending order by their value-to-
weight ratios, so that
v1/w1 ≥ v2/w2 ≥ . . . . . . vn/wn
• The state space tree is a binary tree
• Each ith level of this tree corresponds to the ith item in the ordered list
• Left branch indicates the inclusion of the next item and right branch indicates its
exclusion

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• This is an optimization problem which maximizes the value in the knapsack
• Each node in the state space tree consists of value v, weight w and upper bound ub
• Upper bound is calculated as follows
ub = v + (W – w)(vi+1/wi+1)

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• Let the weights be (7, 3, 4, 5), values be (42, 12, 40, 25) and the knapsack
capacity W=10
item weight value value/weight
1 7 42 6
2 3 12 4
3 4 40 10
4 5 25 5

After sorting
item weight value value/weight
1 4 40 10
2 7 42 6
3 5 25 5
4 3 12 4
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
Find the maximum value of the subset of items for the given Knapsack
instance

Weights = (2, 1, 3, 2)
Values = (12, 10, 20, 15)
W=5

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Knapsack using - Least Cost Branch and Bound
• Find the subset of items which has the maximum profit that fits into the knapsack
• In LC branch and bound the problem is stated as
• Minimize − σ𝑛𝑖=1 𝑝𝑖𝑥𝑖
• Subject to σ𝑛𝑖=1 𝑤𝑖𝑥𝑖 ≤ 𝑚
• xi = 0 or 1, 1 ≤ i ≤ n

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Procedure
1. Set upper = ∞
2. Compute ĉ(x) and u(x) for each node
3. ĉ(x) = Profit by considering fraction of the last item that is considered in the knapsack =
− σ𝑛𝑖=1 𝑝𝑖𝑥𝑖 where 0 ≤ xi ≤ 1
4. u(x) = Profit without considering fraction = − σ𝑛𝑖=1 𝑝𝑖𝑥𝑖 where xi = 1 or 0
5. Choose the node with minimum u(x) among siblings and If u(x) < upper, then upper = u(x)
6. If ĉ(x) > upper, then kill node x
7. Otherwise node with minimum ĉ(x) among its siblings becomes the next E-node
8. Generate children for E-node
9. Repeat steps 2 to 8 until all nodes are killed leaving one as the solution
ĉ(.) is the intelligent rank function which speeds up branching by choosing the least cost
u(.) is the upper bound
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
n=4
m = 15
-22

Item 1 2 3 4
Weights 2 4 6 9
Profits 10 10 12 18

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
 Todo
n=5
m = 30

Weights 4 5 8 10 12
Profits 20 15 16 20 16

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Knapsack using - FIFO Branch and Bound
• Find the subset of items which has the maximum profit that fits into the knapsack
• In FIFO branch and bound the problem is stated as
• Minimize − σ𝑛𝑖=1 𝑝𝑖𝑥𝑖
• Subject to σ𝑛𝑖=1 𝑤𝑖𝑥𝑖 ≤ 𝑚
• xi = 0 or 1, 1 ≤ i ≤ n

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Procedure
1. Set upper = ∞
2. Compute ĉ(x) and u(x) for each node
3. ĉ(x) = Profit by considering fraction of the last item that is considered in the knapsack =
− σ𝑛𝑖=1 𝑝𝑖𝑥𝑖 where 0 ≤ xi ≤ 1
4. u(x) = Profit without considering fraction = − σ𝑛𝑖=1 𝑝𝑖𝑥𝑖 where xi = 1 or 0
5. Choose the node with minimum u(x) among siblings and if u(x) < upper, then upper = u(x)
6. If ĉ(x) > upper, then kill node x
7. Otherwise the next node in queue becomes the E-node
8. Generate children for E-node and if it remains alive insert into queue
9. Repeat steps 2 to 8 until all nodes are killed leaving one as the solution
ĉ(.) is the intelligent rank function which speeds up branching by choosing the least cost
u(.) is the upper bound
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
n=4
m = 15
-22

Item 1 2 3 4
Weights 2 4 6 9
Profits 10 10 12 18

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 Todo
n=4
m = 20

Weights 4 5 8 10
Profits 20 15 16 20

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

 NP Hard and NP Complete Problems - Basic Concepts
Polynomial time algorithms
• Algorithms for which solution times are bound by polynomials of small degree
• Ordered searching – O(log n)
• Polynomial evaluation – O(n)
• Sorting – O(n log n)
• String editing – O(mn)
Non Polynomial time algorithms (Exponential time algorithms)
• Algorithms for which polynomial time algorithms does not exist
• Travelling salesman – O(n22n)
• Knapsack – O(2n/2)
• Graph Coloring - O(2n)
• Subset sum - O(2n)
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
Non Deterministic algorithms

• If the result of every operation is uniquely defined then such algorithms are called
deterministic algorithms
• Algorithms which contain operations whose outcome is not uniquely defined are
called non deterministic algorithms
• Consider the following functions

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga
• Aim of a researcher should be to convert non deterministic non polynomial time
algorithms into deterministic polynomial time algorithms
• Since this is a topic which takes many years for research, the work done by a
researcher has to be saved in whatever state it is.
• This helps the other researcher to continue the research by continuing the work done
by the previous researchers
• For this the concepts of NP Hard and NP Complete provides certain guidelines
about how to classify the research work and how to continue with research

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

Two things to focus in research for non polynomial time algorithms
• Solve by developing non deterministic polynomial time algorithms
• If not solved directly, find similarities in the non polynomial time algorithms,
relate them and try to solve any one which is taken as reference so that the
rest can be solved – (Reference problems: Knapsack Decision problem, CNF
Probability, Max Clique etc)

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• P - is the set of all decision problems which has deterministic polynomial time
algorithms
• NP - is the set of all decision problems which has non deterministic polynomial
time algorithms
• P  NP

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga

• L1  L2 means problem L1 is polynomially equivalent to L2 and vice versa
• NP Hard – These are non deterministic non polynomial time algorithms
• NP Complete – These are non deterministic polynomial time algorithms

Sudeep Manohar, Dept. of ISE, JNNCE, Shivamogga