Backtracking

UNIT - IV
General Recursive backtracking Algorithm:-
Algorithm Backtrack( k )
// The first k-1 values have been assigned.
// x[ ] and n are global.
{
for ( each x[k] € T(x[1],……x[k-1] )
{
if( Bk( x[1],x[2],…..x[k] ≠ 0) then
{
if(x[1],x[2],…..x[k] is a path to answer node )
then write( x[1:k] );
if ( k < n) then Backtrack( k+1 );
}
}
}
 n-Queens Problem
A queen that is placed on an n x n chessboard,
may attack any piece placed in the same
column, row, or diagonal.

8x8 Chessboard
 n-Queens Problem
Can n queens be placed on an n x n
chessboard so that no queen may attack
another queen?

4x4
n-Queens Problem

8x8
 Difficult Problems

• Many require you to find either a subset or

permutation that satisfies some constraints
and (possibly also) optimizes some
objective function.
• May be solved by organizing the solution
space into a tree and systematically
searching this tree for the answer.
 Solution Space
• Solution Space is a set that includes at least one
solution to the problem.
• Subset problem.
 n = 2, {00, 01, 10, 11}
 n = 3, {000, 001, 010, 100, 011, 101, 110, 111}
• Solution space for subset problem has 2n
members.
• Nonsystematic search of the space for the answer
takes O(p2n) time, where p is the time needed to
evaluate each member of the solution space.
 Solution Space
• Permutation problem.
 n = 2, {12, 21}
 n = 3, {123, 132, 213, 231, 312, 321}
• Solution space for a permutation problem has n!
members.
• Nonsystematic search of the space for the answer
takes O(pn!) time, where p is the time needed to
evaluate a member of the solution space.
 Backtracking and Branch and Bound
• Backtracking and branch and bound perform a
systematic search; often taking much less time
than taken by a nonsystematic search.
 Tree Organization Of Solution Space
• Set up a tree structure such that the paths from root to
leaves represent members of the solution space.
• For a size n subset problem, this tree structure has 2n
leaves.
• For a size n permutation problem, this tree structure has
n! leaves.
• The tree structure is too big to store in memory; it also
takes too much time to create the tree structure.
• Portions of the tree structure are created by the
backtracking and branch and bound algorithms as
needed.
 Subset Tree For n = 4
( fixed – sized tuple )
x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

x3=1 x3 = 0

x4=1 x4=0

1110 1011 0111 0001

 Permutation Tree For n = 3

x1=1 x1 = 3
x1=2

x2 = 2 x2 = 3 x2 = 1 x2 = 3 x2 = 1 x2 = 2

x3=3 x3=2 x3=3 x3=1 x3=2 x3=1

123 132 213 231 312 321

 Backtracking

• Searches the solution space tree in a depth-

first manner.
• May be done recursively or use a stack to
retain the path from the root to the current
node in the tree.
• The solution space tree exists only in your
mind, not in the computer.
 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

n – queens problem:-

The problem is to place n queens on an n×n

chessboard so that no two “attack” that is no
two queens on the same row, column, or
diagonal.
Defining the problem:-
 Assume rows and columns of chessboard are
numbered 1 through n.

 Queens also be numbered 1 through n.

 Since each queen must be on a different row ,hence

assume queen i is to be placed on row i.

 Therefore all solutions to the n-queens problem can be

represented as n-tuples ( x1,x2,…..xn), where xi is the
column on which queen i is placed.
Tree structure for the case n=4.
 1
X1=1
X1=4
X1=2 X1=3

2 34 50
18

x 2= 2 3 1 3 1 2 4 1 2 3
4 4

3 8 13 19 24 29 35 40 45 51 56 61
x3=
3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2

4 6 9 11 14 16 20 22 25 27 30 32 36 38 41 43 46 48 52 54 57 59 62 64
x4=
4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1

5 7 10 12 15 17 21 23 26 28 31 33 37 39 42 44 47 9 53 55 58 60 63 65

Tree organization of the 4-queens solution space. Nodes are numbered as in depth
first search.
 1

x1=1 x1=2

2
18

x2=2 x2=3 x2=4 x2=4

3 8 13 29
19 24
B x3=1
B B
9 11 14 16 30
B B B
x4=3
15 31
B
Portion of the tree that is generated during backtracking( n=4 ).
 n - queens problem algorithm
• Every element on the same diagonal that
runs from the upper left to the lower right
has the same row – column value.

• Similarly, every element on the on the same

diagonal that goes from the upper right to
the lower left has the same row + column
value.
• If two queens are placed at positions ( i, j ) and ( k, l ) ,
then they are on the same diagonal only if

i – j = k – l or i + j = k + l
First equation implies

j–l=i–k
Second equation implies

j–l=k–i
• Therefore, two queens lie on the same diagonal if
and only if j – l = i – k
Algorithm place( k, l )
// It returns true if a queen can be placed in column i
// It tests both whether i is distinct from all previous values
// x [ 1 ], ….x [ k-1 ] and there is other queen on the same
//diagonal.
// Abs(r) returns the absolute value of r.
{
for j = 1 to k-1 do // for all previous queens
{ // Two in the same column or in the same diagonal
if ( ( x [ j ] = l ) or ( Abs( x[ j ] - l ) = Abs( j - k ) ) ) then
return false;
}
return true ;

}
Algorithm NQueen(k,n)
//Using backtracking, this procedure prints all
//possible placements of n queens on an n×n
//chessboard so that they are nonattacking.
{
for l =1 to n do // check place of column for queen k
{
if place( k, l ) then
{
x[ k ] = l;
if( k = n )then write( x[1:n] );
else NQueens( k+1, n);
}
}

}
 General method of Backtracking
• Let T ( x[1], x[2],…..x[ k-1 ] be the partial solution
Algorithm Backtrack( k )
{
for( each x[k] € T ( x[1], x[2],…..x[ k-1 ] ) do
{
if( Bk (x[1], x[2],…..x[ k-1 ], x[k] is true ) then
{
if ( x[1], x[2],…..x[ k-1 ], x[k] ) is an answer )
then write( x [ 1; k ] );
if ( k < n ) then Backtrack ( k+1 );
}
}
}
 Graph Coloring Problem

• Assign colors to the vertices of a graph so

that no adjacent vertices share the same
color.
– Vertices i, j are adjacent if there is an edge from
vertex i to vertex j.
 Example

No.of Colors used: 2

Number of Possible ways : 2

No.of Colors used: 3

...

Some possible ways

m-colorings problem:-
Find all ways to color a graph with at most m colors.
Problem formulation:-
 Represent the grapth with adjacency matrix G[1:n,1:n].

 The colors are represented by integer numbers 1,2,….m.

 Solution is represented by n- tuple (x1,….xn), where xi is the

color of node i.
 Solution space tree for mColorigng when n=3 and m=3

x1=1 2 3

3 1 3 1 3
x2 =1 2 2 2

X3 =1

2 3
A 4-node graph and all possible 3-colorings
1 2

4 3

x1= 1 2 3

X2= 2 3 1 3 1 2

X3= 1 3 1 2 2 3 1 2 2 3 1 3

X4= 2 3 2 2 3 3 1 3 1 3 1 3 1 1 2 2 1 2
Algorithm :- finding all m- colorings of a graph.
Function mColoring is begun by first assiging the graph to its adjacency
matrix, setting the array x[ ] to zero, and then invoking the statement
mColoring( 1 );

Algorithm mColoring( k )
// k is the index of the next vertex to color.
{
repeat
{ // Generate all legal assignments for x[k]

NextValue( k ); // Assign to x[k] a legal color

if ( x[k]=0 ) then return; // No new color possible

if ( k=n ) then // At most m colors have been used to color the n vertices
write( x[1:n] );
else mColoring( k+1);
} until ( false );
}
Algorithm NextValue( k )
// x[1],…..x[ k-1 ] have been assigned integer values in the range [ 1 ,m ].
// A value for x[ k ] is determined in the range [ 0,m ]

{
repeat
{
x[k] = ( x[k] +1) mod ( m+1 ); // Next highest color.
if ( x[k]=0 ) then return; // All colors have been used.
for j = 1 to n do
{
if (( G[ k,j ] ≠ 0 ) and ( x[k] = x[j] )) then
break;
}
if( j = n+1 ) then return; // Color found
} until ( false );
}
Hamiltonian cycles:-
A Hamiltonian cycle is a round-trip path along n edges of
connected undirected graph G that visits every vertex once and
returns to its starting position.

1 2 3 4

8 7 6 5

The above graph contains the Hamiltonian cycle

1,2,8,7,6,5,4,3,1
 1 2 3

5 4

The above graph does not contain Hamiltonian

cycles.
Find all possible Hamiltonian cycles
Problem formulation:-
 Represent the grapth with adjacency matrix G[1:n,1:n].

 Solution is represented by n- tuple (x1,….xn), where xi

represents the ith visited vertex of the cycle.

 Start by setting x[ 2: n ] to zero and x[ 1 ]=1, and then

executing Hamiltonian( 2 );
Algorithm Hamiltonian( k )
{
repeat
{ // Generate values for x[k]
NextValue( k ); // Assign a legal next value to x[ k ]

if ( x[k]=0 ) then return; // No new value possible

if ( k=n ) then write( x[1:n] );

else Hamiltonian( k+1);
} until ( false );
}
Algorithm NextValue( k )
{
repeat
{
x[k] = ( x[k] +1) mod ( m+1 ); // Next vertex.
if ( x[k]=0 ) then return;
if( G[ x[ k-1], x[ k ] ] ≠ 0 )
{
for j:= 1 to k-1 do if ( x[ j ] = x [k ] ) then break;

if( j = k ) then // if true, then the vertex is distinct

if( ( k < n ) or ( ( k=n ) and G [ x[n], x[1]] ≠ 0 )
then return;
}
} until ( false );
}
State Space :-
All paths from the root to other nodes
define the state space of the problem.

Solution Space :-
All paths from the root to solution states
define the solution space of the problem.
 Ex :- 1
X1=1
X1=4
X1=2 X1=3

2 34 50
18

x 2= 2 3 1 3 1 2 4 1 2 3
4 4

3 8 13 19 24 29 35 40 45 51 56 61
x3=
3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2

4 6 9 11 14 16 20 22 25 27 30 32 36 38 41 43 46 48 52 54 57 59 62 64
x4=
4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1

5 7 10 12 15 17 21 23 26 28 31 33 37 39 42 44 47 9 53 55 58 60 63 65

Tree organization of the 4-queens solution space. Nodes are numbered as in depth
first search.
Problem state :-
Each node in the tree is a problem state.
Ex:-
1 2 18 and so on
Solution States :-
These are those problem states S for
which the path from the root to S define a
tuple in the solution space.

Ex:-
5 7 10 12 and
15 17so
21 on
General Backtracking Algorithms :-
 Assume all answer nodes are to be found.

 Let (x1,x2,…..,xi) be a path from the root to a node xi in a state

space tree .

 Let T(x1,x2,…..xi) be the set of all possible paths for xi+1 such that
(x1,x2,…….., xi+1) is also a path to a problem state.

 Assume bounding functionBi+1 ( implicit conditions) such that if

Bi+1(x1,x2,……,xi+1) is false for a path (x1,x2,…xi+1) , then the
path cannot be expanded to reach an answer node.
0/1 Knapsack problem:

Given n positive weights wi,n positive profits pi,
and a positive number m that is the knapsack
capacity, this problem calls for choosing a subset of
the weights such that
∑wixi<=m and ∑pixi is maximized
1<=i<=n 1<=i<=n

The solution space for this problem consists of the 2
power n distinct ways to assign 0 or 1 values to the
xi

Bounding functions are needed to help kill some
live nodes without expanding them.
0/1 Knapsack problem:

A good bounding function for this problem is
obtained by using an upper bound on the value of
the best feasible solution obtainable by expanding
the given live node and any of its descendants

If this upper bound is not higher than the value of
the best solution determined so far , then that live
node can be killed

Using fixed tuple size formulation , if at node Z the
values of xi, 1<=i<=k, have already been
determined, then an upper bound for Z can be
obtained by relaxing the requirement xi=0 or 1 to
0<=xi<=1 for k+1<=i<=n and using the greedy
algorithm to solve the relaxed problem
0/1 Knapsack problem:

Function Bound(cp,cw,k) determines an upper
bound on the best feasible solution obtainable by
expanding any node Z at level k+1 of the statespace
tree.
Algorithm Bound(cp,cw,k)
{
b:=cp;c:=cw;
for i:=k+1 to n d
{ c:=c+w[i];
if(c<m) then b:=b+p[i];
else return b+(1-(c-m)/w[i])*p[i];
}
return b; }
0/1 Knapsack problem:
Algorithm BKnap(k,cp,cw)
{ //generate left child
if(cw+w[k]<=m) then
{
y[k]:=1;
if(k<n) then BKnap(k+1,cp+p[k],cw+w[k]);
if((cp+p[k]>fp) and (k=n)) then
{
fp:=cp+p[k]; fw:=cw+w[k];
for j:=1 to k do x[j]:=y[j];
}
}
0/1 Knapsack problem:
//generate right child
if (Bound(cp,cw,k) ≥ fp) then
{ y[k]:=0;
if(k<n) then
BKnap(k+1,cp,cw);
if((cp>fp) and (k=n)) then
{
fp:=cp; fw:=cw;
for j:=1 to k do
x[j]:=y[j];
}
} }
0/1 Knapsack problem:
Ex: w=(1,2,3,5,6,8) p=(3,6,7,9,11,18) m=15
 Backtracking
• Space required is O(tree height).
• With effective bounding functions, large instances
can often be solved.
• For some problems (e.g., 0/1 knapsack), the
answer (or a very good solution) may be found
quickly but a lot of additional time is needed to
complete the search of the tree.
• Run backtracking for as much time as is feasible
and use best solution found up to that time.
 Branch And Bound
• Search the tree using a breadth-first search (FIFO
branch and bound).
• Search the tree as in a bfs, but replace the FIFO
queue with a stack (LIFO branch and bound).
• Replace the FIFO queue with a priority queue
(least-cost (or max priority) branch and bound).
The priority of a node p in the queue is based on
an estimate of the likelihood that the answer node
is in the subtree whose root is p.
 Branch And Bound
• Space required is O(number of leaves).
• For some problems, solutions are at different
levels of the tree (e.g., 16 puzzle).

4 14 1 1 2 3 4
13 2 3 12 5 6 7 8
6 11 5 10 9 10 11 12
9 8 7 15 13 14 15
 Branch And Bound
 FIFO branch and bound finds solution closest to root.
 Backtracking may never find a solution because tree
depth is infinite (unless repeating configurations are
eliminated).
• Least-cost branch and bound directs the search to
parts of the space most likely to contain the
answer. So it could perform better than
backtracking.