Backtracking

Difficult Problems

• Partition
– Partition n positive integers s1, s2, s3, …, sn into
two groups A and B such that the sum of the
numbers in each group is the same.
– [9, 4, 6, 3, 5, 1,8]
– A = [9, 4, 5] and B = [6, 3, 1, 8]
• Subset Sum Problem
– Does any subset of n positive integers s1, s2, s3,
…, sn have a sum exactly equal to c?
– [9, 4, 6, 3, 5, 1,8] and c = 18
– A = [9, 4, 5]
n-Queens Problem
A queen that is placed on an n x n chessboard, may
attack with any other queen placed in the same
column, row, or diagonal.

8x8 Chessboard
Can n queens be placed on an n x n chessboard
so that no queen attacks another queen?

4x4
8x8
 Difficult Problems
• Many difficult problems require you to find
either a subset or permutation that satisfies
some constraints and (possibly also)
optimizes some objective function.
• These may be solved by organizing the
solution space into a tree and systematically
searching this tree for the answer.
 Solution Space
• Solution Space is a set that includes at least one
solution to the problem.
• Subset problem.
 n = 2, {00, 01, 10, 11}
 n = 3, {000, 001, 010, 100, 011, 101, 110, 111}
• Solution space for subset problem has 2n
members.
• Non systematic search of the space for the answer
takes O(2n) time.
 Solution Space
• Permutation problem.
 n = 2, {12, 21}
 n = 3, {123, 132, 213, 231, 312, 321}
• Solution space for a permutation problem has n!
members.
• Non systematic search of the space for the answer
takes O(n!) time.
 Backtracking and Branch and Bound
• Backtracking and branch and bound perform a
systematic search and take much less time than the
time taken by a non systematic search.
 Tree Organization Of Solution Space
• Set up a tree structure such that the paths from root to
leaves represent members of the solution space.
• For a size n subset problem, this tree structure has 2n
leaves.
• For a size n permutation problem, this tree structure has
n! leaves.
• The tree structure is too big to store in memory; it also
takes much time to create the tree structure.
• Portions of the tree structure are created by the
backtracking and branch and bound algorithms as
needed.
 Subset Tree For n = 4
( fixed – sized tuple )
x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

x3=1 x3 = 0

x4=1 x4=0

1110 1011 0111 0001

 Permutation Tree For n = 3

x1=1 x1 = 3
x1=2

x2 = 2 x2 = 3 x2 = 1 x2 = 3 x2 = 1 x2 = 2

x3=3 x3=2 x3=3 x3=1 x3=2 x3=1

123 132 213 231 312 321

 Backtracking

• Searches the solution space tree in a depth-

first manner.
• Will be done recursively.
• The solution space tree exists only in your
mind, not in the computer.
 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

 Backtracking Depth-First Search

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

n – queens problem:-

The problem is to place n queens on an n×n

chessboard so that no two “attack” that is no
two queens on the same row, column, or
diagonal.
Defining the problem:-
 Assume rows and columns of chessboard are
numbered 1 through n.

 Queens also be numbered 1 through n.

 Since each queen must be on a different row ,hence

assume queen i is to be placed on row i.

 Therefore all solutions to the n-queens problem can be

represented as n-tuples ( x1,x2,…..xn), where xi is the
column on which queen i is placed.
Tree structure for the case n=4.
 1
X1=1
X1=4
X1=2 X1=3

2 34 50
18

x 2= 2 3 1 3 1 2 4 1 2 3
4 4

3 8 13 19 24 29 35 40 45 51 56 61
x3=
3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2

4 6 9 11 14 16 20 22 25 27 30 32 36 38 41 43 46 48 52 54 57 59 62 64
x4=
4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1

5 7 10 12 15 17 21 23 26 28 31 33 37 39 42 44 47 9 53 55 58 60 63 65

Tree organization of the 4-queens solution space. Nodes are numbered as in depth
first search.
 1

x1=1 x1=2

2
18

x2=2 x2=3 x2=4 x2=4

3 8 13 29
19 24
B x3=1
B B
9 11 14 16 30
B B B
x4=3
15 31
B
Portion of the tree that is generated during backtracking( n=4 ).
 n - queens problem algorithm
• Every element on the same diagonal that
runs from the upper left to the lower right
has the same row – column value.

• Similarly, every element on the on the same

diagonal that goes from the upper right to
the lower left has the same row + column
value.
• If two queens are placed at positions ( i, j ) and ( k, l ) ,
then they are on the same diagonal only if

i – j = k – l or i + j = k + l
First equation implies

j–l=i–k
Second equation implies

j–l=k–i
• Therefore, two queens lie on the same diagonal if
and only if j – l = i – k
Algorithm place( k, l )
// It returns true if a queen can be placed in column i
// It tests both whether i is distinct from all previous values
// x [ 1 ], ….x [ k-1 ] and there is other queen on the same
//diagonal.
// Abs(r) returns the absolute value of r.
{
for j = 1 to k-1 do // for all previous queens
{ // Two in the same column or in the same diagonal
if ( ( x [ j ] = l ) or ( Abs( x[ j ] - l ) = Abs( j - k ) ) ) then
return false;
}
return true ;

}
Algorithm NQueen(k,n)
//Using backtracking, this procedure prints all
//possible placements of n queens on an n×n
//chessboard so that they are nonattacking.
{
for l =1 to n do // check place of column for queen k
{
if place( k, l ) then
{
x[ k ] = l;
if( k = n )then write( x[1:n] );
else NQueens( k+1, n);
}
}

}
 Sum of subsets
• Given n distinct positive numbers wi, and m,
find all subsets that sum to m.
• We can formulate this problem using
either
– Fixed- or variable – sized tuples.
 Variable- sized tuple
• Ex:- n=4, ( w1, w2, w3, w4 )= ( 11,13, 24, 7 ), m=31.
– Solutions are ( 11, 13, 7 ) and ( 24, 7 )
• Rather than representing the solution by wi ’s, we can
represent by giving the indices of these wi
• Now the solutions are ( 1, 2, 4 ) and ( 3, 4 ).
• Different solutions may have different-sized tuples.
• We use the following condition to avoid generating
multiple instances of the same subset ( e.g., ( 1,2,4) and
(1,4,2) )
– xi < xi+1
 Subset Tree for n=4 (variable- sized tuple )
1

x1=1 x1=4
x1=2
x1=3
3 4 5
2

x2=4 x2=3 x2=4 x2=4

x1=2
x2=3
8 9 10 11
6 7
x3=3 x3=4 x3=4 x3=4
12 13 14 9
x4=3
16

Nodes are numbered as in Breadth first search.

Fixed- sized tuple
• In this method, each solution subset is represented
by an n-tuple (x1,x2,…..x n).
• x i=0 if wi is not chosen and xi=1 if wi is chosen

Subset Tree for n=4 (Fixed- sized tuple )

• We have already discussed.
• Copy that tree.
 Subset Tree For n = 4
( fixed – sized tuple )
x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

x3=1 x3 = 0

x4=1 x4=0

1110 1011 0111 0001

 Algorithm of sum of subsets
• Backtracking solution using fixed- sized tuple.
• A simple choice for bounding function is Bk(x1,x2,
…..,xk)= true iff
k n
∑ wixi + ∑ wi ≥ m
i=1 i=k+1

• Clearly x1,x2,……xk cannot lead to an answer

node if this condition is not satisfied.
• The bounding function strength further can be
increased if we assume the wi’s in increasing
order.
• In this case x1,x2,……xk cannot lead to an
answer node if
k
∑ wixi + ∑ wk+1 > m
i=1

• Therefore, the bounding functions we use are

Bk(x1,x2,…..,xk)= true iff

and
 nw )
The initial call is SumOfSub( 0, 1, ∑ i=1 i

Algorithm SumOfSub( s, k, r )
// Find all subsets of w[ 1:n ] that sum to m
n
// It is assumed that w[1] ≤ m and ∑ wi ≥ m
i=1
// The values of x[j] 1<=j<k, have already been determined.
K-1 n
// s= ∑ w[j]*x[j] and r= ∑ w[j] . W[j]’s in increasing order.
j=1 j=k
{
x[ k ]=1; // left child
if( s + w[k] = m ) then write( x[ 1: k ] ) ; // Subset found
else if ( s + w [ k ] + w[ k+1 ] ≤ m )
then SumOfSub( s+ w[k], k+1, r- w[k] )
// Generate right child and evaluate B k

if ( ( s + r – w[ k ] ≥ m ) and ( s + w[ k+1 ] ≤ m ) ) then

{
x[ k ] = 0;
} SumOfSub( s, k+1, r- w[k] ) ;
}
Ex:- n=6, m=30, w [ 1:6 ]= { 5,10,12,13,15,18 }
Portion of state space tree generated by SumOfSub.
circular nodes indicate subsets with sums equal to m.

• Example 7.6 (Figure 7.10)

– n=6, w[1:6]={5,10,12,13,15,18}, m=30
 General method of Backtracking
• Let T ( x[1], x[2],…..x[ k-1 ] be the partial solution
Algorithm Backtrack( k )
{
for( each x[k] € T ( x[1], x[2],…..x[ k-1 ] ) do
{
if( Bk (x[1], x[2],…..x[ k-1 ], x[k] is true ) then
{
if ( x[1], x[2],…..x[ k-1 ], x[k] ) is an
answer )
then write( x [ 1; k ] );
if ( k < n ) then Backtrack ( k+1 );
}
}
}
 Graph Coloring Problem

• Assign colors to the vertices of a graph so

that no adjacent vertices share the same
color.
– Vertices i, j are adjacent if there is an edge from
vertex i to vertex j.
 Example

No.of Colors used: 2

Number of Possible ways : 2

No.of Colors used: 3

...

Some possible ways

• M-colorability optimization problem asks
for the smallest integer for which the graph
can be colored.
• This number is called chromatic number.
• A planar graph is a graph that can be
embedded in the plane, i.e., it can be drawn
on the plane in such a way that its edges
intersect only at their endpoints.
• The number of colors required to color
every planar graph is 3.
m-colorings problem:-
Find all ways to color a graph with at most m colors.
Problem formulation:-
 Represent the grapth with adjacency matrix G[1:n,1:n].

 The colors are represented by integer numbers 1,2,….m.

 Solution is represented by n- tuple (x1,….xn), where xi is the

color of node i.
 Solution space tree for mColorigng when n=3 and m=3

x1=1 2 3

3 1 3 1 3
x2 =1 2 2 2

X3 =1

2 3
A 4-node graph and all possible 3-colorings
1 2

4 3

x1= 1 2 3

X2= 2 3 1 3 1 2

X3= 1 3 1 2 2 3 1 2 2 3 1 3

X4= 2 3 2 2 3 3 1 3 1 3 1 3 1 1 2 2 1 2
Algorithm :- finding all m- colorings of a graph.
Function mColoring is begun by first assiging the graph to its adjacency
matrix, setting the array x[ ] to zero, and then invoking the statement
mColoring( 1 );

Algorithm mColoring( k )
// k is the index of the next vertex to color.
{
repeat
{ // Generate all legal assignments for x[k]

NextValue( k ); // Assign to x[k] a legal color

if ( x[k]=0 ) then return; // No new color possible

if ( k=n ) then // At most m colors have been used to color the n

vertices
write( x[1:n] );
else mColoring( k+1);
} until ( false );
}
Algorithm NextValue( k )
// x[1],…..x[ k-1 ] have been assigned integer values in the range [ 1 ,m ].
// A value for x[ k ] is determined in the range [ 0,m ]

{
repeat
{
x[k] = ( x[k] +1) mod ( m+1 ); // Next highest color.
if ( x[k]=0 ) then return; // All colors have been used.
for j = 1 to n do
{
if (( G[ k,j ] ≠ 0 ) and ( x[k] = x[j] )) then
break;
}
if( j = n+1 ) then return; // Color found
} until ( false );
}
Hamiltonian cycles:-
A Hamiltonian cycle is a round-trip path along n edges of
connected undirected graph G that visits every vertex once and
returns to its starting position.

1 2 3 4

8 7 6 5

The above graph contains the Hamiltonian cycle

1,2,8,7,6,5,4,3,1
 1 2 3

5 4

The above graph does not contain Hamiltonian

cycles.
Find all possible Hamiltonian cycles
Problem formulation:-
 Represent the grapth with adjacency matrix G[1:n,1:n].

 Solution is represented by n- tuple (x1,….xn), where xi

represents the ith visited vertex of the cycle.

 Start by setting x[ 2: n ] to zero and x[ 1 ]=1, and then

executing Hamiltonian( 2 );
Algorithm Hamiltonian( k )
{
repeat
{ // Generate values for x[k]
NextValue( k ); // Assign a legal next value to x[ k ]

if ( x[k]=0 ) then return; // No new value possible

if ( k=n ) then write( x[1:n] );

else Hamiltonian( k+1);
} until ( false );
}
Algorithm NextValue( k )
{
repeat
{
x[k] = ( x[k] +1) mod ( n+1 ); // Next vertex.
if ( x[k]=0 ) then return;
if( G[ x[ k-1], x[ k ] ] ≠ 0 )
{
for j:= 1 to k-1 do if ( x[ j ] = x [k ] ) then break;

if( j = k ) then // if true, then the vertex is distinct

if( ( k < n ) or ( ( k=n ) and G [ x[n], x[1]] ≠ 0 )
then return;
}
} until ( false );
}
State Space :-
All paths from the root to other nodes
define the state space of the problem.

Solution Space :-
All paths from the root to solution states
define the solution space of the problem.
 Ex :- 1
X1=1
X1=4
X1=2 X1=3

2 34 50
18

x 2= 2 3 1 3 1 2 4 1 2 3
4 4

3 8 13 19 24 29 35 40 45 51 56 61
x3=
3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2

4 6 9 11 14 16 20 22 25 27 30 32 36 38 41 43 46 48 52 54 57 59 62 64
x4=
4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1

5 7 10 12 15 17 21 23 26 28 31 33 37 39 42 44 47 9 53 55 58 60 63 65

Tree organization of the 4-queens solution space. Nodes are numbered as in depth
first search.
Problem state :-
Each node in the tree is a problem
state.
Ex:- 1 2 18
and so on
Solution States :-
These are those problem states
S for which the path from the root to S
define a tuple in the solution space.

Ex:-
5 7 10 12 15 17 21 and so on
General Backtracking Algorithms :-
 Assume all answer nodes are to be found.

 Let (x1,x2,…..,xi) be a path from the root to a node xi in a state

space tree .

 Let T(x1,x2,…..xi) be the set of all possible paths for xi+1 such that
(x1,x2,…….., xi+1) is also a path to a problem state.

 Assume bounding functionBi+1 ( implicit conditions) such that if

Bi+1(x1,x2,……,xi+1) is false for a path (x1,x2,…xi+1) , then the
path cannot be expanded to reach an answer node.
General Recursive backtracking Algorithm:-
Algorithm Backtrack( k )
// The first k-1 values have been assigned.
// x[ ] and n are global.
{
for ( each x[k] € T(x[1],……x[k-1] )
{
if( Bk( x[1],x[2],…..x[k] ≠ 0) then
{
if(x[1],x[2],…..x[k] is a path to answer node )
then write( x[1:k] );
if ( k < n) then Backtrack( k+1 );
}
}
}
 O(2n) Subet Sum & Bounding Functions
{10, 5, 2, 1}, c = 14

x1=1 x1= 0

x2=1 x2= 0 x2=1 x2= 0

Each forward and backward move takes O(1) time.

 Backtracking
• Space required is O(tree height).
• With effective bounding functions, large instances
can often be solved.
• For some problems (e.g., 0/1 knapsack), the
answer (or a very good solution) may be found
quickly but a lot of additional time is needed to
complete the search of the tree.
• Run backtracking for as much time as is feasible
and use best solution found up to that time.
 Branch And Bound
• Search the tree using a breadth-first search (FIFO
branch and bound).
• Search the tree as in a bfs, but replace the FIFO
queue with a stack (LIFO branch and bound).
• Replace the FIFO queue with a priority queue
(least-cost (or max priority) branch and bound).
The priority of a node p in the queue is based on
an estimate of the likelihood that the answer node
is in the subtree whose root is p.
 Branch And Bound
• Space required is O(number of leaves).
• For some problems, solutions are at different
levels of the tree (e.g., 16 puzzle).

4 14 1 1 2 3 4
13 2 3 12 5 6 7 8
6 11 5 10 9 10 11 12
9 8 7 15 13 14 15
 Branch And Bound
 FIFO branch and bound finds solution closest to root.
 Backtracking may never find a solution because tree
depth is infinite (unless repeating configurations are
eliminated).
• Least-cost branch and bound directs the search to
parts of the space most likely to contain the
answer. So it could perform better than
backtracking.