CHAROTAR INSTITUTE OF TECHNOLOGY -

CHANGA

SMT K. D. PATEL DEPARTMENT OF INFORMATION TECHNOLOGY

BackTracking
 Department of Information Technology

BackTracking

 Suppose you have to make a series of decisions, among

various choices, where
 You don’t have enough information to know what to
choose
 Each decision leads to a new set of choices
 Some sequence of choices (possibly more than one) may
be a solution to your problem

 Backtracking is a technique used to solve problems with a

large search space, by systematically trying out
sequences of decisions, until we find one that gives the
solution.

2
 Department of Information Technology

Brute Force Approach & Backtracking Approach

 Brute Force Approach

 In this approach it forms all the possibilities to
determine the desired solution.

 Backtracking Approach
 It requires less possibilities to find desired solution.
 Form a solution and check at every step if this has
any chance of success or not. If the solution at any
point seems not-success (not promising) than ignore
it.

Charotar Institute of Technology, Changa 3

 Department of Information Technology

Backtracking

 Main idea: Backtracking consists of doing a DFS of the

state space tree, checking whether each node is
promising and if the node is non-promising backtracking
to the node’s parent node.

 We call a node non-promising if it cannot lead to a

feasible (or optimal) solution, otherwise node is
promising.

Charotar Institute of Technology, Changa 4

 Department of Information Technology

Backtracking (animation)

dead end

?
dead end
dead end

?
start ? ?
dead end
dead end

success!

5
 Department of Information Technology

Backtracking

 In the backtracking method ,

1. The desired solution is expressible as an n tuple ( X1,X2,….,Xn) where Xi
is chosen from finite set Si.
2. The solution maximizes or minimizes or satisfies a criterion function C (
X1,X2,….,Xn) .
 The problem can be categorized into three categories.
 For instance – For a problem P let C be the set of constrains for P.
Let D be the set containing all solutions satisfying C then
1) Decision Problem = Finding whether there is any feasible solution ?
2) Optimization Problem = What is the best solution ?
3) Enumeration Problem = Listing of all the feasible solutions.
 Department of Information Technology

Constraints

 All the solutions require a set of constraints divided

into two categories:
 explicit and implicit constraints
 Department of Information Technology

Explicit Constraint

 Definition : Explicit constraints are rules that restrict each xi

to take on values only from a given set.
 Explicit constraints depend on the particular instance I of
problem being solved
 All tuples that satisfy the explicit constraints define a
possible solution space for I
 Examples of explicit constraints
 Department of Information Technology

Implicit Constraint

 Definition: Implicit constraints are rules that

determine which of the tuples in the solution space of
I satisfy the criterion function.
 Implicit constraints describe the way in which the xi s
must relate to each other.
 Determine problem solution by systematically
searching the solution space for the given problem
instance.
 Use a tree organization for solution space
 Department of Information Technology

N Queen’s Problem
 The N queen’s problem can be stated as follows:
 Consider a N X N chessboard on which we have to place N queens so
that no two queens attack each other by being in the same row or in
the same column or on the same diagonal.
 For example , consider 2-Queen’s problem which is not solvable.
Q Q Q Q Q
Q Q Q Q Q

Illegal Illegal Illegal Illegal Illegal

 What about 3-Queen’s problem ?

 Department of Information Technology

N Queen’s Problem
 But 4 Queen’s Problem is solvable.
 Note that no two queens can attack each other.
 Let’s take 4-queens and 4 X 4 chessboard.
 Now we start with empty chessboard.
 Department of Information Technology

4 Queen’s Problem
 Try all the 16 places for the each queen
 Total possibilities =16C4 = 1820
 First improvement : One queen in One
Row
 Total possibilities = 4 x 4 x 4 x 4 = 44 = 256
 Second improvement : No two queens in
same column
 Total possibilities = 4 x 3 x 2 x 1 = 4! = 24

Charotar Institute of Technology, Changa 12

 Department of Information Technology

4 Queen’s Problem
 Step 1: Place queen 1 in the first possible position of its row. i.e. on (1,1) .
Q
 Department of Information Technology

N Queen’s Problem
 Step 1: Place queen 1 in the first possible position of its row. i.e. on (1,1) .
Q
 Department of Information Technology

4 Queen’s Problem
Q
Q

 Step 2: Then place queen 2 on the position (2,3) .

 Department of Information Technology

4 Queen’s Problem
Q
Q

 Step 3: Thus this is the Dead End because a 3rd queen cannot be placed in next
row, as there is no acceptable position for queen 3 . Hence algorithm
backtracks to previous queen (i.e. queen 2)
 queen 2 on the position (2,4) Q
Q
 Department of Information Technology

4 Queen’s Problem
 queen 2 on the position (2,4)
Q Q
Q Q
Q

 Step 4: Then place 3rd queen at (3,2) but it is again another dead end as
next queen 4th cannot be placed at permissible position.
 Department of Information Technology

4 Queen’s Problem
Q
Q

 Step 5: Hence we need to backtrack all the way Q

up to queen 1 and move it to (1,2).

Q Q Q Q
Q Q Q
Q Q
Q
 Place queen 1 at (1,2),queen 2 at (2,4), queen 3
at (3,1) and queen 4 at (4,3).
 Thus solution is obtained (2,4,1,3) in row
wise manner.
(Q1, Q2, Q3, Q4) = (2, 4, 1, 3)
 Department of Information Technology

4 Queen’s Problem
 Two solutions for 4 Queen Problem

Q Q
Q Q
Q Q
Q Q

 (Q1,Q2,Q3,Q4) = (3,1,4,2)
 (Q1,Q2,Q3,Q4) = (2,4,1,3)
 Department of Information Technology

4 Queen Problem : state space tree

Charotar Institute of Technology, Changa 20

 Department of Information Technology

State space Tree for 4 Queen’s Problem

 Department of Information Technology

The General Method

 4-Queen Problem
 Department of Information Technology

Charotar Institute of Technology, Changa 23

 Department of Information Technology

4 Queen Problem

 Explicit Constraint :
 Si= {1,2,3,4}, 1<= i <=4
 Here the solution space is 44

 Implicit Constraint :
 No two Si can be same OR All queens are in different
column
 Reduces the solution space 44 to 4!
 No two queens can be on the same diagonal

Charotar Institute of Technology, Changa 24

 Department of Information Technology

Questions….

 Generate first 2-solution for 5-Queen’s

problem using pruned state space tree.

(1,3,5,2,4)
(1,4,2,5,3)

Charotar Institute of Technology, Changa 25

5 Queen Problem : Department of Information Technology

Total 10 solutions and only 2 unique solutions

(4,2,5,3,1)
(2,4,1,3,5)
(1,3,5,2,4)
(1,4,2,5,3)
(5,3,1,4,2)
(3,1,4,2,5)
(3,5,2,4,1)
(5,2,4,1,3)
(2,5,3,1,4)
(4,1,3,5,2)

Charotar Institute of Technology, Changa 26

 Department of Information Technology

Charotar Institute of Technology, Changa 27

 Department of Information Technology

Backtracking : 8 Queen Problem

What is 8-queens problem?

 Place eight queens on an 8 × 8 chessboard so that no
queen attacks another queen.

 No two queens should share same row, column or

diagonal.
 https://en.wikipedia.org/wiki/Eight_queens_puzzle
Charotar Institute of Technology, Changa 28
 Department of Information Technology

Charotar Institute of Technology, Changa 29

 Department of Information Technology

Eight Queen’s Problem

Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
 Department of Information Technology

Backtracking Approach

 A given problem has a set of constraints and

possibly an objective function
 The solution optimizes an objective function, and/or
is feasible.
 We can represent the solution space for the problem
using a state space tree
 The root of the tree represents 0 choices,
 Nodes at depth 1 represent first choice
 Nodes at depth 2 represent the second
choice, etc.
 In this tree a path from a root to a leaf
represents a candidate solution.
 Department of Information Technology

 Bounding function
 No two queens placed on the same column  xi’s are
distinct
 No two queens placed on the same diagonal  how to
test?
 The same value of “row-column” or “row+column”
 Supposing two queens on (i,j) and (k,l)
 i-j=k-l (i.e., j-l=i-k) or i+j=k+l (i.e., j-l=k-i)
 So, two queens placed on the same diagonal iff
|j-l| = |i-k|
 Department of Information Technology

N- Queen Algorithm….

Invoked : NQueens(1,n)
void NQueens(int k, int n)
// Using backtracking, this procedure prints all
// possible placements of n queens on an n x n
// chessboard so that they are non attacking.
{
for (int i=1; i<=n; i++) {
if (Place(k, i)) {
x[k] = i;
if (k==n) { for (int j=1;j<=n;j++)
cout << x[j] << ' '; cout << endl;}
else NQueens(k+1, n);
}
}
}

Charotar Institute of Technology, Changa 33

 Department of Information Technology

N- Queen Algorithm….

bool Place(int k, int i)

// Returns true if a queen can be placed in kth row and
// ith column. Otherwise it returns false. x[] is a
// global array whose first (k-1) values have been set.
// abs(r) returns the absolute value of r.
{ Two in the same column
for (int j=1; j < k; j++)
if ((x[j] = = i) || (abs(x[j]-i) = = abs(j-k)))
return(false);
return(true);
} or in the same diagonal

Charotar Institute of Technology, Changa 34

 Department of Information Technology

Backtracking : 0/1 Knapsack Problem

 Given a set S of n objects, where the object i has value

vi and weight wi , and a knapsack with weight capacity
C, find the maximum of value of objects in S which can
be put in the knapsack.

 General problem statement

 Given vi and wi , for 1  i  n ,
find the set K such that for each i in K, 1  i  n,
 vi is maximum while  wi  C.
iK iK

 Problem space
 Any subset of S.
 Department of Information Technology

State-Space Tree : 0/1 Knapsack

 Given a set of objects o1, …, o5.
{}

{1} {2} {3} {4} {5}

{1,2} {1,3} {1,4} {1,5}

{1,2,3} {1,2,4} {1,2,5} {1,3,4} {1,3,5} {1,4,5}

{1,2,3,4} {1,2,3,5} {1,2,4,5} {1,3,4, 5}

{1,2,3,4,5}
 Department of Information Technology

0/1 Knapsack: Backtracking

{}
0,0 Capacity = 5
Sack
Node

Current weight, current value

{1} {2} {3} {4}
2,5 1,4 3,8 2,7

{1,2} {1,3} {1,4} {2,3} {2,4} {3,4}

3,9 5,13 4,12 4,12 3,11 5,15
object weight value

{1,2,3} {1,2,4} {2,3,4} 1 2 5

6, 17 5, 16 6,19 2 1 4

3 3 8
4 2 7
 Department of Information Technology

Questions….

 Draw a pruned state space tree for given

knapsack problem. Capacity = 8 unit

item weight value

1 2 3
2 3 5

3 4 6
4 5 10

Charotar Institute of Technology, Changa 38

 Department of Information Technology

General Backtracking Algorithm

Algorithm Backtrack ()
// This is recursive backtracking algorithm.
// a[k] is a solution vector. On entering (k-1) remaining next values can be
computed.
// T ( a1,a2,….,an) be the set of all values for an , such that ( a1,a2,….,an) is a
path to problem state.
// Bk is a bounding function.
{
For (each an that belongs to T (a1,a2,….,an) do )
{ if (Bk (a1,a2,….,an) = true) then // feasible sequence
{ if ((a1,a2,….,an) is a path to answer node ) then
print (a1,a2,….,an) ;
if ( k < n ) then
Backtrack ( k+1 ) ; // find the next set
} }}
Charotar Institute of Technology, Changa 39
Branch and Bound Department of Information Technology

Example : Assignment Problem

 Input: n jobs, n employees, and an n x n matrix A where Aij be

the cost if person i performs job j.

 Problem: find a one-to-one matching of the n employees to the n

jobs so that the total cost is minimized.
1 2 3
A 4 7 3
B 2 6 1
C 3 9 4

 A brute-force method would generate the whole solution tree,

where every path from the root to any leaf is a solution, then
evaluate the Cost of each solution, and finally choose the path
with the minimum cost.
 Department of Information Technology

Example : Assignment Problem

J1 J2 J3 J4
A 11 12 18 40
B 14 15 13 22
C 11 17 19 23
D 17 14 20 28

Person D takes 17 unit of time to

finish Job J1.

Charotar Institute of Technology, Changa 41

 Department of Information Technology

Example : Assignment Problem

J1 J2 J3 J4
A 11 12 18 40 Range : [58…73]
B 14 15 13 22
C 11 17 19 23
D 17 14 20 28

 Upper Bound :
 Top Left to Bottom Right Diagonal
 11+15+19+28 = 73
 Top Right to Bottom Left Diagonal
 40+13+17+17 = 87
 Lower Bound :
 Row wise Minimum
 11+13+11+14 = 49
 Column wise Minimum
 11+12+13+22 = 58

Charotar Institute of Technology, Changa 42

 Department of Information Technology

State Space Tree Range : [58…73]

A-1 A-2 A-3 A-4

11+14+13+22 58 65 78*
=60

J1 J2 J3 J4
A 11 12 18 40
B 14 15 13 22
C 11 17 19 23
D 17 14 20 28

Charotar Institute of Technology, Changa 43

 Department of Information Technology

State Space Tree Range : [58…73]

A-1 A-2 A-3 A-4

11+14+13+22 58 65 78*
=60

A-2 A-2 A-2

B-1 B-3 B-4
68 59 64 J1 J2 J3 J4
A 11 12 18 40
B 14 15 13 22
C 11 17 19 23
D 17 14 20 28

Charotar Institute of Technology, Changa 44

 Department of Information Technology

State Space Tree Range : [58…64]

A-1 A-2 A-3 A-4

11+14+13+22 58 65 78*
=60

A-2 A-2 A-2

B-1 B-3 B-4
68 59 64 J1 J2 J3 J4
A 11 12 18 40
A-2 A-2
B-3 B-3 B 14 15 13 22
Now we have New C-1 C-4 C 11 17 19 23
Upper Bound : 64 D-4 D-1 D 17 14 20 28
64 65
Charotar Institute of Technology, Changa 45
 Department of Information Technology

State Space Tree Range : [58…64]

A-1 A-2 A-3 A-4

11+14+13+22 58 65* 78*

=60

Now we need to A-2 A-2 A-2

explore it. B-1 B-3 B-4
68* 59 64* J1 J2 J3 J4
A 11 12 18 40
A-2 A-2
B-3 B-3 B 14 15 13 22
C-1 C-4 C 11 17 19 23
D-4 D-1 D 17 14 20 28
64 65*
Charotar Institute of Technology, Changa 46
 Department of Information Technology

State Space Tree Range : [58…64]

A-1 A-2 A-3 A-4

60 58 65* 78*

A-1 A-1 A-1 A-2 A-2 A-2

B-2 B-3 B-4 B-1 B-3 B-4
68* 61 66* 68* 59 64*

A-1 A-1 A-2 A-2 J1 J2 J3 J4

B-3 B-3 B-3 B-3 A 11 12 18 40
C-2 C-4 C-1 C-4 B 14 15 13 22
D-4 D-2 D-4 D-1 C 11 17 19 23
69* 61 64 65* D 17 14 20 28

Charotar Institute of Technology, Changa 47

 Department of Information Technology

Example : Assignment Problem

 Assigning 4 people to 4 jobs so that the total cost is

minimized. Each person does one job and each job is
assigned to one person. Use branch and bound
method to solve the assignment problem
(Minimization) with the following cost matrices.

J1 J2 J3 J4
P1 9 2 7 8
P2 6 4 3 7
P3 5 8 1 8
P4 7 6 9 4

Charotar Institute of Technology, Changa 48

 J1 J2 J3 J4
Department of Information Technology
P1 9 2 7 8
Example : Assignment Problem
P2 6 4 3 7
P3 5 8 1 8
P4 7 6 9 4

Charotar Institute of Technology, Changa 49

 Department of Information Technology

Branch and Bound

 Branch and bound is a systematic method for solving

optimization problems.
 B&B is a rather general optimization technique that applies
where the greedy method and dynamic programming fail.
 However, it is much slower. Indeed, it often leads to
exponential time complexities in the worst case.
 On the other hand, if applied carefully, it can lead to
algorithms that run reasonably fast on average.
 Department of Information Technology

Branch and Bound

 The general idea of B&B is a BFS-like search for the

optimal solution, but not all nodes get expanded (i.e., their
children generated).
 Rather, a carefully selected criterion determines which node
to expand and when, and another criterion tells the
algorithm when an optimal solution has been found.
 Department of Information Technology

Branch and Bound

The iteration has three main components:

1) Selection of the node to process,
2) Bound calculation,
3) Branching.

Charotar Institute of Technology, Changa 52

 Department of Information Technology

Differences

 First, the branch-and-bound technique is restricted to

optimization problems.
 The second difference relates to how the state space tree
can be traversed. A backtracking problem must do a pre-
order depth first search of the state space tree. A
branch-and-bound attack is not restricted to a depth first
search of the state space tree, but it may use a breath
first search if it needs to.
 The third difference is that a branch-and-bound must
completely search the state space tree.
 All branch-and-bound algorithms search for an optimum
value. If the entire state-space tree is not searched,
then the possibility exists that the optimum value is
located in the un-searched portion of the state space
tree. This would imply that the found solution is not the
optimum.