Chapter-5

Backtracking Algorithms
Topics

 What is Backtracking
 N-Queens Problem
 Sum of Subsets
 Graph Coloring
 Hamiltonian Circuits
 Other Problems

2
 Algorithm Design

Human
Problems Result

Input Data Output Data

Structures Processing Structures

Computer
Algorithms
3
Algorithm Design …
For a problem? What is an Optimal Solution?

• Minimum CPU time

• Minimum memory
Example: Given 4 numbers, sort it to non-increasing order.
Method 1: Sequential comparison
1. Find the largest (3 comparisons)
2. Find the second largest (2 comparisons)
3. Find the third largest (1 comparisons)
4. Find the fourth largest
A total of 6 comparisons

4
Algorithm Design …
For a problem? What is an Optimal Solution?

Minimum CPU time

•
• Minimum memory
Example: Given 4 numbers, sort it to nonincreasing order.
Method 2: Somewhat clever method
a1 a2 a3 a4 a2 a3
(4 comparisons)

a2 a4 a3
(5 comparisons)
a2 a3 a1 a3
a4
5
a2 a3 or a1
 Backtracking Problems

 Findyour way through the well-known maze of hedges

by Hampton Court Palace in England?

 Until you reached a dead end.

 0-1 Knapsack problem – exponential time complexity.

 N-Queens problem.
6
Backtracking
 Suppose you have to make a series of decisions, among
various choices, where

– You don’t have enough information to know what to choose

– Each decision leads to a new set of choices
– Some sequence of choices (possibly more than one) may be a
solution to your problem

 Backtracking is a methodical way of trying out various

sequences of decisions, until you find one that “works”
7
 Introduction

 Backtracking is used to solve problems in which a

sequence of objects is chosen from a specified set so that
the sequence satisfies some criterion.
 Backtracking is a modified depth-first search of a tree.
 Backtracking involves only a tree search.
 BFS-Demo.ppt, depthfirst-Demo.pptx
 Backtracking is the procedure whereby, after determining
that a node can lead to nothing but dead nodes, we go
back (“backtrack”) to the node’s parent and proceed with
8
the search on the next child.
 Introduction …

 We call a node nonpromising if when visiting the node

we determine that it cannot possibly lead to a solution.
Otherwise, we call it promising.
 In summary, backtracking consists of
– Doing a depth-first search of a state space tree,
– Checking whether each node is promising, and, if it
is nonpromising, backtracking to the node’s parent.
 This is called pruning the state space tree, and the
subtree consisting of the visited nodes is called the 9
pruned state space tree.
 Solving a maze

 Given a maze, find a path from start to finish

 At each intersection, you have to decide between three or fewer
choices:
 Go straight

 Go left

 Go right

 You don’t have enough information to choose correctly

 Each choice leads to another set of choices
 One or more sequences of choices may (or may not) lead to a
solution
 10
Many types of maze problem can be solved with backtracking
Backtracking
 Backtracking is a technique
i on Portion A
used to solve problems withncta
large search space, by Ju

Portion B
systematically trying and
eliminating possibilities.

 A standard example of
backtracking would be going
through a maze.
◦ At some point in a maze,
you might have two options
of which direction to go:
 Backtracking
 One strategy would be to try
i on
going through Portion A of nc t
Ju
the maze.
 If you get stuck before Portion B
you find your way out,
then you "backtrack" to

Portion A
the junction.

 At
this point in time you
know that Portion A will
NOT lead you out of the
maze,
 so you then start searching
in Portion B
Backtracking
 Clearly,at a single junction
you could have even more
than 2 choices.

 The backtracking strategy

says to try each choice, one
after the other,
t i o n
◦ if you ever get stuck,
J u nc
"backtrack" to the junction C
and try the next choice. B
A
 Ifyou try all choices and
never found a way out, then
there IS no solution to the
maze.
 Coloring a map

 You wish to color a map with

not more than four colors
– red, yellow, green, blue
 Adjacent countries must be in
different colors
 You don’t have enough information to choose colors
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not) lead to
a solution
 Many coloring problems can be solved with backtracking
14
 Coloring a map

 You wish to color a map with

not more than four colors
◦ red, yellow, green, blue
 Adjacent countries must be in
different colors
 You don’t have enough information to choose colors
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not) lead to
a solution
 Many coloring problems can be solved with backtracking

15
Solving a puzzle

 In this puzzle, all holes but one are filled

with white pegs
 You can jump over one peg with another
 Jumped pegs are removed
 The object is to remove all but the last peg
 You don’t have enough information to jump correctly
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not) lead to a
solution
 Many kinds of puzzle can be solved with backtracking
16
 Backtracking (animation)

? dead end
dead end
dead end

?
start ? ? dead end
dead end
?
success!

17
N-Queens Problem

 Tryto place N queens on an N * N board such that

none of the queens can attack another queen.

 Remember that queens can move horizontally,

vertically, or diagonally any distance.

 Let’s consider the 8 queen example…

18
The 8-Queens Example

19
 0 1 2 3 4 5 6 7
0
1
2
3
4
5
6
7

20
A t (4,4) attack from (0,0) A t (5,4) attack from (2,1) A t (6,4) attack from (4,2) A t (7,4) success

0 1 2 3 4 5 6 7
0
1
2
3
4
5
6

7 21
Backtracking – Eight Queens Problem
 Findan arrangement of 8
queens on a single chess
board such that no two
queens are attacking one
another.

 Inchess, queens can move all

the way down any row,
column or diagonal (so long
as no pieces are in the way).

◦ Due to the first two

restrictions, it's clear that each
row and column of the board
will have exactly one queen.
Backtracking – Eight Queens Problem
 The backtracking strategy is as Q
follows:
 Place a queen on the first available Q
square in row 1. Q
 Move onto the next row, placing a Q
queen on the first available square
there (that doesn't conflict with the Q Q
previously placed queens).
 Continue in this fashion until Continue…
either:
 you have solved the problem, or
 you get stuck.
 When you get stuck, remove
the queens that got you there,
until you get to a row where
there is another valid square
to try.
Backtracking – Eight Queens Problem
When we carry out backtracking, an easy way to
visualize what is going on is a tree that shows all the
different possibilities that have been tried.

On the board we will show a visual representation of

solving the 4 Queens problem (placing 4 queens on
a 4x4 board where no two attack one another).
Backtracking – Eight Queens Problem
The neat thing about coding up backtracking,
is that it can be done recursively, without
having to do all the bookkeeping at once.
◦ Instead, the stack or recursive calls does most of
the bookkeeping
◦ (ie, keeping track of which queens we've placed,
and which combinations we've tried so far, etc.)
 1) Start in the leftmost column
 2) If all queens are placed
 return true
 3) Try all rows in the current column.
 Do following for every tried row.
 a) If the queen can be placed safely in this row
 then mark this [row, column] as part of the
 solution and recursively check if placing
 queen here leads to a solution.
 b) If placing the queen in [row, column] leads to
 a solution then return true.
 c) If placing queen doesn't lead to a solution then
 unmark this [row, column] (Backtrack) and go to
 step (a) to try other rows.
 4) If all rows have been tried and nothing worked,
 return false to trigger backtracking.
Backtracking – 8 queens problem -
Analysis
 Another possible brute-force algorithm is generate the
permutations of the numbers 1 through 8 (of which there
are 8! = 40,320),
◦ and uses the elements of each permutation as indices to place
a queen on each row.
◦ Then it rejects those boards with diagonal attacking positions.

 The backtracking algorithm, is a slight improvement on the

permutation method,
◦ constructs the search tree by considering one row of the board
at a time, eliminating most non-solution board positions at a
very early stage in their construction.
◦ Because it rejects row and diagonal attacks even on
incomplete boards, it examines only 15,720 possible queen
placements.

 A further improvement which examines only 5,508 possible

queen placements is to combine the permutation based
method with the early pruning method:
◦ The permutations are generated depth-first, and the search
space is pruned if the partial permutation produces a diagonal
attack
Sudoku and Backtracking
 Anothercommon puzzle that can be solved
by backtracking is a Sudoku puzzle.

 The basic idea behind the solution is as

follows:
1) Scan the board to look for an empty square that
could take on the fewest possible values based
on the simple game constraints.
2) If you find a square that can only be one
possible value, fill it in with that one value and
continue the algorithm.
3) If no such square exists, place one of the
possible numbers for that square in the number
and repeat the process.
4) If you ever get stuck, erase the last number
placed and see if there are other possible
choices for that slot and try those next.
 Mazes and Backtracking
A final example of something that can be
solved using backtracking is a maze.
◦ From your start point, you will iterate through
each possible starting move.
◦ From there, you recursively move forward.
◦ If you ever get stuck, the recursion takes you
back to where you were, and you try the next
possible move.

 In
dealing with a maze, to make sure you
don't try too many possibilities,
◦ one should mark which locations in the maze
have been visited already so that no location in
the maze gets visited twice.
◦ (If a place has already been visited, there is no
point in trying to reach the end of the maze from
Backtracking – homework
problem
Determine how many solutions
there are to the 5 queens
problem.
◦ Demonstrate backtracking for at
least 2 solutions to the 5 queens
problem, by tracing through the
decision tree as shown in class.
Let’s look at it run

31
 Terminology I

A tree is composed of nodes

There are three kinds of

nodes:
The (one) root node
Internal nodes
Backtracking can be thought of
Leaf nodes as searching a tree for a
particular “goal” leaf node 32
Terminology II

 Each non-leaf node in a tree is a parent of one or

more other nodes (its children)
 Each node in the tree, other than the root, has exactly
one parent parent
Usually, however,
we draw our trees
downward, with
parent
the root at the top
children children
33
Real and virtual trees

 There is a type of data structure called a tree

– But we are not using it here

 Ifwe diagram the sequence of choices we make, the

diagram looks like a tree
– In fact, we did just this a couple of slides ago
– Our backtracking algorithm “sweeps out a tree” in “problem
space”
34
The backtracking algorithm

 Backtracking is really quite simple--we “explore”

each node, as follows:
 To “explore” node N:
1. If N is a goal node, return “success”
2. If N is a leaf node, return “failure”
3. For each child C of N,
3.1. Explore C
3.1.1. If C was successful, return “success”
4. Return “failure”
35
 Sum-of-Subsets problem

 Recall the thief and the 0-1 Knapsack problem.

 The goal is to maximize the total value of the stolen items

while not making the total weight exceed W.

 If we sort the weights in nondecreasing order before doing

the search, there is an obvious sign telling us that a node is
nonpromising.

36
 Sum-of-Subsets problem …

 Let total be the total weight of the remaining weights, a

node at the ith level is nonpromising if weight + total > W

37
Example

 Say that our weight values are 5, 3, 2, 4, 1

 W is 8
 We could have
– 5+3
– 5+2+1
– 4+3+1
 We want to find a sequence of values that
satisfies the criteria of adding up to W

38
Tree Space
 Visualize a tree in which the children of the root indicate
whether or not value has been picked (left is picked, right is not
picked)

 Sort the values in non-decreasing order so the lightest value

left is next on list

 Weight is the sum of the weights that have been included at

level I

 Let weight be the sum of the weights that have been included
up to a node at level i. Then, a node at the ith level is
39
nonpromising if weight + wi+1 > W
 Sum-of-Subsets problem …

 Example: Show the pruned state space tree when backtracking

is used with n = 4, W = 13, and w1 = 3, w2 = 4, w3 = 5, and w4
= 6. Identify those nonpromising nodes.

40
Full example: Map coloring

 The Four Color Theorem states that any map on a plane can be
colored with no more than four colors, so that no two countries
with a common border are the same color

 For most maps, finding a legal coloring is easy

 For some maps, it can be fairly difficult to find a legal coloring

 We will develop a complete Java program to solve this

problem 41
Data structures

 We need a data structure that is easy to work with, and supports:

– Setting a color for each country
– For each country, finding all adjacent countries
 We can do this with two arrays
– An array of “colors”, where countryColor[i] is the color of
the ith country
– A ragged array of adjacent countries, where map[i][j] is the jth
country adjacent to country i
Example: map[5][3]==8 means the 3th country adjacent to
country 5 is country 8 42
Recap
 We went through all the countries recursively, starting with
country zero
 At each country we had to decide a color
– It had to be different from all adjacent countries
– If we could not find a legal color, we reported failure
– If we could find a color, we used it and recurred with the
next country
– If we ran out of countries (colored them all), we reported
success
 When we returned from the topmost call, we were done 43
Hamiltonian Circuits Problem

 Hamiltonian circuit (tour) of a graph is a path that

starts at a given vertex, visits each vertex in the graph
exactly once, and ends at the starting vertex.

44
State Space Tree

 Put the starting vertex at level 0 in the tree

 At level 1, create a child node for the root node for
each remaining vertex that is adjacent to the first
vertex.
 At each node in level 2, create a child node for each of
the adjacent vertices that are not in the path from the
root to this vertex, and so on.

45
Example
18
C F

3 15 4 10

12 5 19
A B D H

6 5 4
22
E G

46
Backtracking Algorithms

How can a computer play

the game?

xx Remember Deep Blue?

The tic-tac-toe game

47
Backtracking Algorithms (1,1)C

0 1 2

0
x (0,0)H (0,1)H (0,2)H (1,0)H...

1 (0,0)C, (0,1)C, (1,0)C...

2
x (0,1)H, (1,0)H, …, (2,2)H

: Computer
x : Human
The tic-tac-toe game
(0,1)C, (1,0)C, (1,2)C, (2,0)C...

48
 Backtracking Algorithms

3 missionaries and 2 cannibals want to cross the river

Condition:
1. A boat can take one or two (must include a missionary)
2. At any time, on either bank, the number of missionaries
must not be less than the number of cannibals.
49
Backtracking Search

Essentially a simplified depth-first

algorithm using recursion

50
Backtracking Search
(3 variables)

Assignment = {}
51
 Backtracking Search
(3 variables)

X1

v11

Assignment = {(X1,v11)}
52
 Backtracking Search
(3 variables)

X1

v11

X3

v31

Assignment = {(X1,v11), (X3,v31)}

53
 Backtracking Search
(3 variables)

X1

v11
Then, the search algorithm
X3 backtracks to the previous
variable and tries another
v31 value
X2
Assume that no value of X2
leads to a valid assignment

Assignment = {(X1,v11), (X3,v31)}

54
 Backtracking Search
(3 variables)

X1

v11

X3

v31 v32

X2

Assignment = {(X1,v11), (X3,v32)}

55
 Backtracking Search
(3 variables)

X1
The search algorithm
backtracks to the
v11 previous variable (X3)
and tries another value.
X3 But assume that X3 has
only two possible values.
v31 v32
The algorithm backtracks
to X1
X2 X2
Assume again that no value of
X2 leads to a valid assignment

Assignment = {(X1,v11), (X3,v32)}

56
 Backtracking Search
(3 variables)

X1

v11 v12

X3

v31 v32

X2 X2

Assignment = {(X1,v12)}
57
 Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2

Assignment = {(X1,v12), (X2,v21)}

58
 Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2 The algorithm need not consider

the variables in the same order in
this sub-tree as in the other

Assignment = {(X1,v12), (X2,v21)}

59
 Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2 X3

v32

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

60
 Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

The algorithm need
X2 X2 X3 not consider the values
of X3 in the same order
v32
in this sub-tree

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

61
 Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

Since there are only
X2 X2 X3
three variables, the
assignment is complete
v32

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

62
[This recursive algorithm keeps too much data in memory.
An iterative version could save memory (left as an exercise)]

Backtracking Algorithm

CSP-BACKTRACKING(A)
1. If assignment A is complete then return A
2. X  select a variable not in A
3. D  select an ordering on the domain of X
4. For each value v in D do
a. Add (Xv) to A
b. If A is valid then
i. result  CSP-BACKTRACKING(A)
ii. If result  failure then return result
5. Return failure

Call CSP-BACKTRACKING({})

63
Map Coloring
{}

WA=red WA=green WA=blue

WA=red WA=red
NT=green NT=blue
NT
Q
WA
WA=red WA=red
NT=green NT=green SA NSW
Q=red Q=blue V

T 64
 Chapter Summary

 Backtracking is an algorithm design technique for

solving problems in which the number of choices grows
at least exponentially with their instant size.

 This approach makes it possible to solve many large

instances of NP-hard problems in an acceptable
amount of time.

 The technique constructs a pruned state space tree.

 Backtracking constructs its state-space tree in the
depth-first search fashion in the majority of its
applications.
65