CSC510 Analysis of Algorithms

Backtracking Algorithms
Jose Ortiz
[email protected]

Copyright Jose Ortiz

 Overview
• Sum of Subsets
• N-Queens
• Graph Coloring

2 Copyright Jose Ortiz

What are backtracking algorithms ?
• In a brute force approach, all the solutions (optimal and non-optimal)
are computed and therefore, taken into consideration. On the other
hand, in backtracking algorithms, all the solutions are computed, but
abandoned if the solution cannot be completed.

• Backtracking considers the concept of “partial candidate solution”.

That’s it, it builds candidates to the solutions.

• In general, backtracking algorithms use the depth- rst search

method. However, more optimized backtracking approaches such
as Branch and Bound use the best-case method.

3 Copyright Jose Ortiz

fi
Backtracking Sum of Subsets

4 Copyright Jose Ortiz

 Sum of Subsets Brute Force Approach
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
i=1
{}

x1 = 1 x1 = 0

3 3
x2 = 1 x2 = 0 x2 = 1 x2 = 0

1 1 1 1
x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 0
x3 = 1 x3 = 1 x3 = 0

8 8 8 8 8 8 8 8
x4 = 1 x4 = 1
x4 = 1 x4 = 1 x4 = 1 x4 = 1 x4 = 1 x4 = 1
x4 = 0 x4 = 0 x4 = 0 x4 = 0 x4 = 0 x4 = 0 x4 = 0 x4 = 0

4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4

Copyright Jose Ortiz

fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16 Bounds
x1 = 1 n n

∑ ∑
wi xi ≤ m wi pi ≥ 0
i=1 i=1
x2 = 1
3,13
Operations
4,12
x3 = 1 x3 = 0 1. xi = 1 (included)
xi = 0 (not included)
12,4 4,4
x4 = 1 x4 = 0 2. Add Xi to Wi
3. Substract Xi from Pi
8,0 4,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16 Bounds
x1 = 1 n n

∑ ∑
wi xi ≤ m wi pi ≥ 0
i=1 i=1
x2 = 1
3,13
Operations
4,12
x3 = 1 x3 = 0 1. xi = 1 (included)
xi = 0 (not included)
12,4 4,4
x4 = 1 x4 = 0 2. Add Xi to Wi
3. Substract Xi from Pi
8,0 4,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16 Bounds
x1 = 1 n n

∑ ∑
xi+1 ≤ m wi pi ≥ 0
i=0 i=1
x2 = 1
3,13 x2 = 0
Operations
4,12 3,12
x3 = 1 x3 = 0 1. xi = 1 (included)
x3 = 1 x3 = 0
xi = 0 (not included)
12,4 4,4 11,4 3,4
x4 = 1 2. Add Xi to Wi
x4 = 0 x4 = 1 x4 = 1 x4 = 0
3. Substract Xi from Pi
8,0 4,0 15,0 7,0 3,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16
x1 = 0
x1 = 1

3,13 x2 = 1
0,13
x2 = 1 x2 = 0

4,12 1,12
3,12 x3 = 1
x3 = 1 x3 = 0 x3 = 1 x3 = 0

12,4 4,4 3,4 9,4

11,4
x4 = 1
x4 = 0 x4 = 1
x4 = 1 x4 = 1 x4 = 0

8,0 4,0 7,0 3,0 13,0

15,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16
x1 = 0
x1 = 1

3,13 x2 = 1
0,13
x2 = 1 x2 = 0

4,12 1,12
3,12 x3 = 1 x3 = 0
x3 = 1 x3 = 0 x3 = 1 x3 = 0

12,4 4,4 3,4 9,4 1,4

11,4
x4 = 1 x4 = 1 x4 = 0
x4 = 0 x4 = 1
x4 = 1 x4 = 1 x4 = 0

8,0 4,0 7,0 3,0 13,0 5,0 1,0

15,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Backtracking :
0,16
x1 = 0
x1 = 1

3,13 x2 = 1
0,13 x2 = 0
x2 = 1 x2 = 0

4,12 1,12 0,12

3,12 x3 = 1 x3 = 0
x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1

12,4 4,4 3,4 9,4 4,4 8,4

11,4 x4 = 1
x4 = 1
x4 = 0 x4 = 1 x4 = 0
x4 = 1 x4 = 1 x4 = 0 x4 = 1

8,0 4,0 7,0 3,0 13,0 8,0 4,0 12,0

15,0
Copyright Jose Ortiz
fi
 Sum of Subsets Backtracking
n

∑
Given the following set of items. S = {3,1,8,4} nd all its subsets s where si = m; m = 12
{wi, pi} i=1
Solutions are: {3,1,8} and {8,4}
0,16
x1 = 0
x1 = 1

3,13 x2 = 1
0,13 x2 = 0
x2 = 1 x2 = 0

4,12 1,12 0,12

3,12 x3 = 1 x3 = 0
x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0

12,4 4,4 3,4 9,4 4,4 8,4 0,4

11,4 x4 = 1
x4 = 1
x4 = 1 x4 = 0 x4 = 1 x4 = 0
x4 = 0 x4 = 1 x4 = 1 x4 = 0 x4 = 1

8,0 4,0 7,0 3,0 13,0 8,0 4,0 12,0 4,0 0,0
15,0
Copyright Jose Ortiz
fi
Sum Of Subsets Pseudocode
Backtracking N-Queens

14 Copyright Jose Ortiz

 Backtracking N-Queens
• Problem statement:

• Given a N * N board, place N − Queens such that no queens attack

each other. Attacks occur when queens are placed in any of the
following ways:

• Same column

• Same row

• Diagonal

15 Copyright Jose Ortiz

 Backtracking N-Queens
For example, we are given a 4x4 board, and Q = {q1, q2, q3, q4} representing
4 − Queens:
Diagonal Same column Same row

1 2 3 4 1 2 3 4 1 2 3 4
1 q1 1 q1 1 q1 q2
2 q2 2 q2 2
3 q3 3 q3 3 q3
4 q4 4 q4 4 q4

1 2 3 4 1 2 3 4
1 q1 1 q1
2 q2 2 q2
3 q3 3 q3
4 q4 4 q4

16 Copyright Jose Ortiz

 Brute Force Approach N-Queens
A brute force approach would consider all the steps to nd all the
possible solutions
q1

Image Source: orion.com 17 Copyright Jose Ortiz

fi
 Backtracking N-Queens
Links Numbers Repesent {row, column}

1,1 {} 1,4 1 2 3 4
q1
q1 1 q1
q1 1,3 2 q2
2,1 q1 1,2 q1
3
q2 4

1,1 {} 1,4 1 2 3 4
q1 q1
q1 q1 1
1,3 2 q2
2,1 q1 1,2 q1
3
q2 2,2
q2 q2 4
q2
Copyright Jose Ortiz
 Backtracking N-Queens
Links Numbers Repesent {row, column}

1,1 {} 1,4
q1
q1 q1
2,1 1,2 1,3
q1 q1
q2 2,2 1 2 3 4
q2 q1
q2 2,3 1
q2 q2 2 q2
q2 q 3 q3
2
3,1 4

q3

19 Copyright Jose Ortiz

 Backtracking N-Queens
Links Numbers Repesent {row, column}

1,1 {} 1,4
q1
q1 q1
2,1 1,2 1,3
q1 q1
q2 2,2 1 2 3 4
q2 q1
q2 2,3 1
q2 q2 2 q2
q2 q 3 q3
2
3,1 4

q3 q2
3,2
q2 q2
q3 q
2

20 Copyright Jose Ortiz

 Backtracking N-Queens
Links Numbers Repesent {row, column}

1,1 {} 1,4
q1
q1 q1
2,1 1,2 1,3
q1 q1
q2 2,2 1 2 3 4
q2 q1
q2 2,3 1
q2 q2 2 q2
q2 q 3 q3
2q
2
3,1 3,2 3,3 q3 4

q3 q2 q q3
3
q2 q2

21 Copyright Jose Ortiz

 Backtracking N-Queens
Links Numbers Repesent {row, column}

1,1 {} 1,4
q1
q1 q1
2,1 1,2 1,3
q1 q1
q2 2,2 1 2 3 4
q2 q1
q2 2,3 1
q2 q2 2 q2
q2 q 3,4 3 q3
2q
2
3,1 3,2 q3 4
3,3
q3 q2 q q3
3
q2 q2

22 Copyright Jose Ortiz

 Backtracking N-Queens
After a few more iterations . . .

1,1 {} 1,4
q1 Solution Found!
q1 q1
1,2 1,3 1 2 3 4
q1 q1
2,1 1 q1
2,2 2,4 2 q2
q2 2,3 q3 3 q3
q2 q4
q2 q2q q2 4
2
3,1
q3
q2 4,1
q q3
2 q2
As we continue with the process, more solutions will be found
q4 4,2 4,3
qq24 q4

23
q q Copyright Jose Ortiz
 Backtracking N-Queens
Time Complexity Analysis
• Brute Force:

• Solving this problem by brute force implies that we need to examine all the possible permutations.
For instance, {0,........., (n − 1)} is O(n!)

• Backtracking:

• One solution: the best case occurs if you nd the solution before exploiting all the possible
arrangements (rarely happens). Also the time complexity of nding the rst solution depends on
your implementation

• All the solutions: the same as brute force O(n!). However, in average time complexity hits the
lower bounds of O(n!) since there are many permutations that are abandoned during the process.

• The backtracking algorithm can be further improved using memoization for all the recursive calls.

24 Copyright Jose Ortiz

fi
fi
fi
Backtracking Graph Coloring

25 Copyright Jose Ortiz

 M-Graph Coloring Backtracking
Given an undirected graph G = {v1, v2, . . . , vn}, and a set of colors
M = {c1, c2, . . . cm}, color all the vertices of the graph such that no
two adjacent vertices share the same color.
There are many possible
solutions for
1 2 1 2
G = {1,2,3,4,5}
M = {red, green, blue}
3 4 3 4
Note that two vertices are
adjacent if they are
5 5
connected by an edge, and
two edges are adjacent is
they share a vertex.
Copyright Jose Ortiz 26
 M-Graph Coloring Brute Force Approach
A brute force approach will compute all the possible ways to color a
graph without bounding. That’s it all the steps are considered.
{}
1,B
1,R
1,G
1 2 2,R 2,B

...... ...... ......

2,R 2,G 2,B 2,G
2,R 2,G 2,B

3 4 0
Level 1 = 3 vertex
n 4+1
1
Level 1 = 3 vertices 4 3 −1
∑
3 =
2
Level 1 = 3 vertices i=1
2
3
Level 1 = 3 vertices O(3 n+1
) = O(m n+1
)
4
Level 1 = 3 vertices
Copyright Jose Ortiz
 M-Graph Coloring Backtracking
{}
{}
1
1 1
1
1
1

2 2
2

No! Yes!
1 2 1 2

3 4 3 4

Copyright Jose Ortiz

 M-Graph Coloring Backtracking
{} {}
1 1
1 1
1 1

2 2 2 2

3 3 3

1 2 1 2

3 4 3 4

No! No!

Copyright Jose Ortiz

 M-Graph Coloring Backtracking
{} {}
1 1
1 1
1 1

2 2 2 2

1 2
3 3 3 3
3 3

1 2
4

3 4
We have one of the solutions!
If you keep going, Yes!
3 4
many solutions will be found
Yes!

Copyright Jose Ortiz

M-Graph Coloring Backtracking Pseudocode

0 1

3 2

G 0 1 2 3
0 1 1 0 1
1 1 1 1 1
2 0 1 1 1
3 1 1 1 1
0 = not connected
1 = connected

Copyright Jose Ortiz

 Backtracking M-Graph Coloring
Time Complexity Analysis
• Brute Force:
• Solving this problem by brute force implies that we need to color each node visited
n+1
m times. Therefore, time complexity using a brute force approach is O(m ) where
m is the total number of colors, and n the total number of nodes to be colored.

• Backtracking:

• The recursion takes n times since it needs to visit all the n nodes recursively, for
every visited node, the function needs to check is there is a valid color for that node
by visiting all its adjacent nodes. Therefore, the time complexity of solving this
n
problem with backtracking is O(nm ), which is slightly better than the one from the
brute force approach

32 Copyright Jose Ortiz