Backtracking:

What is Backtracking? Backtracking is nothing but the modified process of the Brute force approach. It
is a technique where multiple solutions to a problem are available, and it searches for the solution to the
problem among all the available options. Let's consider the example of brute force search as backtracking
follows the brute force search approach. Consider the box, and we have three objects of three different
colors. One object is of red color, the second object is of green color, and the third object is of blue color.
Now, we have to keep these objects inside the box, and we have multiple options. As per the Brute force
approach, we have to consider all the options and identify the best option among all the possible options.

In backtracking, solutions are represented in the form of a tree and that tree is known as a state space
tree. Since backtracking follows the DFS, the tree will be formed using DFS, which is known as a State
Space tree.
Branch and Bound
It is similar to backtracking. The concept branch and bound and backtracking follow the Brute force
method and generate the state space tree. But both of them follows different approaches. The way to
generate the tree is different.
Backtracking follows the DFS, whereas the branch n bound follows the BFS to generate the tree. Now we
will understand the branch n bound through an example. As it follows the BFS, so first all the nodes of the
same level are added then we move to the next level
Examples:
The problems that can be solved by using backtracking are:
o 8 Queens problem
o Knapsack problem using backtracking
The problem that can be solved by using branch and bound is:
o Travelling Salesman Problem
Differences between Branch n bound and Backtracking
N Queen Problem | Backtracking
This problem is to find an arrangement of N queens on a chess board, such that no queen can attack any
other queens on the board. The chess queens can attack in any direction as horizontal, vertical,
horizontal and diagonal way.
A binary matrix is used to display the positions of N Queens, where no queens can attack other queens.
• For N = 1, this is trivial case. For N = 2 and N = 3, solution is not possible. So we start with N = 4
and we will generalize it for N queens.

• We have to arrange four queens, Q1, Q2, Q3 and Q4 in 4 x 4 chess board. We will put ith queen
in ith row. Let us start with position (1, 1). Q1 is the only queen, so there is no issue. partial
solution is <1>
• We cannot place Q2 at positions (2, 1) or (2, 2). Position (2, 3) is acceptable. partial solution is
<1, 3>.
• Next, Q3 cannot be placed in position (3, 1) as Q1 attacks her. And it cannot be placed at (3, 2),
(3, 3) or (3, 4) as Q2 attacks her. There is no way to put Q3 in third row. Hence, the algorithm
backtracks and goes back to the previous solution and readjusts the position of queen Q2. Q2 is
moved from positions (2, 3) to
(2, 4). Partial solution is <1, 4>
• Now, Q3 can be placed at position (3, 2). Partial solution is <1, 4, 3>.
• Queen Q4 cannot be placed anywhere in row four. So again, backtrack to the previous solution
and readjust the position of Q3. Q3 cannot be placed on (3, 3) or(3, 4). So the algorithm
backtracks even further.
• All possible choices for Q2 are already explored, hence the algorithm goes back to partial
solution <1> and moves the queen Q1 from (1, 1) to (1, 2). And this process continues until a
solution is found.
• All possible solutions for 4-queen are shown in fig (a) & fig. (b)
We can see that backtracking is a simple brute force approach, but it applies some intelligence to cut out
unnecessary computation. The solution tree for the 4-queen problem is shown in Fig. (c).

Fig. (d) describes the backtracking sequence for the 4-queen problem.
The solution of the 4-queen problem can be seen as four tuples (x1, x2, x3, x4), where xi represents the
column number of queen Qi. Two possible solutions for the 4-queen problem are (2, 4, 1, 3) and (3, 1, 4,
2).

• Complexity analysis :

• In backtracking, at each level branching factor decreases by 1 and it creates a new problem of
size (n – 1) . With n choices, it creates n different problems of size (n – 1) at level 1.
• PLACE function determines the position of the queen in O(n) time. This function is called n
times.
• Thus, the recurrence of n-Queen problem is defined as, T(n) = n*T(n – 1) + n2. Solution to
recurrence would be O(n!).

Sum of subsets
Given a set of positive integers, find the combination of numbers that sum to given value M
Sum of subsets problem is analogous to the knapsack problem. The Knapsack Problem tries to
fill the knapsack using a given set of items to maximize the profit. Items are selected in such a
way that the total weight in the knapsack does not exceed the capacity of the knapsack. The
inequality condition in the knapsack problem is replaced by equality in the sum of subsets
problem. Given the set of n positive integers, W = {w1, w2, …, wn}, and given a positive integer
M, the sum of the subset problem can be formulated as follows (where wi and M correspond to
item weights and knapsack capacity in the knapsack problem):
\sum_{i=1}^{n} w_i x_i = M∑i=1nwixi=M
Where, x_i \in \{ 0, 1 \}xi∈{0,1}
Numbers are sorted in ascending order, such that w1 < w2 < w3 < …. < wn. The solution is often
represented using the solution vector X. If the ith item is included, set xi to 1 else set it to 0. In
each iteration, one item is tested. If the inclusion of an item does not violet the constraint of the
problem, add it. Otherwise, backtrack, remove the previously added item, and continue the same
procedure for all remaining items. The solution is easily described by the state space tree. Each
left edge denotes the inclusion of wi and the right edge denotes the exclusion of w i. Any path
from the root to the leaf forms a subset. A state-space tree for n = 3 is demonstrated in Fig. (a).

The algorithm for solving the sum of subsets problem using recursion is stated below:
Complexity Analysis
It is intuitive to derive the complexity of sum of the subset problem. In the state-
space tree, at level i, the tree has 2i nodes. So, given n items, the total number of
nodes in the tree would be 1 + 2 + 22 + 23 + .. 2n.

T(n) = 1 + 2 + 22 + 23 + .. 2n = 2n+1 – 1 = O(2n)

Hamiltonian Cycle using Backtracking

The Hamiltonian cycle is the cycle in the graph which visits all the vertices in graph exactly once and
terminates at the starting node. It may not include all the edges

• The Hamiltonian cycle problem is the problem of finding a Hamiltonian cycle in a graph if there
exists any such cycle.
• The input to the problem is an undirected, connected graph. For the graph shown in Figure (a), a
path A – B – E – D – C – A forms a Hamiltonian cycle. It visits all the vertices exactly once, but
does not visit the edges <B, D>.

• The Hamiltonian cycle problem is also both, decision problem and an optimization
problem. A decision problem is stated as, “Given a path, is it a Hamiltonian cycle of the
graph?”.
• The optimization problem is stated as, “Given graph G, find the Hamiltonian cycle for the
graph.”
• We can define the constraint for the Hamiltonian cycle problem as follows:
o In any path, vertex i and (i + 1) must be adjacent.
o 1st and (n – 1)th vertex must be adjacent (nth of cycle is the initial vertex itself).
o Vertex i must not appear in the first (i – 1) vertices of any path.
• With the adjacency matrix representation of the graph, the adjacency of two vertices can
be verified in constant time
Complexity Analysis
Looking at the state space graph, in worst case, total number of nodes in tree would be,

T(n) = 1 + (n – 1) + (n – 1)2 + (n – 1)3 + … + (n – 1)n – 1

T(n) = O(nn). Thus, the Hamiltonian cycle algorithm runs in exponential time.

Solution:
The backtracking approach uses a state-space tree to check if there exists a Hamiltonian cycle in
the graph. Figure (f) shows the simulation of the Hamiltonian cycle algorithm. For simplicity, we
have not explored all possible paths, the concept is self-explanatory.
Step 1: Tour is started from vertex 1. There is no path from 5 to 1. So it’s the dead-end state.

Step 2: Backtrack to the node from where the new path can be explored, that is 3 here
 Step 3: New path also leads to a dead end so backtrack and explore all possible paths

Step 4: Next path is also leading to a dead-end so keep backtracking until we get some node that
can generate a new path, i.e .vertex 2 here
Step 5: One path leads to Hamiltonian cycle, next leads to a dead end so backtrack and explore all
possible paths at each vertex

Step 6: Total two Hamiltonian cycles are detected in a given graph

Branch and bound - Knapsack problem
Branch and bound is one of the techniques used for problem solving. It is similar to the backtracking
since it also uses the state space tree. It is used for solving the optimization problems and minimization
problems. If we have given a maximization problem then we can convert it using the Branch and bound
technique by simply converting the problem into a maximization problem
we shall use the Least Cost- Branch and Bound method, since this shall help us eliminate exploring
certain branches of the tree. We shall also be using the fixed-size solution here. Another thing to be noted
here is that this problem is a maximization problem, whereas the Branch and Bound method is for
minimization problems. Hence, the values will be multiplied by -1 so that this problem gets converted
into a minimization problem.
Now, consider the 0/1 knapsack problem with n objects and total weight W. We define the upper
bound(U) to be the summation of vixi (where vi denotes the value of that objects, and xi is a binary value,
which indicates whether the object is to be included or not), such that the total weights of the included
objects is less than W. The initial value of U is calculated at the initial position, where objects are added in
order until the initial position is filled
Here, the procedure to solve the problem is as follows are:
• Calculate the cost function and the Upper bound for the two children of each node. Here, the (i +
1)th level indicates whether the ith object is to be included or not.
• If the cost function for a given node is greater than the upper bound, then the node need not be
explored further. Hence, we can kill this node. Otherwise, calculate the upper bound for this node.
If this value is less than U, then replace the value of U with this value. Then, kill all unexplored
nodes which have cost function greater than this value.
• The next node to be checked after reaching all nodes in a particular level will be the one with the
least cost function value among the unexplored nodes.
 • While including an object, one needs to check whether the adding the object crossed the
threshold. If it does, one has reached the terminal point in that branch, and all the succeeding
objects will not be included.
In this manner, we shall find a value of U at the end which eliminates all other possibilites. The path to
this node will determine the solution to this problem

Time and Space Complexity

Even though this method is more efficient than the other solutions to this problem, its worst case time
complexity is still given by O(2n), in cases where the entire tree has to be explored. However, in its best
case, only one path through the tree will have to explored, and hence its best case time complexity is
given by O(n)

Example:
Consider the problem with n =4, V = {10, 10, 12, 18}, w = {2, 4, 6, 9} and W = 15. Here, we calculate
the initital upper bound to be U = 10 + 10 + 12 = 32. Note that the 4th object cannot be included here,
since that would exceed W. For the cost, we add 3/9 th of the final value, and hence the cost function is 38.
Remember to negate the values after calculation before comparison.
After calculating the cost at each node, kill nodes that do not need exploring. Hence, the final state space
tree will be as follows

Branch and bound- Travelling salesman problem

 Given the vertices, the problem here is that we have to travel each vertex exactly once and reach back to
the starting point. We have to find the shortest path that goes through all the vertices once and returns
back to the starting vertex
• A salesman has to visit every city exactly once.
• He has to come back to the city from where he starts his journey.
• What is the shortest possible route that the salesman must follow to complete his tour?

If salesman starting city is A, then a TSP tour in the graph is-

A→B→D→C→A

Cost of the tour

= 10 + 25 + 30 + 15
= 80 units
Rules:

• To reduce a matrix, perform the row reduction and column reduction of the matrix separately.
• A row or a column is said to be reduced if it contains at least one entry ‘0’ in it.
 Step-02:

• We consider all other vertices one by one.

• We select the best vertex where we can land upon to minimize the tour cost
Choosing To Go To Vertex-B: Node-2 (Path A → B)

• From the reduced matrix of step-01, M[A,B] = 0

• Set row-A and column-B to ∞
• Set M[B,A] = ∞

Now,

• We reduce this matrix.

• Then, we find out the cost of node-02
•
Step 3:
Proceeding the same way

Thus,

• Optimal path is: A → C → D → B → A

• Cost of Optimal path = 25 units

What is the final time complexity ?

• Now this thing is tricky and need a deeper understanding of what we are doing. We are actually
creating all the possible extenstions of E-nodes in terms of tree nodes. Which is nothing but
a permutation. Suppose we have N cities, then we need to generate all the permutations of
the (N-1) cities, excluding the root city. Hence the time complexity for generating the
permutation is O((n-1)!), which is equal to O(2^(n-1)).

Hence the final time complexity of the algorithm can be O(n^2 * 2^n

Graphs
A Graph is a non-linear data structure consisting of nodes and edges. The nodes are sometimes also
referred to as vertices and the edges are lines or arcs that connect any two nodes in the graph.

Graphs are used to solve many real-life problems. Graphs are used to represent networks. The networks
may include paths in a city or telephone network or circuit network. Graphs are also used in social
networks like linkedIn, Facebook. For example, in Facebook, each person is represented with a vertex(or
node).

DFS & BFS

Depth First Traversal (or Search) for a graph is similar to Depth First Traversal of a tree. The only catch
here is, unlike trees, graphs may contain cycles (a node may be visited twice). To avoid processing a node
more than once, use a boolean visited array.

Approach:
Depth-first search is an algorithm for traversing or searching tree or graph data structures. The algorithm
starts at the root node (selecting some arbitrary node as the root node in the case of a graph) and explores
as far as possible along each branch before backtracking. So the basic idea is to start from the root or any
arbitrary node and mark the node and move to the adjacent unmarked node and continue this loop until
there is no unmarked adjacent node. Then backtrack and check for other unmarked nodes and traverse
them. Finally, print the nodes in the path.

Algorithm:
Create a recursive function that takes the index of the node and a visited array.
1. Mark the current node as visited and print the node.
2. Traverse all the adjacent and unmarked nodes and call the recursive function with the index of the
adjacent node.
Complexity Analysis:
• Time complexity: O(V + E), where V is the number of vertices and E is the number of edges in
the graph.
 • Space Complexity: O(V), since an extra visited array of size V is required

Breadth-First Traversal (or Search) for a graph is similar to Breadth-First Traversal of a tree (See method
2 of this post). The only catch here is, unlike trees, graphs may contain cycles, so we may come to the
same node again. To avoid processing a node more than once, we use a boolean visited array. For
simplicity, it is assumed that all vertices are reachable from the starting vertex

For example, in the following graph, we start traversal from vertex 2. When we come to vertex 0, we look
for all adjacent vertices of it. 2 is also an adjacent vertex of 0. If we don’t mark visited vertices, then 2
will be processed again and it will become a non-terminating process. A Breadth-First Traversal of the
following graph is 2, 0, 3, 1
Time Complexity: O(V+E), where V is the number of nodes and E is the number of edges.
Example:

Floyd Warshall Algorithm

The Floyd Warshall Algorithm is for solving the All Pairs Shortest Path problem. The problem is to find
shortest distances between every pair of vertices in a given edge weighted directed Graph.

Floyd Warshall Algorithm is as shown below-

Create a |V| x |V| matrix // It represents the distance between

every pair of vertices as given
For each cell (i,j) in M do-
if i = = j
M[ i ][ j ] = 0 // For all diagonal elements, value = 0
if (i , j) is an edge in E
M[ i ][ j ] = weight(i,j) // If there exists a direct edge between the
vertices, value = weight of edge
else
M[ i ][ j ] = infinity // If there is no direct edge between the
vertices, value = ∞
for k from 1 to |V|
for i from 1 to |V|
for j from 1 to |V|
if M[ i ][ j ] > M[ i ][ k ] + M[ k ][ j ]
M[ i ][ j ] = M[ i ][ k ] + M[ k ][ j ]

Time Complexity-

• Floyd Warshall Algorithm consists of three loops over all the nodes.
• The inner most loop consists of only constant complexity operations.
• Hence, the asymptotic complexity of Floyd Warshall algorithm is O(n3).
• Here, n is the number of nodes in the given graph.