Course Overview

Mr.C.Karthikeyan
Assistant Professor, Department of CSE
Karpagam Institute of Technology
CS3401 Algorithms | Unit IV 1
 Course Objectives
• Understand and apply the algorithm analysis techniques on
searching and sorting algorithms
• Critically analyze the efficiency of graph algorithms
• Understand different algorithm design techniques
• Solve programming problems using state space tree
• Understand the concepts behind NP Completeness,
Approximation algorithms and randomized algorithms.
CS3401 Algorithms | Unit IV 2
 Course Outcomes
• Develop algorithms for various computing problems and time and space
complexity analysis.
• Apply graph algorithms to solve problems and analyze their efficiency.
• Analyze the different design techniques like divide and conquer, dynamic
programming and Greedy techniques to solve problems.
• Make use of the state space tree method for solving computational
problems.
• Solve problems using approximation algorithms and randomized
algorithms.
CS3401 Algorithms | Unit IV 3
Course Syllabus

COURSE SYLLABUS

CS3401 Algorithms | Unit IV 4

 Text Books
1. Thomas H. Cormen, Charles E. Leiserson,
Ronald L. Rivest and Clifford Stein,
"Introduction to Algorithms", 3rd Edition,
Prentice Hall of India, 2009.
2. Ellis Horowitz, Sartaj Sahni, Sanguthevar
Rajasekaran “Computer Algorithms/C++”
Orient Blackswan, 2nd Edition, 2019.

CS3401 Algorithms | Unit IV 5

Reference Books
1. Anany Levitin, ― Introduction to the Design
and Analysis of Algorithms, Third Edition,
Pearson Education, 2012.
2. Alfred V. Aho, John E. Hopcroft and Jeffrey
D. Ullman, "Data Structures and
Algorithms", Reprint Edition, Pearson
Education, 2006.
3. S. Sridhar, ― Design and Analysis of
Algorithms, Oxford university press, 2014.

CS3401 Algorithms | Unit IV 6

Unit IV : STATE SPACE SEARCH ALGORITHMS

Mr.C.Karthikeyan
Assistant Professor, Department of CSE
Karpagam Institute of Technology
CS3401 Algorithms | Unit IV 7
Backtracking: N-Queens problem - Hamiltonian
Circuit Problem - Subset Sum Problem – Graph
colouring problem Branch and Bound: Solving 15-
Puzzle problem - Assignment problem - Knapsack
Problem - Travelling Salesman Problem

CS3401 Algorithms | Unit IV 8

CS3401 Algorithms | Unit IV 9
 Backtracking: Introduction
• Backtracking is an algorithmic technique whose goal is to use brute force to find all
solutions to a problem
• Backtracking can also be said as an improvement to the brute force approach.
• Backtracking can be defined as a general algorithmic technique that considers searching
every possible combination in order to solve a computational problem.
Basic terminologies:
• Solution vector:
• The desired solution X to a problem instance P of input size n is as a vector of
candidate solutions that are selected from some finite set of possible solutions S.
• Thus, a solution can be represented as an n-tuple (X1, X2, …, Xn) and its partial
solution is given as (X1, X2,…, Xi) where i<n.
E.g. for a 4-queens problem X = {2,4,1,3) is a solution vector.
CS3401 Algorithms | Unit IV
10
 Backtracking: Introduction
• Constraints:
• Constraints are the rules to confine the solution vector (X1, X2…… Xa).
• They determine the values of candidate solutions and their relationship with each
other. There are two types of constraints:
• Implicit constraints.
• Explicit constraints.
• Solution space:
• All candidate solutions xi’s satisfying the explicit constraints form the solution space S
of a problem instance P. In a state space tree, all paths from the root node to a leaf
node describe the solution space.
• Example of in the case of N-queens problem, all n! orderings ( x_{1}, x_{2} ,…,xn )
form the solution space of that problem instance.
CS3401 Algorithms | Unit IV
11
 Backtracking: Introduction
• State space tree:
• A representation of the solution space S of a problem instance P in the form of a tree is
defined as the state space tree.
• It facilitates systematic search in the solution space to determine the desired solution to
a problem.
• A solution space of a given problem can be represented by different state space trees.
• Problem state:
• Each node in a state space tree describes a problem state or a partial solution formed
by making choices from the root of the tree to that node.
• Solution states:
• These are the problem states producing a tuple in the solution space S. At every
internal node, the solution states are partitioned into disjoint sub-solution spaces. In a
state space tree for a variable tuple size, all nodes are solution states.
• CS3401
In Algorithms
a state | Unitspace
IV tree for a fixed tuple size, only the leaf nodes are solution states
12
 Backtracking: Introduction
• State space:
• The state space of a problem is described by all paths from a root node to other nodes
in a state space tree.
• Promising node:
• A node is promising if it eventually leads to the desired solution.
• A promising node corresponds to a partial solution that is still feasible.
• Any time the partial node becomes infeasible, that branch will no longer be pursued.
• Non-promising node:
• A node is non-promising if it eventually leads to a state that cannot produce the desired
solution. A non-promising node corresponds to a partial solution that shows infeasibility
to get a complete solution.
• Live node:
• It is a node that has been generated and all of whose children are not yet been
generated.
• E-node:
• It is a live node whose children are currently being generated. 13
 Backtracking: Introduction
• Dead node:
• A node that is either not to be expanded further or for which all its children have been
generated is known as a dead node.
General Algorithm for Backtracking:
ALGORITHM Backtrack(X [1..i] )
//Gives a template of a generic backtracking algorithm
//Input: X[1..i] specifies first i promising components of a solution
//Output: All the tuples representing the problem’s solutions
if X[1..i] is a solution write X[1..i]
else
for each element x ∈ Si+1 consistent with X[1..i] and the constraints do
X[i + 1] ← x
Backtrack(X[1..i + 1])
CS3401 Algorithms | Unit IV
14
N-Queens problem

CS3401 Algorithms | Unit IV 15

 N-Queens problem : Introduction
• Consider a N X N chessboard on which we have to place N queens so that no two queens
attack each other by being in the same row or in the same column or on the same diagonal.
• Example:

• State-space tree of solving the four-queens

problem by backtracking.
• × denotes an unsuccessful attempt to place a
queen in the indicated column.
• The numbers above the nodes indicate the order
in which the nodes are generated.

CS3401 Algorithms | Unit IV

16
 N-Queens problem : Algorithm
Step 1 : Initialize an empty chessboard of size NxN.
Step 2 : Start with the leftmost column and place a queen in the first row of that column.
Step 3 : Move to the next column and place a queen in the first row of that column.
Step 4 : Repeat step 3 until either all N queens have been placed or it is impossible to place a
queen in the current column without violating the rules of the problem.
Step 5 : If all N queens have been placed, print the solution.
Step 6 : If it is not possible to place a queen in the current column without violating the rules
of the problem, backtrack to the previous column.
Step 7 : Remove the queen from the previous column and move it down one row.
Step 8 : Repeat steps 4-7 until all possible configurations have been tried.
CS3401 Algorithms | Unit IV
17
 N-Queens problem

CS3401 Algorithms | Unit IV

Complete Solution for 4 Queens Problem. 18
 8-Queens problem : All Solutions

CS3401 Algorithms | Unit IV

19
Hamiltonian Circuit Problem

CS3401 Algorithms | Unit IV 20

 Hamiltonian Circuit Problem: Introduction
• Given an undirected connected graph and two nodes X and Y then find a path from X to Y
visiting each node in the graph exactly once.
• Example:

CS3401 Algorithms | Unit IV

21
 Hamiltonian Circuit Problem: Algorithm
Example 2:

Solution: Start our search with vertex 'a'. This vertex 'a' becomes the root of our implicit tree.

CS3401 Algorithms | Unit IV 22

 Hamiltonian Circuit Problem: Algorithm
Next, we choose vertex 'b' adjacent to 'a' as it comes first in lexicographical order (b, c, d).
Next, we select 'd' adjacent to 'c.'

Next, we select 'c' adjacent to 'b.'

CS3401 Algorithms | Unit IV 23

 Hamiltonian Circuit Problem: Algorithm
Next, we select 'e' adjacent to 'd.' Next, we select vertex 'f' adjacent to 'e.' The vertex
adjacent to 'f' is d and e, but they have already visited.
Thus, we get the dead end, and we backtrack one step
and remove the vertex 'f' from partial solution.

CS3401 Algorithms | Unit IV 24

 Hamiltonian Circuit Problem: Algorithm
• From backtracking, the vertex adjacent to 'e' is b, c, d, and f from which vertex 'f' has already been checked,
and b, c, d have already visited.
• So, again we backtrack one step. Now, the vertex adjacent to d are e, f from which e has already been
checked, and adjacent of 'f' are d and e. If 'e' vertex, revisited them we get a dead state.
• So again we backtrack one step. Now, adjacent to c is 'e' and adjacent to 'e' is 'f' and adjacent to 'f' is 'd' and
adjacent to 'd' is 'a.' Here, we get the Hamiltonian Cycle as all the vertex other than the start vertex 'a' is
visited only once. (a - b - c - e - f -d - a).

CS3401 Algorithms | Unit IV 25

Subset Sum Problem

CS3401 Algorithms | Unit IV 26

 Subset Sum Problem : Introduction
27
• Find a subset of a given set A = {a1, . . . , an} of n positive integers whose sum is equal to a
given positive integer d. For Example: A={1, 2, 5, 6, 8} and d = 9, there are two
solutions: {1, 2, 6} and {1, 8}.
• It is convenient to sort the set’s elements in increasing order. So, we will assume that
a1 < a2 < . . . < a n .
• ALGORITHM Backtrack(X[1..i])
//Gives a template of a generic backtracking algorithm
//Input: X[1..i] specifies first i promising components of a solution
//Output: All the tuples representing the problem’s solutions
if X[1..i] is a solution write X[1..i]
else
for each element x ∈ Si+1 consistent with X[1..i] and the constraints do
X[i + 1]←x
Backtrack(X[1..i + 1]) CS3401 Algorithms | Unit IV 27
 Subset Sum Problem : Example
28

• Complete state-space tree of the backtracking algorithm applied to the instance A = {3, 5,
6, 7} and d = 15 of the subset-sum problem.
• The number inside a node is the sum of the elements already included in the subsets
represented by the node.
• The inequality below a leaf indicates CS3401
the reason forIV its termination.
Algorithms | Unit 28
 Subset Sum Problem : Example
29
Example 2: There are 5 distinct numbers {1,2,5,6,8}. Find the combinations of these numbers
such that the sum is 9. Use the backtracking model to arrive at the solution.

Initially subset=[] Sum=0

1 1 Now add the next element
1,2 3<9 Add the next element
1,2,5 8<9 Add the next element
1,2,5,6 14 Sum exceeds the given constraint.
Hence backtrack
1,2,5,8 16 Sum exceeds the given constraint.
Hence backtrack
1,2,6 9=9 Solution is found
CS3401 Algorithms | Unit IV 29
Subset Sum Problem : Example
30

30
Graph colouring problem

CS3401 Algorithms | Unit IV 31

 Graph colouring problem
32
• Graph coloring refers to the problem of coloring vertices of a graph in such a way that no
two adjacent vertices have the same color. This is also called as vertex coloring
problem.

Applications of Graph coloring

• Assignment
• Map coloring
• Scheduling the tasks
• Sudoku
• Prepare time table
• Conflict resolution
CS3401 Algorithms | Unit IV 32
 Graph colouring problem
33
Step 1:
A Graph G Consists of vertices from A to F. There are three colours used Red, Green and Blue.
(1 indicates Red, 2 indicates Green and 3 indicates Blue Color)

CS3401 Algorithms | Unit IV 33

 Graph colouring problem
34
Step 2

34
 Graph colouring problem
35
Step 3

Thus the graph coloring

problem is solved. The state-space
tree can be drawn for better
understanding of graph coloring
technique using backtracking
approach. 35
CS3401 Algorithms | Unit IV 36
 Branch and Bound: Introduction
37
• Branch and Bound method creates the state space tree by discovering the branches that need
to be expanded.
• In order to limit the number of possible solutions, the branch and bound method make use of
estimated bounds.
• There are different types of strategies that are used to explore the search tree and generate
branches. There strategies are:
• LIFO Search (Last In First Out).
• This search is based on Depth First Search Technique, in which the branch is extended
through every first child discovered at a certain depth, until a leaf node is reached.
• Stack data structure is used.
• FIFO Search (First In First Out)
• Uses Breadth First Search technique, where all nodes at the depth d are visited first,
before any nodes at the depth d+1 are visited.
• Queue data structure is used.
CS3401 Algorithms | Unit IV 37
 Branch and Bound: Introduction
38
Differences between Backtracking and Branch and Bound

Parameter Backtracking Branch and Bound

Backtracking traverses the state space tree Branch-and-Bound traverse the tree in any
Traversal
by DFS (Depth First Search) manner. manner, DFS or BFS.

Branch-and-Bound involves a bounding

Function Backtracking involves feasibility function.
function.

Backtracking is used for solving Decision Branch-and-Bound is used for solving

Problems
Problem. Optimization Problem.

Efficiency Backtracking is more efficient. Branch-and-Bound is less efficient.

Useful in solving N-Queen Problem, Sum of

Useful in solving Knapsack
Applications subset, Hamilton cycle problem, graph
Problem, Travelling Salesman Problem.
coloring problem
CS3401 Algorithms | Unit IV 38
15 Puzzle Problem

CS3401 Algorithms | Unit IV 39

15 Puzzle Problem : Introduction
40

CS3401 Algorithms | Unit IV 40

15 Puzzle Problem : Introduction
41

CS3401 Algorithms | Unit IV 41

15 Puzzle Problem : Introduction
42

CS3401 Algorithms | Unit IV 42

Assignment Problem

CS3401 Algorithms | Unit IV 43

 Assignment Problem : Introduction
44
• Let us illustrate the branch-and-bound approach by applying it to the problem of assigning
n people to n jobs so that the total cost of the assignment is as small as possible.
• An instance of the assignment problem is specified by an n × n cost matrix C.

• We have to find a lower bound on the cost of an optimal selection without actually solving
the problem. We can do this by several methods. For example, it is clear that the cost of any
solution, including an optimal one, cannot be smaller than the sum of the smallest elements
in each of the matrix’s rows.
• For the instance here, this sum is 2 + 3+ 1+ 4 = 10. It is important to stress that this is not the
cost of any legitimate selection (3 and 1 came from the same column of the matrix); it is just
a lower bound on the cost of any legitimate selection.
• We can and will apply the same thinking to partially constructed solutions. For example, for
any legitimate selection that selects 9 from the first row, the lower bound will be 9 + 3 + 1+ 4
44
= 17.
 Assignment Problem : Introduction
• It is sensible to consider a node with the best bound as most promising,45 although this does
not, of course, preclude the possibility that an optimal solution will ultimately belong to a
different branch of the state-space tree. This variation of the strategy is called the best-first
branch-and bound.
• The lower-bound value for the root, denoted lb, is 10. The nodes on the first level of the
tree correspond to selections of an element in the first row of the matrix, i.e., a job for
person a as Shown below.

In the above Figure, Levels 0 and 1 of the state-space tree for the instance of the assignment
problem being solved with the best-first branch-and-bound algorithm. The number above a
node shows the order in which the node was generated. A node’s fields indicate the job
number assigned to person a and the lower bound value, lb, for this node. 45
 Assignment Problem : Introduction
46

In the above figure, Levels 0, 1, and 2 of the state-space tree for the instance of the assignment problem being
solved with the best-first branch-and-bound algorithm.
46
Assignment Problem : Introduction
47

Complete state-space tree for the instance of the assignment problem

solved with the best-first branch-and-bound algorithm. 47
Knapsack Problem

CS3401 Algorithms | Unit IV 48

 Knapsack Problem : Introduction
• Branch-and-bound technique can be applied to solving the knapsack49problem. Given n
items of known weights wi and values vi , i = 1, 2, . . . , n, and a knapsack of capacity W, find
the most valuable subset of the items that fit in the knapsack. It is convenient to order the
items of a given instance in descending order by their value-to-weight ratios. Then the first
item gives the best payoff per weight unit and the last one gives the worst payoff per weight
unit, with ties resolved arbitrarily:
v1/w1 ≥ v2/w2 ≥ . . . ≥ vn/wn.
• It is natural to structure the state-space tree for this problem as a binary tree constructed as
follows. Each node on the ith level of this tree, 0 ≤ i ≤ n, represents all the subsets of n items
that include a particular selection made from the first i ordered items. This particular
selection is uniquely determined by the path from the root to the node: a branch going to the
left indicates the inclusion of the next item, and a branch going to the right indicates its
exclusion. We record the total weight w and the total value v of this selection in the node,
along with some upper bound ub on the value of any subset that can be obtained by adding
zero or more items to this selection.
CS3401 Algorithms | Unit IV 49
 Knapsack Problem : Introduction
50
Example:

A Simple way to compute the upper bound ub is to add to v, the total value of the items
already selected, the product of the remaining capacity of the knapsack W − w and the best
per unit payoff among the remaining items, which is vi+1/wi+1:
ub = v + (W − w)(vi+1/wi+1).
= 0+(10-0) (10)
= 100
50
 Knapsack Problem : Introduction
51
Example:

The knapsack’s capacity W is 10.

State-space tree of the best-first branch-and-bound

51
algorithm for the instance of the knapsack problem.
Travelling Salesman Problem

CS3401 Algorithms | Unit IV 52

 Travelling Salesman Problem : Introduction
53
• Branch-and-bound technique can be applied to instances of the travelling salesman
problem if we come up with a reasonable lower bound on tour lengths. One very simple
lower bound can be obtained by finding the smallest element in the intercity distance
matrix D and multiplying it by the number of cities n.
• But there is a less obvious and more informative lower bound for instances with symmetric
matrix D, which does not require a lot of work to compute. It is not difficult to show that
we can compute a lower bound on the length l of any tour as follows. For each city i, 1≤ i ≤
n, find the sum si of the distances from city i to the two nearest cities; compute the sum s
of these n numbers, divide the result by 2, and, if all the distances are integers, round up
the result to the nearest integer:
lb = s/2. 53
 Travelling Salesman Problem : Introduction
54
Solution:

(a) Weighted graph. (b) State-space tree of the

branch-and-bound algorithm to find a shortest
Hamiltonian circuit in this graph. The list of vertices
in a node specifies a beginning part of the
Hamiltonian circuits represented by the node.
54
 Travelling Salesman Problem : Introduction
55
Detailed Solution:

CS3401 Algorithms | Unit IV 55

 Travelling Salesman Problem : Introduction
56
Detailed Solution:

CS3401 Algorithms | Unit IV 56

 Travelling Salesman Problem : Introduction
57
Detailed Solution:

CS3401 Algorithms | Unit IV 57