Noida Institute of Engineering and Technology, Greater

Noida

Dynamic Programming, Backtracking

and Branch and Bound

Unit: 4

Design & Analysis of Algorithms

Sachin Singh
Assistant Professor
CS(AI)

Sachin Singh DAA Unit IV

1
12/22/2024
 Brief Introduction About Me

Name : Sachin Singh

Designation :Assistant Professor CS(AI) Department

NIET Greater Noida

Qualification:

 B.Tech AKGEC Ghaziabad in 2017.

 M.Tech Delhi Technological University(formely DCE) in 2020.

12/22/2024 Sachin Singh DAA Unit IV 2

 Evaluation Scheme

12/22/2024 Sachin Singh DAA Unit IV 3

 Syllabus

Sachin Singh DAA Unit IV

12/22/2024 4
 Branch wise Application

• First, we will start with the internet which is very much important for our daily life and we
cannot even imagine our life without the internet and it is the outcome of clever and creative
algorithms. Numerous sites on the internet can operate and falsify this huge number of data
only with the help of these algorithms.

• The everyday electronic commerce activities are massively subject to our data, for example,
credit or debit card numbers, passwords, OTPs, and many more. The centre technologies
used incorporate public-key cryptocurrency and digital signatures which depend on
mathematical algorithms.

• Even an application that doesn't need algorithm content at the application level depends
vigorously on the algorithm as the application relies upon hardware, GUI, networking, or
object direction and all of these create a substantial use of algorithms.

• There are some other vital use cases where the algorithm has been used such as if we watch
any video on YouTube then next time we will get related-type advice as recommended videos
for us.

12/22/2024 Sachin Singh DAA Unit IV 5

 Course Objective

• Upon completion of this course, students will be able to do the

following:
• Analyze the asymptotic performance of algorithms.
• Write rigorous correctness proofs for algorithms.
• Demonstrate a familiarity with major algorithms and data structures.
• Apply important algorithmic design paradigms and methods of
analysis.
• Synthesize efficient algorithms in common engineering design
situations.

Sachin Singh DAA Unit IV

12/22/2024 6
 Course Outcome

At the end of the semester, the student will be able:

Description Bloom’s Taxonomy
CO To have knowledge of basic principles of algorithm design and Knowledge, analysis
1 Analysis, asymptotic notations and growth of functions for time And design
and space complexity analysis and applying the same in different
sorting algorithms
CO To apply different problem-solving approaches for advanced Knowledge, analysis
2 data structures And apply
CO To apply divide and conquer method for solving merge sort, Knowledge, analysis and
3 quick sort, matrix multiplication and Greedy Algorithm for Apply
solving different Graph Problem.

CO To analyze and apply different optimization techniques like Knowledge, Analysis And
4 dynamic programming, backtracking and Branch & Bound to Apply
solve the complex problems
CO To understand the advanced concepts like NP Completeness and Knowledge, Analysis and
5 Fast Fourier Transform, to analyze and apply String Matching, Apply
Approximation and Randomized Algorithms to solve the
complex problems

12/22/2024 Sachin Singh DAA Unit IV 7

 Program Outcome

At the end of the semester, the student will be able:

POs Engineering Graduates will be able to
PO1 Engineering Knowledge
PO2 Problem Analysis
PO3 Design & Development of solutions
PO4 Conduct Investigation of complex problems
PO5 Modern Tool Usage
PO6 The Engineer and Society
PO7 Environment and sustainability
PO8 Ethics
PO9 Individual & Team work
PO10 Communication
PO11 Project management and Finance
PO12 Life Long Learning

12/22/2024 Sachin Singh DAA Unit IV 8

 CO-PO and PSO Mapping

Design and Analysis of Algorithm (kCS-502)

PO1
CO.K PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO12
1

ACSE0401.1 3 3 3 3 2 - - - 2 2 - 3

ACSE0401.2 3 3 3 3 2 2 - 1 1 1 - 3

ACSE0401.3 3 3 2 3 3 2 - 2 1 1 2 3

ACSE0401.4 3 3 3 3 2 2 - 2 2 1 3 3

ACSE0401.5 2 2 2 2 2 2 - 2 1 1 1 2

Average 2.8 2.8 2.6 2.8 2.2 1.6 - 1.8 1.4 1.2 1.2 2.8

12/22/2024 Sachin Singh DAA Unit IV 9

 Program Educational Objectives(PEOs)

PEO1: To have an excellent scientific and engineering breadth so as to

comprehend, analyze, design and provide sustainable solutions for
real-life problems using state-of-the-art technologies.
PEO2:To have a successful career in industries, to pursue higher studies or
to support enterpreneurial endeavors and to face global challenges.
PEO3:To have an effective communication skills, professional attitude,
ethical values and a desire to learn specific knowledge in emerging
trends, technologies for research, innovation and product
development and contribution to society.
PEO4: To have life-long learning for up-skilling and re-skilling for
successful professional career as engineer, scientist, enterpreneur
and bureaucrat for betterment of society

12/22/2024 Sachin Singh DAA Unit IV 10

 Result Analysis

12/22/2024 Sachin Singh DAA Unit IV 11

 End Semester Question Paper Template

B TECH
(SEM-V) THEORY EXAMINATION 20__-20__
COMPILER DESIGN
Time: 3 Hours Total
Marks: 100
Note: 1. Attempt all Sections. If require any missing data; then choose
suitably.
SECTION A
1.Q.No.
Attempt all questions in brief.
Question Marks 2 xCO10 =
20
1 2
2 2
. .
10 2

12/22/2024 Sachin Singh DAA Unit IV 12

 End Semester Question Paper Templates

SECTION B
2. Attempt any three of the following: 3 x 10 = 30

Q.No. Question Mark CO

s
1 10
2 10
. .
5 SECTION C 10
3. Attempt any one part of the following: 1 x 10 = 10

Q.No. Question Marks CO

1 10
2 10
12/22/2024 Sachin Singh DAA Unit IV 13
 End Semester Question Paper Templates
4. Attempt any one part of the following: 1 x 10 = 10
Q.No. Question Marks CO

1 10
2 10
5. Attempt any one part of the following: 1 x 10 = 10
Q.No. Question Marks CO
1 10
2 10

6. Attempt any one part of the following: 1 x 10 = 10

Q.No. Question Mark CO
s

1 10
2 10

12/22/2024 Sachin Singh DAA Unit IV 14

 Prerequisite and Recap

• Prerequisite
• Basic concept of c programming language.
• Concept of stack, queue and link list.

• Recap
• Flow Chart
• Algorithm

12/22/2024 Sachin Singh DAA Unit IV 15

 Unit Content

• Dynamic programming
• Knapsack
• . All pair shortest paths – Warshal’s and Floyd’s algorithms,
• Resource allocation problem
• Backtracking
• Graph Coloring
• n-Queen Problem
• Hamiltonian Cycles
• Sum of subsets
• Branch and Bound with examples such as Travelling Salesman Problem

12/22/2024 Sachin Singh DAA Unit IV 16

 Unit Objective

• This. objective of this unit is to make students understand about

• Dynamic Programming
• Back tracking
• Branch and Bound ..

12/22/2024 Sachin Singh DAA Unit IV 17

 Topic Objective

• This objective of this topic is to make students understand about

• Concepts of dynamic programming.
• Knapsack Problem(0/1)
• Floyd Warshall Algorithm
• Resource Allocation problem

12/22/2024 Sachin Singh DAA Unit IV 18

 Dynamic Programming(CO4)

• In the divide-and-conquer strategy, you divide the problem to

be solved into subproblems.
• The subproblems are further divided into smaller subproblems.
• That task will continue until you get subproblems that can be
solved easily.
• The basic idea of dynamic programming is to use a table to
store the solutions of solved subproblems.
• If you face a subproblem again, you just need to take the
solution in the table without having to solve it again.
• Therefore, the algorithms designed by dynamic programming
are very effective.

12/22/2024 Sachin Singh DAA Unit IV 19

 Dynamic Programming

To solve a problem by dynamic programming, you need to do the

following tasks:
 Find solutions of the smallest subproblems.
 Find out the formula (or rule) to build a solution of subproblem
through solutions of even smallest subproblems.
 Create a table that stores the solutions of subproblems. Then
calculate the solution of subproblem according to the found
formula and save to the table.
 From the solved subproblems, you find the solution of the
original problem.

12/22/2024 Sachin Singh DAA Unit IV 20

 Dynamic Programming(CO4)

Knapsack problem(0/1)

• Consider a thief gets into a home to rob and he carries a knapsack.

There are fixed number of items in the home — each with its own
weight and value — Jewelry, with less weight and highest value vs
tables, with less value but a lot heavy.

• To add fuel to the fire, the thief has an old knapsack which has
limited capacity.

• Obviously, he can’t split the table into half or jewelry into 3/4ths. He
either takes it or leaves it.

12/22/2024 Sachin Singh DAA Unit IV 21

 Dynamic Programming(CO4)

Dynamic-Programming Approach
• Let i be the highest-numbered item in an optimal
solution S for W dollars. Then S' = S - {i} is an optimal solution
for W - wi dollars and the value to the solution S is Vi plus the value
of the sub-problem.
• We can express this fact in the following formula: define c[i, w] to
be the solution for items 1,2, … , i and the maximum weight w.
• The algorithm takes the following inputs
• The maximum weight W
• The number of items n
• The two sequences v = <v1, v2, …, vn> and w = <w1, w2, …, wn>

12/22/2024 Sachin Singh DAA Unit IV 22

 Dynamic Programming(CO4)

Dynamic-0-1-knapsack (v, w, n, W)

• for w = 0 to W do
• c[0, w] = 0
• for i = 1 to n do
• c[i, 0] = 0
• for w = 1 to W do
• if wi ≤ w then
• if vi + c[i-1, w-wi] then
• c[i, w] = vi + c[i-1, w-wi]
• else c[i, w] = c[i-1, w]
• else
• c[i, w] = c[i-1, w]

12/22/2024 Sachin Singh DAA Unit IV 23

 Dynamic Programming(CO4)

• Knapsack problem(0/1)

• The set of items to take can be deduced from the table, starting
at c[n, w] and tracing backwards where the optimal values came
from.
• If c[i, w] = c[i-1, w], then item i is not part of the solution, and we
continue tracing with c[i-1, w]. Otherwise, item i is part of the
solution, and we continue tracing with c[i-1, w-W].

Analysis

• This algorithm takes θ(n, w) times as table c has (n + 1).(w + 1)

entries, where each entry requires θ(1) time to compute.

12/22/2024 Sachin Singh DAA Unit IV 24

 Dynamic Programming(CO4)
Knapsack problem(0/1)

Steps
• To calculate the maximum value that we can obtain with item i, we
first need to compare the weight of item i with the knapsack’s weight
capacity.

• Obviously, if item i weighs more than what the knapsack can hold, we
can’t include it, so it does not make sense to perform the calculation.

• In that case, the solution to this problem is simply the maximum value
that we can obtain without item i (i.e. the value in the row above, at
the same column).

12/22/2024 Sachin Singh DAA Unit IV 25

 Dynamic Programming(CO4)

Knapsack problem(0/1)
• However, suppose that item i weighs less than the knapsack’s
capacity.
• We thus have the option to include it, if it potentially increases the
maximum obtainable value.
• The maximum obtainable value by including item i is thus = the
value of item i itself + the maximum value that can be obtained with
the remaining capacity of the knapsack.
• We obviously want to make full use of the capacity of our knapsack,
and not let any remaining capacity go to waste.
• Therefore, at row i and column j (which represents the maximum
value we can obtain there), we would pick either the maximum
value that we can obtain without item i, or the maximum value that
we can obtain with item i, whichever is larger.
12/22/2024 Sachin Singh DAA Unit IV 26
 Dynamic Programming(CO4)
• For Example

• At row 3 (item 2), and column 5 (knapsack capacity of 4), we can

choose to either include item 2 (which weighs 4 units) or not.

• If we choose not to include it, the maximum value we can obtain is

the same as if we only have item 1 to choose from (which is found
in the row above, i.e. 0).

12/22/2024 Sachin Singh DAA Unit IV 27

 Dynamic Programming(CO4)

Knapsack problem(0/1)

• If we want to include item 2, the maximum value we can obtain with

item 2 is the value of item 2 (40) + the maximum value we can
obtain with the remaining capacity (which is 0, because the
knapsack is completely full already).

• At row 3 (item 2), and column 10 (knapsack capacity of 9), again,

we can choose to either include item 2 or not.

• If we choose not to, the maximum value we can obtain is the same
as that in the row above at the same column, i.e. 10 (by including
only item 1, which has a value of 10).

12/22/2024 Sachin Singh DAA Unit IV 28

 Dynamic Programming(CO4)

• Knapsack problem(0/1)

• If we include item 2, we have a remaining knapsack capacity of 9 -

4 = 5.

• We can find out the maximum value that can be obtained with a
capacity of 5 by looking at the row above, at column 5.

• Thus, the maximum value we can obtain by including item 2 is 40

(the value of item 2) + 10 = 50.

12/22/2024 Sachin Singh DAA Unit IV 29

 Dynamic Programming(CO4)

Knapsack problem(0/1)

• We pick the larger of 50 vs 10, and so the maximum value we can

obtain with items 1 and 2, with a knapsack capacity of 9, is 50.

• Once the table has been populated, the final solution can be found at
the last row in the last column, which represents the maximum value
obtainable with all the items and the full capacity of the knapsack.

• For more examples watch the following youtube link

• https://www.youtube.com/watch?v=oTTzNMHM05I

12/22/2024 Sachin Singh DAA Unit IV 30

 Dynamic Programming(CO4)

All Pair Shortest Path

• The all pair shortest path algorithm is also known as Floyd-Warshall

algorithm is used to find all pair shortest path problem from a given
weighted graph.

• As a result of this algorithm, it will generate a matrix, which will represent

the minimum distance from any node to all other nodes in the graph.

• At first the output matrix is same as given cost matrix of the graph. After
that the output matrix will be updated with all vertices k as the intermediate
vertex.

• The time complexity of this algorithm is O(V3), here V is the number of

vertices in the graph.
12/22/2024 Sachin Singh DAA Unit IV 31
 Dynamic Programming(CO4)
All Pair Shortest Path
• Representation

12/22/2024 Sachin Singh DAA Unit IV 32

 Dynamic Programming(CO4)

All Pair Shortest Path

12/22/2024 Sachin Singh DAA Unit IV 33

 Dynamic Programming(CO4)
All Pair Shortest Path
Example: Input-The cost matrix of the graph.
0 3 6

3 0 2 1

6 2 0 1 4 2

1 1 0 2 4

4 0 2 1

2 2 0 1

4 1 1 0

12/22/2024 Sachin Singh DAA Unit IV 34

 Dynamic Programming(CO4)

• All Pair Shortest Path

• Output-Matrix of all pair shortest paths
0 3 4
3 0 2 1
4 2 0 1 3 3

1 1 0 2 3

3 0 2 1

2 2 0 1
3 1 1 0

For more examples refer this link

https://www.youtube.com/watch?v=oNI0rf2P9gE

12/22/2024 Sachin Singh DAA Unit IV 35

 Dynamic Programming(CO4)

Resource Allocation problem

• A resource allocation problem in which ‘m’ resources are to be

allocated to ‘n’ projects. If ‘j’ resources are allocated to project I then
profit is P(i,j).

• The problem is to allocate the resource to the ‘n’ projects in such a

way as to maximize total next profit.

• This problem can be formulated as an n+1 stage graph problem as

follows . Stage i, 0 i ≤ n-1, represents project I, there are m+1
vertices, associated with stage i, 1 i ≤ n-1, stage 0 and n each one
vertex S and T respectively. Vertex (i,j) represents I, resources
allocated to projects 1,2,……..j.
12/22/2024 Sachin Singh DAA Unit IV 36
 Dynamic Programming(CO4)

Resource Allocation problem

• An optimal allocation of resources is defined by a maximum cost of a

path from S to T.

• A dynamic programming formulation for a k-stage is obtained by first

noticing that every S to T path is result of a sequence k-2 decisions.
• Let c(S,k) be a cost of path from node S to k and d( k, T) be the cost
of path from node k to T, via. some intermediate node , which can be
calculated recursively.

d(S,T)= max{c(S,k) +d(k,T)}

12/22/2024 Sachin Singh DAA Unit IV 37

 Dynamic Programming(CO4)
• Consider the following example to understand the concept of
multistage graph.

https://www.tutorialspoint.com/design_and_analysis_of_algorithms/d
esign_and_analysis_of_algorithms_multistage_graph.htm

12/22/2024 Sachin Singh DAA Unit IV 38

 Dynamic Programming(CO4)

• According to the formula, we have to calculate the cost (i, j) using

the following steps

• Step-1: Cost (K-2, j)

• In this step, three nodes (node 4, 5. 6) are selected as j. Hence, we

have three options to choose the minimum cost at this step.

Cost(3, 4) = min {c(4, 7) + Cost(7, 9),c(4, 8) + Cost(8, 9)} = 7

Cost(3, 5) = min {c(5, 7) + Cost(7, 9),c(5, 8) + Cost(8, 9)} = 5
Cost(3, 6) = min {c(6, 7) + Cost(7, 9),c(6, 8) + Cost(8, 9)} = 5

12/22/2024 Sachin Singh DAA Unit IV 39

 Dynamic Programming(CO4)

• Step-2: Cost (K-3, j)

• Two nodes are selected as j because at stage k - 3 = 2 there are two
nodes, 2 and 3. So, the value i = 2 and j = 2 and 3.

Cost(2, 2) = min {c(2, 4) + Cost(4, 8) + Cost(8, 9),c(2, 6) +

Cost(6, 8) + Cost(8, 9)}
=8
Cost(2, 3) = {c(3, 4) + Cost(4, 8) + Cost(8, 9), c(3, 5) + Cost(5, 8)+
Cost(8, 9), c(3, 6) + Cost(6, 8) + Cost(8, 9)}
= 10

12/22/2024 Sachin Singh DAA Unit IV 40

 Dynamic Programming(CO4)

• Step-3: Cost (K-4, j)

Cost (1, 1) = {c(1, 2) + Cost(2, 6) + Cost(6, 8) + Cost(8, 9), c(1, 3)
+
Cost(3, 5) + Cost(5, 8) + Cost(8, 9))}
= 12
c(1, 3) + Cost(3, 6) + Cost(6, 8 + Cost(8, 9))} = 13
Hence, the path having the minimum cost is 1→ 3→ 5→ 8→ 9.

For more examples refer the video

https://www.youtube.com/watch?v=9iE9Mj4m8jk

12/22/2024 Sachin Singh DAA Unit IV 41

 Backtracking(CO4)- Objective

This objective of this topic is to make students understand

about
• Concepts of Backtracking
• Graph Colouring
• N-Queens
• Sum of Subsets
• Hamiltonian Cycle

12/22/2024 Sachin Singh DAA Unit IV 42

 Prerequisite and Recap

• Prerequisite
• Algorithms Concepts
• C programming
• Graph

• Recap
• Dynamic Programming

12/22/2024 Sachin Singh DAA Unit IV 43

 Backtracking (CO4)

• Backtracking is an algorithmic-technique for solving problems

recursively by trying to build a solution incrementally, one piece
at a time, removing those solutions that fail to satisfy the
constraints of the problem at any point of time.

• It is a technique based on algorithm to solve problem. It uses

recursive calling to find the solution by building a solution step by
step increasing values with time.

• It removes the solutions that doesn't give rise to the solution of the
problem based on the constraints given to solve the problem.

12/22/2024 Sachin Singh DAA Unit IV 44

 Backtracking

• Backtracking algorithm is applied to some specific types of

problems.
• Decision problem used to find a feasible solution of the
problem.
• Optimisation problem used to find the best solution that can
be applied.
• Enumeration problem used to find the set of all feasible
solutions of the problem

• In backtracking problem, the algorithm tries to find a sequence

path to the solution which has some small checkpoints from
where the problem can backtrack if no feasible solution is found
for the problem.

12/22/2024 Sachin Singh DAA Unit IV 45

 Backtracking

For Example,

• Here, Green is the start point, blue is the intermediate point, red are
points with no feasible solution, dark green is end solution.

• when the algorithm propagates to an end to check if it is a solution

or not, if it is then returns the solution otherwise backtracks to the
point one step behind it to find track to the next point to find
solution.

12/22/2024 Sachin Singh DAA Unit IV 46

 Backtracking

ALGORITHM
Step 1 − if current_position is goal, return success

Step 2 − else,

Step 3− if current_position is an end point, return failed.

Step 4− else, if current_position is not end point, explore

and repeat above steps.

12/22/2024 Sachin Singh DAA Unit IV 47

 Backtracking

Graph Colouring Problem

• Given an undirected graph and a number m, determine if the
graph can be coloured with at most m colours such that no two
adjacent vertices of the graph are colored with the same color.

• Here coloring of a graph means the assignment of colors to all

vertices.

12/22/2024 Sachin Singh DAA Unit IV 48

 Backtracking

Graph Colouring Problem

Input:
• A 2D array graph[V][V] where V is the number of vertices
in graph and graph[V][V] is adjacency matrix
representation of the graph.
• A value graph[i][j] is 1 if there is a direct edge from i to j,
otherwise graph[i][j] is 0.
• An integer m which is the maximum number of colors that
can be used

Output:
An array color[V] that should have numbers from 1 to m.
color[i] should represent the color assigned to the ith vertex.

12/22/2024 Sachin Singh DAA Unit IV 49

 Backtracking

N- Queens Problem

• N - Queens problem is to place n - queens in such a manner on an

n x n chessboard that no queens attack each other by being in the
same row, column or diagonal.

• It can be seen that for n =1, the problem has a trivial solution, and
no solution exists for n =2 and n =3.

• So first we will consider the 4 queens problem and then generate

it to n - queens problem.

• Given a 4 x 4 chessboard and number the rows and column of the

chessboard 1 through 4.

12/22/2024 Sachin Singh DAA Unit IV 50

 Backtracking

N- Queens Problem

Since, we have to place 4 queens such as q1 q2 q3 and q4 on the

chessboard, such that no two queens attack each other. In such a
conditional each queen must be placed on a different row, i.e.,
we put queen "i" on row "i."

12/22/2024 Sachin Singh DAA Unit IV 51

 Backtracking

N- Queens Problem

• Now, we place queen q1 in the very first acceptable position (1, 1).

• Next, we put queen q2 so that both these queens do not attack each
other.

• We find that if we place q2 in column 1 and 2, then the dead end is

encountered.

• Thus the first acceptable position for q 2 in column 3, i.e. (2, 3) but
then no position is left for placing queen 'q 3' safely.

12/22/2024 Sachin Singh DAA Unit IV 52

 Backtracking

N- Queens Problem

• So we backtrack one step and place the queen 'q 2' in (2, 4), the
next best possible solution. Then we obtain the position for
placing 'q3' which is (3, 2).

• But later this position also leads to a dead end, and no place is
found where 'q4' can be placed safely.

• Then we have to backtrack till 'q1' and place it to (1, 2) and then
all other queens are placed safely by moving q2 to (2, 4), q3 to (3,
1) and q4 to (4, 3).

12/22/2024 Sachin Singh DAA Unit IV 53

 Backtracking

N- Queens Problem

• That is, we get the solution (2, 4, 1, 3).

• This is one possible solution for the 4-queens problem.

• For another possible solution, the whole method is repeated for

all partial solutions. The other solutions for 4 - queens problems
is (3, 1, 4, 2) i.e.

12/22/2024 Sachin Singh DAA Unit IV 54

 Backtracking

N- Queens Problem

• Fig shows the complete state space for 4 - queens problem. But
we can use backtracking method to generate the necessary node
and stop if the next node violates the rule, i.e., if two queens are
attacking.

• It can be seen that all the solutions to the 4 queens problem can
be represented as 4 - tuples (x1, x2, x3, x4) where xi represents the
column on which queen "qi" is placed.

12/22/2024 Sachin Singh DAA Unit IV 55

 Backtracking

4- Queens Problem

12/22/2024 Sachin Singh DAA Unit IV 56

 Backtracking

N- Queens Problem

Place (k, i)
{
For j ← 1 to k - 1
do if (x [j] = i)
or (Abs x [j]) - i) = (Abs (j - k))
then return false;
return true;
}
• Place (k, i) return true if a queen can be placed in the kth row
and ith column otherwise return is false.
• x [] is a global array whose final k - 1 values have been set. Abs
(r) returns the absolute value of r.

12/22/2024 Sachin Singh DAA Unit IV 57

 Backtracking

N- Queens Problem
• x [] is a global array whose final k - 1 values have been set.
Abs (r) returns the absolute value of r.

N - Queens (k, n)
{
For i ← 1 to n
do if Place (k, i) then
{
x [k] ← i;
if (k ==n) then
write (x [1....n));
else
N - Queens (k + 1, n);
}
}
12/22/2024 Sachin Singh DAA Unit IV 58
 Backtracking

8- Queens Problem

1.Thus, the solution for 8 -

queen problem for (4, 6, 8, 2, 7, 1, 3, 5).
2.If two queens are placed at position (i, j) and (k, l).
3.Then they are on same diagonal only if (i - j) = k - l or i
+ j = k + l.
4.The first equation implies that j - l = i - k.
5.The second equation implies that j - l = k - i.
6.Therefore, two queens lie on the duplicate diagonal if an
d only if |j-l|=|i-k|
12/22/2024 Sachin Singh DAA Unit IV 59
 Backtracking

Hamiltonian Cycles
• Hamiltonian Path in an undirected graph is a path that visits each
vertex exactly once.
• A Hamiltonian cycle (or Hamiltonian circuit) is a Hamiltonian
Path such that there is an edge (in the graph) from the last vertex
to the first vertex of the Hamiltonian Path.

12/22/2024 Sachin Singh DAA Unit IV 60

 Backtracking

Sum of Subsets Problem

• Subset sum problem is to find subset of elements that are
selected from a given set whose sum adds up to a given
number K.

• We are considering the set contains non-negative values. It is

assumed that the input set is unique (no duplicates are
presented).

• Here backtracking approach is used for trying to select a valid

subset when an item is not valid, we will backtrack to get the
previous subset and add another element to get the solution.

12/22/2024 Sachin Singh DAA Unit IV 61

 Backtracking

Sum of Subsets Problem

• Subset sum problem is to find subset of elements that are
selected from a given set whose sum adds up to a given
number K.

• We are considering the set contains non-negative values. It is

assumed that the input set is unique (no duplicates are
presented).

• Here backtracking approach is used for trying to select a valid

subset when an item is not valid, we will backtrack to get the
previous subset and add another element to get the solution.

12/22/2024 Sachin Singh DAA Unit IV 62

 Backtracking

Sum of Subsets Problem

Backtracking
. Algorithm for Subset Sum
Using exhaustive search we consider all subsets irrespective of
whether they satisfy given constraints or not. Backtracking can be
used to make a systematic consideration of the elements to be selected.

Algorithm
subsetSum(set, subset, n, subSize, total, node, sum)
Input: The given set and subset, size of set and subset, a total of the
subset, number of elements in the subset and the given sum.
Output: All possible subsets whose sum is the same as the given sum.

12/22/2024 Sachin Singh DAA Unit IV 63

 Backtracking

Sum of Subsets Algorithm

12/22/2024 Sachin Singh DAA Unit IV 64

 Backtracking

Sum of Subsets Example

Given a set S = (3, 4, 5, 6) and X =9. Obtain the subset sum using
Backtracking approach.
Solution:
Initially S = (3, 4, 5, 6) and X =9.
S'= (∅)

• The implicit binary tree for the subset sum problem is shown in Fig.

• The number inside a node is the sum of the partial solution elements
at a particular level.

• Thus, if our partial solution elements sum is equal to the positive

integer 'X' then at that time search will terminate, or it continues if all
the possible solution needs to be obtained.

12/22/2024 Sachin Singh DAA Unit IV 65

 Backtracking

Sum of Subsets Example

12/22/2024 Sachin Singh DAA Unit IV 66

 Backtracking

Sum of Subsets Example

Input: The Set: {10, 7, 5, 18, 12, 20, 15}

The sum Value: 35
Output:
All possible subsets of the given set, where sum of each element for
every subsets is same as the given sum value.
{10, 7, 18}
{10, 5, 20}
{5, 18, 12}
{20, 15}

12/22/2024 Sachin Singh DAA Unit IV 67

 Branch and Bound(CO4)- Objective

This objective of this topic is to make students understand

about
• Concepts of Branch and Bound
• Travelling Salesman Problem

12/22/2024 Sachin Singh DAA Unit IV 68

 Prerequisite and Recap

• Prerequisite
• Algorithms Concepts
• C programming
• Graph

• Recap
• Backtracking

12/22/2024 Sachin Singh DAA Unit IV 69

 Branch and Bound (CO4)

• Branch and bound is an algorithm design paradigm which is

generally used for solving combinatorial optimization problems.

• These problems are typically exponential in terms of time

complexity and may require exploring all possible permutations in
worst case.

• The Branch and Bound Algorithm technique solves these problems

relatively quickly.

12/22/2024 Sachin Singh DAA Unit IV 70

 Branch and Bound

Travelling Salesman Problem

Given a set of cities and distance between every pair of cities, the
problem is to find the shortest possible tour that visits every city
exactly once and returns to the starting point.

• In Branch and Bound method, for current node in tree, we compute

a bound on best possible solution that we can get if we down this
node.

• If the bound on best possible solution itself is worse than current

best (best computed so far), then we ignore the subtree rooted with
the node.

12/22/2024 Sachin Singh DAA Unit IV 71

 Branch and Bound

Travelling Salesman Problem

Note that the cost through a node includes two costs.

1) Cost of reaching the node from the root (When we reach a node, we
have this cost computed)

2) Cost of reaching an answer from current node to a leaf (We compute

a bound on this cost to decide whether to ignore subtree with this node
or not).

12/22/2024 Sachin Singh DAA Unit IV 72

 Branch and Bound

Travelling Salesman Problem- Example

• In this method we expand the node which is most promising means the
node which promises that expanding or choosing it will give us the
optimal solution.

• So we prepare the tree starting from root then we expand it.

• The cost matrix is defined by

C(i,j) = W(i,j) , if there is a direct path from Ci to Cj.
=, If there is no direct path from Ci to Cj.

12/22/2024 Sachin Singh DAA Unit IV 73

 Branch and Bound

Travelling Salesman Problem- Example

Consider the following cost matrix

Here C(0,2)=30, C(4,0)=16, C(1,1)= and so on. Below is the

state space tree for the TSP , which shows optimal solution
marked in green
(https://www.techiedelight.com/travelling-salesman-problem-using-branch-and-
bound/
)

12/22/2024 Sachin Singh DAA Unit IV 74

 Branch and Bound

12/22/2024 Sachin Singh DAA Unit IV 75

 Branch and Bound

Travelling
. Salesman Problem- Steps

• As we can see from above diagram , every node has a cost associated
to it.

• When we go from city I to city j, cost of a node j will be sum of cost of

parent node I, cost of the edge (I,j) and lower bound of the path starting
at node j.

• As root node is the first node to be expanded , it doesn’t have any

parent. So cost will be only lower bound of the path starting at root.

• To get the lower bound of the path starting from the node , we reduce
each row and column in such a wat that there must be atleast one zero
in each row and column.
12/22/2024 Sachin Singh DAA Unit IV 76
 Branch and Bound

• . For doing this, we need to reduce the minimum value from each
element in each row and column.

Let’s start from root node

• We reduce the minimum value in each row from each element in that
row.
• Minimum in each row of cost matrix M is marked by blue[10,2,2,3,4]
below.

12/22/2024 Sachin Singh DAA Unit IV 77

 Branch and Bound

• . After reducing the row, we get the below matrix. We then reduce
the minimum value in each column from each element in that
column.

• Minimum in each column is marked by blue[1 0 3 0 0]. After

reducing the column, we get below reduced matrix. This matrix
will be further processed by child nodes of root node to calculate
their lower bound.

12/22/2024 Sachin Singh DAA Unit IV 78

 Branch and Bound

• The total expected cost at the root node is sum of all reductions.
• Cost=[10 2 2 3 4] + [1 0 3 0 0]= 25
• Since we have discovered the root node C0, the next node to be
expanded can be any node from C1,C2, C3, C4. Whichever node
has minimum cost, that node will be expanded further. So we have
to find out the expanding cost of each node.

12/22/2024 Sachin Singh DAA Unit IV 79

 Branch and Bound
• The parent node (C0) has below reduced matrix-
.

• Let us consider edge from 0 1

1. As we are adding edge (0,1) to our search space, we get outgoing

edges for city 0 to infinity and all incoming edges to city 1 to
infinity. We also set (1,0) to infinity.

2. So in reduced matrix of parent node we change all the elements in

row 0 and column 1 and at index (1,0) to infinity(marked in red).

12/22/2024 Sachin Singh DAA Unit IV 80

 Branch and Bound

The resulting cost matrix is

12/22/2024 Sachin Singh DAA Unit IV 81

 Branch and Bound

.
3. We try to calculate lower bound of the path starting at node 1 using
above resulting cost matrix.

4. The lower bound is 0 as matrix is already in reduced form. i.e. all

rows and all columns have zero value.

Therefore for node 1, cost will be

Cost= cost of node 0 + cost of the edge (0,1) + lower bound of the
path starting at node 1
= 25 + 10 +0= 35

12/22/2024 Sachin Singh DAA Unit IV 82

 Branch and Bound

.Lets consider edge from 0 2

1. Change all the elements in row 0 and column 2 and at index(2,0) to

infinity(marked in red).

The resulting cost matrix is:

12/22/2024 Sachin Singh DAA Unit IV 83

 Branch and Bound

• Now calculate lower bound of the path starting at node 2 using the
Approach discussed earlier. Resulting matrix will be-

Therefore the cost of node 2, cost will be

Cost = Cost of node 0 + cost of the edge (0,2) + Lower bound of the
path starting at node 2
= 25 + 17 + 11= 53

12/22/2024 Sachin Singh DAA Unit IV 84

 Branch and Bound

Let us consider edge from 0 3

1. Change the elements in row 0 and column 3 and at index(3,0) to
Infinity(marked in red).

The Resulting cost matrix is :

12/22/2024 Sachin Singh DAA Unit IV 85

 Branch and Bound

• Now calculate lower bound of the path starting at node 3 using the
Approach discussed earlier.
• Lower bound of the path starting at node 3 is 0 as it is already in
reduced form, i.e. all rows and all columns have zero value.

Therefore for node 3, cost will be

Cost = Cost of node 0 + cost of the edge (0,3) + Lower bound of the
path starting at node 3
= 25 + 0 + 0= 25

Similarly we calculate for 0 4. Its cost will be 31.

12/22/2024 Sachin Singh DAA Unit IV 86

 Branch and Bound

• Now we find a live node with estimated cost. Live nodes 1,2,3 and 4
has costs 35, 53, 25 and 31 respectively.

• The minimum among them is node 3 having cost 25. So node 3 will
be expanded further as shown in state space tree diagram.

• After adding its children to list of live nodes, we again find a live
node with least cost and expand it.

• We continue the search till a leaf is encountered in space search tree.

If a leaf is encountered, then the tour is completed and we will
return back to the root node.

12/22/2024 Sachin Singh DAA Unit IV 87

 Faculty Video Links, Youtube & NPTEL Video Links and
Online Courses Details

• Youtube/other Video Links

• https://www.youtube.com/watch?v=oNI0rf2P9gE
• https://www.youtube.com/watch?v=9iE9Mj4m8jk
• https://www.youtube.com/watch?v=kyLxTdsT8ws
• https://nptel.ac.in/courses/106106133/

12/22/2024 Sachin Singh DAA Unit IV 88

 Quiz
1. The problem of placing n queens in a chessboard such that no two
queens attack each other is called as________________.

2. ___________ enumerates a list of promising nodes that could be

computed to give the possible solutions of a given problem.

3. When dynamic programming is applied to a problem, it takes far

less time as compared to other methods that don’t take advantage
of overlapping subproblems.(True/False).

4. Branch and bound is a __________

5. Both LIFO branch and bound strategy and backtracking leads to

depth first search.(True/False)

12/22/2024 Sachin Singh DAA Unit IV 89

 Quiz
6. __________can traverse the state space tree only in DFS manner?

7. A node is said to be ____________ if it has a possibility of

reaching a complete solution.

8. A state-space tree for a backtracking algorithm is constructed using

______________________.

9. The leaves in a state-space tree represent only complete solutions.

(True / False)

10. Backtracking algorithm is faster than the brute force

technique(True/ False)

12/22/2024 Sachin Singh DAA Unit IV 90

 Weekly Assignment
1. Explain Floyd algorithm and also analyze its complexity.
[CO4]
2. What is nQueens problem? Write the steps for solving the nQueens
problem. [CO4]
3. Explain Graph colouring Problem with the help of suitable
example.
[CO4]
4. Find the optimal solution to the knapsack 0/1 instances n=7,m=15.
(V1,V2,V3,V4,V5,V6,V7)=(10,5,15,7,6,18,3)
(W1,W2,W3,W4,W5,W6,W7 )=(2,3,5,7,1,4,1)
[CO4]
5. Let S={4,6,7,8} and m=18. Find all possible subsets of S that sums
to m. Draw state space tree that is generated. [ CO4]
6. Use Floyd Warshall algorithm to find shortest path for all of the
vertices of the given graph.Sachin
12/22/2024
Give the complexity
Singh DAA Unit IV
of the algorithm . 91
 Weekly Assignment

6. Consider following instance for simple Knapsack Problem.

Find the solution using greedy method. N:8, Value of knapsack
is 110.
V : { 30,20,100,90,160,200,180,220}
W : {5, 10, 20,22,33, 43, 45, 55}
[CO4]

7. Write short note on Branch and Bound Technique. [CO4]

12/22/2024 Sachin Singh DAA Unit IV 92

 MCQ

1. Which of the following is/are property/properties of a dynamic

programming problem?
a) Optimal substructure
b) Overlapping subproblems
c) Greedy approach
d) Both optimal substructure and overlapping subproblems

2. If a problem can be broken into subproblems which are reused

several times, the problem possesses ____________ property.
a) Overlapping subproblems
b) Optimal substructure
c) Memoization
d) Greedy

12/22/2024 Sachin Singh DAA Unit IV 93

 MCQ

3. When dynamic programming is applied to a problem, it takes far

less time as compared to other methods that don’t take advantage
of overlapping subproblems.
a) True
b) False

4. Which of the following problems is NOT solved using dynamic

programming?
a) 0/1 knapsack problem
b) Matrix chain multiplication problem
c) Edit distance problem
d) Fractional knapsack problem

12/22/2024 Sachin Singh DAA Unit IV 94

 MCQ

5. Which of the following problems should be solved using dynamic

programming?
a) Mergesort
b) Binary search
c) Longest common subsequence
d) Quicksort

6. If a problem can be solved by combining optimal solutions to non-

overlapping problems, the strategy is called _____________
a) Dynamic programming
b) Greedy
c) Divide and conquer
d) Recursion

12/22/2024 Sachin Singh DAA Unit IV 95

 MCQ

7. Which of the problems cannot be solved by backtracking method?

a) n-queen problem
b) subset sum problem
c) hamiltonian circuit problem
d) travelling salesman problem

8. Backtracking algorithm is implemented by constructing a tree of

choices called as?
a) State-space tree
b) State-chart tree
c) Node tree
d) Backtracking tree

12/22/2024 Sachin Singh DAA Unit IV 96

 MCQ

9. In what manner is a state-space tree for a backtracking algorithm

constructed?
a) Depth-first search
b) Breadth-first search
c) Twice around the tree
d) Nearest neighbour first

10. The problem of finding a subset of positive integers whose sum is

equal to a given positive integer is called as?
a) n- queen problem
b) subset sum problem
c) knapsack problem
d) hamiltonian circuit problem

12/22/2024 Sachin Singh DAA Unit IV 97

 Glossary Question

Q.1The 0/1 Knapsack problem is an example of ____________

a) Greedy algorithm
b) 2D dynamic programming
c) 1D dynamic programming
d) Divide and conquer
Q.2 __________method can be used to solve the Knapsack problem?
a) Brute force algorithm
b) Recursion
c) Dynamic programming
d) Brute force, Recursion and Dynamic Programming

Q.3 ______ is used to solve the matrix chain multiplication problem?

a) Dynamic programming
b) Brute force
c) Recursion
d) Dynamic Programming, Brute force, Recursion

12/22/2024 Sachin Singh DAA Unit IV 98

 Glossary Question

Q.4 ________ problems can be solved using the longest subsequence problem?
a) Longest increasing subsequence
b) Longest palindromic subsequence
c) Longest bitonic subsequence
d) Longest decreasing subsequence
Q.5 ___________is the worst case time complexity of dynamic programming solution
of the subset sum problem(sum=given subset sum)?
a) O(n)
b) O(sum)
c) O(n2)
d) O(sum*n)
Q.6 In ______ directions queens attack each other?
a) 1
b) 2
c) 3
d) 4

12/22/2024 Sachin Singh DAA Unit IV 99

 Glossary Question
Q.7 Placing n-queens so that no two queens attack each other is called__________
a) n-queen’s problem
b) 8-queen’s problem
c) Hamiltonian circuit problem
d) subset sum problem

Q.8 The n-queens problem implemented in _________.

a) carom
b) chess
c) ludo
d) cards
Q.9 __________methods can be used to solve n-queen’s problem.
a) greedy algorithm
b) divide and conquer
c) iterative improvement
d) backtracking

12/22/2024 Sachin Singh DAA Unit IV 100

 Old Question Papers(2019-2020)

12/22/2024 Sachin Singh DAA Unit IV 101

 Old Question Papers(2019-2020)

12/22/2024 Sachin Singh DAA Unit IV 102

 Old Question Papers(2019-2020)

12/22/2024 Sachin Singh DAA Unit IV 103

 Old Question Papers(2019-2020)

12/22/2024 Sachin Singh DAA Unit IV 104

 Old Question Papers(2019-2020)

12/22/2024 Sachin Singh DAA Unit IV 105

 Old Question Papers(2018-2019)

12/22/2024 Sachin Singh DAA Unit IV 106

 Old Question Papers(2018-2019)

12/22/2024 Sachin Singh DAA Unit IV 107

 Old Question Papers(2018-2019)

12/22/2024 Sachin Singh DAA Unit IV 108

 Old Question Papers(2018-2019)

12/22/2024 Sachin Singh DAA Unit IV 109

 Old Question Papers(2018-2019)

12/22/2024 Sachin Singh DAA Unit IV 110

 Old Question Papers(2017-2018)

12/22/2024 Sachin Singh DAA Unit IV 111

 Old Question Papers(2017-2018)

12/22/2024 Sachin Singh DAA Unit IV 112

 Old Question Papers(2017-2018)

12/22/2024 Sachin Singh DAA Unit IV 113

 Old Question Papers(2017-2018)

12/22/2024 Sachin Singh DAA Unit IV 114

 Old Question Papers(2017-2018)

12/22/2024 Sachin Singh DAA Unit IV 115

 Expected Questions for University Exam

1. Solve the problem of travelling salesman problem on

following graph and perform operation for each vertex.
[CO4]

2. Write Short notes on the following. [CO4]

i) Graph Coloring
ii)Sub Set Sum Problem
iii)Hamiltonian Cycle

12/22/2024 Sachin Singh DAA Unit IV 116

 Expected Questions for University Exam

3. Find out one optimal solution for following using back

tracking.
4*4 chessboard,4 Queen Problem
8*8 chessboard,8 Queen Problem
[CO4]

4. Write the difference between the Greedy method and

Dynamic programming.
[CO4]

5. Explain the term state space search tree with suitable

example.
[CO4]

6. Explain Floyd algorithm and also analyze its complexity

[CO4]
12/22/2024 Sachin Singh DAA Unit IV 117
 Recap of Unit

This unit gives the insights of different techniques like Dynamic

Programming, Back tracking and Branch and Bound . Different
real world problem have been solved with these approaches.
The problems discussed here are Travelling salesman Problem,
n Queens problem , Sum of subsets problem, Floyd Warshall
algorithm, Knapsack Problem, Graph Colouring and
Hamiltonion Cycle.

12/22/2024 Sachin Singh DAA Unit IV 118

 Unit 4

Thank You

12/22/2024 Sachin Singh DAA Unit IV 119