UNIT 7: DYNAMIC PROGRAMMING

Mr. Prayag Patel

Assistant Professor
IT-ICT Department, LJIET
[email protected]
 Topics to be Covered
▪ Introduction
▪ The Principle of optimality
▪ Problem Solving using Dynamic Programming :
1) Calculating the Binomial Coefficient
2) Longest Common Subsequence
3) Making Change Problem
4) Knapsack Problem
5) Assembly Line-Scheduling
6) Matrix Chain Multiplication
7) All Points Shortest Path
Mr. Prayag Patel Department: IT/ICT-LJIET
 TOPIC : INTRODUCTION

Mr. Prayag Patel

 Introduction
▪ In the word Dynamic programming the word programming
stands for planning.
▪ Dynamic programming algorithms: The result of smaller
reoccurring instances are obtained to solve the problem.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Introduction
▪ A box contains 4 stones worth $1400, $3000,
$4200 and $7100. They weigh 20, 50,
60 and 100 grams respectively.
▪ Your bag can only hold 110 grams.
1400 3000 4200 7100
20 50 60 100
70 60 70 71

▪ The greedy algorithm selects the best

candidate with the highest profit, i.e., stone
worth $7100.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Introduction
▪ A box contains 4 stones worth $1400,
$3000, $4200 and $7100. They weigh
20, 50, 60 and 100 grams respectively.
▪ Your bag can only hold 110 grams.
▪ But, the optimal choice would be to
select the two stones worth $3000 and
$4200.
▪ Their combined weight is 110 grams,
which means they will fit into your bag
and you will get a profit of $7200.
▪ If there are 100 stones, how to efficiently
find the optimal answer?
Mr. Prayag Patel Department: IT/ICT-LJIET
 The Principle of Optimality
▪ A dynamic-programming algorithm solves every sub-problem just
once and then saves its answer in a table.
▪ It avoids the work of re-computing the answer every time the sub
problem is encountered.
▪ The dynamic programming algorithm obtains the solution using
principle of optimality.
▪ The principle of optimality states that “in an optimal sequence of
decisions or choices, each subsequence must also be optimal.”
▪ If it is not possible to apply the principle of optimality then it is
almost impossible to obtain the solution using the dynamic
programming approach.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Steps of Dynamic programming algorithms
▪ Steps of Dynamic programming algorithms :
1) Characterize the structure of an optimal solution.
2) Recursively define the value of an optimal solution.
3) Compute the value of an optimal solution, typically in a
bottom-up fashion.
4) Construct an optimal solution from the computed
information.

Mr. Prayag Patel Department: IT/ICT-LJIET

 TOPIC : KNAPSACK PROBLEM

Mr. Prayag Patel

 Knapsack Problem
• Problem Definition
– Want to carry essential items in one bag
– Given a set of items, each has
• A weight w(i.e., 12kg)
• A value v (i.e., 4$)
• A capacity of bag W(i.e., 15kg)

• Goal
― Item to include in a collection so that
• The total weight is less than some given limit (capacity of bag)
• And the total value is as large as possible.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Knapsack Problem
Types of Knapsack problem
1) fractional Knapsack Problem
fractional knapsack
2) 0/1 knapsack problem problem is that you CAN
take a fraction of an item.

0/1 knapsack problem is

that you CANNOT take a
fraction of an item.
1 You can either take it or not

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Example: For the given instance of problem obtain the
optimal solution for the knapsack problem using
Dynamic programming. The capacity of Knapsack is
W=5.
Object Weight(wi) Value(vi)
1 2 3
2 3 4
3 4 5
1 4 5 6

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-1:Characterize the structure of an optimal solution.

Formula for 0/1 Knapsack Problem :

1) B[i,0] = B[0,j] = 0
2) B[i, j] = B[i - 1, j], if j < wi
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
Capacity
j
i/j 0 1 2 3 4 5
i 0
1

Object
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0
1
2
3
4
B[i,0] = B[0,0] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0
1 0
2
3
4
B[i,0] = B[1,0] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0
1 0
2 0
3
4
B[i,0] = B[2,0] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0
1 0
2 0
3 0
4
B[i,0] = B[3,0] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0
1 0
2 0
3 0
4 0
B[i,0] = B[4,0] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0 0
1 0
2 0
3 0
4 0
B[0, j] = B[0,1] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0 0 0
1 0
2 0
3 0
4 0
B[0, j] = B[0,2] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0 0 0 0
1 0
2 0
3 0
4 0
B[0, j] = B[0,3] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0 0 0 0 0
1 0
2 0
3 0
4 0
B[0, j] = B[0,4] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

j
i i/j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
B[0, j] = B[0,5] = 0
1
Formula for 0/1 Knapsack Problem :
1) B[i,0] = B[0,j] = 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi vi 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3 3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
For B[1,1] , i=1 , j=1 & w1= 2 , v1 = 3
3 4 5
4 5 6
j
j < wi ? i/j 0 1 2 3 4 5
1 < 2 ? Yes i 0 0 0 0 0 0 0
B[1, 1] = B[0,1] 1 0
=0 1 2 0
3 0
4 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi vi 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3 3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
For B[1,2] , i=1 , j=2 & w1= 2 , v1 = 3
3 4 5
4 5 6
j
j < wi ? i/j 0 1 2 3 4 5
2 < 2 ? No 0 0 0 0 0 0 0
i
1 0 0
1 2 0
3 0
4 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j ≥ wi ? For B[1,2] , i=1 , j=2 & w1= 2 , v1 = 3
3 4 5
2 ≥ 2 ? Yes
4 5 6
B[1,2] = max { B[1 - 1, 2], v1 + B[1 - 1, 2 – w1]}
= max { B[0, 2], 3 + B[0, 0]} i/j 0 1 2 3 4 5
= max { 0, 3 + 0]} 0 0 0 0 0 0 0
=3 1 0 0
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[1,3] , i=1 , j=3 & w1= 2 , v1 = 3
3 4 5
3 < 2 ? No
4 5 6

i/j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j ≥ wi ? For B[1,3] , i=1 , j=3 & w1= 2 , v1 = 3
3 4 5
3 ≥ 2 ? Yes
4 5 6
B[1,3] = max { B[1 - 1, 3], v1 + B[1 - 1, 3 – w1]}
= max { B[0, 3], 3 + B[0, 1]} i/j 0 1 2 3 4 5
= max { 0, 3 + 0]} 0 0 0 0 0 0 0
=3 1 0 0 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[1,4] , i=1 , j=4 & w1= 2 , v1 = 3
3 4 5
4 < 2 ? No
4 5 6

i/j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j ≥ wi ? For B[1,4] , i=1 , j=4 & w1= 2 , v1 = 3
3 4 5
4 ≥ 2 ? Yes
4 5 6
B[1,4] = max { B[1 - 1, 4], v1 + B[1 - 1, 4 – w1]}
= max { B[0, 4], 3 + B[0, 2]} i/j 0 1 2 3 4 5
= max { 0, 3 + 0]} 0 0 0 0 0 0 0
=3 1 0 0 3 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[1,5] , i=1 , j=5 & w1= 2 , v1 = 3
3 4 5
5 < 2 ? No
4 5 6

i/j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j ≥ wi ? For B[1,4] , i=1 , j=5 & w1= 2 , v1 = 3
3 4 5
5 ≥ 2 ? Yes
4 5 6
B[1,5] = max { B[1 - 1, 5], v1 + B[1 - 1, 5 – w1]}
= max { B[0, 5], 3 + B[0, 3]} i/j 0 1 2 3 4 5
= max { 0, 3 + 0]} 0 0 0 0 0 0 0
=3 1 0 0 3 3 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[2,1] , i=2 , j=1 & w2= 3 , v2 = 4
3 4 5
1 < 3 ? Yes
4 5 6

B[2, 1] = B[1,1] i/j 0 1 2 3 4 5

=0 0 0 0 0 0 0 0
1 0 0 3 3 3 3
1 2 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[2,2] , i=2 , j=2 & w2= 3 , v2 = 4
3 4 5
2 < 3 ? Yes
4 5 6

B[2, 2] = B[1,2] i/j 0 1 2 3 4 5

=3 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[2,3] , i=2 , j=3 & w2= 3 , v2 = 4
3 4 5
3<3?
4 5 6
j ≥ wi ?
3 ≥ 3 ? Yes
i/j 0 1 2 3 4 5
B[2,3] = max { B[2 - 1, 3], v2 + B[2 - 1, 3 – w2]}
= max { B[1, 3], 4 + B[1, 0]} 0 0 0 0 0 0 0
= max { 3, 4 + 0]} 1 0 0 3 3 3 3
=4 2 0 0 3
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[2,4] , i=2 , j=4 & w2= 3 , v2 = 4
3 4 5
4<3?
4 5 6
j ≥ wi ?
4 ≥ 3 ? Yes
i/j 0 1 2 3 4 5
B[2,4] = max { B[2 - 1, 4], v2 + B[2 - 1, 4 – w2]}
= max { B[1, 4], 4 + B[1, 1]} 0 0 0 0 0 0 0
= max { 3, 4 + 0]} 1 0 0 3 3 3 3
=4 2 0 0 3 4
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[2,5] , i=2 , j=5 & w2= 3 , v2 = 4
3 4 5
5<3?
4 5 6
j ≥ wi ?
5 ≥ 3 ? Yes
i/j 0 1 2 3 4 5
B[2,5] = max { B[2 - 1, 5], v2 + B[2 - 1, 5 – w2]}
= max { B[1, 5], 4 + B[1, 2]} 0 0 0 0 0 0 0
= max { 3, 4 + 3]} 1 0 0 3 3 3 3
=7 2 0 0 3 4 4
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[3,1] , i=3 , j=1 & w3= 4 , v3 = 5
3 4 5
1 < 4 ? Yes
4 5 6

B[3, 1] = B[2,1] i/j 0 1 2 3 4 5

=0 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[3,2] , i=3 , j=1 & w3= 4 , v3 = 5
3 4 5
2 < 4 ? Yes
4 5 6

B[3, 2] = B[2,2] i/j 0 1 2 3 4 5

=3 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[3,3] , i=3 , j=3 & w3= 4 , v3 = 5
3 4 5
3 < 4 ? Yes
4 5 6

B[3, 3] = B[2,3] i/j 0 1 2 3 4 5

=4 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[3,4] , i=3 , j=4 & w3= 4 , v3 = 5
3 4 5
4<4?
4 5 6
j ≥ wi ?
5 ≥ 4 ? Yes
i/j 0 1 2 3 4 5
B[3,4] = max { B[3 - 1, 4], v3 + B[3 - 1, 4 – w3]}
= max { B[2, 4], 5 + B[2, 0]} 0 0 0 0 0 0 0
= max { 4, 5 + 0]} 1 0 0 3 3 3 3
=5 2 0 0 3 4 4 7
3 0 0 3 4
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[3,5] , i=3 , j=5 & w3= 4 , v3 = 5
3 4 5
5<4?
4 5 6
j ≥ wi ?
5 ≥ 4 ? Yes
i/j 0 1 2 3 4 5
B[3,5] = max { B[3 - 1, 5], v3 + B[3 - 1, 5 – w3]}
= max { B[2, 5], 5 + B[2, 1]} 0 0 0 0 0 0 0
= max { 7, 5 + 0]} 1 0 0 3 3 3 3
=7 2 0 0 3 4 4 7
3 0 0 3 4 5
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[4,1] , i=4 , j=1 & w4= 5 , v4 = 6
3 4 5
1 < 5 ? Yes
4 5 6

B[4, 1] = B[3,1] i/j 0 1 2 3 4 5

=0 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[4,2] , i=4 , j=2 & w4= 5 , v4 = 6
3 4 5
2 < 5 ? Yes
4 5 6

B[4, 2] = B[3,2] i/j 0 1 2 3 4 5

=3 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[4,3] , i=4 , j=3 & w4= 5 , v4 = 6
3 4 5
3 < 5 ? Yes
4 5 6

B[4, 3] = B[3,3] i/j 0 1 2 3 4 5

=4 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[4,4] , i=4 , j=4 & w4= 5 , v4 = 6
3 4 5
4 < 5 ? Yes
4 5 6

B[4, 4] = B[3,4] i/j 0 1 2 3 4 5

=5 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution
wi v i 2) B[i, j] = B[i - 1, j], if j < wi
1 2 3
3) B[i, j] = max { B[i - 1, j], vi + B[i - 1, j - wi]}, otherwise(j ≥ wi)
2 3 4
j < wi ? For B[4,5] , i=4 , j=5 & w4= 5 , v4 = 6
3 4 5
5<5?
4 5 6
j ≥ wi ?
5 ≥ 5 ? Yes
i/j 0 1 2 3 4 5
B[4,5] = max { B[4 - 1, 5], v4 + B[4 - 1, 5 – w4]}
= max { B[3, 5], 6 + B[3, 0]} 0 0 0 0 0 0 0
= max { 7, 6 + 0]} 1 0 0 3 3 3 3
=7 2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
Step-2: Recursively define the value of an optimal solution

i/j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7

Total Profit

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
How to find which Items are selected?
i = n = number of Objects (Items) = 4
k = W = Capacity of Knapsack = 5
For i>0 & k>0

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k].

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi].

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=n=4
k=W=5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
i 4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=n=4
k=W=5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
B[i,k] = B[i-1,k] ? 3 0 0 3 4 5 7
B[4,5] = B[3,5] ? YES i 4 0 0 3 4 5 7
then then No item is selected move up to B[3, 5]

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k]

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi]

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=3 i/k 0 1 2 3 4 5
k=5 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
B[i,k] = B[i-1,k] ? i 3 0 0 3 4 5 7
B[3,5] = B[2,5] ? YES 4 0 0 3 4 5 7
then then No item is selected move up to B[2, 5]

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k]

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi]

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=2 wi = w2 = 3
k=5 k-wi = 5 -3 =2 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0
B[i,k] = B[i-1,k] ? 1 0 0 3 3 3 3
B[2,5] = B[1,5] ? NO
i 2 0 0 3 4 4 7
Means 3 0 0 3 4 5 7
B[i,k] ≠ B[i-1,k]
4 0 0 3 4 5 7
B[2,5] ≠ B[1,5] ? Yes
then mark 2nd item and move left to B[1, 2]

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k]

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi]

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=1 wi = w1 = 2
k=2 k-wi = 2 -2 =0 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0
B[i,k] = B[i-1,k] ? i 1 0 0 3 3 3 3
B[1,2] = B[0,2] ? NO 2 0 0 3 4 4 7
Means 3 0 0 3 4 5 7
B[i,k] ≠ B[i-1,k]
4 0 0 3 4 5 7
B[1,2] ≠ B[0,2] ? Yes
then mark 1st item and move left to B[0, 0]

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k]

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi]

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-3:Compute the value of an optimal solution, typically
k
in a bottom-up fashion.
i=0
k=0 i/k 0 1 2 3 4 5
i 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7

1) If B[i,k] = B[i-1,k] then No item is selected move up to B[i-1, k]

2) If B[i,k] ≠ B[i-1,k] then mark ith item and move left to B[i-1,k-wi]

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
Step-4: Construct an optimal solution from the computed
information. i/k 0 1 2 3 4 5
wi vi 0 0 0 0 0 0 0
1 0 0 3 3 3 3
1 2 3 2 0 0 3 4 4 7
2 3 4 3 0 0 3 4 5 7
3 4 5 4 0 0 3 4 5 7
4 5 6 Selected items are

Total Profit + =
Mr. Prayag Patel Department: IT/ICT-LJIET
 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0 0 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0 0 0 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5

Object
4 5 6 0 0 0 0 0 0 0
1
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0

Object
4 5 6
1 0
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0

Object
4 5 6
1 0 0
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0

Object
4 5 6
1 0 0 3
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0

Object
4 5 6
1 0 0 3 3
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
0 0 0 0 0 0 0

Object
4 5 6
1 0 0 3 3 3
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0 0
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0 0 3
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0 0 3 4
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0 0 3 4 4
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
3+4 =7
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3

Object
2 0 0 3 4 4 7
3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0 0
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0 0 3
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0 0 3 4
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0 0 3 4 5
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4 3+4 =7
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7

Object
3 0 0 3 4 5 7
4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0 0 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0 0 3 4

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0 0 3 4 5

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
Capacity
1 2 3
2 3 4
3+4 =7
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7

Object
4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
k
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
i 4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
k
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
i 3 0 0 3 4 5 7
4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
k
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
1 0 0 3 3 3 3
i 2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
k
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 0 0 0 0 0 0 0
i 1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
wi vi
k
1 2 3
2 3 4
3 4 5 i/k 0 1 2 3 4 5
4 5 6 i 0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7

Mr. Prayag Patel Department: IT/ICT-LJIET

 0/1 Knapsack Problem
i/k 0 1 2 3 4 5
wi vi 0 0 0 0 0 0 0
1 0 0 3 3 3 3
1 2 3 2 0 0 3 4 4 7
2 3 4 3 0 0 3 4 5 7
3 4 5 4 0 0 3 4 5 7
4 5 6 Selected items are

Total Profit + =
Mr. Prayag Patel Department: IT/ICT-LJIET
 TOPIC : LONGEST COMMON SUBSEQUENCE

Mr. Prayag Patel

 Longest Common Subsequence
▪ Subsequence
▪ A subsequence of a string S, is a set of characters that
appear in left to-right order, but not necessarily
consecutively.
▪ Example: A CTTGCG
➢ ACT is a subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪ Subsequence
▪ A subsequence of a string S, is a set of characters that
appear in left to-right order, but not necessarily
consecutively.
▪ Example: A CTTGCG
➢ ATTC is a subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪ Subsequence
▪ A subsequence of a string S, is a set of characters that
appear in left to-right order, but not necessarily
consecutively.
▪ Example: A CTTGCG
➢ T is a subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪ Subsequence
▪ A subsequence of a string S, is a set of characters that
appear in left to-right order, but not necessarily
consecutively.
▪ Example: A CTTGCG
➢ ACTTGC is a subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪ Subsequence
▪ A subsequence of a string S, is a set of characters that
appear in left to-right order, but not necessarily
consecutively.
▪ Example: A CTTGCG
➢ TTA is not a subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪Common Subsequence
▪ A common subsequence of two strings is a subsequence
that appears in both strings.
▪ Given sequences X[1..m] and Y[1..n], find a common
subsequence of both.
▪ Example: X = A B C B D A B and Y = B D C A B A
➢BCA is a common subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪Common Subsequence
▪ A common subsequence of two strings is a subsequence
that appears in both strings.
▪ Given sequences X[1..m] and Y[1..n], find a common
subsequence of both.
▪ Example: X = A B C B D A B and Y = B D C A B A
➢ BDA is a common subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪Common Subsequence
▪ A common subsequence of two strings is a subsequence
that appears in both strings.
▪ Given sequences X[1..m] and Y[1..n], find a common
subsequence of both.
▪ Example: X = A B C B D A B and Y = B D C A B A
➢ BDA is a common subsequence.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪Longest Common Subsequence
▪ A longest common subsequence is a common
subsequence of maximal length.
▪ Given sequences X[1..m] and Y[1..n], find a longest
common subsequence of both.
▪ Example: X = A B C B D A B and Y = B D C A B A
➢ BCBA is a longest common subsequence

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
▪Longest Common Subsequence
▪ A longest common subsequence is a common
subsequence of maximal length.
▪ Given sequences X[1..m] and Y[1..n], find a longest
common subsequence of both.
▪ Example: X = A B C B D A B and Y = B D C A B A
➢ BDAB is a longest common subsequence

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Steps of Longest Common Subsequence:
1) Characterize the structure of an optimal solution.
2) Recursively define the value of an optimal solution.
3) Compute the value of an optimal solution, typically in a bottom-
up fashion.
4) Construct an optimal solution from the computed information.

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-1: Characterize the structure of an optimal solution
Formula for Longest Common Subsequence :
Let A and B be sequences.
Let c[i,j] be the length of an element in LCS(Ai, Bj).

0 • if i=0 or j=0

c[i,j] = c[i-1,j-1]+1 • if i,j>0 and Ai = Bj

1
max(c[i,j-1],c[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
X = A B C B D and Y = B D C A B A
2nd Sequence
j
B D C A B A
i/j 0 1 2 3 4 5 6
i 0

1st Sequence
A 1
B 2
1 C 3
B 4
D 5

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
X = A B C B D and Y = B D C A B A
2nd Sequence
j
B D C A B A
i/j 0 1 2 3 4 5 6
i 0 0 0 0 0 0 0 0

1st Sequence
A 1 0
B 2 0
1 C 3 0
B 4 0
D 5 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
X = A B C B D and Y = B D C A B A
2nd Sequence
j
B D C A B A
i/j 0 1 2 3 4 5 6
i 0 0 0 0 0 0 0 0

1st Sequence
A 1 0 0 0 0 1 1 1
B 2 0 1 1 1 1 2 2
1 C 3 0 1 1 2 2 2 2
B 4 0 1 1 2 2 3 3
D 5 0 1 2 2 2 3 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
X = A B C B D and
nd
Y = B D C A B A
2 Sequence
j
B D C A B A
i/j 0 1 2 3 4 5 6
i 0 0 0 0 0 0 0 0

1st Sequence
A 1 0 0 0 0 1 1 1
B 2 0 1 1 1 1 2 2
1 C 3 0 1 1 2 2 2 2
B 4 0 1 1 2 2 3 3
D 5 0 1 2 2 2 3 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-4:Construct an optimal solution from the computed
information.
X = A B C B D and
nd
Y = B D C A B A
2 Sequence
j
B D C A B A
i/j 0 1 2 3 4 5 6
i 0 0 0 0 0 0 0 0

1st Sequence
A 1 0 0 0 0 1 1 1
B 2 0 1 1 1 1 2 2
1 C 3 0 1 1 2 2 2 2
B 4 0 1 1 2 2 3 3
D 5 0 1 2 2 2 3 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Example: Find any one Longest Common Subsequence
of given two strings using Dynamic Programming.
A=MLNOM
B=LMOM

1 2 3 4 5
A[ i ] M L N O M
B[ j ] L M O M

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-1: Characterize the structure of an optimal solution
Formula for Longest Common Subsequence :
Let A and B be sequences.
Let c[i,j] be the length of an element in LCS(Ai, Bj).

0 • if i=0 or j=0

c[i,j] = c[i-1,j-1]+1 • if i,j>0 and Ai = Bj

1
max(c[i,j-1],c[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
2nd Sequence
A=MLNOM j
L M O M
B=LMOM
i/j 0 1 2 3 4
i 0
M 1

1st Sequence
L 2
N 3
1
O 4
M 5

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[0,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1
L 2
N 3
O 4
M 5
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[1,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1 0
L 2
N 3
O 4
M 5
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[2,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1 0
L 2 0
N 3
O 4
M 5
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[3,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1 0
L 2 0
N 3 0
O 4
M 5
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[4,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[i,0] = C[5,0] = 0 j L M O M
i i/j 0 1 2 3 4
0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5 0
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[0,j] = C[0,1] = 0 j L M O M
i i/j 0 1 2 3 4
0 0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5 0
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[0,j] = C[0,2] = 0 j L M O M
i i/j 0 1 2 3 4
0 0 0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5 0
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[0,j] = C[0,3] = 0 j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5 0
1

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.

C[0,j] = C[0,4] = 0 j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0 0
M 1 0
L 2 0
N 3 0
O 4 0
M 5 1 0

c[i,j] = 0 • if i=0 or j=0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[1,1] , i=1 , j=1 &
A1= M , B1 = L M 1 0
L 2 0
A1= B1 ?
M = L ? No N 3 0
C[1, 1] = max(C[1,0], C[0,1]) O 4 0
= max(0, 0) M 5 0
=0 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[1,2] , i=1 , j=2 &
A1= M , B2 = M M 1 0 0
L 2 0
A1= B2 ?
M = M ? YES N 3 0
C[1, 2] = C[0,1] + 1 O 4 0
=0+1 M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[1,3] , i=1 , j=3 &
A1= M , B3 = O M 1 0 0 1
L 2 0
A1= B3 ?
M = O ? NO N 3 0
C[1, 3] = max(C[1,2], C[0,3]) O 4 0
= max(1, 0) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[1,4] , i=1 , j=4 &
A1= M , B4 = M M 1 0 0 1 1
L 2 0
A1= B4 ?
M = M ? YES N 3 0
C[1, 4] = C[0,3] + 1 O 4 0
=0+1 M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[2,1] , i=2 , j=1 &
A2= L , B1 = L M 1 0 0 1 1 1
L 2 0
A2= B1 ?
L = L ? YES N 3 0
C[2, 1] = C[1,0] + 1 O 4 0
=0+1 M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[2,2] , i=2 , j=2 &
A2= L , B2 = M M 1 0 0 1 1 1
L 2 0 1
A2= B2 ?
L = M ? NO N 3 0
C[2, 2] = max(C[2,1], C[1,2]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[2,3] , i=2 , j=3 &
A2= L , B3 = O M 1 0 0 1 1 1
L 2 0 1 1
A2= B3 ?
L = 0 ? NO N 3 0
C[2, 3] = max(C[2,2], C[1,3]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[2,4] , i=2 , j=4 &
A2= L , B4 = M M 1 0 0 1 1 1
L 2 0 1 1 1
A2= B4 ?
L = M ? NO N 3 0
C[2, 4] = max(C[2,3], C[1,4]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[3,1] , i=3 , j=1 &
A3= N , B1 = L M 1 0 0 1 1 1
L 2 0 1 1 1 1
A3= B1 ?
N = L ? NO N 3 0
C[3, 1] = max(C[3,0], C[2,1]) O 4 0
= max(0, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[3,2] , i=3 , j=2 &
A3= N , B2 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A3= B2 ?
N = M ? NO N 3 0 1
C[3, 2] = max(C[3,1], C[2,2]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[3,3] , i=3 , j=3 &
A3= N , B3 = O M 1 0 0 1 1 1
L 2 0 1 1 1 1
A3= B3 ?
N = O ? NO N 3 0 1 1
C[3, 3] = max(C[3,2], C[2,3]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[3,4] , i=3 , j=4 &
A3= N , B4 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A3= B4 ?
N = M ? NO N 3 0 1 1 1
C[3, 4] = max(C[3,3], C[2,4]) O 4 0
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[4,1] , i=4 , j=1 &
A4= O , B1 = L M 1 0 0 1 1 1
L 2 0 1 1 1 1
A4= B1 ?
O = L ? NO N 3 0 1 1 1 1
C[4, 1] = max(C[4,0], C[3,1]) O 4 0
= max(0, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[4,2] , i=4 , j=2 &
A4= O , B2 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A4= B2 ?
O = M ? NO N 3 0 1 1 1 1
C[4, 2] = max(C[4,1], C[3,2]) O 4 0 1
= max(1, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[4,3] , i=4 , j=3 &
A4= O , B3 = O M 1 0 0 1 1 1
L 2 0 1 1 1 1
A4= B3 ?
O = O ? YES N 3 0 1 1 1 1
C[4, 3] = C[3,2] + 1 O 4 0 1 1
=1+1 M 5 0
=2 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[4,4] , i=4 , j=4 &
A4= O , B4 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A4= B4 ?
O = M ? NO N 3 0 1 1 1 1
C[4, 4] = max(C[4,3], C[3,4]) O 4 0 1 1 2
= max(2, 1) M 5 0
=2 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[5,1] , i=5 , j=1 &
A5= M , B1 = L M 1 0 0 1 1 1
L 2 0 1 1 1 1
A5 = B1 ?
M = L ? NO N 3 0 1 1 1 1
C[5, 1] = max(C[5,0], C[4,1]) O 4 0 1 1 2 2
= max(0, 1) M 5 0
=1 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[5,2] , i=5 , j=2 &
A5= M , B2 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A5 = B2 ?
M = M ? YES N 3 0 1 1 1 1
C[5, 2] = C[4,1] + 1 O 4 0 1 1 2 2
=1+1 M 5 0 1
=2 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[5,3] , i=5 , j=3 &
A5= M , B3 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A5 = B3 ?
M = O ? NO N 3 0 1 1 1 1
C[5, 3] = max(C[5,2], C[4,3]) O 4 0 1 1 2 2
= max(2, 2) M 5 0 1 2
=2 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
For C[5,4] , i=5 , j=4 &
A5= M , B4 = M M 1 0 0 1 1 1
L 2 0 1 1 1 1
A5 = B4 ?
M = M ? YES N 3 0 1 1 1 1
C[5, 4] = C[4,3] + 1 O 4 0 1 1 2 2
=2+1 M 5 0 1 2 2
=3 1
C[i,j] = C[i-1,j-1]+1 • if i,j>0 and Ai = Bj
C[i,j] = max(C[i,j-1],C[i-1,j]) • if i,j>0 and Ai ≠ Bj

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-2: Recursively define the value of an optimal solution.
1 2 3 4 5
A[ i ] M L N O M j L M O M
B[ j ] L M O M i i/j 0 1 2 3 4
0 0 0 0 0 0
M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1

Longest Common Subsequence = 3

Mr. Prayag Patel Department: IT/ICT-LJIET
 Longest Common Subsequence
How to find which characters are Selected?
1 2 3 4 5
i = length of first list = 5
A[ i ] M L N O M j = length of first list = 4
B[ j ] L M O M

Start Backtracking form C[5,4]

1) If C[i,j] = Cross arrow then select ith character & move to the
block C[i-1, j-1]
2) If C[i,j]1 = left arrow then move to the block C[i , j-1]
3) If C[i,j] = Up arrow then move to the block C[i-1 , j]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
j L M O M
Start Backtracking form C[5,4] i/j 0 1 2 3 4
i
0 0 0 0 0 0
1) If C[i,j] = Cross arrow then
M 1 0 0 1 1 1
select ith character & move L 2 0 1 1 1 1
to the block C[i-1, j-1] N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1
In C[5,4] there are Cross arrow then select 5th character
move to the block C[4, 3]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
Consider C[4, 3] j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0 0
1) If C[i,j] = Cross arrow then
M 1 0 0 1 1 1
select ith character & move L 2 0 1 1 1 1
to the block C[i-1, j-1] N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1
In C[4,3] there are Cross arrow then select 4th character &
move to the block C[3, 2]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
Consider C[3, 2] j L M O M
i i/j 0 1 2 3 4
3) If C[i,j] = Up arrow then 0 0 0 0 0 0
move to the block C[i-1 , j] M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1

In C[3,2] there are Up arrow then move to the block C[2, 2]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
Consider C[2, 2] j L M O M
i i/j 0 1 2 3 4
3) If C[i,j] = Up arrow then 0 0 0 0 0 0
move to the block C[i-1 , j] M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1

In C[2,2] there are Up arrow then move to the block C[1, 2]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
Consider C[1, 2] j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0 0
M 1 0 0 1 1 1
1) If C[i,j] = Cross arrow then L 2 0 1 1 1 1
select ith character & move N 3 0 1 1 1 1
to the block C[i-1, j-1] O 4 0 1 1 2 2
M 5 0 1 2 2 3
1
In C[1,2] there are Cross arrow then select 1st character &
move to the block C[0, 1]

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
Consider C[0, 1] j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0 0
M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-4: Construct an optimal solution from the computed
information.
j L M O M
i i/j 0 1 2 3 4
0 0 0 0 0 0
M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3
1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Longest Common Subsequence
Step-4: Construct an optimal solution from the computed
information.
1 2 3 4 5 j L M O M
A[ i ] M L N1 O M i i/j 0 1 2 3 4
B[ j ] L M O M 0 0 0 0 0 0
M 1 0 0 1 1 1
L 2 0 1 1 1 1
N 3 0 1 1 1 1
O 4 0 1 1 2 2
M 5 0 1 2 2 3

Longest Common Subsequence = M O M

Mr. Prayag Patel Department: IT/ICT-LJIET
 Longest Common Subsequence
Example: Find any one Longest Common Subsequence of
given two strings using Dynamic Programming.

1) A = X Y Z Y T X Y & B = Y T Z X Y X
2) A = a b b a c d c b a & B = b c d b b c a a c
3) A = 1 0 0 1 0 1 1 0 1 1 0 1 & B = 0 1 1 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 TOPIC : MATRIX CHAIN MULTIPLICATION

Assistant Professor
IT-ICT Department, LJIET
[email protected]
Mr. Prayag Patel
 Matrix Chain Multiplication
▪ Problem Definition
➢ Input:
― N matrices A1, A2, A3,………… An of dimensions P1 x P2 ,
P2 x P3 , P3 x P4 ,………… Pn x Pn+1 respectively.
➢ Goal:
― To compute the matrix product A1 x A2 x A3 x…………x An
➢ Problem:
― In what order should A1, A2, A3,………… An be multiplied
so that it would take the minimum number of
computations to derive the product.

A B C
PxQ QxR PxR
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
▪ Problem Definition

A B C
2x3 3x4 2x4
PxQ QxR PxR

➢ The total number of scaler multiplication required will be:

=PxQxR
=2x3x4
= 24

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
▪ Problem Definition
▪ Given a sequence of matrices, find the most efficient way to
multiply these matrices together.
▪ Here to decide in which order to perform the multiplications.
▪ We have many options to multiply a chain of matrices.
▪ No matter how we parenthesize the product, the result will be
the same. But Number of multiplication is different

▪ Example: Three matrices A, B and C

➢ Possible multiplication are:
1) (AB)C
2) A(BC)

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
▪ Problem Definition
▪ Example: Three matrices A, B and C

A B C
5x4 4x6 6x2

X C
5x6 6x2

➢ Possible multiplication are:

1) (AB)C = (5 * 4 * 6) + (5 * 6 * 2)
= 120 + 60
= 180
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
▪ Problem Definition
▪ Example: Three matrices A, B and C

A B C
5x4 4x6 6x2

A X
5x4 4x2
➢ Possible multiplication are:
2) A (BC) = (4 * 6 * 2) + (5 * 4 * 2)
= 48 + 40
= 88
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
▪ Problem Definition
▪ Example: Three matrices A, B and C

A B C
5x4 4x6 6x2

➢ Possible multiplication are:

1) (AB)C = 180 Total number of scaler
2) A (BC) = 88 multiplication required

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Example: Consider the case multiplying these 4 matrices

A B C D
2x4 4x2 2x3 3x1
➢ Possible multiplication are:

multiplication required
Total number of scaler
1) A(B(CD)) = (2*3*1) + (4*2*1) + (2*4*1) = 6 + 8 + 8 = 22
2) A((BC)D) = (4*2*3) + (4*3*1) + (2*4*1) = 24 + 12 + 8 = 44
3) (AB)(CD) = (2*4*2) + (2*3*1) + (2*2*1) = 16 + 6 + 4 = 26
4) ((AB)C)D = (2*4*2) + (2*2*3) + (2*3*1) = 16 + 12 + 6 = 34
5) (A(BC))D = (4*2*3) + (2*4*3) + (2*3*1) = 24 + 24 + 6 = 54

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Example: Find an optimal parenthesizing of matrix
chain product For the given matrices using Dynamic
programming.
Matrix Dimension
A1 5x4
A2 4x6
A3 6x2
A4 2x7

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-1: Characterize the structure of an optimal solution
Formula for Matrix Chain

0 • if i = j

m[i,j] =
min(m[i,k]+m[k+1,j]+ pi-1pkpj) • if i < j
With i≤ k ≤j-1
1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution

Matrix Dimension
A1 5x4
Here consider
A2 4x6
p0 = 5
A3 6x2 p1 = 4
A4 2x7 p2 = 6
p3 = 2
p4 = 7
1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
5x4
4 1 i
A1 j 3 2
A2 4x6
2 3
A3 6x2
1 4
A4 2x7

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
5x4
4 1 i
A1 j 3 2
A2 4x6
2 3
A3 6x2
1 4
A4 2x7 0

m[1,1] = 0

m[i, j] = 0 • if i = j
0
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
j 4 1
A1 5x4
3 2 i
A2 4x6
2 3
A3 6x2
1 4
A4 2x7 0 0

m[2,2] = 0

m[i, j] = 0 • if i = j
0
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
5x4
4 1 i
A1
j 3 2
A2 4x6
2 3
A3 6x2
1 4
A4 2x7 0 0 0

m[3, 3] = 0

m[i, j] = 0 • if i = j
0
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
5x4
4 1 i
A1
j 3 2
A2 4x6
2 3
A3 6x2
1 4
A4 2x7 0 0 0 0

m[4, 4] = 0

m[i, j] = 0 • if i = j
0
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
A1 5x4 p0 = 5
j 4 1 i
p1 = 4 3 2
A2 4x6
p2 = 6 3
A3 6x2 2
p3 = 2 1 4
A4 2x7 p4 = 7
0 0 0 0
m[1, 2] = min(m[1,1]+m[2,2]+ p0p1p2) i=1&j=2
= min(0 + 0 + 5*4*6) i ≤ k ≤ j-1
= 120 1 ≤ k ≤ 1 So k = 1
m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj) • if i < j With i≤ k ≤j-1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
p0 = 5 4 1
p1 = 4 j 3 2 i
p2 = 6 3
2
p3 = 2 1 120 4
p4 = 7
0 0 0 0
m[2, 3] = min(m[2,2]+m[3,3]+ p1p2p3) i=2&j=3
= min(0 + 0 + 4*6*2) i ≤ k ≤ j-1
= 48 2 ≤ k ≤ 2 So k = 2
m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj) • if i < j With i≤ k ≤j-1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
p0 = 5 4 1 i
p1 = 4 j 3 2
p2 = 6 3
2
p3 = 2 1 120 48 4
p4 = 7
0 0 0 0
m[3, 4] = min(m[3,3]+m[4,4]+ p2p3p4) i=3&j=4
= min(0 + 0 + 6*2*7) i ≤ k ≤ j-1
= 84 3 ≤ k ≤ 3 So k = 3
m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj) • if i < j With i≤ k ≤j-1

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
p0 = 5 4 1
p1 = 4 i=1&j=3
3 2 i
i ≤ k ≤ j-1
p2 = 6
1≤k≤2 j2 3
p3 = 2 1 120 48 84 4
So k = 1, 2
p4 = 7 0 0 0 0
m[1, 3] = min m[1,1]+m[2,3]+ p0p1p3) k=1
m[1,2]+m[3,3]+ p0p2p3) k=2
= min 0 + 48 + 5*4*2 k=1
120 + 0 + 5*6*2 k=2
88 k=1 M[1,3] = 88 k=1
= min
180 k=2 m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj)

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
p0 = 5 4 1
i=2&j=4 j 3
p1 = 4 2 i
i ≤ k ≤ j-1
p2 = 6 2 88 3
2≤k≤3
p3 = 2 1 120 48 84 4
So k = 2, 3
p4 = 7 0 0 0 0
m[2, 4] = min m[2,2]+m[3,4]+ p1p2p4) k=2
m[2,3]+m[4,4]+ p1p3p4) k=3
= min 0 + 84 + 4*6*7 k=2
48 + 0 + 4*2*7 k=3
252 k=2 M[2,4] = 104 k=3
= min
104 k=3 m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj)

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
p0 = 5 4 1
p1 = 4 i=1&j=4 j 3 2 i
i ≤ k ≤ j-1
p2 = 6
1≤k≤3 2 88 104 3
p3 = 2 1 120 48 84 4
So k = 1, 2, 3
p4 = 7 0 0 0 0
m[1,1]+m[2,4]+ p0p1p4) k=1
m[1, 4] = min m[1,2]+m[3,4]+ p0p2p4) k=2 244 k=1
m[1,3]+m[4,4]+ p0p3p4) k = 3 = min 414 k=2
158 k=3
0 + 104 + 5*4*7 k=1
= min 120 + 84 + 5*6*7 k=2 M[1,4] = 158 k=3
88 + 0 + 5*2*7 k=3
m[i, j] = min(m[i,k]+m[k+1,j]+ pi-1pkpj)

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-2: Recursively define the value of an optimal solution
Matrix Dimension
A1 5x4 1
4
A2 4x6 j 3 158 2 i
A3 6x2 2 88 104 3
A4 2x7 1 120 48 84 4
0 0 0 0

Total number of scaler multiplication required = 158

Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
How to find the optimal parenthesization?
Matrix Dimension
A1 5x4
A1 A2 A3 A4
A2 4x6
A3 6x2 n=4
A4 2x7

A1.........An=1 A1…………As[1,n] * As[1,n]+1 …………An

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i j 3 2 i
2 88 104 3 2 3
1 120 48 84 4
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i j 3 2 i
2 88 104 3 2 3
1 120 48 84 4 1
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i
j 3 2 i
2 88 104 3 2 3
1 120 48 84 4 1 2
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i i
j 3 2
2 88 104 3 2 3
1 120 48 84 4 1 2 3
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i i
j 3 2
2 88 104 3 2 1 3
1 120 48 84 4 1 2 3
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i j 3 2 i
2 88 104 3 2 1 3 3
1 120 48 84 4 1 2 3
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-3: Compute the value of an optimal solution, typically
in a bottom-up fashion.
m S
4 1 4 1
j 3 158 2 i j 3 i
3 2
2 88 104 3 2 1 3 3
1 120 48 84 4 1 2 3
0 0 0 0

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-4: Construct an optimal solution from the computed
information.
Matrix Dimension S
A1 5x4 4 1 i
A1 A2 A3 A4 j 3 3 2
A2 4x6
2 1 3 3
A3 6x2 n=4 1 2 3
A4 2x7

A1.........An= A1…………As[1,n] * As[1,n]+1 …………An

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-4: Construct an optimal solution from the computed
information.
A1.........An= A1…………As[1,n] * As[1,n]+1 …………An
A1 A2 A3 A4 n=4
A1.....A4= A1…..As[1,4] * As[1,4]+1 …..A4
= A1…..A3 * A4 …..A4 S
4 1 i
A1 A2 A3 A4 j 3 3 2
2 1 3 3
1 2 3

Mr. Prayag Patel Department: IT/ICT-LJIET

 Matrix Chain Multiplication
Step-4: Construct an optimal solution from the computed
information. A1.........An= A1…………As[1,n] * As[1,n]+1 …………An
A1 A2 A3 A4
n=3
A1.....A3= A1…..As[1,3] * As[1,3]+1 …..A3 S
= A1…..A1 * A2…..A3 j4 1
i
3 3 2
A1 A2 A3 2 1 3 3
1 2 3
A1 A2 A3 A4
Mr. Prayag Patel Department: IT/ICT-LJIET
 Matrix Chain Multiplication
Step-4: Construct an optimal solution from the computed
information. Matrix Dimension
A1 5x4
Y A2 4x6
X 5x2 X A3 6x2
4x2 4x2
A4 2x7
A1 A2 A3 A4
Y
5x2
Total number of scaler
multiplication required = (4*6*2) + (5*4*2) + (5*2*7)
= 48 + 40 + 70 = 158

Mr. Prayag Patel Department: IT/ICT-LJIET

 THANK YOU FOR
WATCHING!

Mr. Prayag Patel [email protected]