Design and Analysis of Algorithms

(Dynamic Programming - LCS)

Dr. D. P. Acharjya
Professor, SCOPE
Office: SJT Annex 201E
Email: [email protected]

2/10/2024 Dr. D. P. Acharjya 1

Dynamic Programming
 Dynamic programming is a mathematical optimization
method.
 It was developed by Richard Bellman in the 1950s.
 It simplify a complicated problem by breaking it down
into simpler sub-problems in a recursive manner.
 In computer science, if a problem can be solved
optimally by breaking it into sub-problems and then
recursively finding the optimal solutions to the sub-
problems, then it is said to have optimal substructure.
 In dynamic programming, sub-problems are nested
recursively inside larger problems.
2/10/2024 Dr. D. P. Acharjya 2
Basic Problems
 0/1 Knapsack Problem (Syllabus)
 Travelling Salesman Problem (Syllabus)
 Multistage Graph
 String Editing
 Longest Common Subsequence (Syllabus)
 Matrix Chain Multiplication (Syllabus)
 Flow Shop Scheduling
 Assembly Line Scheduling (Syllabus)
2/10/2024 Dr. D. P. Acharjya 3
Subsequence and Substring
 A subsequence of a given sequence is a sequence that can
be derived from the given sequence by deleting some or no
elements without changing the order of the remaining
elements.
 For example, <A,B,D> is a subsequence of
<A,B,C,D,E,F>
 If the sequence is consecutive, then it is also a subsequence
but, generally called as substring.
 For example, <B,C,D> is a subsequence of
<A,B,C,D,E,F> but it is called as substring.

2/10/2024 Dr. D. P. Acharjya 4

Longest Common Subsequence
 Given two sequences X and Y, a sequence Z is said to be a
common subsequence of X and Y, if Z is a subsequence of
both X and Y.
 Finding a subsequence of maximum length is called
longest common subsequence.
 For example, Let X = <A,C,B,D,E,G,C,E,D,B,G>
 and Let Y = <B,E,G,J,C,F,E,K,B>
 Z = <B,E,E> is a common subsequence of X and Y.
 Z = <B,E,E,B> is a longest common subsequence of X
and Y.
 Applications are DNA, RNA, Protein subsequence
2/10/2024 Dr. D. P. Acharjya 5
Properties of LCS
 First Property
 If the last symbol of both the strings are equal, like LCS(XA,YA),
then it reduces to a sub problem of LCS(X,Y) concatenated with
A.
 LCS(XA,YA) = LCS(X,Y)A, for all strings X, Y
 For example: LCS(BANANA, ATANA)
 = LCS(BANAN, ATAN)A
 = LCS(BANA, ATA)NA
 = LCS(BAN, AT)ANA
 Second property
 If the last symbol of both the strings are not equal, like
LCS(XA,YB), then it reduces to two sub problem of LCS(XA,Y)
and LCS(X, YB)
2/10/2024 Dr. D. P. Acharjya 6
Continued …
 For example: LCS(ABCDEFG, BCDGK)
 It leads to two sub-problems LCS(ABCDEFG, BCDG)
 and LCS(ABCDEF, BCDGK)
 If both happened to be of equal length, one of them could be
chosen arbitrarily.
 Mathematical Function

2/10/2024 Dr. D. P. Acharjya 7

 Numerical Illustration
Consider R=<FZK> ={R1,R2,R3} LCS (R0,C0) = 
C = <ZFKZK> = {C1,C2,C3,C4,C5} LCS (R0,C1) = 
C0 C1 C2 C3 C4 C5 LCS (R0,C2) = 
 Z F K Z K LCS (R0,C3) = 
R0        LCS (R0,C4) = 
LCS (R0,C5) = 
R1 F 
LCS (R1,C0) = 
R2 Z 
LCS (R2,C0) = 
R3 K 
LCS (R3,C0) = 
2/10/2024 Dr. D. P. Acharjya 8
 Continued …
LCS (R1,C1) = LCS (F, Z); FZ
LCS (R1,C1) = Max{LCS (R1,C0)= , LCS (R0,C1) = } = 

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F  

R2 Z 

R3 K 

2/10/2024 Dr. D. P. Acharjya 9

 Continued …
LCS (R1,C2) = LCS (F, F); F=F
LCS (R1,C2) = LCS (R0,C1)F= F=F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F

R2 Z 

R3 K 

2/10/2024 Dr. D. P. Acharjya 10

 Continued …
LCS (R1,C3) = LCS (F, K); FK
LCS (R1,C3) = Max{LCS (R1,C2)= F, LCS (R0,C3) = } = F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F

R2 Z 

R3 K 

2/10/2024 Dr. D. P. Acharjya 11

 Continued …
LCS (R1,C4) = LCS (F, Z); FZ
LCS (R1,C4) = Max{LCS(R1,C3)=F, LCS(R0,C4) = } = F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F

R2 Z 

R3 K 

2/10/2024 Dr. D. P. Acharjya 12

 Continued …
LCS (R1,C5) = LCS (F, K); FK
LCS (R1,C5) = Max{LCS(R1,C4)=F, LCS(R0,C5) = } = F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z 

R3 K 

2/10/2024 Dr. D. P. Acharjya 13

 Continued …
LCS (R2,C1) = LCS (Z, Z); Z=Z
LCS (R2,C1) = LCS(R1,C0)Z = Z = Z

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z

R3 K 

2/10/2024 Dr. D. P. Acharjya 14

 Continued …
LCS (R2,C2) = LCS (Z, F); Z  F
LCS (R2,C2) =Max{LCS (R2,C1) = Z, LCS (R1,C2)=F}= Z & F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F

R3 K 

2/10/2024 Dr. D. P. Acharjya 15

 Continued …
LCS (R2,C3) = LCS (Z, K); Z  K
LCS (R2,C3) =Max{LCS (R2,C2) = Z&F, LCS (R1,C3)=F}= Z & F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F

R3 K 

2/10/2024 Dr. D. P. Acharjya 16

 Continued …
LCS (R2,C4) = LCS (Z, Z); Z = Z
LCS (R2,C4) =LCS (R1,C3)Z=FZ

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ

R3 K 

2/10/2024 Dr. D. P. Acharjya 17

 Continued …
LCS (R2,C5) = LCS (Z, K); Z  K
LCS (R2,C5) =Max{LCS (R2,C4)=FZ, LCS(R1,C5)=F}=FZ

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K 

2/10/2024 Dr. D. P. Acharjya 18

 Continued …
LCS (R3,C1) = LCS (K, Z); K  Z
LCS (R3,C1) =Max{LCS (R2,C1)=Z, LCS(R3,C0)=}=Z

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K  Z

2/10/2024 Dr. D. P. Acharjya 19

 Continued …
LCS (R3,C2) = LCS (K, F); K  F
LCS (R3,C2) =Max{LCS (R2,C2)=Z&F, LCS(R3,C1)=Z}=Z&F

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K  Z Z&F

2/10/2024 Dr. D. P. Acharjya 20

 Continued …
LCS (R3,C3) = LCS (K, K); K = K
LCS (R3,C3) =LCS (R2,C2)K={Z&F}K =ZK & FK

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K  Z Z&F ZK & FK

2/10/2024 Dr. D. P. Acharjya 21

 Continued …
LCS (R3,C4) = LCS (K, Z); K  Z
LCS (R3,C4) =Max{LCS (R2,C4)=FZ, LCS (R3,C3)=ZK & FK}= FZ, ZK & FK

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K  Z Z&F ZK & FK FZ, ZK & FK

2/10/2024 Dr. D. P. Acharjya 22

 Continued …
LCS (R3,C5) = LCS (K, K); K =K
LCS (R3,C5) = LCS (R2,C4)K = FZK

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0       

R1 F   F F F F

R2 Z  Z Z&F Z&F FZ FZ

R3 K  Z Z&F ZK & FK FZ, ZK & FK FZK

2/10/2024 Dr. D. P. Acharjya 23

Trace-back Approach
 LCS computation of a row requires LCS table
values of the current and previous row.
 For long sequences, these sequences can get
numerous and long.
 It leads to storage space.
 Storage space can be saved by saving the length of
the subsequence and the direction of the arrows
rather than the actual subsequences.
 It will help to find the length and by bottom up
approach we will get the subsequence.
2/10/2024 Dr. D. P. Acharjya 24
 Illustration
Consider the problem discussed earlier.

C0 C1 C2 C3 C4 C5
 Z F K Z K

R0  0 0 0 0 0 0

R1 F 0 0 1 1 1 1

R2 Z 0 1 1 1 2 2

R3 K 0 1 1 2 2 3

2/10/2024 Dr. D. P. Acharjya 25

Illustration
 LCS computation of a row requires LCS table
values of the current and previous row.
 For long sequences, these sequences can get
numerous and long.
 It leads to storage space.
 Storage space can be saved by saving the length of
the subsequence and the direction of the arrows
rather than the actual subsequences.
 It will help to find the length and by bottom up
approach we will get the subsequence.
2/10/2024 Dr. D. P. Acharjya 26
LCS Length Algorithm
1. LCS length (X[1, …, m], Y[1, …, n])
2. C = Array(0 … m, 0 … n)
3. For i = 0 to m
4. C [i, 0] = 0
5. For j = 0 to n
6. C [0, j] = 0

2/10/2024 Dr. D. P. Acharjya 27

LCS Length Algorithm
7. For i = 1 to m
8. For j = 0 to n
9. if X[i] = Y[j]
10. C[i, j] = C[i-1, j-1] + 1
11. Else
12. C[i, j] = Max (C [i, j-1], C [i-1, j] )
13.Return C[m, n]
2/10/2024 Dr. D. P. Acharjya 28