Design and Analysis of Algorithms
(Dynamic Programming - LCS)
Dr. D. P. Acharjya
Professor, SCOPE
Office: SJT Annex 201E
Email: [email protected]
2/10/2024 Dr. D. P. Acharjya 1
Dynamic Programming
Dynamic programming is a mathematical optimization
method.
It was developed by Richard Bellman in the 1950s.
It simplify a complicated problem by breaking it down
into simpler sub-problems in a recursive manner.
In computer science, if a problem can be solved
optimally by breaking it into sub-problems and then
recursively finding the optimal solutions to the sub-
problems, then it is said to have optimal substructure.
In dynamic programming, sub-problems are nested
recursively inside larger problems.
2/10/2024 Dr. D. P. Acharjya 2
Basic Problems
0/1 Knapsack Problem (Syllabus)
Travelling Salesman Problem (Syllabus)
Multistage Graph
String Editing
Longest Common Subsequence (Syllabus)
Matrix Chain Multiplication (Syllabus)
Flow Shop Scheduling
Assembly Line Scheduling (Syllabus)
2/10/2024 Dr. D. P. Acharjya 3
Subsequence and Substring
A subsequence of a given sequence is a sequence that can
be derived from the given sequence by deleting some or no
elements without changing the order of the remaining
elements.
For example, <A,B,D> is a subsequence of
<A,B,C,D,E,F>
If the sequence is consecutive, then it is also a subsequence
but, generally called as substring.
For example, <B,C,D> is a subsequence of
<A,B,C,D,E,F> but it is called as substring.
2/10/2024 Dr. D. P. Acharjya 4
Longest Common Subsequence
Given two sequences X and Y, a sequence Z is said to be a
common subsequence of X and Y, if Z is a subsequence of
both X and Y.
Finding a subsequence of maximum length is called
longest common subsequence.
For example, Let X = <A,C,B,D,E,G,C,E,D,B,G>
and Let Y = <B,E,G,J,C,F,E,K,B>
Z = <B,E,E> is a common subsequence of X and Y.
Z = <B,E,E,B> is a longest common subsequence of X
and Y.
Applications are DNA, RNA, Protein subsequence
2/10/2024 Dr. D. P. Acharjya 5
Properties of LCS
First Property
If the last symbol of both the strings are equal, like LCS(XA,YA),
then it reduces to a sub problem of LCS(X,Y) concatenated with
A.
LCS(XA,YA) = LCS(X,Y)A, for all strings X, Y
For example: LCS(BANANA, ATANA)
= LCS(BANAN, ATAN)A
= LCS(BANA, ATA)NA
= LCS(BAN, AT)ANA
Second property
If the last symbol of both the strings are not equal, like
LCS(XA,YB), then it reduces to two sub problem of LCS(XA,Y)
and LCS(X, YB)
2/10/2024 Dr. D. P. Acharjya 6
Continued …
For example: LCS(ABCDEFG, BCDGK)
It leads to two sub-problems LCS(ABCDEFG, BCDG)
and LCS(ABCDEF, BCDGK)
If both happened to be of equal length, one of them could be
chosen arbitrarily.
Mathematical Function
2/10/2024 Dr. D. P. Acharjya 7
Numerical Illustration
Consider R=<FZK> ={R1,R2,R3} LCS (R0,C0) =
C = <ZFKZK> = {C1,C2,C3,C4,C5} LCS (R0,C1) =
C0 C1 C2 C3 C4 C5 LCS (R0,C2) =
Z F K Z K LCS (R0,C3) =
R0 LCS (R0,C4) =
LCS (R0,C5) =
R1 F
LCS (R1,C0) =
R2 Z
LCS (R2,C0) =
R3 K
LCS (R3,C0) =
2/10/2024 Dr. D. P. Acharjya 8
Continued …
LCS (R1,C1) = LCS (F, Z); FZ
LCS (R1,C1) = Max{LCS (R1,C0)= , LCS (R0,C1) = } =
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F
R2 Z
R3 K
2/10/2024 Dr. D. P. Acharjya 9
Continued …
LCS (R1,C2) = LCS (F, F); F=F
LCS (R1,C2) = LCS (R0,C1)F= F=F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F
R2 Z
R3 K
2/10/2024 Dr. D. P. Acharjya 10
Continued …
LCS (R1,C3) = LCS (F, K); FK
LCS (R1,C3) = Max{LCS (R1,C2)= F, LCS (R0,C3) = } = F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F
R2 Z
R3 K
2/10/2024 Dr. D. P. Acharjya 11
Continued …
LCS (R1,C4) = LCS (F, Z); FZ
LCS (R1,C4) = Max{LCS(R1,C3)=F, LCS(R0,C4) = } = F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F
R2 Z
R3 K
2/10/2024 Dr. D. P. Acharjya 12
Continued …
LCS (R1,C5) = LCS (F, K); FK
LCS (R1,C5) = Max{LCS(R1,C4)=F, LCS(R0,C5) = } = F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z
R3 K
2/10/2024 Dr. D. P. Acharjya 13
Continued …
LCS (R2,C1) = LCS (Z, Z); Z=Z
LCS (R2,C1) = LCS(R1,C0)Z = Z = Z
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z
R3 K
2/10/2024 Dr. D. P. Acharjya 14
Continued …
LCS (R2,C2) = LCS (Z, F); Z F
LCS (R2,C2) =Max{LCS (R2,C1) = Z, LCS (R1,C2)=F}= Z & F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F
R3 K
2/10/2024 Dr. D. P. Acharjya 15
Continued …
LCS (R2,C3) = LCS (Z, K); Z K
LCS (R2,C3) =Max{LCS (R2,C2) = Z&F, LCS (R1,C3)=F}= Z & F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F
R3 K
2/10/2024 Dr. D. P. Acharjya 16
Continued …
LCS (R2,C4) = LCS (Z, Z); Z = Z
LCS (R2,C4) =LCS (R1,C3)Z=FZ
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ
R3 K
2/10/2024 Dr. D. P. Acharjya 17
Continued …
LCS (R2,C5) = LCS (Z, K); Z K
LCS (R2,C5) =Max{LCS (R2,C4)=FZ, LCS(R1,C5)=F}=FZ
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K
2/10/2024 Dr. D. P. Acharjya 18
Continued …
LCS (R3,C1) = LCS (K, Z); K Z
LCS (R3,C1) =Max{LCS (R2,C1)=Z, LCS(R3,C0)=}=Z
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K Z
2/10/2024 Dr. D. P. Acharjya 19
Continued …
LCS (R3,C2) = LCS (K, F); K F
LCS (R3,C2) =Max{LCS (R2,C2)=Z&F, LCS(R3,C1)=Z}=Z&F
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K Z Z&F
2/10/2024 Dr. D. P. Acharjya 20
Continued …
LCS (R3,C3) = LCS (K, K); K = K
LCS (R3,C3) =LCS (R2,C2)K={Z&F}K =ZK & FK
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K Z Z&F ZK & FK
2/10/2024 Dr. D. P. Acharjya 21
Continued …
LCS (R3,C4) = LCS (K, Z); K Z
LCS (R3,C4) =Max{LCS (R2,C4)=FZ, LCS (R3,C3)=ZK & FK}= FZ, ZK & FK
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K Z Z&F ZK & FK FZ, ZK & FK
2/10/2024 Dr. D. P. Acharjya 22
Continued …
LCS (R3,C5) = LCS (K, K); K =K
LCS (R3,C5) = LCS (R2,C4)K = FZK
C0 C1 C2 C3 C4 C5
Z F K Z K
R0
R1 F F F F F
R2 Z Z Z&F Z&F FZ FZ
R3 K Z Z&F ZK & FK FZ, ZK & FK FZK
2/10/2024 Dr. D. P. Acharjya 23
Trace-back Approach
LCS computation of a row requires LCS table
values of the current and previous row.
For long sequences, these sequences can get
numerous and long.
It leads to storage space.
Storage space can be saved by saving the length of
the subsequence and the direction of the arrows
rather than the actual subsequences.
It will help to find the length and by bottom up
approach we will get the subsequence.
2/10/2024 Dr. D. P. Acharjya 24
Illustration
Consider the problem discussed earlier.
C0 C1 C2 C3 C4 C5
Z F K Z K
R0 0 0 0 0 0 0
R1 F 0 0 1 1 1 1
R2 Z 0 1 1 1 2 2
R3 K 0 1 1 2 2 3
2/10/2024 Dr. D. P. Acharjya 25
Illustration
LCS computation of a row requires LCS table
values of the current and previous row.
For long sequences, these sequences can get
numerous and long.
It leads to storage space.
Storage space can be saved by saving the length of
the subsequence and the direction of the arrows
rather than the actual subsequences.
It will help to find the length and by bottom up
approach we will get the subsequence.
2/10/2024 Dr. D. P. Acharjya 26
LCS Length Algorithm
1. LCS length (X[1, …, m], Y[1, …, n])
2. C = Array(0 … m, 0 … n)
3. For i = 0 to m
4. C [i, 0] = 0
5. For j = 0 to n
6. C [0, j] = 0
2/10/2024 Dr. D. P. Acharjya 27
LCS Length Algorithm
7. For i = 1 to m
8. For j = 0 to n
9. if X[i] = Y[j]
10. C[i, j] = C[i-1, j-1] + 1
11. Else
12. C[i, j] = Max (C [i, j-1], C [i-1, j] )
13.Return C[m, n]
2/10/2024 Dr. D. P. Acharjya 28
