Analysis and Synthesis of Algorithms

Design of Algorithms

Dynamic Programming
Application Examples:
Combinatorial Arrangements, Matrix-Matrix
Multiplication, 0/1 Knapsack, Longest Common
Sub-Sequence and Coin Change
Copyright 2024, Pedro C. Diniz, all rights reserved.
Students enrolled in the DA class at Faculdade de Engenharia da Universidade do Porto (FEUP)
have explicit permission to make copies of these materials for their personal use.
 Outline
• Application Examples of Dynamic Programming
– Combinatorial Arrangements
– Multiplication of Matrices
– 0/1 Knapsack Problem
– Longest Common Sub-sequence (LCS)
– Coin Change
• Dynamic Programming Features

Analysis and Synthesis of Algorithms 2

 Dynamic Programming

• Basic Steps for the implementation of an Algorithm

based on Dynamic Programming:
– Characterize optimal structure of the solution
– Define the value of the optimal solution - the recurrence
– Compute the value of the optimal solution using a
Bottom-Up (or reverse) Approach (using a table)
– Construct the final Optimal Solution

Analysis and Synthesis of Algorithms 3

 Combinatorial Arrangements
Recursive Formulation:

ì1 if k = 0 or k = n
æ n ö ïïæ n - 1ö æ n - 1ö
çç ÷÷ = íçç ÷÷ + çç ÷÷ if 0 < k < n
è k ø ïè k - 1ø è k ø
ïî0 otherwise

Analysis and Synthesis of Algorithms 4

 Example (cont.)
Recursive Solution:
function C(n,k)
if k = 0 or k = n then
return 1
else
return C(n-1, k-1)+C(n-1,k)
Each invocation of C(n,k) returns 1 invokes the
computation and addition of two sub-problems
Solution is obtained adding 1s !
ææ nöö
Execution Time: Wçç çç ÷÷ ÷÷
èè k øø

Analysis and Synthesis of Algorithms 5

 Example (cont.)
• Number of distinct C(n,k) problems is n×k
• Complexity of Naïve solution derived from the
repeated computation of sub-problems:
– C(5,3) = C(4,2) + C(4,3)
– C(4,2) = C(3,1) + C(3,2)
– C(4,3) = C(3,2) + C(3,3)

• Solution: Constructive Approach (bottom-up)

– Fill in table (n x k) (Pascal triangule)

Analysis and Synthesis of Algorithms 6

 Construction of Table
0 1 2 … k-1 k
0 1
1 1 1
2 1 2 1
…
n-1 c[n-1,k-1] c[n-1,k]
n c[n,k]

• Comments:
– Artificial problem
– Ilustrates the elimination of the repeated computation of
sub-problems

Analysis and Synthesis of Algorithms 7

 Matrix-Matrix Multiplication Chains
• Chains of Matrix-Matrix Multiplications
– A1, A2, …, An
• Ai (pi-1 x pi)
– Multiplication time of the n matrices is dominated by the
time required to compute scalar multiplications
• When multiplying two matrices (r x s) e (s x t), the number of
scalar multiplications is: r x s x t
– Number of products depends on the way the products
are organized
• Placement of parenthesis defines organization of the
multiplications in the chain

Analysis and Synthesis of Algorithms 8

 Matrix-Matrix Multiplication Chains

• Concret case: A (13x5); B (5x89); C (89x3); D (3x34)

– (((AxB)xC)xD):
• 13x5x89 + 13x89x3 + 13x3x34 = 10582 products
– ((AxB)x(CxD)):
• 13x5x89 + 89x3x34 + 13x89x34 = 54201 products
– ((Ax(BxC))xD):
• 5x89x3 + 13x5x3 + 13x3x34 = 2856 products!
– (Ax((BxC)xD)):
• 5x89x3 + 5x3x34 + 13x5x34 = 4055 products
– (Ax(Bx(CxD)))
• 89x3x34 + 5x89x34 + 13x5x34 = 26418 products

Analysis and Synthesis of Algorithms 9

 Problem Formulation
• Place the parenthesis as to minimize the number of
scalar multiplications

• Number of possible parenthesis placement grows

exponentially with the number of matrices.
ì 1 n =1
ï n -1
P ( n) = í P ( k ) P ( n - k ) n ³ 2
ïîå
k =1

1 æ 2n ö
çç ÷÷ = Wç 4 n 2 ö÷
æ 3
P(n) = C (n - 1) C ( n) = n

n +1 è n ø è ø

Analysis and Synthesis of Algorithms 10

Characterization of the Optimal Solution
• Let A1..n be the solution with optimal placement of parenthesis
– Assume optimal solution with parenthesis in k, A1..k Ak+1..n
– Fact:
• Placement of parenthesis in A1..k is optimal
– Why?
• Otherwise is would be possible to find a better placement for A1..k
and therefore for A1..n

– Conclusion:
• Optimal solution for the placement of parenthesis is composed by
optimal solution for sub-problems

Analysis and Synthesis of Algorithms 11

 Recursive Solution
• m[i,j]:
– lowest number of multiplications to compute matrix Ai..j
– Optimal solution for A1..n is m[1,n]
– i = j: m[i,j] = 0
– i < j:
• Assume optimal solution places parenthesis in k:
– m[i,j] = m[i,k] + m[k+1,j] + pi-1pkpj
• But what is the value of k?
– clearly k must have value between i and j-1
– algorithm must therefore consider all possible k values
• s[i,j]:
– defines optimal placement of parenthesis between i and j

Analysis and Synthesis of Algorithms 12

 Recursive Solution (cont.)
Definition of m[i,j]:

ì0 i= j
m[i, j ] = í
îmin i £ k < j { m[i, k ] + m[k + 1, j ] + pi -1 pk p j } i < j

Definition of s[i,j]:

Analysis and Synthesis of Algorithms 13

 Computing the Values of m[i,j]

• Number of distinct sub-problems:

– 1 for each: 1 £ i £ j £ n
– Number of problems: Qn ( ) 2

• Problem:
– Recursive solution requires exponential time
– Repeated solution of sub-problems
• Solution:
– Constructive solution (bottom-up)
– Execution Time: Q(n 3 )

Analysis and Synthesis of Algorithms 14

 Example
i

1 2 3 4 5 6
A1 30 x 35
A2 35 x 15 6 15,125 10,500 5,375 3,500 5,000 0

A3 15 x 5 5 11,875 7,125 2,500 1,000 0

A4 5 x 10
A5 10 x 20 4 9,375 4,375 750 0

A6 20 x 25 j
3 7,875 2,625 0

2 15,750 0

1 0

ì m[2,2] + m[3,5] + p1 p 2 p5 = 0 + 2500 + 35.15.20 = 13,000

ï
m[2,5] = ím[2,3] + m[4,5] + p1 p3 p5 = 2625 + 1000 + 35.5.20 = 7125
ï m[2,4] + m[5,5] + p1 p 4 p5 = 4375 + 0 + 35.10.20 = 11,375
î
Analysis and Synthesis of Algorithms 15
 Example

i
A1 30 x 35 1 2 3 4 5 6
A2 35 x 15
A3 15 x 5 6 15,125 10,500 5,375 3,500 5,000 0

A4 5 x 10 5 11,875 7,125 2,500 1,000 0

A5 10 x 20
A6 20 x 25 4 9,375 4,375 750 0

j
3 7,875 2,625 0

2 15,750 0

1 0

ì m[2,2] + m[3,5] + p1 p 2 p5 = 0 + 2500 + 35.15.20 = 13,000

ï
m[2,5] = ím[2,3] + m[4,5] + p1 p3 p5 = 2625 + 1000 + 35.5.20 = 7125
ï m[2,4] + m[5,5] + p1 p 4 p5 = 4375 + 0 + 35.10.20 = 11,375
î
Analysis and Synthesis of Algorithms 16
 Example

i
A1 30 x 35 1 2 3 4 5 6
A2 35 x 15
A3 15 x 5 6 15,125 10,500 5,375 3,500 5,000 0

A4 5 x 10 5 11,875 7,125 2,500 1,000 0

A5 10 x 20
A6 20 x 25 4 9,375 4,375 750 0

j
3 7,875 2,625 0

2 15,750 0

1 0

ì m[2,2] + m[3,5] + p1 p 2 p5 = 0 + 2500 + 35.15.20 = 13,000

ï
m[2,5] = ím[2,3] + m[4,5] + p1 p3 p5 = 2625 + 1000 + 35.5.20 = 7125
ï m[2,4] + m[5,5] + p1 p 4 p5 = 4375 + 0 + 35.10.20 = 11,375
î
Analysis and Synthesis of Algorithms 17
 Example

i
A1 30 x 35 1 2 3 4 5 6
A2 35 x 15
A3 15 x 5 6 15,125 10,500 5,375 3,500 5,000 0

A4 5 x 10 5 11,875 7,125 2,500 1,000 0

A5 10 x 20
A6 20 x 25 4 9,375 4,375 750 0

j
3 7,875 2,625 0

2 15,750 0

1 0

ì m[2,2] + m[3,5] + p1 p 2 p5 = 0 + 2500 + 35.15.20 = 13,000

ï
m[2,5] = ím[2,3] + m[4,5] + p1 p3 p5 = 2625 + 1000 + 35.5.20 = 7125
ï m[2,4] + m[5,5] + p1 p 4 p5 = 4375 + 0 + 35.10.20 = 11,375
î
Analysis and Synthesis of Algorithms 18
 Code for Matrix-Multiplication

m = length[p]-1;
for i = 1 to n do
m[i,i] = 0;
for l = 2 to n do
for i = 1 to n - l + 1 do
for j = 1 to i + l - 1 do
m[i,j] = ∞;
for k = i to j - 1 do
q = m[i,k]+m[k+1,j]+pi-1pkpj;
if (q < m[i,j]) then
m[i,j] = q;
s[i,j] = k;
end
end
end Finding the best k
end
Analysis and Synthesis of Algorithms 19
 0/1 Knapsack Problem
• Problem Definition:
– Given n objects (1,…,n) and a knapsack
– Each object has value vi and weight wi
– Knapsack weight cannot exceed W
– Objective:
• maximize the value in the knapsack while meeting its weight limit
• Formalization:
– xi = 1 if object i is included in the knapsack; 0 otherwise
n
max åxv
i =1
i i

n
s.t. åx w
i =1
i i £W

Analysis and Synthesis of Algorithms 20

 A Simple (Greedy) Algorithm
• Algoritm selects at each step the “best” object, i.e., the
object with best value/weight ratio vi/wi

• Problem:
v1 = 8, w1 = 6
v2 = 5, w2 = 5
v3 = 5, w3 = 5

W = 10

– First selected object is v1;

– Prevents inclusion of any other object
– Cannot reach Optimal Solution (with objects 2 and 3)

Analysis and Synthesis of Algorithms 21

 Using Dynamic Programming
Optimality Reasoning
The Optimal Solution for weight W must also be the Optimal
solution to the problem with weight W-wn if it includes object n;

But, we also need to consider the case where object n is not

included, in which case, the optimal solution may not include it.

• Each decision of including object k needs to:

– Consider the use of weight wk and hence its value vk
– Save the weight for other objects

Analysis and Synthesis of Algorithms 22

 Using Dynamic Programming
• "#$ %, '
– Defines the maximum value that is possible to transport
with a weight limit of j (j ≤ W )
– Allowing only object numbered from 1 to i to be included

• Opimal solution is defined by "#$ (, )

– At each decision consider two alternatives: do not include
object i
"#$ %, ' = max("#$ % − 1, ' , "#$ % − 1, ' − 12 + "2 )
Include
"#$ %, 0 = 0 object i
"#$ 0, ' = 0, ' ≥ 0 "#$ %, ' = −∞, ' < 0
Analysis and Synthesis of Algorithms 23
 Using Dynamic Programming
• "#$ %, '
– Defines the maximum value that is possible to transport
with a weight limit of j (j ≤ W )
– Allowing only object numbered from 1 to i to be included

• Opimal solution is defined by "#$ (, )

– At each decision consider two alternatives: Still j capacity
available

"#$ %, ' = max("#$ % − 1, ' , "#$ % − 1, ' − 12 + "2 )

Reduced
"#$ %, 0 = 0 capacity
"#$ 0, ' = 0, ' ≥ 0 "#$ %, ' = −∞, ' < 0 available

Analysis and Synthesis of Algorithms 24

 Comments
• Optimal Solution is composed of Optimal solutions
to sub-problems:

!"# $, & = max(!"# $ − 1, & , !"# $ − 1, & − ./ + !/ )

– If val[i,j] is an optimal solution:

• If object i is not included, and val[i-1,j] is optimal sub-
problem solution for the same capacity
• If object i is included, and val[i-1,j-wi] is optimal sub-
problem solution for a lower capacity, but including item i.
– Otherwise, it would be possible to obtained solution with value
greater than optimal solution to each sub-probems and thus to the
overall problem: a contradiction !

Analysis and Synthesis of Algorithms 25

 Example
v1 = 8, w1 = 6
v2 = 5, w2 = 5
v3 = 5, w3 = 5 Weight
W = 10 0 1 5 10
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 8 8 8 8 8
Object

2 0 0 0 0 0 5 8 8 8 8 8
3 0 0 0 0 0 5 5 5 5 5 10

!"# $, & = max(!"# $ − 1, & , !"# $ − 1, & − ./ + !/ )

Analysis and Synthesis of Algorithms 26

 Example
v1 = 8, w1 = 6
v2 = 5, w2 = 5
v3 = 5, w3 = 5 Weight
W = 10 0 1 5 10
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 8 8 8 8 8
Object

2 0 0 0 0 0 5 8 8 8 8 8
3 0 0 0 0 0 5 5 5 5 5 10

!"# $, & = max(!"# $ − 1, & , !"# $ − 1, & − ./ + !/ )

Analysis and Synthesis of Algorithms 27

 Comments (cont.)
• Recursive Solution
– Execution Time Exponential wrt n and W
– Number of distinct sub-problems is n×#
– Repeated computation of the same sub-problems !

• Use Dynamic Programming (Bottom-Up) Approach

– Fill in table (n×#), top to bottom
– Check which is the highest entry in Table

• Execution time: Q(nW )

Analysis and Synthesis of Algorithms 28

 Comments (cont.)
• Recursive Solution
– Execution Time Exponential wrt n and W
– Number of distinct sub-problems is n×#
– Repeated computation of the same sub-problems !

• Use Dynamic Programming (Bottom-Up) Approach

– Fill in table (n×#), top to bottom
– Check which is the highest entry in Table

• Execution time: Q(nW )

– Pseudo-polinomial
Analysis and Synthesis of Algorithms 29
Longest Common Sub-Sequence (LCS)
• Given a sequence X = x1,…,xn, the sequence
Z = z1,…,zk is a sub-sequence of X if there exists
a strictly increasing sequence of indeces {i1,…,ik}
such that xi j = z j for all j = 1,…,k,

• Given the sequences X = x1,…,xn and Y = y1,…,ym,

Z is a common sub-sequence if Z is a sub-sequence
of X and Y

• Problem: Find the longest common subsequence

(LCS) of two sequences X and Y

Analysis and Synthesis of Algorithms 30

 Examples
• Example:
– X = abefcghd
– Y = eagbcfdh
– Z = abcd is a common sub-sequence of X and Y

• An Exhaustive Solution in impractical

– Consider the inclusion (or exclusion) of each character
from X and Y
n
• Total number of sub-sequences in X: 2
• Total number of sub-sequences in Y: 2 m
• Total number of cases to analyze: 2 n+ m
– Infeasible for high values of n and m…

Analysis and Synthesis of Algorithms 31

 String Results
• Results:
– Let X = x1,…,xn and Y = y1,…,ym be two sequences, and let
Z = z1,…,zk be a LCS of X a Y
• If xn = ym, then zk = xn = ym and Zk-1 is LCS of Xn-1 and Ym-1
• If xn ¹ ym, then zk ¹ xn implies that Z is LCS of Xn-1 and Y
• If xn ¹ ym, then zk ¹ ym implies that Z is LCS of X and Ym-1

• Approach:
– If xn = ym, find LCS W for Xn-1 and Ym-1 then
Add xn = ym to W to obtain Z
– If xn ¹ ym, find LCS’s for Xn-1 and Y and for X and Ym-1 then
Select the longest sub-sequence of two sub-problems

Analysis and Synthesis of Algorithms 32

 String Results
Case 1. If xn = ym, find LCS W for Xn-1 and Ym-1
n
X
m
Y

Analysis and Synthesis of Algorithms 33

 String Results
Case 1. If xn = ym, find LCS W for Xn-1 and Ym-1
n-1 n
X
m-1 m
Y

Analysis and Synthesis of Algorithms 34

 String Results
Case 1. If xn = ym, find LCS W for Xn-1 and Ym-1
n-1
X
m-1
Y

Analysis and Synthesis of Algorithms 35

 String Results
Case 2. If xn ¹ ym, find LCS’s for Xn-1 and Y &
and for X and Ym-1
n
X
m
Y

Analysis and Synthesis of Algorithms 36

 String Results
Case 2. If xn ¹ ym, find LCS’s for Xn-1 and Y &
and for X and Ym-1
n
X
m-1
Y

Analysis and Synthesis of Algorithms 37

 String Results
Case 2. If xn ¹ ym, find LCS’s for Xn-1 and Y &
and for X and Ym-1
n-1
X
m
Y

Analysis and Synthesis of Algorithms 38

 LCS Formalization

• c[i,j]:
– Length of the LCS for the sequences Xi and Yj

• Recurrence:
ì0 if i = 0 or j = 0
ï
c[i, j ] = íc[i - 1, j - 1] + 1 if i, j > 0 and xi = y j
ïmax (c[i - 1, j ], c[i, j - 1]) if i, j > 0 and xi ¹ y j
î

• Execution Time: Q(n × m )

Analysis and Synthesis of Algorithms 39

 LCS - Algorithm
procedure LCS(string x, string y, c, dir)
1. for i = 1 to m do
2. c[i , 0] = 0;
3. for j = 0 to n do
4. c[0, j] = 0;
5. for i = 1 to m do
6. for j = 1 to n do
7. if (x[i] == y[j]) then
8. c[i,j] = c[i-1,j-1]+1;
9. dir[i,j] = 1; // diagonal
10. else if (c[i-1,j] >= c[i,j-1]) then
11. c[i,j] = c[i-1,j];
12. dir[i,j] = 2; // up
13. else
14. c[i,j] = c[i,j-1];
15. dir[i,j] = 3; // forward
16. end
17. end

dir[i,j] stores the directions. 1-diagonal, 2-up, 3-forward.

Analysis and Synthesis of Algorithms 40

 LCS - Algorithm
procedure LCS(string x, string y, c, dir)
1. for i = 1 to m do
2. c[i , 0] = 0; Y
X
3. for j = 0 to n do
4. c[0, j] = 0; 0 0 0 0 0 0 0 0
5. for i = 1 to m do 0
6. for j = 1 to n do
7. if (x[i] == y[j]) then 0
8. c[i,j] = c[i-1,j-1]+1;
0
9. dir[i,j] = 1; // diagonal
10. else if (c[i-1,j] >= c[i,j-1]) then 0
11. c[i,j] = c[i-1,j];
12. dir[i,j] = 2; // up 0
13. else 0
14. c[i,j] = c[i,j-1];
15. dir[i,j] = 3; // forward
16. end
17. end

b[i,j] stores the directions. 1-diagonal (left), 2-up, 3-forward (left).

Analysis and Synthesis of Algorithms 41

 LCS - Algorithm
procedure LCS(string x, string y, c, dir)
1. for i = 1 to m do
2. c[i , 0] = 0; Y
X
3. for j = 0 to n do
4. c[0, j] = 0; 0 0 0 0 0 0 0 0
5. for i = 1 to m do 0
6. for j = 1 to n do
7. if (x[i] == y[j]) then 0
8. c[i,j] = c[i-1,j-1]+1;
0
9. dir[i,j] = 1; // diagonal
10. else if (c[i-1,j] >= c[i,j-1]) then 0
11. c[i,j] = c[i-1,j];
12. dir[i,j] = 2; // up 0
13. else 0
14. c[i,j] = c[i,j-1];
15. dir[i,j] = 3; // forward
16. end
17. end

b[i,j] stores the directions. 1-diagonal (left), 2-up, 3-forward (left).

Analysis and Synthesis of Algorithms 42

 LCS - Algorithm
procedure LCS(string x, string y, c, dir)
1. for i = 1 to m do
2. c[i , 0] = 0; Y
X
3. for j = 0 to n do
4. c[0, j] = 0; 0 0 0 0 0 0 0 0
5. for i = 1 to m do 0
6. for j = 1 to n do
7. if (x[i] == y[j]) then 0
8. c[i,j] = c[i-1,j-1]+1;
0
9. dir[i,j] = 1; // diagonal
10. else if (c[i-1,j] >= c[i,j-1]) then 0
11. c[i,j] = c[i-1,j];
12. dir[i,j] = 2; // up 0
13. else 0
14. c[i,j] = c[i,j-1];
15. dir[i,j] = 3; // forward
16. end
17. end

b[i,j] stores the directions. 1-diagonal (left), 2-up, 3-forward (left).

Analysis and Synthesis of Algorithms 43

 LCS - Algorithm
procedure LCS(string x, string y, c, dir)
1. for i = 1 to m do
2. c[i , 0] = 0; Y
X
3. for j = 0 to n do
4. c[0, j] = 0; 0 0 0 0 0 0 0 0
5. for i = 1 to m do 0
6. for j = 1 to n do
7. if (x[i] == y[j]) then 0
8. c[i,j] = c[i-1,j-1]+1;
0
9. dir[i,j] = 1; // diagonal
10. else if (c[i-1,j] >= c[i,j-1]) then 0
11. c[i,j] = c[i-1,j];
12. dir[i,j] = 2; // up 0
13. else 0
14. c[i,j] = c[i,j-1];
15. dir[i,j] = 3; // forward
16. end
17. end

b[i,j] stores the directions. 1-diagonal (left), 2-up, 3-forward (left).

Analysis and Synthesis of Algorithms 44

 LCS - Example
Y B D C A B A
X
0 0 0 0 0 0 0
A 0 ­0 ­0 ­0 1 1 1

¬
¬
B 0
C 0
B 0
D 0
A 0
B 0

Analysis and Synthesis of Algorithms 45

 LCS - Example
Y B D C A B A
X
0 0 0 0 0 0 0
A 0 ­0 ­0 ­0 1 1 1

¬
¬
B 0 1 1 1 ­1 2 2

¬
¬ ¬ ¬
C ­1 ­1 2 2 ­2 ­2

¬
0 ¬
1
¬ ­1 ­2 ­2 3 3

¬
B 0 ¬
1 ­2 ­2 ­2 ­ 3 ­3
¬

D 0
1 ­2 ­2 3 ­3 4
¬

¬
A 0
­2 ­2 ­ 3 ­4
¬

1 4
¬
B 0

Analysis and Synthesis of Algorithms 46

 LCS - Example
Y B D C A B A
X
0 0 0 0 0 0 0
A 0 ­0 ­0 ­0 1 1 1

¬
¬
B 0 1 1 1 ­1 2 2

¬
¬ ¬ ¬
C ­1 ­1 2 2 ­2 ­2

¬
0 ¬
1
¬ ­1 ­2 ­2 3 3

¬
B 0 ¬
1 ­2 ­2 ­2 ­ 3 ­3
¬

D 0
1 ­2 ­2 3 ­3 4
¬

¬
A 0
­2 ­2 ­ 3 ­4
¬

1 4
¬
B 0

Analysis and Synthesis of Algorithms 47

 LCS - Example
Y B D C A B A
X
0 0 0 0 0 0 0
A 0 ­0 ­0 ­0 1 1 1

¬
¬
B 0 1 1 1 ­1 2 2

¬
¬ ¬ ¬
C ­1 ­1 2 2 ­2 ­2

¬
0 ¬
1
¬ ­1 ­2 ­2 3 3

¬
B 0 ¬
1 ­2 ­2 ­2 ­ 3 ­3
¬

D 0
1 ­2 ­2 3 ­3 4
¬

¬
A 0
­2 ­2 ­ 3 ­4
¬

1 4
¬
B 0

LCS(X,Y) = BCBA with length 4

Analysis and Synthesis of Algorithms 48

 LCS - Example
Y B D C A B A
X
0 0 0 0 0 0 0
A 0 ­0 ­0 ­0 1 1 1

¬
¬
B 0 1 1 1 ­1 2 2

¬
¬ ¬ ¬
C ­1 ­1 2 2 ­2 ­2

¬
0 ¬
1
¬ ­1 ­2 ­2 3 3

¬
B 0 ¬
1 ­2 ­2 ­2 ­ 3 ­3
¬

D 0
1 ­2 ­2 3 ­3 4
¬

¬
A 0
­2 ­2 ­ 3 ­4
¬

1 4
¬
B 0

LCS(X,Y) = BCBA with length 4

LCS(X,Y) = BCAB with length 4

Analysis and Synthesis of Algorithms 49

 Constructing an LCS (Backtracking)
• We can find an LCS using dir[i,j]’s.
• We start with dir[n,m] and trace back to some cell c[0,i] or c[i,0].
• The algorithm to construct an LCS
1. i = m
2. j = n;
3. if (i == 0 or j == 0) then
exit;
4. if dir[i,j] == 1) then
i = i - 1;
j = j - 1;
print “xi”;
5. if dir[i,j] == 2 then i = i - 1
6. if dir[i,j] == 3 then j = j - 1
7. goto Step 3.

• TimeComplexity: Q(n × m )
Analysis and Synthesis of Algorithms 50
 Coin Change
• Given a set of coins, from denominations c1,…,cn,
with values d1,…,dn, compute the lowest number of
coins such that its sums is N
– Assume: unlimited number of coins of each denomination

• Greedy solution may not work:

– v1 = 1; v2 = 5; v3 = 20; v4 = 25
– Change for 40?!

• Solution based on Dynamic Programming?

Analysis and Synthesis of Algorithms 51

 Coin Change Formulation
• c[i,j]:
– Lowest number of coins required to cover j units , 0 £j£ N,
only using coins with denominations between 1 and i, 1 £i£ n
– Objective is to compute c[n,N]

• Formulation:
c[i,0] = 0 1 £ i £ n
c[i, j ] = +¥ i = 0 or j < 0 do not include coin i

c[i, j ] = min(c[i - 1, j ], 1 + c[i, j - d i ])

• Execution Time: Q(n × N )
include coin i
Analysis and Synthesis of Algorithms 52
 Coin Change Example

• Example: c[i,0] = 0 1 £ i £ n

– v1 = 1; v2 = 5; v3 = 20; v4 = 25 c[i, j ] = +¥ i = 0 or j < 0

– N= 40 c[i, j ] = min(c[i - 1, j ], 1 + c[i, j - d i ])

0 1 15 20 25 30 35 40
0 0 0 0 0 0 0 0

Analysis and Synthesis of Algorithms 53

 Coin Change Example

• Example: c[i,0] = 0 1 £ i £ n

– v1 = 1; v2 = 5; v3 = 20; v4 = 25 c[i, j ] = +¥ i = 0 or j < 0

– N= 40 c[i, j ] = min(c[i - 1, j ], 1 + c[i, j - d i ])

j 0 1 15 20 25 30 35 40
i
0 0 0 0 0 0 0 0
0 0 1 1 0 0 1 1 0

Analysis and Synthesis of Algorithms 54

 Coin Change Example

• Example: c[i,0] = 0 1 £ i £ n
– v1 = 1; v2 = 5; v3 = 20; v4 = 25 c[i, j ] = +¥ i = 0 or j < 0
– N= 40 c[i, j ] = min(c[i - 1, j ], 1 + c[i, j - d i ])

j 0 1 15 20 25 30 35 40
i
0 0 0 0 0 0 0 0
0 0 1 1 0 0 1 1 0
2 0 0 1 0 2 1 1 0

• Implementation:
– Using a sparse Matrix Representation

Analysis and Synthesis of Algorithms 55

 Dynamic Programming Features
• Optimal solution composed of optimal solutions for
sub-problems

• Recursive solution repeatedly resolves the same sub-

problems
– Superimposition of problems lead to increased complexity
– Use table to avoid recomputing the same solution more than once

• Reconstruction of the optimal solution from sub-

problems’ solutions

• Use of Memoization

Analysis and Synthesis of Algorithms 56

 Summary
• Application Examples of Dynamic Programming
– Combinatorial Arrangements
– Multiplication of Matrices
– 0/1 Knapsack problem
– Longest Common Sub-sequence (LCS)
– Coin Change
• Dynamic Programming Features

Analysis and Synthesis of Algorithms 57