Dynamic Programming
Dynamic Programming
An algorithm design technique (like divide and
conquer)
Divide and conquer
Partition the problem into independent subproblems
Solve the subproblems recursively
Combine the solutions to solve the original problem
Dynamic Programming
Applicable when subproblems are not independent
Subproblems share subsubproblems
E.g.: Combinations:
n
k
n
1
=1
n-1
k
n-1
k-1
n
n
=1
A divide and conquer approach would repeatedly solve the
common subproblems
Dynamic programming solves every subproblem just once and
stores the answer in a table
Example: Combinations
Comb (6,4)
=
Comb (5, 3)
Comb (4,2)
Comb (4, 3)
Comb (3, 1)+
+ Comb (2,
+ 1) + Comb (2, 2) + +Comb (2, 1) + Comb (2,
+ 2) +
Comb (3, 2)
Comb (3, 2)
Comb (5, 4)
n-1
k
Comb (4, 3)
Comb
+ (3, 3)
Comb
+ (3, 2)
Comb (4, 4)
Comb
+ (3, 3) +
+ +Comb (2, 1) + Comb (2,
+ 2) +
1+
n
k
n-1
k-1
4
Dynamic Programming
Used for optimization problems
A set of choices must be made to get an optimal
solution
Find a solution with the optimal value (minimum or
maximum)
There may be many solutions that lead to an optimal
value
Our goal: find an optimal solution
Dynamic Programming Algorithm
1. Characterize the structure of an optimal
solution
2. Recursively define the value of an optimal
solution
3. Compute the value of an optimal solution in a
bottom-up fashion
4. Construct an optimal solution from computed
information (not always necessary)
6
Assembly Line Scheduling
Automobile factory with two assembly lines
Each line has n stations: S1,1, . . . , S1,n and S2,1, . . . , S2,n
Corresponding stations S1, j and S2, j perform the same function
but can take different amounts of time a1, j and a2, j
Entry times are: e1 and e2; exit times are: x1 and x2
Assembly Line Scheduling
After going through a station, can either:
stay on same line at no cost, or
transfer to other line: cost after Si,j is ti,j , j = 1, . . . , n - 1
Assembly Line Scheduling
Problem:
what stations should be chosen from line 1 and which
from line 2 in order to minimize the total time through the
factory for one car?
One Solution
Brute force
Enumerate all possibilities of selecting stations
Compute how long it takes in each case and choose
the best one
Solution:
0 if choosing line 2
at step j (= 3)
1 if choosing line 1
at step j (= n)
There are 2n possible ways to choose stations
Infeasible when n is large!!
10
1. Structure of the Optimal Solution
How do we compute the minimum time of going through
a station?
11
1. Structure of the Optimal Solution
Lets consider all possible ways to get from the
starting point through station S1,j
We have two choices of how to get to S1, j:
Through S1, j - 1, then directly to S1, j
Through S2, j - 1, then transfer over to S1, j
Line 1
S1,j-1
S1,j
a1,j-1
a1,j
t2,j-1
Line 2
a2,j-1
S2,j-1
12
1. Structure of the Optimal Solution
Suppose that the fastest way through S1, j is
through S1, j 1
We must have taken a fastest way from entry through S1, j 1
If there were a faster way through S1, j - 1, we would use it instead
Similarly for S2, j 1
Line 1
S1,j-1
S1,j
a1,j-1
a1,j
t2,j-1
Optimal Substructure
Line 2
a2,j-1
S2,j-1
13
Optimal Substructure
Generalization: an optimal solution to the
problem find the fastest way through S1, j contains
within it an optimal solution to subproblems: find
the fastest way through S1, j - 1 or S2, j 1.
This is referred to as the optimal substructure
property
We use this property to construct an optimal
solution to a problem from optimal solutions to
subproblems
14
2. A Recursive Solution
Define the value of an optimal solution in terms of the optimal
solution to subproblems
15
2. A Recursive Solution (cont.)
Definitions:
f* : the fastest time to get through the entire factory
fi[j] : the fastest time to get from the starting point through station Si,j
f* = min (f1[n] + x1, f2[n] + x2)
16
2. A Recursive Solution (cont.)
Base case: j = 1, i=1,2 (getting through station 1)
f1[1] = e1 + a1,1
f2[1] = e2 + a2,1
17
2. A Recursive Solution (cont.)
General Case: j = 2, 3, ,n, and i = 1, 2
Fastest way through S1, j is either:
the way through S1, j - 1 then directly through S1, j, or
f1[j - 1] + a1,j
the way through S2, j - 1, transfer from line 2 to line 1, then through S1, j
f2[j -1] + t2,j-1 + a1,j
Line 1
S1,j-1
S1,j
a1,j-1
a1,j
f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)
Line 2
t2,j-1
a2,j-1
S2,j-1
18
2. A Recursive Solution (cont.)
f1[j] =
f2[j] =
e1 + a1,1
if j = 1
min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j) if j 2
e2 + a2,1
if j = 1
min(f2[j - 1] + a2,j ,f1[j -1] + t1,j-1 + a2,j) if j 2
19
3. Computing the Optimal Solution
f* = min (f1[n] + x1, f2[n] + x2)
f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)
f2[j] = min(f2[j - 1] + a2,j ,f1[j -1] + t1,j-1 + a2,j)
1
f1[j]
f1(1)
f1(2)
f1(3)
f1(4)
f1(5)
f2[j]
f2(1)
f2(2)
f2(3)
f2(4)
f2(5)
4 times
2 times
Solving top-down would result in exponential
running time
20
3. Computing the Optimal Solution
For j 2, each value fi[j] depends only on the
values of f1[j 1] and f2[j - 1]
Idea: compute the values of fi[j] as follows:
in increasing order of j
1
f1[j]
f2[j]
Bottom-up approach
First find optimal solutions to subproblems
Find an optimal solution to the problem from the
subproblems
21
Example
e1 + a1,1,
f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)
1
f1[j]
18[1]
20[2]
24[1]
32[1]
f2[j]
12
16[1]
22[2]
25[1]
30[2]
if j = 1
if j 2
f* = 35[1]
22
FASTEST-WAY(a, t, e, x, n)
1. f1[1] e1 + a1,1
2. f2[1] e2 + a2,1
Compute initial values of f1 and f2
3. for j 2 to n
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
do if f1[j - 1] + a1,j f2[j - 1] + t2, j-1 + a1, j
then f1[j] f1[j - 1] + a1, j
l1[j] 1
else f1[j] f2[j - 1] + t2, j-1 + a1, j
O(N)
Compute the values of
f1[j] and l1[j]
l1[j] 2
if f2[j - 1] + a2, j f1[j - 1] + t1, j-1 + a2, j
then f2[j] f2[j - 1] + a2, j
l2[j] 2
else f2[j] f1[j - 1] + t1, j-1 + a2, j
l2[j] 1
Compute the values of
f2[j] and l2[j]
23
FASTEST-WAY(a, t, e, x, n) (cont.)
14. if f1[n] + x1 f2[n] + x2
15.
16.
17.
18.
then f* = f1[n] + x1
l* = 1
else f* = f2[n] + x2
Compute the values of
the fastest time through the
entire factory
l* = 2
24
4. Construct an Optimal Solution
Alg.: PRINT-STATIONS(l, n)
i l*
print line i , station n
for j n downto 2
do i li[j]
print line i , station j - 1
1
f1[j]/l1[j]
18[1]
20[2]
24[1]
32[1]
f2[j]/l2[j]
12
16[1]
22[2]
25[1]
30[2]
l* = 1
25
Elements of Dynamic Programming
Optimal Substructure
An optimal solution to a problem contains within it an
optimal solution to subproblems
Optimal solution to the entire problem is build in a
bottom-up manner from optimal solutions to
subproblems
Overlapping Subproblems
If a recursive algorithm revisits the same subproblems
over and over the problem has overlapping
subproblems
26
Parameters of Optimal Substructure
How many subproblems are used in an optimal
solution for the original problem
Assembly line: One subproblem (the line that gives best time)
Matrix multiplication: Two subproblems (subproducts Ai..k, Ak+1..j)
How many choices we have in determining
which subproblems to use in an optimal solution
Assembly line: Two choices (line 1 or line 2)
Matrix multiplication: j - i choices for k (splitting the product)
27
Parameters of Optimal Substructure
Intuitively, the running time of a dynamic
programming algorithm depends on two factors:
Number of subproblems overall
How many choices we look at for each subproblem
Assembly line
(n) subproblems (n stations)
(n) overall
2 choices for each subproblem
Matrix multiplication:
(n2) subproblems (1 i j n)
At most n-1 choices
(n3) overall
28
Longest Common Subsequence
Given two sequences
X = x1, x2, , xm
Y = y1, y2, , yn
find a maximum length common subsequence
(LCS) of X and Y
E.g.:
X = A, B, C, B, D, A, B
Subsequences of X:
A subset of elements in the sequence taken in order
A, B, D, B, C, D, B, etc.
29
Example
X = A, B, C, B, D, A, B
X = A, B, C, B, D, A, B
Y = B, D, C, A, B, A
Y = B, D, C, A, B, A
B, C, B, A and B, D, A, B are longest common
subsequences of X and Y (length = 4)
B, C, A, however is not a LCS of X and Y
30
Brute-Force Solution
For every subsequence of X, check whether its
a subsequence of Y
There are 2m subsequences of X to check
Each subsequence takes (n) time to check
scan Y for first letter, from there scan for second, and
so on
Running time: (n2m)
31
Making the choice
X = A, B, D, E
Y = Z, B, E
Choice: include one element into the common
sequence (E) and solve the resulting
subproblem
X = A, B, D, G
Y = Z, B, D
Choice: exclude an element from a string and
solve the resulting subproblem
32
Notations
Given a sequence X = x1, x2, , xm we define
the i-th prefix of X, for i = 0, 1, 2, , m
Xi = x1, x2, , xi
c[i, j] = the length of a LCS of the sequences
Xi = x1, x2, , xi and Yj = y1, y2, , yj
33
A Recursive Solution
Case 1: xi = yj
e.g.:
Xi = A, B, D, E
Yj = Z, B, E
c[i, j] = c[i - 1, j - 1] + 1
Append xi = yj to the LCS of Xi-1 and Yj-1
Must find a LCS of Xi-1 and Yj-1 optimal solution to
a problem includes optimal solutions to subproblems
34
A Recursive Solution
Case 2: xi yj
e.g.:
Xi = A, B, D, G
Yj = Z, B, D
c[i, j] = max { c[i - 1, j], c[i, j-1] }
Must solve two problems
find a LCS of Xi-1 and Yj: Xi-1 = A, B, D and Yj = Z, B, D
find a LCS of Xi and Yj-1: Xi = A, B, D, G and Yj = Z, B
Optimal solution to a problem includes optimal
solutions to subproblems
35
Overlapping Subproblems
To find a LCS of X and Y
we may need to find the LCS between X and Yn-1 and
that of Xm-1 and Y
Both the above subproblems has the subproblem of
finding the LCS of Xm-1 and Yn-1
Subproblems share subsubproblems
36
3. Computing the Length of the LCS
c[i, j] =
0
c[i-1, j-1] + 1
max(c[i, j-1], c[i-1, j])
xi
x1
x2
m xm
0
yj:
1
y1
2
y2
if i = 0 or j = 0
if xi = yj
if xi yj
n
yn
0
first
0
0
0
0
0
second
j
37
Additional Information
0
if i,j = 0
c[i, j] = c[i-1, j-1] + 1
if xi = yj
max(c[i, j-1], c[i-1, j]) if xi yj
b & c:
0
xi
m D
0
yj:
1
A
2
C
3
D
0
0
0
0
0
0
0
c[i-1,j]
c[i,j-1]
n
F
A matrix b[i, j]:
For a subproblem [i, j] it
tells us what choice was
made to obtain the
optimal value
If xi = yj
b[i, j] =
Else, if
c[i - 1, j] c[i, j-1]
b[i, j] =
else
b[i, j] =
38
LCS-LENGTH(X, Y, m, n)
1. for i 1 to m
2.
do c[i, 0] 0
The length of the LCS if one of the sequences
3. for j 0 to n
is empty is zero
4.
do c[0, j] 0
5. for i 1 to m
6.
do for j 1 to n
7.
do if xi = yj
8.
then c[i, j] c[i - 1, j - 1] + 1 Case 1: xi = yj
9.
b[i, j ]
10.
else if c[i - 1, j] c[i, j - 1]
11.
then c[i, j] c[i - 1, j]
12.
b[i, j]
Case 2: xi yj
13.
else c[i, j] c[i, j - 1]
14.
b[i, j]
15. return c and b
Running time: (mn)
39
Example
0
if i = 0 or j = 0
if xi = yj
max(c[i, j-1], c[i-1, j]) if xi yj
X = A, B, C, B, D, A
c[i, j] = c[i-1, j-1] + 1
Y = B, D, C, A, B, A
If xi = yj
b[i, j] =
Else if
c[i - 1, j] c[i, j-1]
b[i, j] =
else
b[i, j] =
0
yj
1
B
2
D
3
C
4
A
5
B
6
A
xi
2
3
4
40
4. Constructing a LCS
Start at b[m, n] and follow the arrows
When we encounter a in b[i, j] xi = yj is an element
of the LCS
0
yj
1
B
2
D
3
C
4
A
5
B
6
A
xi
1
2
2
3
4
41
PRINT-LCS(b, X, i, j)
1. if i = 0 or j = 0
Running time: (m + n)
2. then return
3. if b[i, j] =
4.
then PRINT-LCS(b, X, i - 1, j - 1)
5.
print xi
6. elseif b[i, j] =
7.
then PRINT-LCS(b, X, i - 1, j)
8.
else PRINT-LCS(b, X, i, j - 1)
Initial call: PRINT-LCS(b, X, length[X], length[Y])
42
Improving the Code
What can we say about how each entry c[i, j] is
computed?
It depends only on c[i -1, j - 1], c[i - 1, j], and
c[i, j - 1]
Eliminate table b and compute in O(1) which of the
three values was used to compute c[i, j]
We save (mn) space from table b
However, we do not asymptotically decrease the
auxiliary space requirements: still need table c
43
Improving the Code
If we only need the length of the LCS
LCS-LENGTH works only on two rows of c at a time
The row being computed and the previous row
We can reduce the asymptotic space requirements by
storing only these two rows
44