Dynamic Programming
Dynamic Programming
Dynamic Programming
2
DP - Two key ingredients
• Two key ingredients for an optimization problem
to be suitable for a dynamic-programming
solution:
4
Fibonacci numbers
F 0
0
F 1
1
F F F for i>1 .
i i 1 i 2
5
How to compute F10 ?
F8
F9
F7 ……
F10
F7
F8
F6
6
Dynamic Programming
• Applicable when subproblems are not independent
– Subproblems share subsubproblems
E.g.: Fibonacci numbers:
• Recurrence: F(n) = F(n-1) + F(n-2)
• Boundary conditions: F(1) = 0, F(2) = 1
• Compute: F(5) = 3, F(3) = 1, F(4) = 2
– A divide and conquer approach would repeatedly solve the
common subproblems
– Dynamic programming solves every subproblem just once and
stores the answer in a table
7
Tabular computation
• The tabular computation can avoid
recompuation.
F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10
0 1 1 2 3 5 8 13 21 34 55
Result
8
Dynamic Programming Algorithm
1. Characterize the structure of an optimal
solution
2. Recursively define the value of an optimal
solution
3. Compute the value of an optimal solution in a
bottom-up fashion
4. Construct an optimal solution from computed
information
9
Longest increasing subsequence(LIS)
10
A naive approach for LIS
• Let L[i] be the length of a longest increasing
subsequence ending at position i.
L[i] = 1 + max j = 0..i-1{L[j] | aj < ai}
(use a dummy a0 = minimum, and L[0]=0)
Index 0 1 2 3 4 5 6 7 8 9 10
Input 0 9 2 5 3 7 11 8 10 13 6
Length 0 1 1 2 2 3 4 4 5 6 3
Prev -1 0 0 2 2 4 5 5 7 8 4
Path 1 1 1 1 1 2 2 2 2 2 2
9 –3 1 7 –15 2 3 –4 2 –7 6 –2 8 4 -9
Try Yourself
13
The Knapsack Problem
• The 0-1 knapsack problem
– A thief robbing a store finds n items: the i-th item is
worth vi dollars and weights wi pounds (vi, wi integers)
– The thief can only carry W pounds in his knapsack
– Items must be taken entirely or left behind
– Which items should the thief take to maximize the
value of his load?
• The fractional knapsack problem
– Similar to above
– The thief can take fractions of items
14
The 0-1 Knapsack Problem
weight wi
• Goal:
– find xi such that for all xi = {0, 1}, i = 1, 2, .., n
wixi W and
xivi is maximum
15
0-1 Knapsack - Greedy Strategy
• E.g.:
Item 3 30 $120
Item 2 50 50 50 +
20 $100
Item 1 30
20 + 20 $100
10 10 $60
16
0-1 Knapsack - Dynamic Programming
P(i, w) =P(i - 1, w)
17
0-1 Knapsack – DP Algorithm
•
for i ← 0 to n
• do P(i,0) = 0
• for w ← 0 to W
• do P(0,w) = 0
• for i from 1 to n
• do for w from 0 to W
• do if wi > w //if the capacity is not enough
• then P(i, w) = P(i-1, w)
• else
• P(i,w) = max{ vi + P(i-1, w-wi), P(i-1, w) }
0-1 Knapsack - Dynamic Programming
0 0 0 0 0 0 0 0 0 0 0 0
0 first
0 second
i-1 0
i 0
0
n 0
19
W = 5 Item
Example: Weight Value
1 2 12
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
2 1 10
w 3 3 20
0 1 2 3 4 5
4 2 15
0 0 0 0 0 0 0 P(1, 1) = P(0, 1) = 0
1 0 0 12 12 12 12 P(1, 2) = max{12+0, 0} = 12
2 0 10 12 22 22 22 P(1, 3) = max{12+0, 0} = 12
3 0 10 12 22 30 32 P(1, 4) = max{12+0, 0} = 12
4 0 10 15 25 30 37 P(1, 5) = max{12+0, 0} = 12
• Start at P(n, W)
• When you go left-up item i has been taken
• When you go straight up item i has not been
taken
21
Overlapping Subproblems
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
0: 1 w W
0 0 0 0 0 0 0 0 0 0 0 0
0
0
i-1 0
i 0
0
n 0
E.g.: all the subproblems shown in grey may
depend on P(i-1, w)
22
Longest Common Subsequence (LCS)
23
Longest Common Subsequence
• Given two sequences
X = x1, x2, …, xm
Y = y1, y2, …, yn
find a maximum length common subsequence
(LCS) of X and Y
• E.g.:
X = A, B, C, B, D, A, B
• Subsequences of X:
– A subset of elements in the sequence taken in order
A, B, D, B, C, D, B, etc.
24
Example
X = A, B, C, B, D, A, B X = A, B, C, B, D, A, B
Y = B, D, C, A, B, A Y = B, D, C, A, B, A
25
Brute-Force Solution
26
LCS Algorithm
c[i 1, j 1] 1 if x[i ] y[ j ],
c[i, j ]
max(c[i, j 1], c[i 1, j ]) otherwise
27
LCS recursive solution
c[i 1, j 1] 1 if x[i ] y[ j ],
c[i, j ]
max(c[i, j 1], c[i 1, j ]) otherwise
• We start with i = j = 0 (empty substrings of x and y)
• Since X0 and Y0 are empty strings, their LCS is
always empty (i.e. c[0,0] = 0)
• LCS of empty string and any other string is empty,
so for every i and j: c[0, j] = c[i,0] = 0
28
LCS recursive solution
c[i 1, j 1] 1 if x[i ] y[ j ],
c[i, j ]
max(c[i, j 1], c[i 1, j ]) otherwise
• When we calculate c[i,j], we consider two cases:
• First case: x[i]=y[j]:
– one more symbol in strings X and Y matches, so the length
of LCS Xi and Yj equals to the length of LCS of smaller
strings Xi-1 and Yi-1 , plus 1
29
LCS recursive solution
c[i 1, j 1] 1 if x[i ] y[ j ],
c[i, j ]
max(c[i, j 1], c[i 1, j ]) otherwise
• Second case: x[i] != y[j]
– As symbols don’t match, our solution is not improved, and
the length of LCS(Xi , Yj) is the same as before (i.e.
maximum of LCS(Xi, Yj-1) and LCS(Xi-1,Yj)
0 b[i, j] = “ ”
m D 0 else
j b[i, j] = “ ”
32
LCS-LENGTH(X, Y, m, n)
1. for i ← 1 to m
2. do c[i, 0] ← 0 The length of the LCS if one of the sequences
3. for j ← 0 to n is empty is zero
4. do c[0, j] ← 0
5. for i ← 1 to m
6. do for j ← 1 to n
7. do if xi = yj
Case 1: xi = yj
8. then c[i, j] ← c[i - 1, j - 1] + 1
9. b[i, j ] ← “ ”
10. else if c[i - 1, j] ≥ c[i, j - 1]
11. then c[i, j] ← c[i - 1, j]
12. b[i, j] ← “↑” Case 2: xi yj
13. else c[i, j] ← c[i, j - 1]
14. b[i, j] ← “←”
15. return c and b Running time: (mn)
33
Example
0 if i = 0 or j = 0
X = A, B, C, B, D, A
c[i, j] = c[i-1, j-1] + 1 if xi = yj
Y = B, D, C, A, B, A
max(c[i, j-1], c[i-1, j]) if xi yj
0 1 2 3 4 5 6
If xi = yj yj B D C A B A
0 xi
b[i, j] = “ ” 0 0 0 0 0 0 0
1 A
0 1
Else if c[i - 0 0 0 1 1
2 B
1, j] ≥ c[i, j-1] 0 1 1 1 1 2 2
b[i, j] = “ ” 3 C 0 1 1 2 2 2 2
4 B
else 0 1 1 2 2 3 3
b[i, j] = “ ”5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
34
4. Constructing a LCS
• Start at b[m, n] and follow the arrows
• When we encounter a “ “ in b[i, j] xi = yj is an element
of the LCS 0 1 2 3 4 5 6
yj B D C A B A
0 xi 0 0 0 0 0 0 0
1 A
0 0 0 0 1 1 1
2 B
0 1 1 1 1 2 2
3 C
0 1 1 2 2 2 2
4 B 0 1 1 2 2 3 3
5 D
0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
35
PRINT-LCS(b, X, i, j)
1. if i = 0 or j = 0 Running time: (m + n)
2. then return
3. if b[i, j] = “ ”
4. then PRINT-LCS(b, X, i - 1, j - 1)
5. print xi
6. elseif b[i, j] = “↑”
7. then PRINT-LCS(b, X, i - 1, j)
8. else PRINT-LCS(b, X, i, j - 1)
37
LCS Algorithm Running Time
O(m*n)
since each c[i,j] is calculated in constant
time, and there are m*n elements in the
array
38
Rock Climbing Problem
• A rock climber wants to get from
the bottom of a rock to the top
by the safest possible path.
Suppose we have a
wall instead of the rock. 4
5 3
2
At every step our climber can reach exactly three
handholds: above, above and to the right and
above and to the left.
• Let C(i,j) be the rating of the hold (i,j). There are three cases for
A(i,j):
• Middle: C(i,j)+min{A(i-1,j-1),A(i-1,j),A(i-1,j+1)}
A(i,j) = C(i,j)+min{A(i-1,j-1),A(i-1,j),A(i-1,j+1)}
Rock climbing: example
C(i,j): A(i,j):
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1
3 2 5 4 8 2
3
4
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2
3
4
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7
3
4
A(2,1)=5+min{,3,2}=7.
Rock climbing: example
C(i,j): A(i,j):
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9
3
4
A(2,1)=5+min{,3,2}=7. A(2,2)=7+min{3,2,5}=9
Rock climbing: example
C(i,j): A(i,j):
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7
3
4
A(2,1)=5+min{,3,2}=7. A(2,2)=7+min{3,2,5}=9
A(2,3)=5+min{2,5,4}=7.
Rock climbing: example
C(i,j): A(i,j):
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7 10 5
3
4
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7 10 5
3 11 11 13 7 8
4
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7 10 5
3 11 11 13 7 8
4 13 19 16 12 15
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7 10 5
3 11 11 13 7 8
4 13 19 16 12 15
2 8 9 5 8 i\j 0 1 2 3 4 5 6
4 4 6 2 3 0 0 0 0 0 0
5 7 5 6 1 1 3 2 5 4 8
3 2 5 4 8 2 7 9 7 10 5
3 11 11 13 7 8
4 13 19 16 12 15
We are done!
Printing out the solution recursively
PrintBest(A,i,j) // Printing the best path ending at (i,j)
if (i==0) OR (j=0) OR (j=m+1)
return;
if (A[i-1,j-1]<=A[i-1,j]) AND (A[i-1,j-1]<=A[i-1,j+1])
PrintBest(A,i-1,j-1);
elseif (A[i-1,j]<=A[i-1,j-1]) AND (A[i-1,j]<=A[i-1,j+1])
PrintBest(A,i-1,j);
elseif (A[i-1,j+1]<=A[i-1,j-1]) AND (A[i-1,j+1]<=A[i-1,j])
PrintBest(A,i-1,j+1);
printf(i,j)