Dynamic Programming

∗
Hoa T. Vu

1 Introduction
The main ideas behind dynamic programming are as follows. Dynamic programming is basically
smart recursion. One may think of dynamic programming as recursion with memoization.

2 First example: Fibonacci sequence

The Fibonacci sequence is defined as follows. F0 = 0, F1 = 1. For i > 1, Fi = Fi−1 + Fi−2 . So the
sequence goes like this 0, 1, 1, 2, 3, 5, 8, 13, . . . .
This suggests a recursive algorithm
Algorithm 1: Recursive Fibonacci
1 Function Fibonacci(i)
2 if i = 0 then return 0
3 else if i = 1 then return 1
4 else Return F ibonacci(i − 1) + F ibonacci(i − 2)
The running time of this algorithms is

T (n) = T (n − 1) + T (n − 2) + O(1) ≤ 2T (n − 1) + c

Exercise: Use recursion tree method to show that the running time is Ω(2n ).
Why is the above algorithm wasteful? That’s because there is a lot of repeated computation.
fib(5)

fib(4) fib(3)

fib(3) fib(2) fib(2) fib(1)

fib(2) fib(1) fib(1) fib(0) fib(1)

fib(1) fib(0)
∗
San Diego State University, [email protected]

1
 So, we try to avoid repeated computation by checking if the computation has been done before.
Algorithm 2: Dynamic Programming (top down) for computing the nth Fibonacci number
1 Initialize F [1 . . . n] where each entry is NULL.
2 Function ComputeFibonacci(i)
3 if i = 0 then return 0
4 else if i = 1 then return 1
5 else if F [i − 1] = N U LL then ComputeFibonacci(i − 1)
6 if F [i − 2] = N U LL then ComputeFibonacci(i − 2)
7 Return F [i − 1] + F [i − 2]
8 Call ComputeFibonacci(n).
The above approach is call top-down. We can also fill the array F bottom up deliberately.
Algorithm 3: Dynamic Programming (bottom up) for computing the nth Fibonacci
number
1 Initialize F [1 . . . n] where each entry is NULL.
2 F [0] = 0, F [1] = 1.
3 for i = 2, 3, . . . , n do
4 F [i] = F [i − 1] + F [i − 2].
5 end for

The running time is O(n) which is a great improvement over naive recursion.

3 Shortest path in Directed Acyclic Graphs

We will talk more about graphs and related terminology later. For now, think of a graph as a set of
vertices/nodes V and a set of edges E that connecting them. A directed graph is a graph where
each edge has a direction. Often we use n = |V | and m = |E|.
We consider a directed graph in topological order. For simplicity, assume we have vertices
v1 , v2 , . . . , vn and directed edges in the graph only go from vi to vj where i < j. Note that this
graph has no cycle (why?).
We are also given an array ` where `[vi , vj ] is the length of edges vi → vj . Furthermore,
incoming(vi ) is the list of all vertices that have an edge going toward vi .
For example, see the figure below.

The goal is to compute dist[1 . . . n] where dist[i] is the length of the shortest path from v1 to vi .

• Clearly, dist[1] = 0.
• Consider any node vi . The shortest path from v1 to vi (denoted by v1 vi ) must go through
some vi predecessor vj (ie, j < i where there is an edge vj → vi ) and then then go from vj to
vi using the edge vj → vi (it is possible that vj = v1 if there is an edge from v1 to vi ).

2
 • If v1 vi goes through vj before going from vj to vi using the edge vj → vi , then the length
of that path would be the length of the shortest path v1 vj plus the length of the edge
vj → vi .

• Thus,
dist[vi ] = min (dist[vj ] + `(vj , vi )).
j<i:vj →vi ∈E

The formal algorithm is as follows.

Algorithm 4: Dynamic programming for shortest path in directed acyclic graphs
1 Initialize dist[1 . . . n] where each entry is ∞.
2 Initialize Z[1 . . . n] where each entry is NULL. This array is used to keep track of the actual
path (we will usually omit this for future examples).
3 dist[1] = 0.
4 for i = 2, 3, . . . , n do
5 min = ∞
6 for vj ∈ incoming(vi ) do
7 if dist[vj ] + `(vj , vi ) < min then min = dist[vj ] + `(vj , vi ) and Z[vi ] = vj ;
8 end for
9 end for
Lines 3, 4 executes O(|V |) times. Lines 5, 6 are executed O(|E|) times (ie, once for each edge).
Hence, the running time is O(|V | + |E|).

4 Longest Increasing Subsequence

Given an array A[1 . . . n] what is the longest increasing subsquence (LIS) in A?
Motivation: we may want to test how sorted a database is. If the length of the LIS is close to
n, then A is nearly sorted.
Let us create a graph where node i corresponds to A[i]. If A[i] ≤ A[j] and i ≤ j, then create an
edge i → j with length 1. Otherwise, `(i, j) = N U LL. Note that this graph is also directed acyclic.
Clearly, an increasing subsequence corresponds to a directed path. Hence, the goal is is to find
the longest directed path in the graph we created.
Let L[i] be the length of the longest path ending at vertex i plus 1 (ie, this corresponds to the
length of the longest increasing subsequence ending at A[i]). Again,

• L[1] = 1.

• Consider any node i. The longest path ending at i must go through some vi predecessor vj (ie,
j < i where there is an edge vj → vi ) and then then go from vj to vi using the edge vj → vi .
If i has no predecessor, then the longest path ending at i is just i itself.

• Thus, (
maxj<i (L[j] + 1) if there is some j s.t A[j] ≤ A[j]
L[i] =
1 otherwise

3
 Algorithm 5: Dynamic programming for longest increasing subsequence
1 Initialize L[1 . . . n] where each entry is 1.
2 L[1] = 1.
3 for i = 2, 3, . . . , n do
4 max = 1
5 for j = 1, 2, . . . , i − 1 do
6 if A[j] ≤ A[i] and L[j] + 1 > max then
7 max = L[j] + 1
8 end if
9 end for
10 end for

5 Common theme
There are a few things that we should consider when designing a dynamic programming algorithm.

• Define the dynamic programming table.

• What are the border (base) cases?

• How to fill the table?

6 Edit distance
We are given 2 input strings A and B of length n and m respectively. What is the minimum number
of insertions, deletions, and replacements to transform A into B?

Applications: Comparing documents or DNA sequences.

Example 1. Consider A = SN OW Y and B = SU N N Y . Think of this process as placing gap

into 2 strings to align them together.
For example, the below corresponds to 1) insert U between S and N, 2) Replace O with N, and
3) Delete W. There are 3 edits in total.
S N O W Y
S U N N Y

Example 2. Let’s consider the same 2 strings, but a different set of edits. The below corresponds
to 1) insert S in the beginning 2) Replace S with U, and 3) Delete O, 4) Delete W, 5) insert N.
There are 5 edits in total.
S N O W Y
S U N N Y
The question is to place the gaps that correspond to the smallest number of edits which is the
edit distance between A and B.

4
A DP approach. We define the dynamic programming table as follows.

ED[i, j] = edit distance between A[1 . . . i] and B[1 . . . j].

Consider the optimal gap placing. There are 3 cases for the last column:
• Case 1: deleting A[i] at the end
... ... A[i]
... ...
The cost would be ED[i − 1, j] + 1. This corresponds to first transform A[1 . . . i − 1] to B[1 . . . j]
and then delete A[i] at the end.
• Case 2: inserting B[j] at the end
... ...
... ... B[j]
The cost would be ED[i, j − 1] + 1. This corresponds to first transform A[1 . . . i] to B[1 . . . j − 1]
and then insert B[j] at the end.
• Case 3: substituting A[i] with B[j]
... ... A[i]
... ... B[j]
(
1 if A[i] 6= B[j]
The cost would be ED[i − 1, j − 1] + .
0 if A[i] = B[j]
This corresponds to first transform A[1 . . . i − 1] to B[1 . . . j − 1] and then replace A[i] with
B[j] at the end if A[i] 6= B[j].
(
1 if A[i] 6= B[j]
Let dif f (i, j) = .
0 if A[i] = B[j]
We have the following recursive relationship
ED[i, j] = min{ED[i − 1, j] + 1, ED[i, j − 1] + 1, ED[i − 1, j − 1] + dif f (i, j)}.
Base case ED[i, 0] = i and ED[0, j] = j (Why?).
Algorithm 6: Dynamic programming for Edit distance
1 For each i = 0, 1, 2, . . . , n: ED[i, 0] = i
2 For each j = 0, 1, 2, . . . , m: ED[0, j] = j.
3 for i = 1, 3, . . . , n do
4 for j = 1, 2, . . . , n do
5 ED[i, j] = min{ED[i − 1, j] + 1, ED[i, j − 1] + 1, ED[i − 1, j − 1] + dif f (i, j)}.
6 end for
7 end for
8 Return ED[n, m].

7 Knapsack
Let us assume that we have n items (without repetition) each of which has a weight w[i] and a
value v[i]. We are allowed to carry at most W unit of weight in total. Assume weights and values
are all integers, how to pick items to carry such that we maximize the value?

5
Applications: This arises in various problems where values correspond to utilities and B corre-
sponds to some resource constraints.
The greedy algorithm that picks items with highest value/weight fails to find the best solution.
For example, consider the input W = [3, 2, 2], V = [1.65, 1, 1], B = 4.
Let T [i, j] be the maximum value that we can carry using j unit of weights from items 1, . . . , i.
The optimal solution has to either we pick item i or we don’t.
If we pick item i, the largest total value we could get is T [i − 1, j − w[i]] + V [i]. If we do not
pick item i, the largest total value we could get is T [i − 1, j].
Algorithm 7: Dynamic programming for Knapsack without repetition
1 For each i = 0, 1, 2, . . . , n: T [i, 0] = 0
2 For each j = 0, 1, 2, . . . , B: T [0, j] = 0.
3 for i = 1, 3, . . . , n do
4 for j = 1, 2, . . . , B do
5 if W [i] ≤ j then
6 T [i, j] = max{T [i − 1, j − W [i]] + V [i], T [i − 1, j]}.
7 else
8 T [i, j] = T [i − 1, j]
9 end if
10 end for
11 end for
12 Return T [n, B].
It is easy to see that the running time is O(nB).

8 Subset Sum
Consider a multiset A of n positive integers (represented as an array A[1 . . . n]). We want to output
true if there is a subset of A that sum to a target integer T .
Convention: empty subset sums to 0.
Let S[i, j] = true if and only if there is a subset of A[1 . . . i] that sums to j. Suppose there is a
subset of A that sum to j. There are 2 cases:

• Case 1: This subset contains A[i] which means there must be a subset of A[1 . . . i − 1] that
sum to j − A[i]. Therefore, if this is the case, then S[i − 1, j − A[i]] = true.

• Case 1: This subset does not A[i] which means there must be a subset of A[1 . . . i − 1] that
sum to j. Therefore, if this is the case, then S[i − 1, j] = true.

Therefore, S[i, j] = true if and only if S[i − 1, j − A[i]] = true or S[i − 1, j] = true.

6
 Algorithm 8: Dynamic programming for subset sum
1 Corner cases:
2 S[0, 0] = true
3 S[0, j] = f a`se for all j = 1, 2, . . . , T
4 for i = 1, 3, . . . , n do
5 for j = 1, 2, . . . , T do
6 if A[i] ≤ j then
7 S[i, j] = S[i − 1, j − A[i]] ∨ S[i − 1, j] = true
8 else
9 S[i, j] = S[i − 1, j] = true
10 end if
11 end for
12 end for
13 Return T [n, T ].
It is easy to see that the running time is O(nT ).

9 Longest Palindromic Subsequence

A palindrome is a string that is the same as its reverse, e.g., racecar, tenet.
Input: A string A[1 . . . n]. Output: length of the longest palindromic subsequence (LPS).
Example: ARACABERCKAR. The output should be 7 since the longest palindromic subsequence
is RACECAR.
Let T [i, j] be the length of the longest palindromic subsequence of A[i . . . j].
Corner cases: T [i, i] = 1 and T [i, j] = 0 for i > j.
We have 3 cases:
• Case 1: the LPS of A[i . . . j] contains both A[i] and A[j] (only possible if A[i] = A[j]), then
T [i, j] = 2 + T [i + 1, j − 1].
• Case 2: the LPS of A[i . . . j] does not contain A[i], then T [i, j] = T [i + 1, j].
• Case 3: the LPS of A[i . . . j] does not contain A[j], then T [i, j] = T [i, j − 1].
This problem is simpler if we use top-down DP. The algorithm is as follows.
Algorithm 9: Dynamic Programming (top down) for LPS
1 Initialize T [1 . . . n][1 . . . n] where each entry is NULL.
2 Function F (i, j)
3 if i = j then T [i, j] = 1
4 if i > j then T [i, j] = 0
5 if i < j then
6 If T [i + 1, j − 1] = N U LL, Call F (i + 1, j − 1). If T [i, j − 1] = N U LL, Call
F (i, j − 1). If T [i + 1, j] = N U LL, Call F (i + 1, j). if A[i] = A[j] then
7 T [i, j] = max{2 + T [i + 1, j − 1], T [i + 1, j], T [i, j − 1]}
8 else
9 T [i, j] = max{T [i + 1, j], T [i, j − 1]}
10 end if
11 end if
12 Return F [1, n].

7
 The running time is O(n2 ) since each T [i, j] is called at most once and the non-recursive work is
O(1).
Exercise: How to implement this bottom-up?