451: Dynamic Programming I

G. Miller, K. Sutner
Carnegie Mellon University
2020/09/08/03
1 Recursion

2 Longest Common Subsequences

3 Scheduling

4 Knapsack
Tired Old Example 2

The usual definition of the Fibonacci numbers uses recursion:

F (0) = 0
F (1) = 1
F (n) = F (n − 1) + F (n − 2) n≥2

In a decent environment, this definition translates directly into code:

fib[0] = 0;
fib[1] = 1;
fib[x_] := fib[x-1] + fib[x-2];

This is rather painless from the programmer’s perspective, and correctness is

obvious (assuming the system is properly implemented).
Recursion, the Idea 3

As Kleene has shown, recursion is the only power tool one needs to obtain all
of computability (abstract computability, much more than what we are looking
for in algorithms).

To make sure the mechanism of recursion works and everything terminates we

need a well-order ≺ on the problem domain D.

Total Recall:
A well-order hD, ≺i is a total order with the special property that for all
∅=
6 X ⊆ D, there is a ≺-minimal element in X.

Alternatively, there is no infinite descending chain

x0  x1  x2  . . .  xn  xn+1  . . .
Prototype 4

defun f (x : D)
if x is minimal
then
return answer
else
compute yi = f (xi ) for i = 1, . . . , k
return answer constructed from the yi and x

Here it is understood that xi ≺ x for all x.

For example, in the natural numbers we can use the standard order xi < x.

Length-lex order on words also works, but lexicographic order fails.

Incidentally, a while ago there were people who thought that recursion has no
place in a real programming language.
What Could Go Wrong? 5

6 5

5 4 4 3

4 3 3 2 3 2 2 1

3 2 2 1 2 1 1 0 2 1 1 0 1 0

2 1 1 0 1 0 1 0 1 0

1 0

The call tree of fib[7], an unmitigated disaster! There is a ton of recompu-

tation going on: e.g., the call to argument 4 appears 3 times. We get
exponentially man calls.
Dynamic Programming 6

There are two standard ways to deal with this issue:

Top-Down Aka memoization: keep the recursive program, but store all
computed results in a hash table, do not recompute.

Bottom-Up Lovingly construct a lookup table by hand, replace recursive

calls by filling in the table in a corresponding order.

Memoization has the great advantage that it requires no further thought,

everything happens automatically.

fib[0] = 0;
fib[1] = 1;
fib[x_] := ( fib[x] = fib[x-1] + fib[x-2] );

Very elegant and easy to use.

But . . . 7

If efficiency is paramount, one may be better off by constructing a table of

known values by hand.

Just to be clear: the Fibonacci example is a bit lame, obviously one could
compute the values
F (0), F (1), F (2), . . . , F (n)
bottom-up by using an array of size n + 1.

Which method is easily improved, since F (i) depends only on F (i − 1) and

F (i − 2) we can easily reduce the space requirement to O(1) in this case, using
plain iteration.
Dynamic Programming 8

Definition
Dynamic programming refers to a type of algorithm that solves large problem
instances by systematically breaking them up into smaller sub-instances, solving
these separately and then combining the partial solutions.

The key difference to divide-and-conquer is that the sub-instances are not

required to be independent: two sub-instances may well contain the same
sub-sub-instances.

Bookkeeping becomes an important task here: we have to keep track of all the
sub-instances and their associated solutions. We tackle this by constructing a
table (usually one- or two-dimensional) for all the sub-instances and their
solutions.
The Key Steps 9

Here are the key ingredients in any dynamic programming algorithm.

Find a suitable notion of (smaller) sub-instance for any given instance.

Find a recursive way to express the solution of an instance in terms of the

solutions of the sub-instance(s).

For bottom-up: organize this recursive computation into a neat table,

possibly with some pre-computation.

It is very helpful when the number of sub-instances is O(1). Alas, in general

one may have to deal with an unbounded number of sub-instances.
DP Tables 10

Again: Constructing an explicit table may be more efficient, but it tends to be

more painful for the programmer.

In the RealWorldTM , building the table comes down to two key problems:

Boundary We have to figure out the correct boundary values and initialize
the table correspondingly.

Dependency We have to fill the table in the correct order: never read a
position without having written to it first.
1 Recursion

2 Longest Common Subsequences

3 Scheduling

4 Knapsack
Longest Common Subsequences 12

Definition
Consider a sequence A = a1 , a2 , . . . , an . A subsequence of A if is any sequence
S = s1 , s2 , . . . , sk such that there is an index sequence
1 ≤ p1 < p2 < . . . < pk ≤ n with

si = api .

Given two sequences A and B, S is a common subsequence if S is a

subsequence of both A and B.
S is a longest common subsequence (LCS) if its length is maximal among all
common subsequences.

So this is an optimization problem: we have to maximize the length of a

common subsequence.
Note: we are dealing with scattered subsequences here, the elements need not
be contiguous.
Example 13

A = 3, 5, 4, 3, 1, 5, 4, 2, 4, 5, 3, 1, 3, 5, 2, 1, 5, 2, 1, 3
B = 4, 3, 5, 1, 5, 3, 1, 3, 3, 2, 2, 2, 5, 4, 4, 4, 4, 5, 4, 4

Claim: An LCS for A and B is

S = 3, 5, 1, 5, 3, 1, 3, 5, 5

This is not exactly obvious.

Verification? 14

It’s not hard to check that S really is a subsequence: use a greedy approach to
find an occurrence of S in A and B:

A = 3, 5, 4, 3, 1, 5, 4, 2, 4, 5, 3, 1, 3, 5, 2, 1, 5, 2, 1, 3

B = 4, 3, 5, 1, 5, 3, 1, 3, 3, 2, 2, 2, 5, 4, 4, 4, 4, 5, 4, 4

But it is far from clear that we have the longest possible subsequence.

Uniqueness is another interesting question.

The Sub-Instances 15

So suppose we have sequences A = a1 , a2 , . . . , am and B = b1 , b2 , . . . , bn .

What are the sub-instances that we need to consider?

It is tempting to just shorten the sequence by chopping off the last item.

To keep notation simple we will write

Aa

for a sequence with last element a and initial segment A.

The Recursion 16

Let a 6= b. Then the recursion is fairly clear:

LCS(Ac, Bc) = LCS(A, B) c

LCS(Aa, Bb) = LCS(A, Bb) or LCS(Aa, B)

Read this with a grain of salt – the “or” in the second case means we don’t
know which sub-instance is going to produce the solution.

Exercise
Explain how to make sense out of these equations.
The Optimal Value 17

Here is a standard trick: first ignore the optimal solution, and only compute the
optimal value. Then, in a second round, construct the optimal solution from
the optimal value.

Let’s write lcs(A, B) for the length of any LCS for A and B.

Then assuming a 6= b:

lcs(Ac, Bc) = lcs(A, B) + 1

lcs(Aa, Bb) = max( lcs(A, Bb), lcs(Aa, B) )

There is nothing fishy about these equations, they are literally correct.

If we use memoizing, this is essentially the solution.

Building a Table 18

Write Ai for the prefix of A of length i: Ai = a1 , a2 , . . . , ai−1 , ai and likewise

for Bj . We compute an (n + 1) × (m + 1) table:

L(i, j) = lcs(Ai , Bj )

Initialization is easy: L(0, j) = L(i, 0) = 0. Update:

(
L(i − 1, j − 1) + 1 if ai = bj ,
L(i, j) =
max(L(i − 1, j), L(i, j − 1)) otherwise.

for all 0 < i, j.

Dependencies 19

Possible approaches:
column-by-column, top-down
row-by-row, left-to-right
sweeping counterdiagonal
Example 20

A = d, d, d, c, c, b, c, a and B = d, c, c, d, b, c, a, c, c, d produces

1 1 1 1 1 1 1 1 1 1
1 1 1 2 2 2 2 2 2 2
1 1 1 2 2 2 2 2 2 3
1 2 2 2 2 3 3 3 3 3
1 2 3 3 3 3 3 4 4 4
1 2 3 3 4 4 4 4 4 4
1 2 3 3 4 5 5 5 5 5
1 2 3 3 4 5 6 6 6 6

We have left off the 0 row and column.

So lcs(A, B) = 6 and an LCS is d, c, c, b, c, a.

Extracting the LCS 21

Read off d, c, c, b, c, a by following diagonal moves.

d c c d b c a c c d
d 1 1 1 1 1 1 1 1 1 1
d 1 1 1 2 2 2 2 2 2 2
d 1 1 1 2 2 2 2 2 2 3
c 1 2 2 2 2 3 3 3 3 3
c 1 2 3 3 3 3 3 4 4 4
b 1 2 3 3 4 4 4 4 4 4
c 1 2 3 3 4 5 5 5 5 5
a 1 2 3 3 4 5 6 6 6 6

Exercise
Explain how to turn this into an algorithm. How would you generate all LCS?
And Memoizing? 22

Here is a typical call structure with memoizing:

Around 75% of the matrix is used.

Small Alphabet 23

About 60% of the matrix is used.

Food for Thought 24

Exercise
What is the least amount of memory needed with memoizing? How important
is this going to be in practical applications?

Exercise
Can memoizing ever require mn entries? How important is this going to be in
practical applications?

Exercise
Implement both methods and try to determine which one is better.

Exercise
How would one go about constructing an actual LCS in the memoizing case?
1 Recursion

2 Longest Common Subsequences

3 Scheduling

4 Knapsack
A Scheduling Problem 26

Suppose you have a resource (such as a conference room) and requests for its
use. The requests have the form (t1 , t2 ) where t1 < t2 ∈ N (the begin and end
time of a meeting).
There are two opposing requirements:
The resource is supposed to be as busy as possible, but
no overlapping requests can be scheduled.

By overlap we mean that the two intervals have an interior point in common, so
(1, 5), (5, 7) would be admissible whereas (1, 5), (4, 6) would not. So we want
to find a non-overlapping schedule that maximizes the use of the resource.
Example 27

Intervals are sorted by right endpoint.

In this case, using (human) geometric intuition, it’s easy to find an optimal
schedule that fills the whole interval by visual inspection.
But How About This? 28
A Solution 29
Less Informally 30

The input for the scheduling problem is given as a list of intervals I1 , I2 , . . . , In

where
Ii = (ai , bi ) ai < bi ∈ N.
Let’s write `k = bk − ak for the length of interval Ik . We’ll be sloppy and also
speak of “interval k”.

Definition
A schedule is a set S ⊆ [n] = {1, 2, . . . , n} of intervals subject to the constraint
that for i 6= j ∈ S intervals Ii and Ij do not overlap.

We need to maximize X
val(S) = `i
i∈S

over all (exponentially many) schedules S.

Again, this is an optimization problem and val is the objective function.

Computing the Optimal Value 31

Again, as usual, we will first focus on computing the optimal value val(S) of
the solution, then handle the actual solution S in a second phase.

Sub-instances are easy: consider only the first k intervals, k ≤ n. For simplicity
we write val(k) for the value of an optimal solution using only the intervals
I1 , I2 , . . . , Ik . So we are only looking at S ⊆ {1, 2, . . . , k}.

Furthermore, let us assume that the Ii = (ai , bi ) are already sorted by right
endpoint:
b1 ≤ b2 ≤ . . . . . . ≤ bn−1 ≤ bn .
If not, this can be ensured by a simple pre-processing step (linear time in the
RealWorldTM ).
Breaking Things Up 32

For the recursion we consider two cases.

Case Out
The easy case is when Ik is not part of the optimal solution for k: then
val(k) = val(k − 1).

Case In
In this case val(k − 1) won’t be of much help in general since there may be
overlap between some of these intervals and Ik .

Recall our convention: intervals are sorted by right endpoint. So, we need to
consider val(k0 ) for some k0 < k: k0 has to be small enough so that no
collisions can occur between the intervals in I1 , . . . , Ik0 and Ik .

To make sure we don’t miss out on any possible solutions we will choose k0
maximal.
More Formally 33

To this end we compute the constraint function c(k):

c(k) = max i < k | Ii and Ik do not overlap

Now we can compute val(k) as follows:

 
val(k) = max val(k − 1), val(c(k)) + `k

Exercise
Explain how this really implements our informal strategy.
Implementation Details 34

How do we compute the c-array?

Recall that we have the intervals sorted by their right endpoints (bi ). We try to
avoid collision with Ik , so we need to worry about the left endpoint ak . So we
can perform binary search to find

max(i < k bi ≤ ak ).

The total cost of this is O(n log n).

Exercise
Find a linear time method to compute the constraint function assuming that we
have interval lists sorted by left endpoints, and also sorted by right endpoints.
Efficiency 35

Given the constraint function one can easily construct a table

val(1), val(2), . . . , val(n − 1), val(n)

using a single pass from left to right in time Θ(n).

So time and space complexity are both Θ(n).

Just to be clear: unlike with the Fibonacci example, we cannot simply dump
the array and use an iterative method.
Search Problem 36

Once the table has been constructed we can now trace back the final answer
val(n) to obtain the corresponding actual solution S. We must have

val(k) = val(k − 1)

or
val(k) = val(c(k)) + `k

The second pass is also linear time.

Exercise
Figure out exactly how to construct an actual optimal solution.
1 Recursion

2 Longest Common Subsequences

3 Scheduling

4 Knapsack
The Knapsack Problem 38

Suppose we have a list of sizes s1 , s2 , . . . , sn and a list of values v1 , v2 , . . . , vn

for n items. We are also given an upper bound B for the allowable total size of
chosen items. Assume everything is positive integers.

The goal is to choose a subset I ⊆ [n] such that

X
si ≤ B
i∈I
X
vi is maximal
i∈I

The decision version of this problem has an additional lower bound for value
and is one of the classical problems shown to be NP-complete in Karp’s seminal
paper. Without getting involved in a philosophical discussion, this probably
means that, in general, there are no good algorithms.
Sub-Instances 39

The right sub-instances are again fairly natural: constrain the selected set I to
[k] and march from k to k + 1.
P
Define val(k, b) = i∈I vi to be the maximal value obtainable with

I ⊆ [k]
X
si ≤ b
i∈I

Clearly, val(0, b) = 0.
The recursion now looks like so:


val(k − 1, b) if sk > b,
val(k, b) =
max val(k − 1, b), vk + val(k − 1, b − sk )
otherwise.
Call Structure 40

Here is a typical call structure. Note the lovely gap on the right.
And Another . . . 41

Sizes increasing, values decreasing.

Dire Warning 42

The table has size n × B and each update is clearly O(1), at least in the
uniform cost model.

At first glance, this may look like a polynomial time algorithm, but it’s most
definitely not: all the numbers are given in binary, including the bound B. So
its size is just log B.

All we get is pseudo-polynomial time, but at least Knapsack is not strongly

NP-complete. So there is a good chance that for small “practical” instances we
have a reasonable algorithm.

By contrast, the similar Bin Packing problem is strongly NP-complete.