Dynamic Programming

CSE 301: Combinatorial Optimization

Fibonacci Numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …
Computing the nth Fibonacci number recursively:
F(n) = F(n-1) + F(n-2), when n>1
F(0) = 0
F(1) = 1

Top-down (recursive) algorithm F(n)

Algorithm Fibo(n):
if n ≤ 1 then
return n F(n-1) + F(n-2)
else
return Fibo(n-1)+Fibo(n-2)
F(n-2) + F(n-3) F(n-3) + F(n-4)
Time Complexity of Top-Down Algorithm

F(9)
F(8) F(7)

h=n F(7) F(6) F(6) F(5) h = n/2

F(6) F(5) F(5) F(4) F(5) F(4) F(4) F(3)

Time complexity between 2n/2 and 2n

Recursion Tree of Top-Down Algorithm
Why is the top-down approach so inefficient?
Recomputes many sub-problems (a.k.a. overlapping subproblem).
F(5)
+

F(4) F(3)
+ +

F(3) F(2) F(2) F(1)

+ + +

F(2) F(1) F(1) F(0) F(1) F(0)

+

F(1) F(0)

Same subproblem is
solved multiple times
A Top-down Algorithm that memorizes previous
solutions (memoized top-down approach)
Algorithm Fib(n):
for i = 0 to n //Non-memoized Top-Down Approach
Algorithm Fibo(n):
F[i] = -∞
if n ≤ 1 then
Fib_rec(n) return n
else
Algorithm Fib_rec(n): return Fibo(n-1)+Fibo(n-2)
if F[n]≥0 then
return F[n]
if n ≤ 1 then
F[n] = n
else
F[n] = Fib_rec(n-1)+Fib_rec(n-2)
return F[n]
Time complexity?
Each of the F[i] values (i={0, 1,…, n}) is computed only once. So time is O(n).
Recursion Tree of Memoized Algorithm
Fib(5)

Fib_rec(5)
+

Fib_rec(4) Fib_rec(3)
+

Fib_rec(3)
Fib_rec(2)
+

Fib_rec(2) Fib_rec(1)
+

Fib_rec(1) Fib_rec(0)
A Bottom-up (Iterative) Algorithm
Algorithm fib_bu(n):
for i = 0 to n
if i ≤ 1 then
F[i] = i
else
F[i] = F[i-1] + F[i-2]
return F[n]

Time complexity?
The for loop here runs O(n) times and all the other (first and last)
statements take O(1) time. So total time is O(n).
Summery
Problem with the recursive Fib algorithm:
Each subproblem was solved for many times!

Solution: avoid solving the same subproblem more than once

(1) pre-compute all subproblems that may be needed later (bottom-up)
(2) Compute on demand, but memorize the solution to avoid recomputing
(memoized top-down)

Can you always speedup a recursive algorithm by memoization

(memoized top-down) or pre-computation (bottom-up)?
E.g., merge sort
No. since there is no overlap between the two sub-problems
Dynamic Programming
• Dynamic Programming (DP) is an algorithm design
technique for optimization problems.

• Like divide and conquer (D&C), DP solves problems by

combining solutions to sub-problems.

• Unlike D&C, sub-problems are not independent.

• Sub-problems may share sub-sub-problems
Dynamic Programming
The term Dynamic Programming comes from Control Theory,
not computer science. Programming refers to the use of
tables (arrays) to construct a solution.
In dynamic programming we usually reduce time by
increasing the amount of space
We solve the problem by solving sub-problems of increasing
size and saving each optimal solution in a table (usually).
The table is then used for finding the optimal solution to
larger problems.
Time is saved since each sub-problem is solved only once.
Designing a DP Algorithm
1. Characterize the structure of an optimal solution.
2. Recursively define the value of an optimal solution.
3. Compute the value of an optimal solution in a bottom up
fashion.
4. Construct an optimal solution from computed
information.
Example: NSU Number
• Almost like Fibonacci number; with a simple difference: we take the summation
of last three NSU numbers, instead of last two.
• Assuming A(n) represents the n-th NSU number,
A(n) = n , if n <= 2
= A(n-1) + A(n-2) +A(n-3) , otherwise
A(0) = 0
A(1) = 1
A(2) = 2
A(3) = 0+1+2 = 3
A(4) = 1+2+3 = 6
A(5) = 2+3+6 = 11
….

• Design (i) a memoized top-down algorithm and (ii) a bottom-up algorithm to

compute A(n).
• Show that the problem of computing A(n) satisfies overlapping subproblem
property.
Example:
n n! n choose r (combinations)
   ,1  r  n (closed form )
• Given
 r  r!(n  r )! many different sets of size r can be chosen?
n things, how
 n   n  1  n  1
       ,1  r  n (recursive )
 r   r  1  r 
with base cases:
n n
   n and    1
1 n

Assuming C(n,r) represents the value of n choose r,

C(n,r) = n , if r =1
=1 , if r = n or r = 0
= C(n-1,r-1) + C(n-1, r) , otherwise
Combinations: Top Down Algorithm
Algorithm Comb(n,r):

if(r == 1) // base case 1

return n
else if r = n or r = 0 // base case 2
return 1
else //general case
return Comb(n-1, r-1)+Comb(n-1, r)
Top Down Memoized Algorithm
Algorithm Comb(n,r):
for i=0 to n
for j=0 to min(i,r)
• C[i,j] = -∞
• return rec(n,r)

• Algorithm rec(n,r):
• if C[n,r] != -∞ then
• return C[n,r]
• if r = 1 then // base case 1
• C[n,r] = n
• else if r = n or r = 0 then // base case 2
• C[n,r] = 1
• else //general case
C[n,r] = rec(n-1, r-1)+rec(n-1, r)
return C[n,r]
Bottom-up Algorithm
Algorithm Combination (n,r):

for i = 0 to n
for j = 0 to min(i,r)
if j = 1 then
• C[i,j] = i
• else if j = i or j = 0 then
• C[i,j] = 1
• else //general case
C[i,j] = C[i-1, j-1]+C[i-1, j]

return C[n,r]

Time Complexity:
Θ (nr), for both memorized top-down & bottom-up algorithms
Example: Rod Cutting
You are given a rod of length n ≥ 0 (n in inches)
A rod of length i inches will be sold for pi dollars

Cutting is free (simplifying assumption)

Problem: given a table of prices pi determine the maximum
revenue rn obtainable by cutting up the given rod (of length
n) and selling the pieces.
Length i 1 2 3 4 5 6 7 8 9 10
Price pi 1 5 8 9 10 17 17 20 24 30
Example: Rod Cutting
Problem: given a table of prices pi determine the maximum revenue rn
obtainable by cutting the given rod (of length n) and selling the pieces.
Can we use a greedy algorithm (like fractional knapsack) that always
takes the length with highest price/length value?
Length i 1 2 3 4
Price pi 1 20 33 36
Value pi/i 1 10 11 9
No, for e.g. by greedy algorithm, optimal revenue of a 4 inch rod
should be 33+1=34 which we get by cutting a rod of length 3 first
(which has the maximum value of 11) and then taking the remaining
rod of length 1 (i.e., the cutting lengths here are in order: (3,1)).
However, the optimal cutting is: (2,2) which gives us optimal revenue
of 20+20 = 40.
Example: Rod Cutting
Can we use a brute-force/exhaustive algorithm that tries all possible
cuts of a rod of length n?

Question: in how many different ways can we cut a rod of length n?

For a rod of length 4:

24 - 1 = 2 3 = 8

Yes but the cost would be too high!!! There are 2n-1 possible ways to cut a rod of
length n (Exponential). We cannot try all possibilities for "large“ n; the exhaustive
approach isn’t practically feasible.
Example: Rod Cutting
Let us find a way to solve the problem recursively:
Let rn = the maximum revenue obtainable from a rod of length n

How can we construct a recurrence relation for rn?

Advice: when you don’t know what to do next, start with a simple
example and hope that some idea will click…
Example: Rod Cutting

Maximum revenue obtainable by cutting a rod of length n, rn = ? (for n = 1)

r1 = p1 = 1
Example: Rod Cutting

Maximum revenue obtainable by cutting a rod of length n, rn = ? (for n = 2)

r2 = max(p2, p1+r1) = max(5, 1+1) = 5

Example: Rod Cutting

Maximum revenue obtainable by cutting a rod of length n, rn = ? (for n = 3)

r3 = max(p3, p1+r2 , p2+r1) = max(8, 1+5, 5+1) = 8

Example: Rod Cutting

Maximum revenue obtainable by cutting a rod of length n, rn = ? (for n =4)

r4 = max(p4, p1+r3, p2+r2, p3+r1) = max(9, 1+8, 5+5, 8+1) = 10

Example: Rod Cutting
In general, for any n:
rn = max(pn, p1+ rn-1, p2+ rn-2, p3+ rn-3, …, pn-1+ r1)

Þ rn = max(pn+r0, p1+ rn-1, p2+ rn-2, p3+ rn-3, …, pn-1+ r1) :[Let, r0 = 0]

Þ rn = max1≤i≤n(pi + rn-i)
In other words,
• maximal revenue rn is obtained by cutting the rod into smaller pieces of lengths: i

and (n-i), for some value of i (for which pi+rn-i is maximum) where1≤i≤n and

 ri = pi, i.e., the piece of length i need no more cut (because cutting it into smaller
pieces will not increase its revenue) whereas
 the piece of length (n-i) may need more cut; but we have already calculated the
maximal revenue, rn-i of a rod of length (n-i) before calculating rn.. We can use that
Optimal Substructure Property of Rod Cutting Problem

 The recurrence relation: rn = max1≤i≤n(pi + rn-i) shows that the Rod

Cutting problem has optimal substructure property:

an optimal solution to the problem (here rn) can be calculated using

optimal solutions to its sub problems (here rn-i).

This recurrence relation can be implemented as a simple top-down

recursive procedure
Top Down algorithm for Rod Cutting

Computing the recursion leads to re-computing some numbers (overlapping

subproblems) again and again – how many?
Overlapping subproblem property of Rod Cutting Problem

Let’s call Cut-Rod(p, 4), to see the effects on a simple case:

T(n) = total # of calls made to CUT-ROD for an initial call of CUT-ROD(p,n)

= The # of nodes for a recursion tree corresponding to a rod of size n = ?

Solution: CUT-ROD(p,n) calls CUT-ROD(p, n-i) for i = 1,2,3, …, n, i.e., CUT-ROD(p,n)

calls CUT-ROD(p, j) for j = 0,1,2, …, n-1

T 0   c,T (n)  c   j 0 T ( j )  O(2 ),n  1.

n 1 n
How to decrease time?
We have a problem: “reasonable size” problems are not solvable in “reasonable
time” (but, in this case, they are solvable in “reasonable space”).

Specifically:
• Note that navigating the whole tree requires 2n function calls
•However, no more than n different values need to be computed or used.

We exploit these observations to come up with two ways to decrease time:

• Memoized Top-down algorithm: store the value of rk (once computed by a call
of CUT-ROD) in a table and reuse it in later calls of CUT-ROD as needed. This
technique allows us to compute rk only once.

The technique of storing and reusing values computed by a previous call of a

recursive function is called “memoizing” (i.e., writing yourself a memo).

• Bottom Up Algorithm: Compute small subproblems first, then gradually solve

larger and larger subproblems by using the pre-computed solutions of smaller
subproblems. For e.g., for rod-cutting problem: compute r0, r1, r2, …., rn
Memoized Top-down Algorithm for
Rod-Cutting Problem
X X X

X
Memoized Top-down Algorithm for
Rod-Cutting Problem

Time: Θ(n2)
Each r[i], for i = 1,2,3, …, n, is computed only once and this
computation takes Θ(n) time (due to the for loop in lines 6-7).
As we compute n values of r[i] (for i=1,2,…,n), total time is Θ(n2)
 Bottom Up Algorithm

Bottom-up approach also takes Θ(n2) time.

Why??
The most heavily executed statement in bottom-up approach is line 6 which is executed
j times for each value of jɛ {1, 2,…, n}. So it is executed 1+2+…+n = Θ(n2) times.
Simulation: Rod Cutting
Length i 1 2 3 4 5 6 7
Price pi 1 5 8 9 10 12 17

We begin by constructing (by hand) the optimal solutions for i = 0, 1, …, 10:

r0 = 0
r1 = p1 = 1 (no cuts)
r2 = max(p2, p1+r1) = max(5,1+1) = 5 (no cuts)
r3 = max(p3, p2+r1, p1+r2) = max(8,5+1,1+5) = 8 (no cuts)
r4 = max(p4, p3+r1, p2+r2, p1+r3) = max(9, 8+1, 5+5, 1+8) = 10
r5 = ?
r6 = ?
r7 = ?
Simulation: Rod Cutting
Length i 1 2 3 4 5 6 7
Price pi 1 5 8 9 10 12 17

We begin by constructing (by hand) the optimal solutions for i = 1, …, 10:

r0 = 0
r1 = p1 = 1 (no cuts)
r2 = max(p2, p1+r1) = max(5,1+1) = 5 (no cuts)
r3 = max(p3, p2+r1, p1+r2) = max(8,5+1,1+5) = 8 (no cuts)
r4 = max(p4, p3+r1, p2+r2, p1+r3) = max(9, 8+1, 5+5, 1+8) = 10
r5 = max{p5, p4+r1, p3+r2, p2+r3, p1+r4}
= max(10, 9+1, 8+5, 5+8, 1+10) = 13
r6 = max{p6, p5+r1, p4+r2, p3+r3, p2+r4, p1+r5}
= max{12,11,14,16,15,14} = 16
r7 = … = p2 + r5 = 5 + 13 = 18
Now we know maximum revenue obtainable from a 7 inch rod;
but we don’t know how to cut the rod to get that much revenue
Reconstructing a Solution

Save cutting points

Reconstructing a Solution
Simulation: Rod Cutting
Length i 1 2 3 4 5 6 7
Price pi 1 5 8 9 10 12 17
si 1 2 3 2 2 3 2

We begin by constructing (by hand) the optimal solutions for i = 1, …, 10:

r0 = 0
r1 = p1 = 1 (no cuts)
r2 = max(p2, p1+r1) = max(5,1+1) = 5 (no cuts)
r3 = max(p3, p2+r1, p1+r2) = max(8,5+1,1+5) = 8 (no cuts)
r4 = max(p4, p3+r1, p2+r2, p1+r3) = max(9, 8+1, 5+5, 1+8) = 10
r5 = max{p5, p4+r1, p3+r2, p2+r3, p1+r4}
= max(10, 9+1, 8+5, 5+8, 1+10) = 13
r6 = max{p6, p5+r1, p4+r2, p3+r3, p2+r4, p1+r5}
= max{12,11,14,16,15,14} = 16
r7 = p2 + r5 = 5 + 13 = 18
Simulation: Rod Cutting
Length i 1 2 3 4 5 6 7
Price pi 1 5 8 9 10 12 17
si 1 2 3 2 2 3 2

Optimal Cutting lengths of a rod of length 7:

s7 = 2
Þ 2, optimal cutting of a rod of remaining length (7-2) is: s7-2 = s5
Þ s5 = 2, optimal cutting of a rod of remaining length (5-2) is: s5-2 = s3
Þ s3= 3, remaining length = 3-3 = 0; so stop

Therefore optimal cutting lengths of a rod of length 7 is: 2,2,3

Practice: Find optimal cutting and max. revenue for this rod cutting
problem

Length i 1 2 3 4 5 6 7 8
Price pi 1 5 8 9 10 12 17 19
si

r8 = max (p8, p7+r1, p6+r2, p5+r3, p4+r4, p3+r5, p2+r6, p1+r7)

= max(19, 17+1, 12+5, 10+8, 9+10, 8+13, 5+16, 1+18)
= max(19, 18, 17, 18, 19, 21, 21, 19) = 21
Multiple Rod Cutting
You are given m rods of different lengths
A rod of length i inches will be sold for pi dollars

Problem: given a table of prices pi determine the maximum

revenue obtainable by cutting m rods and selling the pieces.
Length i 1 2 3 4 5 6 7 8 9 10
Price pi 1 5 8 9 10 17 17 20 24 30

Idea: apply the DP algorithm for rod-cutting on the longest rod and
then use its solution (r,s) to compute optimal cutting and optimal
revenue of each rod.
Rod Cutting having cutting-cost
You are given a rod of length n
A rod of length i inches will be sold for pi dollars
Each cut costs you c dollars.
Problem: given a table of prices pi determine the maximum
revenue obtainable by cutting a rod of length n and selling the
pieces.
Length i 1 2 3 4 5 6 7 8 9 10
Price pi 1 5 8 9 10 17 17 20 24 30
Idea: Add cutting cost in the recurrence relation of rod-cutting:
rn = max1≤i≤n(pi + rn-i - c) for n>0; r0 = 0
 Chocolate Cutting/Breaking
You are given a Mimi chocolate having m*n blocks
In each step, we can break one piece of chocolate into two
rectangular pieces horizontally/vertically (so no L shaped
piece ever appears). A chocolate of i*j blocks can be sold for
pi,j dollars. Given pi,j determine the maximum revenue rm,n
obtainable by breaking a chocolate with m*n blocks and then
selling the pieces.
Price pi,j 1 2 3 4 5 6 7 8 9 10
1 1 5 8 9 10 17 17 20 24 30
2 5 6 18 22 31 35 37 39 40 45
3 8 18 22 34 37 39 42 43 45 48
4 9 22 34 40 44 47 48 50 52 53
 Chocolate Cutting
Recurrence relation:

{ }
rm,n = max max{pi,n+rm-i,n}, max{pm,j+rm,n-j} , when n>0
1≤i≤n 1≤j≤n

What are the base cases?

j n-j
A possible cut
i

m-i
 Chocolate Cutting
Recurrence relation:

{ }
rm,n = max max{pi,n+rm-i,n}, max{pm,j+rm,n-j} , when n>0
1≤i≤n 1≤j≤n

Can you write memoized top-down and bottom-up algorithms

to solve this problem?
j n-j
A possible cut
i

m-i
Properties of DP Problems
How can one tell if a DP algorithm will be able to solve an optimization
problem i.e., whether a problem is a “DP problem”?
There are 2 properties exhibited by most DP Problems (problems
solvable via a DP algorithm):
1. Overlapping Subproblem Property: To solve a large problem, the
same subproblem is required to be solved again & again
• It implies that a we can save time by solving each subproblem exactly once
& saving that in a table, so that whenever that solution is required later, we
can simply look up that table and reuse that value
2. Optimal Sub-structure Property: Optimal solution of a problem
depends on optimal solution(s) of its subproblem(s)
• It implies that the optimal solution(s) of subproblem(s) can be combined
together to obtain the optimal solution of the problem itself