Topic 26

Dynamic Programming
"Thus, I thought dynamic programming
was a good name. It was something not
even a Congressman could object to. So I
used it as an umbrella for my activities"
- Richard E. Bellman
 Origins
8 A method for solving complex problems by
breaking them into smaller, easier, sub
problems
8 Term Dynamic Programming coined by
mathematician Richard Bellman in early
1950s
– employed by Rand Corporation
– Rand had many, large military contracts
– Secretary of Defense, Charles Wilson “against research,
especially mathematical research”
– how could any one oppose "dynamic"?

CS314 Dynamic Programming 2

 Dynamic Programming
8 Break big problem up into smaller
problems ...

8 Sound familiar?

8 Recursion?
N! = 1 for N == 0
N! = N * (N - 1)! for N > 0

CS314 Dynamic Programming 3

 Fibonacci Numbers
8 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 114, …
8 F1 = 1
8 F2 = 1
8 FN = F N - 1 + F N - 2

8 Recursive Solution?

CS314 Dynamic Programming 4

 Failing Spectacularly
8 Naïve recursive method
// pre: n > 0
// post: return the nth Fibonacci number
public int fib(int n) {
if (n <= 2)
return 1;
else
return fib(n – 1) + fib (n – 2);
}
8 Clicker 1 - Order of this method?
A. O(1) B. O(log N) C. O(N) D. O(N 2) E. O(2N)
CS314 Dynamic Programming 5
 Failing Spectacularly

CS314 Dynamic Programming 6

 Failing Spectacularly

CS314 Dynamic Programming 7

 Clicker 2 - Failing Spectacularly

8 How long to calculate the 70th Fibonacci

Number with this method?
A. 37 seconds
B. 74 seconds
C. 740 seconds
D. 14,800 seconds
E. None of these
CS314 Dynamic Programming 8
 Aside - Overflow
8 at 47th Fibonacci number overflows int
8 Could use BigInteger class instead
private static final BigInteger one
= new BigInteger("1");

private static final BigInteger two

= new BigInteger("2");

public static BigInteger fib(BigInteger n) {

if (n.compareTo(two) <= 0)
return one;
else {
BigInteger firstTerm = fib(n.subtract(two));
BigInteger secondTerm = fib(n.subtract(one));
return firstTerm.add(secondTerm);
}
}
CS314 Dynamic Programming 9
 Aside - BigInteger
8 Answers correct beyond 46th Fibonacci number
8 Even slower, math on BigIntegers,
object creation, and garbage collection

CS314 Dynamic Programming 10

 Slow Fibonacci
8 Why so slow?
8 Algorithm keeps calculating the same
value over and over
8 When calculating the 40th Fibonacci
number the algorithm calculates the 4th
Fibonacci number 24,157,817 times!!!

CS314 Dynamic Programming 11

 Fast Fibonacci
8 Instead of starting with the big problem
and working down to the small
problems
8 ... start with the small problem and work
up tostatic
public theBigInteger
big problem
fastFib(int n) {
BigInteger smallTerm = one;
BigInteger largeTerm = one;
for (int i = 3; i <= n; i++) {
BigInteger temp = largeTerm;
largeTerm = largeTerm.add(smallTerm);
smallTerm = temp;
}
return largeTerm;
}
CS314 Dynamic Programming 12
 Fast Fibonacci

CS314 Dynamic Programming 13

 Fast Fibonacci

CS314 Dynamic Programming 14

 Memoization
8 Store (cache) results from computations
for later lookup
8 Memoization of Fibonacci Numbers
public class FibMemo {

private static List<BigInteger> lookupTable;

private static final BigInteger ONE

= new BigInteger("1");

static {
lookupTable = new ArrayList<>();
lookupTable.add(null);
lookupTable.add(ONE);
lookupTable.add(ONE);
CS314
} Dynamic Programming 15
 Fibonacci Memoization
public static BigInteger fib(int n) {
// check lookup table
if (n < lookupTable.size()) {
return lookupTable.get(n);
}

// Calculate nth Fibonacci.

// Don't repeat work. Start with the last known.
BigInteger smallTerm
= lookupTable.get(lookupTable.size() - 2);
BigInteger largeTerm
= lookupTable.get(lookupTable.size() - 1);
for(int i = lookupTable.size(); i <= n; i++) {
BigInteger temp = largeTerm;
largeTerm = largeTerm.add(smallTerm);
lookupTable.add(largeTerm); // memo
smallTerm = temp;
}
return largeTerm;
}
 Dynamic Programming
8 When to use?
8 When a big problem can be broken up into sub
problems.
8 Solution to original problem can be
calculated from results of smaller problems.
– larger problems depend on previous solutions
8 Sub problems must have a natural ordering
from smallest to largest (simplest to
hardest)
8 Multiple techniques within DP
CS314 Dynamic Programming 17
 DP Algorithms
8 Step 1: Define the *meaning* of the subproblems
(in English for sure, Mathematically as well if you
find it helpful).
8 Step 2: Show where the solution will be found.
8 Step 3: Show how to set the first subproblem.
8 Step 4: Define the order in which the subproblems
are solved.
8 Step 5: Show how to compute the answer to each
subproblem using the previously computed
subproblems. (This step is typically polynomial,
once the other subproblems are solved.)

CS314 Dynamic Programming 18

 Dynamic Programming Requires:
8 overlapping sub problems:
– problem can be broken down into sub problems
– obvious with Fibonacci
– Fib(N) = Fib(N - 2) + Fib(N - 1) for N >= 3
8 optimal substructure:
– the optimal solution for a problem can be
constructed from optimal solutions of its sub
problems
– In Fibonacci just sub problems, no optimality
– min coins opt(36) = 112 + opt(24) [1, 5, 12]
CS314 Dynamic Programming 19
 Dynamic Programing Example
8 Another simple example
8 Finding the best solution involves finding the
best answer to simpler problems
8 Given a set of coins with values (V1, V2, … VN)
and a target sum S, find the fewest coins
required to equal S
8 What is Greedy Algorithm approach?
8 Does it always work?
8 {1, 5, 12} and target sum = 15 (12, 1, 1, 1)
8 Could use recursive backtracking …
CS314 Dynamic Programming 20
 Minimum Number of Coins
8 To find minimum number of coins to sum to
15 with values {1, 5, 12} start with sum 0
– recursive backtracking would likely start with 15
8 Let M(S) = minimum number of coins to sum
to S
8 At each step look at target sum,
coins available, and previous sums
– pick the smallest option

CS314 Dynamic Programming 21

 Minimum Number of Coins
8 M(0) = 0 coins
8 M(1) = 1 coin (1 coin)
8 M(2) = 2 coins (1 coin + M(1))
8 M(3) = 3 coins (1 coin + M(2))
8 M(4) = 4 coins (1 coin + M(3))
8 M(5) = interesting, 2 options available:
1 + others OR single 5
if 1 then 1 + M(4) = 5, if 5 then 1 + M(0) = 1
clearly better to pick the coin worth 5

CS314 Dynamic Programming 22

 Minimum Number of Coins
8 M(0) = 0 8 M(11) = 2 (1 coin + M(10))
8 M(1) = 1 (1 coin) options: 1, 5
8 M(2) = 2 (1 coin + M(1)) 8 M(12) = 1 (1 coin + M(0))
8 M(3) = 3 (1 coin + M(2)) options: 1, 5, 12
8 M(4) = 4 (1 coin + M(3)) 8 M(13) = 2 (1 coin + M(12))
options: 1, 12
8 M(5) = 1 (1 coin + M(0))
8 M(14) = 3 (1 coin + M(13))
8 M(6) = 2 (1 coin + M(5))
options: 1, 12
8 M(7) = 3 (1 coin + M(6))
8 M(15) = 3 (1 coin + M(10))
8 M(8) = 4 (1 coin + M(7)) options: 1, 5, 12
8 M(9) = 5 (1 coin + M(8))
8 M(10) = 2 (1 coin + M(5))
options: 1, 5

CS314 Dynamic Programming 23

 KNAPSACK PROBLEM -
RECURSIVE BACKTRACKING
AND DYNAMIC PROGRAMMING

CS314 Dynamic Programming 24

 Knapsack Problem
8 A variation of a bin packing problem
8 Similar to fair teams problem from
recursion assignment
8 You have a set of items
8 Each item has a weight and a value
8 You have a knapsack with a weight limit
8 Goal: Maximize the value of the items you
put in the knapsack without exceeding the
weight limit

CS314 Dynamic Programming 25

 Knapsack Example
8 Items: Item
Number
Weight
of Item
Value of
Item
Value
per unit
Weight
1 1 6 6.0
2 2 11 5.5
3 4 1 0.25
8 Weight 4 4 12 3.0
Limit = 8 5 6 19 3.167
6 7 12 1.714

8 One greedy solution: Take the highest ratio

item that will fit: (1, 6), (2, 11), and (4, 12)
8 Total value = 6 + 11 + 12 = 29
8 Clicker 3 - Is this optimal? A. No B.
Yes
Knapsack - Recursive Backtracking
private static int knapsack(ArrayList<Item> items,
int current, int capacity) {

int result = 0;
if (current < items.size()) {
// don't use item
int withoutItem
= knapsack(items, current + 1, capacity);
int withItem = 0;
// if current item will fit, try it
Item currentItem = items.get(current);
if (currentItem.weight <= capacity) {
withItem += currentItem.value;
withItem += knapsack(items, current + 1,
capacity - currentItem.weight);
}
result = Math.max(withoutItem, withItem);
}
return result;
}
Knapsack - Dynamic Programming
8 Recursive backtracking starts with max
capacity and makes choice for items:
choices are:
– take the item if it fits
– don't take the item
8 Dynamic Programming, start with
simpler problems
8 Reduce number of items available
8 … AND Reduce weight limit on knapsack
8 Creates a 2d array of possibilities
CS314 Dynamic Programming 28
 Knapsack - Optimal Function
8 OptimalSolution(items, weight) is best
solution given a subset of items and a weight
limit
8 2 options:
8 OptimalSolution does not select ith item
– select best solution for items 1 to i - 1with weight
limit of w
8 OptimalSolution selects ith item
– New weight limit = w - weight of ith item
– select best solution for items 1 to i - 1with new
weight limit
29
 Knapsack Optimal Function
8 OptimalSolution(items, weight limit) =

0 if 0 items

OptimalSolution(items - 1, weight) if weight of

ith item is greater than allowed weight
wi > w (In others ith item doesn't fit)

max of (OptimalSolution(items - 1, w),

value of ith item +
OptimalSolution(items - 1, w - wi)
CS314 Dynamic Programming 30
 Knapsack - Algorithm
8 Create a 2d array to store Item Weight Value of
Number of Item Item
value of best option given 1 1 6
subset of items and 2 2 11
possible weights 3 4 1
4 4 12
5 6 19

8 In our example 0 to 6 6 7 12

items and weight limits of of 0 to 8

8 Fill in table using OptimalSolution Function

CS314 Dynamic Programming 31

 Knapsack Algorithm
Given N items and WeightLimit

Create Matrix M with N + 1 rows and WeightLimit + 1 columns

For weight = 0 to WeightLimit

M[0, w] = 0

For item = 1 to N
for weight = 1 to WeightLimit
if(weight of ith item > weight)
M[item, weight] = M[item - 1, weight]
else
M[item, weight] = max of
M[item - 1, weight] AND
value of item + M[item - 1, weight - weight of item]
 Knapsack - Table Item
1
Weight
1
Value
6
2 2 11
3 4 1
4 4 12
5 6 19
6 7 12

items / capacity 0 1 2 3 4 5 6 7 8
{} 0 0 0 0 0 0 0 0 0
{1}
{1,2}
{1, 2, 3}
{1, 2, 3, 4}
{1, 2, 3, 4, 5}
{1,CS314
2, 3, 4, 5, 6} Dynamic Programming 33
 Knapsack - Completed Table
items / weight 0 1 2 3 4 5 6 7 8

{}
0 0 0 0 0 0 0 0 0
{1}
[1, 6]
0 6 6 6 6 6 6 6 6
{1,2}
[2, 11]
0 6 11 17 17 17 17 17 17
{1, 2, 3}
[4, 1]
0 6 11 17 17 17 17 18 18
{1, 2, 3, 4}
[4, 12]
0 6 11 17 17 18 23 29 29
{1, 2, 3, 4, 5}
[6, 19]
0 6 11 17 17 18 23 29 30
{1, 2, 3, 4, 5, 6}
[7,CS314
12]
0 6Dynamic
11Programming
17 17 18 23 29 3430
 Knapsack - Items to Take
items / weight 0 1 2 3 4 5 6 7 8

{}
0 0 0 0 0 0 0 0 0
{1}
[1, 6]
0 6 6 6 6 6 6 6 6
{1,2}
[2, 11]
0 6 11 17 17 17 17 17 17
{1, 2, 3}
[4, 1]
0 6 11 17 17 17 17 17 17
{1, 2, 3, 4}
[4, 12]
0 6 11 17 17 18 23 29 29
{1, 2, 3, 4, 5}
[6, 19]
0 6 11 17 17 18 23 29 30
{1, 2, 3, 4, 5, 6}
[7,CS314
12]
0 6Dynamic
11Programming
17 17 18 23 29 3530
 Dynamic Knapsack
// dynamic programming approach
public static int knapsack(ArrayList<Item> items, int maxCapacity) {
final int ROWS = items.size() + 1;
final int COLS = maxCapacity + 1;
int[][] partialSolutions = new int[ROWS][COLS];
// first row and first column all zeros

for(int item = 1; item <= items.size(); item++) {

for(int capacity = 1; capacity <= maxCapacity; capacity++) {
Item currentItem = items.get(item - 1);
int bestSoFar = partialSolutions[item - 1][capacity];
if( currentItem.weight <= capacity) {
int withItem = currentItem.value;
int capLeft = capacity - currentItem.weight;
withItem += partialSolutions[item - 1][capLeft];
if (withItem > bestSoFar) {
bestSoFar = withItem;
}
}
partialSolutions[item][capacity] = bestSoFar;
}
}
return partialSolutions[ROWS - 1][COLS - 1];
}
 Dynamic vs. Recursive
Backtracking Timing Data
Number of items: 32. Capacity: 123
Recursive knapsack. Answer: 740, time: 10.0268025
Dynamic knapsack. Answer: 740, time: 3.43999E-4

Number of items: 33. Capacity: 210

Recursive knapsack. Answer: 893, time: 23.0677814
Dynamic knapsack. Answer: 893, time: 6.76899E-4

Number of items: 34. Capacity: 173

Recursive knapsack. Answer: 941, time: 89.8400178
Dynamic knapsack. Answer: 941, time: 0.0015702

Number of items: 35. Capacity: 93

Recursive knapsack. Answer: 638, time: 81.0132219
Dynamic knapsack. Answer: 638, time: 2.95601E-4

CS314 Dynamic Programming 37

 Clicker 4
8 Which approach to the knapsack problem
uses more memory?
A. the recursive backtracking approach
B. the dynamic programming approach
C. they use about the same amount of memory

CS314 Dynamic Programming 38