Dynamic Programming

Course Code: CSC 2211 Course Title: Algorithms

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: F01 Week No: 08 Semester Summer 2022-2023

:
Lecturer: Pritam Khan Boni, [email protected]
Lecture Outline

1. Introduction to Dynamic Programming

2. Elements of Dynamic Programming
3. Designing a Dynamic Programming Algorithm
4. 0/1 Knapsack Problem
 …Fibonacci Numbers

? F(n)= F(n-1)+ F(n-2)

? F(0) =0, F(1) =1

? 0, 1, 1, 2, 3, 5, 8, 13, 21, 34 …
FibonacciR(n)
FibonacciR(n)
01 if nn ≤≤11 then
01 if then return
return nn
02
02 else
else return
return FibonacciR(n-1)
FibonacciR(n-1) ++ FibonacciR(n-2)
FibonacciR(n-2)

? Straightforward recursive procedure is slow!

 …Fibonacci Numbers

F(6) =
8

F(5 F(4
) )
F(4 F(3 F(3 F(2
) ) ) )
F(3 F(2 F(2 F(1 F(2 F(1 F(1 F(0
) ) ) ) ) ) ) )
F(2 F(1 F(1 F(0 F(1 F(0 F(1 F(0
) ) ) ) ) ) ) )
F(1 F(0
) )

? We keep calculating the same value over and over!

? Subproblems are overlapping – they share sub-subproblems
 …Fibonacci Numbers

? How many summations are there in F(n)?

? F(n) = F(n – 1) + F(n – 2) + 1
? F(n) ≥ 2F(n – 2) +1 and F(1) = F(0) = 0
? Solving the recurrence we get
F(n) ≥ 2n/2 – 1 ≈ 1.4n

? Running time is exponential!

 …Fibonacci Numbers
? We can calculate F(n) in linear time by remembering solutions
to the solved sub-problems (= dynamic programming).
? Compute solution in a bottom-up fashion

? Trade space for time!

Fibonacci(n)
Fibonacci(n)
01
01 F[0]←0
F[0]←0
02
02 F[1]←1
F[1]←1
03 for ii ←
03 for ← 22 to
to nn do
do
04
04 F[i] ←
F[i] ← F[i-1]
F[i-1] ++ F[i-2]
F[i-2]
05
05 return
return F[n]
F[n]
 …Fibonacci Numbers
? In fact, only two values need to be remembered
at any time!

FibonacciImproved(n)
FibonacciImproved(n)
01 if nn ≤
01 if ≤11 then
then return
return nn
02 Fim2 ←0
02 Fim2 ←0
03 Fim1 ←1
03 Fim1 ←1
04 for ii ←
04 for ← 22 to
to nn do
do
05
05 Fi ←
Fi ← Fim1
Fim1 ++ Fim2
Fim2
06
06 Fim2 ←
Fim2 ← Fim1
Fim1
07
07 Fim1 ←
Fim1 ← Fi
Fi
05
05 return
return FiFi
 History

? Dynamic programming
? Invented in the 1957 by Richard Bellman as a general method for
optimizing multistage decision processes
? The term “programming” refers to a tabular method.
? Often used for optimization problems.
 Dynamic Programming

🡽 Solves problems by combining the solutions to sub problems that contain common
sub-sub-problems.
🡽 Difference between DP and Divide-and-Conquer
🡽 Using Divide and Conquer to solve these problems is inefficient as the same
common sub-sub-problems must be solved many times.
🡽 DP will solve each of them once and their answers are stored in a table for future
reference.
 Intuitive Explanation

? Given a problem P, obtain a

sequence of problems Q0, Q1,
…., Qm, where:
? You have a solution to Q0

? The solution to a problem Qj,

j > 0, can be obtained from
solutions to problems Q0...Qk,
k < j, that appear earlier in the
“sequence”.
 Elements of Dynamic Programming

? DP is used to solve problems with the following characteristics:

? Optimal sub-structure (Principle of Optimality)
? an optimal solution to the problem contains within its optimal
solutions to sub-problems.
? Overlapping sub problems
? there exist some places where we solve the same sub problem more
than once
Example: Fibonacci Numbers

F(6) =
8

F(5 F(4
) )
F(4 F(3 F(3 F(2
) ) ) )
F(3 F(2 F(2 F(1 F(2 F(1 F(1 F(0
) ) ) ) ) ) ) )
F(2 F(1 F(1 F(0 F(1 F(0 F(1 F(0
) ) ) ) ) ) ) )
F(1 F(0
) )

? We keep calculating the same value over and over!

? Subproblems are overlapping – they share sub-subproblems
 Steps to Designing a
Dynamic Programming Algorithm

1. Characterize optimal sub-structure

2. Recursively define the value of an optimal solution

3. Compute the value bottom up

4. (if needed) Construct an optimal solution

 1.Optimal Sub-Structure

? An optimal solution to the problem contains optimal solutions to sub-

problems
? Solution to a problem:
? Making a choice out of a number of possibilities (look what possible choices there can be)

? Solving one or more sub-problems that are the result of a choice (characterize the space of
sub-problems)

? Show that solutions to sub-problems must themselves be optimal for the whole
solution to be optimal.
 2. Recursive Solution

? Write a recursive solution for the value of an optimal solution.

? Show that the number of different instances of sub-problems is

bounded by a polynomial.
 3. Bottom Up

? Compute the value of an optimal solution in a bottom-up fashion,

so that you always have the necessary sub-results pre-computed
(or use memoization)

? Check if it is possible to reduce the space requirements, by

“forgetting” solutions to sub-problems that will not be used any
more
 4. Construct Optimal Solution

? Construct an optimal solution from computed

information (which records a sequence of choices
made that lead to an optimal solution)
Knapsack Problem
 The Knapsack Problem

? The famous knapsack problem:

? A thief breaks into a museum. Fabulous paintings, sculptures,
and jewels are everywhere. The thief has a good eye for the
value of these objects, and knows that each will fetch hundreds
or thousands of dollars on the clandestine art collector’s market.
But, the thief has only brought a single knapsack to the scene of
the robbery, and can take away only what he can carry. What
items should the thief take to maximize the haul?
 0-1 Knapsack problem

? Given a knapsack with maximum capacity W, and a set S

consisting of n items

? Each item i has some weight wi and benefit value bi (all wi , bi

and W are integer values)

? Problem: How to pack the knapsack to achieve maximum

total value of packed items?
Knapsack Size = 25

Knapsack size = 60
0-1 Knapsack problem: a picture
Weight Benefit value
Item wi bi
s 2
3

4
This is a knapsack 3

Max weight: W = 20 4 5

5 8
W = 20

9 10
 The Knapsack Problem

? More formally, the 0-1 knapsack problem:

? The thief must choose among n items, where the ith item worth vi
dollars and weighs wi pounds
? Carrying at most W pounds, maximize value
? Note: assume vi, wi, and W are all integers
? “0-1” b/c each item must be taken or left in entirety

? A variation, the fractional knapsack problem:

? Thief can take fractions of items
? Think of items in 0-1 problem as gold ingots, in fractional problem as
buckets of gold dust
 0-1 Knapsack problem
? Problem, in other words, is to find

● The problem is called a “0-1” problem,

because each item must be entirely
accepted or rejected.
● Just another version of this problem is the
“Fractional Knapsack Problem”, where we
can take fractions of items.
0-1 Knapsack problem: brute-force approach

Let’s first solve this problem with a straightforward algorithm

? Since there are n items, there are 2n possible
combinations of items.
? We go through all combinations and find the one with
the most total value and with total weight less or equal to
W
? Running time will be O(2n)
0-1 Knapsack problem: brute-force approach

? Can we do better?

? Yes, with an algorithm based on dynamic programming

? We need to carefully identify the subproblems

Let’s try this:

If items are labeled 1..n, then a
subproblem
would be to find an optimal solution for
Sk = {items labeled 1, 2, .. k}
 Defining a Subproblem

If items are labeled 1..n, then a subproblem would be to find an

optimal solution for Sk = {items labeled 1, 2, .. k}

? This is a valid subproblem definition.

? The question is: can we describe the final solution (Sn ) in

terms of subproblems (Sk)?

? Unfortunately, we can’t do that. Explanation follows….

 Defining a Subproblem
Weight Benefit
w1 =2 w2 =3 w3 =4 w4 =5
b1 =3 b2 =4 b3 =5 b4 =8
Ite wi bi
?
m
#1 2 3
Max weight: W = 20
S4 2 3 4
For S4:
Total weight: 14; S5
total benefit: 20 3 4 5

4 5 8

5 9 10
w1 =2 w3 =4 w4 =5 w5 =9
b1 =3 b3=5 b4 =8 b5 =10

For S5: Solution for S4 is

Total weight: 20
total benefit: 26
not part of the
solution for S5!!!
 Defining a Subproblem
(continued)

? As we have seen, the solution for S4 is not part of the

solution for S5

? So, our definition of a subproblem is flawed and we

need another one!
? Let’s add another parameter: w, which will represent
the exact weight for each subset of items
? The subproblem then will be to compute B[k,w]
Recursive Formula for subproblems
● Recursive formula for subproblems:

? It means, that the best subset of Sk that has total weight

w is one of the two:

1) the best subset of Sk-1 that has total weight w, or

2) the best subset of Sk-1 that has total weight w-wk plus the
item k
 Recursive Formula

? The best subset of Sk that has the total weight w, either

contains item k or not.

? First case: wk>w. Item k can’t be part of the solution, since if

it was, the total weight would be > w, which is unacceptable

? Second case: wk <=w. Then the item k can be in the solution,

and we choose the case with greater value
The Knapsack Problem And Optimal Substructure

? Both variations exhibit optimal substructure

? To show this for the 0-1 problem, consider the most valuable
load weighing at most W pounds
? If we remove item j from the load, what do we know about the
remaining load?
? A: remainder must be the most valuable load weighing at most
W - wj that thief could take from museum, excluding item j
 Solving The Knapsack Problem

? The optimal solution to the fractional knapsack problem can be

found with a greedy algorithm
? How?

? The optimal solution to the 0-1 problem cannot be found with

the same greedy strategy
? Greedy strategy: take in order of dollars/pound
? Example: 3 items weighing 10, 20, and 30 pounds, knapsack can
hold 50 pounds
? Suppose item 2 is worth $100. Assign values to the other items so
that the greedy strategy will fail
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 The Knapsack Problem: Greedy Vs. Dynamic

? The fractional problem can be solved greedily

? The 0-1 problem cannot be solved with a greedy approach

? As you have seen, however, it can be solved with dynamic
programming
 Fractional-knapsack

? Greedy-fractional-knapsack (w, v, W)
? FOR i =1 to n
do x[i] =0
weight = 0
while weight < W
do i = best remaining item
IF weight + w[i] ≤ W
then x[i] = 1
weight = weight + w[i]
else
x[i] = (w - weight) / w[i]
weight = W
return x
 0-1 Knapsack Algorithm
for w = 0 to W
B[0,w] = 0
for i = 0 to n
B[i,0] = 0
for w = 0 to W
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Running time

for w = 0 to W O(W)
B[0,w] = 0

for i = 0 to n
Repeat n
B[i,0] = 0 times
for w = 0 to W O(W)
< the rest of the code >
What is the running time of this
algorithm?
O(n*W)
Remember that the brute-force algorithm takes O(2n)
 Example

Let’s run our algorithm on the

following data:

n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
 Example (2)
i 0 1 2 3 4
W
0 0

1 0

2 0

3 0
4 0

5 0

for w = 0 to W
B[0,w] = 0
 Example (3)
i 0 1 2 3 4
W
0 0 0 0 0 0

1 0

2 0

3 0
4 0

5 0

for i = 0 to n
B[i,0] = 0
 Example (4) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 i=1 4: (5,6)
2 0 bi=3
3 0 wi=2
4 0 w=1
5 0
w-wi =-1
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (5) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 i=1 4: (5,6)
2 0 3 bi=3
3 0 wi=2
4 0 w=2
5 0
w-wi =0
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (6) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 i=1 4: (5,6)
2 0 3 bi=3
3 0 3 wi=2
4 0 w=3
5 0
w-wi=1
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (7) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 i=1 4: (5,6)
2 0 3 bi=3
3 0 3 wi=2
4 0 3 w=4
5 0
w-wi=2
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (8) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 i=1 4: (5,6)
2 0 3 bi=3
3 0 3 wi=2
4 0 3 w=5
5 0 3
w-wi=2
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (9) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 i=2 4: (5,6)
2 0 3 bi=4
3 0 3 wi=3
4 0 3 w=1
5 0 3
w-wi=-2
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (10) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 i=2 4: (5,6)
2 0 3 3 bi=4
3 0 3 wi=3
4 0 3 w=2
5 0 3
w-wi=-1
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (11) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 i=2 4: (5,6)
2 0 3 3 bi=4
3 0 3 4 wi=3
4 0 3 w=3
5 0 3
w-wi=0
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (12) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 i=2 4: (5,6)
2 0 3 3 bi=4
3 0 3 4 wi=3
4 0 3 4 w=4
5 0 3
w-wi=1
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (13) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 i=2 4: (5,6)
2 0 3 3 bi=4
3 0 3 4 wi=3
4 0 3 4 w=5
5 0 3 7
w-wi=2
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (14) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 0 i=3 4: (5,6)
2 0 3 3 3 bi=5
3 0 3 4 4 wi=4
4 0 3 4 w=1..3
5 0 3 7

if wi <= w // item i can be part of the solution

if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (15) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 0 i=3 4: (5,6)
2 0 3 3 3 bi=5
3 0 3 4 4 wi=4
4 0 3 4 5 w=4
5 0 3 7
w- wi=0
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (15) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 0 i=3 4: (5,6)
2 0 3 3 3 bi=5
3 0 3 4 4 wi=4
4 0 3 4 5 w=5
5 0 3 7 7
w- wi=1
if wi <= w // item i can be part of the solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (16) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 0 0 i=3 4: (5,6)
2 0 3 3 3 3 bi=5
3 0 3 4 4 4 wi=4
4 0 3 4 5 5 w=1..4
5 0 3 7 7

if wi <= w // item i can be part of the solution

if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Example (17) Items:
i 0 1 2 3 4
1: (2,3)
W 2: (3,4)
0 0 0 0 0 0 3: (4,5)
1 0 0 0 0 0 i=3 4: (5,6)
2 0 3 3 3 3 bi=5
3 0 3 4 4 4 wi=4
4 0 3 4 5 5 w=5
5 0 3 7 7 7

if wi <= w // item i can be part of the solution

if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // wi > w
 Comments

? This algorithm only finds the max possible value that can
be carried in the knapsack
? To know the items that make this maximum value, an
addition to this algorithm is necessary
? Please see LCS algorithm lecture for the example how to
extract this data from the table we built
 Books

1. Introduction to Algorithms, Third Edition, Thomas H. Cormen, Charle E. Leiserson,

Ronald L. Rivest, Clifford Stein (CLRS).
2. Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj Sahni, Sanguthevar
Rajasekaran (HSR)
 References

🡽 https://algorithmist.com/wiki/Dynamic_programming

🡽 https://www.topcoder.com/community/competitive-programming/tutorials/dynami
c-programming-from-novice-to-advanced/

🡽 CLRS: 15.3

🡽 HSR: 5.1, 5.5