Dynamic Programming

1
 Dynamic Programming
Example 1:

Shortest Path Problem

2
 Consider the following graph. For each stage j and
location s, we have
fj(s) = Min(all z of stage j+1) { Csz + fj+1(z) }

B 7

4 E 1
3
2 4
H
2 6 3
4
4 C F3
J
A 2
3 I 4
2
2 1 G 3

D 5
3
 We have five stages in previous figure, namely S1,
S2, S3 S4 S5 and contains following nodes

Csz means cost (length) of arc s  z

We will be solving our problem in bottom up fashion.

i.e. we will be moving from stage S5 to stage S1

By above definition, for stage S5 we will have

f5(J) = 0

4
 S4 Csz + f5(z) f4(S4) Decision to
go
J
H 3+0 = 3 3 J
I 4+0= 4 4 J

S3 Csz + f4(z) f3(S3) Decision to go

H I
E 1+3 = 4 4+4 = 8 4 H
F 6+3 = 9 3+4 = 7 7 I
G 3+3 = 6 3+4 = 7 6 H
5
 S2 Csz + f3(z) f2(S2) Decision to go
E F G
B 11 11 9 9 G
C 6 11 8 6 E
D 6 8 11 6 E

S1 Csz + f2(z) f1(S1) Decision to go

B C D
A 11 10 8 8 D

 So from above DP approach we found that

6
A D E H J is the shortest possible path.
 Time complexity
What it should be?
stages :- m
nodes :- n
let us say, at every stage we have n- locations and for
that we calculated values from all previous stage values.

So we have O(n2) for every stage. And if we have m

stages then for all stages it should be O(mn2).

7
 Space complexity
What it should be?

Space complexity = O(n), as we calculate n –

locations at every stage. So stage doesn't matter
actually but the number of nodes does matter at
every stage.

8
 Knapsack Problem
You have a knapsack that has capacity (weight) C.
You have several items I1,…,In.
Each item Ij has a weight wj and a benefit bj.
You want to place a certain number of copies of
each item Ij in the knapsack so that:
The knapsack weight capacity is not exceeded and
The total benefit is maximal.

9
 Example

Item Weight Benefit

A 2 60

B 3 75

C 4 90

Capacity = 5
10
 Key question
Suppose f(w) represents the maximal possible benefit
of a knapsack with weight w.
We want to find (in the example) f(5).
Is there anything we can say about f(w) for arbitrary
w?

11
 Key observation
To fill a knapsack with items of weight w, we must
have added items into the knapsack in some order.
Suppose the last such item was Ij with weight wi
and benefit bi.
Consider the knapsack with weight (w- wi).
Clearly, we chose to add Ij to this knapsack
because of all items with weight wi or less, Ij had
the max benefit bi.

12
 Key observation
Thus, f(w) = MAX { bj + f(w-wj) | Ij is an item}.
This gives rise to an immediate recursive algorithm to
determine how to fill a knapsack.

13
 Example

Item Weight Benefit

A 2 60

B 3 75

C 4 90

14
 f(0), f(1)
f(0) = 0. Why? The knapsack with capacity 0 can have
nothing in it.
f(1) = 0. There is no item with weight 1.

15
 f(2)
f(2) = 60. There is only one item with weight 60.
Choose A.

16
 f(2)
f(2) = MAX { bj + f(w-wj) | Ij is an item}.
= MAX { 60+f(2-2)}
= MAX { 60 + 0}
= 60.
Choose A .

17
 f(3)
f(3) = MAX { bj + f(w-wj) | Ij is an item}.
= MAX { 60+f(3-2), 75 + f(3-3)}
= MAX { 60 + 0, 75 + 0 }
= 75.
Choose B.

18
 f(4)
f(4) = MAX { bj + f(w-wj) | Ij is an item}.
= MAX { 60 + f(4-2), 75 + f(4-3), 90+f(4-4)}
= MAX { 60 + 60, 75 + f(1), 90 + f(0)}
= MAX { 120, 75, 90}
=120.
Choose A.

19
 f(5)
f(5) = MAX { bj + f(w-wj) | Ij is an item}.
= MAX { 60 + f(5-2), 75 + f(5-3), 90+f(5-4)}
= MAX { 60 + f(3), 75 + f(2), 90 + f(1)}
= MAX { 60 + 75, 75 + 60, 90+0}
= 135.
Choose A and B.

20
 Result
Optimal knapsack weight is 135.
Two possible optimal solutions:
Choose A during computation of f(5). Choose B in
computation of f(3).
Choose B during computation of f(5). Choose A in
computation of f(2).
Both solutions coincide. Take A and B.

21
 Another example
Knapsack of capacity 50.
3 items
Item 1 has weight 10, benefit 60
Item 2 has weight 20,benefit 100
Item 3 has weight 30, benefit 120.

22
 f(0),..,f(9)
All have value 0.

23
 f(10),..,f(19)
All have value 10.
Choose Item 1.

24
 f(20),..,f(29)
F(20) = MAX { 60 + f(20-10), 100 + f(20-20) }
= MAX { 60 + f(10), 100 + f(0) }
= MAX { 60+60, 100+0}
=120.
Choose Item 1.

25
 f(30),…,f(39)
f(30) = MAX { 60 + f(30-10), 100 + f(30-20), 120 + f(30-30) }
= MAX { 60 + f(20), 100 + f(10), 120 + f(0) }
= MAX { 60 + 120, 100+60, 120+0}
= 180
Choose item 1.

26
 f(40),…,f(49)
F(40) = MAX { 60 + f(30), 100 + f(20), 120 + f(10)}
= MAX { 60 + 180, 100+120, 120 + 60}
= 240.
Choose item 1.

27
 f(50)
f(50) = MAX { 60 + f(40), 100 + f(30), 120 + f(20) }
= MAX { 60 + 240, 100+180, 120 + 120}
= 300.
Choose item 1.

28
 Another Example:
Truck – 10t Capacity
Truck – 10t capacity

Optimum cargo combination:

• Item 1: $5 (3t)
• Item 2: $7 (4t)
• Item 3: $8 (5t)

29
 In tabular form

Item Weight Benefit

Item 1 3 $5

Item 2 4 $7

Item 3 5 $8

30
 f(10)
f(10) = MAX { 5 + f(10-3), 7 + f(10-4), 8 + f(10-5) }

= MAX { 5 + f(7), 7 + f(6), 8 + f(5) }

So we need to calculate f(7), f(6), f(5)

= MAX { …, …, …}

= value.

Choose items combination that given value =‘value’

31
 Other Functions are
f(1)=0
f(2)=0
f(3)=max(5+f(3-3))= max(5+f(0))= 5
f(4)=max(5+f(4-3), 7+f(4-4))= max(5,7)= 7
f(5)=max(5+f(5-3), 7+f(5-4) , 8+f(5-5))= max(5,7,8)= 8
f(6)=max(5+f(6-3), 7+f(6-4) , 8+f(6-5))= max(10,7,8)= 10
f(7)=max(5+f(7-3), 7+f(7-4) , 8+f(7-5))= max(12,12,8)= 12
f(8)=max(5+f(8-3), 7+f(8-4) , 8+f(8-5))= max(13,14,13)= 14
f(9)=max(5+f(9-3), 7+f(9-4) , 8+f(9-5))= max(15,15,15)= 15
f(10)=max(5+f(7), 7+f(6) , 8+f(5))= max(17,17,16)= 17
32