Knapsack problem
There are two versions of the problem:
(1) “0-1 knapsack problem” and
(2) “Fractional knapsack problem”
(1) Items are indivisible; you either take an item or
not. Solved with dynamic programming
(2) Items are divisible: you can take any fraction of
an item. Solved with a greedy algorithm.
1
Knapsack problem 0-1
Thief has a knapsack with maximum capacity W, and a
set S consisting of n items
Each item i has some weight wi and benefit value vi (all
wi , vi and W are integer values)
Problem: How to pack the knapsack to achieve
maximum total value of packed items?
Goal:
find xi such that for all xi = {0, 1}, i = 1, 2, .., n
wixi W and
xivi is maximum
2
Knapsack - Greedy Strategy 0-1
E.g.1:
Item 3 30 $120
Item 2 50 50 50 +
20 $100
Item 1 30
20 + 20 $100
10 10 $60
$60 $100 $120 W $160 $220
$6/pound $5/pound $4/pound
E.g.2:
Item: 1 2 3 4 5 6 7
Benefit: 5 8 3 2 7 9 4
Weight: 7 8 4 10 4 6 4
Knapsack holds a maximum of 22 pounds
Fill it to get the maximum benefit
Knapsack problem: brute-force 0-1
approach
Let’s first solve this problem with a straightforward
algorithm
Since there are n items, there are 2n possible
combinations of items.
We go through all combinations and find the one
with the most total value and with total weight less
or equal to W
Running time will be O(2n)
4
Knapsack - Dynamic Programming 0-1
P(i, w) – the maximum profit that can be
obtained from items 1 to i, if the
knapsack has size w
Case 1: thief takes item i
P(i, w) =vi + P(i - 1, w-wi)
Case 2: thief does not take item i
P(i, w) =P(i - 1, w)
Recursive Formula
P[i 1, w] if wi w
P[i, w]
max{vi P[i 1, w wk ], P[i 1, w]} else
The best subset that has the total weight w, either
contains item i or not.
First case: wi>w. Item i can’t be part of the
solution, since if it was, the total weight would be
> w, which is unacceptable
Second case: wi <=w. Then the item i can be in
the solution, and we choose the case with greater
value.
6
Knapsack - Dynamic Programming 0-1
Item i was taken Item i was not taken
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
0: 1 w - wi w W
0 0 0 0 0 0 0 0 0 0 0 0
0 first
0 second
i-1 0
i 0
0
n 0
W = 5 Item Weight Value
Example: 1 2 12
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) } 2 1 10
3 3 20
0 1 2 3 4 5
4 2 15
0 0 0 0 0 0 0 P(1, 1) = P(0, 1) = 0
1 0 0 12 12 12 12 P(1, 2) = max{12+0, 0} = 12
2 0 10 12 22 22 22 P(1, 3) = max{12+0, 0} = 12
3 0 10 12 22 30 32 P(1, 4) = max{12+0, 0} = 12
4 0 10 15 25 30 37 P(1, 5) = max{12+0, 0} = 12
P(2, 1)= max{10+0, 0} = 10 P(3, 1)= P(2,1) = 10 P(4, 1)= P(3,1) = 10
P(2, 2)= max{10+0, 12} = 12 P(3, 2)= P(2,2) = 12 P(4, 2)= max{15+0, 12} = 15
P(2, 3)= max{10+12, 12} = 22 P(3, 3)= max{20+0, 22}=22 P(4, 3)= max{15+10, 22}=25
P(2, 4)= max{10+12, 12} = 22 P(3, 4)= max{20+10,22}=30 P(4, 4)= max{15+12, 30}=30
P(2, 5)= max{10+12, 12} = 22 P(4, 5)= max{20+12,22}=32 P(4, 5)= max{15+22, 32}=37
Reconstructing the Optimal Solution
0 1 2 3 4 5
0 0 0 0 0 0 0 • Item 4
1 0 0 12 12 12 12
• Item 2
2 0 10 12 22 22 22
3 0 10 12 22 30 32 • Item 1
4 0 10 15 25 30 37
• Start at P(n, W)
• When you go left-up item i has been taken
• When you go straight up item i has not
been taken
Optimal Substructure
Consider the most valuable load that weights
at most W pounds
If we remove item j from this load
The remaining load must be the most
valuable load weighing at most W – wj that
can be taken from the remaining n – 1 items
Knapsack Algorithm 0-1
for w = 0 to W
P[0,w] = 0
O(n*W)
for i = 0 to n
P[i,0] = 0
for w = 0 to W
if wi <= w // item i can be part of the solution
if vi + P[i-1,w-wi] > P[i-1,w]
P[i,w] = vi + P[i-1,w- wi]
else
P[i,w] = P[i-1,w]
else P[i,w] = P[i-1,w] // wi > w
11
Overlapping Subproblems
P(i, w) = max {vi + P(i - 1, w-wi), P(i - 1, w) }
0: 1 w W
0 0 0 0 0 0 0 0 0 0 0 0
0
0
i-1 0
i 0
0
n 0
E.g.: all the subproblems shown in grey
may depend on P(i-1, w)
Example
Let’s run our algorithm on the
following data:
n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
13
Conclusion
Dynamic programming is a useful technique of
solving certain kind of problems
When the solution can be recursively described in
terms of partial solutions, we can store these partial
solutions and re-use them as necessary
14
