0-1 Knapsack problem

 Given a knapsack with maximum capacity W, and a set

S consisting of n items
 Each item i has some weight wi and benefit value bi (all
wi , bi and W are integer values)
 Problem: How to pack the knapsack to achieve
maximum total value of packed items?
 0-1 Knapsack problem:
a picture
Weight Benefit value
Items wi bi

2 3
This is a knapsack 3 4
Max weight: W = 20 4 5

5 8
W = 20

9 10
 0-1 knapsack problem
 More formally, the 0-1 knapsack problem:
» The thief must choose among n items, where the ith item
worth vi dollars and weighs wi pounds
» Carrying at most W pounds, maximize value
• Note: assume vi, wi, and W are all integers
• “0-1” b/c each item must be taken or left in entirety
 0-1 Knapsack problem
 Problem, in other words, is to find

max  bi subject to  wi W
iT iT
 The problem is called a “0-1” problem,
because each item must be entirely
accepted or rejected.
 0-1 Knapsack problem: brute-
force approach
Let’s first solve this problem with a straightforward
algorithm
 Since there are n items, there are 2n possible
combinations of items.
 We go through all combinations and find the one
with the most total value and with total weight less
or equal to W
 Running time will be O(2n)
 0-1 Knapsack problem: brute-
force approach
 Can we do better?
 Yes, with an algorithm based on dynamic
programming
 We need to carefully identify the subproblems

Let’s try this:

If items are labeled 1..n, then a subproblem
would be to find an optimal solution for
Sk = {items labeled 1, 2, .. k}
 Defining a Subproblem
If items are labeled 1..n, then a subproblem would be
to find an optimal solution for Sk = {items labeled
1, 2, .. k}
 This is a valid subproblem definition.
 The question is: can we describe the final solution
(Sn ) in terms of subproblems (Sk)?
 Unfortunately, we can’t do that. Explanation
follows….
 Defining a Subproblem
w1 =2 w2 =4 w3 =5 w4 =3 Weight Benefit
b1 =3 b2 =5 b3 =8 b4 =4
Item wi bi
?
Max weight: W = 20
#
1 2 3
S4 2 3 4
For S4:
S5
Total weight: 14; 3 4 5
total benefit: 20 4 5 8
5 9 10
w1 =2 w2 =4 w3 =5 w4 =9
b1 =3 b2 =5 b3 =8 b4 =10

For S5:
Solution for S4 is
Total weight: 20 not part of the
total benefit: 26 solution for S5!!!
 Defining a Subproblem
(continued)
 As we have seen, the solution for S4 is not part of
the solution for S5
 So our definition of a subproblem is flawed and
we need another one!
 Let’s add another parameter: w, which will
represent the exact weight for each subset of
items
 The subproblem then will be to compute B[k,w]
 Recursive Formula for
subproblems
 Recursive formula for subproblems:
 B[k  1, w] if wk  w
B[k , w] 
max{B[k  1, w], B[k  1, w  wk ]  bk } else
 It means, that the best subset of Sk that has total
weight w is one of the two:
1) the best subset of Sk-1 that has total weight w, or
2) the best subset of Sk-1 that has total weight w-wk
plus the item k
 Recursive Formula
 B[k  1, w] if wk  w
B[k , w] 
max{B[k  1, w], B[k  1, w  wk ]  bk } else
 The best subset of Sk that has the total weight w,
either contains item k or not.
 First case: wk>w. Item k can’t be part of the
solution, since if it was, the total weight would be
> w, which is unacceptable
 Second case: wk <=w. Then the item k can be in
the solution, and we choose the case with greater
value
 The Knapsack Problem
And Optimal Substructure
 Both variations exhibit optimal substructure
 To show this for the 0-1 problem, consider the most
valuable load weighing at most W pounds
» If we remove item j from the load, what do we know about the
remaining load?
» A: remainder must be the most valuable load weighing at
most W - wj that thief could take from museum, excluding
item j
 Solving The Knapsack Problem
 The optimal solution to the fractional knapsack
problem can be found with a greedy algorithm
» How?
 The optimal solution to the 0-1 problem cannot be
found with the same greedy strategy
» Greedy strategy: take in order of dollars/pound
» Example: 3 items weighing 10, 20, and 30 pounds, knapsack
can hold 50 pounds
• Suppose item 2 is worth $100. Assign values to the other items so that
the greedy strategy will fail
 0-1 Knapsack Algorithm
for w = 0 to W
B[0,w] = 0
for i = 0 to n
B[i,0] = 0
for w = 0 to W
if wi <= w // item i can be part of the
solution
if bi + B[i-1,w-wi] > B[i-1,w]
B[i,w] = bi + B[i-1,w- wi]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w > w
 Running time
for w = 0 to W O(W)
B[0,w] = 0
for i = 0 to n
Repeat n times
B[i,0] = 0
for w = 0 to W O(W)
< the rest of the code >
What is the running time of this
algorithm?
O(n*W)
Remember that the brute-force algorithm
takes O(2n)
 Example
Let’s run our algorithm on the
following data:

n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
 Example (2)

i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1
2
3
4

for w = 0 to W
V[0,w] = 0
 Example (3)

i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

for i = 1 to n
V[i,0] = 0
 Items:
Example (4) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0
wi=2
2 0
3 0 w=1
4 0 w-wi =-1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (5) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3
wi=2
2 0
3 0 w=2
4 0 w-wi =0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (6) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3
wi=2
2 0
3 0 w=3
4 0 w-wi =1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (7) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3
wi=2
2 0
3 0 w=4
4 0 w-wi =2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (8) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3 3
wi=2
2 0
3 0 w=5
4 0 w-wi =3
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (9) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
3 0 w=1
4 0 w-wi =-2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (10) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3
3 0 w=2
4 0 w-wi =-1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (11) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
3 0 w=3
4 0 w-wi =0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (12) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
3 0 w=4
4 0 w-wi =1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (13) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
3 0 w=5
4 0 w-wi =2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (14) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 w= 1..3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (15) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 w= 4
4 0 w- wi=0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (16) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 w- wi=1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (17) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 1..4
4 0 0 3 4 5
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
Example (18) 1: (2,3)
2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 0 3 4 5 7 w- wi=0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w