Design and Analysis of Algorithm (DAA) Study material

Knapsack Problem

1 Introduction
1.1 Problem statement
A thief robbing a store finds n items; the ith item is worth vi dollars and weighs wi pounds, where vi
and wi are integers. He wants to take as valuable a load as possible, but he can carry at most W
pounds in his knapsack for some integer W. Which items should he take?
There are two versions of knapsack problem.
a) 0/1 knapsack problem
In the 0/1 knapsack problem the set up is same as mentioned in the above problem statement,
but there is a constraint that is the items available in the store can not be broken into smaller
pieces. So each item must either be taken or left behind by the thief, the thief cannot take a
fractional amount of an item or take an item more than once.
Since 0/1 knapsack problem exhibit optimal substructure property, therefore for this
problem only dynamic programming algorithm exists.

b) Fractional knapsack problem

In the fractional knapsack problem the set up is same as mentioned in the above problem
statement, but there is a constraint that is the items available in the store can be broken into
smaller pieces. So the thief can take fractions of items, rather than having to make a binary (0–1)
choice for each item.
Since fractional knapsack problem exhibit greedy choice property, so for this problem
greedy algorithm exists.

2 Fractional knapsack problem

2.1 Problem statement
A thief robbing a store with a knapsack finds n items/objects; the ith item/object is worth vi dollars
and weighs wi pounds. The knapsack capacity is W pounds that is the thief can carry at most W
pounds in his knapsack. If a fraction xi, 0 ≤ xi ≤ 1, of item/object i is placed into the knapsack,
then a profit of vixi is earned. The fractional problem states “How much of which items should the
thief take in his knapsack that maximize the profit earned.”
Symbolically the fractional knapsack problem is defined as follows:

Maximize vixi

Subjects to wixi ≤ W

Where 0 ≤ xi ≤ 1 and 0 ≤ i ≤ 1

Page 1 /4
 Example-1:
Consider the following instance of the knapsack: n = 3, W = 20, (v1, v2, v3) = (25, 24, 14) and (w1,
w2. w3) = (18, 15, 10)
(You may find in the exam. in a different notation- m=20, (p1, p2, p3) = (25, 24, 14) etc., m=20
means knapsack capacity)
Four feasible solutions are
Solution items Wights Profit earned
(x1, x2, x3) wixi vixi

1 (1/2, 1/3, 1/4) 16.5 24.25

2 (1, 2/15, 0) 20 28.2
3 (0, 2/3, 1) 20 31
4 (0, 1, 1/2) 20 31.5

Of these four feasible solutions, solution 4 yields maximum profit. There may be other feasible
solutions are also possible. Out of all feasible solutions, the one which shows the maximum profit
earned is the optimal solution. How to get the optimal solution. Apply greedy greedy knapsack
algorithm.
2.2 Greedy Knapsack Algorithm
GREEDY-KNAPSACK(W, n)
/*w[1:n] and v[1:n] contains the weights and values/profits respectively. n
items/objects ordered such that v[i]/w[i] ≥ v[i+1]/w[i+1]. W is the knapsack size
and x[1:n] is the solution vector.
{
for i ← 1 to n do
x[i] ← 0 // Initilize x
U←W
for i ← 1 to n do
{
if (w[i] > U) then break;
x[i] ← 1.0
U ← (U – w[i])
}
if (i ≤ n) the x[i] ← U/w[i]
}

In other words, sort the n items in decreasing order of their value per pound (or value per pound or
profit per weight) vi/wi. Choose items one by one till the knapsack is full.

2.3 Time complexity of Greedy Knapsack Algorithm

The time complexity is O(n log n) where n is the number of items.

3 Examples
3.1 Find an optimal solution to the above example given.
Step-1
Item/ Value/Profit Weight Value per
Object (i) (vi) (wi) weight (vi/wi)
1 25 18 25/18 = 1.38
2 24 15 24/15 = 1.6
3 14 10 14/10 = 1.4

Page 2 /4
 Step-2
Item/ Value/Profit Weight Value per Weight taken xi
Object (i) (vi) (wi) weight (vi/wi)
2 24 15 24/15 = 1.6 15 ≤ 20, take the 1
whole item
Remaining weight
U = 20-5 = 5
3 14 10 14/10 = 1.4 As 14 > 5 so take 1/2
a fraction of it.
Fraction= U/wi =
5/10 = 1/2
1 25 18 25/18 = 1.38 As no weight is 0
left so don’t take
this item

The solution vector or the optimal solution is (x1, x2, x3) = (0, 1, 1/2)

3.2 Find an optimal solution to the knapsack instance n=7, W=15. (v1, v2, v3, v4, v5, v6, v7) = (10, 5,
15, 7, 6, 18, 3) and (w1, w2, w3, w4, w5, w6, w7) = (2,3, 5, 7, 1, 4, 1)
Solution
Step-1
Item/ Value/Profit Weight Value per
Object (i) (vi) (wi) weight (vi/wi)
1 10 2 5
2 5 3 1.66
3 15 5 3
4 7 7 1
5 6 1 6
6 18 4 4.5
7 3 1 3
Step-2
U=W=15
Item/ Value/Profit Weight Value per Weight taken xi
Object (i) (vi) (wi) weight (vi/wi)
5 6 1 6 1 ≤ 15 1
U=15-1=14
1 10 2 5 2 ≤ 14 1
U=14-2=12
6 18 4 4.5 4 ≤ 12 1
U=12-4=8
3 15 5 3 5≤8 1
U=8-5=3
7 3 1 3 1≤3 1
U=3-1=2
2 5 3 1.66 3 >2 2/3
Fraction=2/3
4 7 7 1 - 0

The solution vector or the optimal solution is (x1, x2, x3, x4, x5, x6, x7) = (1, 2/3, 1, 0, 1, 1, 1)
Profit earned = vixi = 1x10 + 2/3 x 5 + 1x15 + 0x7 + 1x6 + 1x18 +1x3
=10 + 3.33 + 15 + 0 + 6 + 18 + 3 =55.33 (check this figure with class discussion)

Page 3 /4
 Solve it by 0/1 knapsack.

4 Home Task
a) Prove that the fractional knapsack problem has the greedy-choice property.
b) Give a dynamic-programming solution to the 0–1 knapsack problem that runs in O(n W) time,
where n is number of items and W is the maximum weight of items that the thief can put in his
knapsack.

Page 4 /4