Understanding the Knapsack Problem

The Knapsack Problem is a classic optimization problem where we are given a set of items, each with a weight and a value, and a
knapsack with a certain weight capacity. The objective is to find the most valuable combination of items to fill the knapsack without
exceeding its weight capacity.

Certainly! Let’s dive into the Knapsack problem and explore how it can be solved using the greedy method. 🎒
Knapsack Problem Overview
The knapsack problem involves selecting items to maximize profit while adhering to a weight constraint. Imagine
you have a bag (the knapsack) with a limited weight capacity. Your goal is to choose items from a given set, each
with its own weight and profit, such that the total profit is maximized without exceeding the knapsack’s weight
limit.
Types of Knapsack Problems
1. 0/1 Knapsack Problem:
o In this variant, items can either be fully included or not included at all.
o For example, if we have items with weights {2kg, 3kg} and profits {2, 3}, we cannot take a fraction of an item; it’s either the
whole item or none.
o Dynamic programming is commonly used to solve this problem.
2. Fractional Knapsack Problem:
o Here, items can be divided (fractional) to fit into the knapsack.
o For instance, if we have an item weighing 3kg, we can take part of it (e.g., 2kg) and leave the rest.
o The fractional knapsack problem is solved using the greedy approach.

Example: Fractional Knapsack

Let’s consider an example with the following data:

 Weights: {3, 4, 6, 5}
 Profits: {2, 3, 1, 4}
 Knapsack capacity: 8 kg

CMPS 6610 Algorithms 1

 Greedy Approach:

1. Sort items by their profit-to-weight ratio in non-increasing order.

2. Start with an empty knapsack.
3. Add items to the knapsack until the weight limit is reached (or exceeded).

Solution:

1. Take the 4th item (profit = 15, weight = 5).

2. Remaining capacity: 8 - 5 = 3 kg.
3. Take the 3rd item (profit = 14, weight = 7).
4. Total profit: 15 + 14 = 29.

Optimal 0/1 Knapsack Solution:

1. Take items 2 and 3 (profit = 19).

2. Weight: 3 kg (within the capacity).

In summary, the greedy approach works well for the fractional knapsack problem but not necessarily for the 0/1
knapsack problem. For the latter, dynamic programming is more suitable

In this method, the Knapsack’s filling is done so that the maximum capacity of the knapsack is utilized so that
maximum profit can be earned from it. The knapsack problem using the Greedy Method is referred to as:
Given a list of n objects, say {I1, I2,……, In) and a knapsack (or bag).
The capacity of the knapsack is M.

If a fraction xj (where x ∈ {0…., 1)) of an object Ij is placed into a knapsack, then a profit of pjxj is earned.
Each object Ij has a weight wj and a profit of pj

The problem (or Objective) is to fill the knapsack (up to its maximum capacity M), maximizing the total profit earned.

CMPS 6610 Algorithms 1

Mathematically:
Note that the value of xj will be any value between 0 and 1 (inclusive). If any object Ij is completely placed into a
knapsack, its value is 1 (xj = 1). If we do not pick (or select) that object to fill into a knapsack, its value is 0 ( x j = 0).
Otherwise, if we take a fraction of any object, then its value will be any value between 0 and 1.

Knapsack Problem Algorithm Using Greedy Method

Knapsack Problem Using Greedy Method Pseudocode

A pseudo-code for solving knapsack problems using the greedy method is;
greedy fractional-knapsack (P[1…n], W[1…n], X[1..n]. M)
/*P[1…n] and W[1…n] contain the profit and weight of the n-objects ordered such that X[1…n] is a solution set and M
is the capacity of knapsack*/
{

For j ← 1 to n do
X[j]← 0
profit ← 0 // Total profit of item filled in the knapsack
weight ← 0 // Total weight of items packed in knapsacks
j←1
While (Weight < M) // M is the knapsack capacity

CMPS 6610 Algorithms 1

 Greedy Strategy
1. Repeatedly identify a decision to be made (
recursion)
2. Make a locally optimal choice for each decision

In order to reach a globally optimal solution, the

problem must have appropriate recursive substructure:
optimal solution = locally optimal choice
+ optimal solution for the remainder
of the problem

CMPS 6610 Algorithms 2

 Knapsack Problem
• Given a knapsack with weight capacity , and
given items of positive integer weights 1 n

and positive integer values 1 n.

(So, item has value i and weight i.)

• 0-1 Knapsack Problem: Compute a subset of
items that maximize the total value (sum), and they
all fit into the knapsack (total weight at most W).
• Fractional Knapsack Problem: Same as before but
we are allowed to take fractions of items ( gold
dust).
CMPS 6610 Algorithms 3
 Greedy Knapsack
• Greedy Strategy:
vi
– Compute
wi for each
– Greedily take as much as possible of the item
with the highest value/weight. Then
repeat/recurse.
 Sort items by value/weight
 runtime

CMPS 6610 Algorithms 4

 Knapsack Example
item 1 2 3
value 12 15 4 W=4
weight 2 3 1
value/weight 6 5 4
• Greedy fractional: Take item 1 and 2/3 of item 2
 weight=4, value=12+2/315 = 12+10 = 22
• Greedy 0-1: Take item 1 and then item 3
greedy 0-1
 weight = 1+2=3, value=12+4=16 ≠ optimal 0-1
• Optimal 0-1: Take items 2 and 3, value =19
CMPS 6610 Algorithms 5
 Optimal Substructure
• Let 1 n be an optimal solution, where i =
amount of item that is taken; i

• Suppose we remove one item.  items left

• Is the remaining “solution” still an optimal solution
for items?
• Yes; cut-and-paste.

CMPS 6610 Algorithms 6

 Correctness Proof for Greedy
• Suppose items are numbered in decreasing order
by value/weight.
∗
• Greedy solution G: Takes all elements -1 and a
fraction of ∗.
• Assume optimal solution S is different from G. Assume S takes
only a fraction 1 ∗
a of item , for -1.
• Create new solution S’ from S by taking j
weight away from items , and add j of
item back in. Hence, all of item is taken.
 New solution S’ has the same weight but increased value.
This contradicts the assumption that S was optimal.
 S=G.
CMPS 6610 Algorithms 7
 General Solution: DP
• = max value possible for taking a subset of items
with knapsack constraint .
• for all and
for
i i

don’t take item i take item i

• Compute by filling an DP-table.

 Two nested for-loops, runtime and space
• Trace back from by redoing computation or
following arrows.  runtime
CMPS 6610 Algorithms 8
 Solution:
DP Example w Take items 3 and 2

W=4
W= 4 0 12 15 19
item 1 2 3 3 0 12 15 16
value 12 15 4 2 0 12 12 12
weight 2 3 1 1 0 0 0 4
value/weight 6 5 4
0 0 0 0 0
Take item i: 0 1 2 3 i
n

Don’t take item i:

i i

don’t take item i take item i

CMPS 6610 Algorithms 9
 Conclusion

Greedy algorithms provide a powerful approach for solving optimization problems efficiently. In the case of the Knapsack Problem,
the greedy algorithm proves to be a useful technique for minimizing time complexity while achieving a locally optimal solution.

Remember that greedy algorithms may not always guarantee an optimal solution for every problem. However, they provide a handy
tool in your algorithmic arsenal and are worth considering, especially when applying them to optimization problems.

Introduction to Knapsack Problem, its Types and How to

solve them
Last Updated : 06 Apr, 2023





The Knapsack problem is an example of the combinational optimization problem. This problem is
also commonly known as the “Rucksack Problem“. The name of the problem is defined from the
maximization problem as mentioned below:
Given a bag with maximum weight capacity of W and a set of items, each having a weight and a value
associated with it. Decide the number of each item to take in a collection such that the total weight is
less than the capacity and the total value is maximized.
Types of Knapsack Problem:
The knapsack problem can be classified into the following types:
1. Fractional Knapsack Problem
2. 0/1 Knapsack Problem
3. Bounded Knapsack Problem
4. Unbounded Knapsack Problem

CMPS 6610 Algorithms 1

0
1. Fractional Knapsack Problem
The Fractional Knapsack problem can be defined as follows:
Given the weights and values of N items, put these items in a knapsack of capacity W to get the
maximum total value in the knapsack. In Fractional Knapsack, we can break items for maximizing the
total value of the knapsack.
Some practice problems on 0/1 Knapsack:
Sl. No. Problem

1 Fractional Knapsack Problem

2 Fractional Knapsack Queries

Unbounded Fractional
3
Knapsack

2. 0/1 Knapsack Problem

The 0/1 Knapsack problem can be defined as follows:
We are given N items where each item has some weight (wi) and value (vi) associated with it. We are
also given a bag with capacity W. The target is to put the items into the bag such that the sum of
values associated with them is the maximum possible.
Note that here we can either put an item completely into the bag or cannot put it at all.
Mathematically the problem can be expressed as:

Maximize subject to and xi ∈ {0, 1}

Some practice problems on 0/1 Knapsack:
Sl. No. Problem

1 0/1 Knapsack Problem

CMPS 6610 Algorithms 1

1
Sl. No. Problem

2 Printing Items in 0/1 Knapsack

0/1 Knapsack Problem to print all possible

3
solutions

4 Knapsack with large Weights

5 0/1 knapsack queries

6 0/1 Knapsack using Branch and Bound

7 0/1 Knapsack using Least Cost Branch and Bound

Following are the differences between the 0/1 knapsack problem and the Fractional
knapsack problem.
Sr.
No 0/1 knapsack problem Fractional knapsack problem

The 0/1 knapsack problem is solved using dynamic

Fractional knapsack problem is solved using a greedy approach.
1. programming approach.

2. The 0/1 knapsack problem has not an optimal structure. The fractional knapsack problem has an optimal structure.

In the 0/1 knapsack problem, we are not allowed to break Fractional knapsack problem, we can break items for maximizing
3. items. the total value of the knapsack.

CMPS 6610 Algorithms 1

2
 Sr.
No 0/1 knapsack problem Fractional knapsack problem

0/1 knapsack problem, finds a most valuable subset item with In the fractional knapsack problem, finds a most valuable subset
4. a total value less than equal to weight. item with a total value equal to the weight.

In the 0/1 knapsack problem we can take objects in an integer In the fractional knapsack problem, we can take objects in
5. value. fractions in floating points.

3. Bounded Knapsack Problem

The Bounded Knapsack problem can be defined as follows:
Given N items, each item having a given weight wi and a value vi, the task is to maximize the value
by selecting a maximum of K items adding up to a maximum weight W.
Mathematically the problem can be expressed as:

Maximize subject to and xi ∈ {0, 1, . . . , K}

Some practice problems on Bounded Knapsack:
Sl. No. Problem

1 Extended Knapsack Problem

2 Partition Problem

3 Count of subsets with sum equal to X

Length of longest subset consisting of A 0s and B 1s from an array of

4
strings

5 Breaking an Integer to get Maximum Product

CMPS 6610 Algorithms 1

3
4. Unbounded Knapsack Problem
The Unbounded Knapsack problem can be defined as follows:
Given a knapsack weight W and a set of N items with certain value vi and weight wi, we need to
calculate the maximum amount that could make up this quantity exactly. This is different from 0/1
Knapsack problem, here we are allowed to use an unlimited number of instances of an item.
Mathematically the problem can be expressed as:

Maximize subject to and and xi ≥ 0.

Some practice problems on Unbounded Knapsack:
Sl. No. Problem

1 Unbounded Fractional Knapsack

2 Unbounded Knapsack (Repetition of items allowed)

Unbounded Knapsack (Repetition of items allowed) | Set

3
2

4 Find minimum number of coins that make a given value

5 Coin Change

Variations of Knapsack Problem:

There are several variations possible for the Knapsack Problem. Some of the well-known variations are
provided below:

1. Multi-objective Knapsack problem:

In this variation, the goal of filling the knapsack changes. Instead of maximizing only the value, there
can be several other objectives.
CMPS 6610 Algorithms 1
4
For example: Consider you are organizing a music show in a hall that has a capacity of 10,000. You
are organizing a show and the size of the audience depends on the popularity of the singers. Also, the
more popular the singer is, the more the fee. You want to maximize the profit and minimize the
amount spend on the singer simultaneously and also want to bring as many singers as possible.

2. Multi-dimensional Knapsack problem:

In this variation of the problem, the weight of any item i is given by an M dimensional vector {wi1, wi2, .
. . wiM} and similarly, the capacity of the knapsack is also an M dimensional vector {W 1, W2, . . . , WM}.

3. Multiple Knapsack problem:

This variation of the knapsack problem is similar to the Bin Packing algorithm. The difference in
both the problem is here we can pick a subset of the items whereas, in the Bin Packing problem, we
have to pack all the items in any of the bins. The idea is that there are multiple knapsacks which may
seem like adding capacity to the initial knapsack, but it is not similar to that at all.
 Double Knapsack

4. Quadratic Knapsack problem:

This variation has the goal of achieving the maximum value of a quadratic objective function that is
subjected to binary and linear capacity constraints.

5. Geometric Knapsack problem:

In this variation, there is a set of rectangles with different values and a rectangular knapsack. The goal
is to pack the largest possible value into the knapsack.
Applications of the Knapsack Problem:
The Knapsack problem has several real-life applications. Some of them are mentioned here:

CMPS 6610 Algorithms 1

5
 One of the early applications of the Knapsack problem was in construction and scoring of exams in
which the test takers have a choice as to which questions they answer.
 The subset sum problem is solved using the concept of the Knapsack problem.
 The multiple objective variations of the Knapsack problem is frequently used for transportation
logistics optimization problems.
 The multiple knapsack problem is often used in many loading and scheduling algorithms in
Operational Research.

CMPS 6610 Algorithms 1

6