Algorithms

AKC Notes
The Greedy Method - Knapsack Problem
1. Greedy Method
The greedy method is one of the strategies like Divide and conquer used to solve the problems. This
method is used for solving optimization problems. An optimization problem is a problem that
demands either maximum or minimum results. Let's understand through some terms. The main
function of this approach is that the decision is taken on the basis of the currently available
information. Whatever the current information is present, the decision is made without worrying
about the effect of the current decision in future.
This technique is basically used to determine the feasible solution that may or may not be optimal.
The feasible solution is a subset that satisfies the given criteria. The optimal solution is the solution
which is the best and the most favourable solution in the subset.

2. Characteristics of Greedy method

i. To construct the solution in an optimal way, this algorithm creates two sets where one set
contains all the chosen items, and another set contains the rejected items.
ii. A Greedy algorithm makes good local choices in the hope that the solution should be either
feasible or optimal.

3. Components of Greedy Algorithm

i. Candidate set: A solution that is created from the set is known as a candidate set.
ii. Selection function: This function is used to choose the candidate or subset which can be
added in the solution.
iii. Feasibility function: A function that is used to determine whether the candidate or subset
can be used to contribute to the solution or not.
iv. Objective function: A function is used to assign the value to the solution or the partial
solution.
v. Solution function: This function is used to intimate whether the complete function has been
reached or not.

4. Pseudo code of Greedy Algorithm

AKC NOTES | Semester IV | Algorithms 1

 The function Select selects an input from a[ ] and removes it. The selected input's value is assigned
to x. Feasible is a Boolean-valued function that determines whether x can be included into the
solution vector. The function Union combines x with the solution and updates the objective
function. The function Greedy describes the essential way that a greedy algorithm will look, once a
particular problem is chosen and the functions Select, Feasible, and Union are properly
implemented.

5. Knapsack Problem
We are given n objects and a knapsack or bag. Object i has a weight w; and the knapsack has a
capacity m. If a fraction xi, 0 ≤ i ≤ 1, of object i is placed into the knapsack, then a profit of p ixi is
earned. The objective is to obtain a filling of the knapsack that maximizes the total profit earned.
Since the knapsack capacity is m, we require the total weight of all chosen objects to be at most m.
Formally, the problem can be stated as

The profits and weights are positive numbers.

A feasible solution (or filling) is any set (1,..., In) satisfying above equations. An optimal solution is
a feasible solution for which first equation is maximized.

Algorithm GreedyKnapsack (m, n)

// p[1 : n] and w[1 : n] contain the profits and weights respectively of objects ordered so that p[i]
/ w[i] > // p[i + 1] / w[i + 1]. m is the knapsack size and x[1: n] is the solution vector.
{

for i := 1 to n do
x[i] := 0.0
U := m;
for i := 1 to n do
{
if (w(i) > U) then break;
x [i] := 1.0;
U := U – w[i];
}
if (i < n) then x[i] := U / w[i];
}

Complexity
The objects are to be sorted into non-decreasing order of pi / wi ratio. But if we disregard the time
to initially sort the objects, the algorithm requires only O(n) time.

AKC NOTES | Semester IV | Algorithms 2

6. Theorem: If p1/w1 ≥ p2/w2 ≥ ... ≥ pn/wn, then Greedy Knapsack generates an optimal
solution to the given instance of the knapsack problem.
Let x = (x1,...,xn) be the solution generated by Greedy Knapsack. If all the xi equal one, then clearly
the solution is optimal. So, let j be the least index such that xj ≠ 1. From the algorithm it follows that
x = 1 for 1 < i < j, x = 0 for j<i≤ n, and 0 < x < 1. Let y = (y1,..., yn) be an optimal solution. We know
that assume that Σwiyi = m. Let k be the least index such that yk ≠ xk. Clearly, such a k must exist. It
also follows that yk < xk. To see this, consider the three possibilities k < j, k = j, or k > j.

7. Theorem: All optimal solutions will fill the knapsack exactly.

It is true because we can always increase the contribution of some object i by a fractional amount
until the total weight is exactly m.
<Take one example and explain the steps briefly in your own words>

8. Greedy Algorithm vs Divide and Conquer Algorithm vs Dynamic Algorithm

Divide and Dynamic

Sl.No Greedy Algorithm
conquer Programming
Follows Top-down Follows bottom-up
1 Follows Top-down approach
approach approach
Used to solve optimization Used to solve Used to solve
2
problem decision problem optimization problem
The optimal solution is Solution of The solution of
generated without revisiting subproblem is subproblems is
3 previously generated computed computed once and
solutions; thus, it avoids the recursively more stored in a table for
re-computation than once. later use.
It is used to obtain
a solution to the
It may or may not generate an given problem, it It always generates
4
optimal solution. does not aim for optimal solution.
the optimal
solution
Recursive in
5 Iterative in nature. Recursive in nature.
nature.
More efficient but
Efficient and fast than divide slower than greedy.
and conquer. For instance, For instance, single
Less efficient and
6 single source shortest path source shortest path
slower.
finding using Dijkstra’s Algo finding using Bellman
takes O(ElogV) time Ford Algo takes O(VE)
time.
Some memory is More memory is
7 Extra memory is not required.
required. required to store

AKC NOTES | Semester IV | Algorithms 3

 subproblems for later
use.
Examples: Merge Examples: 0/1
Examples: Fractional
sort, Knapsack,
Knapsack problem,
8 Quick sort, All pair shortest path,
Activity selection problem,
Strassen’s matrix Matrix-chain
Job sequencing problem.
multiplication. multiplication.

9. Difference between Greedy Approach and Dynamic Programming

Feature Greedy Approach Dynamic Programming

Guarantees an optimal solution if
May not always provide an
Optimality the problem exhibits the
optimal solution.
principle of optimality.
Subproblem Does not reuse solutions to Reuses solutions to overlapping
Reuse subproblems. subproblems.
May involve backtracking,
Does not involve
Backtracking especially in top-down
backtracking.
implementations.
Typically, simpler and faster May be more complex and
Complexity
to implement. slower to implement.
Suitable for problems where Suitable for problems with
Application local optimization leads to overlapping subproblems and
global optimization. optimal substructure.
Minimum Spanning Tree, Fibonacci sequence, Longest
Examples
Shortest Path algorithms. Common Subsequence.

AKC NOTES | Semester IV | Algorithms 4