Chapter-Three

Greedy–Method
 Basic Topics of Chapter-Three

• At the end of this chapter, you will be able to:

– Understand the concept of greedy algorithms
– Solve job sequencing problem using greedy algorithms
– Optimal Merge Pattern
– Find Minimum spanning tree using greedy algorithms
• Prim’s algorithm

• Kruskal’s algorithm

– Fractional Knapsack

2
 Introduction
• Greedy algorithms are simple and the strategies for solving the
problem.
• Just like divide –conquer and other strategies greed algorithm is one of
the approach solving the problem, but the difference is that solutions to
the sub problems do not have to be known at each stage.
• They are short-sighted in their approach

• It is used to solve the optimization problems.

• An optimization problem is one in which you want to find, not just a

solution, but the best solution.
 What is greedy Algorithm?
• A greedy algorithm is an algorithm that follows the problem-solving
heuristic of making the locally optimal choice at each stage with the hope
of finding a global optimum.
• A feasible solution to which the optimization function has the best
possible value is called an optimal solution.
• In greedy algorithm approach, decisions are made from the given solution
domain.

• As being greedy, the closest solution that seems to provide an

optimum solution is chosen.
– At each choice, we make a decision that appears to be the best at that time.
4
 General Example: Greedy algorithm
• Suppose Ali want to travel from country A to B within 6 hrs. there are some
possibilities he can travel from country A to B some of them are: (walk, bus,
car, train, flight), but there are constraints in the problem. He want to cover
within 6 hrs. but he can not cover by walk and by car, he can only cover by
train or by flight. The train and flight satisfies this condition, so this kind of
condition is called feasible solutions.(i.e. solution which satisfies some
constrains.)
• From the two feasible solution one of the solution take minimum cost. Suppose
the train take minimum cost that is optimal solution(i.e. solution which is
achieving the objective of the problem).
• Note: optimization problems –the problem which is required either minimum
5
or maximum results.
 General Greedy Method Algorithm
Let a[ ] be an array of elements that may contribute to a solution.
Let S be a solution set,

Algorithm: Greedy (a[ ],n)

{
S = empty;
for each element (i) from a[ ], i = 1 to n
{
x = Select (a, i);
if (Feasible (S, x))
S = Union (S, x);
}
return S;
}

6
 …continue

• Select:

Selects an element from a[ ] and removes it. Selection is optimized

to satisfy an objective function.
• Feasible:

True if selected value can be included in the solution vector, False

otherwise.
• Union:

Combines value with solution and updates objective function.

7
 Greedy Algorithm: Counting Coins
• This problem is to count to a desired value by choosing the least possible
coins and the greedy approach forces the algorithm to pick the largest
possible coin.
E.g. Greedy algorithm to find minimum number of coins:
Q. Given a set of coins and a value we have to find the minimum number of coins
which satisfies the value.

8
 Most networking algorithms use the greedy approach.
• Here is a list of few of them −
– Job Scheduling Problem

– Minimum Spanning Trees

• Prim's Minimal Spanning Tree Algorithm

• Kruskal's Minimal Spanning Tree Algorithm

– Single-Source Shortest Paths Problem

• Dijkstra's Minimal Spanning Tree Algorithm

– Graph - Map Coloring

– Graph - Vertex Cover

– Fractional Knapsack Problem

• There are lots of similar problems that uses the greedy approach to find an optimum
solution. 9
 Job sequencing with deadlines

• Job sequencing, as the name suggests, is the ordering of jobs with

given deadlines and profits, in order to maximize the profit earned. Or

• The sequencing of jobs on a single processor with deadline

constraints is called as Job Sequencing with Deadlines.
– N jobs to execute, each of which takes unit time.

– At any time t = 1, 2, 3… we can execute exactly one job.

– Job i earns us a profit Pi > 0 if and only if it is executed no later

than time di.

• That is; we have N Jobs, Profits Pi and Deadlines di for each job.
10
 Job S. with deadlines: Problem Statement

• In job sequencing problem, the objective is to find a sequence of jobs, which is

completed within their deadlines and gives maximum profit.
• Here-

– You are given a set of jobs.

– Each job has a defined deadline and some profit associated with it.

– The profit of a job is given only when that job is completed within its deadline.

– Only one processor is available for processing all the jobs.

– Processor takes one unit of time to complete a job.

• The problem states-

• “How can the total profit be maximized if only one job can be completed
at a time?”
11
 Approach to Solution
• Let us consider, a set of n given jobs which are associated with deadlines
and profit is earned, if a job is completed by its deadline.
• These jobs need to be ordered in such a way that there is maximum profit.
• It may happen that all of the given jobs may not be completed within their
deadlines.
• A feasible solution would be a subset of jobs where each job of the
subset gets completed within its deadline.
• Value of the feasible solution would be the sum of profit of all the jobs
contained in the subset.
• An optimal solution of the problem would be a feasible solution which
gives the maximum profit. 12
 Greedy Approach
• Greedy Algorithm is adopted to determine how the next job is
selected for an optimal solution.
• Feasibility of a Set of Jobs: A set of jobs is feasible if there exists
at least one sequence that allows all the jobs to be executed no
later than their deadlines.
• Construct the schedule step by step.
• Add the job with highest value of Pi among those not considered,
provided that
– The chosen set of jobs remains feasible.
• Assumption every job takes single unit of time 13
 Steps to solve Job S. with deadlines
Step1:
 Sort all the given jobs in decreasing order of their profit.
 Initially, these jobs are ordered according to profit, i.e. p1 ⩾p2 ⩾p3 ⩾... ⩾pn
 if two or more jobs having the same profit then sort them as per their entry
in the job.
Step2:
 Check the value of maximum deadline.

 Draw a Gantt chart where maximum time on Gantt chart is the value of maximum

deadline
Step3: Pick up the jobs one by one.

– Put the job on Gantt chart as far as possible from 0 ensuring that the job gets

completed before its deadline.

14
 PRACTICE PROBLEM BASED ON JOB SEQUENCING WITH
DEADLINES-
• Problem-

Given the jobs, their deadlines and associated profits as shown-

• Answer the following questions-

a. Write the optimal schedule that gives maximum profit.(i.e. find a sequence of jobs, which

will be completed within their deadlines )

b. Are all the jobs completed in the optimal schedule?

c. What is the maximum earned profit? 15

 Solution
Step-01:
• Sort all the given jobs in decreasing order of their profit-
– Jobs J4 J1 J3 J2 J5 J6
– Deadlines 2 5 3 3 4 2
– Profits 300 200 190 180 120 100
Step-02:
• Value of maximum deadline = 5.
• So, draw a Gantt chart with maximum time on Gantt chart = 5 units as shown-

• Now,
• We take each job one by one in the order they appear in Step-01.
• We place the job on Gantt chart as far as possible from 0.
16
 Solution….
• Step-03:
• We take job J4.
• Since its deadline is 2, so we place it in the first empty cell before deadline 2 as-

• Step-04:
• We take job J1.
• Since its deadline is 5, so we place it in the first empty cell before deadline 5 as-

17
 Solution….
• Step-06:
• We take job J2.
• Since its deadline is 3, so we place it in the first empty cell before deadline 3.
• Since the second and third cells are already filled, so we place job J2 in the first cell
as-

• Step-07:
• Now, we take job J5.
• Since its deadline is 4, so we place it in the first empty cell before deadline 4 as-

• Now,
• The only job left is job J6 whose deadline is 2.
• All the slots before deadline 2 are already occupied.
• Thus, job J6 can not be completed. 18
 Solution….
• Now, the given questions may be answered as-
a
– The optimal schedule is-
– J2 , J4 , J3 , J5 , J1
– This is the required order in which the jobs must be completed in order to
obtain the maximum profit.
b:
– All the jobs are not completed in optimal schedule.
– This is because job J6 could not be completed within its deadline.
c:
– Maximum earned profit = Sum of profit of all the jobs in optimal schedule
= Profit of job J2 + Profit of job J4 + Profit of job J3 + Profit of job J5 +
Profit of job J1
= 180 + 300 + 190 + 120 + 200
= 990 units
19
 Job sequencing with deadlines: Exercise #1
• Let us consider a set of given jobs as shown in the following table.

• Answer the following questions-

a. Write the optimal schedule that gives maximum profit.(i.e. find a

sequence of jobs, which will be completed within their deadlines )

b. Are all the jobs completed in the optimal schedule?

c. What is the maximum earned profit?

20
 Job Sequencing Algorithm

Steps:
1) Sort all jobs in decreasing order of profit.
2) Initialize the result sequence as first job in sorted jobs.
3) Do following for remaining n-1 jobs
If the current job can fit in the current result …….sequence without
missing the deadline,
add …..current job to the result.
Else
ignore the current ……….job

21
 Job Sequencing Algorithm
Algorithm greedy job(d,j,n)
{
for i = 1 to n do
Set k = min(dmax, DEADLINE(i)) //where DEADLINE(i) denotes deadline of ith
job
while k >= 1 do
if timeslot[k] is EMPTY then
timeslot[k] = job(i)
break
endif
Set k = k - 1
end while
End for
• Analysis: In this algorithm, we are using two loops, one is within another. Hence, the
complexity of this algorithm is O(n2)
22
 Optimal Merge Pattern
• Merge a set of sorted files of different length into a single sorted file.
• We need to find an optimal solution, where the resultant file will be
generated in minimum time.
• If the number of sorted files are given, there are many ways to merge
them into a single sorted file.
• This merge can be performed pair wise. Hence, this type of merging is
called as 2-way merge patterns.
• As, different pairings require different amounts of time, in this strategy
we want to determine an optimal way of merging many files together.
• At each step, two shortest sequences are merged.
23
 Optimal Merge Pattern
• To merge a p-record file and a q-record file requires possibly p +
q record moves, the obvious choice being, merge the two smallest files
together at each step.
• When two or more sorted files are to be merged all together to form a
single file, the minimum computations done to reach this file are known
as Optimal Merge Pattern.
• Two-way merge patterns can be represented by binary merge trees. Let
us consider a set of n sorted files {f1, f2, f3, …, fn}.

• Initially, each element of this is considered as a single node binary tree.

• To find this optimal solution, the following algorithm is used.

24
 Optimal Merge Pattern: Algorithm
Algorithm: TREE (n)
for i := 1 to n – 1 do
declare new node
node.leftchild := least (list)
node.rightchild := least (list)
node.weight) := ((node.leftchild).weight) +
((node.rightchild).weight)
insert (list, node);
return least (list);

• At the end of this algorithm, the weight of the root node

25
represents the optimal cost.
 Example #1

• Let us consider the given files, A, B, C, and D with 6, 5, 2, and 3

number of elements respectively. What is total cost?

• Therefore: this observation we say that greedy method

• Total cost =31
26
 Example: #2

• Let us consider the given files, f1, f2, f3, f4 and f5 with 20, 30, 10, 5 and 30
number of elements respectively.
• If merge operations are performed according to the provided sequence,
then
– M1 = merge f1 and f2 => 20 + 30 = 50

– M2 = merge M1 and f3 => 50 + 10 = 60

– M3 = merge M2 and f4 => 60 + 5 = 65

– M4 = merge M3 and f5 => 65 + 30 = 95

• Hence, the total number of operations is 50 + 60 + 65 + 95 = 270

• Now, the question arises is there any better solution?

27
 Solution…
• Sorting the numbers according to their size in an ascending order, we get the
following sequence −

• f4 , f 3 , f 1 , f 2 , f 5

• Hence, merge operations can be performed on this sequence

– M1 = merge f4 and f3 => 5 + 10 = 15

– M2 = merge M1 and f1 => 15 + 20 = 35

– M3 = merge M2 and f2 => 35 + 30 = 65

– M4 = merge M3 and f5 => 65 + 30 = 95

• Therefore, the total number of operations is

• 15 + 35 + 65 + 95 = 210

• Obviously, this is better than the previous one. 28

 Solution…
• In this context, we are now going
to solve the problem using this
algorithm.

Hence, the solution takes 15 + 35 +

60 + 95 = 205 number of
comparisons.

29
 Spanning Tree

• A spanning tree is a subset of an undirected Graph that has all the

vertices connected by minimum number of edges.
• If all the vertices are connected in a graph, then there exists at least
one spanning tree.
• In a graph, there may exist more than one spanning tree.
• Properties
– A spanning tree does not have any cycle.
– Any vertex can be reached from any other vertex.

30
Spanning Tree: Example #1

31
 Minimum Spanning Trees
• A Minimum Spanning Tree (MST) is a subset of edges of a
connected weighted undirected graph that connects all the vertices
together with the minimum cost for that graph.
• One graph may have more than one spanning tree.
• If there are n number of vertices, the spanning tree should have

n - 1 number of edges.
• If each edge of the graph is associated with a weight and there exists
more than one spanning tree, we need to find the minimum spanning
tree of the graph.
• To derive an MST, Prim’s algorithm or Kruskal’s algorithm can be used.
32
 MST : Example #2

• Note: We will use Kruskal’s algorithm and Prim’s algorithm to find

the minimum spanning tree.
33
 Algorithm Characteristics

• Both Prim’s and Kruskal’s Algorithms work with undirected graphs

• Both work with weighted and unweighted graphs but are more

interesting when edges are weighted.

• Both are greedy algorithms that produce optimal solutions

34
 Prim's Algorithm

• This algorithm starts with one node.

• It then, one by one, adds a node that is unconnected to the new

graph, each time selecting the node whose connecting edge has the

smallest weight out of the available nodes’ connecting edges.

• Prim's Algorithm Steps:

1st. Choose an arbitrary start vertex with minimum edge

2nd. Keep including connected min edge[NO CYCLE]

3rd . Calculate total cost

35
 MST: Example
• Take vertices containing the minimum edge and expand the
tree by comparing(take the minimum one) the edges
connected to the tree.

• The cost of the tree is 10+25+22+12+16+14=99

36
 Prim's Algorithm brief Example
• Consider the connected graph below

37
 Kruskal's Algorithm

• This algorithm creates a forest of trees.

• Initially the forest consists of n single node trees (and no edges).

• At each step, we add one edge (the cheapest one) so that it joins

two trees together.

• If it were to form a cycle, it would simply link two nodes that were

already part of a single connected tree, so that this edge would not

be needed.
38
 Kruskal's Algorithm: The Steps
• The steps are:

1. The forest is constructed - with each node in a separate tree.

2. The edges are placed in a priority queue.
3. Until we've added n-1 edges,
a. Extract the cheapest edge from the queue,
b. If it forms a cycle, reject it,
c. Else add it to the forest. Adding it to the forest will join two
trees together.
• Every step will have joined two trees in the forest together, so that at the
end, there will only be one tree in T. 39
Kruskal's Algorithm: Example

40
 Solution

• 1st iteration • 2nd iteration

41
 Solution…

• 3rd iteration • 4th iteration

42
 Solution…

• 5th-Iteration • 6th-iteration

43
 Solution…

• 7th-Iteration • 8th-Iteration

44
Final solution…

45
 Knapsack-Problem
 The Greedy algorithm could be understood very well with a well-
known problem referred to as Knapsack problem.
 The Idea of Knapsack Problem:
– list of items is given, each item has its own value and weight. Items can be
placed in a knapsack whose maximum weight limit is W. The problem is to
find the weight that is less than or equal to W, and value is maximized.

Example: look at this picture

46
 Knapsack-Problem Cont’d
• Problem Scenario
• A thief is robbing a store and can carry a maximal weight of W into his
knapsack. There are n items available in the store and weight of ith item
is wi and its profit is pi.

• What items should the thief take?

– In this context, the items should be selected in such a way that the
thief will carry those items for which he/she will gain maximum profit.
– Hence, the objective of the thief is to maximize the profit.
• Based on the nature of the items, Knapsack problems are categorized as

i. Fractional Knapsack and ii. 0 – 1 Knapsack

47
 Fractional Knapsack-Greedy method
• In Fractional Knapsack, we can break(divisible) items for maximizing the total
value of knapsack.
• This problem in which we can break an item is also called the fractional
knapsack problem.
• For the 0 – 1 Knapsack, items cannot be divided into smaller pieces (we will
discus later)

• Fractional Knapsack uses 3 approach to get maximum profits:

1st. Select item having maximum profit

2nd. Select item having minimum weight

3rd . Find out the ratio of profit by weight and then select the item having
maximum profit by ratio 48
 Fractional Knapsack: Example #1
• Let us consider that the capacity of the knapsack W = 60 and the list of provided
items are shown in the following table −

• As the provided items are not sorted based on pi/wi. After sorting,
the items are as shown in the following table.

49
 Solution
 After sorting all the items according to pi/wi.
 First all of B is chosen as weight of B is less than the capacity of
the knapsack.
 Next, item A is chosen, as the available capacity of the knapsack is
greater than the weight of A.
 Now, C is chosen as the next item.
 However, the whole item cannot be chosen as the remaining
capacity of the knapsack is less than the weight of C.

50
 Solution…

• Hence, fraction of C (i.e. (60 − 50)/20) is chosen.

• Now, the capacity of the Knapsack is equal to the selected items.
Hence, no more item can be selected.
• The total weight of the selected items is 10 + 40 + 20 * (10/20) =
60
• And the total profit is 100 + 280 + 120 * (10/20) = 380 + 60 = 440
• This is the optimal solution. We cannot gain more profit selecting
any different combination of items.

51
 Example #2:Fractional Knapsack

• Number of item n=7, Bag capacity W=15 & the weights and profits
are given below.
Item A B C D E F G
Profit(P) 5 10 15 7 8 9 4
Weight(w) 1 3 5 4 1 3 2

Question: Find the chosen item with maximum profit ?

52
Question

53