Design & Analysis of Algorithm

(DAA)
Module 2b
(greedy Applications)

Department of CSE, RCCIIT

1
 Table of Content Module 3
• Greedy method:
– definition,
– characteristics , difference with DP,
– Fractional Knapsack algorithm with example,
– job sequencing with deadline algorithm, time complexity, examples
• Spanning tree & Minimal Spanning Tree
– definition,
– kruskal’s algorithm with time complexity, examples
– prim’s algorithm with time complexity, examples
 GREEDY TECHNIQUE
• A greedy algorithm, as the name suggests, always makes the choice that
seems to be the best at that moment. This means that it makes a
locally-optimal choice in the hope that this choice will lead to a
globally-optimal solution.
• Assume that we have an objective function that needs to be optimized
(either maximized or minimized) at a given point.
• A Greedy algorithm makes greedy choices at each step to ensure that the
objective function is optimized.
• The Greedy algorithm has only one shot to compute the optimal solution so
that it never goes back and reverses the decision.

• A Greedy technique consists of four function to find optimal solution –

❑ A function that checks whether chosen set of items provides a solution
❑ A function that checks the feasibility of a set.
❑ The solution function tells which of the candidates is the most promising.
❑ An objective function, which does not appear explicitly gives the value of
a solution.
 GREEDY TECHNIQUE
Greedy Procedure
• The Greedy method works in stages taking only one input at a time.
• At each stage an optimal solution is obtained by making decision on
involvement of the input.
• This is done through a selection function which determines inputs in an
order.
• The next input is only added to the partially optimal solution if it does not
produce any infeasible solution.
• The selection itself is again based on some objective functions.
• For a given problem more objective functions are possible.
• Most of these generate sub optimal solutions.
• This is called subset paradigm of the Greedy approach.
For examples,
Fractional Knapsack problem
Job sequencing With Deadlines problem,
Activity selection problem,
Huffman Codes
Minimum Spanning Tree By Prim's And Kruskal's Algorithms
etc.
GREEDY TECHNIQUE
 Dynamic Programming VS GREEDY
• Dynamic Programming • Greedy Techniques
1) Choice is made at each step 1) We make the greedy choice
2) The choices depends on 2) Solve the sub-problem arising
solutions to sub-problems after the choice is made
3) Decisions are based on all the 3) The choice we make may depend
decisions made in the on previous choices.
previous stage.
4) Uses bottom up approach 4) Uses top down solution,
from smaller to larger problems get decreased in size.
sub-problems.
 Greedy: Fractional Knapsack problem
Fractional Knapsack problem
• There are n items in a store. For I =1,2, . . . , n, item i has weight wi
> 0 and worth vi > 0. Thief can carry a maximum weight of W
pounds in a knapsack. In this case of a problem the items can be
broken into smaller piece,
– Fractional Knapsack is clear that an optimal solution must fill
the knapsack exactly, for otherwise we could add a fraction of
one of the remaining objects and increase the value of the
load.
– Fractional Knapsack problem can be solvable by the greedy
strategy.
 Greedy: Fractional Knapsack problem
Algorithm: Greedy-Fractional-Knapsack (Array w, Array v, W)
Steps:
1. for i =1 to n do
1. p[i]🡪v[i]/w[i]
2. Sort-Descending(p)
3. i🡪1
4. while W>0 do
1. amount🡪min(W, w[i])
2. solution[i]🡪amount
3. W🡪W-amount
4. i🡪i+1
5. return solution.

Analysis
If the items are already sorted into decreasing order of vi / wi, then the
while-loop takes a time in O(n). Therefore, the total time including the
sort is in O(n log n).
 Greedy: Fractional Knapsack problem
• Example 1: Consider 5 items along their respective weights and
values are
wi =(5, 10, 20, 30, 40) and
vi =(30, 20, 100, 90, 160).
The capacity of Knapsack W=60. Find the solution to the
fractional knapsack problem. [Ans: The profit earned is 270]
• Example 2: Consider the following Table that consists of some
items with weights and cost values.
Items I1 I2 I3 I4 I5
Weights 5 10 15 22 25
Cost 30 40 45 77 90

If the Knapsack capacity W=60Kg. Find the feasible solution.

[Ans: The profit earned is 200 ]
 Greedy: Fractional Knapsack problem
Solution:
Given, No. of items n=5, Value of items vi=(30, 20, 100, 90, 160), Weight of
items wi=(5, 10, 20, 30, 40) and the capacity of Knapsack W=60.

We apply Greedy fractional Knapsack Algo: Greedy-knapsack (Array w, Array v, W)

algorithm
Step 1: Initially:
Steps:
1. for i =1 to n do
Item no. wi vi
I1 5 30
1. p[i]🡪v[i]/w[i]
I2 10 20 2. Sort-Descending(p)
I3 20 100 3. i🡪1
I4 30 90
4. while W>0 do
I5 40 160
1. amount🡪min(W, w[i])
Step 2: Taking value per weight
ratio pi=vi/wi 2. solution[i]🡪amount
Item no. wi vi pi=vi/wi 3. W🡪W-amount
I1 5 30 6.0 4. i🡪i+1
I2 10 20 2.0
5. return solution.
I3 20 100 5.0
I4 30 90 3.0
I5 40 160 4.0
 Greedy: Fractional Knapsack problem
Step 3: Now, arranging items with
decreasing order of pi
Algo: Greedy-knapsack (Array w, Array v, W)
Item no. wi vi pi=vi/wi Steps:
I1 5 30 6.0
1. for i =1 to n do
I3 20 100 5.0
I5 40 160 4.0 1. p[i]🡪v[i]/w[i]
I4 30 90 3.0 2. Sort-Descending(p)
I2 10 20 2.0 3. i🡪1
Step 4: Filling the Knapsack according to 4. while W>0 do
decreasing value of pi.
First, we
1. amount🡪min(W, w[i])
•Choose item I1 whose weight is 5 then total 2. solution[i]🡪amount
weight is 5 and value is 30 and remaining
weight is 55
3. W🡪W-amount
•Choose item I3 whose weight is 20 the total 4. i🡪i+1
weight is 5+20=25 and value is 30+100=130 5. return solution.
and remaining weight is 35
•Choose item I5 whose weight is 40 but
remaining weight is 35. so choose fractional
part of weight i.e. 35 and value is
30+100+(160/40)*35=270
 Greedy: Job Sequencing Problem
Job Sequencing Problem
Job sequencing problem is the problem of optimally sequencing
unit-time tasks on a single processor, where each task is a job such
as a problem to be run on a computer that requires exactly one unit
of time to complete.
Given a finite set S of unit-time tasks, a schedule for S is a permutation of S
specifying the order in which these tasks are to be performed.
The 1st job in the schedule begins at time 0 and finishes at time 1 , the 2 nd
job begins at time 1 and finishes at time 2 and so on.
The problem of job sequencing unit time tasks with dead lines and
penalties for a single processor has the following inputs –
▪ A set S={ 1, 2, 3, ………, n} of n- unit time tasks.
▪ A set of n integer deadlines d1, d2, d3, …….,dn such that each di satisfies 1≤i
≤n and task 1 is supposed to finish by time di.
▪ A set of n non-negative weights or penalties w1, w2, …,wn such that a
penalty wi is incurred if task i is not finished by di and no penalty is incurred if
a task finishes by its deadline.
 Greedy: Job Sequencing Problem
Job Sequencing Problem with Deadline problem consists of n jobs each
associated with a deadline and profit and our objective is to earn maximum
profit. We will earn profit only when job is completed on or before deadline.
We assume that each job will take unit time to complete.

Points to remember
---------------------------------------------------------------------------------------------------------------------
1. In this problem we have n jobs j1, j2, … jn each has an associated deadline d1, d2, … dn
and profit p1, p2, ... pn.
2. Profit will only be awarded or earned if the job is completed on or before the deadline.
3. We assume that each job takes unit time to complete.
4. The objective is to earn maximum profit when only one job can be scheduled or
processed at any given time.
---------------------------------------------------------------------------------------------------------------------
Consider the following 5 jobs and their associated deadline and profit.
job id deadline profit
j1 2 60
j2 1 100
j3 3 20
j4 2 40
j5 1 20
 Greedy: Job sequencing Problem
Algorithm: Greedy-Job-sequencing(P, D, n, FS, Profit)
Steps: Analysis: Time complexity of job
1. D[0]🡪FS[0]🡪0 //Initialisation and 1st job inclusion sequencing algorithm is O(n2)
1. FS[1]🡪1
2. K🡪1
3. Profit🡪0
2. For i🡪2 to n Do //For feaseible solution
1. r🡪k
2. While (D[FS[r]]>D[i]) and (D[FS[r]]≠r) Do
1. r🡪r-1
2. If D[FS[r]]≤D[i] and D[i]>r then //Checking deadline violation
1. For m<--k to (r+1) step by -1 Do
1. FS[m+1]🡪FS[m]
2. FS[m+1]🡪I
3. K🡪k+1
3. For i🡪1 to k Do
1. Profit🡪Profit+P[FS[i]]
2. Write FS[i] //Display job sequence
4. Write “Maximum Profit Earned”, Profit. //Display profit
5. Return
 Greedy: Job sequencing Problem
Example 1: Find the feasible solution for the following list of jobs
and also find the total profit earned. Assume that each job needs
one unit time in a single machine.
Job No. I II III IV V
Profit 50 30 10 20 15
Deadline 2 1 3 2 4

Solution: The feasible solution can be obtained step-by-step as follows-

First, Sort the profit values in descending order according place job
positions and keep their deadlines as they are
Profit 50 30 20 15 10
New Job No. I II III IV V
Deadline =
Deadline 2 1 3 2 4
Old Job No. I II IV V III

Max(deadline of all jobs)=Max(2,1,3,2,4)=4. Four times slot will be there i.e. at most 4
jobs will be completed within their deadlines
 Greedy: Job sequencing Problem
Now, considering the new job numbering in the above table, jobs are
scheduled as-
Job I is allotted in [0, 1] time slot.
Time Slot 0-1 1-2 2-3 3-4 4-5
New Job No. 1
Deadline 2
Old Job No. 1

Job II is allotted in [1, 2] time slot. But its deadline is 1. So we have to

shift job 1 upward
Time Slot 0-1 1-2 2-3 3-4
New Job No. 1🡪
Deadline 2
Old Job No. 1
 Greedy: Job sequencing Problem
Time Slot 0-1 1-2 2-3 3-4
New Job No. II 1
Deadline I 2
Old Job No. II 1

Job III is allotted in [2, 3] time slot.

Time Slot 0-1 1-2 2-3 3-4

New Job No. II I III
Deadline 1 2
Old Job No. II I
 Greedy: Job sequencing Problem
Job IV is allotted in [3, 4] time slot. But its deadline is 2. Already slot
[1,2] has been used. So this job IV is rejected. Now, Job V is allotted
in [3, 4] time slot.
Time Slot 0-1 1-2 2-3 3-4
New Job No. II I III V
Deadline 1 2 3 4
Old Job No. II I IV II

Thus, the feasible solution =< I, II, III, V > [new job no.]
Execution sequence = < II, I, III, V > [new job no.]
Profit earned = Profit[II] + Profit[I] + Profit[III] + Profit[V]
= 30 + 50 + 10 + 15
= 105
 MCQ
• Fractional knapsack problem is also known as __________
a) 0/1 knapsack problem
b) Continuous knapsack problem
c) Divisible knapsack problem
d) Non continuous knapsack problem
• Fractional knapsack problem is solved most efficiently by which of the
following algorithm?
a) Divide and conquer
b) Dynamic programming
c) Greedy algorithm
d) Backtracking
• What is the objective of the knapsack problem?
a) To get maximum total value in the knapsack
b) To get minimum total value in the knapsack
c) To get maximum weight in the knapsack
d) To get minimum weight in the knapsack
 MCQ
• Time complexity of fractional knapsack problem is ____________
a) O(n log n)
b) O(n)
c) O(n2)
d) O(nW)
• Given items as {value,weight} pairs {{40,20},{30,10},{20,5}}. The capacity of
knapsack=20. Find the maximum value output assuming items to be
divisible.
a) 60
b) 80
c) 100
d) 40
• The main time taking step in fractional knapsack problem is ___________
a) Breaking items into fraction
b) Adding items into knapsack
c) Sorting
d) Looping through sorted items
 MCQ
• Given items as {value,weight} pairs {{60,20},{50,25},{20,5}}. The capacity of
knapsack=40. Find the maximum value output assuming items to be divisible and
nondivisible respectively.
a) 100, 80
b) 110, 70
c) 130, 110
d) 110, 80
• job sequencing with deadline is based on ____________method
a. greedy method
b. branch and bound
c. dynamic programming
d. divide and conquer
• Greedy job sequencing with deadlines algorithms’ complexity is defined as
a. O(N)
b. Ω( n log n)
c. O (n2 log n)
d. O ( n log n)
 Greedy: Spanning Tree
Spanning Tree:
– A spanning tree of a graph is any tree that includes every vertex in the
graph.
– A spanning tree of a graph G is a subgraph of G that is a tree and contains all
the vertices of G. An edge of a spanning tree is called a branch; an edge in
the graph that is not in the spanning tree is called a chord.
– Given a connected undirected graph, a spanning tree of that graph is a
subgraph that is a tree and joined all vertices. A single graph can have many
spanning trees.
Greedy: Spanning Tree
 Greedy: Minimum Spanning Tree
A minimum spanning tree (MST) of a weighted graph G is a spanning tree of G
whose edges sum is minimum weight. Let G=(V, E) be a connected, undirected
graph where V is a set of vertices (nodes) and E is the set of edges. Each edge has a
given non negative length.
In other words, a MST is a tree formed from a subset of the edges in a given
undirected graph, with three properties:
✔ It spans the graph, i.e., it includes every vertex of the graph without form of
any cycle.
✔ It is a minimum, i.e., the total weight of all the edges is as minimum as
possible.
✔ Connects all vertices in V

There are two methods to find

Minimum Spanning Tree –

1.Kruskal's Algorithm

2.Prim's Algorithm
 Greedy: Prime’s
• Prim's Algorithm is used to find the minimum spanning tree from a graph.
Prim's algorithm finds the subset of edges that includes every vertex of the
graph such that the sum of the weights of the edges can be minimized.
• Prim's algorithm starts with the single node and explore all the adjacent nodes
with all the connecting edges at every step. The edges with the minimal
weights causing no cycles in the graph got selected.
•Prime’s algorithm for finding the minimum
spanning tree of a given weighted graph G.
Steps:
1.Select a starting vertex
2.Select an edge e connecting the tree
vertex and fringe vertex that has
minimum weight
3.Add the selected edge and the vertex
to the minimum spanning tree T
4.Repeat 2 until all vertices are
connected
5.Exit
 Greedy: Prime’s Example
Construct a minimum spanning tree of the graph given in the following
figure by using prim's

Solution
Step 1 : Choose a starting vertex B.

Step 2: Add the vertices that are

adjacent to A. the edges that
connecting the vertices are shown
by dotted lines.
 Greedy: Prime’s Example
Step 3: Choose the edge with the minimum weight
among all. i.e. BD and add it to MST. Add the
adjacent vertices of D i.e. C and E.

Step 4: Choose the edge with the minimum weight

among all. In this case, the edges DE and CD are
such edges. Add them to MST and explore the
adjacent of C i.e. E and A.

Step 5: Choose the edge with the minimum weight

i.e. CA. We can't choose CE as it would cause cycle
in the graph.
The graph produces in the step 5 is the minimum
spanning tree of the graph shown in the above figure.
The cost of MST will be calculated as;
cost(MST) = 4 + 2 + 1 + 3 = 10 units.
 Greedy: Kurskal’s
• Kurskal’s algorithm for finding the minimum spanning tree of a given
weighted graph G for applying this algorithm, a list of all edges of G in
the non-decreasing order is maintained but this is similar to the Prime’s
algorithm.
• It is a Greedy Algorithm. The Greedy Choice is to put the smallest
weight edge that does not because a cycle in the MST constructed so
far.
 Greedy: Kurskal’s Algorithm
Steps for finding MST using Kruskal's Algorithm
Steps:
1.Arrange all the edges in increasing order of their weight
2.Include to minimum spanning tree edge if it does not
form a cycle
3.Continue till all the edges are visited and an MST is
formed
4.Add the weight of all edges(take union) in MST to get
the minimum cost of the spanning tree.
5.Stop.

Analysis: In this algorithm,

running time to sort the edges
takes O(|E|log|E|) time and
other steps takes, time less than
or equal to this,

So the Kruskal’s algorithm takes

O(|E|log|E|) or O(|E|log|V|)
time.
Difference between Kurskal’s and Prim’s algorithm
 Greedy: Kurskal’s & Prim’s
• Example, Find a minimum cost spanning tree using Prim’s and
Kruskal’s algorithms of the given graph.
 Greedy: Kurskal’s Example
Solution for Kurskal’s
•The graph contains 9 vertices and 14 edges. So,
the minimum spanning tree formed will be
having (9 – 1) = 8 edges.
•After sorting:

Now pick all edges one by one from the sorted

list of edges
1. Pick edge 7-6: No cycle is formed, include it.

2. Pick edge 8-2: No cycle is formed, include it.

3. Pick edge 6-5: No cycle is formed, include it.

4. Pick edge 0-1: No cycle is formed, include it.

5. Pick edge 2-5: No cycle is formed, include it.

 Greedy: Kurskal’s Example
6. Pick edge 8-6: Since including this edge results in the
cycle, discard it.
7. Pick edge 2-3: No cycle is formed, include it.
8. Pick edge 7-8: Since including this edge results in the
cycle, discard it.
9. Pick edge 0-7: No cycle is formed, include it.
10. Pick edge 1-2: Since including this edge results in
the cycle, discard it.
11. Pick edge 3-4: No cycle is formed, include it.
Since the number of edges included equals (V – 1), the
algorithm stops here.
Prim’s Algorithm Vs Kruskal’s Algorithm
 MCQ
• Kruskal’s algorithm is used to ______
a) find minimum spanning tree
b) find single source shortest path
c) find all pair shortest path algorithm
d) traverse the graph

• Kruskal’s algorithm is a ______

a) divide and conquer algorithm
b) dynamic programming algorithm
c) greedy algorithm
d) approximation algorithm

• What is the time complexity of Kruskal’s algorithm?

a) O(log V)
b) O(E log V)
c) O(E2)
d) O(V log E)
 MCQ
• What is the weight of the minimum spanning tree using the Kruskal’s
algorithm?
a) 24
b) 23
c) 15
d) 19

• Which of the following is true?

a) Prim’s algorithm can also be used for disconnected graphs
b) Kruskal’s algorithm can also run on the disconnected graphs
c) Prim’s algorithm is simpler than Kruskal’s algorithm
d) In Kruskal’s sort edges are added to MST in decreasing order of their
weights
 MCQ
• Prim’s algorithm is a ______
a) Divide and conquer algorithm
b) Greedy algorithm
c) Dynamic Programming
d) Approximation algorithm
• Worst case is the worst case time complexity of Prim’s algorithm if
adjacency matrix is used?
a) O(log V)
b) O(V2)
c) O(E2)
d) O(V log E)
• What is the weight of the minimum spanning tree using the Prim’s
algorithm,starting from vertex a?
a) 23
b) 28
c) 27
d) 11
 MCQ
• Using Prim's algorithm to construct a minimum spanning tree starting with
node A, which one of the following sequences of edges represents a possible
order in which the edges would be added to construct the minimum
spanning tree?
a. (E, G), (C, F), (F, G), (A, D), (A, B), (A, C)
b. (A, D), (A, B), (A, C), (C, F), (G, E), (F, G)
c. (A, B), (A, D), (D, F), (F, G), (G, E), (F, C)
d. (A, D), (A, B), (D, F), (F, C), (F, G), (G, E)

• Consider a complete graph G with 4 vertices. The graph G has ____ spanning
trees.
a) 15
b) 8
c) 16
d) 13
 References
• https://youtu.be/-Cg-aL1D8CM
• https://youtu.be/MIvWBI8Xlyk
• https://youtu.be/7FtGk9yr66A
• https://youtu.be/M9FMtiCvvhs
Thank you !

H. Tunga 40