Admas University

Department of Computer Science

Design and Analysis of

Algorithms

Chapter-3
Greedy Algorithms

1
 Greedy Algorithm
•A Greedy algorithm is an approach to solving a problem that selects the
most appropriate option based on the current situation.
•This algorithm ignores the fact that the current best result may not bring
about the overall optimal result.
•Even if the initial decision was incorrect, the algorithm never reverses
it.
•This simple, intuitive algorithm can be applied to solve any
optimization problem which requires the maximum or minimum
optimum result.
•The best thing about this algorithm is that it is easy to understand and
implement.

2
 Greedy Algorithm…
• The runtime complexity associated with a greedy solution is
pretty reasonable.
• However, you can implement a greedy solution only if the problem
statement follows two properties mentioned below:
• Greedy Choice Property: Choosing the best option at each phase can
lead to a global (overall) optimal solution.

• Optimal Substructure: If an optimal solution to the complete problem

contains the optimal solutions to the subproblems, the problem has an
optimal substructure.

3
 Greedy Algorithm…
 Greedy algorithms are characterized by the following
features.
1. Greedy approach forms a set or list of candidates 𝑪.
2. Once a candidate is selected in the solution, it is there forever: once
a candidate is excluded from the solution, it is never reconsidered.
3. To construct the solution in an optimal way, Greedy Algorithm
maintains two sets.

One set contains candidates that have already been considered and
chosen, while the other set contains candidates that have been
considered but rejected.

4
 Greedy Algorithm…
The greedy algorithm consists of four functions.
i. Solution Function:- A function that checks whether chosen set
of items provides a solution.
ii. Feasible Function:- A function that checks the feasibility of a
set.
iii. Selection Function:- The selection function tells which of the
candidates is the most promising.
iv. Objective Function:- An objective function, which does not
appear explicitly, but gives the value of a solution.

5
Greedy Algorithm…

Algorithm 3.1 Greedy method

6
 Greedy Algorithm…
Example of Greedy Algorithm
• Problem Statement: Find the best route to reach the
destination city from the given starting point using a greedy
method.

7
 Greedy Algorithm…
• Greedy Solution: In order to tackle this problem, we need to maintain
a graph structure.
• And for that graph structure, we'll have to create a tree structure, which
will serve as the answer to this problem.
• The steps to generate this solution are given below:
 Start from the source vertex.
 Pick one vertex at a time with a minimum edge weight (distance)
from the source vertex.
 Add the selected vertex to a tree structure if the connecting edge
does not form a cycle.
 Keep adding adjacent fringe vertices to the tree until you reach
the destination vertex.

8
 Greedy Algorithm…
• The animation given below explains how paths will be picked up in
order to reach the destination city.

9
 Greedy Algorithm…
• Limitations of Greedy Algorithm
 The greedy algorithm makes judgments based on the
information at each iteration without considering the
broader problem; hence it does not produce the best
answer for every problem.
 The problematic part for a greedy algorithm is analyzing
its accuracy. Even with the proper solution, it is difficult to
demonstrate why it is accurate.
 Optimization problems (Dijkstra’s Algorithm) with
negative graph edges cannot be solved using a greedy
algorithm.
 Moving forward, let’s look at some applications of a
greedy algorithm.

10
 Greedy Algorithm…
• Applications of Greedy Algorithm
 Used for Constructing Minimum Spanning Trees: Prim’s and
Kruskal’s Algorithms used to construct minimum spanning trees are
greedy algorithms.

 Used to Implement Huffman Encoding: A greedy algorithm is

utilized to build a Huffman tree that compresses a given image,
spreadsheet, or video into a lossless compressed file.

 Used to Solve Optimization Problems: Graph - Map Coloring, Graph

- Vertex Cover, Knapsack Problem, Job Scheduling Problem, and
activity selection problem are classic optimization problems solved
using a greedy algorithmic paradigm.

11
 Greedy Algorithm…
•Components of a Greedy Algorithm
A set of candidate solutions (typically represented as a graph)
A way of ranking the candidates according to some criteria
A selection function that picks the best candidate from the set,
according to the ranking
A way of "pruning" the set of candidates, so that it doesn't contain
any solutions that are worse than the one already chosen.
•The first two components are straightforward - the candidate solutions
can be anything, and the ranking criteria can be anything as well.

12
 Greedy Algorithm…
•The selection function is usually just a matter of picking the
candidate with the highest ranking.
•The pruning step is important, because it ensures that the
algorithm doesn't waste time considering candidates that are
already known to be worse than the best one found so far.

13
 Spanning trees
 A spanning tree of a weighted graph with N nodes is a subgraph
that includes all vertices of the graph with the minimum possible
number of edges i.e. there is a unique path between any two nodes
of the subgraph.
 There can be more than one spanning tree.
 A spanning tree is a subset of Graph G, which has all the vertices
covered with minimum possible number of edges.
 Hence, a spanning tree does not have cycles and it cannot be
disconnected.

14
 Spanning trees…
 General Properties of Spanning Tree
 A connected graph G can have more than one spanning tree.
 All possible spanning trees of graph G, have the same number of edges
and vertices.
 The spanning tree does not have any cycle (loops).
 Removing one edge from the spanning tree will make the graph
disconnected.
 Adding one edge to the spanning tree will create a circuit or loop.
 Mathematical Properties of Spanning Tree
 Spanning tree has n-1 edges, where n is the number of nodes (vertices).
 A complete graph can have maximum nn-2 number of spanning trees.
 From a complete graph, by removing maximum e - n + 1 edges, we can
construct a spanning tree.
15
 Spanning trees…
• In the example below given a graph and its spanning trees

• The spanning trees of the above-weighted graph are shown below

16
 Minimum Spanning-Tree Algorithm
 In a weighted graph, a minimum spanning tree is a
spanning tree that has minimum weight than all other
spanning trees of the same graph.
• The minimum spanning tree of a weighted graph
having N nodes is the spanning tree having the minimum
sum of weights of all edges.

17
 Cont…
There are two types of MST algorithms
• Prim's algorithm and Kruskal's algorithm are greedy
algorithms used to find the minimum spanning tree in a
weighted graph.
• Prim's and Kruskal's algorithms have many use cases they are
used in solving traveling salesman
problems, LAN networks, TV networks.
• And in general for any network where we need to find the least
cost subset of this network such that any two nodes are
connected.

18
 Prim's Algorithm
• Prim's algorithm is used to find the minimum spanning tree.
• It is a greedy algorithm that constructs the minimum spanning
tree by selecting the most optimal edge at that point.
 Given a weighted graph of n nodes, prim's algorithm is used to
find the tree having n nodes and n-1 edges such that the sum of
weights of these n−1 edges is the minimum of all possible trees.
• Below are the steps of Prim's algorithm:
 Start with any node in the graph and mark it visited and
include it in the MST.
 Iterate over all adjacent vertices of the chosen vertex which
are not included in the MST and update their values and
parent.
 Repeat the above two steps until all the nodes are included in
the MST. 19
 Cont…
 Prim's algorithm to find minimum cost spanning tree uses the greedy approach.
 Prim's algorithm shares a similarity with the shortest path first algorithms.
 Prim's algorithm, treats the nodes as a single tree and keeps on adding new nodes to
the spanning tree from the given graph. The tree starts from an arbitrary root
vertex r and grows until the tree spans all the vertices in V.

Prim: O(NlogN) search the

least weight edge for every
vertices

20
Cont…

21
 Cont…

Step 1 - Remove all loops and parallel edges

22
 Cont…

 Remove all loops and parallel edges from the

given graph.
In case of parallel edges, keep the one which has
the least cost associated and remove all others.

23
 Cont…
 Step 2 - Choose any arbitrary node as root node
 In this case, we choose S node as the root node of Prim's
spanning tree.
 This node is arbitrarily chosen, so any node can be the root
node.
 One may wonder why any video can be a root node.
 So the answer is, in the spanning tree all the nodes of a
graph are included and because it is connected then there
must be at least one edge, which will join it to the rest of the
tree.

24
 Cont…
 Step 3 - Check outgoing edges and select the one with less
cost
 After choosing the root node S, we see that S,A and S,C are
two edges with weight 7 and 8, respectively. We choose the
edge S,A as it is lesser than the other.

25
 Cont…
Now, the tree S-7-A is treated as one node and we
check for all edges going out from it. We select the
one which has the lowest cost and include it in the
tree.

26
 Cont…
After this step, S-7-A-3-C tree is formed. Now we'll
again treat it as a node and will check all the edges again.
However, we will choose only the least cost edge. In this
case, C-3-D is the new edge, which is less than other
edges' cost 8, 6, 4, etc.

27
 Cont…
 After adding node D to the spanning tree, we now have two
edges going out of it having the same cost, i.e. D-2-T and D-
2-B.
 Thus, we can add either one. But the next step will again
yield edge 2 as the least cost.
 Hence, we are showing a spanning tree with both edges
included.

28
 Kruskal’s algorithm
• Kruskal’s algorithm finds a safe edge to add to the growing forest by
finding, of all the edges that connect any two trees in the forest, an
edge .u; / of least weight.
• Let C1 and C2 denote the two trees that are connected by .u; /.
Since .u; / must be a light edge connecting C1 to some other tree .u; /
is a safe edge for C1.
• Kruskal’s algorithm qualifies as a greedy algorithm because at each
step it adds to the forest an edge of least possible weight.
• It uses a disjoint-set data structure to maintain several disjoint sets of
elements.

29
 Cont…
• Each set contains the vertices in one tree of the current forest.
• The operation FIND-SET.u/ returns a representative element from the
set that contains u.
• Thus, we can determine whether two vertices u and belong to the same
tree by testing whether FIND-SET.u/ equals FIND-SET./.
• To combine trees, Kruskal’s algorithm calls the UNION procedure.

30
 Cont…
A Step 2: Taking next Step 4: Taking next
4 5
6 min edge (B,C) min edge (A,B)
6
B E B
3 A
4
5 7 2
2
C D B E
C D 1 3
1
2
Step 3: Taking next

Step 1: Taking min

min edge (B,E) C D
edge (C,D) B E 1
3
So, we obtained a
C D 2
1 C D minimum
1 spanning tree of cost:
4 + 2 + 1 + 3 = 10
31
 The difference between Prim’s and Kruskal algorithm

Prim’s Algorithm Kruskal’s Algorithm

It starts to build the Minimum Spanning Tree
It starts to build the Minimum Spanning Tree
from the vertex carrying minimum weight in the
from any vertex in the graph.
graph.

It traverses one node more than one time to get

It traverses one node only once.
the minimum distance.

Prim’s algorithm has a time complexity of

O(V2), V being the number of vertices and can Kruskal’s algorithm’s time complexity is O(E
be improved up to O(E log V) using Fibonacci log V), V being the number of vertices.
heaps.

Kruskal’s algorithm can generate

Prim’s algorithm gives connected component as forest(disconnected components) at any instant
well as it works only on connected graph. as well as it can work on disconnected
components

32
 Cont…
Prim’s Algorithm Kruskal’s Algorithm
Kruskal’s algorithm runs faster in sparse
Prim’s algorithm runs faster in dense graphs.
graphs.

It generates the minimum spanning tree starting It generates the minimum spanning tree starting
from the root vertex. from the least weighted edge.

Applications of prim’s algorithm are Travelling

Applications of Kruskal algorithm are LAN
Salesman Problem, Network for roads and Rail
connection, TV Network etc.
tracks connecting all the cities etc.

Prim’s algorithm prefer list data structures. Kruskal’s algorithm prefer heap data structures.

33
 Job Scheduling in Greedy Algorithm
• Job scheduling is the problem of scheduling jobs out of a set of
N jobs on a single processor which maximizes profit as much
as possible.
• Consider N jobs, each taking unit time for execution.
• Each job is having some profit and deadline associated with it.
• Profit earned only if the job is completed on or before its
deadline.
• Otherwise, we have to pay a profit as a penalty.
• Each job has deadline di ≥ 1 and profit pi ≥ 0.
• At a time, only one job can be active on the processor.

34
 Cont…
• The job is feasible only if it can be finished on or before its
deadline.
• A feasible solution is a subset of N jobs such that each job can
be completed on or before its deadline.
• An optimal solution is a solution with maximum profit.
• The simple and inefficient solution is to generate all subsets of
the given set of jobs and find the feasible set that maximizes the
profit.
• For N jobs, there exist 2N schedules, so this brute force
approach runs in O(2N) time

35
 Cont…
• Algorithm for Job Scheduling
• Algorithm for job scheduling is described below:

36
 Cont…
Complexity Analysis of Job Scheduling
• Simple greedy algorithm spends most of the time looking for
the latest slot a job can use.
• On average, N jobs search N/2 slots.
• This would take O(N2) time.
• However, with the use of set data structure (find and union), the
algorithm runs nearly in O(N) time.

37
Cont…

38
Cont…

39
Cont…

40
End of Chapter Three!

41