Greedy Algorithms

Chapter Three
 Introduction
• The greedy method is one of the strategies like Divide and conquer used to
solve the problems.
• This method is used for solving optimization problems.
• Optimization problem is a mathematical problem that involves finding the
best solution from a set of possible solutions that meet certain constraints.
• The goal is to maximize or minimize a specific objective function
• The Greedy method is the simplest and straightforward approach. It is not an
algorithm, but it is a technique.
• The main function of this approach is that the decision is taken on the basis of
the currently available information.
• Whatever the current information is present, the decision is made without
worrying about the effect of the current decision in future.
 Cont’d…
• Greedy algorithm is an algorithmic paradigm based on heuristic that
follows local optimal choice at each step with the hope of finding global
optimal solution.
• Heuristics are a method of problem solving which uses shortcuts in a
given limited time frame to produce almost perfect solutions.
• this technique is basically used to determine the feasible solution that
may or may not be optimal.
• The feasible solution is a subset that satisfies the given criteria.
• The optimal solution is the solution which is the best and the most
favorable solution in the subset.
 General Characteristic of Greedy
Algorithms
• Greedy algorithms are a class of algorithms that make locally optimal choices at
each stage with the hope of finding a global optimum. They are characterized by
their simplicity and efficiency but may not always guarantee the best solution.
Here are some general characteristics of greedy algorithms:
• Greedy Choice Property:
• Greedy algorithms make the best possible choice at each step, based on the
current information.
• The choice made at each stage does not depend on the choices made in previous
stages.
• Optimal Substructure:
• A problem has an optimal substructure if an optimal solution to the problem
contains optimal solutions to its subproblems.
• Greedy algorithms often work well when a problem exhibits optimal substructure,
meaning that an optimal solution to the problem can be constructed from optimal
solutions to its subproblems.
Cont..
• Greedy Choice vs. Global Optimum:
• The fact that a greedy choice is locally optimal does not guarantee that it will lead to a
globally optimal solution.
• In some cases, a sequence of locally optimal choices might not result in the best overall
solution.
• Efficiency:
• Greedy algorithms are often efficient and easy to implement.
• They are suitable for solving optimization problems when the problem structure allows for
making greedy choices.
• Not Always Correct:
• Greedy algorithms do not always produce correct or optimal solutions for every problem.
• It is crucial to analyze the problem and determine whether the greedy approach is suitable
and likely to lead to the desired solution.
• No Backtracking:
• Greedy algorithms make decisions by selecting the best available option at the current stage
without reconsidering previous choices.
• Once a decision is made, it is not revisited or changed later, even if a better solution might be
found by reconsidering previous decisions.
Pseudo code
• Algorithm Greedy (a, n)
• {
• Solution : = 0;
• for i = 0 to n do
• {
• x: = select(a);
• if feasible(solution, x)
• {
• Solution: = union(solution , x)
• }
• return solution;
• }}
Examples of some greedy
approaches
• Most networking algorithms use the greedy approach.
Here is a list of few of them −
• Prim's Minimal Spanning Tree Algorithm
• Kruskal's Minimal Spanning Tree Algorithm
• Dijkstra's algorithm for finding the shortest path
• Job Scheduling Problem
Prim’s Minimal Spanning Tree
• Prim’s minimal spanning tree algorithm is one of the efficient methods to
find the minimum spanning tree of a graph.
• A minimum spanning tree is a subgraph that connects all the vertices
present in the main graph with the least possible edges and minimum
cost (sum of the weights assigned to each edge) ,it does not form any
cycle
• The algorithm, similar to any shortest path algorithm, begins from a
vertex that is set as a root and walks through all the vertices in the graph
by determining the least cost adjacent edges.
Cont’d…
Prim’s Algorithm
• To execute the prim’s algorithm, the inputs taken by the algorithm are
the graph G {V, E}, where V is the set of vertices and E is the set of
edges, and the source vertex S.
• A minimum spanning tree of graph G is obtained as an output.
Algorithm
• Declare an array visited[] to store the visited vertices and firstly, add the
arbitrary root, say S, to the visited array.
• Check whether the adjacent vertices of the last visited vertex are present in
the visited[] array or not.
• If the vertices are not in the visited[] array, compare the cost of edges and
add the least cost edge to the output spanning tree.
• The adjacent unvisited vertex with the least cost edge is added into
the visited[] array and the least cost edge is added to the minimum spanning
tree output.
• Steps 2 and 4 are repeated for all the unvisited vertices in the graph to
obtain the full minimum spanning tree output for the given graph.
• Add the chosen edge to the MST if it does not form any cycle.
• Calculate the cost of the minimum spanning tree obtained.
Examples
• Find the minimum spanning tree using prim’s method (greedy
approach) for the graph given below with S as the arbitrary root.
Solution
• Step 1
• Create a visited array to store all the visited vertices into it.
• V={}
• The arbitrary root is mentioned to be S, so among all the edges that
are connected to S we need to find the least cost edge.
• S→B=8
• V = {S, B}
Cont’d…
• Step 2
• Since B is the last visited, check for the least cost edge that is
connected to the vertex B.
•B→A=9
• B → C = 16
• B → E = 14
• Hence, B → A is the edge added to the spanning tree.
• V = {S, B, A}
Cont’d…
• Step 3
• Since A is the last visited, check for the least cost edge that is
connected to the vertex A.
• A → C = 22
• A→B=9
• A → E = 11
• But A → B is already in the spanning tree, check for the next least cost
edge. Hence, A → E is added to the spanning tree.
• V = {S, B, A, E}
Cont’d…
• Step 4
• Since E is the last visited, check for the least cost edge that is connected
to the vertex E.
• E → C = 18
• E→D=3
• Therefore, E → D is added to the spanning tree.
• V = {S, B, A, E, D}
Cont’d…
• Step 5
• Since D is the last visited, check for the least cost edge that is
connected to the vertex D.
• D → C = 15
• E→D=3
• Therefore, D → C is added to the spanning tree.
• V = {S, B, A, E, D, C}
• The minimum spanning tree is obtained with the minimum cost = 46
Time complexity of MST

Data structure used for the minimum edge Time Complexity

weight

Adjacency matrix, linear searching O(|V|2)

Adjacency list and binary heap O(|E| log |V|)

Adjacency list and Fibonacci heap O(|E|+ |V| log |V|)

 Kruskal’s Minimal Spanning Tree
• Kruskal’s minimal spanning tree algorithm is one of the efficient methods to
find the minimum spanning tree of a graph.
• A minimum spanning tree is a subgraph that connects all the vertices present
in the main graph with the least possible edges and minimum cost (sum of
the weights assigned to each edge).
• The algorithm first starts from the forest which is defined as a subgraph
containing only vertices of the main graph of the graph, adding the least cost
edges later until the minimum spanning tree is created without forming
cycles in the graph.
• Kruskal’s algorithm has easier implementation than prim’s algorithm, but has
higher complexity.
Kruskal’s Algorithm
• The inputs taken by the kruskal’s algorithm are the graph G {V, E},
where V is the set of vertices and E is the set of edges, and the source
vertex S and the minimum spanning tree of graph G is obtained as an
output.
Algorithm
• Sort all the edges in the graph in an ascending order and store it in an
array edge[].
Edge
Cost
Steps of Algorithms
• Construct the forest of the graph on a plane with all the vertices in it.
• Select the least cost edge from the edge[] array and add it into the
forest of the graph. Mark the vertices visited by adding them into the
visited[] array.
• Repeat the steps 2 and 3 until all the vertices are visited without
having any cycles forming in the graph
• When all the vertices are visited, the minimum spanning tree is
formed.
• Calculate the minimum cost of the output spanning tree formed.
Examples
• Construct a minimum spanning tree using kruskal’s algorithm for the
graph given below −
Cont’d…
• Solution
• As the first step, sort all the edges in the given graph in an ascending
order and store the values in an array.

Edge B→D A→B C→F F→E B→C G→F A→G C→D D→E C→G
Cost 5 6 9 10 11 12 15 17 22 25
Cont’d…
Cont’d…
• From the list of sorted edge costs, select the least cost edge and add
it onto the forest in output graph.
• B→D=5
• Minimum cost = 5
• Visited array, v = {B, D}
Cont’d…
• Similarly, the next least cost edge is B → A = 6; so we add it onto the
output graph.
• Minimum cost = 5 + 6 = 11
• Visited array, v = {B, D, A}
Cont’d…
• The next least cost edge is C → F = 9; add it onto the output graph.
• Minimum Cost = 5 + 6 + 9 = 20
• Visited array, v = {B, D, A, C, F}
Cont’d…
• The next edge to be added onto the output graph is F → E = 10.
• Minimum Cost = 5 + 6 + 9 + 10 = 30
• Visited array, v = {B, D, A, C, F, E}
Cont’d…
• The next edge from the least cost array is B → C = 11, hence we add it
in the output graph.
• Minimum cost = 5 + 6 + 9 + 10 + 11 = 41
• Visited array, v = {B, D, A, C, F, E}
Cont’d…
• The last edge from the least cost array to be added in the output
graph is F → G = 12.
• Minimum cost = 5 + 6 + 9 + 10 + 11 + 12 = 53
• Visited array, v = {B, D, A, C, F, E, G}
• The obtained result is the minimum spanning tree of the given graph
with cost = 53.
• The time complexity of this algorithm is O(E log E) or O(E log V),
where E is a number of edges and V is a number of vertices.
Dijkstra’s Shortest Path Algorithm
• Dijkstra’s shortest path algorithm is similar to that of Prim’s algorithm as
they both rely on finding the shortest path locally to achieve the global
solution.
• However, unlike prim’s algorithm, the dijkstra’s algorithm does not find
the minimum spanning tree; it is designed to find the shortest path in
the graph from one vertex to other remaining vertices in the graph.
• Dijkstra’s algorithm can be performed on both directed and undirected
graphs.
• Since the shortest path can be calculated from single source vertex to all
the other vertices in the graph, Dijkstra’s algorithm is also called single-
source shortest path algorithm.
• The output obtained is called shortest path spanning tree.
 Dijkstra’s Algorithm

• The dijkstra’s algorithm is designed to find the shortest path between two
vertices of a graph.
• These two vertices could either be adjacent or the farthest points in the
graph.
• The algorithm starts from the source.
• The inputs taken by the algorithm are the graph G {V, E}, where V is the set
of vertices and E is the set of edges, and the source vertex S.
• And the output is the shortest path spanning tree.
Algorithm
• Declare two arrays − distance[] to store the distances from the source vertex
to the other vertices in graph and visited[] to store the visited vertices.
• Set distance[S] to ‘0’ and distance[v] = ∞, where v represents all the other
vertices in the graph.
• Add S to the visited[] array and find the adjacent vertices of S with the
minimum distance.
• The adjacent vertex to S, say A, has the minimum distance and is not in the
visited array yet. A is picked and added to the visited array and the distance
of A is changed from ∞ to the assigned distance of A, say d1, where d1 < ∞.
• Repeat the process for the adjacent vertices of the visited vertices until the
shortest path spanning tree is formed.
Examples
• To understand the dijkstra’s concept better, let us analyze the
algorithm with the help of an example graph −
Cont’d…
• Step 1
• Initialize the distances of all the vertices as ∞, except the source node
S.
Vertex S A B C D E
Distance 0 ∞ ∞ ∞ ∞ ∞

• Now that the source vertex S is visited, add it into the visited array.
• visited = {S}
Cont’d…
• Step 2
• The vertex S has three adjacent vertices with various distances and
the vertex with minimum distance among them all is A.
• Hence, A is visited and the dist[A] is changed from ∞ to 6.
• S→A=6
Vertex S A B C D E
• S→D=8
Distance 0 6 ∞ ∞ 8 7
• S→E=7
• Visited = {S, A}
Cont’d…
Cont’d…
• Step 3
• There are two vertices visited in the visited array, therefore, the
adjacent vertices must be checked for both the visited vertices.
• Vertex S has two more adjacent vertices to be visited yet: D and E.
Vertex A has one adjacent vertex B.
• Calculate the distances from S to D, E, B and select the minimum
distance −
• S → D = 8 and S → E = 7.
• S → B = S → A + A → B = 6 + 9 = 15
• Visited = {S, A, E}
Cont’d…

Vertex S A B C D E
Distance 0 6 15 ∞ 8 7
Cont’d…
• Step 4
• Calculate the distances of the adjacent vertices – S, A, E – of all the
visited arrays and select the vertex with minimum distance.
• S→D=8
• S → B = 15
• S → C = S → E + E → C = 7 + 5 = 12
• Visited = {S, A, E, D}
Vertex S A B C D E
Distance 0 6 15 12 8 7
Cont’d…
Cont’d…
• Step 5
• Recalculate the distances of unvisited vertices and if the distances
minimum than existing distance is found, replace the value in the
distance array.
• S → C = S → E + E → C = 7 + 5 = 12
• S → C = S → D + D → C = 8 + 3 = 11
• dist[C] = minimum (12, 11) = 11
• S → B = S → A + A → B = 6 + 9 = 15
• S → B = S → D + D → C + C → B = 8 + 3 + 12 = 23
Cont’d…
• dist[B] = minimum (15,23) = 15

Vertex S A B C D E

Distanc 0 6 15 11 8 7
e

• Visited = { S, A, E, D, C}
Cont’d…
Cont’d…
• Step 6
• The remaining unvisited vertex in the graph is B with the minimum
distance 15, is added to the output spanning tree.
• Visited = {S, A, E, D, C, B}
• It has a time complexity of O ( V 2 ) O(V^2) O(V2) using the adjacency
matrix representation of graph.
Job scheduling
• Job scheduling algorithm is applied to schedule the jobs on a single
processor to maximize the profits.
• The greedy approach of the job scheduling algorithm states that,
“Given ‘n’ number of jobs with a starting time and ending time, they
need to be scheduled in such a way that maximum profit is received
within the maximum deadline”.
Job Scheduling Algorithm
• Set of jobs with deadlines and profits are taken as an input with the job
scheduling algorithm and scheduled subset of jobs with maximum
profit are obtained as the final output.
• Algorithm
• Find the maximum deadline value from the input set of jobs.
• Once, the deadline is decided, arrange the jobs in descending order of
their profits.
• Selects the jobs with highest profits, their time periods not exceeding
the maximum deadline.
• The selected set of jobs are the output.
Cont’d…
• Examples
• Consider the following tasks with their deadlines and profits.
Schedule the tasks in such a way that they produce maximum profit
after being executed −
S. No. 1 2 3 4 5

Jobs J1 J2 J3 J4 J5

Deadlines 2 2 1 3 4

Profits 20 60 40 100 80
Cont’d…
• Step 1
• Find the maximum deadline value, dm, from the deadlines given.
• dm = 4.
• Step 2
• Arrange the jobs in descending order of their profits.
S. No. 1 2 3 4 5
Jobs J4 J5 J2 J3 J1
Deadlines 3 4 2 1 2
Profits 100 80 60 40 20
Cont’d…
• The maximum deadline, dm, is 4. Therefore, all the tasks must end
before 4.
• Choose the job with highest profit, J4. It takes up 3 parts of the
maximum deadline.
• Therefore, the next job must have the time period 1.
• Total Profit = 100.
• Step 3
• The next job with highest profit is J5. But the time taken by J5 is 4, which exceeds
the deadline by 3. Therefore, it cannot be added to the output set.
• Step 4
• The next job with highest profit is J2. The time taken by J2 is 2, which also
exceeds the deadline by 1. Therefore, it cannot be added to the output set.
• Step 5
• The next job with higher profit is J3. The time taken by J3 is 1, which does not
exceed the given deadline. Therefore, J3 is added to the output set.
• Total Profit: 100 + 40 = 140
• Step 6
• Since, the maximum deadline is met, the algorithm comes to an end. The output
set of jobs scheduled within the deadline are {J4, J3} with the maximum profit of
140.
Q &A