Design and Analysis of Algorithms

CoSc 3094

Unit-3
Greedy Algorithms

Computer Science Department

Oda Bultum University, Chiro, Ethiopia
[email protected]
915980493
 Outline
Looping
 General Characteristics of greedy algorithms
 Minimum Spanning trees (MST)
 Kruskal’s algorithm
 Prim’s algorithm
 Shortest Paths
 Scheduling(reading assignment)
Characteristics of Greedy Algorithms
 Greedy algorithms are characterized by the following features.
1. Greedy approach forms a set or list of candidates 𝑪.
2. Once a candidate is selected in the solution, it is there forever: once a candidate is excluded from the
solution, it is never reconsidered.
3. To construct the solution in an optimal way, Greedy Algorithm maintains two sets.
4. One set contains candidates that have already been considered and chosen, while the other set contains
candidates that have been considered but rejected.

 The greedy algorithm consists of four functions.

i. Solution Function:- A function that checks whether chosen set of items provides a solution.
ii. Feasible Function:- A function that checks the feasibility of a set.
iii. Selection Function:- The selection function tells which of the candidates is the most promising.
iv. Objective Function:- An objective function, which does not appear explicitly, but gives the value of a
solution.

Hadi Hussen  Unit 3 – Greedy Algorithms 4

Introduction to Minimum Spanning Tree (MST)
 Let 𝐺 = 𝑁, 𝐴 be a connected, undirected graph where,
1. N is the set of nodes and
2. A is the set of edges.
 Each edge has a given positive length or weight.
 A spanning tree of a graph 𝐺 is a sub-graph which is basically a tree and it contains all the
vertices of 𝐺 but does not contain cycle.
 A minimum spanning tree (MST) of a weighted connected graph 𝐺 is a spanning tree with
minimum or smallest weight of edges.
 Two Algorithms for constructing minimum spanning tree are,
1. Kruskal’s Algorithm
2. Prim’s Algorithm

Hadi Hussen  Unit 3 – Greedy Algorithms 6

Spanning Tree Examples
Graph A Spanning Tree A
B C B C

D E F G D E F G

H H

Graph Spanning Tree

A
A
B C E
B C E
D F
D F

Hadi Hussen  Unit 3 – Greedy Algorithms 7

Kruskal’s Algorithm for MST – Example 1
A Step 2: Taking next Step 4: Taking next
4 5
6 min edge (B,C) min edge (A,B)
6
B 3
E B A
4
5 7 2
2
C D B E
C D 1 3
1
2
Step 3: Taking next

Step 1: Taking min

min edge (B,E) C D
edge (C,D) B E 1
3
So, we obtained a
C D 2
1 C D minimum
1 spanning tree of cost:
4 + 2 + 1 + 3 = 10
Hadi Hussen  Unit 3 – Greedy Algorithms 8
 Step:1
Kruskal’s Algorithm for MST – Example 2
Sort the edges in increasing order of their weight.

Edges Weight

1 2 {1, 2} 1
1 2 3 {2, 3} 2
{4, 5} 3
4 6 4 5 6
{6, 7} 3
{1, 4} 4
4 3 5 8 6 {2, 5} 4
{4, 7} 4
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Hadi Hussen  Unit 3– Greedy Algorithms 9
 Step:2
Kruskal’s Algorithm for MST – Example 2
Select the minimum weight edge but no cycle.

Edges Weight

1 2 {1, 2} 1 
1 2 3 {2, 3} 2 
{4, 5} 3 
4 6 4 5 6
{6, 7} 3 
{1, 4} 4 
4 3 5 8 6 {2, 5} 4
{4, 7} 4 
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Hadi Hussen  Unit 3– Greedy Algorithms 10
 Step:3
Kruskal’s Algorithm for MST – Example 2
The minimum spanning tree for the given graph.

Edges Weight

1 2 {1, 2} 1 
1 2 3 {2, 3} 2 
{4, 5} 3 
4
{6, 7} 3 
{1, 4} 4 
4 3 5 6 {4, 7} 4 

4 3 Total Cost = 17

Hadi Hussen  Unit 3– Greedy Algorithms 11

Kruskal’s Algorithm – Example 2
Step Edges considered - Connected Components Edges Weight
{u, v}
Init. - {1} {2} {3} {4} {5} {6} {7} {1, 2} 1

1 {1,2} {1,2} (3} {4} {5} {6} {7} {2, 3} 2

2 {2,3} {1,2,3} {4} {5} {6} {7} {4, 5} 3
3 {4,5} {1,2,3} {4,5} {6} {7} {6, 7} 3
4 {6,7} {1,2,3} {4,5} {6,7} {1, 4} 4
5 {1,4} {1,2,3,4,5} {6,7} {4, 7} 4
6 {2,5} Rejected Total Cost = 17
7 {4,7} {1,2,3,4,5,6,7}

Hadi Hussen  Unit 3 – Greedy Algorithms 12

Exercises – Home Work
 The complexity for the Kruskal’s algorithm is in 𝜽(𝒂 𝒍𝒐𝒈 𝒏) where 𝒂 is total number of edges
and 𝒏 is the total number of nodes in the graph 𝐺.

 Write the kruskal’s Algorithm to find out Minimum Spanning Tree. Apply the same and find MST
for the graph given below.
𝟏. 𝟐.
10
3
3 A 1
F C
A 4 3 7
B 2 4
C 5
8 4
5 6
4 B D D E F G
4
H 3 2 8
2 1 9
3
G E
3 H

Hadi Hussen  Unit 3 – Greedy Algorithms 14

Prim’s Algorithm
 In Prim's algorithm, the minimum spanning tree grows in a natural way, starting from an
arbitrary root.
 At each stage we add a new branch to the tree already constructed; the algorithm stops when
all the nodes have been reached.
 The complexity for the Prim’s algorithm is 𝜽(𝒏𝟐 ) where 𝑛 is the total number of nodes in the
graph 𝐺.

Hadi Hussen  Unit 3 – Greedy Algorithms 15

Prim’s Algorithm for MST – Example 1 Step:1 Select an arbitrary node.

Node - Set B Edges

1 2
1 2 3 1

4 6 4 5 6

4 3 5 8 6
7
4 3

Hadi Hussen  Unit 3– Greedy Algorithms 16

Prim’s Algorithm for MST – Example 1 Step:2 Find an edge with minimum weight.

Node - Set B Edges

1 2
1 2 3 1 {1, 2}, {1, 4}

4 6 1, 2 {1, 4}, {2, 3} {2, 4}, {2, 5}

6 4 5
1, 2, 3 {1,4}, {2,4}, {2,5}, {3,5},
4 3 5 8 6 {3,6}
1, 2, 3, 4 {2,4} {2,5} {3,5} {3,6} {4,5}
7 {4,7}
4 3 1, 2, 3, 4, 5 {2,4} {2,5} {3,5} {3,6} {4,7}
{5,6} {5,7}
7 1, 2, 3, 4, 5, 7 {2,4} {2,5} {3,5} {3,6} {5,6}
{5,7} {6,7}
1, 2, 3, 4, 5, 6, 7
Hadi Hussen  Unit 3– Greedy Algorithms 17
Prim’s Algorithm for MST – Example 1 Step:3

The minimum spanning tree for the given graph.

Node Edges
1 2
1 2 3 1
1, 2 {1, 2}
4
1, 2, 3 {2, 3}
1, 2, 3, 4 {1, 4}
4 3 5 6 1, 2, 3, 4, 5 {4, 5}
1, 2, 3, 4, 5, 7 {4, 7}
4 3 1, 2, 3, 4, 5, 6, 7 {6, 7}

Total Cost = 17
7

Hadi Hussen  Unit 3– Greedy Algorithms 18

 Cost = 17
Prim’s Algorithm – Example 1

Step Edge Set B Edges Considered

Selected
{u, v}
Init. - {1} --
1 {1, 2} {1,2} {1,2} {1,4}
2 {2, 3} {1,2,3} {1,4} {2,3} {2,4} {2,5}
3 {1, 4} {1,2,3,4} {1,4} {2,4} {2,5} {3,5} {3,6}
4 {4, 5} {1,2,3,4,5} {2,4} {2,5} {3,5} {3,6} {4,5} {4,7}
5 {4, 7} {1,2,3,4,5,7} {2,4} {2,5} {3,5} {3,6} {4,7} {5,6} {5,7}
6 {6,7} {1,2,3,4,5,6,7} {2,4} {2,5} {3,5} {3,6} {5,6} {5,7} {6,7}
Exercises – Home Work
 Write the Prim’s Algorithm to find out Minimum Spanning Tree. Apply the same and find MST
for the graph given below.

𝟏. 𝟐.
10
3
3 A 1
F C
A 4 3 7
B 2 4
C 5
8 4
5 6
4 B D D E F G
4
H 3 2 8
2 1 9
3
G E
3 H

Hadi Hussen  Unit 3 – Greedy Algorithms 21

Introduction
 Consider now a directed graph 𝐺 = (𝑁, 𝐴) where 𝑁 is the set of nodes and 𝐴 is the set of
directed edges of graph 𝐺.
 Each edge has a positive length.
 One of the nodes is designated as the source node.
 The problem is to determine the length of the shortest path from the source to each of the
other nodes of the graph.
 Dijkstra’s Algorithm is for finding the shortest paths between the nodes in a graph.
 For a given source node, the algorithm finds the shortest path between the source node and
every other node.
 The algorithm maintains a matrix 𝑳 which gives the length of each directed edge:

𝐿[𝑖, 𝑗] ≥ 0 if the edge (𝑖, 𝑗) ∈ 𝐴, and

𝐿[𝑖, 𝑗] = ∞ otherwise.

Hadi Hussen  Unit 3 – Greedy Algorithms 23

Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10

Compare cost of 1–5–4

Is there path from 1 - 5 - 432 No
Yes (20) and 1–4 (100)
Hadi Hussen  Unit 3 – Greedy Algorithms 24
Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10

1–4–2
Compare cost of 1–4–3
Is there
Is there path
path from
from 11 -- 44 -- 532 Yes
No
(40) and
(70) and 1–3
1–2(50)
(30)
Hadi Hussen  Unit 3 – Greedy Algorithms 25
Dijkstra’s Algorithm - Example

1 Single source shortest path algorithm

10 50

5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10
3 3 {2} 35 30 20 10

Compare cost of
Is there path from 1 - 3 - 542 Yes
No
1–3–2 and 1–2
Hadi Hussen  Unit 3 – Greedy Algorithms 26
Exercises – Home Work
 Write Dijkstra’s Algorithm for shortest path. Use the algorithm to find the shortest path from the
following graph.

1. 2.
2
2 A B
B D
10 4 1 3 10

1 4 8 7 9 2 2
A C D E

3 2 5 8 4 6
C E
1
F G

Hadi Hussen  Unit 3 – Greedy Algorithms 27

Assignment(Individual Assignment)
1. Write an algorithm for:
 Prims algorithm
 Kruskal’s algorithm
2. Scheduling(Reading Assignment)

Hadi Hussen  Unit 3 – Greedy Algorithms 53