ANALYSIS AND DESIGN OF

ALGORITHMS

UNIT-IV
CHAPTER 9:
GREEDY TECHNIQUE

1
 OUTLINE
 Greedy Technique
 Prim’s Algorithm
 Kruskal’s Algorithm
 Dijkstra’s Algorithm

2
 Minimum Spanning Tree
Definition: A spanning tree of a connected graph is its connected
acyclic subgraph (i.e., a tree) that contains all the vertices of the
graph. A minimum spanning tree of a weighted connected graph is
its spanning tree of smallest weight, where the weight of a tree is
defined as the sum of the weights on all its edges. The minimum
spanning tree problem is the problem of finding a minimum spanning tree
for a given weighted connected graph.

Figure: Graph and its spanning trees: T1 is the minimum spanning tree.3
 Try to use exhaustive search ?

 If we were to try an exhaustive-search approach to

constructing a minimum spanning tree, we would face
two serious obstacles.

1) the number of spanning trees grows exponentially

with the graph size (at least for dense graphs).

2) generating all spanning trees for a given graph is not

easy;

4
 GREEDY APPROACH
 The greedy approach suggests constructing a solution
through a sequence of steps, each expanding a partially
constructed solution obtained so far, until a complete
solution to the problem is reached. On each step—and
this is the central point of this technique—the choice
made must be

 feasible, i.e., it has to satisfy the problem’s constraints.

 locally optimal, i.e., it has to be the best local choice

among all feasible choices available on that step.

 irrevocable, i.e., once made, it cannot be changed on

subsequent steps of the algorithm.
5
 PRIM’S ALGORITHM
 Prim’s algorithm constructs a minimum spanning tree through a
sequence of expanding subtrees.

 The initial subtree in such a sequence consists of a single vertex

selected arbitrarily from the set V of the graph’s vertices.

 On each iteration, we expand the current tree in the greedy manner

by simply attaching to it the nearest vertex not in that tree.

 The algorithm stops after all the graph’s vertices have been included
in the tree being constructed.

 Since the algorithm expands a tree by exactly one vertex on each of

its iterations, the total number of such iterations is n-1, where n is the
number of vertices in the graph.

6
PRIM’S ALGORITHM

7
 PRIM’S ALGORITHM
 The nature of Prim’s algorithm makes it necessary to provide each
vertex not in the current tree with the information about the shortest
edge connecting the vertex to a tree vertex.

 We can provide such information by attaching two labels to a vertex:

(1) The name of the nearest tree vertex and the length (the weight) of
the corresponding edge.

(2) Vertices that are not adjacent to any of the tree vertices can be
given the ∞ label indicating their “infinite” distance to the tree
vertices and a null label for the name of the nearest tree vertex.

 With such labels, finding the next vertex to be added to the current
tree T={VT, ET} becomes a simple task of finding a vertex with the
smallest distance label in the set V - VT

8
 PRIM’S ALGORITHM
 After we have identified a vertex u* to be added to the tree,
we need to perform two operations:

 Move u* from the set V - VT to the set of tree vertices VT.

 For each remaining vertex u in V - VT that is connected to

u* by a shorter edge than the u’s current distance label,
update its labels by u* and the weight of the edge
between u* and u, respectively.

9
 PRIM‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

10
 PRIM‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

11
 PRIM‘S ALGORITHM
5
A B
4 6 2

2 D 3
C

3 1 2

E F
4

12
 PRIM‘S ALGORITHM

A B
2

2 D 3
C

3 1 2
E F
4

13
 PRIM‘S ALGORITHM

A B
2

2 D 3
C

3 1 2
E F

14
 PRIM‘S ALGORITHM

A B
2

2 D 3
C

3 1 2
E F

15
 PRIM‘S ALGORITHM

A B
2

2 D
C

3 1 2
E F

16
 PRIM‘S ALGORITHM

A B
2

2 D
C

3 1 2
E F

17
 PRIM‘S ALGORITHM
minimum- spanning tree

A B
2

2 D
C

3 1 2
E F

18
 Figure: Application of Prim’s Algorithm. The parenthesized labels of a vertex in the
middle column indicate the nearest tree vertex and edge weight; selected vertices
and edges are shown in bold.

b 1 c
3 4 4 6
a 5 5
f d
6 2 8
e
Tree vertices Remaining vertices Illustration

3 b 1 c
a(-, -) b(a, 3) c(-, ∞) d(-, ∞) e(a, 6)
6
4 4
f(a, 5) a f d
5 5
2
6 8
e
3
b 1
c
6
b(a, 3) c(b, 1) d(-, ∞) e(a, 6) f(b, 4) 4 4
a f 5 d
5
2
6 8 19

e
 Figure: Application of Prim’s Algorithm. The parenthesized labels of a vertex in the
middle column indicate the nearest tree vertex and edge weight; selected vertices
and edges are shown in bold.
Tree vertices Remaining vertices Illustration

3
b 1 c
c(b, 1) d(c, 6) e(a, 6) f(b, 4) 6
4 4
a 5 f 5 d
6 2 8
e
3 b 1
c
6
f(b, 4) d(f, 5) e(f, 2) 4 4
a f 5 d
5
2 8
6
e
3
b 1
c
6
e(f, 2) d(f, 5) 4 4
a f 5 d
5
2 8 20
6
d(f, 5) e
 PRIM’S ALGORITHM ANALYSIS

 Efficiency of Prim’s algorithm depends on the data

structures chosen for the graph and for the priority queue
of the set V-VT whose vertex priorities are the distances to
the nearest tree vertices.

 For example, if a graph is represented by its weight

matrix and the priority queue is implemented as an
unordered array, the algorithm’s running time will be in
θ(|V |2).

21
 PRIM’S ALGORITHM
 A min-heap is a complete binary tree in which every element is less
than or equal to its children.

 Deletion of the smallest element from and insertion of a new element

into a min-heap of size n are O(log n) operations, and so is the
operation of changing an element’s priority.

 If a graph is represented by its adjacency linked lists and the priority

queue is implemented as a min-heap, the running time of the
algorithm is in O(|E| log |V |).

 This is because the algorithm performs |V| - 1 deletions of the

smallest element and makes |E| verifications and, possibly, changes
of an element’s priority in a min-heap of size not greater than |V |.
Each of these operations, as noted earlier, is a O(log |V |) operation.
Hence, the running time of this implementation of Prim’s algorithm
is in
(|V| - 1+ |E|)O(log |V |) = O(|E| log |V |)

22
 KRUSKAL’S ALGORITHM
 This is another greedy algorithm for the minimum
spanning tree problem that also always yields an
optimal solution.

 It is named Kruskal’s algorithm, after Joseph Kruskal,

who discovered the algorithm.

 Kruskal’s algorithm looks at a minimum spanning tree

for a weighted connected graph G={V, E} as an acyclic
subgraph with |V| - 1 edges for which the sum of the
edge weights is the smallest.

23
 KRUSKAL’S ALGORITHM
 The algorithm constructs a minimum spanning tree as an
expanding sequence of subgraphs, which are always
acyclic but are not necessarily connected on the
intermediate stages of the algorithm.

 The algorithm begins by sorting the graph’s edges in

nondecreasing order of their weights.

 Then, starting with the empty subgraph, it scans this

sorted list adding the next edge on the list to the current
subgraph if such an inclusion does not create a cycle and
simply skipping the edge otherwise.

24
KRUSKAL’S ALGORITHM

25
KRUSKAL’S ALGORITHM ANALYSIS
 On each of its iterations, Kruskal’s algorithm has to check whether the
addition of the next edge to the edges already selected would create a
cycle.

 A new cycle is created if and only if the new edge connects two
vertices already connected by a path, i.e., if and only if the two
vertices belong to the same connected component as shown in below
figure.

Figure: New edge connecting two vertices may (a) or may not (b) create a cycle

 Each connected component of a subgraph generated by Kruskal’s

algorithm is a tree because it has no cycles. 26
KRUSKAL‘S ALGORITHM
5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

27
 KRUSKAL‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

28
 KRUSKAL‘S ALGORITHM

5
A B

4 6 2

2 D 3
C

3 1 2
E F
4

29
KRUSKAL‘S ALGORITHM

5
A B
4 6 2

2
C D 3

3 1 2
E F
4

30
 KRUSKAL‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

31
 KRUSKAL‘S ALGORITHM
5
A B
4 6 2

2 D 3 cycle!!
C

3 1 2
E F
4

32
 KRUSKAL‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

33
 KRUSKAL‘S ALGORITHM

5
A B
4 6 2

2 D 3
C

3 1 2
E F
4

34
 KRUSKAL‘S ALGORITHM
minimum- spanning tree
A B
2

2 D
C

3 1 2
E F

35
 Figure: Application of Kruskal’s Algorithm. Selected edges are shown in bold.

b 1 c
3 4 4 6
a 5 5
f d
6 2 8
e
Tree vertices Sorted list of edges Illustration

3 b 1 c
bc ef ab bf cf af df ae cd de
6
4 4
1 2 3 4 4 5 5 6 6 8 a f d
5 5
2
6 8
e
b 1
c
3 6
bc bc ef ab bf cf af df ae cd de 4 4
1 1 2 3 4 4 5 5 6 6 8 a f 5 d
5
2
6 8 36

e
 Figure: Application of Kruskal’s Algorithm. Selected edges are shown in bold.

Tree vertices Sorted list of edges Illustration

3
b 1 c
6
ef bc ef ab bf cf af df ae cd de 4 4
2 1 2 3 4 4 5 5 6 6 8 a 5 f 5 d
2
6 8
e
ab bc ef ab bf cf af df ae cd de 3 b 1
c
6
3 1 2 3 4 4 5 5 6 6 8 4 4
a f 5 d
5
2 8
6
e
bf bc ef ab bf cf af df ae cd 3
b 1
c
6
de 4 4
4 1 2 3 4 4 5 5 6 6 8 a f 5 d
5
df 2
6 8 37
5 e
 KRUSKAL’S ALGORITHM ANALYSIS
 With an efficient Union-find algorithm (check whether two
vertices belong to the same tree) , the running time of
Kruskal’s algorithm will be dominated by the time needed
for sorting the edge weights of a given graph.

 Hence, with an efficient sorting algorithm, the time

efficiency of Kruskal’s algorithm will be in O(|E| log |V |).

38
 DIJKSTRA’S ALGORITHM
 single-source shortest-paths problem: for a given vertex
called the source in a weighted connected graph, find
shortest paths to all its other vertices.

 The single-source shortest-paths problem asks for a family

of paths, each leading from the source to a different vertex
in the graph, though some paths may, of course, have edges
in common.

 The best-known algorithm for the single-source shortest-

paths problem,: Dijkstra’s algorithm.

 However, this algorithm is applicable to graphs with

nonnegative weights only.

 Edsger W. Dijkstra discovered this algorithm in mid-1950s.

39
DIJKSTRA’S ALGORITHM
 Assumptions:
 the graph is connected
 the edges are undirected (or directed)
 the edge weights are nonnegative

40
DIJKSTRA’S ALGORITHM
Idea of Dijkstra’s algorithm

The subtree of the shortest

paths already found is
shown in bold. The next
nearest to the source v0
vertex, u*, is selected by
comparing the lengths of the
subtree’s paths increased by
the distances to vertices
adjacent to the subtree’s
vertices.
41
 DIJKSTRA’S ALGORITHM
 First, it finds the shortest path from the source to a vertex
nearest to it,

 then to a second nearest, and so on.

 Before its ith iteration commences, the algorithm has

already identified the shortest paths to i - 1 other vertices
nearest to the source.

 These vertices, the source, and the edges of the shortest

paths leading to them from the source form a subtree Ti of
the given graph.

 The set of vertices adjacent to the vertices in Ti can be

referred to as “fringe vertices”; they are the candidates
from which Dijkstra’s algorithm selects the next vertex
nearest to the source. 42
 DIJKSTRA’S ALGORITHM
 To identify the ith nearest vertex, the algorithm computes,
for every fringe vertex u, the sum of the distance to the
nearest tree vertex v (given by the weight of the edge (v,
u)) and the length dv of the shortest path from the source
to v.

 Then selects the vertex with the smallest such sum.

43
 DIJKSTRA’S ALGORITHM
 To facilitate the algorithm’s operations, we label each
vertex with two labels:

(1) The numeric label d indicates the length of the shortest

path from the source to this vertex found by the
algorithm so far; when a vertex is added to the tree, d
indicates the length of the shortest path from the source
to that vertex.

(2) The other label indicates the name of the next-to-last

vertex on such a path, i.e., the parent of the vertex in the
tree being constructed.

 With such labeling, finding the next nearest vertex u*

becomes a simple task of finding a fringe vertex with the
smallest d value. 44
 DIJKSTRA’S ALGORITHM
 After we have identified a vertex u* to be added to the tree,
we need to perform two operations:

 Move u* from the fringe to the set of tree vertices.

 For each remaining fringe vertex u that is connected to u*

by an edge of weight w(u*, u) such that du* + w(u*, u) < du,
update the labels of u by u* and du* + w(u*, u),
respectively.
Note that in the following pseudocode, VT contains a given
source vertex and the fringe contains the vertices adjacent to
it after iteration 0 is completed.
45
 Figure: Application of Dijkstra’s Algorithm. The next closest vertex
is shown in bold.
4
b c
3 6
2 5
a d e
7 4

Tree vertices Remaining vertices Illustration

4
3 b c
a(-, 0) b(a, 3) c(-, ∞) d(a, 7) e(-,
6
2 5
∞) a d e
7 4

4
3
b c
b(a, 3) c(b, 3+4) d(b, 3+2) e(-, ∞) 6
2 5
a d e
7 4
46
 Figure: Application of Dijkstra’s Algorithm. The next closest vertex
is shown in bold.

Tree vertices Remaining vertices Illustration

4
d(b, 5) c(b, 7) e(d, 5+4) 3 b c
6
2 5
a d e
7 4
4
c(b, 7) e(d, 9) 3
b c
6
2 5
e(d, 9) a d e
7 4
The shortest paths and their lengths are:
from a to b: a -b of length 3
from a to d: a–b–d of length 5
from a to c: a–b–c of length 7
from a to e: a–b–d–e of length 9 47
DIJKSTRA’S ALGORITHM

48
 DIJKSTRA’S ALGORITHM ANALYSIS
 The time efficiency of Dijkstra’s algorithm depends on the
data structures used for implementing the priority queue
and for representing an input graph itself.

 It is in θ(|V |2) for graphs represented by their weight

matrix and the priority queue implemented as an
unordered array.

 For graphs represented by their adjacency linked lists and

the priority queue implemented as a min-heap, it is in O(|
E| log |V |).

49
End of Chapter 9

50