Algorithmics

CT065-3.5-3

Shortest Path Algorithms

Level 3 – Computing (Software Engineering)
 Topics & Structure of Lesson

• Greedy Algorithms
• Applications of Greedy Algorithms:
– Minimum Spanning Trees (MST)
• Prim’s Algorithm
• Kruskal’s Algorithm
– Shortest Path Trees
• Dijkstra’s Algorithm

Module Code and Module Title Title of Slides Slide 2 (of 42)
 Learning Outcomes

By the end of this lesson you should be able

to:
• Explain what is a greedy algorithm
• Give examples of applications which use
greedy algorithms
• Apply Prim’s, Kruskal’s and Dijkstra’s
algorithms on given problems

Module Code and Module Title Title of Slides Slide 3 (of 42)
 Key terms

• Greedy algorithm
• Spanning tree
• Minimum spanning tree (MST)
• Prim’s algorithm
• Kruskal’s algorithm
• Dijkstra’s algorithm

Module Code and Module Title Title of Slides Slide 4 (of 42)
 Greedy Algorithms

A greedy algorithm is a method for finding an

optimal solution to some problem involving a
large, homogeneous data structure (such as an
array, a tree, or a graph) by starting from an
optimal solution to some component or small
part of the data structure and extending it, by
considering additional components of the data
structure one by one, to arrive at an optimal
global solution.

Module Code and Module Title Title of Slides Slide 5 (of 42)
 Greedy Algorithms

• Key idea: Makes the choice that looks best at the

moment with the hope that a locally optimal choice
will lead to a globally optimal solution.

• Real world examples:

– Getting your money’s worth out of a theme park’s
rides/carnival/buffet meal
– Tour planning
– Room scheduling

Module Code and Module Title Title of Slides Slide 6 (of 42)
 Applications of Greedy Algorithms

• Graph Algorithms
– Minimum Spanning Trees
– Shortest Path Trees

• Data compression
– Huffman Encoding

• Combinatorial problems
– Traveling salesman problem
– Knapsack problem

Module Code and Module Title Title of Slides Slide 7 (of 42)
 Spanning Trees
• Spanning Trees: A subgraph T of an undirected graph
G=(V,E) is a spanning tree of G if it is a tree and contains every
vertex of G.

E.g:

Module Code and Module Title Title of Slides Slide 8 (of 42)
 Weighted Graphs

• Weighted Graphs: A weighted graph is a graph in

which each edge has a weight.

• Weight of a Graph: The sum of the weights of all

edges.

Module Code and Module Title Title of Slides Slide 9 (of 42)
 Weighted Graphs
• Example: Spanning trees for a weighted graph

Module Code and Module Title Title of Slides Slide 10 (of 42)
 Minimum Spanning Trees (MST)

• Minimum Spanning Trees (MST) : A MST of

an undirected connected weighted graph is a
spanning tree with minimum weight (among all
spanning trees).

Module Code and Module Title Title of Slides Slide 11 (of 42)
 Minimum Spanning Trees
• Example: Minimum spanning tree for a weighted graph

Module Code and Module Title Title of Slides Slide 12 (of 42)
 Minimum Spanning Trees (MST)

Note:
The MST may not be unique. However, if the weights of
all the edges are pair-wise distinct then it is indeed
unique.

Module Code and Module Title Title of Slides Slide 13 (of 42)
 Minimum Spanning Tree Problem

• The MST for a connected undirected weighted graph

can be found using the following algorithms:
– Prim’s algorithm
– Kruskal’s algorithm

• General approach to problem: start with an empty

graph and try to add edges one at a time, ensuring
that the graph (tree) that is built is acyclic. And if we
are sure that the every time the resulting graph is
always a subset of some MST, we are done.
Module Code and Module Title Title of Slides Slide 14 (of 42)
 Prim’s Algorithm

• The algorithm was discovered by Vojtech Jarnik and

later rediscovered by Robert Prim.
• In this method:
1. All edges in the graph are initially ordered.
2. Find the shortest edge leaving the (partial) tree and add it
to the tree. The chosen edge should not create cycles
within the tree.
3. The steps are repeated until all vertices are added to the
MST.

Module Code and Module Title Title of Slides Slide 15 (of 42)
 Prim’s Algorithm

tree = null
E = sequence of all edges of graph sorted by weight;
for i = 1 to |V| // V = no. of vertices in graph
for j = 1 to |E|
if ej from edges does not form a cycle with
edges in tree and is incident to a
vertex in tree
add ej to tree;
break;

Module Code and Module Title Title of Slides Slide 16 (of 42)
 Prim’s Algorithm - Example

Let us construct a MST for the graph below, using

Prim’s algorithm.
6
a b

5 9
13
16
c d
15
7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 17 (of 42)
 Prim’s Algorithm - Example

Step 1 f

3
g

Step 2 f

3
g

Module Code and Module Title Title of Slides Slide 18 (of 42)
 Prim’s Algorithm - Example

Step 3 d

7
e f

8
3
g

Module Code and Module Title Title of Slides Slide 19 (of 42)
 Prim’s Algorithm - Example

Step 4
c d

7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 20 (of 42)
 Prim’s Algorithm - Example

Step 5 a

c d

7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 21 (of 42)
 Prim’s Algorithm - Example

Step 6 a
6
b

c d

7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 22 (of 42)
 Prim’s Algorithm

The running time of Prim’s algorithm:

- the inner loop of the algorithm can be O(E) in the
worst case, and since the outer loop iterates |V|-1
times, the inner loop may iterate O(|V||E|) in total.

Module Code and Module Title Title of Slides Slide 23 (of 42)
 Kruskal’s Algorithm

• The algorithm was devised by Joseph Kruskal.

• In this method:
1. All edges are ordered by weight.
2. Each edge in this sequence is checked to see whether it
can be considered part of the tree under construction.
3. The edge is added to the tree if no cycle arises after its
inclusion.
4. The steps are repeated until all vertices are added to the
MST.

Module Code and Module Title Title of Slides Slide 24 (of 42)
 Kruskal’s Algorithm
Let T be the set of selected edges // for the MST to be built
Initialize T to the empty set
Let E be the set of graph edges
while ( E ! = the empty set ) && (T ! = V-1) //graph has V vertices
{ Let (u, v) be a least cost edge in E
E = E – {u, v} //delete the edge from E
if (u, v) does not create a cycle in T)
add edge (u, v) to T }
If (T == n – 1)
T is a minimum cost spanning tree
Else
The network is not connected and has no spanning tree

Module Code and Module Title Title of Slides Slide 25 (of 42)
 Kruskal’s Algorithm - Example

Let us construct a MST for the graph below, using

Kruskal’s algorithm.
6
a b

5 9
13
16
c d
15
7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 26 (of 42)
 Kruskal’s Algorithm - Example

f
Step 1
3
g

a
Step 2
5

c f

3
g
Module Code and Module Title Title of Slides Slide 27 (of 42)
 Kruskal’s Algorithm - Example

Step 3 a
6
b

3
g

Module Code and Module Title Title of Slides Slide 28 (of 42)
 Kruskal’s Algorithm - Example

Step 4 a
6
b

c d

7
f

3
g

Module Code and Module Title Title of Slides Slide 29 (of 42)
 Kruskal’s Algorithm - Example

Step 5 a
6
b

c d

7
e f

8
3
g

Module Code and Module Title Title of Slides Slide 30 (of 42)
 Kruskal’s Algorithm - Example

Step 6 a
6
b

c d

7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 31 (of 42)
 Kruskal’s Algorithm

The running time of Kruskal’s algorithm:

- The complexity of the algorithm is determined by
the complexity of the sorting method applied and
the method used for cycle detection.

- The complexity for the algorithm is O(|E|lg|V|)

Module Code and Module Title Title of Slides Slide 32 (of 42)
 Shortest Path Problem

• The weighted path length of a path is the sum of

the edge costs (weight) on the path.

• In the positive weight shortest path problem, the

edges have non-negative weights and we want to
find the shortest weighted path from some start
vertex to all other vertices.

Module Code and Module Title Title of Slides Slide 33 (of 42)
 Dijkstra’s Algorithm

• The algorithm was devised by Edsger Wybe Dijkstra.

• In this method:
1. A record is maintained for vertices in increasing order of
their distance from the source vertex.
2. Construct the shortest path tree edge by edge – at each
step, add one new edge, corresponding to the construction
of the shortest path to the new vertex.

Module Code and Module Title Title of Slides Slide 34 (of 42)
 Dijkstra’s Algorithm

DijkstraAlgorithm (weighted graph, vertex first)

for all vertices v
currDist(v) = 
currDist(first) = 0
toBeChecked = all vertices
While toBeChecked is not empty {
v = a vertex in toBeChecked with minimal currDist(v)
remove v from toBeChecked
for all vertices u adjacent to v and in toBeChecked
if currDist(u) > currDist(v) + weight(edge(vu)) {
currDist(u) = currDist(v) + weight(edge(vu))
predecessor(u) = v }
}

Module Code and Module Title Title of Slides Slide 35 (of 42)
 Dijkstra’s Algorithm - Example

Let us construct a shortest path tree from vertex a (rooted at a)

for the following graph, using Dijkstra’s algorithm.
6
a b

5 9
13
16
c d
15
7
e 12 f

8
3
g

Module Code and Module Title Title of Slides Slide 36 (of 42)
 Dijkstra’s Algorithm - Example
Trace of algorithm:
Iteration initial 1 2 3 4 5 6 7
Active vertex a c b f e g
d
currDist (a) 0
currDist (b)  6 6
currDist (c)  5
currDist (d)   21 21 21 21
currDist (e)    19 19
currDist (f )   17 17
currDist (g)     20 20

Module Code and Module Title Title of Slides Slide 37 (of 42)
 Dijkstra’s Algorithm - Example

Trace of algorithm (cont’d):

Predecessor list:
predecessor(a) = nil
predecessor(b) = a
predecessor(c) = a
predecessor (d) = c
predecessor (e) = b
predecessor (f) = c
predecessor (g) = f

Module Code and Module Title Title of Slides Slide 38 (of 42)
 Dijkstra’s Algorithm - Example

Based on the predecessor list, we can now construct

the shortest path tree starting from vertex A.
6
a b

5
13
16
c d
15
7
e 12 f

8
3
g
Module Code and Module Title Title of Slides Slide 39 (of 42)
 Dijkstra’s Algorithm

The running time of Dijkstra’s algorithm:

- The complexity of the algorithm is O(|V|2)

- The complexity for the algorithm can be improved

to O((|E|+ |V|) lg |V|) by using a heap to store and
order vertices and adjacency lists .

Module Code and Module Title Title of Slides Slide 40 (of 42)
 Summary

•Greedy Algorithms
•Applications of Greedy Algorithms:
–Minimum Spanning Trees (MST)
•Prim’s Algorithm
•Kruskal’s Algorithm

–Shortest Path Trees

•Dijkstra’s Algorithm

Module Code and Module Title Title of Slides Slide 41 (of 42)
 Next Session

• Information Coding Techniques

– Huffman encoding
– Arithmetic coding
– RSA Encrytion

Module Code and Module Title Title of Slides Slide 42 (of 42)