Graphs : Shortest Path Algorithm

Dijkstra algorithm and the algorithm of its

time complexity
Outline
 Problem Definition and Representation of the Shortest-Paths
Problems
 Single-Source Shortest Path

2
Problem Definition and Motivation
An abstract data type (ADT) is the realization of a data type as a software component.
The interface of the ADT is defined in terms of a type and a set of operations on that type. School

Graph is a rich ADT that provides the ultimate in data structure flexibility. Graphs can model V
both real-world systems and abstract problems, so they are used in hundreds of applications.

Graphs can be used to:

15
1. Modeling connectivity in computer and communications networks.
S

5
2. Representing a map as a set of locations with distances between locations;
used to compute shortest routes between locations.
3. Modeling flow capacities in transportation networks. Home

4
4. Modeling computer algorithms, showing transitions from one program state
to another.
5. Finding an acceptable order for finishing subtasks in a complex activity,
such as constructing large buildings or completing a degree program with U
pre-requisites.
Weighted Directed Acyclic Graph (DAG).

3
Definitions and Representation
A graph G = (V;E) consists of a set of vertices V and a set of edges E, such that each edge in E is a connection
between a pair of vertices in V. A graph with relatively few edges is called sparse,
while a graph with many edges is called dense. A graph containing all possible edges is said to be complete.

Two vertices are adjacent if they are joined by an edge. Such vertices are also called neighbors. An edge connecting
Vertices U and V is written (U, V). Such an edge is said to be incident on Vertices U and V.
Associated with each edge may be a cost or weight.

The number of edges in a complete undirected graph : V

V

15
E = n(n-1)/2 where n is the number of vertices

15
15
The number of edges in a complete DAG: E = n(n-1)
where n is the number of vertices ç è S

5
5
S

4
4
4
U U

Undirected Graph

4
Definitions and Representation

In either case (Directed or undirected graph), the space requirements for the adjacency matrix are θ (|V|2).
The storage requirements for the adjacency list depend on both the number of edges and the number of
vertices in the graph. There must be an array entry for each vertex (even if the vertex is not adjacent to
any other vertex and thus has no elements on its linked list), and each edge must appear on one of the
lists. Thus, the cost is θ (|V| + |E|).
5
 Shortest Path Problem
Given such a graph, a typical problem is to find the total length
of the shortest path between two specified vertices. This is
not a trivial problem, because the shortest path may not be
along the edge (if any) connecting two vertices, but rather may
be along a path involving one or more intermediate vertices.

We use the notation d(A, D) = 10 to indicate that the shortest distance

from A to D is 10.

There is no path from E to B, so we set d(E, B) = ∞.

We define w(A, D) = 20 to be the weight of edge (A, D), that is, the
weight of the direct connection from A to D.
Because there is no edge from E to B, w(E, B) = ∞.

6
Single-Source Shortest Path: Dijkstra's
Algorithm
Dijkstra’s algorithm presents an algorithm to solve the single-source shortest-paths problem.
Given Vertex S in Graph G, find a shortest path from S to every other vertex in G.

The algorithm processes a set of labelled vertices v0 to vn-1, with S = v0.

Assume that we have processed in order of distance from S to the first i - 1 vertices that are closest to
S; call this set of vertices S. We are now about to process the ith closest vertex; call it X. A shortest
path from S to X must have its next-to-last vertex in S. Thus,

In other words, the shortest path from S to X is the minimum over all paths that go from S to U, then have
an edge from U to X, where U is some vertex in S.

How it Works: It works by maintaining a distance estimate D(X) for all vertices X in V. The elements of D
are initialized to the value INFINITE. Vertices are processed in order of distance from S. Whenever a
vertex V is processed, D(X) is updated for every neighbor X of V.

7
Dijkstra’s Pseudocode
Dijkstra(Graph G, Vertex source)
initialize distances to
- Set all nodes is infinity, except for the start which is 0
mark source as distance 0 - Start at the start vertex to be the current vertex
mark all vertices unprocessed - For the current vertex, look at all of the outgoing
while(there are unprocessed vertices){ neighbors/their edges. If the distance to the current node (from
the source)+ that edge weight is smaller than the proposed
let u be the closest unprocessed vertex
estimated distance for that neighbor, relax the neighbor node
foreach(edge (u,v) leaving u){ (update the proposed estimated distance and predecessor
// u is the current vertex
edge).
if(u’s dist+weight(u,v) < v’s dist){ -Update the current vertex to be the vertex with the next
smallest estimated distance that hasn’t been processed
v.dist = u.dist+weight(u,v)
v.predecessor = (u,v)
}
}
mark u as processed
}

8
 Example
Input Output
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)

A 0 A 0
B B A
C C A
D D A
E E
Initialization
Processed = [ ]
UnProcessed = [ A,B,C,D,E]
Select the closest unprocessed: vertex =A

At this step the distance to the source (initialization): A -> B = , A-> D = , A -> C =

Find ALL edges leaving A

A -> B = 10 , A-> D = 20 , A -> C = 3
Since 10 < , 20 < , 3 < then update the original values to the new values. Note
this is the (shortest) distance from the source at this step/iteration
Processed = [A ] ,
UnProcessed = [ B,C,D,E]

9
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)

A 0 A 0
B A B C
C A C A
D A D A
E E C
Processed = [A ]
UnProcessed = [ B,C,D,E]
Select the closest unprocessed: vertex = C
At this step (initialization): C -> B = , C-> E =
Find ALL edges leaving C : C -> B = , C-> E =

Find the shortest distance form the source

A-> C -> B = shortest distance from A -> C computed previously + shortest distance C->B computed at this step
A-> C -> B = 3 +2 = 5  A->B = 5 this is a shorter path from A to B going through C compared to the previously computed one (10)  Update
A->C->E = 3+15 = 18  A->E = 18 this is a shorter path from A to E going through C compared to the previously computed one ()  Update
Processed = [A,C ] ,
UnProcessed = [ B,D,E]

10
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)

A 0 A 0
B C B C
C A C A
D A D B
E C E C
Processed = [A,C ]
UnProcessed = [ B,D,E]
Select the closest unprocessed: vertex = B
At this step (initialization): B -> D =
Find ALL edges leaving B : B -> D =
Find the shortest distance form the source
A-> B -> D = shortest distance from A -> B computed previously + shortest distance B->D computed at this step
shortest distance from A -> B computed previously = A->C->B = 5
shortest distance B->D computed at this step = B->C = 5
A->C->B->D = 5+5 = 10  A->D this is a shorter path from A to D going through C and B compared to the previously computed one (20)  Update
Processed = [A,C,B ] ,
UnProcessed = [ D,E]

11
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)

A 0 A 0
B C B C
C A C A
D B D B
E C E C
Processed = [A,C,B ]
UnProcessed = [ D,E]
Select the closest unprocessed: vertex = D
At this step (initialization): D -> E =
Find ALL edges leaving D : D -> E =
Find the shortest distance form the source
A-> D -> E = shortest distance from A -> D computed previously + shortest distance D->E computed at this step
shortest distance from A -> D computed previously = A->C->B->D = 10
shortest distance D->E computed at this step = D->E = 11
A->C->B->D->E = 10+11 = 21  A->D this is a longer path from A to E going through C, B, D compared to the previously computed one (18)  NO Update

Processed = [A,C,B,D ]
UnProcessed = [E]
12
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)

A 0 A 0
B C B C
C A C A
D B D B
E C E C
Processed = [A,C,B ]
UnProcessed = [E] What did we learn?
Select the closest unprocessed: vertex = E 1- This is a Directed Acyclic Graph (DAG) with No negative edges
Find ALL edges leaving E : NULL
2- We iterate over every single Vertex and for each Vertex we iterate over
every single outgoing edge (TWO nested loop)
Terminate
3- Some Vertex has few outgoing edge (Sparse Graph)
Processed = [A,C,B,D,E ] 4- The execution time of the Dijkstra’s algorithm will be determined by the
UnProcessed = [ ] number of vertices and the outgoing edges.
5- Some vertices will be processed faster than others.
6- Vertices without outgoing edges will not be processed

13
 Example: Undirected Graph Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A 5
6

B B A D 2 H

C C A 3
G
D D A
5
E E C E

F F
G G
H H
Processed = [ ]
UnProcessed = [ A,B,C,D,E,F,G,H]
Select the closest unprocessed: vertex = A
Find ALL edges leaving A (DO NOT include edges toward the processed vertices) :
A->B = 8, A->D = 5, A->C = 2  if shorter path than previous to the source, then update, otherwise No update
Processed = [ A]
UnProcessed = [ B,C,D,E,F,G,H]

14
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A 5
6

B A B A D 2 H
C A C A 3
G
D A D C
5
E E C C E

F F
G G
H H
Processed = [ A]
UnProcessed = [ B,C,D,E,F,G,H]
Select the closest unprocessed: vertex = C
Find ALL edges leaving C (DO NOT include edges toward the processed vertices) :
C->D = 2, C->E = 5  A->C->D = 4, A->C->E = 7. if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C]
UnProcessed = [ B,C,D,E,F,G,H]

15
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A 5
6

B A B D D 2 H
C A C A 3
G
D C D C
5
E C E D C E

F F D
G G D
H H
Processed = [A,C]
UnProcessed = [ B,C,D,E,F,G,H]
Select the closest unprocessed: vertex = D
Find ALL edges leaving D (DO NOT include edges toward the processed vertices) :
D->B = 2, D->E = 1, D->G = 3, D->F = 6  A->C->D->B = 6, A->C->D->E = 5, A->C->D->G = 7, A->C->D->F = 10
if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C,D]
UnProcessed = [ B,E,F,G,H]

16
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A 5
6

B D B D D 2 H

C A C A 3
G
D C D C
5
E D E D C E

F D F D
G D G E
H H
Processed = [A,C,D]
UnProcessed = [ B,E,F,G,H]
Select the closest unprocessed: vertex = E
Find ALL edges leaving E (DO NOT include edges toward the processed vertices) :
E->G = 1, A->C->D->E->G = 6
if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C,D,E]
UnProcessed = [B,F,G,H]

17
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A
6
5
B D B D D 2 H
C A C A 3
G
D C D C
5
E D E D C E

F D F D
G E G E
H H
Processed = [A,C,D,E]
UnProcessed = [B,F,G,H]
Select the closest unprocessed: vertex = B
Find ALL edges leaving B (DO NOT include edges toward the processed vertices) :
B->F = 13, A->C->D->B->F =19
if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C,D,E,B]
UnProcessed = [F,G,H]
18
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A
6
5
B D B D D 2 H
C A C A 3
G
D C D C
5
E D E D C E

F D F G
G E G E
H H G
Processed = [A,C,D,E,B]
UnProcessed = [F,G,H]
Select the closest unprocessed: vertex = G
Find ALL edges leaving G (DO NOT include edges toward the processed vertices) :
G->F = 2, G->H = 6 , A->C->D->E->G->F =8, A-C->D-E->G ->H = 12
if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C,D,E,B]
UnProcessed = [F,H]
19
 Example Cont’d Output
Input
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor B F
to Source (A) to Source (A) 13

A 0 A 0 A
6
5
B D B D D 2 H
C A C A 3
G
D C D C
5
E D E D C E

F G F G
G E G E
H G H F
Processed = [A,C,D,E,B]
UnProcessed = [F,H]
Select the closest unprocessed: vertex = F
Find ALL edges leaving F (DO NOT include edges toward the processed vertices) :
F-> H = 3 , A->C->D->E->G->F->H =11
if shorter path than previous to the source, then update, otherwise No update
Processed = [A,C,D,E,B,F]
UnProcessed = [H]

20
 Example Cont’d
Input Output
Vertex Shortest Distance Predecessor Vertex Shortest Distance Predecessor
to Source (A) to Source (A)
B F
13
A 0 A 0
6
B D B D A 5

C A D 2 H
C A
3
D C D C
G
E D E D
5
C E
F G F G
G E G E
H G H F
Processed = [A,C,D,E,B,F] What did we learn?
UnProcessed = [H]
Select the closest unprocessed: vertex = H 1- Do you think D is an extremely important vertex in this graph? What about
Find ALL edges leaving H (DO NOT include edges toward the
B?
processed vertices) :
NULL  No update 2- Should we remove the edge between B and F?
3- What if we want to consider multiple weight per edge (Length of the road
Processed = [A,C,D,E,B,F.H] and Travel Time)?
UnProcessed = [ ]
4- We found the shortest path. Can we find the second shortest path!
5- Is this a sparse or dense graph?
21
Revisit - What is Asymptotic Analysis?
 Asymptotic analysis attempts to estimate the resource consumption of an algorithm.
 It allows us to compare the relative costs of two or more algorithms for solving the same problem.
 Asymptotic analysis also gives algorithm designers a tool for estimating whether a proposed
solution is likely to meet the resource constraints for a problem before they implement an
actual program.
 Asymptotic analysis measures the efficiency of an algorithm, or its implementation as a program,
as the input size becomes large.
 The growth rate (GR) “Dominant Term” for an algorithm is the rate at which the cost of the
algorithm grows as the size of its input grows.
 Big-Oh notation ‘O (GR)’: it is the upper bound for an algorithm’s growth rate on the best-case,
average-case, or worst-case inputs.
 Big-Omega ‘Ω (GR)’: it is used to describe the least amount of a resource (time) that an algorithm
needs for some class of input.
 Big-Theta ‘θ (GR)’ : it used to indicate that there is no meaningful difference between what we
know about the growth rates of the upper and lower bound (which is usually the case for simple
algorithms)

22
Dijkstra’s Pseudocode - Revisited
Dijkstra(Graph G, Vertex source)
initialize distances to
- Set all nodes is infinity, except for the start which is 0
mark source as distance 0 - Start at the start vertex to be the current vertex
mark all vertices unprocessed - For the current vertex, look at all of the outgoing
while(there are unprocessed vertices){ neighbors/their edges. If the distance to the current node (from
the source)+ that edge weight is smaller than the proposed
let u be the closest unprocessed vertex
estimated distance for that neighbor, relax the neighbor node
foreach(edge (u,v) leaving u){ (update the proposed estimated distance and predecessor
// u is the current vertex
edge).
if(u’s dist+weight(u,v) < v’s dist){ -Update the current vertex to be the vertex with the next
smallest estimated distance that hasn’t been processed
v.dist = u.dist+weight(u,v)
v.predecessor = (u,v)
}
}
mark u as processed
}

23
Dijkstra’s Time Complexity - Using Adjacency
Matrix/List
The simplest implementation of the Dijkstra's algorithm stores vertices in an ordinary linked list or array
 Adjacency Matrix is Good for dense graphs (many edges)
 Linked List for sparse graphs (less edges) 3 8
Question: How to store weighted graphs? 8 2

7
3
 |V| vertices and |E| edges 4 7

6
 Initialization O(|V|) 4
2
 While loop O(|V|) 6
 For loop to Find and Remove minimum distance vertices O(|V|)

 Potentially |E| updates

 Update costs O(1)

Total time O(|V2| + |E|) = O(|V2| )  because |E| is in O(|V|2).

24
Dijkstra's algorithm Implementation using
Adjacency Matrix

Using MinVertex to scan the vertex list for the minimum value is more efficient when the graph is dense,
that is, when |E| approaches |V|2.
Because this scan is done |V| times, and because each edge requires a constant time update to D, the total
cost for this approach is θ (|V|2 + |E|) = θ (|V|2), because |E| is in O(|V|2).
25
MinHeap
 A min heap is a complete (almost complete) binary tree, where
the entry at each node is less than or equal to the entries in its
children.
 To add an entry to a heap, place the new entry at the next
available spot, and perform a reheapify.
 To remove the biggest entry, move the last node onto the root,
and perform a reheapify.  θ (log(n))

See Section 5.5: Heaps and Priority Queues 26

Dijkstra’s Pseuodocode using MinHeap
Dijkstra(Graph G, Vertex source)
initialize distances to
mark source as distance 0
mark all vertices unprocessed
initialize MPQ as a Min Priority Queue, add source
while(there are unprocessed vertices){
u = MPQ.removeMin();
foreach(edge (u,v) leaving u){
if(u’s dist+weight(u,v) < v’s dist){
v.dist = u.dist+weight(u,v)
v.predecessor = (u,v)
}
}
mark u as processed
}

27
Dijkstra’s Time Complexity – Using Priority
Queue
For sparse graphs, (i.e., graphs with much less than |V | edges) Dijkstra's
2

implemented more efficiently using a priority queue

• Initialization O(|V|) using O(|V|) buildHeap (MinHeap)

• While loop O(|V|)
• Find and remove min distance vertices O(log |V|)

• Potentially |E| updates

• Update costs O(log |V|) using decreaseKey

Total time O(|V|log|V| + |E|log|V|) = O(|E|log|V|)?! Why not O(|V|log|V|)

28
Dijkstra's algorithm Implementation using
Priority Queue

Using a priority queue is more efficient when the graph is sparse because its cost is θ ((|V| + |E|) log |E|).
However, when the graph is dense, this cost can become as great as θ (|V|2 log |E|) = θ (|V|2 log |V|).
29
Summary - Take-home Messages
 Given a weighted directed acyclic graph, we can find the shortest distance
between two vertices by:
 starting with a trivial path containing the initial vertex

 growing this path by always going to the next vertex which has the shortest

current path
 Dijkstra’s Algorithm time complexity in the worst-case input depends on the
ADT used to store the graph’s vertices (Adjacency Matrix/List, MinHeap)

30
What is next

Chapter 9: Sorting Algorithms

31