CSE408

Minimum Spanning
Tree(Prims,Kruskshal)
Lecture #26
Minimum Spanning Tree

• A minimum spanning tree connects all nodes

in a given graph
• A MST must be a connected and undirected
graph
• A MST can have weighted edges
• Multiple MSTs can exist within a given
undirected graph
More about Multiple MSTs

• Multiple MSTs can be generated depending on

which algorithm is used
• If you wish to have an MST start at a specific
node
• However, if there are weighted edges and all
weighted edges are unique, only one MST will
exist
Real Life Application of a MST

A cable TV company is laying cable in a new

neighborhood. If it is constrained to bury the cable
only along certain paths, then there would be a
graph representing which points are connected by
those paths. Some of those paths might be more
expensive, because they are longer, or require the
cable to be buried deeper; these paths would be
represented by edges with larger weights. A
minimum spanning tree would be the network with
the lowest total cost.
Prim’s Algorithm

• Initially discovered in 1930 by Vojtěch Jarník,

then rediscovered in 1957 by Robert C. Prim
• Similar to Dijkstra’s Algorithm regarding a
connected graph
• Starts off by picking any node within the graph
and growing from there
Prim’s Algorithm Cont.

• Label the starting node, A, with a 0 and all

others with infinite
• Starting from A, update all the connected
nodes’ labels to A with their weighted edges if
it less than the labeled value
• Find the next smallest label and update the
corresponding connecting nodes
• Repeat until all the nodes have been visited
Prim’s Algorithm Example
Prim’s Algorithm Example
Kruskal’s Algorithm

• Created in 1957 by Joseph Kruskal

• Finds the MST by taking the smallest weight in the graph and
connecting the two nodes and repeating until all nodes are
connected to just one tree
• This is done by creating a priority queue using the weights as
keys
• Each node starts off as it’s own tree
• While the queue is not empty, if the edge retrieved connects
two trees, connect them, if not, discard it
• Once the queue is empty, you are left with the minimum
spanning tree
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
 CSE408
Minimum Spanning
Tree(Prims,Kruskshal)
Lecture #26
Minimum Spanning Tree

• A minimum spanning tree connects all nodes

in a given graph
• A MST must be a connected and undirected
graph
• A MST can have weighted edges
• Multiple MSTs can exist within a given
undirected graph
More about Multiple MSTs

• Multiple MSTs can be generated depending on

which algorithm is used
• If you wish to have an MST start at a specific
node
• However, if there are weighted edges and all
weighted edges are unique, only one MST will
exist
Real Life Application of a MST

A cable TV company is laying cable in a new

neighborhood. If it is constrained to bury the cable
only along certain paths, then there would be a
graph representing which points are connected by
those paths. Some of those paths might be more
expensive, because they are longer, or require the
cable to be buried deeper; these paths would be
represented by edges with larger weights. A
minimum spanning tree would be the network with
the lowest total cost.
Prim’s Algorithm

• Initially discovered in 1930 by Vojtěch Jarník,

then rediscovered in 1957 by Robert C. Prim
• Similar to Dijkstra’s Algorithm regarding a
connected graph
• Starts off by picking any node within the graph
and growing from there
Prim’s Algorithm Cont.

• Label the starting node, A, with a 0 and all

others with infinite
• Starting from A, update all the connected
nodes’ labels to A with their weighted edges if
it less than the labeled value
• Find the next smallest label and update the
corresponding connecting nodes
• Repeat until all the nodes have been visited
Prim’s Algorithm Example
Prim’s Algorithm Example
Kruskal’s Algorithm

• Created in 1957 by Joseph Kruskal

• Finds the MST by taking the smallest weight in the graph and
connecting the two nodes and repeating until all nodes are
connected to just one tree
• This is done by creating a priority queue using the weights as
keys
• Each node starts off as it’s own tree
• While the queue is not empty, if the edge retrieved connects
two trees, connect them, if not, discard it
• Once the queue is empty, you are left with the minimum
spanning tree
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
Kruskal’s Algorithm Example
 CSE408
Dijkstra,Huffmancoding

Lecture # 27
Dijkstra’s Algorithm
Edge Relaxation

• Consider an edge e = (u,z) such

that d(u) = 50
– u is the vertex most recently d(z) = 75
u e
added to the cloud s z
– z is not in the cloud

• The relaxation of edge e

updates distance d(z) as
follows:
d(u) = 50
d(z)  min{d(z),d(u) + weight(e)} d(z) = 60
u e
s z
Example
0 0
8 A 4 8 A 4
2 2
8 7 2 1 4 8 7 2 1 3
B C D B C D

 3 9  5 3 9 8
2 5 2 5
E F E F

0 0
8 A 4 8 A 4
2 2
8 7 2 1 3 7 7 2 1 3
B C D B C D

5 3 9 11 5 3 9 8
2 5 2 5
E F E F
Example (cont.)

0
8 A 4
2
7 7 2 1 3
B C D

5 3 9 8
2 5
E F
0
8 A 4
2
7 7 2 1 3
B C D

5 3 9 8
2 5
E F
Dijkstra’s Algorithm

• A priority queue stores Algorithm DijkstraDistances(G, s)

the vertices outside the Q  new heap-based priority queue
cloud for all v  G.vertices()
if v = s
– Key: distance setDistance(v, 0)
– Element: vertex else
• Locator-based methods setDistance(v, )
– insert(k,e) returns a l  Q.insert(getDistance(v), v)
locator setLocator(v,l)
– replaceKey(l,k) changes while Q.isEmpty()
the key of an item u  Q.removeMin()
for all e  G.incidentEdges(u)
• We store two labels with
{ relax edge e }
each vertex: z  G.opposite(u,e)
– Distance (d(v) label) r  getDistance(u) + weight(e)
– locator in priority queue if r < getDistance(z)
setDistance(z,r)
Q.replaceKey(getLocator(z),r)
Analysis

• Graph operations
– Method incidentEdges is called once for each vertex
• Label operations
– We set/get the distance and locator labels of vertex z O(deg(z)) times
– Setting/getting a label takes O(1) time
• Priority queue operations
– Each vertex is inserted once into and removed once from the priority
queue, where each insertion or removal takes O(log n) time
– The key of a vertex in the priority queue is modified at most deg(w) times,
where each key change takes O(log n) time
• Dijkstra’s algorithm runs in O((n + m) log n) time provided the graph is
represented by the adjacency list structure
– Recall that Sv deg(v) = 2m
• The running time can also be expressed as O(m log n) since the graph
is connected
Extension

• Using the template Algorithm DijkstraShortestPathsTree(G, s)

method pattern, we can
extend Dijkstra’s …
algorithm to return a
tree of shortest paths for all v  G.vertices()
from the start vertex to …
all other vertices setParent(v, )
…
• We store with each
vertex a third label: for all e  G.incidentEdges(u)
– parent edge in the { relax edge e }
shortest path tree z  G.opposite(u,e)
• In the edge relaxation r  getDistance(u) + weight(e)
step, we update the if r < getDistance(z)
parent label setDistance(z,r)
setParent(z,e)
Q.replaceKey(getLocator(z),r)
Why Dijkstra’s Algorithm Works

• Dijkstra’s algorithm is based on the greedy method.

It adds vertices by increasing distance.
◼ Suppose it didn’t find all shortest
distances. Let F be the first wrong 0
vertex the algorithm processed. 8 A 4
◼ When the previous node, D, on the 2
7 7 2 1 3
true shortest path was considered, its B C D
distance was correct.
5 3 9 8
◼ But the edge (D,F) was relaxed at that 2 5
time! E F

◼ Thus, so long as d(F)>d(D), F’s distance

cannot be wrong. That is, there is no
wrong vertex.
Purpose of Huffman Coding

• Proposed by Dr. David A. Huffman in 1952

– “A Method for the Construction of Minimum
Redundancy Codes”
• Applicable to many forms of data
transmission
– Our example: text files
The Basic Algorithm

• Huffman coding is a form of statistical coding

• Not all characters occur with the same
frequency!
• Yet all characters are allocated the same amount
of space
– 1 char = 1 byte, be it e or x
The Basic Algorithm

• Any savings in tailoring codes to frequency of

character?
• Code word lengths are no longer fixed like
ASCII.
• Code word lengths vary and will be shorter for
the more frequently used characters.
The (Real) Basic Algorithm

1. Scan text to be compressed and tally

occurrence of all characters.
2. Sort or prioritize characters based on number of
occurrences in text.
3. Build Huffman code tree based on
prioritized list.
4. Perform a traversal of tree to determine all code words.
5. Scan text again and create new file using the
Huffman codes.
Building a Tree Scan the original text

• Consider the following short text:

Eerie eyes seen near lake.

• Count up the occurrences of all characters in the

text
Building a Tree Scan the original text

Eerie eyes seen near lake.

• What characters are present?

E e r i space
ysnarlk.
Building a Tree Scan the original text

Eerie eyes seen near lake.

• What is the frequency of each character in the text?

Char Freq. Char Freq. Char Freq.

E 1 y 1 k 1
e 8 s 2 . 1
r 2 n 2
i 1 a 2
space 4 l 1
Building a Tree Prioritize characters

• Create binary tree nodes with character

and frequency of each character
• Place nodes in a priority queue
– The lower the occurrence, the higher the
priority in the queue
Building a Tree Prioritize characters

• Uses binary tree nodes

public class HuffNode
{
public char myChar;
public int myFrequency;
public HuffNode myLeft, myRight;
}
priorityQueue myQueue;
Building a Tree

• The queue after inserting all nodes

E i y l k . r s n a sp e
1 1 1 1 1 1 2 2 2 2 4 8

• Null Pointers are not shown

Building a Tree

• While priority queue contains two or more nodes

– Create new node
– Dequeue node and make it left subtree
– Dequeue next node and make it right subtree
– Frequency of new node equals sum of frequency of left
and right children
– Enqueue new node back into queue
Building a Tree

E i y l k . r s n a sp e
1 1 1 1 1 1 2 2 2 2 4 8
Building a Tree

y l k . r s n a sp e
1 1 1 1 2 2 2 2 4 8

E i
1 1
Building a Tree

y l k . r s n a sp e
2
1 1 1 1 2 2 2 2 4 8
E i
1 1
Building a Tree

k . r s n a sp e
2
1 1 2 2 2 2 4 8
E i
1 1

y l
1 1
Building a Tree

2
k . r s n a 2 sp e
1 1 2 2 2 2 4 8
y l
1 1
E i
1 1
Building a Tree

r s n a 2 2 sp e
2 2 2 2 4 8
y l
E i 1 1
1 1

k .
1 1
Building a Tree

r s n a 2 2 sp e
2
2 2 2 2 4 8
E i y l k .
1 1 1 1 1 1
Building a Tree

n a 2 sp e
2 2
2 2 4 8
E i y l k .
1 1 1 1 1 1

r s
2 2
Building a Tree

n a 2 sp e
2 2 4
2 2 4 8

E i y l k . r s
1 1 1 1 1 1 2 2
Building a Tree

2 4 e
2 2 sp
8
4
y l k . r s
E i 1 1 1 1 2 2
1 1

n a
2 2
Building a Tree

2 4 4 e
2 2 sp
8
4
y l k . r s n a
E i 1 1 1 1 2 2 2 2
1 1
Building a Tree

4 4 e
2 sp
8
4
k . r s n a
1 1 2 2 2 2

2 2

E i y l
1 1 1 1
Building a Tree

4 4 4
2 sp e
4 2 2 8
k . r s n a
1 1 2 2 2 2
E i y l
1 1 1 1
Building a Tree

4 4 4
e
2 2 8
r s n a
2 2 2 2
E i y l
1 1 1 1

2 sp
4
k .
1 1
Building a Tree

4 4 4 6 e
2 sp 8
r s n a 2 2
4
2 2 2 2 k .
E i y l 1 1
1 1 1 1

What is happening to the characters with a low number of occurrences?

Building a Tree

4 6 e
2 2 2 8
sp
4
E i y l k .
1 1 1 1 1 1
8

4 4

r s n a
2 2 2 2
Building a Tree

4 6 e 8
2 2 2 8
sp
4 4 4
E i y l k .
1 1 1 1 1 1
r s n a
2 2 2 2
Building a Tree

8
e
8
4 4
10
r s n a
2 2 2 2 4
6
2 2
2 sp
4
E i y l k .
1 1 1 1 1 1
Building a Tree

8 10
e
8 4
4 4
6
2 2 2
r s n a sp
2 2 2 2 4
E i y l k .
1 1 1 1 1 1
Building a Tree

10
16
4
6
2 2 2 e 8
sp 8
4
E i y l k . 4 4
1 1 1 1 1 1

r s n a
2 2 2 2
Building a Tree

10 16
4
6
e 8
2 2 8
2 sp
4 4 4
E i y l k .
1 1 1 1 1 1
r s n a
2 2 2 2
Building a Tree

26

16
10

4 e 8
6 8
2 2 2 sp 4 4
4
E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2
Building a Tree

After enqueueing this node

there is only one node left in
priority queue.
26

16
10

4 e 8
6 8
2 2
2 sp 4 4
4
E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2
Building a Tree

Dequeue the single node left in the

queue.
26

This tree contains the new code 10

16
words for each character.
4 e 8
6 8
Frequency of root node should 2 2 2 sp 4 4
equal number of characters in text. 4
E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2

Eerie eyes seen near lake. 26 characters

Encoding the File Traverse Tree for Codes

• Perform a traversal of the

tree to obtain new code
words 26
• Going left is a 0 going right is 16
10
a1
• code word is only completed 4
6
e 8
8
when a leaf node is reached 2 2 2 sp 4 4
4
E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2
Encoding the File Traverse Tree for Codes

Char Code
E 0000
i 0001
y 0010
l 0011 26
k 0100 16
. 0101 10
space 011 4
e 10 e 8
6 8
r 1100 2 2 2
s 1101 sp 4 4
4
n 1110 E i y l k .
a 1111 1 1 1 1 1 1 r s n a
2 2 2 2
Encoding the File

• Rescan text and encode file

using new code words Char Code
E 0000
Eerie eyes seen near lake. i 0001
y 0010
l 0011
000010110000011001110001010110110100
k 0100
111110101111110001100111111010010010
. 0101
space 011
1 e 10
r 1100
s 1101
n 1110
a 1111
• Why is there no need for a
separator character?
.
Encoding the File

• Have we made things any

000010110000011001110001010110110100
better? 111110101111110001100111111010010010
1
• 73 bits to encode the text
• ASCII would take 8 * 26 =
208 bits

If modified code used 4 bits per

character are needed. Total bits
4 * 26 = 104. Savings not as great.
Decoding the File

• How does receiver know what the codes are?

• Tree constructed for each text file.
– Considers frequency for each file
– Big hit on compression, especially for smaller files
• Tree predetermined
– based on statistical analysis of text files or file types
• Data transmission is bit based versus byte based
Decoding the File

• Once receiver has tree it

scans incoming bit stream 26

• 0  go left 10
16

• 1  go right 4 e 8
6 8
2 2 2 sp 4 4
4
101000110111101111011 E i y l k .
1 1 1 1 1 1 r s n a
2 2 2 2
11110000110101
Summary

• Huffman coding is a technique used to compress files

for transmission
• Uses statistical coding
– more frequently used symbols have shorter code words
• Works well for text and fax transmissions
• An application that uses several data structures
 CSE408
Single Source Shortest
path

Lecture #28
Outline and Reading

• Weighted graphs
– Shortest path problem
– Shortest path properties
• Dijkstra’s algorithm
– Algorithm
– Edge relaxation
• The Bellman-Ford algorithm
• All-pairs shortest paths
Weighted Graphs

• In a weighted graph, each edge has an associated numerical value,

called the weight of the edge
• Edge weights may represent, distances, costs, etc.
• Example:
– In a flight route graph, the weight of an edge represents the distance in
miles between the endpoint airports

PVD
ORD
SFO
LGA

HNL
LAX
DFW
MIA
Shortest Path Problem

• Given a weighted graph and two vertices u and v, we want to find a

path of minimum total weight between u and v.
– Length of a path is the sum of the weights of its edges.
• Example:
– Shortest path between Providence and Honolulu
• Applications
– Internet packet routing
– Flight reservations
– Driving directions
PVD
ORD
SFO
LGA

HNL
LAX
DFW
MIA
Shortest Path Properties

Property 1:
A subpath of a shortest path is itself a shortest path
Property 2:
There is a tree of shortest paths from a start vertex to all the other vertices
Example:
Tree of shortest paths from Providence

PVD
ORD
SFO
LGA

HNL
LAX
DFW
MIA
Dijkstra’s Algorithm
Edge Relaxation

• Consider an edge e = (u,z) such

that d(u) = 50
– u is the vertex most recently d(z) = 75
u e
added to the cloud s z
– z is not in the cloud

• The relaxation of edge e

updates distance d(z) as
follows:
d(u) = 50
d(z)  min{d(z),d(u) + weight(e)} d(z) = 60
u e
s z
Example
0 0
8 A 4 8 A 4
2 2
8 7 2 1 4 8 7 2 1 3
B C D B C D

 3 9  5 3 9 8
2 5 2 5
E F E F

0 0
8 A 4 8 A 4
2 2
8 7 2 1 3 7 7 2 1 3
B C D B C D

5 3 9 11 5 3 9 8
2 5 2 5
E F E F
Example (cont.)

0
8 A 4
2
7 7 2 1 3
B C D

5 3 9 8
2 5
E F
0
8 A 4
2
7 7 2 1 3
B C D

5 3 9 8
2 5
E F
Dijkstra’s Algorithm

• A priority queue stores Algorithm DijkstraDistances(G, s)

the vertices outside the Q  new heap-based priority queue
cloud for all v  G.vertices()
if v = s
– Key: distance setDistance(v, 0)
– Element: vertex else
• Locator-based methods setDistance(v, )
– insert(k,e) returns a l  Q.insert(getDistance(v), v)
locator setLocator(v,l)
– replaceKey(l,k) changes while Q.isEmpty()
the key of an item u  Q.removeMin()
for all e  G.incidentEdges(u)
• We store two labels with
{ relax edge e }
each vertex: z  G.opposite(u,e)
– Distance (d(v) label) r  getDistance(u) + weight(e)
– locator in priority queue if r < getDistance(z)
setDistance(z,r)
Q.replaceKey(getLocator(z),r)
Analysis

• Graph operations
– Method incidentEdges is called once for each vertex
• Label operations
– We set/get the distance and locator labels of vertex z O(deg(z)) times
– Setting/getting a label takes O(1) time
• Priority queue operations
– Each vertex is inserted once into and removed once from the priority
queue, where each insertion or removal takes O(log n) time
– The key of a vertex in the priority queue is modified at most deg(w) times,
where each key change takes O(log n) time
• Dijkstra’s algorithm runs in O((n + m) log n) time provided the graph is
represented by the adjacency list structure
– Recall that Sv deg(v) = 2m
• The running time can also be expressed as O(m log n) since the graph
is connected
Extension

• Using the template Algorithm DijkstraShortestPathsTree(G, s)

method pattern, we can
extend Dijkstra’s …
algorithm to return a
tree of shortest paths for all v  G.vertices()
from the start vertex to …
all other vertices setParent(v, )
…
• We store with each
vertex a third label: for all e  G.incidentEdges(u)
– parent edge in the { relax edge e }
shortest path tree z  G.opposite(u,e)
• In the edge relaxation r  getDistance(u) + weight(e)
step, we update the if r < getDistance(z)
parent label setDistance(z,r)
setParent(z,e)
Q.replaceKey(getLocator(z),r)
Why Dijkstra’s Algorithm Works

• Dijkstra’s algorithm is based on the greedy method.

It adds vertices by increasing distance.
◼ Suppose it didn’t find all shortest
distances. Let F be the first wrong 0
vertex the algorithm processed. 8 A 4
◼ When the previous node, D, on the 2
7 7 2 1 3
true shortest path was considered, its B C D
distance was correct.
5 3 9 8
◼ But the edge (D,F) was relaxed at that 2 5
time! E F

◼ Thus, so long as d(F)>d(D), F’s distance

cannot be wrong. That is, there is no
wrong vertex.
Why It Doesn’t Work for Negative-Weight Edges

Dijkstra’s algorithm is based on the greedy method. It adds vertices by

increasing distance.

0
8 A 4
– If a node with a negative incident 6
edge were to be added late to the 7 7 5 1 4
cloud, it could mess up distances B C D
for vertices already in the cloud. 0 -8
5 9
2 5
E F

C’s true distance is 1, but it is already

in the cloud with d(C)=5!
Bellman-Ford Algorithm

• Works even with negative- Algorithm BellmanFord(G, s)

weight edges for all v  G.vertices()
• Must assume directed edges if v = s
(for otherwise we would have setDistance(v, 0)
negative-weight cycles) else
setDistance(v, )
• Iteration i finds all shortest
for i  1 to n-1 do
paths that use i edges. for each e  G.edges()
• Running time: O(nm). { relax edge e }
• Can be extended to detect a u  G.origin(e)
negative-weight cycle if it z  G.opposite(u,e)
exists r  getDistance(u) + weight(e)
– How? if r < getDistance(z)
setDistance(z,r)
Bellman-Ford Example
Nodes are labeled with their d(v) values

8 0 4 8 0 4
-2 -2
7 1 8 7 -2 1 4
     
3 9 3 9
-2 5 -2 5
   

8 0 4 8 0 4
-2 -2
5 8 7 1 -1 7 1
-2 4 5 -2 -1
1 3 9 3 9
-2 6 9 5 -2 4 5
  1 9
All-Pairs Shortest Paths

• Find the distance between Algorithm AllPair(G) {assumes vertices 1,…,n}

every pair of vertices in a for all vertex pairs (i,j)
weighted directed graph G. if i = j
• We can make n calls to D0[i,i]  0
else if (i,j) is an edge in G
Dijkstra’s algorithm (if no
D0[i,j]  weight of edge (i,j)
negative edges), which
else
takes O(nmlog n) time. D0[i,j]  + 
• Likewise, n calls to Bellman- for k  1 to n do
Ford would take O(n2m) for i  1 to n do
time. for j  1 to n do
• We can achieve O(n3) time Dk[i,j]  min{Dk-1[i,j], Dk-1[i,k]+Dk-1[k,j]}
using dynamic return Dn
programming (similar to the i Uses only vertices numbered 1,…,k
Floyd-Warshall algorithm). (compute weight of this edge)
j
Uses only vertices
numbered 1,…,k-1 k Uses only vertices
numbered 1,…,k-1
 CSE408
All pairs shortest path

Lecture #29
All pairs shortest path
• The problem: find the shortest path between every
pair of vertices of a graph
• The graph: may contain negative edges but no
negative cycles

• A representation: a weight matrix where

W(i,j)=0 if i=j.
W(i,j)= if there is no edge between i and j.
W(i,j)=“weight of edge”
• Note: we have shown principle of optimality
applies to shortest path problems
The weight matrix and the graph

1
1 2 3 4 5 3 v1 v2
9
1 0 1  1 5 5
1 2 3
2 9 0 3 2  v5

3   0 4  3
v4
2
v3
4   2 0 3 4

5 3    0
The subproblems

• How can we define the shortest distance di,j in

terms of “smaller” problems?

• One way is to restrict the paths to only include

vertices from a restricted subset.

• Initially, the subset is empty.

• Then, it is incrementally increased until it

includes all the vertices.
The subproblems

• Let D(k)[i,j]=weight of a shortest path from

vi to vj using only vertices from {v1,v2,…,vk}
as intermediate vertices in the path

– D(0)=W
– D(n)=D which is the goal matrix

• How do we compute D(k) from D(k-1) ?

The Recursive Definition:
Case 1: A shortest path from vi to vj restricted to using only
vertices from {v1,v2,…,vk} as intermediate vertices does not
use vk. Then D(k)[i,j]= D(k-1)[i,j].

Case 2: A shortest path from vi to vj restricted to using only

vertices from {v1,v2,…,vk} as intermediate vertices does use
vk. Then D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].

Shortest path using intermediate vertices

{V1, . . . Vk } Vk

Vj
Vi

Shortest Path using intermediate vertices { V1, . . . Vk -1 }

The recursive definition
• Since
D(k)[i,j]= D(k-1)[i,j] or
D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].
We conclude:
D(k)[i,j]= min{ D(k-1)[i,j], D(k-1)[i,k]+ D(k-1)[k,j] }.

Shortest path using intermediate vertices

{V1, . . . Vk } Vk

Vj
Vi

Shortest Path using intermediate vertices { V1, . . . Vk -1 }

The pointer array P

• Used to enable finding a shortest path

• Initially the array contains 0

• Each time that a shorter path from i to j is found

the k that provided the minimum is saved (highest
index node on the path from i to j)

• To print the intermediate nodes on the shortest

path a recursive procedure that print the shortest
paths from i and k, and from k to j can be used
Floyd's Algorithm Using n+1 D matrices

Floyd//Computes shortest distance between all pairs of

//nodes, and saves P to enable finding shortest paths
1. D0  W // initialize D array to W [ ]
2. P  0 // initialize P array to [0]
3. for k  1 to n
4. do for i  1 to n
5. do for j  1 to n
6. if (Dk-1[ i, j ] > Dk-1 [ i, k ] + Dk-1 [ k, j ] )
7. then Dk[ i, j ]  Dk-1 [ i, k ] + Dk-1 [ k, j ]
8. P[ i, j ]  k;
9. else Dk[ i, j ]  Dk-1 [ i, j ]
 Example
1 2 3
1 0 4 5
W= D0 =
5
2 2 0 
1
3  -3 0
4 2 3
1 2 3
-3 1 0 0 0
2
P= 2 0 0 0
3 0 0 0
 1 2 3
k=1
1 5 D0 =
1 0 4 5 Vertex 1
4 3
2
-3
2 2 0  can be
2 3  -3 0
1 2 3
intermediat
D1 =
1 0 4 5 1 0
e node
D [2,3] = min( D [2,3], 0 0
D [2,1]+D [1,3] )
2 2 0 7 = min (, 7)
=7
3  -3 0

1 2 3
D1[3,2] = min( D0[3,2], D0[3,1]+D0[1,2] )
1 0 0 0
= min (-3,)
P= 2 0 0 1 = -3
3 0 0 0
 1 2 3 k=2
1 5 D1 = 1
3 2
0 4 5
Vertices 1,
4 2 2 0 7

2
-3 3  -3 0
2 can be
1 2 3 intermediat
D2 =
1 0 4 5 2 1
e
D [1,3] = min( D [1,3],
= min (5, 4+7)
1 1
D [1,2]+D [2,3] )
2 2 0 7
=5
3 -1 -3 0

1 2 3
1 0 0 0
D2[3,1] = min( D1[3,1], D1[3,2]+D1[2,1] )
P= 2 0 0 1 = min (, -3+2)
3 2 0 0 = -1
 1 5 D2 =
1 2 3 k=3
1 0 4 5
4 2 3
2 2 0 7
Vertices 1,
2
-3
3 -1 -3 0 2, 3 can be
1 2 3 intermediat
D3 =
1 0 2 5 3 2e D [1,3]+D [3,2] )
D [1,2] = min(D [1,2], 2 2

2 2 0 7 = min (4, 5+(-3))

3 -1 -3 0 =2

1 2 3
1 0 3 0 D3[2,1] = min(D2[2,1], D2[2,3]+D2[3,1] )
= min (2, 7+ (-1))
P= 2 0 0 1
=2
3 2 0 0
Floyd's Algorithm: Using 2 D matrices

Floyd
1. D  W // initialize D array to W [ ]
2. P  0 // initialize P array to [0]
3. for k  1 to n
// Computing D’ from D
4. do for i  1 to n
5. do for j  1 to n
6. if (D[ i, j ] > D[ i, k ] + D[ k, j ] )
7. then D’[ i, j ]  D[ i, k ] + D[ k, j ]
8. P[ i, j ]  k;
9. else D’[ i, j ]  D[ i, j ]
10. Move D’ to D.
Can we use only one D matrix?

• D[i,j] depends only on elements in the kth

column and row of the distance matrix.
• We will show that the kth row and the kth
column of the distance matrix are unchanged
when Dk is computed
• This means D can be calculated in-place
The main diagonal values

• Before we show that kth row and column of D

remain unchanged we show that the main
diagonal remains 0

• D(k)[ j,j ] = min{ D(k-1)[ j,j ] , D(k-1)[ j,k ] + D(k-1)[ k,j

]}
= min{ 0, D(k-1)[ j,k ] + D(k-1)[ k,j ] }
=0
• Based on which assumption?
The kth column

• kth column of Dk is equal to the kth column of

Dk-1
• Intuitively true - a path from i to k will not
become shorter by adding k to the allowed
subset of intermediate vertices

• For all i, D(k)[i,k] =

= min{ D(k-1)[i,k], D(k-1)[i,k]+ D(k-1)[k,k] }
= min { D(k-1)[i,k], D(k-1)[i,k]+0 }
= D(k-1)[i,k]
The kth row

• kth row of Dk is equal to the kth row of Dk-1

For all j, D(k)[k,j] =

= min{ D(k-1)[k,j], D(k-1)[k,k]+ D(k-1)[k,j] }
= min{ D(k-1)[ k,j ], 0+D(k-1)[k,j ] }
= D(k-1)[ k,j ]
Floyd's Algorithm using a single D

Floyd
1. D  W // initialize D array to W [ ]
2. P  0 // initialize P array to [0]
3. for k  1 to n
4. do for i  1 to n
5. do for j  1 to n
6. if (D[ i, j ] > D[ i, k ] + D[ k, j ] )
7. then D[ i, j ]  D[ i, k ] + D[ k, j ]
8. P[ i, j ]  k;
Printing intermediate nodes on shortest path from q to r

path(index q, r) 1 2 3
if (P[ q, r ]!=0)
1 0 3 0
path(q, P[q, r])
println( “v”+ P[q, r]) P= 2 0 0 1
path(P[q, r], r)
3
return; 2 0 0
//no intermediate nodes
else return
1 5
Before calling path check D[q, r] < , and print
node q, after the call to
4 3
path print node r 2
2
-3
Example
 The final distance matrix and P

The values in parenthesis are the non zero P values.

 The call tree for Path(1, 4)

Path(1, 4)

Path(1, 6) Path(6, 4)
Print
v6
P(1, 6)=0

Path(6, 3) Print Path(3, 4)

v3

P(6, 3)=0 P(3, 4)=0

The intermediate nodes on the shortest path from 1 to 4 are v6, v3.
The shortest path is v1, v6, v3, v4.
 CSE408
Maximum Flow

Lecture #30
Maximum Flow

• Maximum Flow Problem

• The Ford-Fulkerson method
• Maximum bipartite matching
Flow networks:

• A flow network G=(V,E): a directed graph, where each edge

(u,v)E has a nonnegative capacity c(u,v)>=0.
• If (u,v)E, we assume that c(u,v)=0.
• two distinct vertices :a source s and a sink t.

12

16 20

s 10 9 7 t
4

13 4

14
Flow:

• G=(V,E): a flow network with capacity function c.

• s-- the source and t-- the sink.
• A flow in G: a real-valued function f:V*V → R satisfying
the following two properties:
• Capacity constraint: For all u,v V,
we require f(u,v)  c( u,v).
• Flow conservation: For all u V-{s,t}, we require

 f (e) =  f (e)
e.in.v e.out .v
Net flow and value of a flow f:

• The quantity f (u,v) is called the net flow from vertex

u to vertex v.
• The value of a flow is defined as

f =  f ( s, v )
– The total flow fromvsource
V
to any other vertices.
– The same as the total flow from any vertices to
the sink.
 12/12

15/20
11/16

s 10 1/4 4/9 7/7 t

8/13 4/4

11/14

A flow f in G with value . f = 19

Maximum-flow problem:

• Given a flow network G with source s and sink t

• Find a flow of maximum value from s to t.

• How to solve it efficiently?

The Ford-Fulkerson method:

• This section presents the Ford-Fulkerson method for solving

the maximum-flow problem.We call it a “method” rather than
an “algorithm” because it encompasses several
implementations with different running times.The Ford-
Fulkerson method depends on three important ideas that
transcend the method and are relevant to many flow
algorithms and problems: residual networks,augmenting
paths,and cuts. These ideas are essential to the important
max-flow min-cut theorem,which characterizes the value of
maximum flow in terms of cuts of the flow network.
Continue:

• FORD-FULKERSON-METHOD(G,s,t)
• initialize flow f to 0
• while there exists an augmenting path p
• do augment flow f along p
• return f
Residual networks:

• Given a flow network and a flow, the residual

network consists of edges that can admit more net
flow.
• G=(V,E) --a flow network with source s and sink t
• f: a flow in G.
• The amount of additional net flow from u to v
before exceeding the capacity c(u,v) is the residual
capacity of (u,v), given by: cf(u,v)=c(u,v)-f(u,v)
in the other direction: cf(v, u)=c(v, u)+f(u, v).
 Example of residual network

12 4/12
16 v1 v3 20 4/16 v1 v3 20

s 10 4 7 t s 7
9 10 4 t
4/9
13 4 13 4/4
v2 v4 v2 v4
14 4/14

(a)
 Example of Residual network (continued)

8
12
v1 v3 20
4
4
s 10 4 4
7 t
5
13 10 4
v2 v4
4

(b)
Fact 1:

• Let G=(V,E) be a flow network with source s and sink t, and let
f be a flow in G.
• Let Gf be the residual network of G induced by f,and let f’ be a
flow in Gf.Then, the flow sum f+f’ is a flow in G with
value .
f + in
• f+f’: the flow = fsame
f ' the + f 'direction will be added.
the flow in different directions will be cnacelled.
Augmenting paths:

• Given a flow network G=(V,E) and a flow f, an augmenting

path is a simple path from s to t in the residual network Gf.

• Residual capacity of p : the maximum amount of net flow that

we can ship along the edges of an augmenting path p, i.e.,
cf(p)=min{cf(u,v):(u,v) is on p}.

2 3 1

The residual capacity is 1.

 Example of an augment path (bold edges)

8
12
v1 v3 20
4
4
s 10 4 4
7 t
5
13 10 4
v2 v4
4

(b)
The basic Ford-Fulkerson algorithm:

• FORD-FULKERSON(G,s,t)
• for each edge (u,v) E[G]
• do f[u,v]  0
• f[v,u] 0
• while there exists a path p from s to t in the residual network
Gf
• do cf(p) min{cf(u,v): (u,v) is in p}
• for each edge (u,v) in p
• do f[u,v] f[u,v]+cf(p)
•
Example:

• The execution of the basic Ford-Fulkerson algorithm.

• (a)-(d) Successive iterations of the while loop: The left side of
each part shows the residual network Gf from line 4 with a
shaded augmenting path p.The right side of each part shows
the new flow f that results from adding fp to f.The residual
network in (a) is the input network G.(e) The residual network
at the last while loop test.It has no augmenting paths,and the
flow f shown in (d) is therefore a maximum flow.
 12 4/12
16 v1 v3 20 4/16 v1 v3 20

s 10 4 7 t s 7
9 10 4 t
4/9
13 4 13 4/4
v2 v4 v2 v4
14 4/14

(a)
 8
12 4/12
v1 v3 20 v1 v3 7/20
4 11/16
4
s 10 4 4
7 t s 7/10 4 7/7 t
5 4/9
13 10 4 13 4/4
v2 v4 v2 v4
4 11/14

(b)
 8 12/12
5 v1 v3 13
4 11/16 v1 v3 15/20
11 4
s 3 11 7 7 t s 7/7
10 1/4 t
5 4/9
13 3
4 8/13 4/4
v2 v4 v2 v4
11 11/14

(c)
 12 12/12
5 v1 v3 5 11/16 v1 v3 19/20
11 4
s 11 3 7 15 t s 7/7
5 10 1/4 t
5 9
8 3
4 12/13 4/4
v2 v4 v2 v4
11 11/14

(d)
 12
5 v1 v3 1

s 3 19
1 11 9 7 t

12 3 4
v2 v4
11

(e)
Time complexity:

• If each c(e) is an integer, then time complexity is O(|E|f*),

where f* is the maximum flow.
• Reason: each time the flow is increased by at least one.
• This might not be a polynomial time algorithm since f* can be
represented by log (f*) bits. So, the input size might be log(f*).
The Edmonds-Karp algorithm

• Find the augmenting path using breadth-first search.

• Breadth-first search gives the shortest path for
graphs (Assuming the length of each edge is 1.)
• Time complexity of Edmonds-Karp algorithm is
O(VE2).
• The proof is very hard and is not required here.
Maximum bipartite matching:

• Bipartite graph: a graph (V, E), where V=LR, L∩R=empty, and

for every (u, v)E, u L and v R.
• Given an undirected graph G=(V,E), a matching is a subset of
edges ME such that for all vertices vV,at most one edge of
M is incident on v.We say that a vertex v V is matched by
matching M if some edge in M is incident on v;otherwise, v is
unmatched. A maximum matching is a matching of maximum
cardinality,that is, a matching M such that for any matching M’,
we have .

M  M'
 L R L R
(a) (b)

A bipartite graph G=(V,E) with vertex partition V=LR.(a)A matching with

cardinality 2.(b) A maximum matching with cardinality 3.
Finding a maximum bipartite matching:

• We define the corresponding flow network G’=(V’,E’) for the

bipartite graph G as follows. Let the source s and sink t be
new vertices not in V, and let V’=V{s,t}.If the vertex
partition of G is V=L R, the directed edges of G’ are given by
E’={(s,u):uL} {(u,v):u L,v R,and (u,v) E} {(v,t):v
R}.Finally, we assign unit capacity to each edge in E’.

• We will show that a matching in G corresponds directly to a

flow in G’s corresponding flow network G’. We say that a flow
f on a flow network G=(V,E) is integer-valued if f(u,v) is an
integer for all (u,v) V*V.
 s
t

L R L R
(a) (b)

(a)The bipartite graph G=(V,E) with vertex partition V=LR. A

maximum matching is shown by shaded edges.(b) The corresponding
flow network.Each edge has unit capacity.Shaded edges have
a flow of 1,and all other edges carry no flow.
Continue:

• Lemma .
• Let G=(V,E) be a bipartite graph with vertex partition
V=LR,and let G’=(V’,E’) be its corresponding flow network.If
M is a matching in G, then there is an integer-valued flow f in
G’ with value .Conversely, if f is an integer-valued
flow in G’,then there is a matchingfM=inMG with
cardinality .
• Reason: The edges incidentM =tof s and t ensures this.
– Each node in the first column has in-degree 1
– Each node in the second column has out-degree 1.
– So each node in the bipartite graph can be involved once in the flow.
Example:
Aug. path:
 1

1
1

1
1
1

Residual network. Red edges are new edges in the residual network.
The new aug. path is bold. Green edges are old aug. path. old flow=1.