CSE 101

Algorithm Design and Analysis

Russell Impagliazzo
[email protected]
Today’s plan
• BFS
• Shortest path
• Shortest path with edge weights
• Dijkstra’s
• Priority queue
Breadth first search
• BFS (G, s)
• Initialize array Visited to False, Visited[s]=True
• Initialize Queue Q to include s
• While Q is not empty do:
• v= Q.Dequeue
• For u ∊ N(v) IF Visited[u]== False do:
• Visited[u]= True; Q.Enqueue(u)
• Return set of u so that Visited[u]==True
 Example
A
B
C
s=A, Q= (A)
v=A Q=(B, D)
D
v=B , Q= (D, C, E)
v=D, Q=(C,E,F)
H v=C , Q= (E,F)
E
v= E, Q= (F)
v=F , Q= (G)
v=G, Q= (H)
v=H, Q is empty

G F
BFS explores the closest vertices first
• So the intuition is that it should find shortest paths.
• How can we keep track of shortest paths/minimum
distances?
Augmented BFS
procedure BFS(G, s)
Input: Graph G = (V,E), (directed or undirected) and a vertex s in V.
Output: For all vertices u reachable from s, dist(u) is the distance from s to u. and for all
vertices u not reachable from s, dist(u) = ∞
for each vertex u in V:
dist(u)=∞
dist(s) = 0
Q = [s] (queue just containing s)
while Q is not empty
v = Q.dequeue
for all edges (v,u) in E
if dist(u)=∞ then
Q.enqueue(u); prev(u)=v;
dist(v)=dist(u) + 1
 Example
A
B
C

H
E A B C D E F G H

G F
Proof of correctness
• For each vertex v, we want to show that dist(v) is the
minimum distance of all paths from s to v.
• Claim: at the first time when the head of the queue has
distance marked d:
• (1) all vertices with distances at distance ≤ 𝑑 from s have their
distance values correctly set.
• (2). All vertices in the queue are exactly those of distance d.
• (2) All other vertices are marked distance infinity
 Claim: for each distance value d = 0, 1, 2,…:

Proof of correctness (1) all vertices at distance ≤ 𝑑 from s have their

distance values correctly set.
(2) all other vertices (distance > d from s) have
distances set to ∞
• Proof: (3) The queue contains exactly the nodes at
distance d.
• Base Case: (for d = 0)
• (1) dist(s) = 0 is the correct distance value
• (2) all other vertices have distances set to ∞
• (3) The queue contains only s which is the only vertex at distance 0
 Claim: for each distance value d = 0, 1, 2,…:

Proof of correctness (1) all vertices at distance ≤ 𝑑 from s have their

distance values correctly set.
(2) all other vertices (distance > d from s) have
distances set to ∞
• Proof: (3) The queue contains exactly the nodes at
distance d.
• Induction step:
• Let k be an arbitrary integer such that 𝑘 ≥ 0. Assume that
the above three statements are true for when d=k.
• (WTS the three statements are true for when d=k+1.)
• (1)
• (2)
• (3)
 Claim: for each distance value d = 0, 1, 2,…:

Proof of correctness (1) all vertices at distance ≤ 𝑑 from s have their

distance values correctly set.
(2) all other vertices (distance > d from s) have
distances set to ∞
• Proof: (3) The queue contains exactly the nodes at
• Induction step (continued): distance d.
• All vertices distance ≤ 𝑘 have been set and the queue only contains vertices at distance = k,
and the remaining vertices are marked distance infinity
• Since queues are FIFO, the head of the queue will have distance marked k until the current
queue is empty.
• (1) All new vertices added to the queue during this time have distance k+1 and
are set correctly, since they are neighbors of distance k vertices and didn’t have
distances set before.
• (2) All vertices of distance k+1 have been added to the queue, since each one is
the neighbor of some distance k vertex. .
• The first time we see the head of the queue at distance k+1 is right after the last
time the head of the queue has distance k.
 Claim: for each distance value d = 0, 1, 2,…:

Proof of correctness (1) all vertices at distance ≤ 𝑑 from s have their

distance values correctly set.
(2) all other vertices (distance > d from s) have
distances set to ∞
• Proof: (3) The queue contains exactly the nodes at
distance d.
• Using induction claim.
• Led dmax be the maximum distance of a vertex from s.
• Applying the induction claim to dmax, the first time the head
of the queue has distance dmax, all vertices have their correct distances
defined (since all vertices have distance ≤ dmax).
Runtime of BFS
• Notice that in BFS, each vertex enters the queue (F) at
most one time.
• This was the assumption we made about graphsearch
when we calculated its runtime.
• So BFS runs in 𝑂 𝑉 + 𝐸 time.

•Corollary:
For each vertex 𝑣, dist(𝑣) is set at most one time.
Edge lengths (weights)
edges can be given values such as
• Distance
• Cost
• Time
• Bandwidth
• Value
BFS on weighted graphs.
• BFS only works to find shortest distances on graphs in
which each edge has equal weight.
A 3 B 2 C

2 1 4 1

D 4 E F
BFS on weighted graphs.
• Discuss how we can use a reduction to solve the
problem of shortest paths with edge lengths.
A 3 B 2 C

2 1 4 1

D 4 E F
 BFS on weighted graphs.
• On a graph 𝐺 with integer edge lengths, form 𝐺 ! by adding ℓ" − 1 many new
vertices between 𝑢 and 𝑣 for every edge 𝑒 = (𝑢, 𝑣). Then run BFS on 𝐺′.

A 3 B 2 C A B C

2 1 4 1

D 4 E F D E F

𝐺 𝐺′
Problems with this method
If the edge lengths
(weights) are large
integers then it is
impractical.

In this example
with 10 vertices,
we must add 1,783
more vertices!!!!!!!!
Dijkstra’s algorithm
• Simulates what would happen if we introduced
intermediate vertices, but without actually introducing
intermediate vertices.

Think of an ant colony, exploring each edge in the graph as

soon as it discovers it, discovering vertices in order of
distance from source.
Dijkstra Example
BOS

60
ORD 80
250

IAD

0
10
DEN

10
0

60
0
25 100
ATL
SAN 120
0
DFW 30 0
9
 Graph reachability:
Dijkstra’s at a high-level
procedure GraphSearch (Dijkstra’s) (G: directed graph with edgeweights, s: vertex)

Initialize X = empty, F=V

Initialize dist(v) = ∞ for all v
Initialize dist(s) = 0
While F is not empty:
Pick v in F with smallest dist(v)
For each neighbor u of v in F:
If dist(u) > dist(v) + ℓ(𝑣, 𝑢):
set dist(u) = dist(v) + ℓ(𝑣, 𝑢):
Move v from F to X.
 BOS,
BOS,∞
280
80
ORD,
ORD,
ORD, ∞

60
200
250
250
IAD,
IAD, ∞
0 270
420

110000
25

60
DEN,
DEN, ∞
100 100
SAN, 0 0
30 ATL,
ATL, ∞
210

10
120

0
90
DFW,
DFW,
120∞
DFW,
120
Correctness of Dijkstra’s
Claim: After Dijkstra’s is done, dist(v) is the length of the
shortest path from s to v.

(Existence) After every iteration, dist(v) is the length of

some path from s to v.

Exercise: loop invariant

Increasing order of distance found
• Lemma: Let 𝑣!, 𝑣", . . 𝑣# be the vertices in the graph in the
order they are put into X in Dijkstra’s algorithm
• Then 𝑑𝑖𝑠𝑡 𝑣! ≤ 𝑑𝑖𝑠𝑡 𝑣" ≤ ⋯ ≤ 𝑑𝑖𝑠𝑡(𝑣# )

Note: we haven’t yet shown that dist(v) is the actual

shortest path distance, so its not clear why this is helpful.
Increasing order of distance found
• Lemma: Let 𝑣!, 𝑣", . . 𝑣# be the vertices in the graph in the
order they are put into X in Dijkstra’s algorithm
• Then 𝑑𝑖𝑠𝑡 𝑣! ≤ 𝑑𝑖𝑠𝑡 𝑣" ≤ ⋯ ≤ 𝑑𝑖𝑠𝑡(𝑣# )
Proof: When we chose 𝑣$ , it was the smallest distance
in F, so at that point 𝑑𝑖𝑠𝑡 𝑣$ ≤ 𝑑𝑖𝑠𝑡(𝑣$%!). After, we might
have not changed 𝑑𝑖𝑠𝑡 𝑣$%! or we might set it to
𝑑𝑖𝑠𝑡 𝑣$%! = 𝑑𝑖𝑠𝑡 𝑣$ + 𝑙((𝑣$ , 𝑣$%!))≥ 𝑑𝑖𝑠𝑡(𝑣$ ). So after we
move 𝑣$ into X, 𝑑𝑖𝑠𝑡(𝑣$ ) ≤ 𝑑𝑖𝑠𝑡(𝑣$%!). Afterwards, we won’t
change either one.
Correctness of Dijkstra’s
Claim: After Dijkstra’s is done, dist(v) is the length of the
shortest path from s to v for all vertices 𝑣 ∈ 𝑉.

(optimization) After Dijkstra’s is done, dist(v) is the length of

the shortest path from s to v for all vertices 𝑣 ∈ 𝑉.
Correctness of Dijkstra’s
Claim: After Dijkstra’s is done, dist(v) is the length of the
shortest path from s to v for all vertices 𝑣 ∈ 𝑉.

(optimization) After Dijkstra’s is done, dist(v) is the length of

the shortest path from s to v for all vertices 𝑣 ∈ 𝑉.

Suppose for some vertex v, that there is a path p from s to

v such that length(p) < dist(v).
 Correctness of Dijkstra’s
(<-) After Dijkstra’s is done, dist(v) is the length of the
shortest path from s to v for all vertices 𝑣 ∈ 𝑉.

Suppose for some vertex v, that there is a path p from s to

v such that length(p) < dist(v).
(Let d(v) be the actual length of the shortest path from s to v.)

S z w v

dist(s) = d(s) dist(z) = d(z) dist(w) > d(w) dist(v) > d(v)
 Correctness of Dijkstra’s
(Let d(v) be the actual length of the shortest path from s to v.)
Then let z be the last vertex in the path s.t. dist(z) = d(z) and
Let w be the first vertex in the path s.t. dist(w) > d(w) (at the end)
Case 1: d(w) = d(z) + ℓ(𝑧, 𝑤). 𝑑𝑖𝑠𝑡 𝑧 = 𝑑 𝑧 ≤ 𝑑 𝑤 < 𝑑𝑖𝑠𝑡 𝑤 . By the
increasing distance lemma, z is moved into X before w. When z is chosen from
F, the algorithm would reset dist(w) = d(z) + ℓ 𝑧, 𝑤 = 𝑑(𝑤). So dist(w) = d(w)
at the end, a contradiction.
Case 2: d(w) < d(z) + ℓ(𝑧, 𝑤). Then the path p is actually not the shortest path.

S z w v

dist(s) = d(s) dist(z) = d(z) dist(w) > d(w) dist(v) > d(v)
 Graph reachability:
Dijkstra’s high-level (Runtime)
procedure GraphSearch (Dijkstra’s) (G: directed graph with edgeweights, s: vertex)

Initialize X = empty, F = {s}

Initialize dist(v) = ∞ for all v
Initialize dist(s) = 0
While F is not empty:
Pick v in F with smallest dist(v)
For each neighbor u of v in F: We move every v from F to X at most once.
If dist(u) > dist(v) + ℓ(𝑣, 𝑢): We ``relax’’ along each edge at most once.
move u to F.
set dist(u) = dist(v) + ℓ(𝑣, 𝑢):
Move v from F to X.
 Graph reachability:
Dijkstra’s high-level (carefully choosing)
Claim: Let d(v) be the length of the shortest path from s to v. Then after every iteration, dist(v) = d(v) for all
vertices v in X.

(This claim implies that once a vertex moves into X, it will never move back to F. Therefore, every vertex
moves from F to X at most once
 Graph reachability:
Dijkstra’s high-level (carefully choosing)
Claim: Let d(v) be the length of the shortest path from s to v. Then after every iteration, dist(v) = d(v) for all
vertices v in X.

Base Case: The first vertex to move into X is s. dist(s) = 0 = d(s).

Inductive Hypothesis: After k vertices have been moved into X, dist(v) = d(v) for all vertices v in X.

Inductive Step: Suppose u is the next vertex to move into X……

Want to show that dist(u) = d(u).
 Graph reachability:
Dijkstra’s high-level (carefully choosing)
Inductive Step: Suppose u is the next vertex to move into X……
Want to show that dist(u) = d(u).
Suppose by contradiction that dist(u) > d(u).
So there is a path P such that length(P) = d(u).

y
X w
P
u
Graph reachability:
Did we use non-negativity?
y
X w
P
u

P goes from s to u. so there is an edge that crosses the boundary of X.

dist(w) = d(w) by IH.
dist(y) ≥ dist(u) by choice of u.
Therefore d(u) = len(P) ≥ dist(w) + ℓ(𝑤, 𝑦) ≥ dist(y) ≥ dist(u) > d(u).
LOW-LEVEL DIJKSTRA
• We’ve reduced the problem of creating a low-level
efficient version of Dijkstra’s algorithm to that of finding a
good implementation of a priority queue.
 Graph reachability:
Dijkstra’s mid-level (modification)
procedure GraphSearch (Dijkstra’s) (G: directed graph with edgeweights, s: vertex)

Initialize X = empty, F = {s}

Initialize dist(v) = ∞ for all v
Initialize dist(s) = 0
While F is not empty:
Pick v in F.
For each neighbor u of v:
If dist(u) > dist(v) + ℓ(𝑣, 𝑢):
move u to F.
set dist(u) = dist(v) + ℓ(𝑣, 𝑢):
Move v from F to X.
 Graph reachability:
Dijkstra’s mid-level (carefully choosing)
procedure GraphSearch (Dijkstra’s) (G: directed graph with edgeweights, s: vertex)

Initialize X = empty, F = {s}

Initialize dist(v) = ∞ for all v
Initialize dist(s) = 0
While F is not empty:
Pick the v in F that has the lowest dist(v) value.
For each neighbor u of v:
If dist(u) > dist(v) + ℓ(𝑣, 𝑢):
move u to F.
set dist(u) = dist(v) + ℓ(𝑣, 𝑢):
Move v from F to X.
Priority Queue
A priority queue is a data structure of a set of objects
(vertices) along with key values for each object that can
be changed. And it can support the following operations.
• insert(H,u): Insert new (object,key) into H
• deletemin(H): find the (object,key) in H with the smallest key
• decreasekey(H,u): decrease and update the key value of a particular object.
• makequeue(S): initialize a priority queue from a set S.
DIJKSTRA’s In TERMS OF PRIORITY
QUEUE Runtime:
procedure dijkstra(G,ℓ,s)
for all u in V O(deletemin(H)*|V|+ decreasekey(H,v)*|E|)
dist(u) = infinity
prev(u) = nil
dist(s) = 0
H = makequeue(V)
while H is not empty
u = deletemin(H)
for all edges (u,v) in E
if dist(v) > dist(u) + ℓ(u,v):
dist(v)= dist(u) + ℓ(u,v)
prev(v)=u
decreasekey(H,v)
 Dijkstra Example PQ s A B C D E F G
s 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞
12
7
s A B A CDF 12 ∞ 20 6 ∞ 15 ∞
10 ACFBE
12 16 14 26 15
G
6 8 5
ACFBE 9
20 10 12 16 14 17 15
15
20 CFBE 16 14 17 15
C D
D E
E
8
3 FBE 16 17 15
10 8
s
F G BE 16 17
D F A 9
E 17
C B G prev
E
Runtime of Dijkstra’s
procedure dijkstra(G,ℓ,s) deletemin × |V| +
for all u in V decreasekey × |E|
dist(u) := infinity
prev(u) := nil
dist(s) := 0
H := makequeue(V)
while H is not empty
u := deletemin(H)
for all edges (u,v) in E
if dist(v) > dist(u) + ℓ(u,v) then
dist(v):= dist(u) + ℓ(u,v)
prev(v):=u
decreasekey(H,v)
LOW-LEVEL DIJKSTRA
• We’ve reduced the problem of creating a low-level
efficient version of Dijkstra’s algorithm to that of finding a
good implementation of a priority queue
• Total time will be
O(|V|(deletemin) + |E|(decrease key))
• Different implementations of priority queue will have
different trade-offs between costs of operations. Might not
be a single one that is optimal for all kinds of graphs. So
flexibility is useful
Array as a priority queue
•Array(hash table): indexed by vertex, giving key value.
Example: (A,2),(B,9),(C,4),(D,1),(E,6),(F,3),(G,4)
H[A] = 2, H[B] = 9 ,…..

• deletemin:
• decreasekey:
Array as a priority queue
• Array(hash table): indexed by vertex, giving key value.
Example: (A,2),(B,9),(C,4),(D,1),(E,6),(F,3),(G,4)
H[A] = 2, H[B] = 9 ,…..

• deletemin: O(|V|)
• decreasekey: O(1)

• Total runtime:
|V|*deletemin + |E|*decreasekey
|V|*O(|V|) + |E|*O(1) = O( 𝑉 ! )
Binary heap
A complete binary tree
of objects (vertices)
with the property that
each key value of an
object is less than or
equal to the key value
of its children.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Binary heap
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

•
 Binary heap (deletemin)
The object with the
minimum key value is
guaranteed to be the
root. Once you take it
out, you must reorder
the tree. You replace
the
root with the last object
and let it trickle
down.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Binary heap (deletemin)

•
Binary heap (decreasekey)
When you decrease a
key, you may have to
adjust the heap by
having the decreased
𝑂!
key object bubble up.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
𝑂!
ONE COMPLICATION
• How do we know where v is in the binary heap?

• Keep a supplemental array (an address book) indexed by

v, and keep pointers in both directions between this array
and binary heap elements.
Binary heap (decreasekey)

•
Binary heap
• deletemin: 𝑂 log 𝑉
• decreasekey: 𝑂 log 𝑉

• Dijkstra’s takes time:

• deletemin ×|𝑉| + decreasekey ×|𝐸|
• 𝑂(log |𝑉|)×|𝑉| + 𝑂(log |𝑉|)×|𝐸| =
𝑂 𝑉 + 𝐸 log 𝑉
Array vs Binary Heap
•
ADVANTAGES OF FLEXIBILITY
• By working with a higher level version of the algorithm, we
can understand what’s going on without getting caught up
in details
• Flexibility allows us to fit data structures to be efficient for
the given circumstance
Optimality proofs for shortest path
reductions
• If you have an algorithm that reduces to a shortest path
algorithm (Dijkstra’s or BFS), then often the proof will have two
parts (path correspondence) and (optimality)
• (Path correspondence): This is the main part of the proof. In
this part of the proof, you establish that the types of objects you
are trying to minimize in the original problem correspond to the
paths in the graph you built. (usually a bidirectional argument.)
• (Optimality): If you have done the path correspondence part
then optimality follows easily:
“There can’t be anything better because Dijkstra’s would have
found a better path….”