Analysis of Algorithms

CSCI 315
 1. Graphs - Definition
Graphs = a set of nodes (vertices) with edges
(links) between them.
Notations:
• G = (V, E) - graph
• V = set of vertices çV ç= n
• E = set of edges çE ç= m

1 2 1 2 1 2

3 4 3 4 3 4

Directed Undirected Acyclic

graph graph graph
2
 2. Graphs - Applications
• Applications that involve not only a set of items, but also the
connections between them

Maps Schedules Computer networks

Hypertext Circuits
3
 3. Types of Graphs
• Directed or undirected 1 2 1 2
• Weighted or unweighted
• Connected or complete 3 4 3 4
– A graph is connected if there is a
Connected Not connected
path between every two vertices
• A bipartite graph is an
undirected graph G = (V, E) in 1 2
9
which V = V1 + V2 and there are 4
8
edges only between vertices in 3 6
V1 and V2 7
4

CS 477/677 - Lecture 18 4
 4. Graph Representation
1. Adjacency list
2. Adjacent Matrix

5
 4.1 Graph Representation
• Adjacency list representation of G = (V, E)
– An array of çV çlists, one for each vertex in V
– Each list Adj[u] contains all the vertices v such that
there is an edge between u and v
• Adj[u] contains the vertices adjacent to u (in arbitrary order)
– Can be used for both directed and undirected graphs
1 2 5 /
1 2 2 1 5 3 4 /

3 3 2 4
4 2 5 3 /
5 4
5 4 1 2
Undirected graph
6
 4.2 Graph Representation
• Adjacency matrix representation of G = (V, E)
– Assume vertices are numbered 1, 2, … çV ç
– The representation consists of a matrix A çV çx çV ç:
– aij = 1 if (i, j) Î E
0 otherwise
1 2 3 4 5

1 2 1 0 1 0 0 1 For undirected
3 2 1 0 1 1 1 graphs matrix A
3 0 1 0 1 0 is symmetric:
5 4
4 0 1 1 0 1 aij = aji
Undirected graph
5 1 1 0 1 0
A = AT

7
 5. Searching in a Graph
• Graph searching = systematically follow the
edges of the graph so as to visit the vertices of
the graph
• Two basic graph searching algorithms:
– Breadth-first search
– Depth-first search
– The difference between them is in the order in which
they explore the unvisited edges of the graph
• Graph algorithms are typically elaborations of
the basic graph-searching algorithms
8
 5.1 Breadth-First Search (BFS)
• Input:
– A graph G = (V, E) (directed or undirected)
– A source vertex s Î V
• Goal:
– Explore the edges of G to “discover” every vertex
reachable from s, taking the ones closest to s first
• Output:
– d[v] = distance (smallest # of edges) from s to v, for
all v Î V
– A “breadth-first tree” rooted at s that contains all
reachable vertices

9
 5.1 Breadth-First Search (cont.)
• Discover vertices in increasing order of distance
from the source s – search in breadth not depth
– Find all vertices at 1 edge from s, then all vertices at 2
edges from s, and so on

1 2

5 4
11

6
7
12
9

7
10
 5.1 Breadth-First Search (cont.)
source
• Keeping track of progress:
1 2
– Color each vertex in either white,
3
gray or black
5 4
– Initially, all vertices are white
1 2
– When being discovered a vertex
3
becomes gray
5 4
– After discovering all its adjacent
vertices the node becomes black
1 2
– Use FIFO queue Q to maintain the
3
set of gray vertices
5 4
11
 Breadth-First Tree
• BFS constructs a breadth-first tree
– Initially contains the root (source vertex s)
– When vertex v is discovered while scanning source
the adjacency list of a vertex u Þ vertex v
1 2
and edge (u, v) are added to the tree
3
– u is the predecessor (parent) of v in the
5 4
breadth-first tree
– A vertex is discovered only once Þ it has at
most one parent
12
 BFS Additional Data Structures
• G = (V, E) represented using adjacency lists
• color[u] – the color of the vertex for all u Î V source
d=1
• p[u] – predecessor of u p=1
1 2
– If u = s (root) or node u has not yet been
3
discovered Þ p[u] = NIL
d=2
5 4
• d[u] – the distance from the source s to p=2
d=1 d=2
vertex u p=1 p=5

• Use a FIFO queue Q to maintain the set of

gray vertices

13
 BFS(V, E, s)
r s t u
1. for each u Î V - {s}
2. do color[u] ¬ WHITE
3. d[u] ← ¥ v w x y
r s t u
4. p[u] = NIL ¥ ¥ ¥
5. color[s] ¬ GRAY
¥ ¥ ¥ ¥
6. d[s] ← 0 v w x y
r s t u
7. p[s] = NIL
¥ 0 ¥ ¥
8. Q ¬ Æ
¥ ¥ ¥ ¥
9. Q ← ENQUEUE(Q, s) v w x y
Q: s

14
 BFS(V, E, s)
r s t u
10. while Q ¹ Æ ¥ 0 ¥ ¥
Q: s
11. do u ← DEQUEUE(Q)
¥ ¥ ¥ ¥
12. for each v Î Adj[u] v w x y
r s t u
13. do if color[v] = WHITE ¥ 0 ¥ ¥
Q: w
14. then color[v] = GRAY
¥ 1 ¥ ¥
15. d[v] ← d[u] + 1 v w x y

16. p[v] = u r s t u
1 0 ¥ ¥
17. ENQUEUE(Q, v) Q: w, r
¥ ¥ ¥
18. color[u] ¬ BLACK 1
v w x y

15
 Example
r s t u r s t u r s t u
¥ 0 ¥ ¥ 1 0 ¥ ¥ 1 0 2 ¥

¥ ¥ ¥ ¥ ¥ 1 ¥ ¥ ¥ 1 2 ¥
v w x y v w x y v w x y
Q: s Q: w, r Q: r, t, x
r s t u r s t u r s t u
1 0 2 ¥ 1 0 2 3 1 0 2 3

2 1 2 ¥ 2 1 2 ¥ 2 1 2 3
v w x y v w x y v w x y
Q: t, x, v Q: x, v, u Q: v, u, y
r s t u r s t u r s t u
1 0 2 3 1 0 2 3 1 0 2 3

2 1 2 3 2 1 2 3 2 1 2 3
v w x y v w x y v w x y
Q: u, y Q: y Q: Æ 16
 Analysis of BFS
1. for each u Î V - {s}
2. do color[u] ¬ WHITE
O(V)
3. d[u] ← ¥
4. p[u] = NIL
5. color[s] ¬ GRAY
6. d[s] ← 0
7. p[s] = NIL Q(1)
8. Q ¬ Æ
9. Q ← ENQUEUE(Q, s)

17
 Analysis of BFS
10. while Q ¹ Æ
11. do u ← DEQUEUE(Q) Q(1)
12. for each v Î Adj[u] Scan Adj[u] for all vertices
in the graph
13. do if color[v] = WHITE • Each vertex is scanned only
14. then color[v] = GRAY once, when the vertex is
dequeued
15. d[v] ← d[u] + 1 • Sum of lengths of all
adjacency lists = Q(E)
16. p[v] = u • Scanning operations:
O(E)
17. ENQUEUE(Q, v) Q(1)
18. color[u] ¬ BLACK
• Total running time for BFS = O(V + E)
18
 Shortest Paths Property
• BFS finds the shortest-path distance from the
source vertex s Î V to each node in the graph
• Shortest-path distance = d(s, u)
– Minimum number of edges in any path from s to u

source
r s t u

1 0 2 3

2 1 2 3
v w x y

19
 5.2 Depth-First Search
• Input: 1 2
– G = (V, E) (No source vertex given!) 3

• Goal: 5 4

– Explore the edges of G to “discover” every vertex in V

starting at the most current visited node
– Search may be repeated from multiple sources
• Output:
– 2 timestamps on each vertex:
• d[v] = discovery time
• f[v] = finishing time (done with examining v’s adjacency list)
– Depth-first forest 20
 5.2 Depth-First Search
• Search “deeper” in the graph whenever
1 2
possible
3
• Edges are explored out of the most recently
discovered vertex v that still has unexplored 5 4

edges
• After all edges of v have been explored, the search
“backtracks” from the parent of v
• The process continues until all vertices reachable from the
original source have been discovered
• If undiscovered vertices remain, choose one of them as a
new source and repeat the search from that vertex
• DFS creates a “depth-first forest”
21
 DFS Additional Data Structures
• Global variable: time-step
– Incremented when nodes are discovered/finished
• color[u] – similar to BFS
– White before discovery, gray while processing and
black when finished processing
• p[u] – predecessor of u
• d[u], f[u] – discovery and finish times

1 ≤ d[u] < f [u] ≤ 2 |V|

WHITE GRAY BLACK

0 d[u] f[u] 2V

22
 DFS(V, E)
1. for each u Î V u v w

2. do color[u] ← WHITE
3. p[u] ← NIL x y z
4. time ← 0
5. for each u Î V
6. do if color[u] = WHITE
7. then DFS-VISIT(u)

• Every time DFS-VISIT(u) is called, u becomes the

root of a new tree in the depth-first forest

23
 DFS-VISIT(u)
1. color[u] ← GRAY u v w

2. time ← time+1
3. d[u] ← time
x y z
4. for each v Î Adj[u] time = 1
5. do if color[v] = WHITE u v w
1/
6. then p[v] ← u
7. DFS-VISIT(v) x y z
8. color[u] ← BLACK u v w

9. time ← time + 1
1/ 2/

10. f[u] ← time

x y z
24
 Example
u v w u v w u v w
1/ 1/ 2/ 1/ 2/

3/
x y z x y z x y z

u v w u v w u v w
1/ 2/ 1/ 2/ 1/ 2/
B B
4/ 3/ 4/ 3/ 4/5 3/
x y z x y z x y z

u v w u v w u v w
1/ 2/ 1/ 2/7 1/ 2/7
B B F B
4/5 3/6 4/5 3/6 4/5 3/6
x y z x y z x y z
25
 Example (cont.)
u v w u v w u v w
1/8 2/7 1/8 2/7 9/ 1/8 2/7 9/
C
F B F B F B

4/5 3/6 4/5 3/6 4/5 3/6

x y z x y z x y z

u v w u v w u v w
1/8 2/7 9/ 1/8 2/7 9/ 1/8 2/7 9/
C C C
F B F B F B
B B
4/5 3/6 10/ 4/5 3/6 10/ 4/5 3/6 10/11
x y z x y z x y z

u v w The results of DFS may depend on:

1/8 2/7
C
9/12 • The order in which nodes are explored
F B in procedure DFS
B
4/5 3/6 10/11 • The order in which the neighbors of a
x y z vertex are visited in DFS-VISIT
26
 Edge Classification
• Tree edge (reaches a WHITE u v w
1/
vertex):
– (u, v) is a tree edge if v was first
discovered by exploring edge (u, v) x y z

• Back edge (reaches a GRAY

u v w
vertex):
1/ 2/
– (u, v), connecting a vertex u to an B
ancestor v in a depth first tree 4/ 3/

– Self loops (in directed graphs) are x y z

also back edges

27
 Edge Classification
• Forward edge (reaches a BLACK u v w
vertex & d[u] < d[v]): 1/ 2/7
F B
– Non-tree edges (u, v) that connect a vertex 4/5 3/6
u to a descendant v in a depth first tree x y z

• Cross edge (reaches a BLACK vertex

& d[u] > d[v]): u v w
1/8 2/7 9/
– Can go between vertices in same depth-first F B
C

tree (as long as there is no ancestor / 4/5 3/6

descendant relation) or between different x y z
depth-first trees

28
 Analysis of DFS(V, E)
1. for each u Î V
2. do color[u] ← WHITE Q(V)
3. p[u] ← NIL
4. time ← 0
5. for each u Î V Q(V) – exclusive
6. do if color[u] = WHITE of time for
7. then DFS-VISIT(u) DFS-VISIT

29
 Analysis of DFS-VISIT(u)
1. color[u] ← GRAY DFS-VISIT is called exactly
2. time ← time+1 once for each vertex

3. d[u] ← time
4. for each v Î Adj[u]
5. do if color[v] = WHITE Each loop takes
6. then p[v] ← u |Adj[v]|
7. DFS-VISIT(v)
8. color[u] ← BLACK
9. time ← time + 1 Total: ΣvÎV |Adj[v]| + Q(V) = Q(V + E)
10. f[u] ← time Q(E)
30
 Properties of DFS
• u = p[v] Û DFS-VISIT(v) was called
during a search of u’s adjacency list
u v w
1/ 2/

• Vertex v is a descendant of vertex u 3/

x y z
in the depth first forest Û v is
discovered during the time in which
u is gray

31
 Parenthesis Theorem
y z s t
In any DFS of a graph G, for 3/6 2/9 1/10 11/16
all u, v, exactly one of the
4/5 7/8 12/13 14/15
following holds: x w v u
1. [d[u], f[u]] and [d[v], f[v]] are
s t
disjoint, and neither of u and v
is a descendant of the other z v u

2. [d[v], f[v]] is entirely within y w

[d[u], f[u]] and v is a
x
descendant of u
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
3. [d[u], f[u]] is entirely within (s (z (y (x x) y) (w w) z) s) (t (v u) (u u) t)

[d[v], f[v]] and u is a Well-formed expression: parenthesis are

descendant of v properly nested
32
 Other Properties of DFS
Corollary
Vertex v is a proper descendant of u u
1/8 2/7 9/12
C
Û d[u] < d[v] < f[v] < f[u] F B
B
4/5 3/6 10/11
v

Theorem (White-path Theorem)

In a depth-first forest of a graph G, vertex u
v is a descendant of u if and only if at time 1/ 2/

d[u], there is a path u ] v consisting of

only white vertices. v

33
 Topological Sort
Topological sort of a directed acyclic graph G =
(V, E): a linear order of vertices such that if there
exists an edge (u, v), then u appears before v in
the ordering.

• Directed acyclic graphs (DAGs)

– Used to represent precedence of events or processes
that have a partial order
a before b b before c What about
a before c
b before c a before c a and b?

Topological sort helps us establish a total order

34
 Topological Sort
undershorts socks

Topological sort:
pants shoes an ordering of vertices along a
horizontal line so that all directed
shirt
edges go from left to right.
belt
watch
tie

jacket

socks undershorts pants shoes watch shirt belt tie jacket

35
 Topological Sort
undershorts 11/ 16 17/18 socks TOPOLOGICAL-SORT(V, E)
1. Call DFS(V, E) to compute
12/15 finishing times f[v] for each
pants shoes 13/14
shirt 1/8 vertex v
6/7 belt 2. When each vertex is finished,
watch 9/10
insert it onto the front of a
tie 2/5
linked list
3. Return the linked list of
jacket 3/4
vertices

socks undershorts pants shoes watch shirt belt tie jacket

Running time: Q(V + E)

36
 Lemma
A directed graph is acyclic Û a DFS on G yields
no back edges.
v
Proof: (u, v)

“Þ”: acyclic Þ no back edge

u
– Assume back edge Þ prove cycle
– Assume there is a back edge (u, v)
Þv is an ancestor of u
Þ there is a path from v to u in G (v ] u)
Þ v ] u + the back edge (u, v) yield a cycle
37
 Lemma
A directed graph is acyclic Û a DFS on G yields
no back edges. v
Proof:
(u, v)
“Ü”: no back edge Þ acyclic
– Assume cycle Þ prove back edge u
– Suppose G contains cycle c
– Let v be the first vertex discovered in c, and (u, v) be
the preceding edge in c
– At time d[v], vertices of c form a white path v ] u
– u is descendant of v in depth-first forest (by white-path
theorem)
Þ (u, v) is a back edge

38
 Strongly Connected Components
Given directed graph G = (V, E):
A strongly connected component (SCC) of G
is a maximal set of vertices C Í V such that for
every pair of vertices u, v Î C, we have both
u ] v and v ] u.

39
 The Transpose of a Graph
• GT = transpose of G
– GT is G with all edges reversed
– GT = (V, ET), ET = {(u, v) : (v, u) Î E}
• If using adjacency lists: we can create GT in
Q(V + E) time

1 2 1 2

3 3

5 4 5 4

40
 Finding the SCC
• Observation: G and GT have the same SCC’s
– u and v are reachable from each other in G Û they are
reachable from each other in GT
• Idea for computing the SCC of a DAG G = (V, E):
– Make two depth first searches: one on G and one on GT

1 2 1 2

3 3

5 4 5 4

41
STRONGLY-CONNECTED-COMPONENTS(G)

1. call DFS(G) to compute finishing times f[u] for

each vertex u
2. compute GT
3. call DFS(GT), but in the main loop of DFS,
consider vertices in order of decreasing f[u]
(as computed in first DFS)
4. output the vertices in each tree of the depth-
first forest formed in second DFS as a separate
SCC
42
 Example
a b c d
DFS on the initial graph G
13/14 11/ 16 1/ 10 8/ 9
b e a c d g h f
16 15 14 10 9 7 6 4
12/15 3/ 4 2/ 7 5/ 6

e f g h

a b c d
DFS on GT:
• start at b: visit a, e
• start at c: visit d
• start at g: visit f
• start at h
e f g h

Strongly connected components: C1 = {a, b, e}, C2 = {c, d}, C3 = {f, g}, C4 = {h}

43