Design and Analysis of Algorithms

CSE 5311
Lecture 18 Graph Algorithm

Junzhou Huang, Ph.D.

Department of Computer Science and Engineering

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 1

 Graphs

• Graph G = (V, E)
– V = set of vertices
– E = set of edges ⊆ (V×V)
• Types of graphs
– Undirected: edge (u, v) = (v, u); for all v, (v, v) ∉ E (No self loops.)
– Directed: (u, v) is edge from u to v, denoted as u → v. Self loops are
allowed.
– Weighted: each edge has an associated weight, given by a weight
function w : E → R.
– Dense: |E| ≈ |V|2.
– Sparse: |E| << |V|2.
• |E| = O(|V|2)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 2

 Graphs

• If (u, v) ∈ E, then vertex v is adjacent to vertex u.

• Adjacency relationship is:
– Symmetric if G is undirected.
– Not necessarily so if G is directed.
• If G is connected:
– There is a path between every pair of vertices.
– |E| ≥ |V| – 1.
– Furthermore, if |E| = |V| – 1, then G is a tree.

• Other definitions in Appendix B (B.4 and B.5) as needed.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 3

 Representation of Graphs

• Two standard ways.

– Adjacency Lists.
a b a b d c

b a c

c d c d a b

d a c

– Adjacency Matrix.

1 2 1 2 3 4
a b
1 0 1 1 1
2 1 0 1 0
c d 3 1 1 0 1
3 4 4 1 0 1 0
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 4
 Adjacency Lists

• Consists of an array Adj of |V| lists.

• One list per vertex.
• For u ∈ V, Adj[u] consists of all vertices adjacent to u.
a b a b d c

b c

d
If weighted, store weights
c d c
also in adjacency lists.
d

a b a b d c

b a c

c d c d a b

d a c
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 5
 Storage Requirement

• For directed graphs:

– Sum of lengths of all adj. lists is
∑out-degree(v) = |E|
v∈V No. of edges leaving v

– Total storage: Θ(V+E)

• For undirected graphs:
– Sum of lengths of all adj. lists is
∑degree(v) = 2|E| No. of edges incident on v. Edge (u,v) is
v∈V
incident on vertices u and v.
– Total storage: Θ(V+E)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 6

 Pros and Cons: adj list

• Pros
– Space-efficient, when a graph is sparse.
– Can be modified to support many graph variants.
• Cons
– Determining if an edge (u,v) ∈G is not efficient.

Have to search in u’s adjacency list. Θ(degree(u)) time.

Θ(V) in the worst case.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 7

 Adjacency Matrix

• |V| × |V| matrix A.

• Number vertices from 1 to |V| in some arbitrary manner.
• A is then given by: A[i, j ] = aij = 1 if (i, j ) ∈ E
1 2 0 otherwise
a b 1 2 3 4
1 0 1 1 1
2 0 0 1 0
c d4 3 0 0 0 1
3 4 0 0 0 0
1 2 1 2 3 4
a b
1 0 1 1 1
2 1 0 1 0 A = AT for undirected graphs.
c d 3 1 1 0 1
3 4 4 1 0 1 0

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 8

 Space and Time

• Space: Θ(V2).
– Not memory efficient for large graphs.
• Time: to list all vertices adjacent to u: Θ(V).
• Time: to determine if (u, v) ∈ E: Θ(1).
• Can store weights instead of bits for weighted graph.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 9

 Graph-searching Algorithms

• Searching a graph:
– Systematically follow the edges of a graph
to visit the vertices of the graph.
• Used to discover the structure of a graph.
• Standard graph-searching algorithms.
– Breadth-first Search (BFS).
– Depth-first Search (DFS).

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 10

 Breadth-first Search
• Input: Graph G = (V, E), either directed or undirected,
and source vertex s ∈ V.
• Output:
– d[v] = distance (smallest # of edges, or shortest path) from s to v,
for all v ∈ V. d[v] = ∞ if v is not reachable from s.
– π[v] = u such that (u, v) is last edge on shortest path s v.

u is v’s predecessor.
– Builds breadth-first tree with root s that contains all reachable
vertices.
Definitions:
Path between vertices u and v: Sequence of vertices (v1, v2, …, vk) such that u=v1
and v =vk, and (vi,vi+1) ∈ E, for all 1≤ i ≤ k-1. Error!
Length of the path: Number of edges in the path.
Path is simple if no vertex is repeated.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 11
 Breadth-first Search

• Expands the frontier between discovered and undiscovered

vertices uniformly across the breadth of the frontier.
– A vertex is “discovered” the first time it is encountered during the
search.
– A vertex is “finished” if all vertices adjacent to it have been
discovered.
• Colors the vertices to keep track of progress.
– White – Undiscovered.
– Gray – Discovered but not finished.
– Black – Finished.

Colors are required only to reason about the algorithm. Can be
implemented without colors.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 12

 BFS for Shortest Paths

2 3 3
2
2 3
1 S
S 2 S
1 1
2
2 3 3

Finished Discovered Undiscovered

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 13

 BFS(G,s)
1. for each vertex u in V[G] – {s}
2 do color[u] ← white
3 d[u] ← ∝
white: undiscovered
4 π[u] ← nil
gray: discovered
5 color[s] ← gray black: finished
6 d[s] ← 0
7 π[s] ← nil
8 Q←Φ Q: a queue of discovered
vertices
9 enqueue(Q,s) color[v]: color of v
10 while Q ≠ Φ d[v]: distance from s to v
11 do u ← dequeue(Q) π[u]: predecessor of v
12 for each v in Adj[u]
13 do if color[v] = white
14 then color[v] ← gray Example: animation.
15 d[v] ← d[u] + 1
16 π[v] ← u
17 enqueue(Q,v)
18 color[u] ← black

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 14

 Example (BFS)
(Courtesy of Prof. Jim Anderson)

r s t u
∞ 0 ∞ ∞

∞ ∞ ∞ ∞
v w x y

Q: s
0

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 15

 Example (BFS)

r s t u
1 0 ∞ ∞

∞ 1 ∞ ∞
v w x y

Q: w r
1 1

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 16

 Example (BFS)

r s t u
1 0 2 ∞

∞ 1 2 ∞
v w x y

Q: r t x
1 2 2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 17

 Example (BFS)

r s t u
1 0 2 ∞

2 1 2 ∞
v w x y

Q: t x v
2 2 2

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 18

 Example (BFS)

r s t u
1 0 2 3

2 1 2 ∞
v w x y

Q: x v u
2 2 3

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 19

 Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: v u y
2 3 3

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 20

 Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: u y
3 3

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 21

 Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: y
3

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 22

 Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

Q: ∅

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 23

 Example (BFS)

r s t u
1 0 2 3

2 1 2 3
v w x y

BF Tree

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 24

 Analysis of BFS

• Initialization takes O(V).

• Traversal Loop
– After initialization, each vertex is enqueued and dequeued at most
once, and each operation takes O(1). So, total time for queuing is
O(V).
– The adjacency list of each vertex is scanned at most once. The sum
of lengths of all adjacency lists is Θ(E).
• Summing up over all vertices => total running time of BFS
is O(V+E), linear in the size of the adjacency list
representation of graph.
• Correctness Proof
– We omit for BFS and DFS.
– Will do for later algorithms.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 25

 Breadth-first Tree

• For a graph G = (V, E) with source s, the predecessor

subgraph of G is Gπ = (Vπ , Eπ) where
– Vπ ={v∈V : π[v] ≠ NIL}U{s}
– Eπ ={(π[v],v)∈E : v ∈ Vπ - {s}}
• The predecessor subgraph Gπ is a breadth-first tree if:
– Vπ consists of the vertices reachable from s and
– for all v∈Vπ , there is a unique simple path from s to v in Gπ that
is also a shortest path from s to v in G.
• The edges in Eπ are called tree edges.
|Eπ | = |Vπ | - 1.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 26

 Depth-first Search (DFS)

• Explore edges out of the most recently discovered vertex

v.
• When all edges of v have been explored, backtrack to
explore other edges leaving the vertex from which v was
discovered (its predecessor).
• “Search as deep as possible first.”
• Continue until all vertices reachable from the original
source are discovered.
• If any undiscovered vertices remain, then one of them is
chosen as a new source and search is repeated from that
source.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 27
 Depth-first Search

• Input: G = (V, E), directed or undirected. No source

vertex given!
• Output:
– 2 timestamps on each vertex. Integers between 1 and 2|V|.

d[v] = discovery time (v turns from white to gray)

f [v] = finishing time (v turns from gray to black)
– π[v] : predecessor of v = u, such that v was discovered during the
scan of u’s adjacency list.
• Uses the same coloring scheme for vertices as BFS.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 28

 Pseudo-code

DFS(G) DFS-Visit(u)
1. for each vertex u ∈ V[G] 1. color[u] ← GRAY ∇ White vertex u
2. do color[u] ← white has been discovered
3. π[u] ← NIL 2. time ← time + 1
4. time ← 0 3. d[u] ← time
5. for each vertex u ∈ V[G] 4. for each v ∈ Adj[u]
6. do if color[u] = white 5. do if color[v] = WHITE
7. then DFS-Visit(u) 6. then π[v] ← u
7. DFS-Visit(v)
8. color[u] ← BLACK ∇ Blacken u;;
Uses a global timestamp time. it is finished.
9. f[u] ← time ← time + 1

Example: animation.
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 29
 Example (DFS)
(Courtesy of Prof. Jim Anderson)

u v w
1/

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 30

 Example (DFS)

u v w
1/ 2/

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 31

 Example (DFS)

u v w
1/ 2/

3/
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 32

 Example (DFS)

u v w
1/ 2/

4/ 3/
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 33

 Example (DFS)

u v w
1/ 2/
B

4/ 3/
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 34

 Example (DFS)

u v w
1/ 2/
B

4/5 3/
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 35

 Example (DFS)

u v w
1/ 2/
B

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 36

 Example (DFS)

u v w
1/ 2/7
B

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 37

 Example (DFS)

u v w
1/ 2/7

F B

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 38

 Example (DFS)

u v w
1/8 2/7

F B

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 39

 Example (DFS)

u v w
1/8 2/7 9/

F B

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 40

 Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6
x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 41

 Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 42

 Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/ B

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 43

 Example (DFS)

u v w
1/8 2/7 9/

F B C

4/5 3/6 10/11 B

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 44

 Example (DFS)

u v w
1/8 2/7 9/12

F B C

4/5 3/6 10/11 B

x y z

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 45

 Analysis of DFS

• Loops on lines 1-2 & 5-7 take Θ(V) time, excluding time
to execute DFS-Visit.

• DFS-Visit is called once for each white vertex v∈V when

it’s painted gray the first time. Lines 3-6 of DFS-Visit is
executed |Adj[v]| times. The total cost of executing DFS-
Visit is ∑v∈V|Adj[v]| = Θ(E)

• Total running time of DFS is Θ(V+E).

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 46

 Parenthesis Theorem

Theorem 22.7
For all u, v, exactly one of the following holds:
1. d[u] < f [u] < d[v] < f [v] or d[v] < f [v] < d[u] < f [u] and neither u nor v
is a descendant of the other.
2. d[u] < d[v] < f [v] < f [u] and v is a descendant of u.
3. d[v] < d[u] < f [u] < f [v] and u is a descendant of v.

So d[u] < d[v] < f [u] < f [v] cannot happen.

Like parentheses:
 OK: ( ) [ ] ( [ ] ) [ ( ) ]
 Not OK: ( [ ) ] [ ( ] )
Corollary
v is a proper descendant of u if and only if d[u] < d[v] < f [v] < f [u].
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 47
 Example (Parenthesis Theorem)

y z s t
3/6 2/9 1/10 11/16

B F C B

4/5 7/8 12/13

C 14/15
C C
x w v u

(s (z (y (x x) y) (w w) z) s) (t (v v) (u u) t)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 48

 Depth-First Trees

• Predecessor subgraph defined slightly different from that of

BFS.
• The predecessor subgraph of DFS is Gπ = (V, Eπ) where
Eπ ={(π[v],v) : v ∈ V and π[v] ≠ NIL}.
– How does it differ from that of BFS?
– The predecessor subgraph Gπ forms a depth-first forest composed
of several depth-first trees. The edges in Eπ are called tree edges.

Definition:
Forest: An acyclic graph G that may be disconnected.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 49

 White-path Theorem

Theorem 22.9
v is a descendant of u if and only if at time d[u], there is a path
u v consisting of only white vertices. (Except for u, which
was just colored gray.)

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 50

 Classification of Edges

• Tree edge: in the depth-first forest. Found by exploring (u, v).

• Back edge: (u, v), where u is a descendant of v (in the depth-first
tree).
• Forward edge: (u, v), where v is a descendant of u, but not a tree
edge.
• Cross edge: any other edge. Can go between vertices in same depth-
first tree or in different depth-first trees.

Theorem:
In DFS of an undirected graph, we get only tree and back edges. No
forward or cross edges.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 51

 Identification of Edges

• Edge type for edge (u, v) can be identified when it is first explored by
DFS.
• Identification is based on the color of v.
– White – tree edge.
– Gray – back edge.
– Black – forward or cross edge.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 52

 Directed Acyclic Graph

• DAG – Directed graph with no cycles.

• Good for modeling processes and structures that have a
partial order:
– a > b and b > c ⇒ a > c.
– But may have a and b such that neither a > b nor b > a.
• Can always make a total order (either a > b or b > a for all
a ≠ b) from a partial order.

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 53

 Example

DAG of dependencies for putting on goalie equipment.

socks shorts T-shirt batting glove

hose chest pad

pants sweater

skates mask

leg pads catch glove

blocker
Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 54
 Characterizing a DAG

Lemma 22.11
A directed graph G is acyclic iff a DFS of G yields no back edges.

Proof:
• ⇒: Show that back edge ⇒ cycle.
– Suppose there is a back edge (u, v). Then v is ancestor of u in
depth-first forest.
– Therefore, there is a path v u, so v u v is a cycle.

T T T
v u
B

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 55

 Characterizing a DAG

Lemma 22.11
A directed graph G is acyclic iff a DFS of G yields no back edges.
Proof (Contd.):
• ⇐ : Show that a cycle implies a back edge.
– c : cycle in G, v : first vertex discovered in c, (u, v) : preceding edge
in c.
– At time d[v], vertices of c form a white path v u. Why?
– By white-path theorem, u is a descendent of v in depth-first forest.
– Therefore, (u, v) is a back edge. T T T
v u
B

Dept. CSE, UT Arlington CSE5311 Design and Analysis of Algorithms 56