COMPSCI 311 Introduction to Algorithms

Lecture 3: Asymptotic Complexity

Graphs and Breadth-First Search

Marius Minea and Ghazaleh Parvini

University of Massachusetts Amherst

13 February 2022

slides credit: Dan Sheldon, Akshay Krishnamurthy, Andrew McGregor

Review: Big-Θ

Definition: the function T (n) is Θ(f (n)) if there exist positive

constants c1 , c2 > 0 and n0 ≥ 0 such that
c1 f (n) ≤ T (n) ≤ c2 f (n) for all n ≥ n0

f is an asymptotically tight bound of T

Equivalent Definition: the function T (n) is Θ(f (n)) if it is both

O(f (n)) and Ω(f (n)).

Result (Θ from limit):

If limn→∞ f (n)/g(n) = c ∈ (0, ∞) then f (n) = Θ(g(n)).
f (n)
Proof: This means ∀ϵ > 0 ∃n0 ≥ 0 ∀n ≥ n0 : c − ϵ ≤ g(n) ≤ c + ϵ.
This implies the definition (pick any ϵ).
Big-Θ gives us a direct comparison

How do we compare the running time of these algorithms?

Algorithm bar
Algorithm foo for i = 1 to n do
for i = 1 to n do for j = 1 to n do
for j = 1 to n do for k = 1 to n do
constant-time op constant-time op
end for end for
end for end for
end for

Answer: foo is Θ(n2 ) and bar is Θ(n3 ).

They do not have the same asymptotic running time.
Big-Θ is an Equivalence Relation

▶ f is Θ(f )

▶ if f is Θ(g), then g is Θ(f )

f = O(g) =⇒ g = Ω(f ), f = Ω(g) =⇒ g = O(f )

▶ if f is Θ(g), and g is Θ(h), then f is Θ(f )

from transitivity of big-O and Ω
Additivity Revisited

Suppose f and g are two (non-negative) functions and f is O(g)

Old version: Then f + g is O(g)

New (stronger) version: Then f + g is Θ(g)

Example:
n2 + 42n + n log n is Θ(n2 )
|{z} | {z }
g f

For an algorithm that is a sequence of parts, the part with the

highest complexity determines (dominates) the running time.
Little-o/little-omega: Growing Much Slower/Faster

What about bounds which are not tight?

Def. f (n) = o(g(n)) if limn→∞ fg(n)
(n)
=0
read: f is dominated asymptotically by g
f grows much slower than g

If f (n) = o(g(n)), then f (n) = O(g(n)), and f (n) is not Ω(g(n)).

Examples:
n = o(n2 )
log n = o(n)
nk = o(2n )

Opposite: little-omega, f (n) = ω(g(n)) if g(n) = o(f (n))

equivalent, limn→∞ fg(n)
(n)
= ∞ (f grows much faster than g)
Running Time Analysis

Mathematical analysis of worst-case running time of an algorithm as

function of input size. Why these choices?
▶ Mathematical: describes the algorithm, not the code
Avoids hard-to-control experimental factors (CPU, programming
language, quality of implementation), while still being predictive.

▶ Worst-case: gives guarantees.

(“average case” appealing, but hard to analyze)

▶ Function of input size: allows predictions.

What will happen on a new input?
Efficiency

When is an algorithm efficient?

Stable Matching Brute force: Ω(n!)

Propose-and-Reject?: O(n2 )
We must have done something clever
Question: Is it Ω(n2 ) ?
Polynomial Time

Definition: an algorithm runs in polynomial time if its running time

is O(nd ) for some constant d

▶ Examples
These are polynomial time:
f1 (n) = n
Not polynomial time:
f2 (n) = 4n + 100
f7 (n) = 2n
f3 (n) = n log(n) + 2n + 20
f8 (n) = 3n
f4 (n) = 0.01n2
f9 (n) = n!
f5 (n) = n2
f6 (n) = 20n2 + 2n + 3
Why Polynomial Time ?

Why is this a good definition of efficiency?

▶ Matches practice: almost all practically efficient algorithms

have this property.

▶ Usually distinguishes a clever algorithm from a “brute force”

approach.

▶ Refutable: gives us a way of saying an algorithm is not efficient,

or that (likely) no efficient algorithm exists.

▶ Compositional: composing two polynomials yields a polynomial

(can use polynomial subroutines)
Exponential Time and Beyond

k
An algorithm is exponential time if it is O(2n ) for some k > 0
equivalently: O(2poly(n) ). Does the base 2 matter?

Other Running Times: How fast does n! grow?

Useful fact: (Stirling’s approximation)
√  n
n
n! ∼ 2πn (ratio is 1+ Θ(1/n))
e

Exercise: Does n! grow faster than exponential of any base?

Exercise: What can you claim from here for growth of log n! ?
Useful facts: Logarithm / Harmonic Series

Pn
H(n) = 1 + 1/2 + ... + 1/n = k=1 1/k

Fact: H(n) diverges. Easy proof: if 2j−1 < k ≤ 2j , 1/k ≥ 1/2j

H(2k ) ≥ 1 + 1/2 + (1/4 + 1/4) + ... + 2k−1 · 1/2k = 1 + k · 1/2.
Can do similar for upper bound: =⇒ H(n) = Θ(log n)

More precisely: ln n < H(n) < 1 + ln n.

Easy with integrals: (ln x)′ = 1/x, and 1/x decreasing
1 R n+1 1
=⇒ n+1 < n 1/x dx = ln(n + 1) − ln n < n
1
Can also write as n+1 < ln(1 + n1 ) < 1
n
Review: Asymptotics

Property Definition / terminology

f (n) is O(g(n)) ∃c, n0 s.t. f (n) ≤ cg(n) for all n ≥ n0
g is an asymptotic upper bound on f

f (n) is Ω(g(n)) ∃c, n0 s.t. f (n) ≥ cg(n) for all n ≥ n0

Equivalently: g(n) is O(f (n))
g is an asymptotic lower bound on f

f (n) is Θ(g(n)) f (n) is O(g(n)) and f (n) is Ω(g(n))

g is an asymptotically tight bound on f
Graphs Are Everywhere

▶ Transportation networks:
hubs, links, routes
shortest paths
▶ Communication networks:
routing, hops,
latency/throughput?
connectivity, spanning trees

▶ Information networks:
WWW, what are important/authoritative pages?

▶ Social networks: study interaction dynamics, find influencers?

How do we build algorithms to answer these questions?

One week of Enron emails

slide credit: Kevin Wayne / Pearson

 key variable of interest was an alter’s obesity at t + 1 to the foregoing models. We also analyzed
time t + 1. A significant coefficient for this vari- the role of geographic distance between egos
Framingham heart study
able would suggest either that an alter’s weight
affected an ego’s weight or that an ego and an
and alters by adding such a variable.
We calculated 95% confidence intervals by sim-
alter experienced contemporaneous events affect- ulating the first difference in the alter’s contem-

Figure 1. Largest Connected Subcomponent of the Social Network in the Framingham Heart Study in the Year 2000.
Each circle (node) represents one person in the data set. There are 2200 persons in this subcomponent of the social
network. Circles with red borders denote women, and circles with blue borders denote men. The size of each circle
is proportional to the person’s body-mass index. The interior color of the circles indicates the person’s obesity status:
yellow denotes an obese person (body-mass index, ≥30) and green denotes a nonobese person. The colors of the
ties between the nodes indicate the relationship between them: purple denotes a friendship or marital tie and orange
denotes a familial tie.

“The Spread of Obesity in a Large Social Network over 32 Years” by Christakis and Fowler in New England Journal of Medicine, 2007 6

n engl j med 357;4 www.nejm.org july 26, 2007 373

slide credit: Kevin Wayne / Pearson
Political blogosphere graph

Node = political blog; edge = link.

The Political Blogosphere and the 2004 U.S. Election: Divided They Blog, Adamic and Glance, 2005
Figure 1: Community structure of political blogs (expanded set), shown using utilizing a GEM
layout [11] in the GUESS[3] visualization and analysis tool. The colors reflect political orientation, 37

red for conservative, and blue for liberal. Orange links go from liberal to conservative, and purple
slide credit: Kevin Wayne / Pearson
More applications

▶ Network science
▶ random graphs: various evolution models
▶ scale-free, small world

▶ Analyzing graph evolution in time

▶ fake news
▶ botnets

▶ Analyzing programs
▶ control flow graph, function call graph
▶ state space search (also in games):
compute reachable states (configurations)
is an error state reachable?
Graphs

A graph is a mathematical representation of a network

▶ Set of nodes (vertices) V
▶ Set of pairs of nodes (edges) E (a relation)
Graph G = (V, E)
Notation: n = |V |, m = |E| (almost always used)
Example: Internet in 1970
2.2. PATHS AND CONNECTIVITY 25
Definitions: Edge, Path
Edge e = {u, v} — but usually written e = (u, v)
u and v are neighbors, adjacent, endpoints of e
26 CHAPTER 2. GRAPHS
e is incident to u and v

SRI UTAH LINC

MIT CASE

UCSB STAN SDC

BBN CARN

UCLA RAND HARV

Figure
A path2.3: is
Anaalternate
sequencedrawing
P =ofvthe
1 , v13-node Internet
2, . . . , v k−1 , vkgraph
suchfrom December
that each 1970.
consecutive pair vi , vi+1 is joined by an edge in G
Paths. Although
Called: pathwe’ve
“frombeen
v1discussing
to vk ”. examples
Or: a v1of–vgraphs
k pathin many di↵erent areas, there
are clearly some common themes in the use of graphs across these areas. Perhaps foremost
Special
among these iscase: empty
the idea that path
things from any vacross
often travel to itself (justofnode,
the edges nomoving
a graph, edges)from
Simple Path, Distance, Cycle

▶ Simple path: path where all vertices are distinct

▶ Exercise. Prove: If there is a path from u to v then there is a
simple path from u to v.

Different terms elsewhere: walk: arbitrary (called path here)

trail: no repeated edges; path (here: simple path)

▶ Distance from u to v:
minimum number of edges in a u–v path

▶ (Simple) Cycle: path v1 , . . . , vk−1 , vk where

▶ v1 = vk
▶ First k − 1 nodes distinct
▶ All edges distinct
Connected Components

Connected component: maximal subset of nodes such that a path

exists between each pair in the set
▶ maximal = if a new node is added to the set, there will no
longer be a path between each pair
1 4 5 6

2 3

Node set: {1, 2, 4}: there is a path between any two nodes
But it is not maximal: can add 3, property still holds
{1, 2, 3, 4} is maximal: can’t add any other node
Trees

Tree = a connected graph with no cycles

Q: Is this equivalent to trees seen in Data Structures?
A: More or less. No root identified.

Tree properties
Let G be an undirected graph with n nodes.
Then any two of the following statements imply the third:
▶ G is connected
▶ G does not contain a cycle
▶ G has n − 1 edges

Rooted tree: tree with parent-child relationship

▶ Pick root r and “orient” all edges away from root
▶ Parent of v = predecessor on path from r to v
Directed Graphs

▶ Directed graph G = (V, E)

▶ Directed edge e = (u, v) is now an ordered pair
▶ e leaves u (source) and enters v (sink)

1 4 5

2 3

▶ Directed path, cycle: same as before, but with directed edges

▶ Strongly connected: directed graph with directed path
between every pair of vertices
▶ Note: graphs undirected if not otherwise specified
Graph Traversal

Thought experiment. World social graph.

▶ Is it connected?
▶ If not, how big is largest connected component?
▶ Is there a path between you and <some famous person>?
“Six degrees of separation” (everyone connected in at most 6 links?)
Erdös number: coauthorship of scientific papers
How can you tell algorithmically?
Answer: graph traversal! (BFS/DFS)
Breadth-First Search
Explore
2.3. outward from
DISTANCE AND starting node
BREADTH-FIRST by distance. “Expanding
SEARCH 33 wave”

you

distance 1 your friends

distance 2 friends of friends

distance 3 friends of friends

of friends

all nodes, not already discovered, that have an

edge to some node in the previous layer
Breadth-First Search: Layers

Explore outward from starting node s.

Define layer Li = all nodes at distance exactly i from s.

Layers
▶ L0 = {s}
▶ L1 = nodes with edge to L0
▶ L2 = nodes with an edge to L1 that don’t belong to L0 or L1
▶ ...
▶ Li+1 = nodes with an edge to Li that don’t belong to any
earlier layer.
Observation:
There is a path from s to t if and only if t appears in some layer.
BFS Implementation

BFS(s):
mark s as "discovered"
L[0] ← {s}, i ← 0 ▷ Discover s
while L[i] is not empty do
L[i + 1] ← empty list
for all nodes v in L[i] do
for all neighbors w of v do ▷ Explore v
if w is not marked "discovered" then
mark w as "discovered" ▷ Discover w
put w in L[i + 1]
end if
end for
end for
i←i+1
end while

Running time? How many times does each line execute?

BFS Running Time
BFS(s):
mark s as "discovered" ▷1
L[0] ← {s}, i ← 0 ▷1
while L[i] is not empty do
L[i + 1] ← empty list ▷≤n
for all nodes v in L[i] do ▷n
for all neighbors w of v do ▷ 2m
if w is not marked "discovered" then ▷ 2m
mark w as "discovered" ▷n
put w in L[i + 1] ▷n
end if
end for
end for
i←i+1 ▷≤n
end while

Running time: O(m + n). Assumption: can efficiently iterate over

neighbors of v. OK with adjacency list.
Exploring all Connected Components

How to explore entire graph even if it is disconnected?

while there is some unexplored node s do

BFS(s) ▷ Run BFS starting from s.
Extract connected component containing s
end while

Usually OK to assume graph is connected.

State if you are doing so and why it does not trivialize the problem.

Running time? Does it change? No, still O(m + n)

BFS Tree
Tree edges = edges that reach a node for the first time.
BFS(s):
mark s as "discovered"
L[0] ← {s}, i ← 0
T ← empty
while L[i] is not empty do
L[i + 1] ← empty list
for all nodes v in L[i] do
for all neighbors w of v do
if w is not marked "discovered" then
mark w as "discovered"
put w in L[i + 1]
put (v, w) in T ▷ tree edge
end if
end for
end for
i←i+1
end while
BFS Tree

blue: “tree edges”, dashed: “non-tree edges”

MIT

UTAH BBN LINC

HARV CASE
SRI SDC RAND

UCSB STAN UCLA CARN

Claim: let T be the tree discovered by BFS on graph G = (V, E),

and let (x, y) be any edge of G. Then the layer of x and y in T
differ by at most 1.
BFS and Non-tree Edges

Claim: let T be the tree discovered by BFS on graph G = (V, E),

and let (x, y) be any edge of G. Then the layer of x and y in T
differ by at most 1.
Proof
▶ Let (x, y) be an edge
▶ Assume x is discovered first and placed in Li
▶ Then y ∈ Lj for j ≥ i
▶ When neighbors of x are explored, y is either already in Li , or
is discovered and added to Li+1