36-781 Lecture:

Limits of (dense) graph sequences

Lecturer: Cosma Shalizi

Scribe: Nicolás Kim
1 November 2016

Contents
1 Warm-up: convergence in distribution 1
1.1 Starting case: empirical distirbution on k categories . . . . . . . 1
1.2 Convergence of distributions on R1 . . . . . . . . . . . . . . . . . 1
1.3 Alternative view of convergence in distribution . . . . . . . . . . 3
1.4 Lessons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Convergence of sequences of graphs 5

2.1 Some symbols and notation . . . . . . . . . . . . . . . . . . . . . 7
2.2 Injective homomorphisms . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Homomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1 Warm-up: convergence in distribution

1.1 Starting case: empirical distirbution on k categories
Pk
Represent a distribution on k categories by a vector pn ∈ Rk≥0 , where i=1 pn =
1. To be more precise, then, pn lives in the k-simplex Sk :
So, pn is any point on the k-simplex with rational coefficients. In other words,
each rational point on the k-simplex represents a distribution on k categories.
Figure 1 depicts the simplices for k = 2 and k = 3.
So, as n → ∞, pn → ρ even if ρ has irrational coefficients. This is the notion
of convergence in distribution for categorical distributions.

1.2 Convergence of distributions on R1

Definition 1. At any finite n, define the empirical cdf to be the function
n
1X
pn (x) = 1(xi ≤ x).
n i=1

1
Figure 1: S2 and S3 are the simplices corresponding to the range of possible
probability distributions on 2 and 3 categories, respectively. Each point in a
simplex is a probability distribution, since each point is a vector whose entries
sum to 1.

2
Figure 2: The empirical cdf (orange) is a step function which converges to the
true cdf (blue) as the sample size increases.

So, at any finite n, pn (x) takes jumps of size 1/n at (at most) n distinct
points. See Figure 2 for an example.
iid
Theorem 1.1. (Glivenko-Cantelli) If Xi ∼ ρ = true cdf, then
a.s.
max |pn (x) − ρ(x)| → 0.
x

The Glivenko-Cantelli thoerem is also known as the fundamental theorem of

statistics, since it essentially claims that we can use a sample from a distribution
to learn that distribution.
Note that, similarly to the case of convergence of categorical distributions,
ρ isn’t necessarily a step function, even though pn will be for any finite n. The
possible cdf’s that these step functions may converge to constitutes a broader
class of functions than just step functions.

1.3 Alternative view of convergence in distribution

Definition 2. Say we have a sequence of probability distributions (µ1 , µ2 , . . . , µn , . . . )
with corresponding pdf ’s (m1 , m2 , . . . , mn , . . . ). These converge on a limit µ
with pdf m, if: Z Z
n→∞
f (x)mn (x) dx −→ f (x)m(x) dx,
R R

3
for all bounded and continuous f (such f are known as “test functions”). If
d
this convergence holds, then we say that µn → µ, i.e. that µn converges in
distribution to µ.
How do we apply this notion to the convergence of empirical distributions?
In particular, the empirical distribution is a jump function, meaning it does not
have a pdf in the standard sense. However, we can consider the empirical pdf
to be a mixture of Dirac delta functions, as follows: put µn to be the empirical
distribution from n samples. Naturally, we let µ denote the distribution that
all data were drawn from. Define the pdf of µn to be
n
1X
δ(x − xi ),
n i=1

where δ(x) is the Dirac delta function 1 ,

(
0, x 6= 0,
δ(x) =
+∞, x = 0,

so that Z
f (x)δ(x) dx = f (0)
R
for any f : R → R.
Now, putting mn to be this mixture of Dirac delta functions, we get
Z n Z
1X
f (x)mn (x) dx = f (x)δ(x − xi ) dx (1)
R n i=1 R
n Z
1X
= f (y + xi )δ(y) dy (2)
n i=1 R
n
1X
= f (xi ). (3)
n i=1

d
So for µn → µ, we need
n
1X d
δ(x − xi ) → µ,
n i=1
which we now know means that
n Z
1X n→∞
f (xi ) −→ f (x) µ(dx),
n i=1 R

1 Alternatively, δ(x) can be thought of as

Z
lim f (x)φ(x; σ) dx,
σ→0 R

where φ(x) is the pdf of a random variable with distribution N (0, σ).

4
for all bounded and continuous test functions f . Since
n Z
1X
f (xi ) ∈ R, f (x) µ(dx) ∈ R,
n i=1 R

this is now just convergence in the sense of a sequence of real numbers.

1.4 Lessons
1. Observed data sets get represented as objects with lots of discreteness.
2. They tend towards continuous limit objects.

3. The discrete ones are special cases of the continuous limits.

4. Convergence in distribution is equivalent to convergence of averages/integrals/expectations
for bounded continuous test functions:
Z Z
d
µn → µ ⇐⇒ f (x) µn (dx) → f (x) µ(dx)
R R

for all bounded and continuous f .

2 Convergence of sequences of graphs

How can we apply the idea of convergence in distribution in order to define
convergence in graphs?
We have a sequence of graphs g1 , g2 , . . . , gm , . . . . Denote by V (gm ) and
E(gm ) the set of nodes and edges, respectively, for gm . In other words, E(gm ) ⊆
V (gm ) × V (gm ).
Fix a favorite graph f , which we will call a motif. For example, f may be
the triange graph K3 , or the four-cycle C4 . In general, a motif is any fixed,
finite graph.
Definition 3. An isomorphism between two graphs f and g is some bijective
function
→ V (g)
φ : V (f ) ,→
such that (i, j) ∈ E(f ) if and only if (φ(i), φ(j)) ∈ E(g). We will sometimes say
f ' g if f and g are isomorphic in this sense. An example is given in Figure 3.

Definition 4. The density of a k-node motif, f , in an n-node graph, g, is the

fraction of k-node induced subgraphs in g that are isomorphic to f (by definition,
the density is 0 if k ≥ n).
These motifs are going to be our test functions. Figure 4 demonstrates this
concept when the motif is a 2-star.

5
Figure 3: Graph isomorphism is determined by the graphs’ structure, not by
the label associated with each node.

Figure 4: Take three arbitrary nodes in g: do they have the same structure as
f ? Here, we identify two successful matchings between a triplet in g and f .

6
Figure 5: The subgraph induced by the three green nodes in g is isomorphic
to f . The same applies to the three pink nodes, and to a several other sets of
node triplets in g. The total number of such triplets that are isomophic to f is
denoted by Iso(f, g).

2.1 Some symbols and notation

Denote by Iso(f, g) the number of mappings from V (f ) to V (g) such that the
induced subgraph of g is isomorphic to f . Figure 5 demonstrates 2 of the several
such mappings from f to g.

Definition 5. The motif density of f in g is

Iso(f, g) Iso(f, g) Iso(f, g)

tiso (f, g) = = = n

.
# potential mappings from f to g n(n − 1) . . . (n − (k − 1)) k k!

This motif density is sometimes referred to as tind (f, g).

Definition 6. A sequence of graphs (g1 , g2 , . . . , gm , . . . ) converges (in the graph
sense) when tiso (f, gm ) converges as m → ∞ for every fixed motif f .2
One additional notation we will require is G[k] = the subgraph induced by
picking k distinct nodes from g at random. In this new notation, we can say

tiso (f, g) = P(f ' G[k]).

2 As a sanity check, if gm = g for every m, then the sequence still converges.

7
 Figure 6: There exists some f 0 ⊇ f such that f 0 ' G[k].

2.2 Injective homomorphisms

Unfortunately, tiso is very hard to calculate. So, define a weaker notion of
matching than tiso :
tinjective(f,g) = P(f ⊆ G[k]),
where f ⊆ G[k] means that E(f ) ⊆ E(G[k]); Figure 6 provides a small example
of this concept. Now, if we know all of the isomorphism densities tiso (f 0 , g), for
all f 0 , we can calculate tinjective (f, g). If f ⊆ G[k], then there is some f 0 with
the same edges as f (plus some more edges), with an isomorphism f 0 ' G[k].
So, X
tinjective (f, g) = tiso (f 0 , g).
f 0 :f ⊆f 0

Since this is a linear system of equations, we can invert to get tiso (f, g) as a linear
combination of tinjective (f, g). This imples that if we have all of the tinjective (f, g),
we can always calculate all of the tiso (f, g), and vice versa. Therefore, the iso-
morphism densities converge if and only if the injective homomorphism densities
converge.

2.3 Homomorphisms
We previously went from a strong notion of graph matching, graph isomorphism,
to a weaker notion, injective homomorphism. We showed that convergence of

8
Figure 7: In a non-injective mapping, two different nodes in the domain graph
f can be mapped to the same node in the image graph g.

one implies the other, so that we could get away with working with the simpler
notion of injective homomorphism.
Naturally, the next step is to define an even weaker notion of graph matching,
which will be easier to calculate as well. To that end, we remove the restriction
of injectivity (in the sense of injective functions mapping to unique elements in
the image), to obtain the graph homomorphism.
Definition 7. A homomorphism from f to g is a mapping φ : V (f ) → V (g)
such that if (i, j) ∈ E(f ), then (φ(i), φ(j)) ∈ E(g). Figure 7 shows an exam-
ple where f is injectively mapped to g, and another example where f is non-
injectively mapped to g.
The notion of graph homomorphism, in contrast to the injective homomor-
phism, is akin to sampling nodes of g with replacement, in the sense that we are
allowing φ to assign the same node in g to more than one node from f .
Just as we defined G[k] as the subgraph induced by picking k nodes without
replacement, we define G0 [k] to be the subgraph induced by picking k (not
necessarily distinct) nodes of g, i.e. with replacement. This leads to a new
notion of motif density:

Hom(f, g)
thom = = P(f ⊆ G0 [k]).
nk