CSE 521: Design and Analysis of Algorithms I Spring 2016

Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality

Lecturer: Shayan Oveis Gharan May 4th Scribe: Gabriel Cadamuro

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.

12.1 Overview of spectral graph theory

Spectral graph theory seeks to relate the eigenvectors and eigenvalues of matrices corresponding to a Graph
to the combinatorial properties of the graph. While generally the theorems are designed for unweighted and
undirected graphs they can be extended to the weighted graph case and (less commonly) the directed graph
case. In these notes we only study undirected graphs.
Given a graph G we use A to denote adjacency matrix of the graph and D to denote the diagonal degree
matrix of the vertex degrees. Recall these are defined as

(
if i ∼ j
wi,j
Ai,j =
0 otherwise,
( P
dw (i) = j wij if i = j
Di,j =
0 otherwise.

If G is unweighted, wij is 1, when there is an edge between vertex i and j and 0 otherwise. Generally in
spectral graph theory we work with a normalization of the adjacency matrix of G, known as graph Laplacian.
The Laplacian of G denoted by L is defined as follows:

L=D−A

The main reason for this normalization choice is that the quadratic form of a Laplacian can be nicely written
as an energy function. For any graph laplacian L and x ∈ Rn

X
xT Lx = (xi − xj )2 wij (12.1)
i∼j

We will see shortly that this formulation can be used to relate eigenvalues of L, A to the combinatorial
properties of G. Let us justify the above identity. Consider a graph G with only two vertices 1, 2 and an
edge (1, 2). In this case      
0 1 1 0 1 −1
A= D= L=
1 0 0 1 −1 1
Therefore,   
1 −1 x1
xT Lx = x1 , x2 = x21 − x1 x2 − x1 x2 + x22 = (x1 − x2 )2
 
−1 1 x2

To prove (12.1), let Lij be the laplacian of the graph with all the vertices in G but only the edge between i
to j. We note then that

12-1
12-2 Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality

 
X X X
xT Lx = xT  Lij  x = xT Lij x = (xi − xj )2
i∼j i∼j i∼j

12.2 Properties of the graph laplacian

One can use the Laplacian of G to state several properties of G. For example, det(L + 11T ) is equal to n
times the number of spanning trees of G. In this section we only describe and prove basic properties of the
Laplacian. We also use λ1 ≤ λ2 ≤ · · · ≤ λn to denote the eigenvalues of L.

Prop 1: For any graph G, L  0. This can be seen by looking at its quadratic form. For any vector
x ∈ Rn ,
X
xT Lx = (xi − xj )2 ≥ 0
i∼j

Prop 2: For any graph G, λ1 = 0. This essentially follows from the variational characterization of eigen-
values. Recall that,
X
λ1 = min xT Lx = min (xi − xj )2 .
kxk=1 kxk=1
i∼j

Since i∼j (xi −xj )2 ≥ 0,, to show λ1 = 0, it is enough to construct a vectors x such that i∼j (xi −xj )2 = 0.
P P
For that matter it is enough to let x = 1. This argument also shows that the all-1s vector is always the first
eigenvector of Laplacian of any graph.

Prop 3: For any graph G, λ2 = 0 iff G is disconnected. We prove this statement in two steps: (i) we first
prove that if λ2 = 0, then G is disconnected, (ii) in the second step, we show that if G is disconnected, then
λ2 = 0.
2
P
(i) Assume λ2 = 0. Let x be the eigenvector corresponding to λ2 . Since λ2 = 0, i∼j (xi − xj ) = 0.
Therefore, for all i ∼ j, we have xi = xj . But this means that any two vertices i, j which are in the same
connected component of G we have xi = xj since equality is transitive. So to show that G has two connected
components it is enough to show that x assigns two distinct values to the vertices of G.
On the other hand, since x is the second eigenvector, we have x ⊥ 1. Therefore, there is i, j such that
xi 6= xj as desired.
(ii) Now, assume that G is disconnected, and we show that λ2 = 0. Let S ⊆ V be a disconnected set of
vertices in G and S be its complement. We construct the vector x as follows:
( 1
|S| if i ∈ S
xi 1
− |S| otherwise.

Since x assigns the same number to the vertices of each connected component of G, for all i ∼ j, we have
xi = xj . So,
X
xT Lx = (xi − xj )2 = 0.
i∼j
Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality 12-3

1 1
Moreover we note that hx, 1i = |S|( |S| ) − |S|( |S| ) = 0. Therefore,

λ2 = min y T Ly = xT Lx = 0.
y⊥1,kyk=1

The third property can be naturally extended to any k. In particular, for any graph G, λk = 0 if and only
if G has at least k connected components. We leave this as exercise.

12.3 Normalized Laplacian

1 1
Let us define the normalized adjacency matrix as à = D− 2 AD− 2 . That since D is a diagonal matrix that
taking the power can be done entrywise and we can simply understand the entires of these matrices as

−1 1
Di,i2 = p
dw (i)
wij
Ãij = p
dw (i)dw (j)

Similarly, we can define the normalized Laplacian matrix

1 1 1 1
L̃ = D− 2 LD− 2 = I − D− 2 AD− 2 = I − Ã.
The above matrix is closely related to the normalized adjacency matrix, if λ is an eigenvalue of L̃, 1 − λ is
an eigenvalue of Ã. This is because

λx = L̃x = (I − Ã)x = x − Ãx ⇒ Ãx = (1 − λ)x.

The normalized adjacency matrix (and the normalized Laplacian matrix) are closely related to the random
walk matrix of G. The matrix P = D−1 A is the transition probability matrix of the simple random walk on
G. It is not hard to see that any eigenvalue of à is also an eigenvalue of P . We will see more details in the
future lectures.
For simplicity of the notation, from now on, we assume that G is d-regular; namely that each vertex in
the graph has precisely d edges incident to it. All the following results work for non-regular (weighted)
graphs but this assumption considerably simplifies the notation. In this case, the normalized laplacian is
just L̃ = Ld = I − A/d..
L
Another reason for working with the normalized Laplacian matrix is that the eigenvalues of d are in the
range
0 ≤ λ1 . . . λn ≤ 2,
L
in other words the eigenvalues are scale free. Note that since L̃ = L/d, any eigenvalue λ of d corresponds
to an eigenvalue dλ of L.

12.4 Cheeger’s inequality

From now on assume that

0 = λ1 ≤ · · · ≤ λn ≤ 2
12-4 Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality

are the eigenvalues of L/d. The following theorem is one of the fundamental results in spectral graph
theory with many applications in complexity theory, coding theory, analysis of random walks, approximation
algorithms, etc.

Theorem 12.1 (Cheeger’s inequality [AM85; Alo86]). For any graph G with normalized Laplacian L̃ and
eigenvalues 0 = λ1 ≤ · · · ≤ λn ≤ 2,
λ2 p
≤ φ(G) ≤ 2λ2
2

To better understand the above example let us consider extreme cases.

Example 1: Disconnected graphs By property 3 of the Laplacian matrix, a graph G is disconnected if

and only if the second smallest eigenvalue of the (normalized) Laplacian matrix is 0. If G is disconnected,
then φ(G) = 0 and λ2 = 0 so the above theorem is tight. Conversely, if λ2 = 0, then G is disconnected and
φ(G) = 0. So, the above theorem can be seen as a robust version of this basic fact in spectral graph theory,
that is λ2 ≈ 0 if and only if G is almost disconnected.
Next, we present an example where the right side of Cheeger’s inequality is tight.

2
Example 1: The cycle graph We recall from the previous notes (Fact 11.3) that φ(G) = n when G is
a cycle: we will show that in this case λ2 . n12 . Consider the following vector x,
(
1
2 − 2in if i ≤ n/2
xi =
− 21 + 2i
n otherwise.

This means that for each pair of adjacent vertices i, i + 1

4
(xi − xi+1 )2 = .
n2
Since x ⊥ 1, we can write,
T
xT Lx n · (4/n2 )
  
x L x
λ2 ≤ ≤ 2
= = O(1/n2 ),
kxk d kxk d · kxk d · (n/2) · (1/4)2

where in last inequality we used that |xi | ≥ 1/4 for at least half of the vertices.
The above calculations show that (up to a constant factor) the right side of Cheeger’s inequality is tight for
a cycle. The left side of the Cheeger’s inequality is tight for the d-dimensional hypercube {0, 1}d . In this
case φ(Hd ) = 1/d and λ2 = Θ(1/d). We leave the details of the calculations as an exercise.
We now examine a different class of graphs in which this inequality is important.

Definition 12.2. An expander graph is one in which the 2nd eigenvalue of the normalized laplacian is lower
bounded by an absolute constant λ2 ≥ Ω(1)

The simplest example of an expander graph is the complete graph on n vertices. In this case it is not hard
to see that λ2 = 1. For an infinite sequence of graphs the above definitions mean that λ2 remain bounded
away from zero as the size of the graph grows. For a single graph it might be more intuitive to think of it as
having λ2 > c for some constant c like 41 . Sparse expander graphs are important in many fields of computer
science including optimization, coding theory and complexity theory. See PS5 for many examples.
Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality 12-5

Though specific construction of sparse expander graphs are hard to describe we can prove that a random
d-regular graph is an expander graph with high probability. In other words, although (sparse) expander
graphs are hard to construct explicitly, most of the graphs are expanders.
From a very highlevel point of view, expander graphs can be see as ”sparse complete graphs”; for example
if G is a d-regular expander, then for all S ⊆ V ,
d
|EG (S, S)| ≈ |EKn (S, S)|
n
where Kn is the complete graph over n vertices. Expander graphs have very small diameter and fast mixing
time of random walks.
We now prove the left side of Cheeger’s inequality.

Lemma 12.3. Let L/d be the normalized Laplacian of a d-regular graph G, and λ2 be the second smallest
eigenvalue of L/d. Then
λ2
≤ φ(G)
4

Proof. For a vector x ∈ Rn let

− xj )2
P
xT L x xT Lx i∼j (xi
R(x) = Td = =
x2i
P
x x dxT x d· i

By variational characterization we can write,

− xj )2
P
i∼j (xi
λ2 = min R(x) = min P . (12.2)
x⊥1 x⊥1 d· x2i

Fix a set S ⊆ V such that vol(S) ≤ vol(V )/2. Next, we show that the conductance of a set S ⊆ V is nothing
but the Rayleigh quotient of the indicator vector of that set. Recall that 1S is the vector where 1Si = 1 if
i ∈ S and 0 otherwise. Then,
S S
P
|E(S, S)| |E(S, S)| i∼j |1i − 1j |
φ(S) = = = P S ,
vol(S) d|S| d i 1i

where in the last equality we used that |1Si − 1Sj | = 1 if and only if i, j lie on opposite side of the (S, S) cut.
Now, observe that for all i, 1Si = (1Si )2 and for all i, j

|1Si − 1Sj | = (1Si − 1Sj )2 .

since the values can only take 0 or 1. Therefore,

|1Si − 1Sj |2
P
i∼j
φ(S) = P S 2 = R(1S ).
d i (1i )

Using the above identity, (12.2) seems to imply that λ2 ≤ φ(S) which should directly conclude the proof.
The catch is that 1S is not orthogonal to 1, the first eigenvector of L/d. This minor issue can be resolved
by projecting 1S to the space orthogonal to 1.
Consider the vector
1 1 |S|
y = 1S − h1S , i = 1S − .
k1k k1k n
12-6 Lecture 12: Introduction to Spectral Graph Theory, Cheeger’s inequality

By construction, we have 1 ⊥ y. In addition, the numerator of the Rayleigh quotient is shift-invariant, i.e.,
for any c,
|xi − xj |2 = |(xi + c) − (xj + c)|2 ,
so |1Si − 1Sj |2 = |yi − yj |2 . To prove the lemma we show that kyk2 ≥ k1S k2 /4. Then, by (12.2)

|1Si − 1Sj |2 S S 2
P P
i∼j i∼j |1i − 1j |
λ2 ≤ R(y) = ≤ = 4φ(S) ≤ 4φ(G).
d · i yi2 d
P P S 2
4 · i (1i )

which completes the proof.

So, it remains to show that kyk2 ≥ k1S k2 . Note that we still have not used the assumption that vol(S) ≤
vol(V )/2, or equivalently that |S| ≤ |V |/2. This implies that |S| ≤ n/2. Therefore,
X X X
kyk2 = yi2 ≥ yi2 = (1 − |S|/n)2 ≥ |S|/4 = k1S k/4.
i i∈S i∈S

This completes the proof of Lemma 12.3.

The right side of Cheeger’s inequality has a more involved proof: please see the ”reading” section of the
course page.

12.5 Relation to spectral partitioning

The above proof shows why the spectral partitioning algorithm generally does well when it comes to finding
φ(G). Here is the highlevel idea: The second eigenvector of L/d is the vector x which minimizes R(x) among
all real-valued vectors (orthogonal to 1). On the other hand, as we saw in the proof of Lemma 12.3, φ(G)
is the {0, 1}n vector y which minimizes R(y). Therefore, the second eigenvalue of the normalized Laplacian
matrix approximates the conductance of G by formulating it as a continuous optimization problem. The
spectral partitioning algorithm can be seen as a rounding of this continuous optimization problem. Given a
real values vector x ∈ Rn , we would like to construct a vector y ∈ {0, 1}n such that R(y) ≈ R(x). Having
such a y, the support of y, i.e., {i : yi = 1} would correspond to a low conductance cut. The spectral
partitioning algorithm uses the most natural rounding idea: Choose a threshold t and let yi = 0 if xi < t and
let yi = 1 otherwise. The algorithm chooses the best possible threshold by going over all possible thresholds.
The following lemma which we state without a proof gives a bound on the performance of the spectral
partitioning algorithm:

Lemma 12.4. For any graph G, the spectral partitioning algorithm returns a set S such that vol(S) ≤
vol(V )/2 and p
φ(S) ≤ 2λ2 ,
where as usual λ2 is the second smallest eigenvalue of the normalized Laplacian matrix L̃.
p
Since by Lemma 12.3, φ(G) ≥ λ2 /2, φ(S) ≤ O( φ(G)). The proof of the above lemma also implies the
right side of the Cheeger’s inequality.

A tight example for spectral partitioning. As we mentioned earlier, the right side of the Cheeger’s
inequality is tight for a cycle of length n. But, it is not hard to see that the spectral partitioning algorithm
returns the minimum conductance cut. So, a cycle is not a tight example for the spectral partitioning
algorithm.
REFERENCES 12-7

Let us conclude this lecture by giving a tight example for the spectral partitioning algorithm. Consider the
weighted ladder graph shown in the left of Figure 12.1. This graph consists of two cycles 1, 2, . . . , n/2 and
n/2, n/2 + 1, . . . , n.
In this case the minimum conductance cut is shown at the middle of Figure 12.1 for which φ(G) ≈ 100 n2 .
However, recall that the Rayleigh quotient of the second eigenvector of a cycle of length n is no more than
20/n2 . Therefore, the second eigenvector of the ladder graph is just two copies of the second eigenvector of a
cycle of length n/2; in particular the endpoints of each dashed has the same value in the second eigenvector.
It follows that the spectral partitioning algorithm cannot distinguish the inner cycle from the outer one,
because the second eigenvector assigns the same number to the endpoints of each dashed edge. The cut
at the right of Figure 12.1 shows the output of the spectral partitioning algorithm; in this case, although
φ(G) = Θ(1/n2 ), the spectral partitioning algorithm returns a set S with φ(S) = Θ(1/n).

Figure 12.1: An illustration of the Ladder graph. Here each solid edge has weight 1 and each dashed edge
has weight 100/n2 where n/2 is the length of the cycles. The middle figure shows the set with the optimum
conductance for a sufficiently large n, and the right figure shows the output of the spectral partitioning
algorithm.

In the next lecture we briefly discuss an extension of the spectral partitioning algorithm based on the first
k eigenvectors of the normalized Laplacian matrix.

References

[Alo86] N. Alon. “Eigenvalues and expanders”. In: Combinatorica 6 (2 Jan. 1986), pp. 83–96 (cit. on
p. 12-4).
[AM85] N. Alon and V. Milman. “Isoperimetric inequalities for graphs, and superconcentrators”. In: Journal
of Combinatorial Theory, Series B 38.1 (Feb. 1985), pp. 73–88 (cit. on p. 12-4).