Graph drawing in spectral layout∗

Maureen Gallagher Colleen Tygh John Urschel Ludmil Zikatanov

Beginning: July 18, 2013; Today is: October 12, 2013

1 Introduction
Our research focuses on the use of spectral graph drawing techniques (see [5] and [3]) and other
mathematical tools to create a two-dimensional (2D) representation of the graph of a sparse matrix.
The actual graph drawing then amounts to displaying a particular arrangement of the vertices and
edges of a given graph in two dimensions. We use a spectral layout that provides coordinates using
the second and third eigenvectors of the associated graph Laplacian. To compute the eigenvectors
and eigenvalues, we implement a subspace correction method proposed in [4] and further analyzed
in [1].

1.1 Some graph drawing applications

Several important applications of graph drawing emerge from the analysis of networks, including
those evident in social media sites. In particular, drawing graphs of networks can aid the following
tasks:

• Exploration of data: Displaying nodes and ties in various layouts and attributing color, size
and other properties to nodes allows for the better visualization and analysis of data.

• Analyzing collaboration graphs:

– Vertices represent participants or people.

– Edges join two participants if a collaborative relationship exists between them.
– Edge weights denote positive and negative relationships.

• Analysis of sociograms: Digraphs that diagram the structures and patterns of group interac-
tions facilitate the study of choices or preferences within a group.

With the quick and recent expansion of social networking, graph drawing is gaining a great deal of
attention. Some of the currently available graph-drawing software includes the following:

• Wolfram Alpha: This online service can connect to Facebook and make charts of friend
connections and relationships.

• Weighted Spectral Distribution: This MATLAB tool applies to undirected graphs.

∗
These notes are based on the 2013 summer research as a part of the Computational Mathematics for Un-
dergraduate Students in the Department of Mathematics, Penn State, University Park, PA, 16802, USA; see
http://sites.psu.edu/cmus2013/ for details

1
 • LTAS Technologies, Inc.: Among many other applications, this web identity search tool for
Facebook finds the degree of separation between any two Facebook users and displays it in a
graph.

2 Graph Theory: Preliminaries

We consider a general combinatorial graph G = (V, E), where

• V is the set of vertices: a set of discrete objects (but usually isomorphic to {1, . . . , n}).

• E is the set of edges: ordered or unordered pairs of vertices.

We will consider two different types of graphs: undirected graphs and directed graphs. In an
undirected graph for an edge (c, m), we always have (c, m) = (m, c), while in a directed graph
(c, m) 6= (m, c). In addition, multigraphs allow multiple edges between vertices and loop graphs
contain an edge that connects a vertex to itself. If the graph is weighted, one may associate a
weight with every edge of a graph. Finally, a simple graph is an undirected graph that contains no
loops or parallel edges. We will focus on weighted and unweighted simple graphs.

2.1 Characteristics of Graphs

.
All graphs have the following characteristics:

• order: V (G) = |V | (the number of vertices)

• size: E(G) = |E| (the number of edges)

• degree of a vertex: the number of edges incident to the vertex (gives an idea of how directly
connected it is)

• connectedness: A graph is connected if for any two vertices v1 , v2 ∈ V there exists a path from
v1 to v2 ; a path being a sequence of vertices v1 = w1 , w2 . . . , wk = v2 such that (wj−1 , wj ) ∈ E,
for all j = 2, . . . , k.

• partition of a graph: splitting V in two disjoint subsets, namely, V = V1 ∪V2 with V1 ∩V2 = ∅.
A graph is connected if for every partition there exists an edge connecting the two sets of
vertices.

2.2 Matrices and graphs

We will now introduce several matrix representations of graphs, such as the adjacency and graph
Laplacian matrices. In this and the following sections, we shall consider simple, undirected graphs
(possibly weighted).

• Adjacency matrix A ∈ IRn×n , n = |V |, where Aij equals the number of edges between vertex
i and vertex j

• graph Laplacian matrix L = D − A, where D is the degree matrix of the graph (D =

diag(d(v1 ), d(v2 ), . . . , d(vn )) and A is the adjacency matrix of the graph. Here, d(vk ) for a
vertex vk ∈ V is the number of edges incident with vk .

2
 Homework Show that for a simple undirected graph, Lij = −1 for i 6= j, and Lii = d(i) (the
degree of vertex i).

3 Graph drawing and eigenvalues

The graph Laplacian L of a simple graph is a symmetric matrix. If σ(L) is the set of eigenvalues
of the symmetric matrix L, we can conclude that all eigenvalues of L are real, that is, σ(L) ⊂ IR.
Moreover, since the row and column sums of a graph Laplacian are zero, we immediately obtain
that L1 = 0, where 1 = (1, . . . , 1)T (the constant vector). In other words, 0 ∈ σ(A).
| {z }
n
On IRn we have an inner (or scalar) product
n
X
(u, v) = u1 v1 + u2 v2 + . . . , +un vn = ui vi . (1)
j=1

Using the inner product, with every matrix, such as L, we associate a bilinear form:
n X
X n
(Lu, v) = Lij uj vi , for all u ∈ IRn , v ∈ IRn .
i=1 j=1

The fact that L1 = 0 implies that

X
Lij = −Lii , for all i = 1, . . . , n.
j6=i

The sum on the left side of the above identity is over all j ∈ {1, . . . , n} such that j 6= i.

Homework Given that L1 = 0, show that the above bilinear form can be written as:
n n
1 XX
(Lu, v) = (−Lij )(ui − uj )(vi − vj ).
2
i=1 j=1

Further show that if all Lij ≤ 0, then (Lu, u) ≥ 0 for all u ∈ IRn . Such an L is called symmetric
positive semidefinite.

The extreme eigenvalues of L will be extreme values of the Rayleigh quotient if

σ(L) = {λ1 , λ2 , . . . , λn }, λ1 ≤ λ2 ≤ . . . , ≤ λn .
(Lv, v) (Lv, v)
λ1 = minn , λn = maxn .
v∈IR (v, v) v∈IR (v, v)
As we checked in the Homework, (Lv, v) ≥ 0 and (L1, 1) = 0. We conclude that λ1 = 0 and
the corresponding eigenvector is 1. For a connected, undirected graph with positive edge weights
it can be shown that 0 is a simple eigenvalue, namely, 0 = λ1 < λ2 ≤ λ2 . . . ≤ λn .
Furthermore, it can be shown that if (λ2 , v2 ) and (λ2 , v3 ) are the eigenvalue and eigenvector
pairs of L, then
(Lv, v) (Lv, v)
λ2 = min , λ3 = min , (2)
v∈V2 (v, v) v∈V3 (v, v)

3
 where V1 is the subspace of vectors orthogonal to 1, and V2 is the subspace of vectors orthogonal
to both 1 and v2 . More precisely,

V1 = v ∈ IRn , (v, 1) = 0 ,
 

V2 = v ∈ IRn , (v, 1) = 0, (v, v2 ) = 0 .

 

Homework Show this by proving the min-max principle (you will find this in [2]).

The spectral layout graph drawing uses v2 and v3 as coordinates of the vertices to display the
graph in two dimensions. In other words, the vertex i will have coordinates (v1,i , v2,i ) ⊂ IR2 , and
an edge is drawn between i and j if (i, j) ∈ E.
Thus, to utilize a spectral layout drawing, the problem is to find (λ2 , v2 ) and (λ3 , v3 ) which
solve
Lv2 = λ2 v2 , Lv3 = λ3 v3 , (v2 , 1) = (v3 , 1) = 0, (v2 , v3 ) = 0.
(Lv, v)
This is equivalent to minimizing the Rayleigh quotient, written as λ = min .
(x,1)=0 (v, v)
We would like to emphasize that instead of the scalar product given in (1), in all of the Rayleigh
quotients we use (Dv, v) instead of (v, v). In such a case we will be solving the so-called generalized
eigenvalue problem

Lv2 = λ2 Dv2 , Lv2 = λ2 Dv2 , (Dv2 , 1) = (Dv3 , 1) = 0, (Dv2 , v3 ) = 0. (3)

Indeed, it is easy to verify that all considerations below are valid with this scalar product.

Homework Using the definition of scalar product in [2], show that

n
X
(Dv, w) = d(i)vi wi
i=1

is indeed a dot product between v and w. For n = 2, (two dimensions) draw the unit sphere, or the
tips of all vectors v ∈ IR2 satisfying (Dv, v) = 1 if: (a) d(1) = d(2) = 1; (b) d(1) = 1 and d(2) = 10.

3.1 Subspace correction method for eigenvalue problem

We consider the subspace correction method for eigenvalue problems (see [4], [1]). We begin by
splitting IRn into the sum of subspaces,

IRn = W1 + W2 + · · · + Wnc .

For each of the subspaces Wk we assume

Wk = Range(Pk ), Pk ∈ IRn×nk , nk = dim Wk .

The algorithm visits each subspace and improves the current eigenvalue approximation from
each of these subspaces. To explain this in more detail, let y be a given approximation to the
eigenvector v2 , with (y, 1) = 0. We want to improve the approximation y by finding y1 ∈ W1 such
that
(L(y + z), (y + z))
y1 = arg min . (4)
z∈W1 (y + z, y + z)

4
The notation arg min means that y1 is the argument z which minimizes the right side of the equation
above. We now show how we can find y1 by solving a (ni + 1) × (ni + 1) problem. We first compute
the numerator on the right hand side of (4):

(L(y + z), (y + z)) = (Ly + Lz, y + z) = (Ly, y) + (Ly, z) + (Lz, y) + (Lz, z)

We compute the denominator in a similar fashion. Denoting now

(Ly, y) + 2(Ly, z) + (Lz, z)
µ = min ,
z∈W1 (y, y) + 2(y, z) + (z, z)
we note (as we also noted before) that y1 ∈ W1 is the minimizer (i.e. the z for which the minimum
is achieved), namely
(Ly, y) + 2(Ly, y1 ) + (Ly1 , y1 ) (Ly, y) + 2(Ly, z) + (Lz, z)
= min .
(y, y) + 2(y, y1 ) + (y1 , y1 ) z∈W1 (y, y) + 2(y, z) + (z, z)
We now try to write the right-hand side of the equation above as a Rayleigh quotient for some
matrix. Recall that W1 = Range(P1 ), and let us denote
n1 columns 1 column
z}|{ z}|{
Pe1 = [ P1 y ]} n rows, n1 = dim W1 .
| {z }
(n1 +1)columns

A direct computation then shows that

 T  T  T
P1 LP1 P1T Ly

T e P1   P1  
L1 = P1 LP1 = T L P1 y = T LP1 Ly = T
e e
y y y LP1 y T Ly
Since z ∈ Range(P1 ), it follows that a dz ∈ IRn1 exists such that

z = P1 dz
|{z} |{z} |{z}
IRn n
IRn×n1 IR 1

Thus, minimizing with respect to z ∈ W1 is the same as finding dz ∈ IRn1 . We then have
(Ly, y) + 2(Ly, z) + (Lz, z)
µ = min
z∈W1 (y, y) + 2(y, z) + (z, z)
(Ly, y) + 2(Ly, P1 dz ) + (LP1 dz , P1 dz )
= e minn
dz ∈IR 1 (y, y) + 2(y, P1 dz ) + (P1 dz , P1 dz )
(Ly, y) + 2(P1T Ly, dz ) + (P1T LP1 dz , dz )
= minnz .
dz ∈IR (y, y) + 2(P1T y, dz ) + (P1T P1 dz , dz )
Introducing the matrix
T 
   T Ty

P 1
 P 1 P 1 P1
Ie1 = T P1 y = T ,
y y P1 y T y
and the vector  
d
dz = 1
e (n1 + 1),
1
it is straightforward to check the following identities:

(Le 1 dez , dez ) = (Ly, y) + 2(P1T Ly, dz ) + (P1T LP1 dz , dz )

(Ie1 dez , dez ) = (y, y) + 2(P1T y, dz ) + (P1T P1 dz , dz ).

5
Consequently, we have that
(L
e 1 dez , dez )
µ = min .
dez (Ie1 dez , dez )
Hence, the minimizer of the above Rayleigh quotient, de1 , is a generalized eigenvector of the gener-
alized eigenvalue problem
L
e 1 de1 = µIe1 de1 . (5)
So to find y1 , we need to solve the (n1 + 1) × (n1 + 1) eigenvalue problem (5).

4 Implementation of the subspace correction method

In this section, we outline the algorithm for computing the eigenvectors. We first split the set of
vertices of the graph into nc non-overlapping subsets {Ωk }nk−1
c
, namely,

{1, . . . , n} = ∪nk=1
c
Ωk , Ωk ∩ Ωj = ∅, whenever k 6= j.

We use a simple “greedy” algorithm to construct such a splitting.

Algorithm 1: Greedy vertex splitting

Set nc = 0;
for i = 1 : n do
if i and all its neighbors have not been visited then
set nc = nc + 1;
label with nc the subgraph whose vertices are i and its neighbors;
mark i and all its neighbors as visited
if at least one neighbor of i has been visited then
continue the loop over the vertices
for isolated vertices do
add each such vertex to a neighboring aggregate with minimal number of vertices

Next, we formalize the subspace correction algorithm as follows. In our computations, we use a
modification that can simultaneously compute nλ eigenvalues. In the tests we have done, we always
set nλ = 3. The number of eigenvalues we find on each subspace is nm = (nλ + 2), because the first
nλ eigenvalues of the subspace problem are more accurately computed.
We find the three smallest eigenvalues of Lv = λDv (0 = λ1 , λ2 , and λ3 ). Here, D is the degree
matrix of the graph, L = D − A is the Laplacian matrix of the graph, and Y is the approximation
of the eigenvectors, which we orthogonalize and normalize. We obtain the eigenvalues of the graph,
and we use the corresponding eigenvectors to draw a representation of the graph. We also consider
subspaces Wi = Range(Pi ) subordinate to the partitioning provided by the clusters Ωi . More
precisely, let
Ωi = {j1 , . . . , jni } ⊂ {1, . . . , n}.
Then Pi ∈ IRn×ni is defined as 
1, k = `
(Pi )j` ,k
0, otherwise.
The algorithm below takes an initial guess Y ∈ IRn×nλ and, as output, provides a new and
improved approximation.

6
 Algorithm 2: Schwartz-based method
Y ∈ IRn×nλ is the current approximation to the first nλ eigenvectors
for i = 1 : nc do
set Pei = [Pi , Y ]
compute L e i = PeT LPei
i
compute Iei = PeiT DPei
let (λk , uk ), k = 1, . . . , nλ , 0 = λ1 ≤ . . . ≤ λnλ be the eigenvector, eigenvalue pair of the
eigenvalue problem L e k uk = λk Iei uk
Set Y = Pei [u1 , . . . , unλ ] (Y is now the new approximation to the eigenvectors)

Figure 1: The red drawing depicts the clusters, the black graph depicts the coarse graph and blue
graph is our regular graph drawing.

4.1 Adding a coarse “grid” correction

A coarser grid (coarser graph) is a simpler representation of a graph in which the vertices are the
clusters {ωk }nk=1
c
. We consider two clusters connected if and only if a member of one cluster is
connected to a member of the other.
A coarse grid correction provides faster and more accurate results because such a correction
is ”global,” and vertices which could be “far” from each other are simultaneously corrected. In
general, the algorithm can be run without changes by introducing a coarse space Wc = Range(P ),
dim Wc = nc , and the k-th column of P equalling 1 at the vertices from Ωk and 0 otherwise. Clearly,
P 1nc = 1n , where 1nc = (1, . . . , 1)T . We can now form
| {z }
nc

Pe = [P, Y ], e = PeT LPe,

L Ie = PeT DPe.

However, if we attempt to solve in this subspace in the same way as we did before, it will not work;
Y contains column 1, and we also have, as we mentioned earlier, P 1nc = 1n . Note that

Lx = λDx (6)

is well defined if D is invertible. If D is not invertible, D has a non-trivial null space and the
eigenvalue problem (6) does not make sense in general. If D is invertible, then clearly any solution
to (6) is a solution to the eigenvalue problem D−1 Lx = λx, and everything is well defined.

7
 These observations are summarized as follows:
     
1 ∗ ∗ 1 1
1 ∗ ∗ 1 1
Pe = [P, Y ], Y =  . . ..  , P . = . .
     
 .. .. .  ..   .. 
1 ∗ ∗ 1 1

We then have (with e1 = [1, 0, 0]T ):

     
1   1 1
1 1 1 1
Pe[1nc , −e1 ] = [P, Y ] [1nc , −e1 ] = P  .  − Y 0 =  .  −  .  = 0.
     
 ..   ..   .. 
0
1 1 1

We thus immediately conclude that Pe is not full rank, and, therefore, PeT DPe is singular, and the
eigenvalue problem on this subspace is not well posed (this is only due to the matrices we use). In
order to eliminate this redundancy, we use the two columns of Y only when we do the coarse grid
correction. In short, we only use the last two columns of Y to form Pe. More precisely, we have,

if Y = [y1 , y2 , y3 ], then set Z = [y2 , y3 ] and Pe = [P, Z].

Another remark we would like to make here is that instead of the P which we described above,
we also use in the algorithm the so-called “smoothed aggregation” P , which is defined as

P ← (I − D−1 L)P.

The results in the next section are obtained using this P for the coarse grid correction.

5 Numerical tests
In our tests, we used several adjacency matrices from The University of Florida Sparse Matrix
Collection http://www.cise.ufl.edu/research/sparse/matrices/.

Figure 2: grid1 dual (left) and delaunay n13 (right)

8
 Iterations Iterations
Figure Graph Name (no coarse grid) (coarse grid) Vertices Edges
2 grid1 dual 127 7 224 420
3 delaunay n13 |> 200 | 5 8192 24547
4 barth5 |> 200 | 5 15606 45878
5 big dual |> 200 | 5 30269 44929
6 Alemdar 194 7 6240 18168
7 fe 4elt2 |> 200 | 5 11143 32818

Table 1: Comparison of a single level method (without coarse grid correction) and two level method
(with coarse grid correction).

We ran our subspace correction algorithm both with and without the coarse grid correction.
A comparison of the number of iterations can be seen in Table 1, and the corresponding graph
drawings obtained using the two-level method (with coarse grid correction) are shown in Figures 2-
4.

Figure 3: barth5 (left) and big dual (right)

Figure 4: Alemdar (left) fe 4elt2 (right)

9
References
[1] Tony F. Chan and Ilya Sharapov, Subspace correction multi-level methods for elliptic eigenvalue
problems, Numer. Linear Algebra Appl. 9 (2002), no. 1, 1–20. MR 1874780 (2002k:65180)

[2] Paul R. Halmos, Finite-dimensional vector spaces, second ed., Springer-Verlag, New York, 1974,
Undergraduate Texts in Mathematics. MR 0409503 (53 #13258)

[3] David Harel and Yehuda Koren, A fast multi-scale method for drawing large graphs, J. Graph
Algorithms Appl. 6 (2002), no. 3, 179–202 (electronic), 2000 Symposium on Graph Drawing
(Williamsburg, VA). MR 1993519 (2004f:68118)

[4] M. S. Kaschiev, An iterative method for minimization of the Rayleigh-Ritz functional, Com-
putational processes and systems, No. 6 (Russian), “Nauka”, Moscow, 1988, pp. 160–170. MR
982053 (90f:65054)

[5] Yehuda Koren, On spectral graph drawing, Computing and combinatorics, Lecture Notes in
Comput. Sci., vol. 2697, Springer, Berlin, 2003, pp. 496–508. MR 2063526

10