An Introduction To The Applications of Linear Algebra in Graph Theory
An Introduction To The Applications of Linear Algebra in Graph Theory
0 C OVER L ETTER
Dear Dusty, Math 22A TFs/CAs, and Math 22A readers,
We hope this cover letter finds you well! In this paper, we present a short introduction to
graph theory through the lens of linear algebra. Specifically, our goal is to give our Math 22A
audience a primer on how to interpret graphs in more abstract terms using only linear algebra
by proving theorems involving eigenvalues, matrices, and other concepts.
In terms of contributions, we worked together to tackle the proofs while writing other
sections independently. Jointly, we wrote up an introduction, decided on notation, talked
through proofs, and solved examples. Separately, Nishant researched the MST application and
drafted the proofs; Erin looked into other applications and created graphics.
After we received our peer review feedback, we wrote up a clearer guiding narrative (Charles
M.); expounded upon concepts like the spectral theorem, simple graphs, and matrices indices,
as well as added another illuminating example (Iris C.); fixed image citations (Lillian P.);
reworked the ordering of sections (Parita S.); adopted the typographical conventions for
definitions/remarks (Jamin L.); and fixed the typos our reviewers collectively uncovered.
Finally, we’d like to point out that one linear algebra concept we use minimally, trace,
might be unfamiliar to a Math 22A student. However, we are careful to construct a clear
definition of trace before applying it, and we hope our explanations add sufficient background
to understand the proofs that involve trace.
We are endlessly grateful to Prof. Grundmeier; the course TFs and CAs; our draft reviewers
Charles, Iris, Lillian, Parita, and Jamin; Albie the Cat; and you, the reader!
Sincerely,
Nishant Mishra and Erin Yuan
1
1 I NTRODUCTION AND M OTIVATION
Say you are an early 18th century traveler to Königsberg, a Prussian port city by the Baltic sea.
You hope to spend the day exploring the city, which is made of two landmasses connected by a
series of seven bridges. To do as much sightseeing as possible, you plan on crossing each of
the seven bridges, but with a catch: to save time, each bridge can only be crossed once and no
more than once. Is it possible?
Figure 1.1: The Seven Bridges of Königsberg [4]. Can you devise a way to cross all seven bridges
only once?
This problem, commonly referred to as The Seven Bridges of Königsberg, was eventually
proven rigorously by the mathematician Leonhard Euler in 1736 to have no solution [3]. His
reasoning laid the foundations for modern graph theory––the mathematical study of graphs,
or pairwise relations between objects––which has since branched out extensive applications
in a variety of contexts. Among the more powerful tools that mathematicians have used to
study this field is linear algebra, the branch of mathematics that seeks to examine linear sets
of equations.
While it may initially seem difficult to reconcile the concrete concept of a graph with the
abstract nature of linear algebra, studying graphs under this lens enables us to derive some
surprising results. To present an introduction to graph theory, we apply key ideas from linear
algebra, including matrix operations, eigenvalues, and the Spectral Theorem, to expand upon
our previous notions of the mathematical definition of a "graph," which has real-world uses in
disciplines ranging even beyond science.
Definition 1.1 (Graph, Vertex, and Edge). A graph is a structure containing elements in which
some pairs of elements are related in a certain way. We refer to these elements as vertices, and
2
any possible relations that might exist between vertices as edges [8].
Visually, vertices are represented by circles and edges are represented by lines connecting
any two circles. Note that graphs can be infinite or finite according to its number of vertices
or edges, although for our purposes we limit ourselves to only finite graphs with finite vertices
and edges.
Figure 1.2: A pictorial representation of a graph. Each of the six circles is a vertex. The eight
lines labeled e 1 , . . . , e 8 are edges. Note that not every pair of vertices has to have an
edge in between them (for example, there is no edge between circles 2 and 3). Also,
an edge can circle back to the same vertex (see e 4 ). Figure by authors.
Graphs have also proven their benefits in medically-oriented fields. In chemistry, graphs
are commonly used to generate molecular models, with vertices acting as atoms and edges as
bonds. Since many physical properties, like boiling point, are directly related to structure, such
graphical representations allow ease of calculations not only for larger collections of known
molecules but also for molecules that may be hard to create in a traditional laboratory [2]. The
significance of graphs in biology hits close to home due to the current challenges facing our
world: with vertices for regions and edges for migration paths, these graphs can trace disease
propagation to predict infectious spread. In molecular and cellular biology, graphs are used to
create "bio-networks" that tabulate protein interactions or transcription of genes in live cells
[7].
3
Even beyond traditional scientific disciplines, graphs serve to be surprisingly versatile in
portraying relationships. As an example, sociologists construct "social networks" with people
as vertices and degrees of familiarity as edges to study influence, friendship, or communica-
tion. Altogether, the flexibility of graph theory to adapt to multiple subjects demonstrates its
importance as an object of study.
Definition 2.1 (Vertex Set). The vertex set is the set of n vertices of a graph that is represented
as V = {v 1 , ..., v n }. As a shortcut, a pair of vertices, say {v i , v j }, is often shortened to simply
{i , j }).
Definition 2.2 (Edge Set). The edge set is the set of m edges of a graph that is represented as
E = {e 1 , ...e m }.
represents all possible pairs of vertices in V , even a pair consisting of the same vertex twice! [8]
Because the notation of ( v2 ) from Definition 2.3 may be confusing, we demonstrate this
¡ ¢
Aside 2.1. Suppose we have a set S = {1, 2, 3}. Then S2 = {{1, 2}, {2, 3}, {1, 3}}, as per the rules of
¡ ¢
basic combinatorics. But we want this set to also have pairs consisting of the same vertex twice,
so we attach an extra parenthesis to S2 as notation representing this expanded set in question.
¡ ¢
So ( S2 ) = {{1, 1}, {2, 2}, {3, 3}, {1, 2}, {2, 3}, {1, 3}}. If we define the function φ : E → ( V2 ), then each
¡ ¢ ¡ ¢
¡V ¢
element of the edge set maps to the pair of vertices in ( 2 ) that the edge connects. Looking
back at our example graph from Figure 1.1, we would say φ(e 1 ) = {1, 2}, and φ(e 4 ) = {5, 5}.
Putting these definitions together, we can describe a finite graph as G(V, E , φ). We now
expand upon some other relevant terminology.
Definition 2.4 (Adjacent). Adjacent vertices are vertices with at least one edge connecting
them.
Definition 2.5 (Incident). An incident edge between two vertices is an edge that connects two
adjacent vertices.
Definition 2.6 (Multiple Edge). If two adjacent vertices have more than one edge incident to
them, we say that there is a multiple edge between them.
4
Definition 2.7 (Loop). An edge that connects a vertex to itself is referred to as a loop.
Definition 2.9 (Complete Graph). A graph is considered to be complete if there exists exactly
one edge between any two distinct vertices. Complete graphs can be uniquely defined by their
number of vertices. Here, let K p represent the complete graph of p vertices.
Figure 2.1: Examples of complete graphs of 1, . . . , 8 vertices. The number of edges is written
¡ ¢ p(p−1)
after each labeled K p and can be found using p2 = 2 since there are no multi-
ple edges in a complete graph and every distinct pair of vertices will have an edge
between them. Figure by authors.
Definition 2.10 (Adjacency Matrix). An adjacency matrix A for a finite graph with n vertices
is an n by n matrix where the entry in the i th row and the j th column is the number of edges
that connect the i th and j th vertices. The name "adjacency" originates from the fact that the
matrix tracks the number of edges between adjacent vertices [8].
5
Figure 2.2: A reexamination of the graph from Figure 1.1, this time linking its visual representa-
tion with its adjacency matrix. Notice that there is a loop in the graph from the edge
labeled e 4 that connects back to circle 5, which shows up as a 1 on the diagonal at
the entry in the 5th row and 5th column.
We note several interesting observations about the adjacency matrix that are summarized in
the following remarks.
Remark 2.1. The way we number the vertices of a graph is arbitrary. Given a graph with n
vertices, we can number them from 1 to n in any order. So a graph can have more than one
valid adjacency matrix.
Remark 2.2. The number of edges connecting the i th and j th vertices is equal to the number
of edges connecting the j th and i th vertices. So any adjacency matrix A must be symmetric
across the main diagonal since we assume that all edges are bidirectional (we can traverse
through them both ways). Thus the adjacency matrix is equal to its transpose: A = A T .
Remark 2.3. Since we know that each entry on the main diagonal must have the same row
and column index, we can say that any nonzero element on the main diagonal represents a
loop connecting a vertex to itself.
Remark 2.4. We can also use the adjacency matrix to classify a simple graph. Recall that a
simple graph can at most have only one edge between vertices. So any entry in the adjacency
matrix of a simple graph must have the maximum value of 1. Additionally, since simple graphs
do not have loops, all entries on the main diagonal must be 0.
3 n- STEP WALKS
3.1 W HAT IS AN n-S TEP WALK ?
Armed with these powerful tools to represent graphs mathematically, we can now begin
applying what we know to a case study––specifically, the n-step walk.
Definition 3.1 (n-step walk). An n-step walk from vertex v x to v y is a sequence that uses n
edges and the vertices between them to take us from v x to v y . More rigorously, this sequence
can be denoted as {v 1 , e 1 , v 2 , ...e n , v n+1 } where v 1 = v x and v n+1 = v y and each of edges in the
sequence is the edge connecting the vertex before and after it [8].
6
There can be many paths from one vertex to another, so there can be multiple valid n-step
walks between any two pairs of vertices. One could use trial-and-error to compute the total
number of n-step walks between two vertices, but this method quickly becomes increasingly
infeasible when tested with larger graphs and/or longer n-step walks. The adjacency matrix,
as we will soon find, provides us with a more systematic way to compute the number of n-step
walks between any two vertices.
Theorem 3.1. For any graph G with m vertices, let A represent its adjacency matrix. Then the
number of n-step walks from vertex v x to vertex v y of length l is the entry in the (x, y) index of
A l , where (x, y) denotes the xth row and yth column [8].
Induction Hypothesis. Assume that the proposed theorem is true for when n = l − 1, where l
is some positive integer. Then the (x, y) index of A = A ( l − 1) is the number of l − 1-step walks
from v x to vertex v y . Let r i j be the value of the (i , j ) entry of A l −1 , and let p i j be the (i , j ) entry
of A. To get A l , we multiply A l −1 and A. By matrix multiplication, the (x, y) index of A l −1 A
would be r x1 p 1y + r x2 p 2y + · · · + r xm p m y where A and A l − 1 are m by m matrices. Now, we
must show r x1 p 1y + r x2 p 2y + · · · + r xm p m y gives the number of l -step walks from v x to v y .
Note that any l -step walk from v x to v y can be seen as a (l − 1)-step walk from v x to a vertex
adjacent to v y , followed by a 1-step walk from that vertex to v y . So to count the total number of
l -step walks, we sum the number of l − 1 step walks to each and every possible vertex adjacent
to v y multiplied by the number of 1-step walks from that vertex to v y .
Our sum r x1 p 1y + r x2 p 2y + ... + r xm p m y does exactly what we have observed: r x1 gives the
number of (l − 1)-step walks to v 1 and p 1y gives the number of 1-step walks to v y . So r x1 p 1y
gives the l -step walks between v x and v y where v 1 is the second to last vertex. Similarly,
r x2 p 2y gives the number of l -step walks where v 2 is the second to last vertex. This pattern
continues all through v m , thus covering all possible walks since every l -step walk must have
some "second to last" vertex. So A l −1 A = A l has the described property that the number of
n-step walks of length l is the (x, y) index of A l . ■
7
When analyzing any graph G, we define the eigenvalues of G as the eigenvalues of the adjacency
matrix of G. For the remainder of this section, we will examine what extra information
eigenvalues bring into the picture when evaluating n-step walks.
As stated in Section 2.2, the definition of the adjacency matrix implies it must be symmetric
across the main diagonal because the number of paths between the (i , j ) vertices must be the
same number of paths that exist between the ( j , i ) vertices. This specific classification lends
itself to the use of the Spectral Theorem, which states that an n by n symmetric matrix must
be orthogonally diagonalizable [6]. We now apply this conclusion to the adjacency matrix and
derive an explicit formula for the number of n-step walks between two vertices, which is more
flexible than the method of always calculating A l from Theorem 3.1.
Theorem 3.2. Let A be the adjacency matrix for a graph G with eigenvalues λ1 , λ2 , . . . , λm (with
the same eigenvalue listed multiple times to match its multiplicity). Let p i , j be the entries of
P in the orthogonal diagonalization P −1 AP = D, where D is the diagonalization matrix for A.
Then the number of n-step walks between vertices v x and v y is given by c 1 λn1 +· · ·+c m λnm where
c k = p x,k p y,k [8].
Proof. Since the adjacency matrix A is symmetric, we use the Spectral Theorem to conclude
that A is orthogonally diagonalizable. Recall that the definition of orthogonally diagonal-
izable implies that there must exist orthogonal matrices P and P −1 such that P −1 AP = D,
where D is a diagonal matrix [6]. Raising D to the nth power, we get D n = (P −1 AP )n =
(P −1 AP )(P −1 AP ) . . . (P −1 AP ) = P −1 A(P P −1 )A . . . (P P −1 )AP . Each paired term P P −1 cancels
out to the identity matrix I , so the product simplifies to D n = P −1 A n P . Rewriting the equation
to isolate A n , we get A n = P D n P −1 . Using Theorem 3.1, we can say that the number of n-step
walks is then the entry in the (x, y) entry of A n . Since A n = P D n P −1 , we can compute the value
of the entry of the (x, y) entry to get the number of n-step walks.
Since D is a diagonalization of A, the entries along the diagonal of D are the eigenvalues
of A. So the diagonal of D n will be λn1 , ..., λnm . Also note that since P is an orthogonal matrix,
P −1 = P T . Using the algebraic formula for matrix multiplication, the value of the (x, y) entry
of P D n P −1 is p x1 λn1 p y1 + ... + p xm λnm p ym , which is in the form c 1 λn1 + · · · + c m λnm where c k =
p x,k p y,k . ■
8
Theorem 3.3. Let A be the adjacency matrix for a graph G with eigenvalues λ1 , λ2 . . . λm (with
the same eigenvalue listed multiple times to match its multiplicity). The number of closed n-step
walks on G is given by λn1 + · · · + λnm [8].
Proof. Recall that closed walks are walks that begin and end at the same vertex. This can be
represented by the diagonal entries of the adjacency matrix, which show the number of walks
from a vertex to itself. So the total number of closed n-step walks in a graph G is essentially
the sum of the diagonal entries of A n , where A is the adjacency matrix. By definition, the
sum of the diagonal entries of A n is the trace of the matrix, and trace equals the sum of its
eigenvalues [1]. From Theorem 3.2, A n can be diagonalized into a matrix with λn1 , . . . , λnm along
its diagonal, so the eigenvalues of A n are λn1 , . . . , λnm . Hence the total number of closed walks is
λn1 + ... + λnm . ■
Theorem 3.4. The eigenvalues of a complete graph K p are p −1 with multiplicity 1 and −1 with
multiplicity p − 1 [8].
Proof. Note that the adjacency matrix A of a complete graph K p will be a p by p matrix with
every entry equal to 1 except on the main diagonal, where all diagonal entries are equal to 0.
We can rewrite the adjacency matrix as A = J − I , where J is a matrix with all entries of 1, and
the identity matrix I is subtracted from J . We know the reduced row echelon form of J has
only one pivot column since all rows are identical to each other and are not equal to 0, so J a
rank of 1. Since J has rank 1, it can only have one nonzero eigenvalue. We briefly depart to
prove this as a lemma.
Lemma 3.5. A matrix with rank 1 has only one nonzero eigenvalue.
Proof. By way of contradiction, assume that a matrix J of rank 1 has more than one nonzero
eigenvalue. Let λ1 and λ2 be two distinct eigenvalues both nonzero. Let v 1 and v 2 be their
respective eigenvectors, making them linearly independent. So J v 1 and J v 2 must also be
linearly independent. This is impossible since the range of J is one-dimensional. Similarly,
this lone nonzero eigenvalue has multiplicity 1, otherwise by the same logic, we would get a
list of two linearly independent vectors in range(J ). So J has one nonzero eigenvalue and p − 1
eigenvalues of 0. Since the diagonal is composed of p ones, the trace of J is p, implying that
the sum of the eigenvalues is p. Thus J has an eigenvalue of p of multiplicity 1 and another
eigenvalue 0 of multiplicity p − 1. ■
Proceeding with A = J − I , we can use the eigenvalues of J to solve for the eigenvalues of A.
For any eigenvector e for J , Ae = (J − I )e = Je − e = λe − e = (λ − 1)e. So every eigenvector of
9
J is an eigenvector of A, but with an eigenvalue shifted down by 1. Thus the corresponding
eigenspaces will be identical, meaning that the multiplicities will be the same. So the eigenval-
ues of A will be p − 1 with multiplicity of 1 and 0 − 1 = −1 with multiplicity of p − 1. Since A is
the adjacency matrix of K p , this proves the desired statement. ■
Now that we have a formula to compute the eigenvalues of K p , we can use it to derive a new
formula to calculate the number closed n-step walks on a complete graph.
Theorem 3.6. For a complete graph K p , the number of closed n-step walks from a vertex v x to
(p−1)n +(p−1)(−1)n
itself is given by p [8].
Proof. By Theorem 3.4, the eigenvalues of K p are p − 1 with multiplicity 1 and −1 with
multiplicity p − 1. So by Theorem 3.3, the total number of closed n-step walks is given by
(p − 1)n + (−1)n + ... + (−1)n. By the symmetry of a complete graph, the number of closed
walks must be equal for each and every vertex. To find the number of closed n-step walks for a
particular vertex, divide the total number by the total number of vertices p, so we arrive at the
(p−1)n +(p−1)(−1)n
expression p . ■
Solution. (1) First, we need to translate this graph into its adjacency matrix. There are six
vertices, so our adjacency matrix will be a 6 by 6 matrix. Given that the graph is simple, we can
assume that none of its edges are multiple edges. Thus if an edge exists between vertices i and
j , then we mark a 1 for the corresponding index (i , j ) of A. Doing so for all vertex combinations,
we get:
0 1 0 0 0 0
1 0 1 0 0 1
0 1 0 0 1 1
A=
0 0 0 0 0 1
0 0 1 0 0 0
0 1 1 1 0 0
10
Now we can continue on our journey to find the eigenvalues of A. We begin by finding the
determinant of A − I λ.
¯ ¯
¯−λ 1 0 0 0 0 ¯¯
¯
¯ 1 −λ 1 0 0 1 ¯¯
¯
¯ 0 1 −λ 0 1 1 ¯¯
d et (A − I λ) = ¯ ¯ = λ6 − 6λ4 − 2λ3 + 6λ2 − 1
¯
¯ 0 0 0 −λ 0 1 ¯
¯ ¯
¯ 0 0 1 0 −λ 0 ¯¯
¯
¯ 0 1 1 1 0 −λ¯
p p p p
Factoring the polynomial, we get (λ + 5+1 2 )2 (λ − 5−1
2 )2 (λ + ( 2 − 1))(λ − ( 2 + 1)). So the
p p p p
eigenvalues are − 25−1 , 5−1 2 , − 2 + 1, 2 + 1.
(2) After successfully collecting these eigenvalues, we now seek out the number of closed
n-step walks using the explicit rule from Theorem 3.3. Since ptwo of pthe eigenvalues
p p
have a
− 5−1 − 5−1 5−1 5−1
p
multiplicity
p of 2, the full list that we input into the formula is 2 , 2 , 2 , 2 , − 2+
1, 2 + 1. We arrive at a general formula for the total number of closed n-step walks:
p p p p p p
5−1 n − 5−1 n 5−1 n 5−1 n
(− 2 ) + ( 2 ) + ( 2 ) + ( n
2 ) + (− 2 + 1) + ( 2 + 1)
n
As an extension, say we want to find the total number of closed 1-step walks. It should be
clear immediately that since this graph has no loops, there is no way to traverse one edge and
get back to the same vertex, so the answer should be 0. Let us verify with Theorem 3.3.
p p p p p p
5−1 1 − 5−1 1 5−1 1 5−1 1
(− 2 ) + ( 2 p) + ( 2 ) + ( 1
2 ) + (− 2 + 1) + ( 2 + 1)
1
=
p p p
2( − 25−1 + 5−1
2 ) + (− 2 + 1 + 2 + 1) = 2(−1) + 2 = 0
What about the number of closed 2-step walks? Since 2 is a relatively low number, so we can
get to this value visually. From the graph, we can tell that 2-step walk that returns to the vertex
it started from involves moving to an adjacent vertex for the first step, and directly coming
back for the second step. So the number of closed 2-step walks starting at a single vertex is
equal the number of vertices that are adjacent to it. Vertices 1, 4, and 5 each only one adjacent
vertex, and 2, 4, and 6 have three adjacent vertexes. So the total number of closed n-step walks
is 1 + 1 + 1 + 3 + 3 + 3 = 12. Again, let us verify mathematically with Theorem 3.3.
p p p p p p
5−1 2
(− 2 ) + ( − 25−1 )2 + ( 5−1 2 5−1 2 2
2 ) + ( 2 ) + (− 2 + 1) + ( 2 + 1)
2
= 12
Finally, to get a sense of how quickly the number of n-step walks can escalate, let us compute
when n = 10.
p p p p p p
( − 25−1 )10 + ( − 25−1 )10 + ( 5−1 10 5−1 10 10 10
2 ) + ( 2 ) + (− 2 + 1) + ( 2 + 1) = 6972
Example 2. Draw out the complete graph K 10 . Then, compute (1) its eigenvalues and (2) the
number of closed n-step from vertex 1 to itself for when n = 8.
Solution. To draw a complete graph with 10 vertices, we plot 10 points and connect every
two distinct vertices with an edge.
11
Figure 4.2: An example of a possible pictorial representation for K 10 . Note that the vertices do
not have to be arranged symmetrically as shown. Figure by authors.
(1) By Theorem 3.4, the eigenvalues of K 10 are 9 with multiplicity 1 and -1 with multiplicity
10 − 1 = 9.
(2) To get the number of closed n-step walks from any vertex to itself, we can use the formula
n n
we proved in Theorem 3.5, and get (10−1) +(10−1)(−1)
10
1
= 10 (9n + 9(−1)n ). Notice that since there
are no loops, there should not be any closed 1-step walks. To test this observation, we substitute
1
1 in for n in the formula, and we indeed get 0. If we try n = 8, we get 10 (98 + 9(−1)8 ) = 4304673
n-step walks.
5 C ONCLUSION
Our examination of graph theory only begins to scratch the surface of this fully-fledged branch
of mathematics. However, the theorems and ideas we have illustrated with the adjacency
matrix and its eigenvalues are inherently fundamental, and may serve as a starting point
for those looking to further explore the graph theory. We hope to have provided a new way
of viewing graphs for our Math 22A readers, and to have demonstrated the beauty in the
relationships between cumulative linear algebra concepts, such as the Spectral Theorem, to
applications in graph theory.
To further continue the study of graph theory, there are many paths to take (pun intended
*wink*). One possible "next step" is to prove formulas to analyze random walks, which have
a succession of random steps through edges and vertices in a graph. Here one would see
the intersection of linear algebra and probability, tying together two profound branches of
mathematics to create something new [8].
R EFERENCES
[1] Sheldon Jay Axler. Linear Algebra Done Right. Undergraduate Texts in Mathematics. New
York: Springer, 1997. ISBN: 0387982582. URL: http://linear.axler.net/.
[2] Kimberly Jordan Burch. An Introduction to Chemical Graph Theory. 2008. URL : https:
//www.maa.org/external_archive/joma/Volume8/Burch/Research.html.
12
[3] Stephan C. Carlson. Königsberg bridge problem. July 2010. URL: https://www.britannica.
com/science/Konigsberg-bridge-problem.
[4] John M. Harris, Jeffry L. Hirst, and Michael J. Mossinghoff. Combinatorics and graph
theory. Springer, 2010.
[5] Pallavi Jayawant and Kerry Glavin. “Minimum spanning trees”. In: Involve 2.4 (2009),
pp. 439–450. DOI: 10.2140/involve.2009.2.439. URL: https://doi.org/10.2140/involve.2009.
2.439.
[6] David C. Lay. Linear Algebra and Its Applications. Reading, MA: Addison-Wesley, 2003.
[7] Oliver Mason and Mark Verwoerd. “Verwoerd M: Graph theory and networks in biology”.
In: IET Systems Biology (), pp. 1–89.
[8] Richard P. Stanley. Algebraic combinatorics: walks, trees, tableaux, and more. Springer.,
2018.
13