0% found this document useful (0 votes)
74 views6 pages

Mathematical Tools Problem

This document contains a student's solutions to problems from Problem Set 8 on mathematical tools in computer science. The student solves the following problems: (1) Showing that the largest eigenvalue of a k-regular graph is k. (2) Proving that the multiplicity of the eigenvalue k is equal to the number of connected components in the graph. (3) Showing that a graph has an eigenvalue of -k if and only if it has a bipartite connected component. (4) Demonstrating that if a graph is bipartite, then for any eigenvector with eigenvalue λ, there exists an eigenvector with eigenvalue -λ.

Uploaded by

Anonymous A9kn0h
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
74 views6 pages

Mathematical Tools Problem

This document contains a student's solutions to problems from Problem Set 8 on mathematical tools in computer science. The student solves the following problems: (1) Showing that the largest eigenvalue of a k-regular graph is k. (2) Proving that the multiplicity of the eigenvalue k is equal to the number of connected components in the graph. (3) Showing that a graph has an eigenvalue of -k if and only if it has a bipartite connected component. (4) Demonstrating that if a graph is bipartite, then for any eigenvector with eigenvalue λ, there exists an eigenvector with eigenvalue -λ.

Uploaded by

Anonymous A9kn0h
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Mathematical Tools in CS - Problem Set 8

Abu Radi Bader *********


December 29, 2017

1 Problem 1
(a) + (b): We show first that λ = k is an eigenvalue and then we show that for
all i ∈ [n], it holds that |λi | ≤ k. Thus, we can conclude that λ1 = k.
 
1
1
• λ = k is an eigenvalue: we show that x =  .  ∈ Rn is an eigenvector with
 
 .. 
1
n n
Acr ·xc = Acr ·1 =
P P P
eigenvalue λ = k, indeed (Ax)r = 1
c=1 c=1 c ∈ neighbors(r)
 
k
k 
G is k−regular
= k. Hence, Ax =  .  = kx.
 
 .. 
k

• For all i ∈ [n], |λi | ≤ k: Let v 6= 0 be some eigenvector of A, that is, there
is λ ∈ R with Av = λv 1 . Let z ∈ [n] be such that |λvz | is maximal, And
assume W.l.o.g that z = 1. We want to show that |λ| ≤ k: it holds that,
n
Ac1 vc =
P P
λv1 = (λv)1 = (Av)1 = vc
c=1 c ∈ neighbors(1)

Note that since |λv1 | is maximal, we get that |v1 | is maximal in |v|. There-
fore, we get from this and the above equality that

P T he triangle0 s inequality P
|λv1 | = | vc | ≤ |vc |
c ∈ neighbors(1) c ∈ neighbors(1)
G is k−regular and |v1 | is maximal
≤ k|v1 |

All in all, we have |λ||v1 | ≤ k|v1 |, that is |λ| ≤ k.


1A is symmetric over the reals, therefore all of its eigenvalues are reals

1
(c) A is symmetric over R, therefore, its eigenvectors form an orthogonal basis.
Now, assume that G has m connected components V1 , ..., Vm . We will show:
1) There are m orthogonal eigenvectors u(1) , ..., u(m) with eigenvalue k each.
Therefore the multiplicity of k is at least m.
2) Any vector u with Au = ku is in Span{u(1) , ..., u(m) }. Therefore the multi-
plicity of k is at most m
(i)
1. For every 1 ≤ i ≤ m, define the c’th coordinate of u(i) to be uc = 1 if c ∈
(i)
Vi and uc = 0 if c ∈ / Vi . In words, the vector u(i) is an indicator to which
vertices are in the Vi component. Its not hard to see that u(1) , ..., u(m) are
independent, indeed since the components partition the graph we get that
for every i 6= j hvi , vj i = 0. Now, what is left to show is that for every
1 ≤ i ≤ m, it holds that Au(i) = ku(i) . This follows from the fact that the
vertex r is in the component Vi all of r’s neighbors are in Vi , indeed
 P
 1 · 1, if r ∈ Vi
n 
(i) c ∈ neighbors(r)
Acr (i)
P
(Au )r = · (u )c = P
c=1 
 1 · 0, if r ∈
/ Vi
c ∈ neighbors(r)
(
k, if r ∈ Vi
= = (ku(i) )r
0, if r ∈
/ Vi

2. Let u be some non-zero vector with Au = ku, fix i ∈ [m] and let r∗ be a
vertex in Vi with maximal |ur∗ |. Now since Au = ku, we get
P
kur∗ = (ku)r∗ = (Au)r∗ = uc
c ∈ neighbors(r ∗ )

Now, since r∗ has k neighbors and since all of r∗ ’s neighbors are in Vi , we


get that the above equality holds iff each uc in the sum equals ur∗ , this
is because ur∗ has bigger magnitude compared to the k uc s. All in all,
we have that ur∗ = uc for every c ∈ neighbors(r∗ ). Repeating a similar
argument on the neighbors of r∗ and so on... we get that for every r ∈ Vi
it holds that ur is the same. Meaning that, for every component Vi the
values of ur where r ∈ Vi , are all equal. Hence, when restricted to Vi , u is
a scaling of u(i) , thus we can write u as αi u(i) .
P
i

(d)

• For the first direction, assume that that λn = −k, that is there is a non-
zero vector u such that Au = −ku, let r be such that | − ku| is maximal.
We have
n
Acr uc =
P P
−kur = (−ku)r = (Au)r = uc
c=1 c ∈ neighbors(r)

2
Note that the above equality holds iff uc = −ur for every c ∈ neighbors(r).
Applying a similar argument on the neighbors of r, we will get that if t is
a neighbor of a neighbor of r then ut = −(−ur ) = ur . If we keep applying
the same argument till we reach every vertex in r’s connected component
we will get that every vertex v in this component either has uv = ur or
uv = −ur , and if uv1 6= uv2 then there is no edge between v1 and v2 .
Hence, r’s component is bipartite w.r.t 1) vertices v with uv = ur . and 2)
vertices v with uv = −ur .
• For the other direction, assume that G has bipartite connected component,
denote one side of this component by L and the other side by R, and
assume that L has m vertices, while R has s vertices. Assume W.l.o.g
that L = [m], R = [m + s] \ L. Define the vector u ∈ Rn to be such that
the first m coordinates in u are 1, the coordinates from m + 1 up to m + s
are all equal −1 and the rest of the coordinates are all zero. We claim
that u is an eigenvector with eigenvalue −k, indeed
P
– For every 1 ≤ r ≤ m, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are in R P
= −1 = −k.
c ∈ neighbors(r)
P
– For every m + 1 ≤ r ≤ m + s, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are in L P
= 1 = k.
c ∈ neighbors(r)
P
– For every m + s < r, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are not in L∪R P
= 0 = 0.
c ∈ neighbors(r)

Therefore, Au = −ku.

(e) G is bipartite, that is, V is partitioned to two sets L and R such that G0 s
edges are only between R and L. Assume W.l.o.g that L = [m], R = [m + s] \ L.
We show now that for every eigenvector u 6= 0 with eigenvalue λ, there exists an
eigenvector v 6= 0 with eigenvalue −λ: let u be some eigenvector with eigenvalue
λ. Define v as follows:

• For every 1 ≤ r ≤ m: vr = ur .
• For every m < r: vr = −ur .
In words, v agrees with u on the first m coordinates and agrees with −u on the
rest of the coordinates. Its not hard to see that since u 6= 0, then so is v. We
claim that v is an eigenvector with eigenvalue λ, indeed

3
P r 0 s neighbors are in R
• For every 1 ≤ r ≤ m, we have that (Av)r = vc =
c ∈ neighbors(r)
P P u0 s eigenvalue is λ
−uc = − uc = −(Au)r =
c ∈ neighbors(r) c ∈ neighbors(r)
−(λu)r .
P r 0 s neighbors are in L
• For every m < r, we have that (Av)r = vc =
c ∈ neighbors(r)
P u0 s eigenvalue is λ
uc = (Au)r = (λu)r .
c ∈ neighbors(r)

Now, since agrees with u on the first m coordinates and agrees with −u on the
other coordinates, we conclude that −λ is an eigenvalue of u. BTW, its not
hard to see that the above describes a linear bijection between the eigenspace
of λ and the eigenspace of −λ.

2 Problem 2
To begin with, we prove the following lemma
Lemma 2.1. Let A be an n×n matrix. It holds that rank(A)+dim(E(0, A)) =
n, where E(0, A) is the eigenspace of the the zero eigenvalues of A.

Proof. A non-zero vector v is an eigenvector with eigenvalue λ iff Av = λv iff


(A − λI)v = 0. That is, the eigenspace of the eigenvalue lambda is Ker(A −
λI). By the nullity theorem on the matrix A, we have rank(A) + ker(A) = n.
Choosing λ = 0, we will get that ker(E(0, A)) = ker(A − 0I) = ker(A), now
substituting this back to rank(A) + ker(A) = n, concludes the result.

Back to the question, the adjacency matrix A of Kn is A = J − I, where J is


the all ones matrix. Since, raank(J) = 1, we get by the above lemma that J
has the 0 eigenvalue with multiplicity n − 1. And its not hard to see that the
vector x ∈ Rn which consists of ones, is an eigenvector of J with eigenvalue n.
Now, since Ax = (J − I)x = Jx − Ix = λx − x = x(λ − 1), we get that the
eigenvalues of A are those of J shifted with -1. That is, A has an eigenvalue of
n − 1 with multiplicity 1, and has the eigenvalue −1 with multiplicity n − 1.

3 Problem 3
To begin with, define for each i ∈ [n], #3 (i) to be the number of different paths
from i to itself. We have seen in recitation that #3 (i) = (A3 )i,i . Now,
of length 3P
n
note that i=1 #3 (i) is 6 times the number of triangles, because each triangle is
counter
Pn twice for each
Pof its nodes. That is, thu number of triangles in the
Pngraph
n
is 61 i=1 #3 (i) = 16 i=1 (A3 )i,i . That is, if we show that trace(A3 ) = i=1 λ3i ,
then we are done. We show this as follows: we know that trace(A3 ) equals to
the sum of A3 eigenvalues (with multiplicity), hence, it is sufficient to show

4
that λ31 , λ32 , ..., λ3n are the eigenvalues of A3 . This is easy! Because, if v is an
eigenvector of A with eigenvalue λ, then v is an eigenvector of A3 with eigenvalue
λ3 , indeed
A3 v = A2 Av = A2 λv = λAAv = λ2 Av = λ3 v

4 Problem 4
(a) To begin with, we use the fact the unit circle is compact. Also, note that
f is continuous (f can be thought of as a polynomial of n variables- x1 , ..., xn ).
Now, since f is a continuous function from a compact set to R, we get that
image(f ) is compact! why?
• image(f ) is closed: let (yn ) ⊆ image(f ) be a sequence that converges
to some y. Hence, by definition there is some sequence (xn ) ⊆ S with
f (xn ) = yn for every n. Now, since S is compact, there is a subsequence
xnk that converges to some x ∈ S. Now, by continuity of f , we have

y = limk→∞ ynk = limk→∞ f (xnk ) = f (x), that is y ∈ image(f ). Hence,


image(f ) is closed.

• image(f) is bounded: assume by contradiction that there is a sequence


(yn ) ⊆ image(f ) such that n ≤ yn for every n. As in the previous point
there is a sequence (xn ) in S with f (xn ) = yn , and since S is compact we
have that there is a convergent subsequence xnk that converges to x ∈ S.
But this is a contradiction, because f (x) ∈ R and:

∞ = limk→∞ ynk = limk→∞ f (xnk ) = f (x) < ∞

All in all, we get that image(f ) is compact, and therefore f obtains a


maximum.
(b) Let v be a vector in S on which f is maximal, we will show that for every
⊥ T
u ∈ span({v})
 ,u  Av = 0. As the hint suggests, define a function g : R → R
v+tu
by g(t) = f ||v+tu|| . By definition of f , we have that for every t ∈ R:

(v+tu)T A(v+tu) v T Av+tuT Av+tv T Au+t2 uT Au


g(t) = ||v+tu||2 = ||v||2 +2thv,ui+t2 ||u||2
0 0
Now, using ( yz )0 = y z−z
z2
y
. Define y(t) = v T Av + tuT Av + tv T Au + t2 uT Au,
and z(t) = ||v|| + 2thv, ui + t2 ||u||2 . We get that
2

limt→0 z 0 (t) = 2hv, ui = 0.2 And limt→0 y 0 (t) = uT Av + v T Au


Therefore,
2u ∈ span({v})⊥

5
T T
y 0 z−z 0 y
limt→0 g 0 (t) = limt→0 z2 = (u Av+v Au) limt→0 z(t)−0·limt→0 y
limt→0 z 2 (t) =
(uT Av+v T Au)||v||2
||v||4

Now, since f (x) obtains maximum at v. We get that g(t) has a critical point
at t = 0. Therefore, limt→0 g 0 (t) has to be 0, that is, uT Av + v T Au = 0. This
happens iff

hu, Avi = −hv, Aui


And since A is symmetric, this happens iff
hu, Avi = −hAv, ui

Meaning that uT Av = hu, Avi must be 0.


(c) Let v2 , ..., vn ∈ Rn be an orthonormal basis for span({v})⊥ . Then v, v2 , ..., vn
is an orthonormal basis for Rn . We want to show that for every 2 ≤ i ≤ n,
(vi )T Av = 0. This is sufficient, because every vector 0 6= u ∈ Rn can be written
as α1 v + α2 v2 + ... + αn vn , therefore, for every 2 ≤ i ≤ n, we have

(vi )T u = hvi , α1 v + α2 v2 + ... + αn vn i = αi ||vi ||2 = αi .


Meaning that u ∈ span({v}) iff (vi )T u = 0 for every 2 ≤ i ≤ n. This implies
that Av ∈ span({v}), that is, v is an eigenvector of A. What is left to show is
that for every 2 ≤ i ≤ n, (vi )T Av = 0. But this follows immediately from (b),
because every vi , 2 ≤ i ≤ n is in span({v})⊥ .
(d) The spectral theorem can be proven by inductively as follows: first, we
find an eigenvector v of A as in ((a), (b), (c)). Now, consider span({v})⊥ , we
claim that we can inductively find another vector, v2 , in span({v})⊥ which is
also an eigenvector of A. (clearly, this vector is orthogonal to v). We find
v2 as follows: the set S is now modified to be S2 = {x ∈ Rn : ||x|| = 1} ∩
{x ∈ Rn : x ∈ span({v})⊥ }, note that this intersection is isomorphic to the
unit circle in Rn−1 . That is, S2 is compact, and thus we can use the same
proof as in (a), (b), (c) to get that there is an eigenvector v2 ∈ span({v})⊥ .
Now, we can keep applying the same argument, where at iteration i: we define
Si = {x ∈ Rn : ||x|| = 1} ∩ {x ∈ Rn : x ∈ span({v, v1 , ..., vi−1 })⊥ }. Note, that
after we get the n’th vector we stop.

You might also like