0% found this document useful (0 votes)
68 views6 pages

Innerproduct 2

1) The document discusses orthogonal and orthonormal sequences of vectors in an inner product space and proves that an orthogonal sequence of non-zero vectors is linearly independent. 2) It defines the orthogonal projection of a vector onto a subspace and proves it is the unique vector in the subspace that minimizes the distance to the original vector. 3) The Gram-Schmidt process is described as a method to transform a basis into an orthonormal basis in a finite-dimensional inner product space.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
68 views6 pages

Innerproduct 2

1) The document discusses orthogonal and orthonormal sequences of vectors in an inner product space and proves that an orthogonal sequence of non-zero vectors is linearly independent. 2) It defines the orthogonal projection of a vector onto a subspace and proves it is the unique vector in the subspace that minimizes the distance to the original vector. 3) The Gram-Schmidt process is described as a method to transform a basis into an orthonormal basis in a finite-dimensional inner product space.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Inner Product Spaces: Part 2

Let V be an inner product space. We let K = R or C according as V is real or complex. A


sequence of vectors u1 , u2 , u3 , ... is said to be orthogonal if < ui , uj >= 0 for i 6= j. If, in addition,
we have ||ui || = 1 for all i then the sequence is said to be orthonormal.
If u1 , u2 , u3 , ..., un is an orthogonal sequence of non-zero vectors of V then u1 , u2 , ..., un is linearly
independent. Indeed, if a1 u1 + a2 u2 + · · · + an un = 0, we have

0 =< a1 u1 + a2 u2 + · · · + an un , ui >= ai < ui , ui >

which implies that ai = 0 since < ui , u, >= ||ui ||2 6= 0. If u1 , u2 , ..., un is a basis of V then

v = a1 u1 + a2 u2 + · · · + an un =⇒ < v, ui >= ai < ui , ui >


<v,ui >
so that ai = <ui ,ui > . If u1 , u2 , ..., un is an orthonormal basis of V then

v = a1 u1 + a2 u2 + · · · + an un =⇒ ai =< v, ui > .

Let u1 , u2 , ..., un be an orthogonal sequence of non-zero vectors of V and let v ∈ V . If


< v, u1 > < v, u2 > < v, un >
w= u1 + u2 + · · · + un
< u1 , u 1 > < u2 , u2 > < u n , un >
then v − w is orthogonal to each vector ui . If u is any vector in the subspace W of V spanned by
u1 , u2 , ..., un , we have v − u = v − w + w − u with v − w orthogonal to w − u and so

||v − u||2 = ||v − w||2 + ||w − u||2 ≥ ||v − w||2

with equality iff u = w. It follows that w is the unique vector of W which minimizes ||v − w||.
The vector PW (v) = w is called the orthogonal projection of v on W . The mappping PW is a
linear operator on V with image W and kernel the set W ⊥ of vectors of V which are orthogonal to
every vector of W . The subspace W ⊥ is called the orthogonal complement of W in V . We have
V = W ⊕ W ⊥.
Theorem 1. Every finite-dimensional inner product space has an orthonormal basis.
Proof. Let u1 , u2 , ..., un be a basis of the inner product space V . Define v1 , v2 , ..., vn inductively by
< ui+1 , v1 > < ui+1 , v2 > < ui+1 , vi >
v1 = u1 , vi+1 = ui+1 − ( + + ··· + ).
< v 1 , v1 > < v2 , v2 > < vi , v i >
This process is well-defined since one proves inductively that

Span(v1 , ..., vi ) = Span(u1 , ..., ui )

so that, in particular, vi 6= 0 for all i. By construction, v1 , ..., vn is orthogonal and so can be


normalized to give an orthonormal basis of V . The above process is known as the Gram-Schmidt
Process.
Corollary 2. An n-dimensional inner product space V is isomorphic to K n with the ususal inner
product.

1
Proof. Let u1 , ..., un be an orthonormal basis of V and let x, y be the coordinate vectors of u, v ∈ V
with respect to this basis. Then

< u, v > =< x1 u1 + x2 u2 + · · · + xn un , y1 u1 + y2 u2 + · · · + xyn un


Xn
= xi yj < ui , uj >
i,j=1

= x1 y 1 + x2 y 2 + · · · + xn y n .

Example 1. If W = Span(1, 1, 1, 1, 1) in R4 then, to find an orthonormal basis for W ⊥ , we first


find a basis of W ⊥ . The following vectors

u1 = (1, −1, 0, 0), u2 = (1, 0, −1, 0), u3 = (1, 0, 0, −1)

are such a basis. Then

v1 = u1 = (1, −1, 0, 0),


< u2 , v1 > 1 1
v2 = u2 − v1 = (1, 0, −1, 0) − (1, −1, 0, 0) = (1, 1, −2, 0),
< v 1 , v1 > 2 2
< u3 , v1 > < u 3 , v2 >
v3 = u3 − v1 − v2
< v 1 , v1 > < v 2 , v2 >
1 1 1
= (1, 0, 0, −1) − (1, −1, 0, 0) − (1/2, 1/2, −1, 0) = (1, 1, 1, −3).
2 3 3
form an orthogonal basis of W ⊥ . Normalizing these vectors, we get
√ √ √
(1/ 2)(1, −1, 0, 0), (1/ 6)(1, 1, −2, 0), (1/2 3)(1, 1, 1, −3)

as an orthonormal basis for W ⊥ .


If X, Y ∈ K n×1 are the coordinate matrices of the vectors u, v with respect an orthonormal basis
of the inner product space V , we have

< u, v >= X t Y .

If f = (f1 , ..., fn ) is an orthonormal basis of V and P = [pij ] is the transition matrix to another
basis g = (g1 , ..., gn ), the j-th column of P is Pj = [gj ]f . It follows that the ij-th entry of P t P is
equal to
Pit P j =< gi , gj > .
Hence, g is an orthonormal basis iff P t P = I or, equivalently,
t
P −1 = P .

Such a matrix P is called an orthogonal matrix if K = R and a unitary matrix if K = C.


t
The adjoint of a complex matrix A is the matrix A∗ = A . We have

(aA + bB)∗ = aA∗ + bB ∗ , (AB)∗ = B ∗ A∗ .

A complex matrix is said to be normal if it commutes with its adjoint, i.e., AA∗ = A∗ A. Such a
matrix is necessarily square. If A = A∗ then A is said to be self-adjoint or Hermitian.

2
Theorem 3. The eigenvalues of a Hermitian matrix are real.
Proof. If AX = λX with X 6= 0 then

λ < X, X >=< AX, X >= (AX)t X = X t At X = X t AX = X t AX =< X, AX >= λ < X, X >

which implies that λ = λ.


Theorem 4. Let A be a normal complex matrix. Then there is a unitary matrix U such that U −1 AU
is a diagonal matrix.
Proof. Let P be an invertible matrix such that P −1 AP is upper triangular. Let U be the unitary
matrix obtained by applying the Gram-Schmidt Process to the columns of P and normalizing the
resulting column matrices. Then, if Pj and Uj are the j-th columns of P and U , we have for
1≤j≤n
Span(P1 , ..., Pj ) = Span(U1 , ..., Un )
and so

AUj = A(c1j P1 + c2j P2 + · · · + cjj Pj )


= c1j AP1 + c2j AP2 + · · · + cjj APj
= b1j U1 + b2j U2 + · · · + bjj Uj

since APj ∈ Span(P1 , ..., Pj ) = Span(U1 , ..., Uj ). Hence B = U −1 AU is upper triangular, i.e., bij = 0
for j < i. Now

BB ∗ = U ∗ AU (U ∗ AU )∗ = U ∗ AU U ∗ A∗ U = U ∗ AA∗ U = U ∗ A∗ AU = U ∗ A∗ U U ∗ AU = B ∗ B

and so B = [bij ] is also normal. Comparing the i-th diagonal entries of BB ∗ and B ∗ B, we get for
1≤i≤n X X
|bij |2 = |bji |2
j≥i j≤i

which implies by induction on i that bij = 0 for j > i. Hence B is a diagonal matrix.
Corollary 5. Two eigenvectors of a normal matrix with distinct eigenvalues are orthogonal.
Corollary 6. A real symmetric matrix is diagonalizable.
Corollary 7. If A is a real symmetric matrix, there is an orthogonal matrix P with P −1 AP a
diagonal matrix.
Let A be a normal complex matrix n × n matrix and suppose that U is a unitary matrix which
diagonalizes A. Then, if Uj is the j-column of U , we have AUj = λj Uj . Let Rj = Uj Uj∗ , an n × n
matrix of rank 1. The matrices Rj have the following properties:
n
X
ARj = λj Rj , Rj∗ = Rj , Rj2 = Rj , Ri Rj = 0 (i 6= j), I= Rj .
j=1
P
To prove the last identity it suffices to prove that X = j Uj Uj∗ X for any column matrix X. But
this identity follows from the fact that Uj∗ X =< X, Uj > . Multiplying the last identity on the left
by A, we get
A = λ1 R1 + λ2 R2 + · · · + λn Rn .

3
If TA is the linear operator on Cn with matrix A with respect to the standard basis, the matrix
Ri is the matrix (with respect to the standard basis) of the orthogonal projection of Cn on the
one-dimensional subspace spanned by the eigenvector of TA having coordinate matrix Ui .
If c1 , .., cm are the distinct eigenvalues of A, let Qi be the sum of the Rj with λj = ci . The
matrices Qi with 1 ≤ i ≤ m have the following properties:
m
X
Q∗j = Qj , Q2j = Qj , Qi Qj = 0 (i 6= j), I= Qj ,
j=1

A = c1 Q1 + c2 Q2 + · · · + cm Qm .
This decomposition of the normal matrix A is called the spectral resolution of A and the matrices
Qi are called the projection matrices of the resolution. The adjective spectral comes from the
fact that in functional analysis the set of eigenvalues of a matrix is called the spectrum of T . The
matrix Qi is the matrix (with respect to the standard basis) of the orthogonal projection of Cn on
the eigenspace of TA corresponding to the eigenvalue ci .
From the spectral resolution of A we get
An = cn1 Q1 + cn2 Q2 + · · · + cnm Qm
for any natural number n and hence, for any polynomial p(λ),
p(A) = p(c1 )Q1 + p(c2 )Q2 + · · · + p(cm )Qm .
If p(ci ) 6= 0 for all i, then p(A) is invertible with
p(A)−1 = p(c1 )−1 Q1 + p(c2 )−1 Q2 + · · · + p(cm )−1 Qm .
Q
If pi (λ) = j6=i cλ−c i
j −ci
, we have pi (ci ) = 1 and pj (ci ) = 0 for j =
6 i. Hence Qi = pi (A).
If f is any complex valued function whose domain is a subset of C and contains the set of
eigenvalues of T , we define
f (A) = f (c1 )Q1 + f (c2 )Q2 + · · · + f (cm )Qm .
P
For example, if f (z) = ez = n≥0 z n /n! then

eA = ec1 E1 + ec2 E2 + · · · + ecm Em .


A Hermitian matrix is said to be positive if the eigenvalues are ≥ 0 and positive definite if the
eigenvalues are > 0. If A is a positive Hermitian matrix, then there is a positive Hermitian matrix
B with B 2 = A, namely
√ √ √ √
B = A = c1 Q1 + c2 Q2 + · · · + cm Qm .
We shall see later that B is in fact unique.
If A is a real symmetric matrix, the projection matrices of its spectral resolution are real since
the eigenvalues of A are real. Indeed, if
A = λ1 R1 + λ2 R2 + · · · + λn Rn
with Ri = Pi Pit , where Pi is the i-th column of a real orthogonal matrix P with P −1 AP equal to a
diagonal matrix with i-th diagonal entry equal to λi , the matrix Qi is the sum of the Rj with λj = ci .
The matrix Qi is the matrix (with respect to the standard basis) of the orthogonal projection of Rn
on the eigenspace of TA corresponding to the eigenvalue ci .

4
 
2 1 1 1
1 2 1 1
Example 2. . If A is the real symmetric matrix   2
1 1 2 1, we have (A − 1) = 4(A − 1) and so
1 1 1 2
(A−1)(A−5) = 0. The eigenspace
√ for the eigenvalue 1 is the subspace W ⊥ of example
√ √ 1 and so has as
orthonormal basis f1 = (1/ 2)(1, −1, 0, 0), f2 = (1/ 6)(1, 1, −2, 0), f3 = (1/2 3)(1, 1, 1, −3). The
eigenspace for the eigenvalue 5 is one-dimensional with basis the unit vector f4 = (1/2)(1, 1, 1, 1).
The matrix  √ √ √ 
1/ √2 1/√6 3/2
√ 1/2
−1/ 2 1/ 6 1/2√3 1/2
P = [1R4 ]f,e =  0
√ 
−2/ 6 1/2 √3 1/2
0 0 −3/2 3 1/2
is an orthogonal matrix with  
1 0 0 0
0 1 0 0
P AP = 
−1
0
.
0 1 0
0 0 0 5
If Pi is the i-th column of P and Ri = Pi Pit , we have
   
1 −1 0 0 1 1 −2 0
1 −1 1 0 0  1 1 1 −2 0
R1 =  
 , R2 = 
 
,
2 0 0 0 0 6 −2 −2 4 0
0 0 0 0 0 0 0 0
   
1 1 1 −3 1 1 1 1
1 
 1 1 1 −3 
 , R4 = 1 1 1 1 1.
R3 =
12  1 1 1 −3 4 1 1 1 1
−3 −3 −3 9 1 1 1 1
We have A = R1 + R2 + R3 + 5R4 = Q1 + 5Q2 , where
   
3 −1 −1 −1 1 1 1 1
1 −1 3 −1 −1 11 1 1 1
Q1 = R1 + R2 + R3 =  , Q2 = R4 =  .
4 −1 −1 3 −1 4 1 1 1 1
−1 −1 −1 3 1 1 1 1
√ √
We thus have An = Q1 + 5n Q2 for all n ∈ Z and, if B = A = Q1 + 5Q2 , we have B 2 = A.
As another application of this consider etA = et Q1 + e5t Q2 with t ∈ R. Then etA is a matrix
C = [cij (t)] whose entries are differentiable real valued functions cij (t) of t. If we define the derivative
of C = C(t) to be dC 0 0 d tA
dt = C (t) = [cij (t)], we have dt e = et Q1 + 5e5t Q2 = AetA . This can be used
to solve the system of differential equations
dx1
= 2x1 + x2 + x3 + x4
dt
dx2
= x1 + 2x2 + x3 + x4
dt
dx3
= x1 + x2 + 2x3 + x4
dt
dx4
= x1 + x2 + x3 + 2x4
dt

5
dX
Indeed, writing this system in the form dt = AX, it is an easy exercise to prove that
 
a
 
tA  b 
X=e  
c
d

is the unique solution with X(0) = [a, b, c, d]t .

You might also like