Inner Product Spaces
Inner Product Spaces
An inner product space is a pair (V, ⟨·, ·⟩), where V is vector space and ⟨·, ·⟩ is
an inner product on V .
⟨v, w⟩ = v1 w1 + · · · + vn wn .
1
Inner product spaces 2
This is the typical example of an inner product, called the Euclidean inner
product, and when Rn is referred to as an inner product space, one should
assume that the inner product is the Euclidean one, unless explicitly stated
otherwise.
a b
2. Let A ∈ M2 (R), A = be a positive definite matrix, that is a >
b c
0, det(A) > 0.
u1
⟨u, v⟩ = v1 v2 A .
u2
It can easily be verified that ⟨·, ·⟩ is an inner product on the real linear space
R2 .
⟨u, v⟩ = u1 v1 + u2 v2 .
Properties. From the definition one can easily deduce the following properties
of an inner product:
1. ⟨v, 0⟩ = ⟨0, v⟩ = 0,
∥·∥:V →R
Example 1.4. 1. On the real linear space Rn one can define a norm in several
ways.
3. One can easily verify that the axioms in the definition of norm are satisfied.
This norm is called the Euclidian norm.
1
∥x∥p = (|x1 |p + |x2 |p + · · · + |xn |p ) p ,
In a real inner product space we can define the angle of two vectors as
\ ⟨v, w⟩
(v, w) = arccos
∥v∥ · ∥w∥
Inner product spaces 5
We have
\ π
v⊥w ⇔ ⟨v, w⟩ = 0 ⇔ (v, w) = .
2
Theorem 1.8. (Parallelogram law) Let V be an inner product space and u, v ∈
V . Then
Proof.
= ⟨u, u + v⟩ + ⟨v, u + v⟩
= ⟨u, u − v⟩ − ⟨v, u − v⟩
We obtain the desired relation by adding the last form for (1.1) and (1.2).
Proof.
||u + v||2 = ⟨u + v, u + v⟩
= ⟨u, u + v⟩ + ⟨v, u + v⟩
Orthonormal Bases
Definition 1.10. Let (V, ⟨·, ·⟩) be an inner product space and let I be an arbitrary
index set. A family of vectors A = {ei ∈ V |i ∈ I} is called an orthogonal family,
if ⟨ei , ej ⟩ = 0 for every i, j ∈ I, i ̸= j. The family A is called orthonormal if it is
orthogonal and ∥ei ∥ = 1 for every i ∈ I.
One of the reason that one studies orthonormal families is that in such special
bases the computations are much more simple.
for all α1 , α2 , . . . , αm ∈ F.
α1 e1 + α2 e2 + · · · + αn en = 0V ⇐⇒ α1 = α2 = · · · = αn = 0.
⟨u3 , u1 ⟩ = 0 ⇐⇒ ⟨v3 + α1 u1 + α2 u2 , u1 ⟩ = 0
⇐⇒ ⟨v3 , u1 ⟩ + α1 ⟨u1 , u1 ⟩ = 0
⟨v3 , u1 ⟩
⇐⇒ α1 = − .
⟨u1 , u1 ⟩
⟨u3 , u2 ⟩ = 0 ⇐⇒ ⟨v3 + α1 u1 + α2 u2 , u2 ⟩ = 0
⇐⇒ ⟨v3 , u2 ⟩ + α2 ⟨u1 , u2 ⟩ = 0
⟨v3 , u2 ⟩
⇐⇒ α2 = − .
⟨u2 , u2 ⟩
⟨v3 ,u1 ⟩ ⟨v3 ,u2 ⟩
So, u3 = v3 − u
⟨u1 ,u1 ⟩ 1
− u,
⟨u2 ,u2 ⟩ 2
and u3 ⊥ u1 , u3 ⊥ u2 .
By induction, we will have that
⟨vk ,u1 ⟩ ⟨vk ,u2 ⟩ ⟨vk ,uk−1 ⟩
uk = vk − u
⟨u1 ,u1 ⟩ 1
− u
⟨u2 ,u2 ⟩ 2
− ··· − u .
⟨uk−1 ,uk−1 ⟩ k−1
u1
Because the set {u1 , u2 , . . . , uk } is orthogonal, then the set {e1 = ,e
||u2 || 2
=
u2 uk
||u2 ||
. . . , ek = ||uk ||
} is orthonormal.
Corolary 1.14. Every finitely dimensional inner product space has an orhtonormal
basis.
Orthogonal complement
Let U ⊆ V be a subset of an inner product space V . The orthogonal complement of
U , denoted by U ⊥ is the set of all vectors in V which are orthogonal to every vector
in U i.e.:
U ⊥ = {v ∈ V |⟨v, u⟩ = 0, ∀u ∈ U }.
Inner product spaces 9
V = U ⊕ U⊥
Solved Problems
Problem 1.1. Let R4 be the inner product space with the canonical inner product.
Apply the Gram-Schmidt orthogonalization method to construct orthogonal basis
for the subspace
⟨v3 , u1 ⟩ ⟨v3 , u2 ⟩
u3 = v3 − u1 − u2
⟨u1 , u1 ⟩ ⟨u2 , u2 ⟩
3 + 4 + 16 + 7 6 + 6 − 24 − 14
= (3, 2, 8, −7) − (1, 2, 2, −1) − (2, 3, −3, 2)
1+4+4+1 4+9+9+4
= (3, 2, 8, −7) − 3(1, 2, 2, −1) + (2, 3, −3, 2)
Solution: We will determine first a basis from the general solution of the
system
(S)
1 1 1 −1 0 0 1 1 1 −1 0 0
L −L ,L −L 2L +L
1 2 1 3 2 3
≃ 0 −1 −2 −1 −1 0 ≃
1 2 3 0 1 0
1 −1 −3 −3 −2 0 0 2 4 2 2 0
1 1 1 −1 0 0
0 −1 −2 −1 −1 0 .
0 0 0 0 0 0
rank (A) = 2 and there are 5 unknowns, so z
= α, t= β and w = γ are free
x + y = −α + β − γ
variables. We can determine x and y solving the system .
−y = 2α + β + γ
Inner product spaces 11
S ={(α + 2β + γ, −2α − β − γ, α, β, γ) | α, β, γ ∈ R}
= span{s1 = (1, −2, 1, 0, 0), s2 = (2, −1, 0, 1, 0), s3 = (1, −1, 0, 0, 1)}
⟨s2 , u1 ⟩
u2 = s2 − u1
⟨u1 , u1 ⟩
3
= (2, −1, 0, 1, 0) − (1, −1, 0, 0, 1)
3
= (1, 0, 0, 1, −1).
⟨s3 , u1 ⟩ ⟨s3 , u2 ⟩
u′3 = s1 − u1 − u2
⟨u1 , u1 ⟩ ⟨u2 , u2 ⟩
3 1
= (1, −2, 1, 0, 0) − (1, −1, 0, 0, 1) − (1, 0, 0, 1, −1)
3 3
1
= (1, −2, 1, 0, 0) − (1, −1, 0, 0, 1) − (1, 0, 0, 1, −1)
3
1 1 2
= (− , −1, 1, − , ).
3 3 3
u2 √1 (1, 0, 0, 1, −1) = ( √1 , 0, 0, √1 , − √1 ),
n2 = ∥u2 ∥
= 3 3 3 3
u3 1
n3 = ∥u3 ∥
= √
2 6
(−1, −3, 3, −1, −2) = (− 2 6 , − 2√3 6 , 2√3 6 , − 2√1 6 , − √16 ).
√1
S = {(α, 2α − β + 3γ, β, γ) | α, β, γ ∈ R}
⟨s3 , u1 ⟩ ⟨s3 , u2 ⟩
u′3 = s3 − u1 − u2
⟨u1 , u1 ⟩ ⟨u2 , u2 ⟩
−3 3
= (0, 3, 0, 1) − (0, −1, 1, 0) − (1, 1, 1, 0)
2 3
3 3
= (0, 3, 0, 1) + (0, − , , 0) + (−1, −1, −1, 0)
2 2
1 1
= (−1, , , 1).
2 2
Inner product spaces 13
S ⊥ = {v ∈ R4 |⟨v, s⟩ = 0, ∀s ∈ S}
The matrix system has the rank 3, so one of the unknowns become free variable,
x2 = α, and we can determine x1 = −2α, x3 = α and x4 = −3α.
S ⊥ = {(−2α, α, α, −3α) | α ∈ R} = span{(−2, 1, 1, −3)}.
An orthogonal basis for BS⊥ = {s4 = (−2, 1, 1, −3)}, an orthonormal basis is
s4 √1 (−2, 1, 1, −3)
BnS ⊥ = {n4 }, where n4 = ∥s4 ∥
= 15
= (− √215 , √115 , √115 , − √315 ).
Problem 1.4. Verify that the vectors v1 = (1, 0, 2, −1), v2 = (1, 2, 0, 1) are orthog-
onal and complete them to form orthogonal basis of R4 .
v ⊥ v 1
⟨v, v1 ⟩ = 0
x + 2z − t = 0
⇐⇒ ⇐⇒ .
v ⊥ v 2
⟨v, v2 ⟩ = 0
x + 2y + t = 0
S = {(−α − β, α, β, β − α) | α, β ∈ R}
Problems
Problem 1.5. Let S be the solutions of the system. Find an orthonormal basis in
S if:
x+y+z−t=0
1. (S) : x + 2y + 3z = 0
x − y − 3z − 3t = 0;
x+y+t=0
2. (S) : 2x + y + z = 0
x − y + 2z − 3t = 0;
Inner product spaces 15
x+y−z+t=0
3. (S) : x − y − z + 2t = 0
x + 3y − z = 0.
Problem 1.6. Let S be the set of solutions of the following systems and find bases
in S and in the orthogonal complement S ⊥ if:
x + y + 2z = 0
1. (S) : 2x + 3y + z = 0
x + 2y − z = 0;
2x − y − z + t = 0
2. (S) :
x + y + 3z − t = 0
x+y−z+t=0
3. (S) : x + y + 3z − t = 0
x + y − 5z + 3t = 0.
Problem 1.7. Verify that the following sets of vectors are orthogonal and complete
them to form orthogonal basis of R4 :
Problem 1.8. Let R4 be the inner product space with the canonical inner product.
Apply the Gram-Schmidt orthogonalizations to construct orthogonal bases for the
subspaces spanned by the following sets of vectors: