Mathematics I

[Inner Product Space]

Dr. Anirban Lakshman

Assistant Professor

Department of Mathematics
IIIT Kalyani

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 1 / 35

 O UTLINE

1 I NNER P RODUCT S PACES

2 O RTHOGONAL AND O RTHONORMAL V ECTORS

3 N ORMED S PACE

4 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

5 R EFERENCES

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 2 / 35

 I NNER P RODUCT S PACES

In this section our main object is to study vector spaces in which it

makes sense to speak of the length of a vector and of the angle
between two vectors.
We shall do this by studying a certain type of scalar-valued function
on pairs of vectors, known as an inner product.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 3 / 35

 I NNER P RODUCT S PACES

Definition of Inner Product Space

Let F be the field of real numbers or the field of complex numbers
and V be a vector space over F .
Then there is a function which assigns to each ordered pair of
vectors α, β ∈ V a scalar hα, βi ∈ F . This function is called a
inner product on V if it satisfies the following axioms:
[i] hu1 + u2 , v i = hu1 , v i + hu2 , v i;
[ii] hcu, v i = chu, v i;
[iii] hu, v i = hv , ui, the bar denoting complex conjugation;
[iv] hu, ui > 0 if u 6= θ; where θ is the zero element of V .
∀u1 , u2 , u, v ∈ V and c ∈ F .

The vector space V with an inner product is called a inner

product space.
Dr. Anirban Lakshman (IIIT Kalyani) MAC101 4 / 35
 I NNER P RODUCT S PACES

Conditions [i], [ii] and [iii] imply that

[v ] hu1 , cu2 + u3 i = chu1 , u2 i + hu1 , u3 i
If F = R, then the complex conjugates appearing in [iii] and [v ]
are superfluous.
If F = C, then the complex conjugates appearing in [iii] and [v ]
are necessary.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 5 / 35

 I NNER P RODUCT S PACES

Let V be a vector space over the field R. Then V with an inner

product defined on it is said to be a Euclidean Space.
Let V be a vector space over the field C. Then V with an inner
product defined on it is said to be a Unitary Space.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 6 / 35

 I NNER P RODUCT S PACES

Example of Euclidean Space

E XAMPLE
Euclidean n-Space Rn
Consider the vector space V = Rn . The dot product or scalar
product in Rn is defined by hu, v i = u · v = a1 b1 + a2 b2 + . . . + an bn ;
where u = {a1 , a2 , . . . , an } and v = {b1 , b2 , . . . , bn }.
This function defines an inner product on Rn .
The norm kuk of the vector u = {a1q
, a2 , . . . , an } in this space is as
p √
follows: kuk = hu, ui = u · u = a12 + a22 + a33 + . . . + an2 .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 7 / 35

 I NNER P RODUCT S PACES

E XAMPLE
Marix Space M = Mm,n
Let M = Mm,n = vector space of all real m × n matrices.
An inner product is defined on M by hA, Bi = tr (B t A).
If A = [aij ]m×n and B = [bij ]m×n , then

m X
X n
T
hA, Bi = tr (B A) = aij bij
i=1 j=1

m X
X n
2
kAk = hA, Ai = aij2
i=1 j=1
qP P
m n 2
So, norm of A is kAk = i=1 j=1 aij

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 8 / 35

 I NNER P RODUCT S PACES

Example of Unitary Space

E XAMPLE
Complex Vector Space Cn
Consider the vector space V = Cn .
Let us define the function hα, βi = ni=1 xi yi ;
P

Where α = (x1 , x2 , . . . , xn ) and β = (y1 , y2 , . . . , yn ) ∈ Cn .

This function defines an inner product on Cn .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 9 / 35

 O UTLINE

1 I NNER P RODUCT S PACES

2 O RTHOGONAL AND O RTHONORMAL V ECTORS

3 N ORMED S PACE

4 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

5 R EFERENCES

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 10 / 35

 O RTHOGONAL AND O RTHONORMAL V ECTORS

Definition
Let v1 , v2 , . . . , vn be non zero vectors in an inner product space V .
If hvi , vj i = 0 whenever i 6= j, then {v1 , v2 , . . . , vn } is said to be an
orthogonal set of vectors.

E XAMPLE
Example of Orthogonal Vectors
Let V = R3 . The inner product on V is the dot product.
The set {(1, 1, 1), (2, 1, −3), (4, −5, 1)} is an orthogonal set in V ,
since h(1, 1, 1), (2, 1, −3)i = 1 · 2 + 1 · 1 + 1 · (−3) = 0;
h(1, 1, 1), (4, −5, 1)i = 1 · 4 + 1 · (−5) + 1 · 1 = 0
and h(2, 1, −3), (4, −5, 1)i = 2 · 4 + 1 · (−5) + (−3) · 1 = 0

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 11 / 35

 O RTHOGONAL AND O RTHONORMAL V ECTORS

Definition
Let v1 , v2 , . . . , vn be non zero vectors in an inner product space V .
Then the set of vectors v1 , v2 , . . . , vn are said to be Orthonormal set
of vectors if
(i) All vi ’s are unit vectors i.e., hvi , vi i = 1, ∀i = 1, 2, . . . , n
and (ii) {v1 , v2 , . . . , vn } is an orthogonal set of vectors.

Given any orthogonal set of non zero vectors v1 , v2 , . . . , vn , it

is possible to form an orthonormal set {u1 , u2 , . . . , un } by defining
ui = ( kv1i k )vi for i = 1, 2, . . . , n.
An orthogonal set of vectors may contain the null vector θ but an
orthonormal set contains only non-null vectors.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 12 / 35

 O RTHOGONAL AND O RTHONORMAL V ECTORS

E XAMPLE
Let v1 = (1, 1, 1), v2 = (2, 1, −3) and v3 = (4, −5, 1).
Here {v1 , v2 , v3 } is an orthogonal set in R3 .
To form an orthonormal set, let u1 = ( kv11 k )v1 = √1 (1, 1, 1);
3
u2 = ( kv12 k )v2 = √1 (2, 1, −3)
14
and u3 = ( kv13 k )v3 = 1
√ (4, −5, 1).
42
Then {u1 , u2 , u3 } is an orthonormal set of R3 .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 13 / 35

 O RTHOGONAL AND O RTHONORMAL V ECTORS

T HEOREM
If {v1 , v2 , . . . , vn } is an orthogonal set of non zero vectors in an
inner product space V , then {v1 , v2 , . . . , vn } are L.I.

C OROLLARY
An orthonormal set of vectors in a inner product space is linearly
independent.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 14 / 35

 O RTHOGONAL AND O RTHONORMAL V ECTORS

T HEOREM
Let {u1 , u2 , . .P
. , un } be an orthonormal basis for an inner product
space V . If v = ni=1 ci ui , then ci = hv , ui i.

C OROLLARY
Let {u1 , u2 , . . . , un } be an orthonormal basis for an inner
product space V . If u = ni=1 ai ui and v = ni=1 bi ui , then
P P

hu, v i = ni=1 ai bi ,
P

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 15 / 35

 O UTLINE

1 I NNER P RODUCT S PACES

2 O RTHOGONAL AND O RTHONORMAL V ECTORS

3 N ORMED S PACE

4 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

5 R EFERENCES

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 16 / 35

 N ORMED S PACE

A significant portion of linear algebra is in fact geometric in nature

because much of the subject grew out of the need to generalize the
basic geometry of R2 and R3 to non visual higher-dimensional
spaces.
The usual approach is to coordinatize geometric concepts in R2
and R3 , and then extend statements concerning ordered pairs and
triples to ordered n-tuples in Rn and Cn .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 17 / 35

 N ORMED S PACE
For example, the length of a vector u ∈ R2 or v ∈ R3 is obtained
from the Pythagorean theorem by computing the length of the
hypotenuse of a right triangle as shown in the below figure -

F IGURE : 1

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 18 / 35

 N ORMED S PACE

p p
This measure of length kuk = x 2 + y 2 and kv k = x 2 + y 2 + z 2
is called the euclidean norm in R2 and R3 , and there is an obvious
extension to higher dimensions.
 
x1
 x2 
For a vector xn×1 =  . , the euclidean norm of x is defined to be
 
 .. 
xn
Pn 1 √
kxk = ( i=1 xi2 ) 2 = x T x; whenever x ∈ Rn×1 .
1 √
kxk = ( ni=1 xi2 ) 2 = x ? x; whenever x ∈ Cn×1 .
P

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 19 / 35

 N ORMED S PACE

E XAMPLE
  
0 i
−1  2 
   
If u =  2  and v = 
 
 1 − i , then

−2  0 
4 1+i
qP √ √
kuk = ui2 = u T u = 0 + 1 + 4 + 4 + 16 = 5
pP √ √
kv k = |vi |2 = v ? v = 1 + 4 + 2 + 0 + 2 = 3

The complex version of kxk includes the real version as a special

case because |z|2 = z 2 whenever z is a real number.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 20 / 35

 N ORMED S PACE

General Vector Norms

A norm for a real or complex vector space V is a function k k
which maps V into R that satisfies the following conditions:
[i] kxk ≥ 0 and kxk = 0 ⇒ x = 0,
[ii] kαxk = αkxk for all scalars α,
[iii] kx + y k ≤ kxk + ky k.

Definition of Normed linear space

A vector space V is said to be a normed linear space if, to each
vector v ∈ V , there is associated a real number kv k, called the norm
of v .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 21 / 35

 N ORMED S PACE

T HEOREM
If V is an inner product space, then for any vectors α, β ∈ V and
any scalar c
(i) kcαk = |c|kαk;
(ii) kαk > 0 for α 6= 0;
(iii) |hα, βi| ≤ kαkkβk; [Cauchy Schwarz Inequality]
(iv ) kα + βk ≤ kαk + kβk. [Triangular Inequality]

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 22 / 35

 N ORMED S PACE

The Pythagorean Law

If u and v are orthogonal vectors in an inner product space V , then
ku + v k2 = kuk2 + kv k2 .

In R2 , this is just the familiar Pythagorean theorem as shown in

this Figure.

F IGURE : 2

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 23 / 35

 N ORMED S PACE

Example: Consider the vector space C[−1, 1] with inner product

R1
defined by hf , gi = −1 f (x)g(x)dx; where f (x), g(x) ∈ C[−1, 1].
=⇒ The vectors 1 and x are orthogonal, since
R1 R1
h1, xi = −1 1.x dx = −1 x dx = 0.
To determine the lengths of these vectors, we compute
R1 R1
h1, 1i = −1 1.1 dx = −1 dx = 2,
R1 R1
hx, xi = −1 x.x dx = −1 x 2 dx = 23 .
1 √ 1
√
6
It follows that, k1k = (h1, 1i) 2 = 2 and kxk = (hx, xi) 2 = 3
Since 1 and x are orthogonal, they satisfy the Pythagorean law:
2
k1 + xk2 = k1k2 + kxk2 = 2 + 3 = 83 .
It also be verify that:
R1 8
k1 + xk2 = h1 + x, 1 + xi = −1 (1 + x)2 dx = 3

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 24 / 35

 N ORMED S PACE

The Parallelogram Law

If u and v be any two vectors in an inner product space V , then
ku + v k2 + ku − v k2 = 2kuk2 + 2kv k2 .

In R2 , this is just the familiar Parallelogram Law as shown in this

Figure.

F IGURE : 3
Dr. Anirban Lakshman (IIIT Kalyani) MAC101 25 / 35
 O UTLINE

1 I NNER P RODUCT S PACES

2 O RTHOGONAL AND O RTHONORMAL V ECTORS

3 N ORMED S PACE

4 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

5 R EFERENCES

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 26 / 35

 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

Let β be a fixed non-zero vector in a inner product spaceV .

Then for non-zero vector α ∈ V there exists a unique real number c
such that α − cβ is orthogonal to β.
c is determined by hα − cβ, βi = 0. Therefore hα, βi = chβ, βi.
hα,βi
So, c = hβ,βi
Here c is said to be the scalar component (or component) of α
along β.
hα,βi
cβ = hβ,βi β is said to be the projection of α upon β

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 27 / 35

 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

F Every non null subspace W of a finite dimensional inner product

space V possesses an orthonormal basis.

=⇒ Let {α1 , α2 , . . . , αr } be a basis of W . An orthogonal basis will be

obtained by a method known as Gram-Schmidt process of
orthogonalisation.
Since the basis vectors are non zero, we pick up one of them, say
α1 , and consider as the first member of the new basis.
For convenience we rename it β1 , i.e., β1 = α1 .
Let β2 = α2 − c1 β1 , where c1 β1 is the projection of α2 upon β1 .
Then β2 is orthogonal to β1 and L{β1 , β2 } = L{β1 , α2 } = L{α1 , α2 }.
hα2 ,β1 i
β 2 = α2 − hβ1 ,β1 i β1 .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 28 / 35

 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

α3 ∈
/ L{β1 , β2 }. Let β3 = α3 − d1 β1 − d2 β2 ,
where d1 β1 , d2 β2 are the projection of α3 upon β1 , β2 respectively.
Then β3 is orthogonal to β1 , β2 and
L{β1 , β2 , β3 } = L{β1 , β2 , α3 } = L{α1 , α2 , α3 }.
hα3 ,β1 i hα3 ,β2 i
β3 = α3 − hβ1 ,β1 i β1 − hβ2 ,β2 i β2 . α4 ∈
/ L{β1 , β2 , β3 }.
Let β4 = α4 − r1 β1 − r2 β2 − r3 β3 , where r1 β1 , r2 β2 , r3 β3 are the
projection of α4 upon β1 , β2 , β3 respectively.
Then β4 is orthogonal to β1 , β2 , β3 and
L{β1 , β2 , β3 , β4 } = L{β1 , β2 , β3 , α4 } = L{α1 , α2 , α3 , α4 }.
hα4 ,β1 i hα4 ,β2 i hα4 ,β3 i
β4 = α4 − hβ1 ,β1 i β1 − hβ2 ,β2 i β2 − hβ3 ,β3 i β3 .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 29 / 35

 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

This process terminates after a finite number of steps because at

every step one vector of the given basis is repalced by vector in the
desired orthogonal basis.
Finally we obtain
hαr ,β1 i hαr ,β2 i hαr ,βr −1 i
βr = αr − hβ1 ,β1 i β1 − hβ2 ,β2 i β2 − ... − hβr −1 ,βr −1 i βr −1 , and
{β1 , β2 , . . . , βr } is an orthogonal basis of the subspace W .
βk
Now we replace each vector βk by γk = kβk k .
Then {γ1 , γ2 , . . . , γr } is an orthonormal basis of W .

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 30 / 35

 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

Example of Gram-Schmidt process

Example: Use Gram-Schmidt process to obtain an orthogonal basis

from the basis set {(1, 0, 1), (1, 1, 1), (1, 3, 4)} of the Euclidean
space R3 with standard inner product.
=⇒ Let α1 = (1, 0, 1), α2 = (1, 1, 1), α3 = (1, 3, 4).
Let β1 = α1 and β2 = α2 − c1 β1 , where c1 is the scalar component
of α2 along β1 .
Then β2 is orthogonal to β1 and L{β1 , β2 } = L{β1 , α2 } = L{α1 , α2 }.
hα2 ,β1 i
c1 = hβ1 ,β1 i =1 ∴ β2 = α2 − β1 = (0, 1, 0).
Let β3 = α3 − d1 β1 − d2 β2 , where d1 , d2 are scalar components of
α3 along β1 , β2 respectively.
Then β3 is orthogonal to β1 , β2 and L{β1 , β2 , β3 } = L{α1 , α2 , α3 }.
Dr. Anirban Lakshman (IIIT Kalyani) MAC101 31 / 35
 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

hα3 ,β1 i hα3 ,β2 i

d1 = hβ1 ,β1 i = 52 , d2 = hβ2 ,β2 i = 3
1 = 3.
So, β3 = (1, 3, 4) − 52 (1, 0, 1) − 3(0, 1, 0) = 32 (−1, 0, 1).
Therefore an orthogonal basis is {(1, 0, 1)(0, 1, 0), 23 (−1, 0, 1)}.

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 32 / 35

 O UTLINE

1 I NNER P RODUCT S PACES

2 O RTHOGONAL AND O RTHONORMAL V ECTORS

3 N ORMED S PACE

4 G RAM -S CHMIDT PROCESS FOR ORTHOGONALISATION

5 R EFERENCES

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 33 / 35

 R EFERENCES

Gilbert Strang, Linear Algebra and Its Applications.

Steven J. Leon, Linear Algebra with Applications, Pearson.
Carl D. Meyer, Matrix Analysis and Linear Algebra,Siam
Kenneth Hoffman; Ray Kunze, Linear Algebra, Pearson

Dr. Anirban Lakshman (IIIT Kalyani) MAC101 34 / 35