Section 6.

3: Gram-Schmidt process;
QR-decomposition

University of Johannesburg: Engineering Linear Algebra B

(MATEAB2)
Dr Wilmari Morton

1/16
Orthogonal and orthonormal sets of vectors

Example
Let S = {u 1 , u 2 , u 3 } be a subset of R3 with the Euclidean inner
product where u 1 = (0, 1, 0), u 2 = ( √1 , 0, √1 ) and
2 2
u 3 = (− √1 , 0, √1 ).
2 2
1 Compute hu 1 , u 2 i, hu 1 , u 3 i and hu 2 , u 3 i. What do you
notice? All the inner products are equal to 0.
2 Compute ||u 1 ||, ||u 2 || and ||u 3 ||. What do you notice?
All the norms are equal to 1, i.e., all the vectors are unit vectors.

Definition
A set of two or more vectors in a real inner product space is
said to be orthogonal if all pairs of distinct vectors in the set are
orthogonal. An orthogonal set in which each vector has norm 1
is said to be orthonormal.

2/16
Orthogonality and linear independence; Orthogonal
and orthonormal bases

Theorem
If S = {v 1 , v 2 , . . . , v n } is an orthogonal set of nonzero vectors
in an inner product space, then S is linearly independent.

Definition
In an inner product space, a basis consisting of orthonormal
vectors is called an orthonormal basis, and a basis consisting
or orthogonal vectors is called an orthogonal basis

Example
From the previous example it follows that
S = {(0, 1, 0), ( √1 , 0, √1 ), (− √1 , 0, √1 )} is an orthonormal
2 2 2 2
basis for R3 with the Euclidean inner product.

3/16
Example

Example
Let R2√be equipped
 with the inner product generated by
2 √0
A= and let B = {(1, 2), (−3, 1)}.
0 3
1 Show that B is an orthogonal set. Why can we now
conclude that B is an orthogonal basis for R2 with respect
to the inner product generated by A? h(1, 2), (−3, 1)i = 0.
2 Find [(1, 16)]B . [(1, 16)]B = (7, 2).
3 Compute
h(1,16),(1,2)i
h(1,2),(1,2)i . 7.
1

h(1,16),(−3,1)i
h(−3,1),(−3,1)i . 2.
2

What do you notice?

4/16
Coordinates relative to orthonormal bases
Theorem
1 If S = {v 1 , v 2 , . . . , v n } is an orthogonal basis for an inner
product space V , an if u is any vector in V , then

hu, v 1 i hu, v 2 i hu, v n i

u= 2
v1 + 2
v2 + · · · + vn
||v 1 || ||v 2 || ||v n ||2

2 If S = {v 1 , v 2 , . . . , v n } is an orthonormal basis for an inner

product space V , then

u = hu, v 1 i v 1 + hu, v 2 i v 2 + · · · + hu, v n i v n

Thus, the coordinate vector of a vector u in V relative to an
orthogonal basis S = {v 1 , v 2 , . . . , v n } is
 
hu, v 1 i hu, v 2 i hu, v n i
(u)S = , , . . . ,
||v 1 ||2 ||v 2 ||2 ||v n ||2
and relative to an orthonormal basis S = {v 1 , v 2 , . . . , v n } is
(u)S = (hu, v 1 i , hu, v 2 i , . . . , hu, v n i) 5/16
Orthogonal projections

Theorem (Projection theorem)

If W is a finite-dimensional subspace of an inner product space
V , then every vector u in V can be expressed in exactly one
way as
u = w1 + w2
where w 1 is in W and w 2 is in W ⊥ .

6/16
Orthogonal projections

Definition
Let W be a finite-dimensional subspace of an inner product
space V . Then the unique vectors w 1 ∈ W and w 2 ∈ W ⊥ such
that u = w 1 + w 2 are called the orthogonal projection of u on
W and the orthogonal projection of u on W ⊥ , respectively, and
are denoted by

w 1 = projW u and w 2 = projW ⊥ u.

Alternatively, w 2 is also called the component of u orthogonal

to W . Moreover, since projW ⊥ u = u − projW u we can express u
as follows:

u = projW u + projW ⊥ u
= projW u + (u − projW u)

7/16
Example

Example
Let W = span(v 1 , v 2 ) = span((2, 0, 0), (0, 3, 0)) a subspace of
R3 with the Euclidean inner product and let u = (2, −1, 3).
1 Express u as a sum of vectors form W and W ⊥ .
u = (2, −1, 0) + (0, 0, 3).
hu,v 1 i hu,v 2 i
2 Calculate v
||v 1 ||2 1
+ v .
||v 2 ||2 2
What do you see? (2, −1, 0).

8/16
Orthogonal projection

Theorem
Let W be a finite-dimensional subspace of an inner product
space V .
1 If {v 1 , v 2 , . . . , v r } is an orthogonal basis for W , and u is
any vector in V , then

hu, v 1 i hu, v 2 i hu, v r i

projW u = 2
v1 + 2
v2 + · · · + vr
||v 1 || ||v 2 || ||v r ||2

2 If {v 1 , v 2 , . . . , v r } is an orthonormal basis for W , and u is

any vector in V , then

projW u = hu, v 1 i v 1 + hu, v 2 i v 2 + · · · + hu, v r i v r

9/16
Finding orthonormal bases

Theorem
Every nonzero finite-dimensional inner product space has an
orthonormal basis.

Example
Let R3 have the Euclidean inner product. Use the
Gram-Schmidt process to transform the basis {u 1 , u 2 , u 3 } into
an orthonormal basis, where un1 = (0, 1, 1), u 2 = (1, 2, 0) and o
u 3 = (3, 1, 1). √1 (0, 1, 1), √1 (1, 1, −1), √1 (2, −1, 1) .
2 3 6

10/16
 The Gram-Schmidt process
To convert a basis {u 1 , u 2 , . . . , u r } into an orthogonal basis
{v 1 , v 2 , . . . , v r } perform the following computations:
Step 1 v 1 = u 1
hu 2 ,v 1 i
Step 2 v 2 = u 2 − ||v 1 ||2 1
v = u 2 − projW1 u 2
where W1 = span(v 1 ).
hu 3 ,v 1 i hu 3 ,v 2 i
Step 3 v 3 = u 3 − ||v 1 ||2 1
v − ||v 2 ||2 2
v = u 3 − projW2 u 3
where W2 = span(v 1 , v 2 ).
..
.
hu r ,v r −1 i
Step r v r = u r − hu r ,v 1 i
||v 1 ||2 1
v − hu r ,v 2 i
||v 2 ||2 2
v − ··· − v
||v r −1 ||2 r −1
=
u r − projWr −1 u r
where Wr −1 = span(v 1 , v 1 , . . . , v r −1 ).

Optional To convert the orthogonal basis into an orthonormal basis

{q 1 , q 2 , . . . , q r }, normalize the orthogonal basis vectors.
11/16
Examples

Example
Let R3 have the weighted inner product with weights 2, 3 and 1.
Use the Gram-Schmidt process to transform the basis
{u 1 , u 2 , u 3 } into an orthogonal basis, where u 1 = (0, 1, 1),
u 2 = (1, 2, 0) and u 3 = (3, 1, 1). {(0, 1, 1), (2, 1, −3), (9, −3, 9)}.

Example
Let M22 have the standard inner product. Use the
Gram-Schmidt process to transform the basis {A1 , A2 , A3 , A4 }
into an orthonormal basis, where
       
0 1 0 1 1 0 0 0
A1 = , A2 = , A3 = , A4 =
1 0 0 1 0 1 1 1
        
√1
0 1 √1
0 1 1 3 −1 1 −1 −1
, , √ ,2 .
2 1 0 6 −1 2 2 3 1 1 1 1

12/16
Finding orthonormal bases

Theorem
If W is a finite-dimensional inner product space, then:
1 Every orthogonal set of nonzero vectors in W can be
enlarged to an orthogonal basis for W .
2 Every orthonormal set in W can be enlarged to an
orthonormal basis for W .

Example
Let R3 have the Euclidean inner product. Extend
{(1, −1, 1), (−1, 1, 2)} to an orthogonal basis for R3 .
{(1, −1, 1), (−1, 1, 2), (1, 1, 0)}

13/16
QR-decomposition

The problem
If A is an m × n matrix with linearly independent column
vectors, and if Q is the matrix that result by applying the
Gram-Schmidt process to the column vectors of A, what
relationship, if any, exists between A and Q?

Theorem (QR-decomposition)
If A is an m × n matrix with linearly independent column
vectors, then A can be factored as

A = QR

where Q is an m × n matrix with orthonormal column vectors,

and R is an n × n invertible upper triangular matrix.

14/16
Examples

Example
Find the QR-decomposition of the matrix where possible:
   √ √ √ 
0 1 3
 1
0 √ √2 2 √2 √2
3 6
1 A = 1 2 1 A =  √12 √1
− √16   0 3 √3
 
3
1 1 1
1 0 1 √
2
− 3
√ √
6
0 0 6
 
1 3 1
2 B= rank(B) = 2 and therefore B doesn’t have a
2 4 3
QR-decomposition.

15/16
Examples

Example
 
u u2
Let M22 have the inner product defined by: for U = 1
u3 u4
 
v v
and V = 1 2
v3 v4

hU, V i = u1 v1 + 2u2 v2 + 2u3 v3 + u4 v4 .

Apply the Gram-Schmidt process to the basis

       
1 0 1 1 1 1 1 1
, , ,
0 0 0 0 1 0 1 1
       
1 0 0 1 0 0 0 0
, , ,
0 0 0 0 1 0 0 1

16/16