Chapter.

5 Orthogonality
5.6. The Gram-Schmidt Orthogonalization Process

1. Background
2. Theorem
3. Gram–Schmidt QR Factorization
4. The Modified Gram–Schmidt Process

1
1. Background

2
2. The Gram–Schmidt Process

3
 2. The Gram–Schmidt Process
Example.1
Find an orthonormal basis for 𝑃3 if the inner product on 𝑃3 is defined by

Solution:
Starting with the basis 1, 𝑥, 𝑥 2 we can use the Gram–Schmidt process to generate an
orthonormal basis.

4
 2. The Gram–Schmidt Process
Example.1 (continued)
1 −1 1
Vector 𝑣1 = 1 = 1 , 𝑣2 = 𝑥 = 0 , 𝑣3 = 𝑥 2 = 0
1 𝑥 𝑥2
1 1 1 𝑥1 1 −1 1
𝑣 1
𝐮1 = 1 = … (𝑖) 𝑥2 1 0 0
𝑣1 1
Now, 𝑣1 = 1 = 1 2 + 1 2 + 1 2 = 3, putting in (i) 𝑥3 1 1 1
1
𝐮1 =
3
Now,
𝑣2 − 𝑝1 𝑥 − 𝑝1
𝐮2 = = … (𝑖𝑖)
𝑣2 − 𝑝1 𝑥 − 𝑝1
−1
1 1 1 1 1 1
Now, 𝑝1 = 𝑣2 , 𝐮1 𝐮1 = 𝑥, 𝐮1 𝐮1 = 0 . = −1 + 0 + 1
3 3 3 3 3 3
1
1
⇒ 𝑝1 = 0 =0
3
5
 2. The Gram–Schmidt Process
Example.1 (continued)

−1 −1
Now, 𝑣2 − 𝑝1 = 𝑥 − 𝑝1 = 0 − 0 = 0 … (𝑎)
1 1
Now, 𝑣2 − 𝑝1 = 𝑥 − 𝑝1 = −1 2 + 0 2 + 1 2 = 2 … (𝑏), putting the value of (a) and
(b) in equation (ii), we get
𝑣2 − 𝑝1 𝑥 − 𝑝1 1 −1 1
𝐮2 = = = 0 = 𝑥
𝑣2 − 𝑝1 𝑥 − 𝑝1 2 1 2
Now,
𝑣3 − 𝑝2 𝑥 2 −𝑝2
𝐮3 = = 2
… (𝑖𝑖)
𝑣3 − 𝑝2 𝑥 −𝑝2
6
 2. The Gram–Schmidt Process
Example.1 (continued)
Now, 0
𝑝2 = 𝑣3 , 𝐮1 𝐮1 + 𝑣3 , 𝐮2 𝐮2 = 𝑥 2 , 𝐮1 𝐮1 + 𝑥 2 , 𝐮2 𝐮2
1
1 1 1 1 1 1 2
Now, 𝑥 2 , 𝐮1 = 0 . = 1 + 0 + 1 = + =
3 3 3 3 3 3 3
1
1 −1
1
Also, 𝑥 2 , 𝐮2 = 0 . 0 =0
2
1 1
2 1 2
∴ 𝑝2 = 𝑥 2 , 𝐮1 𝐮1 = . =
3 3 3
1
1 3
2 2 1 2 2 2 1 2 2
Now, 𝑣3 − 𝑝2 = 𝑥 2 −𝑝2 = 0 − = − 3 , 𝑣3 − 𝑝2 = 𝑥 2 −𝑝2 = + − + =
3 3 3 3 3
1 1
3 7
 2. The Gram–Schmidt Process
Example.1 (continued)
putting the values of 𝑥 2 −𝑝2 and 𝑥 2 −𝑝2 in equation (ii), we get

1
3 1 1
𝑣3 −𝑝2 𝑥 2 −𝑝2 3 2 3 1 1
𝐮3 = = = − = * −2 = −2
𝑣3 −𝑝2 𝑥 2 −𝑝2 2 3 2 9 6
1 1 1
3

∴ the set 𝐮1 , 𝐮2 , 𝐮3 is an orthonormal basis for the given polynomial 𝑃3 .

8
 3. Gram–Schmidt QR Factorization
Theorem. 5.6.2:

Remark: We can obtain a useful factorization (A = QR) if we keep track of all the inner products
and norms computed in the Gram–Schmidt process,

9
 3. Gram–Schmidt QR Factorization

𝑎1 = 𝑟11 𝑞1
𝑟𝑘𝑘 = 𝑎𝑘 − 𝑃𝑘−1 𝑟11 𝑟12 … 𝑟1𝑛 𝑎2 = 𝑟12 𝑞1 + 𝑟22 𝑞2
For 𝑘=2,3,…, n 0 𝑟22 … 𝑟2𝑛 .
𝑅= ⋮ ⋮ ⋮ ⋮ .
𝑟𝑛𝑚 = 𝑞𝑛𝑇 𝑎𝑚 , 𝑚 = 𝑛 + 1 0 0 0 𝑟𝑛𝑛 .
For 𝑛 = 1,2, … , 𝑛 𝑎𝑛 = 𝑟1𝑛 𝑞1 + ⋯ + 𝑟𝑛𝑛 𝑞𝑛

1
𝑞𝑘 = 𝑎 − 𝑃𝑘−1
𝑟𝑘𝑘 𝑘 𝑄 = 𝑞1 , 𝑞2 , … , 𝑞𝑛
For 𝑘=2,3,…, n

10
 3. Gram–Schmidt QR Factorization
Example 3. Compute the Gram–Schmidt QR factorization of the following matrix 𝐴.
1 −2 −1
2 0 1
𝐴=
2 −4 2
4 0 0
Solution: Here, 𝒂1 = 1,2,2,4 𝑇 , 𝒂2 = −2,0, −4,0 𝑇 , 𝒂3 = −1,1,2,0 𝑇
Step.1 Set 𝑟11 = 𝒂1 = 1 2 + 2 2 + 2 2 + 4 2= 5
𝒂1 1 2 2 4 𝑇
𝒒1 = = , , ,
𝒂1 5 5 5 5
Step.2 Set 𝑟12 = 𝒒1𝑇 𝒂2 = −2
𝒑1 = 𝑟12 𝒒1 = −2𝒒1
8 4 16 8 𝑇
𝒂2 − 𝒑1 = − , ,− ,
5 5 5 5
11
 3. Gram–Schmidt QR Factorization
Example 3. (continued)
𝑟22 = 𝒂2 − 𝒑1 = 4
1 2 1 4 2 𝑇
𝒒2 = 𝒂2 − 𝒑1 = − , ,− ,
𝑟22 5 5 5 5
Step.3 Set
𝑟13 = 𝒒1𝑇 𝒂3 = 1, 𝑟23 = 𝒒𝑇2 𝒂3 = −1
3 1 6 2 𝑇
𝒑2 = 𝑟13 𝒒1 + 𝑟23 𝒒2 = 𝒒1 − 𝒒2 = , , ,
5 5 5 5
8 4 4 2 𝑇
𝒂3 − 𝒑3 = − , , ,− , 𝑟33 = 𝒂3 − 𝒑2 = 2
5 5 5 5
1 4 2 2 1 𝑇
𝒒3 = 𝒂3 − 𝒑2 = − , , ,−
𝑟33 5 5 5 5
At each step, we have determined a column of 𝑄 and a column of 𝑅. The factorization is given by
12
 3. Gram–Schmidt QR Factorization
Example 3. (continued)
1 2 4
− −
5 5 5
𝑟11 𝑟12 𝑟13 2 1 2
5 −2 1
0 𝑟22 𝑟23 = 5 5 5
A = 𝑄𝑅 = 𝒒1 , 𝒒2 , 𝒒3 2 4 2 0 4 −1
0 0 𝑟33 − 0 0 2
5 5 5
4 2 1
−
5 5 5

13
 3. Gram–Schmidt QR Factorization
Theorem 5.6.3

14
 3. Gram–Schmidt QR Factorization
Example.4:

Solution
The coefficient matrix of this system was factored in Example 3. Using that factorization, we have

15
 3. The Modified Gram–Schmidt Process
Generally, the 𝑄𝑅 method doesn’t produce accurate results when carried out with finite-precision
arithmetic. In practice, there may be a loss of orthogonality due to round-off error n computing
𝑞1 , 𝑞2 , … , 𝑞𝑛 . We can achieve better numerical accuracy by using the modified version of the
Gram-Schmidt method.

16