Exercises Set 6: Exercises for Lecture 6

1. Show that 𝐀 and 𝐀 have the same nonzero singular values. How are their singular value
decompositions related?

2. Use the method discussed in the lecture to find the FULL singular value decomposition of
each of the following matrices.

1 1 2 −2 −2 0 1 2
a) , b) , c) , d) ,
2 2 1 2 0 −1 2 1
1 3 2 0 0
0 2
3 1 0 2 1
e) 0 0 , f) , g) .
0 0 0 1 2
0 0
0 0 0 0 0

3. For the matrices in Question 2, write down each matrix as a sum of rank-one matrices
associated with the singular values and the singular vectors.

4. For each of the matrices in Question 2:

i) Determine the rank;
ii) Determine the 2-norm ‖⋅‖ ;
iii) Determine the Frobenius norm ‖⋅‖ .

5. Given

−2 8 20 − 0 30 0 0 ⎛ ⎞
𝐀 = 14 19 10 = 0 0 15 0 ⎜ ⎟.
2 −2 1 0 0 3
0 0 1
⎝ ⎠
1. Determine ‖𝐀‖ ;
2. Determine the value of 𝑐𝑜𝑛𝑑 (𝐀);
3. Determine ‖𝐀‖ and ‖𝐀 ‖ ;
4. Determine the value of 𝑐𝑜𝑛𝑑 (𝐀);

6. Matrix
0 1
𝐀= 1 1 .
1 0
has singular value decomposition (SVD)

√ √ √
⎛ ⎞ √3 0 √ √
⎜√ 0 0 1 .
√ ⎟
0 0 √ √
⎝√ √ √ ⎠
a) What is the dimension of the column space of A ?
b) Use the SVD to calculate

1
 i) 𝑐𝑜𝑛𝑑 (𝐀) ;
ii) ‖𝐀‖ .
iii) ‖𝐀‖ .

7. The matrix
2 5 4
6 3 0
𝐀= .
6 3 0
2 5 4

has singular value decomposition

⎛ ⎞ 12 0 0
⎟ 0 6 0 ⎛ ⎞
⎟.
⎜
⎜ ⎟ 0 0 0 ⎜
0 0 0
⎝ ⎠
⎝ ⎠

a) Use the singular value decomposition to find orthonormal bases for the column space of
𝐀 and the nullspace of 𝐀.
b) Use the singular value decomposition to find orthonormal bases for the column space of
𝐀 and the nullspace of 𝐀 .

8. If 𝐀 is a 5 × 3 matrix with ‖𝐀‖ = 8, 𝑐𝑜𝑛𝑑 (𝐀) = 2 and ‖𝐀‖ = 12, determine the singular
values of 𝐀.

9. Given
1 2 6
𝐀= and 𝐛= ,
−1 −2 −4
1
The eigenvalues of 𝐀 𝐀 are 𝜇 = 10, 𝜇 = 0, with associated eigenvectors 𝐯 = and
2
−2
𝐯 = respectively.
1
a) Compute the reduced singular value decomposition of 𝐀
b) Find the minimum norm solution of the system 𝐀𝐱 = 𝐛 ;
c) Find all solutions to the least squares problem 𝐀𝐱 = 𝐛.

10. The matrix

1 2
𝐀 = −1 −2 .
1 2
has singular value decomposition
√
0
√ √
⎛ ⎞ √15 0 √ √
⎜− √ 0 0 .
√ √ ⎟ −
0 0 √ √
−
⎝ √ √ √ ⎠
a) Let 𝐛 = (6, −3,3) . Compute the minimum norm solution of the overdetermined
system 𝐀𝐱 = 𝐛.
b) Write down the general form of other solutions to the system if there exists any.

2
11. Let 𝐀 be an m  n matrix with singular value decomposition 𝐔Σ𝐕 . Show that
‖𝐀𝐱‖
𝑚𝑖𝑛 =𝜎 .
𝐱 𝟎 ‖𝐱‖

12. Let 𝐀 be an 𝑚 × 𝑛 matrix with singular value decomposition 𝐔Σ𝐕 . Show that for any
vector 𝐱 ∈ 𝑅 ,
𝜎 ‖𝐱‖ ≤ ‖𝐀𝐱‖ ≤ 𝜎 ‖𝐱‖ .

13. Prove that if 𝐀 is a symmetric matrix with eigenvalues 𝜆 , 𝜆 , … , 𝜆 , then the singular values
of 𝐀 are |𝜆 |, |𝜆 |, … , |𝜆 |.

14. Let 𝐀 be an 𝑚 × 𝑛 matrix of rank 𝑛 with singular value decomposition 𝐔Σ𝐕 . Let 𝚺 denote
the 𝑛 × 𝑚 matrix
1
𝐎
⎛𝜎 ⎞
⎜ 1 ⎟
⎜ 𝜎 ⎟
⎜ ⎟
⎜ ⋱ ⎟
1
𝐎
⎝ 𝜎 ⎠

and define 𝐀 = 𝐕𝚺 𝐔 . Show that 𝐱 = 𝐀 𝐛 satisfies the normal equations 𝐀 𝐀𝐱 = 𝐀 𝐛.

15. Singular value decomposition for UNDERDETERMINED matrices: Let 𝐀 be an 𝑚 × 𝑛 matrix

(with 𝑚 ≤ 𝑛 ) and 𝐀 = 𝐔𝚺𝐕 is the singular value decomposition of 𝐀.
a) Justify the following
i) the diagonal entries of 𝚺 are the square roots of the eigenvalues of 𝐀𝐀 ;
ii) the columns of 𝐔 are orthonormal eigenvectors of 𝐀𝐀 ;
b) Suppose 𝐀𝐀 has 𝑟 nonzero eigenvalues 𝜇 , 𝜇 , … , 𝜇 , where 𝑟 ≤ 𝑚 .
i) By considering 𝐀 = 𝐕𝚺𝐔 , show that the first 𝑟 columns of 𝐕 are given by
𝐯 = 𝐀 𝐮 ,         𝑖 = 1,2, … , 𝑟 ,
where 𝐮 is the i the column of 𝐔 and 𝜎 = 𝜇 .
ii) Show that the last 𝑛 − 𝑟 columns of 𝐕 forms an orthonormal basis for the null space
of 𝐀.

16. Use results in Exercise 14 to determine the FULL SVD of the following matrices:
1 1 0 1
1 0 1 0
a) , b) 0 0 0 1 .
0 1 0 1
1 1 0 0