23MAT112

Mathematics for Intelligent Systems – 2

Practice Sheet – 4
LU, QR, SVD, Polar Decompositions

Eigenvalues, eigenvectors, Similar matrices, Diagonalisation

 A=[1,2;5,4]; E = eig(A)
% produces a column vector E containing the eigenvalues of a square matrix A.
 A=[1,2;5,4]; [V,D] = eig(A)
% produces a diagonal matrix D of eigenvalues and a full matrix V whose columns are the
corresponding eigenvectors so that A*V = V*D.
 A= [ 1 2 3 ; 2 3 4; 3 5 8];
syms x;
charpoly(A,x)
% Command for obtaining characteristic polynomial of a matrix A in terms of x.

 If B=M-1AM, then A and B are similar. Similar matrices have same eigenvalues.

A=randi([0,9],5,5);
M=randi([0,9],5,5);
B=inv(M)*A*M;
EvalA=eig(A)
EvalB=eig(B)

 AB and BA are similar and so have same non-zero eigenvalues

A=randi([0,9],5,3);
B=randi([0,9],3,5);
EvalAB=eig(A*B)
EvalBA=eig(B*A)

 ATA and AAT are similar and have same non-zero eigenvalues.

A=randi([0,9],7,5);
EvalATA=eig(A'*A)
EvalAAT=eig(A*A')

 Application of diagonalization

Ld1=1;
Ld2=2;
Ld3=3;
Ev1=[1;1;1];
 Ev2=[2;1;-1];
Ev3=[1;2;-1];
V=[Ev1,Ev2,Ev3]
D=diag([Ld1,Ld2,Ld3])
A=V*D*inv(V)
% Generates a matrix A with given eigenvalues Ld1,Ld2,Ld3 and independent eigenvectors
Ev1,Ev2 and Ev3.

Cayley Hamilton theorem: Every square matrix satisfy its own characteristic equation. It has two forms:

Consider 3x3 matrix A. Let its characteristic equation be .

1.

2.
Here are distinct (assumed) non-zero eigenvalues of matrix A

It can be verified that

This result can be used to find 2 matrices whose product has a rank less than the rank of both the matrices.

Another observation:

Consider 3x3 lower triangular matrix with rank 2 and with all eigen values as zero.

, then
 LU Decomposition using MATLAB
>>[L,U] = lu(A)
% returns an upper triangular matrix in U and a permuted lower triangular matrix in L, such
that A = L*U. The input matrix A can be full or sparse.

 Using MATLAB we can find the QR decomposition of a matrix A by:

>> [Q,R]=qr(A,0)

 QR Decomposition for finding eigenvalues of matrices numerically :

A=randi(10,2,2);
eig(A)
B=A;
for i= 1:10
[Q R]=qr(B,0); % B=QR.
B=R*Q;
% Creating a new B=RQ and repeat the computation. B=RQ retains eigenvalues of A (as RQ is similar
to A) but approaches to an upper triangular matrix. That is, diagonal elements of B approaches to
eigenvalues of A.
end
diag(B)
% Note that eig(A) and diag(B) are almost the same values.

 Singular Value Decomposition using MATLAB

>>[U,Z,V] = svd(A)
% produces a diagonal matrix Z, of the same dimension as A and with nonnegative diagonal elements in
decreasing order, and unitary matrices U and V so that A = U*Z*V'.
>>Z = svd(A)
% returns a vector containing the singular values.

 Understanding of SVD through computational experiments using MATLAB

[ ]
9 10
1. First find the SVD of a matrix, A= 10 7 .
2 1
>> format bank % display results in 2 decimal places
>>A=[9 10; 10 7; 2 1];
>>[U Z V]=svd(A); % full SVD

2. Verify that U and V are orthogonal by evaluating the following:

>> U’*U
>>U*U’
 >> V’*V
>>V*V’

3. Compute eigenvectors of ATA and compare with the columns of V.

>> [V1, L1]=eig(A’*A)
>>V
% Notice that the column vectors of V1 and V are the same, that is the eigenvectors of ATA.
4. Compute eigenvectors of AAT and compare with the columns of U.
>> [U1, L]=eig(A*A’)
>>U
% Notice that the column vectors of U1 and U are the same, that is the eigenvectors of AAT.

5. Compute the eigenvalues of ATA and AAT and compare with the singular values of A.
>> eig(A’*A)
>>eig(A*A’)
>>S=diag(Z) % singular values of A are diagonal elements of Z
>>Ssquare=S.^2 % square of singular values
% Notice that the square of singular values of A are same as the non-zero eigenvalues of A TA and AAT.

6. Given the SVD of a matrix is A=U∑ V T , the pseudo-inverse of A can be decomposed as A+ = V ∑ . ¿

+¿

UT
>> A=[1,1,0;0,1,1];[ u,z,v]=svd(A)
>> pseudoA=pinv(A);
>> v*pinv(z)*u’ % Verify if this is same as pseudoA

Practice questions

1) Find a matrix A whose eigenvalues are 24, 10, 5 with corresponding eigenvectors as [1,1,0]T, [1,-1,0]T
and [0,0,1]T Also use diagonalization to find A5.

[ ]
1 3
0 4
2) Find the eigenvalues of AAT and ATA and verify both matrices are similar if A= .
1 6
2 7

[ ]
−4 1 1
3) Consider the matrix, A= 1 −6 7 . Find two matrices B and C that are similar to matrix A and
1 7 −9
verify their eigenvalues are same.
4) Create a 5x5 non-diagonal matrix with eigenvalues as 9,8,7,6,5.
[Hint: Create a similar matrix using a non singular matrix B and then
A=B*diag([9,8,7,6,5])*inv(B)]
5) Using MATLAB generate a 10 ×5 matrix A of rank 3.
(a) Obtain a symmetric matrix S1=A*AT and S2=AT*A
(b) Find the rank of S1 and S2.
 (c) Find the eigenvalues of S1 and S2. What do you notice about them? Explain.
6) Consider the following matrices:

( ) [ ] [ ]
1 3 4 7
0 1 0 1 2 01
A= 2
3 5 8 13 [ ]
4 6 10 , B= 3 4
6 8
,C= 0 0 1,
0 0 0
D= 0 1 1 0 , E=
2 4 02
0 14 0
0 28 0 [ ]
4 6 10 16

F¿ [ 94 35].
(a) Find the eigenvalues of the matrices and determine the definiteness.
(b) Find the SVD of these matrices.

7) Using QR decomposition find the eigenvalues of a random integer symmetric matrix of order 15 and
verify the result using direct evaluation of eigenvalues.
8) Generate a random integer 7 by 9 matrix A of rank 6.
(a) Verify the rank of A is 6
(b) Find the SVD of the matrix
(c) Using SVD find an orthonormal basis for all the fundamental subspaces of the matrix A
(d) Verify the results obtained in (c) by checking the orthogonality of RS & NS and CS & LNS.
(e) Find the singular values of A
(f) Write a MATLAB code to obtain the singular values of A without using the command ‘svd’.
(g) Find A+ , the pseudo-inverse of A and show the decomposition of it
9) Using QR decomposition find the eigenvalues of the following matrices using 50 iterations in
MATLAB.
(a) 4th order Pascal’s matrix

[ ]
1 2 1
(b) A¿ 2 1 3
5 7 0

[ ]
1 −1 0 0
−1 2 −1 0
(c) B=
0 −1 2 −1
0 0 −1 2
(d) a random 50 by 50 matrix
10) Find the pseudo inverse of A (in question 5, 6 and 7) and SVD of A. Then verify that:
11) Generate a 5 by 3 matrix of rank 3. Using SVD find a basis for the left null space of the matrix and the
null space of the matrix.
12) Generate a 6 by 5 matrix of rank 4. Using SVD find a basis for all the fundamental subspaces of matrix
A
13) Solve 9x-2y+3z =1, 4x+9y-5z=3, 6x-y-z=3; using LU decomposition.
14) Express the Pascal matrix of order 7 as LU using elementary matrices, by reducing it to upper triangular
form manually.