Numerical Methods: LU Decomposition

NPTEL ONLINE
IIT ROORKEE CERTIFICATION COURSE
NUMERICAL METHODS
LU decomposition
Dr. Sanjeev Kumar

Department of Mathematics
1
LU decomposition
Introduction
Consider the system of equations
Ax=b
Also known as the decomposition method or the factorization method.
Coefficient matrix A is decomposed into product of lower triangular and upper
triangular matrices.
We can write matrix A as
A=LU
where L is lower triangular matrix and U is upper triangular matrix given as
NPTEL ONLINE
IIT ROORKEE CERTIFICATION COURSE 2
LU Decomposition
L and U Matrices
   
l11 0 0 . . . 0 u11 u12 u13 . . . u1n
 l21 l22 0 . . . 0   0 u22 u23 . . . u2n 
   
L =  l31 l32 l33 . . . 0 , U = 0 0 u33 . . . u3n
   
  
 ..   .. 
 .   . 
ln1 ln2 ln3 . . . lnn 0 0 0 . . . unn
Usual matrix multiplication is used to multiply matrices L and U.
NPTEL ONLINE
LU decomposition
Algorithm
Comparing elements of resulting matrix LU with elements of A. We get
li1 u1j + li2 u2j + . . . + lin unj = aij , j = 1, · · · , n (1)
where lij = 0, j > i and uij = 0, i > j.
The system of equations has n2 + n unknowns.

Therefore, there are n parameter family of solutions.
To produce unique solution, we choose either uii = 1 or lii = 1, i = 1, · · · , n.
NPTEL ONLINE
LU decomposition
Doolittle and Crout’s Schemes

When we choose
lii = 1, the method is known as Doolittle’s method.
uii = 1, the method is called the Crout’s method.
If we take uii = 1, i = 1 · · · n, the system of equations (1) can be written as
j−1
X
lij = aij − lik ukj , i ≥ j
k =1
NPTEL ONLINE
LU decomposition
Crout’s method
i−1
X
uij = (aij − lik ukj )/lii , i < j (2)
k =1
uii = 1.
The first column of matrix L is identical with first column of matrix A. i.e.
li1 = ai1 , i = 1, · · · , n. (3)
Also, u1j = a1j /l11 , j = 2, · · · , n. (4)
NPTEL ONLINE
LU decomposition
Crout’s method
The first column of L and first row of U have been determined.
Now, we proceed to find second column of L and second row of U.
li2 = ai2 − li1 u12 , i = 2, · · · , n
u2j = (a2j − l21 u1j )/l22 , j = 3, · · · , n. (5)
NPTEL ONLINE
LU decomposition
Crout’s Method
Next, we will find the third column of L and third row of U.
Elements of A are to be find out by comparing with elements of matrix LU.
For relevant indices i and j, the elements are computed in order
li1 , u1j ; li2 , u2j ; li3 , u3j ; · · · ; li,n−1 , un−1,j ; lnn
NPTEL ONLINE
LU decomposition
Example: Crout’s method

Consider the matrix
    
1 2 4 l11 0 0 1 u12 u13
A =  3 8 14  =  l21 l22 0   0 1 u23 
2 6 13 l31 l32 l33 0 0 1
 
l11 l11 u12 l11 u13
=  l21 l21 u12 + l22 l21 u13 + l22 u23 
l31 l31 u12 + l32 l31 u13 + l32 u23 + l33
NPTEL ONLINE
LU decomposition
First column gives
l11 = 1; l21 = 3; l31 = 2
First row gives
u12 = 2; u13 = 4
Second column gives
l22 = 2; l32 = 2
Second row gives
u23 = 1
Third column/row gives
l33 = 3
NPTEL ONLINE
LU decomposition

Hence, LU decomposition is
    
1 2 4 1 0 0 1 2 4
A =  3 8 14  =  3 2 0   0 1 1 
2 6 13 2 2 3 0 0 1
NPTEL ONLINE
LU decomposition
Example: Doolittle’s method

Consider the matrix
    
1 2 4 1 0 0 u11 u12 u13
A =  3 8 14  =  l21 1 0   0 u22 u23 
2 6 13 l31 l32 1 0 0 u33
 
u11 u12 u13
=  l21 u11 l21 u12 + u22 l21 u13 + u23 
l31 u11 l31 u12 + l32 u22 l31 u13 + l32 u23 + u33
NPTEL ONLINE
LU decomposition
First row gives

u11 = 1; u12 = 2; u13 = 4
First column gives
l21 = 3; l31 = 2
Similarly making other rows and columns comparisons, we get
    
1 2 4 1 0 0 1 2 4
A =  3 8 14  =  3 1 0   0 2 2 
2 6 13 2 1 1 0 0 3
NPTEL ONLINE
LU decomposition
Example: Doolittle’s method

After determining matrices L and U, the system of equations becomes
LUx=b. (6)
We can write (6) as following two system of equations

Ux=z (7)
Lz=b (8)
The unknowns z1 , z2 , · · · , zn in (8) are determined by forward substitution and

unknowns x1 , x2 , · · · , xn in (7) are obtained by back substitution.
NPTEL ONLINE
Linear System
Example
    
1 2 4 x1 4
 3 8 14   x2  =  12 
2 6 13 x3 11
NPTEL ONLINE
Triangularization Method
Example
This system can be written as in triangular matrices form
     
1 0 0 1 2 4 x1 4
 3 1 0   0 2 2   x2  =  12 
2 1 1 0 0 3 x3 11
Let     
1 2 4 x1 z1
 0 2 2   x2  =  z2 
0 0 3 x3 z3
NPTEL ONLINE
Triangularization Method
Example cont...
    
1 0 0 z1 4
 3 1 0   z2  =  12 
2 1 1 z3 11
Using forward substitution, we get z1 = 4; z2 = 0; z3 = 3. Now, solving Lx = z
using back substitution
    
1 2 4 x1 4
 0 2 2   x2  =  0 
0 0 3 x3 3
We get x1 = 2; x2 = −1; x3 = 1.
NPTEL ONLINE
LU Decomposition
Inverse of a matrix
Also, L−1 and U−1 are calculated to get
z = L−1 b and x = U−1 z (9)
Thus, inverse of A can also be determined from
A−1 = U−1 L−1 . (10)
This method fails if any of diagonal elements lii or uii is zero.

So, LU decomposition is guaranteed if matrix A is positive definite.
NPTEL ONLINE
LU Decomposition
Positive Definite Matrix

A symmetric n × n matrix A is said to be positive definite if for all nonzero vectors
x, the product x T Ax > 0.
Remarks
A positive definite matrix M is invertible.
All the eigenvalues with corresponding real eigenvectors of a positive definite
matrix are positive.
A symmetric matrix is positive definite if: (i) all the diagonal entries are
positive, and (ii) each diagonal entry is greater than the sum of the absolute
values of all other entries in the corresponding row/column.
NPTEL ONLINE
Linear system
Cholesky Method
This method is also known as square-root method.
If the coefficient matrix A is symmetric and positive definite, then, it can be
decomposed as
A=LLT (11)
where L = (lij ), lij = 0, i < j
Also, A can be decomposed as A=UUT (12)
For (11), the system of equations becomes
LLT x = b (13)
NPTEL ONLINE
Linear system
Cholesky Method
(13) can be written as
LT x = z (14)
Lz = b (15)
The values zi , i = 1, · · · , n are obtained from (15) by forward substitution.
xi , i = 1, · · · , n are determined from (14) by back substitution.
Alternatively, we can also find L−1 and obtain
z=L−1 b (16)
and, x=(LT )−1 z = (L−1 )T z
NPTEL ONLINE
Linear system
Cholesky Method
The inverse of A can be obtained from (11)
A−1 =(L−1 )T L−1 (17)
The elements of lower triangular matrix L may be obtained as
i−1 1/2
X
lii = aii − lij2 , i = 1, 2, · · · , n
j=1
NPTEL ONLINE
Linear system
Cholesky Method
j−1
1 X
lij = aij − ljk lik , i = j + 1, j + 2, · · · , n; j = 1, · · · , n
lii
k =1
lij = 0, i < j (18)

The summation is taken as zero, wherever the upper limit is less than the
lower limit.
For (12), the elements uij are given by unn = (ann )1/2
uin = ain /unn , i = 1, 2, · · · , (n − 1)
NPTEL ONLINE
Linear system
Cholesky Method
n
X
uij = aij − uik ujk /ujj ,
k =j+1
i = n − 2, n − 3, · · · , 1; j = i + 1, i + 2, · · · , n − 1
Xn 1/2
2
uii = aii − uik , i = n − 1, n − 2, · · · , 1
k =i+1
uij = 0, i > j (19)
NPTEL ONLINE
Cholesky Method
Example
    
4 −1 0 0 x1 1
 −1 4 −1 0   x2   0 
  x3  =  0
    
 0 −1 4 −1 
0 0 −1 4 x4 0
We will solve the above system using Cholesky method and determine A−1 .
NPTEL ONLINE
Cholesky Method
Example
Suppose  
l11 0 0 0
 l21 l22 0 0 
L=  l31 l32 l33 0 

l41 l42 l43 l44

  
l11 0 0 0 l11 l21 l31 l41
l21 l22 0 0   0 l22 l32 l42 
A = LLT = 
  

 l31 l32 l33 0   0 0 l33 l43 
l41 l42 l43 l44 0 0 0 l44
NPTEL ONLINE
Cholesky Method
Example
2
 
l11 l11 l21 l11 l31 l11 l41
 l21 l11 2 + l2
l21 l21 l31 + l22 l32 l21 l41 + l22 l42 
A= 22 
 l31 l11 l31 l21 + l32 l22 2 2
l31 + l32 + l332 l31 l41 + l32 l42 + l33 l43 
2 + l2 + l2 + l2
l41 l11 l41 l21 + l42 l22 l41 l31 + l42 l32 + l43 l33 l41 42 43 44
Comparing corresponding elements, we get

First row: l11 = 2, l21 = −1/2, l31 = 0, l41 = 0,
NPTEL ONLINE
Cholesky Method
Example
2 + l 2 = 4, or l
p
Second row: l21 22 22 = 15/4
p
l21 l31 + l22 l32 = −1, or l32 = − 4/15
l21 l41 + l22 l42 = 0, or l42 = 0

2 + l2 + l2
p
Third row: l31 32 33 = 4, or l33 = 56/15
p
l31 l41 + l32 l42 + l33 l43 = −1, or l43 = − 15/56
2 + l 2 + l 2 + l 2 = 4, or l
p
Fourth row: l41 42 43 44 44 = 209/56.
NPTEL ONLINE
Cholesky Method
Example
The solution of system Lz=b i.e.
2 p0 0 0
    
z1 1
 −1/2 15/4 0 0   z2   0 
 p p  = 
 0 − 4/15 p56/15 p 0   z3   0 
0 0 − 15/56 209/56 z4 0
is given by z1 = 21 , z2 = √1 , z3
60
= √ 1 , z4
840
= √ 1
11704
.
NPTEL ONLINE
Cholesky Method
Example
The solution vector is obtained from LT x=z, i.e.
−1/2
2 p p0 0 1/2
    
x1 √
 0
 15/4 −
p 4/15 p0   x2
    1/ 60 
= √ 
 0 0 56/15 −
p 15/56   x3   1/ 840 
√
0 0 0 209/56 x4 1/ 11704
1 4 15 56
On back substitution, we get the solution x4 = 209 , x3 = 209 , x2 = 209 , x1 = 209 .
NPTEL ONLINE
Cholesky Method
Example
We also find
1/2 p0 0 0
 
√
 1/ 60
L−1 √ p 4/15 p 0 0 
=
 1/ 840

√ p 2/105 p 15/56 p 0 
1/ 11704 2/1463 15/11704 56/209
 
0.5 0 0 0
 0.1291 0.5164 0 0 
=
 0.0345 0.1380 0.5176

0 
0.0092 0.0370 0.1387 0.5176
NPTEL ONLINE
Cholesky Method
Example
Finally, we obtain
 
0.2679 0.0718 0.0191 0.0048
 0.0718 0.2871 0.0766 0.0192 
A−1 =(L−1 )T L−1 =
 0.0191

0.0766 0.2871 0.0718 
0.0048 0.0192 0.0718 0.2679
NPTEL ONLINE

Numerical Methods: LU Decomposition

Uploaded by

Copyright:

Available Formats

Numerical Methods: LU Decomposition

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Numerical Methods: LU Decomposition

Uploaded by

Copyright:

Available Formats

NPTEL ONLINE

IIT ROORKEE CERTIFICATION COURSE

Dr. Sanjeev Kumar

where lij = 0, j > i and uij = 0, i > j.

The system of equations has n2 + n unknowns.

Doolittle and Crout’s Schemes

Example: Crout’s method

Example: Crout’s method

Example: Doolittle’s method

First row gives

Example: Doolittle’s method

We can write (6) as following two system of equations

The unknowns z1 , z2 , · · · , zn in (8) are determined by forward substitution and

z = L−1 b and x = U−1 z (9)

Thus, inverse of A can also be determined from

A−1 = U−1 L−1 . (10)

This method fails if any of diagonal elements lii or uii is zero.

Positive Definite Matrix

A−1 =(L−1 )T L−1 (17)

The elements of lower triangular matrix L may be obtained as

lij = 0, i < j (18)

uij = 0, i > j (19)

l41 l42 l43 l44

Comparing corresponding elements, we get

l21 l41 + l22 l42 = 0, or l42 = 0

You might also like