CE 601: Numerical Methods

Lecture 5

Course Coordinator:
Dr. Suresh A. Kartha,
Associate Professor,
Department of Civil Engineering,
IIT Guwahati.
 Elimination Methods
• For a system [A]{x} = {b} where [A] is an n X n matrix,
the number of operations = (2/3)n3+(3/2)n2-(7/6)n.
• Note: The formula (2/3)n3+(3/2)n2-(7/6)n will always
yield integer values.
• Like Gauss elimination, you might have studied Gauss-
Jordan elimination method. The objective of Gauss-
Jordan scheme was to convert [A]{x} = {b} => [I]{x} =
{x}, [I] -> Identity matrix, through systematic
elimination process.
• We are not going to discuss on this method in the
class and request you to work on your our referring
literature.
• The Gauss-Jordan scheme is computationally less
efficient than the Gauss elimination scheme. The
no. of operations involved in Gauss-Jordan
scheme = n3+n2-n.
• Like Gauss elimination, Gauss-Jordan, etc. you
might have studied matrix inverse methods to
solve linear systems
i.e. [A]{x} = {b}
so, {x} = [A]-1{b}
There are two evaluations in this scheme
→ First evaluate the inverse of matrix [A]
→Second, to evaluate the product of [A]-1{b}
• In both the evaluations, there are arithmetic
operations involved. For matrix inverse it takes
2n2-n operations i.e. total of 2n3 operations.
• Based on the significant digits assigned to the
variables or components, round-off errors
may creep in the solutions while using Gauss
elimination scheme.
→ These errors can be minimized by
performing partial pivoting or scaled partial
pivoting.
• LU Decomposition
o We have discussed that matrix can be factored i.e., it can
be given as product of two different matrix.
[A] = [B][C]
o There can be many possibilities of obtaining factor
matrices.
→ However if we specify the diagonal elements of either
[L] or [U], then we will have a unique a factorization for
[A].
→The LU decomposition methods like Doolittle and Crout
work on these principles.
o In a similar tone, one can also factorize [A] as product of [L]
and [U] i.e., [A]= [L][U] where [L] is lower triangular and [U]
is a upper triangular matrix.
a11 a12 a1n l11 0 0 u11 u12 u1n
a21 a22 a2 n l21 l22 0 0 u22 u2 n

an1 an 2 ann ln1 ln 2 lnn 0 0 unn

• Property of LU decomposition
o [A]{x} = {b}
[L][U] {x} = {b}
Multiply left and right side by [L]-1
[L]-1[L][U] {x} = [L]-1 {b}
[I][U]{x} = [L]-1 {b}
[U}{x} = [L]-1 {b}
o Let us define the system [U]{x} = {c}
So, we get {c} = [L]-1 {b}
or, [L]{c} = {b}
o The meaning here is:
 Lower triangular matrix [L] will transform RHS vector
{b} to {c} using the relation [L]-1{c} = {b}.
 On obtaining {c} , using the relation [U]{x} = {c}, we
obtain the solution vector.
 So if we can suitable algorithm such that process
involved in LU decomposition are”
 Factorize [A] = [L][U]
 Forward substitution to evaluate {c}
 Backward substitution to evaluate {x]
 If the no. of arithmetic operations in factorization can
be reduced then LU decomposition becomes
efficient.
• The Advantage of LU Decomposition
o If there are many linear systems with same
coefficient matrix i.e.,
[A]{x} = {b}
[A]{y} = {c}
[A]{z} = {d}
o Then if we adopt Gauss elimination, for each
system it may take (2/3)n3+(3/2)n2-(7/6)n
operations. The purpose may become tedious.
o If we do LU decomposition, then you may see that
only once we need to decompose [A]=[L][U]
o After that for each system, we need to utilize only
the forward and backward substitutions.
• Doolittle LU Decomposition
• [A] = [L][U]
• If we factorize in such a way that
a11 a12 a1n 1 0 0 u11 u12 u1n
a21 a22 a2 n l21 1 0 0 u22 u2 n

an1 an 2 ann ln1 ln 2 1 0 0 unn

i.e. diagonal elements of [L] are 1, then the

approach is Doolittle’s method.
• If l11 0 0 1 u12 u1n
l21 l22 0 0 1 u2 n
[ A]

ln1 ln 2 lnn 0 0 1

the approach is called Crout’s method.

Algorithm for Doolittle’s LU Decomposition
• Doolittle algorithm is developed using
knowledge of Gauss elimination.
• Recall Gauss elimination at any step k
aij( k ) aij( k 1)
lik akj( k 1)

k 1, 2,3,..., n 1; j k , k 1,..., n; i k 1, k 2,..., n

( k 1) ( k 1)
lik a
ik a kk

• You can now write

aij( k ) aij( k 1)
lik akj( k 1)

i.e., aij( k ) aij( k 2)

lik akj( k 1)
li ( k 1) a((kk 2)
1) j
k
(k ) (m 1)
Extending, a ij aij lim amj
m 1
k
(k ) ( m 1)
or, aij a ij lim amj ;i k 1, k 2,..., n; j k , k 1,..., n
m 1
• This is nothing but
[ A] [ L][U ]
k
(k ) (m 1)
aij aij lim amj
m 1

where lij elements of lower triangular matrix

lik ; for i k ; k 1, 2,3,..., n 1
such that lij 1 ; for i j
0 ; for i j

• So you get 1 0 0 0 a11 a12 a13 a1n

(1) (1)
l21 1 0 0 0 a22 a23 a2(1)n
(2)
[ L] l31 l32 1 0 ,[U ] 0 0 a33 a3(2)
n

(n 1)
ln1 ln 2 ln 3 1 0 0 0 ann
(1) (1)
where l21 a21 a11 , l32 a32 a22 , etc.
• So the steps involved are:
o No. of steps as in Gauss elimination
k = 1,2,3,…,n-1
o At any k , i= k+1,k+2,…,n and j=k,k+1,…,n
lkk 1
lik 0; i k
uij aij( k ) aij( k 1)
lik akj( k 1)

uij 0
lik aik( k 1)
akk( k 1)
;i k
o Forward substitution for c
c1 b1
k 1
ci bi lim cm ; i 2,3, ,n
m 1

o Back substitution for x

xn cn unn
cn 1 u( n x
1) n n
xn 1
u( n 1)( n 1)
n
ci uij x j
j i 1
In genral, xi ;i (n 1),(n 2),..., 2,1
uii
 80 20 20 x1 20
• Example 20 40 20 x2 20
20 20 130 x3 20
A L U
1 0 0 u11 u12 u13
L l21 1 0 ,U 0 u22 u23
l31 l32 1 0 0 u33
l21 1 / 4, l31 1 / 4, l32 5/7
u11 80, u12 20, u13 20
u21 0, u22 40 ( 1 / 4)( 20) 35
u23 20 ( 1 / 4)( 20) 25
u31 u32 0
k 2
(m 1)
u33 130 l3m amj
m 1

750 / 7
Forward Substitution:
c1 b1 20
c2 20 ( 20 / 4) 25
c3 20 ( 20 / 4) ( 5 25 / 7) 300 / 7
[U ]{x} {c}
80 20 20 x1 20
i.e., 0 35 25 x2 25
0 0 750 / 7 x3 300 / 7

Now you can do back substitution easily.

• Number of operations
• In Doolittle’s algorithm,
o to evaluate lik for any kth step for i> k, it takes
(n-k) operations.
o To evaluate uij = aij(k) it takes 2(n-k)2
operations.
o In (n-k) elimination steps to form [L] and [u],
it takes, n 1 (n p)(2n 2 p 1)
p 1

n 2 3 n2 n
(n 1)(4n 1) n
6 3 2 6
o To perform forward substitution,
as c1 = b1 (No operation required)
i 1
ci bi limcm ; i 2,3,4, , n
m 1

There are 2i operations for each i.

n
2
So, no. of operations = 2i n n
i 2

o To perform backward substitution, it requires

n2 operations (as in Gauss)
2 3 n2 n
o Total no. of operations = n 2n 2 n
3 2 6
2 3 3 2 7
n n n
3 2 6
• No. of operations is same as Gauss elimination
method.
• However if there are many systems involving
[A], then you may need to just add the no. of
operations for forward & backward
substitution for each system.