Chapter.

6 Linear Equations
I. Gaussian Elimination
I.1 Introduction of the method

The goal of this method is to

solve linear equations of the
form [A][X]=[C]
The goal of forward elimination is to transform the
coefficient matrix into an upper triangular matrix
 25 5 1  x1  106.8 
 64 8 1  x  = 177.2 
   2  
144 12 1  x3  279.2

25 5 1   x1   106.8 
 0 − 4.8 − 1.56  x  =  − 96.21
   2  
 0 0 0.7   x3   0.735 
 Forward Elimination
A set of n equations and n unknowns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
. .
. .
. .
an1x1 + an 2 x2 + an3 x3 + ... + ann xn = bn

(n-1) steps of forward elimination

 Forward Elimination
Step 1
For Equation 2, divide Equation 1 by a11 and
multiply by a21 .

 a21 
 a (a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1 )
 11 
a21 a21 a21
a21 x1 + a12 x2 + ... + a1n xn = b1
a11 a11 a11
 Forward Elimination
Subtract the result from Equation 2.
a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
a21 a21 a21
− a21 x1 + a a12 x2 + ... + a a1n xn = a b1
_________________________________________________
11 11 11

 a21   a21  a21

 a22 − a12  x2 + ... +  a2 n − a1n  xn = b2 − b1
 a11   a11  a11

or a x + ... + a x = b
'
22 2
'
2n n
'
2
 Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
'
a32 x2 + a33
'
x3 + ... + a3' n xn = b3'
. . .
. . .
. . .

an' 2 x2 + an' 3 x3 + ... + ann

'
xn = bn'

End of Step 1
 Forward Elimination
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
"
a33 x3 + ... + a3" n xn = b3"
. .
. .
. .

an" 3 x3 + ... + ann

"
xn = bn"
End of Step 2
 Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
a x + ... + a x = b
"
33 3
"
3n n
"
3
. .
. .
. .

( n −1) (n −1 )
ann xn = bn

End of Step (n-1)

Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
0 a '
a ' '
 a 2 n x2     b2' 
 22 23    
0 0 a "
33  a"3n   x3  =  b3" 
    
           
 0 0 0 (n −1 )
0 ann   xn  bn(n-1 ) 
 Back Substitution
Solve each equation starting from the last equation

25 5 1   x1   106.8 
 0 − 4.8 − 1.56  x  =  − 96.21
   2  
 0 0 0.7   x3   0.735 

Example of a system of 3 equations

Back Substitution Starting Eqns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
"
a33 x3 + ... + an" xn = b3"
. .
. .
. .

( n −1) (n −1 )
ann xn = bn
 Back Substitution
Start with the last equation because it has only one unknown
( n −1)
b
xn = n
( n −1)
a nn
 Back Substitution
( n −1)
b
xn = n
( n −1)
a nn

bi(i −1) − ai(,ii−+11) xi +1 − ai(,ii−+12) xi + 2 − ... − ai(,in−1) xn

xi = (i −1) for i = n − 1,...,1
aii

(i −1)
−  aij(i −1) x j
n
bi
j =i +1
xi = (i −1) for i = n − 1,...,1
a ii
 Example 1
The upward velocity of a rocket is given at three
different times

Table 1 Velocity vs. time data.

Time, t (s ) Velocity, v (m/s )

5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

v(t ) = a1t 2 + a 2 t + a3 , 5  t  12.

Find the velocity at t=6 seconds .
 Example 1 Cont.
Assume
v(t ) = a1t 2 + a2t + a3 , 5  t  12.

Results in a matrix template of the form:

t12 t1 1  a1   v1 
 2   a  = v 
t 2 t2 1  2  2
t32 t3 1 
 a3 
  v3 

 

Using data from Table 1, the matrix becomes:

 25 5 1  a1  106.8 
 64 8 1 a  = 177.2 
   2  
144 12 1  a3  279.2
 Example 1 Cont.
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a 2  = 177.2    64 8 1  177.2 
       
144 12 1  a 3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
 Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n−1)=(3−1)=2
 Forward Elimination: Step 1
 25 5 1  106.8  Divide Equation 1 by 25 and
 64 8 1  177.2 
  64
multiply it by 64, = 2.56 .
144 12 1  279.2 25
25 5 1  106.8 2.56 = 64 12.8 2.56  273.408

. 64 8 1  177.2
Subtract the result from − 64 12.8 2.56  273.408
Equation 2
0 − 4.8 − 1.56  − 96.208

 25 5 1  106.8 
Substitute new equation for  0 − 4.8 − 1.56  − 96.208
Equation 2  
144 12 1  279.2 
Forward Elimination: Step 1 (cont.)
 25 5 1  106.8 
Divide Equation 1 by 25 and
 0 − 4.8 − 1.56  − 96.208
  144
144 12 1  279.2  multiply it by 144, 25 = 5.76 .

25 5 1  106.8 5.76 = 144 28.8 5.76  615.168

Subtract the result from 144 279.2

.
12 1 
Equation 3 − 144 28.8 5.76  615.168
0 − 16.8 − 4.76  − 335.968
25 5 1  106.8 
Substitute new equation for  
Equation 3  0 − 4. 8 − 1.56  − 96.208 
 0 − 16.8 − 4.76  − 335.968
 Forward Elimination: Step 2
25 5 1  106.8  Divide Equation 2 by −4.8
 0 − 4.8 − 1.56  − 96.208 
  and multiply it by −16.8,
 0 − 16.8 − 4.76  − 335.968 − 16.8
= 3.5 .
− 4.8
0 − 4.8 − 1.56  − 96.208 3.5 = 0 − 16.8 − 5.46  − 336.728
0 − 16.8 − 4.76  335.968
Subtract the result from
Equation 3 − 0 − 16.8 − 5.46  − 336.728
0 0 0.7  0.76

25 5 1  106.8 
Substitute new equation for  0 − 4.8 − 1.56  − 96.208
Equation 3  
 0 0 0.7  0.76 
Back Substitution
 Back Substitution
25 5 1  106.8  25 5 1   a1   106.8 
 0 − 4.8 − 1.56  − 96.2   0 − 4.8 − 1.56 a  = − 96.208
     2  
 0 0 0.7  0.7   0 0 0.7   a3   0.76 

Solving for a3
0.7a3 = 0.76
0.76
a3 =
0.7
a3 = 1.08571
 Back Substitution (cont.)
25 5 1   a1   106.8 
 0 − 4.8 − 1.56 a  = − 96.208
   2  
 0 0 0.7   a3   0.76 

Solving for a2
− 4.8a2 − 1.56a3 = −96.208
− 96.208 + 1.56a3
a2 =
− 4.8
− 96.208 + 1.56 1.08571
a2 =
− 4.8
a2 = 19.6905
 Back Substitution (cont.)
25 5 1   a1   106.8 
 0 − 4.8 − 1.56 a  = − 96.2
   2  
 0 0 0.7   a3   0.76 

Solving for a1
25a1 + 5a2 + a3 = 106.8
106.8 − 5a2 − a3
a1 =
25
106.8 − 5  19.6905 − 1.08571
=
25
= 0.290472
Gaussian Elimination Solution
 25 5 1  a1  106.8 
 64 8 1 a2  = 177.2 
     
144 12 1  a3  279.2

 a1  0.290472
a  =  19.6905 
 2  
 a3   1.08571 
 Example 1 Cont.
Solution  a1  0.290472
a  =  19.6905 
The solution vector is
 2  
 a3   1.08571 

The polynomial that passes through the three data points is then:

v(t ) = a1t 2 + a2t + a3

= 0.290472t 2 + 19.6905t + 1.08571, 5  t  12

v(6) = 0.290472(6) + 19.6905(6) + 1.08571

2

= 129.686 m/s.
Be careful!! Division by zero
10 x2 − 7 x3 = 3
6 x1 + 2 x2 + 3x3 = 11
5 x1 − x2 + 5 x3 = 9

0 10 − 7  x1   3 
6 2 3   x2  = 11
    
5 − 1 5   x3   9 
Is division by zero an issue here?
12 x1 + 10 x2 − 7 x3 = 15
6 x1 + 5 x2 + 3x3 = 14
5 x1 − x2 + 5 x3 = 9

12 10 − 7  x1  15
6 5 3   x2  = 14
    
 5 − 1 5   x3   9 
 Is division by zero an issue here?
YES
12 x1 + 10 x2 − 7 x3 = 15
6 x1 + 5 x2 + 3x3 = 14
24 x1 − x2 + 5 x3 = 28

12 10 − 7  x1  15 12 10 − 7  x1   15 

6 5 3   x2  = 14  0
     0 6.5  x2  =  6.5
    
24 − 1 5   x3  28 12 − 21 19   x3   − 2

Division by zero is a possibility at any step

of forward elimination
!!! Large Round-off Errors
 20 15 10  x1   45 
− 3 − 2.249 7   x  = 1.751
   2  
 5 1 3   x3   9 

Exact Solution

 x1  1
 x  = 1
 2  
 x3  1
  20 15 10  x1   45 
− 3 − 2.249 7   x  = 1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 6 significant digits with chopping

 x1   0.9625 
 x  =  1.05 
 2  
 x3  0.999995
  20 15 10  x1   45 
− 3 − 2.249 7   x  = 1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 5 significant digits with chopping

 x1   0.625 
 x  =  1.5 
 2  
 x3  0.99995

Is there a way to reduce the round off error?

 Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero
 Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting

• Avoids division by zero
• Reduces round off error
I.2 Gauss Elimination with
Partial Pivoting
 Pitfalls of Naïve Gauss Elimination
• Possible division by zero
• Large round-off errors
 Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero
 Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
 What is Different About Partial
Pivoting?
At the beginning of the kth step of forward elimination,
find the maximum of

akk , ak +1,k ,................, ank

If the maximum of the values is a pk
in the p th row, k  p  n, then switch rows p and k.
Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' a '
 ' 
a 2 n x2   ' 
b2
 22 23    
 0 a '
32 a '
33  a3' n   x3  =  b3' 
    
           
 0 a'n 2 an' 3 an' 4 '
ann   xn  bn' 
 Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0 − 7 6 1 2   x2   − 6
    
0 4 12 1 11  x3  =  8 
    
0 9 23 6 8   x4   9 
0 − 17 12 11 43  x5   3 

Which two rows would you switch?

Example (2nd step of FE)

6 14 5.1 3.7 6   x1   5 
0 − 17 12 11 43  x   3 
  2   
0 4 12 1 11  x3  =  8 
    
 0 9 23 6 8   x 4    9
0 − 7 6 1 2   x5   − 6

Switched Rows
 Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution
 Forward Elimination
Same as naïve Gauss elimination method
except that we switch rows before each
of the (n-1) steps of forward elimination.
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' a '
 ' 
a 2 n x2   ' 
b2
 22 23    
 0 a32'
a '
33  a3n   x3  =  b3 
' '

    
           
 0 a'n 2 a '
n3 a '
n4
'
   
ann   xn  bn 
'
Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
 0 a' a ' '
 a 2 n x2     b2' 
 22 23    
0 0 a "
33  a3n   x3  =  b3 
" "

    
           
 0 0 0 (n −1 )
   
0 ann   xn  bn 
(n-1 )
Back Substitution Starting Eqns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
"
a33 x3 + ... + an" xn = b3"
. .
. .
. .

( n −1) (n −1 )
ann xn = bn
 Back Substitution
( n −1)
b
xn = n
( n −1)
a nn

(i −1) n
(i −1)
bi −  aij x j
j =i +1
xi = (i −1) for i = n − 1,...,1
a ii
 Example 2
Solve the following set of equations
by Gaussian elimination with partial
pivoting

 25 5 1  a1  106.8 
 64   
8 1 a 2 = 177.2  
     
144 12 1  a 3  279.2
 Example 2 Cont.
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a  = 177.2    
    2   64 8 1  177. 2 
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
 Number of Steps of Forward
Elimination

Number of steps of forward elimination is

(n−1)=(3−1)=2
 Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.

 25 5 1  106.8  144 12 1  279.2

 64 8 1  177.2    64 8 1  177.2 
   
144 12 1  279.2  25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 64 8 1  177.2  64
  multiply it by 64, = 0.4444.
 25 5 1  106.8  144

144 12 1  279.2 0.4444 = 63.99 5.333 0.4444  124.1

.
Subtract the result from 64 8 1  177.2
Equation 2 − 63.99 5.333 0.4444  124.1
0 2.667 0.5556  53.10

Substitute new equation for 144 12 1  279.2

Equation 2  0 2.667 0.5556  53.10
 
 25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 0 2.667 0.5556  53.10 25
  multiply it by 25, = 0.1736.
 25 5 1  106.8  144

144 12 1  279.2 0.1736 = 25.00 2.083 0.1736  48.47

.
25 5 1  106.8
Subtract the result from
Equation 3 − 25 2.083 0.1736  48.47
0 2.917 0.8264  58.33

Substitute new equation for 144 12 1  279.2

Equation 3  0 2.667 0.5556  53.10
 
 0 2.917 0.8264  58.33
 Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.

144 12 1  279.2 144 12 1  279.2

 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33
   
 0 2.917 0.8264  58.33  0 2.667 0.5556  53.10
 Forward Elimination: Step 2 (cont.)
Divide Equation 2 by 2.917 and
144 12 1  279.2
 0 2.917 0.8264  58.33 multiply it by 2.667,
  2.667
 0 2.667 0.5556  53.10 = 0.9143.
2.917
0 2.917 0.8264  58.33 0.9143 = 0 2.667 0.7556  53.33

.
0 2.667 0.5556  53.10
Subtract the result from − 0 2.667 0.7556  53.33
Equation 3
0 0 − 0.2  − 0.23

144 12 1  279.2 
Substitute new equation for  0 2.917 0.8264  58.33 
Equation 3  
 0 0 − 0.2  − 0.23
Back Substitution
 Back Substitution
144 12 1  279.2  144 12 1   a1   279.2 
 0 2.917 0.8264  58.33    0 2.917 0.8264 a  =  58.33 
     2  
 0 0 − 0.2  − 0.23  0 0 − 0.2   a3  − 0.23

Solving for a3
− 0.2a3 = −0.23
− 0.23
a3 =
− 0.2
= 1.15
 Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a  =  58.33 
   2  
 0 0 − 0.2   a3  − 0.23

Solving for a2
2.917a2 + 0.8264a3 = 58.33
58.33 − 0.8264a3
a2 =
2.917
58.33 − 0.8264 1.15
=
2.917
= 19.67
 Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a  =  58.33 
   2  
 0 0 − 0.2   a3  − 0.23

Solving for a1
144a1 + 12a2 + a3 = 279.2
279.2 − 12a2 − a3
a1 =
144
279.2 − 12 19.67 − 1.15
=
144
= 0.2917
Gaussian Elimination with Partial
Pivoting Solution

 25 5 1  a1  106.8 
 64 8 1 a  = 177.2 
   2  
144 12 1  a3  279.2

 a1  0.2917
a  =  19.67 
 2  
 a3   1.15 
Gauss Elimination with
Partial Pivoting
Another Example
 Partial Pivoting: Example
Consider the system of equations
10 x1 − 7 x2 = 7
− 3x1 + 2.099 x2 + 6 x3 = 3.901
5 x1 − x2 + 5 x3 = 6
In matrix form
 10 − 7 0  x1   7 
− 3 2.099 6  x  3.901
   2 =  
 5 − 1 5  x3   6 

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping
 Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.

Performing Forward Elimination

 10 − 7 0  x1   7  10 −7 0  x1   7 
− 3 2.099 6  x  = 3.901

 5
 2   
− 1 5  x3   6 
  0 − 0.001 6  x  = 6.001

 0 2.5
 2   
5  x3   2.5 
 Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3

Performing the row swap

10 −7 0  x1   7  10 −7 0  x1   7 
 0 − 0.001 6  x  = 6.001

 0 2.5
 2   
5  x3   2.5 
 0
 2 .5 5   x  =  2.5 
 2   
 0 − 0.001 6  x3  6.001
 Partial Pivoting: Example
Forward Elimination: Step 2

Performing the Forward Elimination results in:

10 − 7 0   x1   7 
 0 2.5 5   x  =  2.5 
  2   
 0 0 6.002  x3  6.002
 Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution

10 − 7
6.002
0   x1   7  x3 = =1
 0 2.5 5   x  =  2.5  6.002
  2   
 0 0 6.002  x3  6.002 2.5 − 5 x3
x2 = = −1
2.5

7 + 7 x 2 − 0 x3
x1 = =0
10
 Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

 x1   0   x1   0 
X  calculated =  x2  = − 1 X  exact =  x 2  = − 1
 x3   1   x3   1 
 II. LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is

needed.

http://numericalmethods.eng.usf.edu
 LU Decomposition
Method
For most non-singular matrix [A] that one could conduct Naïve Gauss
Elimination forward elimination steps, one can always write it as

[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix

http://numericalmethods.eng.usf.edu
How does LU Decomposition work?
If solving a set of linear equations [A][X] = [C]
If [A] = [L][U] then [L][U][X] = [C]
Multiply by [L]-1
Which gives [L]-1[L][U][X] = [L]-1[C]
Remember [L]-1[L] = [I] which leads to [I][U][X] = [L]-1[C]
Now, if [I][U] = [U] then [U][X] = [L]-1[C]
Now, let [L]-1[C]=[Z]
Which ends with [L][Z] = [C] (1)
and [U][X] = [Z] (2)

http://numericalmethods.eng.usf.edu
 LU Decomposition
How can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]

http://numericalmethods.eng.usf.edu
Method: [A] Decomposes to [L] and [U]
1 0 0 u11 u12 u13 
A = LU  =  21 1 0  0 u 22 u 23 
 31  32 1  0 0 u 33 

[U] is the same as the coefficient matrix at the end of the forward elimination step.
[L] is obtained using the multipliers that were used in the forward elimination process

http://numericalmethods.eng.usf.edu
 Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination

 25 5 1
 64 8 1
 
144 12 1
 25 5 1 
= 2.56; Row2 − Row1(2.56) =  0 − 4.8 − 1.56
64
Step 1:
25
144 12 1 

25 5 1 
= 5.76; Row3 − Row1(5.76) =  0 − 4.8 − 1.56 
144
25
 0 − 16.8 − 4.76

http://numericalmethods.eng.usf.edu
 Finding the [U] Matrix
25 5 1 
Matrix after Step 1:  0 − 4.8 − 1.56 
 
 0 − 16.8 − 4.76

25 5 1 
− 16.8
Step 2: = 3.5; Row3 − Row2(3.5) =  0 − 4.8 − 1.56
− 4.8
 0 0 0.7 

25 5 1 
U  =  0 − 4.8 − 1.56
 0 0 0.7 

http://numericalmethods.eng.usf.edu
 Finding the [L] matrix
1 0 0
 1 0
 21
 31  32 1

Using the multipliers used during the Forward Elimination Procedure

a 64
From the first step  25 5 1  21 = 21 = = 2.56
of forward  64 8 1 a11 25
elimination    31 =
a31 144
= = 5.76
144 12 1 a11 25

http://numericalmethods.eng.usf.edu
 Finding the [L] Matrix
From the second 25 5 1 
step of forward  0 − 4.8 − 1.56   32 = a32 = − 16.8 = 3.5
elimination   a 22 − 4.8
 0 − 16.8 − 4.76

 1 0 0
L = 2.56 1 0
5.76 3.5 1

http://numericalmethods.eng.usf.edu
 Does [L][U] = [A]?

 1 0 0 25 5 1 
LU  = 2.56 1 0  0 − 4.8 − 1.56 =
5.76 3.5 1  0 0 0.7 
?

http://numericalmethods.eng.usf.edu
 Example
Set [L][Z] = [C]  1 0 0  z1  106.8 
2.56 1 0  z  = 177.2 
  2   
5.76 3.5 1  z 3  279.2

Solve for [Z] z1 = 10

2.56 z1 + z 2 = 177.2
5.76 z1 + 3.5 z 2 + z 3 = 279.2

http://numericalmethods.eng.usf.edu
 Example
Complete the forward substitution to solve for [Z]

z1 = 106.8
z 2 = 177.2 − 2.56 z1  z1   106.8 
= 177.2 − 2.56(106.8)
= −96.2
Z  =  z2  = − 96.21
z3 = 279.2 − 5.76 z1 − 3.5 z 2  z3   0.735 
= 279.2 − 5.76(106.8) − 3.5(− 96.21)
= 0.735

http://numericalmethods.eng.usf.edu
 Example
Set [U][X] = [Z]
25 5 1   x1   106.8 
 0 − 4.8 − 1.56  x  = − 96.21
   2  
 0 0 0.7   x3   0.735 

Solve for [X] The 3 equations become

25a1 + 5a2 + a3 = 106.8

− 4.8a2 − 1.56a3 = −96.21
0.7a3 = 0.735

http://numericalmethods.eng.usf.edu
 Example
Substituting in a3 and using the
From the 3rd equation second equation

0.7 a3 = 0.735 − 4.8a2 − 1.56a3 = −96.21

a3 =
0.735 − 96.21 + 1.56a3
a2 =
0.7 − 4.8
a3 = 1.050 − 96.21 + 1.56(1.050)
a2 =
− 4.8
a2 = 19.70

http://numericalmethods.eng.usf.edu
 Example
Substituting in a3 and a2 using Hence the Solution Vector is:
the first equation

25a1 + 5a2 + a3 = 106.8  a1  0.2900

106.8 − 5a2 − a3 a  =  19.70 
a1 =
25
 2  
106.8 − 5(19.70) − 1.050
 a3   1.050 
=
25
= 0.2900

http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]

http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

b11  1  b12  0
b  0  b  1
A  21  =   A  22  =  
         
       
bn1  0 bn 2  0

The remaining columns in [B] can be found in the same manner

http://numericalmethods.eng.usf.edu
 Example: Inverse of a Matrix
Find the inverse of a square matrix [A]
 25 5 1
A =  64 8 1
144 12 1

Using the decomposition procedure, the [L] and [U] matrices are found to be

 1 0 0 25 5 1 
A = LU  = 2.56 1 0  0 − 4.8 − 1.56
5.76 3.5 1  0 0 0.7 

http://numericalmethods.eng.usf.edu
 Example: Inverse of a Matrix
Solving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]

 1 0 0  z1  1
Step 1: LZ  = C  → 2.56 1 0  z2  = 0
5.76 3.5 1  z3  0
This generates the equations:
z1 = 1
2.56 z1 + z 2 = 0
5.76 z1 + 3.5 z 2 + z3 = 0
http://numericalmethods.eng.usf.edu
 Example: Inverse of a Matrix
Solving for [Z]

z1 = 1
 z1   1 
z 2 = 0 − 2.56 z1
= 0 − 2.56(1)
Z  =  z2  = − 2.56
= −2.56  z3   3.2 
z3 = 0 − 5.76 z1 − 3.5 z 2
= 0 − 5.76(1) − 3.5(− 2.56 )
= 3.2

http://numericalmethods.eng.usf.edu
 Example: Inverse of a Matrix
25 5 1  b11   1 
Solving [U][X] = [Z] for [X]  0 − 4.8 − 1.56 b  = − 2.56
   21   
 0 0 0.7  b31   3.2 

25b11 + 5b21 + b31 = 1

− 4.8b21 − 1.56b31 = −2.56
0.7b31 = 3.2

http://numericalmethods.eng.usf.edu
 Example: Inverse of a Matrix
Using Backward Substitution
3.2
b31 = = 4.571 So the first column of
0.7
the inverse of [A] is:
−2.56 + 1.560b31
b21 =
−4.8 b11   0.04762 
−2.56 + 1.560(4.571) b  = − 0.9524
= = −0.9524
−4.8  21   
b11 =
1 − 5b21 − b31 b31   4.571 
25
1 − 5(− 0.9524) − 4.571
= = 0.04762
25

http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Repeating for the second and third columns of the inverse

Second Column Third Column

 25 5 1 b12  0  25 5 1  b13  0
 64 8 1 b  = 1  64 8 1 b  = 0
   22       23   
144 12 1 b32  0 144 12 1 b33  1
b12  − 0.08333 b13   0.03571 
b  =  1.417  b  = − 0.4643
 22     23   
b32   − 5.000  b33   1.429 

http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
The inverse of [A] is

 0.04762 − 0.08333 0.03571 

A−1 = − 0.9524 1.417 − 0.4643
 4.571 − 5.000 1.429 

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

http://numericalmethods.eng.usf.edu
Is LU Decomposition better than
Gaussian Elimination?
Solve [A][X] = [B]

T = clock cycle time and nxn = size of the matrix

Forward Elimination Decomposition to LU
 8n 3 32n   8n 3 20n 
CT | FE = T  + 8n 2 −  CT | DE = T  + 4n 2 − 
 3 3 
 3 3 

Back Substitution Forward Substitution

(
CT | BS = T 4n 2 + 12n ) (
CT | FS = T 4n 2 − 4n )
Back Substitution
(
CT | BS = T 4n 2 + 12n )
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than
Gaussian Elimination?
To solve [A][X] = [B]
Time taken by methods

Gaussian Elimination LU Decomposition

 8n 3 4n   8n 3 4n 
T  + 12n 2 +  T  + 12n 2 + 
 3 3   3 3 

T = clock cycle time and nxn = size of the matrix

So both methods are equally efficient.

http://numericalmethods.eng.usf.edu
 To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
= n(CT |FE +CT |BS ) = CT |DE + n  CT |FS + n  CT |BS
 8n 4 4n 2   32n 3 20n 
= T  + 12n +
3
 = T  + 12n 2 − 
 3 3   3 3 

http://numericalmethods.eng.usf.edu
 To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
 8n 4 4n 2   32n 3 20n 
T  + 12n +
3
 T  + 12n 2 − 
 3 3   3 3 

Table 1 Comparing computational times of finding inverse of a matrix using

LU decomposition and Gaussian elimination.
n 10 100 1000 10000
CT|inverse GE / CT|inverse LU 3.288 25.84 250.8 2501

For large n, CT|inverse GE / CT|inverse LU ≈ n/4

http://numericalmethods.eng.usf.edu
Using LU Decomposition to solve SLEs
Solve the following set of  25 5 1  x1  106.8 
linear equations using LU  64 8 1  x  = 177.2 
Decomposition    2  
144 12 1  x3  279.2

Using the procedure for finding the [L] and [U] matrices

 1 0 0 25 5 1 
A = LU  = 2.56 1 0  0 − 4.8 − 1.56
5.76 3.5 1  0 0 0.7 

http://numericalmethods.eng.usf.edu