Gaussian Elimination

Prepared by: Group 1

 Gaussian Elimination
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution
 Forward Elimination
The goal of forward elimination is to
transform the coefficient matrix into an
upper triangular matrix
25 5 1 x1 106.8
64 8 1 x 177.2
2
144 12 1 x3 279.2

25 5 1 x1 106.8
0 4.8 1.56 x 96.21
2
0 0 0.7 x3 0.735
 Forward Elimination
A set of n equations and n unknowns
a11 x1 a12 x2 a13 x3 ... a1n xn b1
a21 x1 a22 x2 a23 x3 ... a2 n xn b2
. .
. .
. .
an1 x1 an 2 x2 an 3 x3 ... ann xn bn

(n-1) steps of forward

elimination
 Forward Elimination
Step 1
For Equation 2, divide Equation 1a11by and
multiply bya21 .

a21
a ( a11 x1 a12 x2 a13 x3 ... a1n xn b1 )
11
a21 a21 a21
a21 x1 a12 x2 ... a1n xn b1
a11 a11 a11
 Forward Elimination
Subtract the result from Equation 2.
a21 x1 a22 x2 a23 x3 ... a2 n xn b2
a21 a21 a21
a21 x1 a a12 x2 ... a a1n xn a b1
11 11 11
_________________________________________________
a21 a21 a21
a22 a12 x2 ... a2 n a1n xn b2 b1
a11 a11 a11

o a x ... a x b
'
22 2
'
2n n
'
2
r
 Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations
as a11 x1 a12 x2 a13 x3 ... a1n xn b1
'
a22 x2 a23
'
x3 ... a2' n xn b2'
'
a32 x2 a33
'
x3 ... a3' n xn b3'
. . .
. . .
. . .

an' 2 x2 an' 3 x3 ... ann

'
xn bn'

End of Step 1
 Forward Elimination
Step 2
Repeat the same procedure for the 3rd
term of Equation 3.
a11 x1 a12 x2 a13 x3 ... a1n xn b1
'
a22 x2 a23
'
x3 ... a2' n xn b2'
"
a33 x3 ... a3" n xn b3"
. .
. .
. .

an" 3 x3 ... ann

"
xn bn"
End of Step 2
 Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a11 x1 a12 x 2 a13 x3 ... a1n x n b1
'
a22 x2 a23
'
x3 ... a2' n xn b2'
a x ... a x b
"
33 3
"
3n n
"
3
. .
. .
. .

n 1 n 1
ann xn bn

End of Step (n-1)

Matrix Form at End of Forward
Elimination
a11 a12 a13 a1n x1 b1
0 '
a22 '
a23 '
a 2 n x2 b2'

0 0 a "
33 a3n x3 b3
" "

0 0 0 0 ann xn bn
(n 1 )
(n-1 )
 Back Substitution
Solve each equation starting from the last
equation
25 5 1 x1 106.8
0 4.8 1.56 x 96.21
2
0 0 0.7 x3 0.735

Example of a system of 3
equations
Back Substitution Starting
Eqns
a11 x1 a12 x 2 a13 x3 ... a1n x n b1
'
a22 x2 a23
'
x3 ... a2' n xn b2'
"
a33 x3 ... an" xn b3"
. .
. .
. .

n 1 n 1
ann xn bn
 Back Substitution
Start with the last equation because it has only one unknown
( n 1)
b
xn n
( n 1)
a nn
 Back Substitution
( n 1)
b
xn n
( n 1)
a nn

bi i 1 ai ,ii11 xi 1 ai,ii12 xi 2 ... ai ,in1 xn

xi i 1 for i n 1,...,1
aii
n
i 1
bi aij i 1 x j
j i 1
xi i 1 for i n 1,...,1
a ii
Gauss Elimination
Example
 Example 1
The upward velocity of a rocket is given at three
different times

Table 1 Velocity vs. time data.

Time, t s Velocity,v m/s
5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial as:

v t a t a t a ,
1
2
2 3 5 t 12.
Find the velocity at t=6 seconds .
 Example 1 Cont.
Assume
v t a1t 2 a2t a3 , 5 t 12.

Results in a matrix template of the form:

t12 t1 1 a1 v1
2 a v
2 t t2 1 2 2
t32 t3 1 a3 v3

Using data from Table 1, the matrix becomes:

25 5 1 a1 106.8
64 8 1 a 177.2
2
144 12 1 a3 279.2
 Example 1 Cont.
25 5 1 a1 106.8 25 5 1 106.8
64 8 1 a 2 177.2 64 8 1 177.2

144 12 1 a 3 279.2 144 12 1 279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of Steps of Forward
Elimination
Number of steps of forward elimination
is
(n1)(31)2
 Forward Elimination: Step 1
25 5 1 106.8 Divide Equation 1 by 25
64 8 1 177.2 and
64
2.56
144 12 1 279.2 25
multiply it by 64,
25 5 1 106.8 2.56 .64 12.8 2.56 273.408

. 64 8 1 177.2
Subtract the result from 64 12.8 2.56 273.408
Equation 2
0 4.8 1.56 96.208

25 5 1 106.8
Substitute new equation 0 4.8 1.56 96.208
for Equation 2
144 12 1 279.2
 Forward Elimination: Step 1
25 5 1
(cont.)
106.8
0
Divide Equation 1 by 25
4.8 1.56 96.208 and 144
144 12 1 279.2 5.76
25
multiply it by 144,
.
25 5 1 106.8 5.76 144 28.8 5.76 615.168

Subtract the result from144 279.2

.
12 1
Equation 3 144 28.8 5.76 615.168
0 16.8 4.76 335.968
25 5 1 106.8
Substitute new equation
for Equation 3 0 4 . 8 1 . 56 96 . 208
0 16.8 4.76 335.968
 Forward Elimination: Step 2
25 5 1 106.8 Divide Equation 2 by
0 4.8 1.56 96.208 4.8

0 16.8 4.76 335.968 16.8multiply it by
and 3.5
4 .8
16.8,
0 4.8 1.56 96.208 3.5 0 16.8 5.46 336.728
0 16.8 4.76 335.968
Subtract the result from
Equation 3 0 16.8 5.46 336.728
0 0 0 .7 0.76

25 5 1 106.8
Substitute new equation 0 4.8 1.56 96.208
for Equation 3
0 0 0.7 0.76
Back Substitution
 Back Substitution
25 5 1 106.8 25 5 1 a1 106.8
0 4.8 1.56 96.2 0 4.8 1.56 a 96.208
2
0 0 0.7 0.7 0 0 0.7 a3 0.76

Solving for a3
0.7a3 0.76
0.76
a3
0 .7
a3 1.08571
 Back Substitution (cont.)
25 5 1 a1 106.8
0 4.8 1.56 a 96.208
2
0 0 0.7 a3 0.76

Solving for a2
4.8a2 1.56a3 96.208
96.208 1.56a3
a2
4.8
96.208 1.56 1.08571
a2
4.8
a2 19.6905
 Back Substitution (cont.)
25 5 1 a1 106.8
0 4.8 1.56 a 96.2
2
0 0 0.7 a3 0.76

Solving for a1
25a1 5a2 a3 106.8
106.8 5a2 a3
a1
25
106.8 5 19.6905 1.08571

25
0.290472
Gaussian Elimination Solution
25 5 1 a1 106.8
64 8 1 a2 177.2

144 12 1 a3 279.2

a1 0.290472
a 19.6905
2
a3 1.08571
 Example 1 Cont.
Solution a1 0.290472
a 19.6905
The solution vector is
2
a3 1.08571

The polynomial that passes through the three data points is then:

v t a1t 2 a2t a3
0.290472t 2 19.6905t 1.08571, 5 t 12

v 6 0.290472 6 19.6905 6 1.08571

2

129.686 m/s.
Gauss Elimination
Pitfalls
Pitfall#1. Division by
zero
10 x2 7 x3 3
6 x1 2 x2 3x3 11
5 x1 x2 5 x3 9

0 10 7 x1 3
6 2 3 x 11
2
5 1 5 x3 9
Is division by zero an issue
here?
12 x1 10 x2 7 x3 15
6 x1 5 x2 3x3 14
5 x1 x2 5 x3 9

12 10 7 x1 15
6 5 3 x2 14

5 1 5 x3 9
 Is division by zero an issue
here? YES
12 x1 10 x2 7 x3 15
6 x1 5 x2 3x3 14
24 x1 x2 5 x3 28

12 10 7 x1 15 12 10 7 x1 15
6 5 3 x 14 0
2 0 6.5 x2 6.5
24 1 5 x3 28 0 21 19 x3 2

Division by zero is a possibility at any

step of forward elimination
Pitfall#2. Large Round-off
Errors 20 15 10 x 45 1
3 2.249 7 x 1.751
2
5 1 3 x3 9

Exact Solution

x1 1
x 1
2
x3 1
Pitfall#2. Large Round-off
Errors 20 15 10 x 45 1
3 2.249 7 x 1.751
2
5 1 3 x3 9

Solve it on a computer using 6

significant digits with
chopping
x1 0.9625
x 1.05
2
x3 0.999995
Pitfall#2. Large Round-off
Errors 20 15 10 x 45 1
3 2.249 7 x 1.751
2
5 1 3 x3 9

Solve it on a computer using 5

significant digits with
chopping
x1 0.625
x 1.5
2
x3 0.99995

Is there a way to reduce the round off

 Avoiding Pitfalls
Increase the number of significant
digits
Decreases round-off error
Does not avoid division by zero
 Avoiding Pitfalls

Gaussian Elimination with Partial

Pivoting
Avoids division by zero
Reduces round off error
Gauss Elimination with
Partial Pivoting
 Pitfalls of Nave Gauss
Elimination
Possible division by zero
Large round-off errors
 Avoiding Pitfalls
Increase the number of significant
digits
Decreases round-off error
Does not avoid division by zero
 Avoiding Pitfalls
Gaussian Elimination with Partial
Pivoting
Avoids division by zero
Reduces round off error
What is Different About Partial
Pivoting?
At the beginning of the kth step of forward
elimination, find the maximum of

akk , ak 1,k ,................, ank

If the maximum of the a pk
values isth
in the p row,k p n, then switch rows p
and k.
Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12 a13 a1n x1 b1
0 '
a22 '
a23 '
a 2 n x2 '
b2

0 a '
32 a '
33 a3n x3 b3
' '

0 '
an 2 ' '
an 3 an 4 ann xn bn
'
'
Example (2nd step of FE)

6 14 5.1 3.7 6 x1 5
0 7 6 1 2 x2 6

0 4 12 1 11 x3 8

0 9 23 6 8 x4 9
0 17 12 11 43 x5 3

Which two rows would you

switch?
Example (2nd step of FE)

6 14 5.1 3.7 6 x1 5
0 17 12 11 43 x 3
2
0 4 12 1 11 x3 8

0 9 23 6 8 x

4 9
0 7 6 1 2 x5 6

Switched Rows
 Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous
linear equations of the form [A]
[X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution
 Forward Elimination
Same as nave Gauss elimination
method except that we switch rows
before each of the (n-1) steps of
forward elimination.
 Example: Matrix Form at
Beginning of 2nd Step of
Forward Elimination
a11 a12 a13 a1n x1 b1
0 a '
a '
'
a 2 n x2 '
b2
22 23
0 a '
32 a '
33 a3n x3 b3
' '

0 a '
n2 a '
n3 a '
n4 ann xn bn
'
'
Matrix Form at End of Forward
Elimination
a11 a12 a13 a1n x1 b1
0 a' a '
'
a2 n x b'
22 23 2 2
0 0 a "
33 a "
3n x3 b "
3


0 0 0 0 ann
(n 1 )
xn bn(n-1 )
Back Substitution Starting
Eqns
a11 x1 a12 x 2 a13 x3 ... a1n x n b1
'
a22 x2 a23
'
x3 ... a2' n xn b2'
"
a33 x3 ... an" xn b3"
. .
. .
. .

n 1 n 1
ann xn bn
Back Substitution
( n 1)
b
xn n
( n 1)
a nn

n
i 1 i 1
bi aij x j
j i 1
xi i 1 for i n 1,...,1
a ii
Gauss Elimination with
Partial Pivoting
Example
 Example 2
Solve the following set of
equations by Gaussian
elimination with partial pivoting

25 5 1 a1 106.8
64
8 1 a 2 177.2

144 12 1 a 3 279.2
 Example 2 Cont.
25 5 1 a1 106.8 25 5 1 106.8
64 8 1 a 177.2
2 64 8 1 177 . 2
144 12 1 a 3 279.2 144 12 1 279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of Steps of Forward
Elimination

Number of steps of forward elimination

is (n1)=(31)=2
 Forward Elimination: Step 1
Examine absolute values of first column, first row
and below.
25 , 64 , 144
Largest absolute value is 144 and exists in row
3.
Switch row 1 and row 3.

25 5 1 106.8 144 12 1 279.2

64 8 1 177.2 64 8 1 177.2

144 12 1 279.2 25 5 1 106.8
 Forward Elimination: Step 1
(cont.)
144 12 1 279.2 Divide Equation 1 by 144
64 8 1 177.2 and 64
0.4444
25 5 1 106.8 144
multiply it by 64,
.
144 12 1 279.2 0.4444 63.99 5.333 0.4444 124.1
Subtract the result from 64 1 177.2
.
8
Equation 2 63.99 5.333 0.4444 124.1
0 2.667 0.5556 53.10

Substitute new equation 144 12 1 279.2

for Equation 2 0 2.667 0.5556 53.10

25 5 1 106.8
 Forward Elimination: Step 1
(cont.)
144 12 1 279.2 Divide Equation 1 by 144
0 2.667 0.5556 53.10 and 25
0.1736
25 5 1 106.8 144
multiply it by 25,
.
144 12 1 279.2 0.1736 25.00 2.083 0.1736 48.47
.
25 5 1 106.8
Subtract the result from
Equation 3 25 2.083 0.1736 48.47
0 2.917 0.8264 58.33

Substitute new equation 144 12 1 279.2

for Equation 3 0 2.667 0.5556 53.10

0 2.917 0.8264 58.33
 Forward Elimination: Step 2
Examine absolute values of second column, second row
and below.

2.667 , 2.917
Largest absolute value is 2.917 and exists in
row 3.
Switch row 2 and row 3.

144 12 1 279.2 144 12 1 279.2

0 2.667 0.5556 53.10 0 2.917 0.8264 58.33

0 2.917 0.8264 58.33 0 2.667 0.5556 53.10
 Forward Elimination: Step 2
(cont.)
Divide Equation 2 by 2.917
144 12 1 279.2
0 2.917 0.8264 58.33 and
2.667
multiply it by 2.667,
0 2.667 0.5556 53.10 0.9143.
2.917
0 2.917 0.8264 58.33 0.9143 0 2.667 0.7556 53.33

.
0 2.667 0.5556 53.10
Subtract the result from 0 2.667 0.7556 53.33
Equation 3
0 0 0.2 0.23

144 12 1 279.2
Substitute new equation 0 2.917 0.8264 58.33
for Equation 3
0 0 0.2 0.23
Back Substitution
 Back Substitution
144 12 1 279.2 144 12 1 a1 279.2
0 2.917 0.8264 58.33 0 2.917 0.8264 a 58.33
2
0 0 0.2 0.23 0 0 0.2 a3 0.23

Solving for a3
0.2a3 0.23
0.23
a3
0 .2
1.15
 Back Substitution (cont.)
144 12 1 a1 279.2
0 2.917 0.8264 a 58.33
2
0 0 0.2 a3 0.23

Solving for a2
2.917 a2 0.8264a3 58.33
58.33 0.8264a3
a2
2.917
58.33 0.8264 1.15

2.917
19.67
 Back Substitution (cont.)
144 12 1 a1 279.2
0 2.917 0.8264 a 58.33
2
0 0 0.2 a3 0.23

Solving for a1
144a1 12a2 a3 279.2
279.2 12a2 a3
a1
144
279.2 12 19.67 1.15

144
0.2917
Gaussian Elimination with
Partial Pivoting Solution

25 5 1 a1 106.8
64 8 1 a 177.2
2
144 12 1 a3 279.2

a1 0.2917
a 19.67
2
a3 1.15
Gauss Elimination with
Partial Pivoting
Another Example
Partial Pivoting: Example
Consider the system of equations
10 x1 7 x2 7
3 x1 2.099 x2 6 x3 3.901
5 x1 x2 5 x3 6

In matrix form
10 7 0 x1 7
3 2.099 6 x 3.901
2 =
5 1 5 x3 6

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping
 Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.

Performing Forward Elimination

10 7 0 x1 7 10 7 0 x1 7
3 2.099 6 x 3.901

5
2
1 5 x3 6
0 0.001 6 x 6.001

0 2.5
2
5 x3 2.5

 Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3

Performing the row swap

10 7 0 x1 7 10 7 0 x1 7
0 0.001 6 x 6.001

0 2.5
2
5 x3 2.5
0
2 . 5 5 x 2.5
2
0 0.001 6 x3 6.001

 Partial Pivoting: Example
Forward Elimination: Step 2

Performing the Forward Elimination results in:

10 7 0 x1 7
0 2.5 5 x 2.5
2
0 0 6.002 x3 6.002
 Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution

10 7
6.002
0 x1 7 x3 1
0 2.5 5 x 2 2.5 6.002

0 0 6.002 x3 6.002 2.5 5 x3
x2 1
2.5

7 7 x 2 0 x3
x1 0
10
 Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

x1 0 x1 0
X calculated x2 1 X exact x 2 1
x3 1 x3 1
 Determinant of a Square
Matrix
Using Nave Gauss Elimination
Example
 Theorem of Determinants
If a multiple of one row of [A]nxn is added or
subtracted to another row of [A]nxn to result
in [B]nxn then det(A)=det(B)
 Theorem of Determinants
The determinant of an upper triangular matrix [A] nxn is
given by

det A a11 a22 ... aii ... ann

n
aii
i 1
 Forward Elimination of a
Square Matrix
Using forward elimination to transform [A]nxn to an
upper triangular matrix, [U]nxn.

A nn U nn

det A det U
 Example
Using nave Gaussian elimination find
the determinant of the following
square matrix.
25 5 1
64 8 1

144 12 1
Forward Elimination
 Forward Elimination: Step 1
25 5 1
64 8 1 Divide Equation 1 by 25
and 64
2.56
144 12 1 25
multiply it by 64,
25 5 1 2.56. 64 12.8 2.56
. 64 8 1
Subtract the result from 64 12.8 2.56
Equation 2
0 4.8 1.56
25 5 1
Substitute new equation 0 4.8 1.56
for Equation 2
144 12 1
 Forward Elimination: Step 1
25 5 1 (cont.)

0 4.8 1.56 Divide Equation 1 by 25
and 144
5.76
144 12
1 multiply it by 144,
25
25 5 1 5.76 144
. 28.8 5.76
.
144 12 1
Subtract the result from 144 28.8 5.76
Equation 3
0 16.8 4.76

25 5 1
Substitute new equation 0 4.8 1.56
for Equation 3
0 16.8 4.76
 Forward Elimination: Step 2
Divide Equation 2 by
25 5 1
0 4.8 1.56 4.8
16.8multiply it by
and
0 16.8 4.76 3 .5
4 .8
16.8,
0 4.8 1.56 3.5 0 16..8 5.46
.
0 16.8 4.76
Subtract the result from
Equation 3 0 16.8 5.46
0 0 0.7

Substitute new equation

25 5 1
0 4.8 1.56
for Equation 3
0 0 0.7
 Finding the Determinant
After forward elimination

25 5 1 25 5 1
64 8 1 0 4.8 1.56

144 12 1 0 0 0.7
.

det A u11 u 22 u33

25 4.8 0.7
84.00
 Summary

-Forward Elimination
-Back Substitution
-Pitfalls
-Improvements
-Partial Pivoting
-Determinant of a Matrix
THE END