Chapter 4

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Linear Systems

5x  y  z  5
x  4y  z  4
x  y  3z  3

x  0.76 y  0.68 z  0.52

2
Linear Systems

4
Linear Systems

5
5
Linear Systems

6
6
 4.1
Naïve Gauss Elimination

7
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Naïve Gauss Elimination
A method to solve simultaneous linear equations
of the form [A][X]=[C]

Two steps:
1. Forward Elimination
2. Back Substitution

8
Naïve Gauss Elimination
1. Forward Elimination
The goal of forward elimination is to transform the
coefficient matrix into an upper triangular matrix
 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 
9
Naïve Gauss Elimination
1. Forward Elimination
A set of n equations and n unknowns
a11x1  a12x2  a13x3  ...  a1n xn  b1
a21x1  a22x2  a23x3  ...  a2n xn  b2
. .
. .
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn

(n-1) steps of forward elimination

10
Naïve Gauss Elimination
1. Forward Elimination
Step 1
For Equation 2, divide Equation 1 by a11 and multiply
by a21

 a21 
 a (a11x1  a12 x2  a13x3  ...  a1n xn  b1 )
 11 
a21 a21 a21
a21x1  a12 x2  ...  a1n xn  b1
a11 a11 a11

11
Naïve Gauss Elimination
1. Forward Elimination
Subtract the result from Equation 2.
a21x1  a22x2  a23x3  ...  a2n xn  b2
a21 a21 a21
− a21x1  a a12 x2  ...  a a1n xn  a b1
11 11 11
_________________________________________________

 a21   a21  a21

 a22  a12  x2  ...   a2 n  a1n  xn  b2  b1
 a11   a11  a11

or a x  ...  a x  b
'
22 2
'
2n n
'
2

12
Naïve Gauss Elimination
1. Forward Elimination
Repeat this procedure for the remaining equations
to reduce the set of equations as
a11x1  a12x2  a13x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
'
a32 x2  a33
'
x3  ...  a3' n xn  b3'
. . .
. . .
. . .

an' 2 x2  an' 3 x3  ...  ann

'
xn  bn'

End of Step 1
13
Naïve Gauss Elimination
1. Forward Elimination
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11x1  a12x2  a13x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3"n xn  b3"
. .
. .
. .

an" 3 x3  ...  ann

"
xn  bn"
End of Step 2
14
Naïve Gauss Elimination
1. Forward Elimination
At the end of (n-1) Forward Elimination steps, the system of
equations will look like
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
a x  ...  a x  b
"
33 3
"
.3n n .
"
3
. .
. .

n 1 n 1 
ann xn  bn

End of Step (n-1)

15
Naïve Gauss Elimination
1. Forward Elimination
Matrix Form at End of Forward Elimination

a11 a12 a13  a1n   x1   b1 

 0 a' '
a23  '
a2 n  x   b' 
 22  2   2 
0 0 "
a33  "
a3n   x3    b3 
"

    
           
 0 0 0 0 ann   xn  bn 
(n 1 )
    (n-1 )

16
Naïve Gauss Elimination
2. Back Substitution
Solve each equation starting from the last equation

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 

Example of a system of 3 equations

17
Naïve Gauss Elimination
2. Back Substitution
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
a22 x2  a23
'
x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

n 1 n 1 
ann xn  bn

18
Naïve Gauss Elimination
2. Back Substitution
Start with the last equation because it has only one unknown
( n 1)
b
xn  n
( n 1)
a nn

19
Naïve Gauss Elimination
2. Back Substitution
( n 1)
b
xn  n
( n 1)
a nn

bii 1  ai,ii11 xi 1  ai,ii12 xi  2  ...  ai,in1 xn

xi  i 1 for i  n  1,...,1
aii

i 1
  aiji 1 x j
n
bi
j i 1
xi  i 1 for i  n  1,...,1
a ii

20
Naïve Gauss Elimination
Example 1
The upward velocity of a rocket is given at
three different times

Table 1 Velocity vs. time data.

Time, t s  Velocity, v m/s 
5 106.8
8 177.2
12 279.2

The velocity data is approximated by a polynomial


as: v t  a t 2  a t  a ,
1 2 3 5  t  12.
Find the velocity at t=6 seconds .
21
Naïve Gauss Elimination
Example 1
vt   a1t 2  a2t  a3 , 5  t  12.
Results in a matrix template of the form:
t12 t1 1  a1   v1 
 2   a   v 
t 2 t2 1  2  2
t32 t3 1 
 a3 
 v3 

 

Using data from Table 1, the matrix becomes:

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2
22
Naïve Gauss Elimination

Example 1
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a 2   177.2    64 8 1  177.2 
       
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution

23
Naïve Gauss Elimination

Example 1
Number of Steps of Forward Elimination

Number of steps of forward elimination is

(n-1)=(3-1)=2

24
 Example 1
Forward Elimination: Step 1
 25 5 1  106.8  Divide Equation 1 by 25 and
 64 8 1  177.2 
  64
multiply it by 64,  2.56
144 12 1  279.2 25
25 5 1  106.8 2.56  64 12.8 2.56  273.408

. 64 8 1  177.2
Subtract the result from  64 12.8 2.56  273.408
Equation 2
0  4.8  1.56   96.208

 25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208
Equation 2  
144 12 1  279.2  25
 Example 1
Forward Elimination: Step 1
 25 5 1  106.8  Divide Equation 1 by 25 and
 0  4.8  1.56   96.208
  144
144 12 1  279.2  multiply it by 144, 25  5. 76

25 5 1  106.8 5.76  144 28.8 5.76  615.168

.
Subtract the result from 144 12 1  279.2
Equation 3  144 28.8 5.76  615.168
0  16.8  4.76   335.968
25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208 
Equation 3  
 0  16.8  4.76   335.968
26
 Example 1
Forward Elimination: Step 2
25 5 1  106.8  Divide Equation 2 by −4.8
 0  4.8  1.56   96.208 
  and multiply it by −16.8,
 0  16.8  4.76   335.968  16.8
 3.5.
 4.8
0  4.8 1.56   96.208 3.5  0 16.8  5.46   336.728
0  16.8  4.76  335.968
Subtract the result from
Equation 3  0  16.8  5.46   336.728
0 0 0.7  0.76

25 5 1  106.8 
Substitute new equation for  0  4.8  1.56   96.208
Equation 3  
 0 0 0.7  0.76 
27
 Example 1
Back Substitution
25 5 1  106.8  25 5 1   a1   106.8 
 0  4.8  1.56   96.2   0  4.8  1.56 a    96.208
     2  
 0 0 0.7  0.7   0 0 0.7   a3   0.76 

Solving for a3
0.7 a3  0.76
0.76
a3 
0.7
a3  1.08571
28
 Example 1
Back Substitution
25 5 1   a1   106.8 
 0  4.8  1.56 a    96.208
   2  
 0 0 0.7   a3   0.76 

Solving for a2
 4.8a2  1.56a3  96.208
 96.208  1.56a3
a2 
 4.8
 96.208  1.56 1.08571
a2 
 4.8
a2  19.6905
29
 Example 1
Back Substitution
25 5 1   a1   106.8 
 0  4.8  1.56 a    96.2
   2  
 0 0 0.7   a3   0.76 

Solving for a1
25a1  5a2  a3  106.8
106.8  5a2  a3
a1 
25
106.8  5 19.6905  1.08571

25
 0.290472
30
Naïve Gauss Elimination
Naïve Gaussian Elimination
Solution
 25 5 1  a1  106.8 
 64   
8 1 a2  177.2  
     
144 12 1  a3  279.2

 a1  0.290472
a    19.6905 
 2  
 a3   1.08571  31
Naïve Gauss Elimination
Example 1
Solution  a1  0.290472
a    19.6905 
The solution vector is  2  
 a3   1.08571 

The polynomial that passes through the three data points is then:
vt   a1t 2  a2t  a3
 0.290472t 2  19.6905t  1.08571, 5  t  12

v6  0.2904726  19.69056  1.08571

2

 129.686 m/s.
32
Naïve Gauss Elimination
Example 2
4 x1  2 x2  x3  11 (a)
 2 x1  4 x2  2 x3  16 (b)
x1  2 x2  4 x3  17 (c)

4 x1  2 x2  x3  11 (a)
 3 x2  1.5 x3  10.5 (b)
 1.5 x2  4 x3  14.25 (c)

4 x1  2 x2  x3  11 (a)
 3 x2  1.5 x3  10.5 (b)
3 x3  9 (c) 33
Naïve Gauss Elimination
Example 2

4 x1  2 x2  x3  11 (a)
 3 x2  1.5 x3  10.5 (b)
3 x3  9 (c)

34
Naïve Gauss Elimination
Example 3

35
Naïve Gauss Elimination
Example 3

36
Naïve Gauss Elimination
Example 3

37
Naïve Gauss Elimination
Example 3

38
 4.2
Naïve Gauss Elimination
Pitfalls

39
Naïve Gauss Elimination Pitfalls
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9

0 10  7  x1   3 
6 2 3   x2   11
    
5  1 5   x3   9 

40
Naïve Gauss Elimination Pitfalls
Is division by zero an issue here?
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9

12 10  7  x1  15
6 5  
3 x2  14   
    
 5  1 5   x3   9 
41
Naïve Gauss Elimination Pitfalls
Is division by zero an issue here?
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28
12 10  7  x1  15 12 10  7  x1   15 
6 5 3   x2   14  0
     0 6.5  x2   6.5
    
24  1 5   x3  28 12  21 19   x3   2

Division by zero is a possibility at any step of

forward elimination 42
Naïve Gauss Elimination Pitfalls
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 
Exact Solution

 x1  1
 x   1
 2  
 x3  1
43
Naïve Gauss Elimination Pitfalls
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 6 significant digits with chopping

 x1   0.9625 
 x    1.05 
 2  
 x3  0.999995
44
Naïve Gauss Elimination Pitfalls
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
 3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 5 significant digits with chopping

 x1   0.625 
 x    1.5 
 2  
 x3  0.99995
Is there a way to reduce the round off error?
45
Naïve Gauss Elimination Pitfalls
Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero

Gaussian Elimination with Partial Pivoting

• Avoids division by zero
• Reduces round off error

46
 4.3
Gauss Elimination with
Partial Pivoting

47
Pivoting
What is Different About Partial
Pivoting?
At the beginning of the kth step of forward elimination, find
the maximum of

akk , ak 1,k ,................, ank

If the maximum of the values is a pk
in the p th row, k  p  n, then switch rows p and k.

48
Pivoting
What is Different About Partial
Pivoting?

49
Pivoting
Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
0 '
a22 '
a23  ' 
a 2 n x2   ' 
b2
    
0 '
a32 '
a33  a3n   x3   b3 
' '

    
           
 0 '
an 2 ' '
an 3 a n 4 ann   xn  bn 
'
    '

50
Pivoting
Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  7 6 1 2   x2   6
    
0 4 12 1 11  x3    8 
    
 0 9 23 6 8   x4  9
 
0  17 12 11 43  x5   3 

Which two rows would you switch?

51
Pivoting
Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 
  2   
0 4 12 1 11  x3    8 
    
 0 9 23 6 8 x
   
4 9
0  7 6 1 2   x5   6

Switched Rows

52
Pivoting

Forward Elimination
Same as naïve Gauss elimination method except
that we switch rows before each of the (n-1)
steps of forward elimination.

53
Pivoting
Example 4
Solve the following set of equations by
Gaussian elimination with partial pivoting

 25 5 1  a1  106.8 
 64   
8 1 a 2  177.2  
     
144 12 1  a3  279.2
54
Pivoting
Example 4
 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a   177.2    
    
2   64 8 1  177.2 
144 12 1  a3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution

55
Pivoting
Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.

 25 5 1  106.8  144 12 1  279.2

 64 8 1  177.2    64 8 1  177.2 
   
144 12 1  279.2  25 5 1  106.8 
56
Pivoting
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 64 8 1  177.2  64
  multiply it by 64,  0.4444
.
 25 5 1  106.8  144

144 12 1  279.2 0.4444  63.99 5.333 0.4444  124.1

.
Subtract the result from 64 8 1  177.2
Equation 2  63.99 5.333 0.4444  124.1
0 2.667 0.5556  53.10

Substitute new equation for 144 12 1  279.2

Equation 2  0 2.667 0.5556  53.10
 
 25 5 1  106.8  57
Pivoting
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 0 2.667 0.5556  53.10 25
  multiply it by 25,  0.1736
.
 25 5 1  106.8  144

144 12 1  279.2 0.1736  25.00 2.083 0.1736  48.47

.
Subtract the result from
25 5 1  106.8
Equation 3  25 2.083 0.1736  48.47
0 2.917 0.8264  58.33

Substitute new equation for 144 12 1  279.2

Equation 3  0 2.667 0.5556  53.10
 
 0 2.917 0.8264  58.33 58
Pivoting
Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.

144 12 1  279.2 144 12 1  279.2

 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33
   
 0 2.917 0.8264  58.33  0 2.667 0.5556  53.10

59
Pivoting
Forward Elimination: Step 2 (cont.)
Divide Equation 2 by 2.917 and
144 12 1  279.2
 0 2.917 0.8264  58.33 multiply it by 2.667,
  2.667
 0 2.667 0.5556  53.10  0.9143.
2.917
0 2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33

.
0 2.667 0.5556  53.10
Subtract the result from  0 2.667 0.7556  53.33
Equation 3
0 0  0.2   0.23

144 12 1  279.2 
Substitute new equation for  0 2.917 0.8264  58.33 
Equation 3  
 0 0  0.2   0.23 60
Pivoting
Back Substitution
144 12 1  279.2  144 12 1   a1   279.2 
 0 2.917 0.8264  58.33    0 2.917 0.8264 a    58.33 
     2  
 0 0  0.2   0.23  0 0  0.2   a3   0.23

Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15
61
Pivoting
Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3   0.23

Solving for a2
2.917a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264 1.15

2.917
 19.67 62
Pivoting
Back Substitution (cont.)
144 12 1   a1   279.2 
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3   0.23

Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917 63
Pivoting
Gaussian Elimination with
Partial Pivoting Solution
 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

 a1  0.2917 
a    19.67 
 2  
 a3   1.15  64
Pivoting
Example 5
Consider the system of equations
10 x1  7 x2  7
 3x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6

In matrix form
 10  7 0  x1   7 
 3 2.099 6  x  3.901
   2 =  
 5  1 5  x 3   6 

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping
65
Pivoting
Example 5
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch row1
with row1.
Performing Forward Elimination
 10  7 0  x1   7  10 7 0  x1   7 
 3 2.099 6  x   3.901

 5
 2   
 1 5  x3   6 
  0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
66
Pivoting
Example 5
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is switched
with row 3

Performing the row swap

10 7 0  x1   7  10 7 0  x1   7 
 0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
 0
 2.5 5   x    2.5 
 2   
 0  0.001 6  x3  6.001

67
Pivoting
Example 5
Forward Elimination: Step 2
Performing the Forward Elimination results in:

10  7 0   x1   7 
 0 2.5 5   x    2.5 
  2   
 0 0 6.002  x3  6.002

68
Pivoting
Example 5
Back Substitution
Solving the equations through back substitution

10  7
6.002
0   x1   7  x3  1
 0 2.5 5   x    2.5  6.002
  2   
 0 0 6.002  x3  6.002 2.5  5 x3
x2   1
2.5

7  7 x 2  0 x3
x1  0
10

69
Pivoting
Example 5
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

 x1   0   x1   0 
X  calculated   x2    1 X  exact   x 2    1
 x3   1   x3   1 

70
Pivoting
Example 6

71
Pivoting
Example 6

72