Lecture 08-Gauss-Elemination
Lecture 08-Gauss-Elemination
ECE 410
Lecture 8
Gauss Elimination
1
Introduction
Nonlinear Equations
a11 x1 a12 x1 x2 a1n ( xn ) 5 b1
a21 ( x1 ) 3 a 22 e x2 a 2 n ( x2 ) 3 / xn b2
a x a x a x b
n1 1 n2 2 nn n n
3
Review of Matrices
a11 a12 a1m
a a 2m
a 22 2nd row Elements are indicated by a
[A]
21 ij
an1 a n2 anm n m
row column
mth column
Square matrix:
c m m 1
- [A]nxm is a square matrix if n=m.
- A system of n equations with n unknonws has a square
coefficient matrix. 4
Review of Matrices
• Symmetric matrix:
If aij = aji [A]nxn is a symmetric matrix
• Diagonal matrix:
[A]nxn is diagonal if aij = 0 for all i=1,...,n ; j=1,...,n
and ij
• Identity matrix:
[A]nxn is an identity matrix if it is diagonal with aii=1
i=1,...,n . Shown as [I] 5
Review of Matrices
• Upper triangular matrix:
6
Special Types of Square Matrices
5 1 2 16 a11 1
1 3 7 39 1
[ A] [ D]
a 22 [I ]
2 7 9 6 1
16 39 6 88 a nn
1
Symmetric Diagonal Identity
a11 a12 a1n a11
a a a a
[ A] 22 2n [ A] 21 22
a nn a n1 ann
Upper Triangular Lower Triangular
7
Review of Matrices
• Matrix multiplication:
r
Note: [A][B] [B][A] cij aik bkj
k 1
8
Review of Matrices
• Determinant of a matrix:
A single number. Determinant of [A] is shown as |A|.
9
Part 3- Objectives
10
Solving Small Numbers of Equations
a11 x1 a12 x2 b1
a21 x1 a22 x2 b2
• Solve both equations for x2: the intersection of the lines presents the
solution.
a11 b1
x2 x1 x2 (slope) x1 intercept
a12 a12
a21 b2
x2
x1
a22 a22
13
Graphical Method
14
2.Determinants and Cramer’s Rule
Determinant can be illustrated for a set of three
equations:
a11 a12 a13
A. x B D a21 a22 a23
a31 a32 a33
Where [A] is the coefficient
a22 a23
matrix: D11 a22 a33 a32 a23
a32 a33
a11 a12 a13 a21 a23
A a21 a22 a23 D12
a31 a33
a21 a33 a31 a23
a31 a32 a33
D13
a21 a22
a21 a32 a31 a22
a31 a32
15
16
Graphical Method
Mathematically
• Coefficient matrices of (a) & (b) are singular. There
is no unique solution for these systems.
Determinants of the coefficient matrices are zero
and these matrices can not be inverted.
21
3. Method of Elimination
• The basic strategy is to successively solve one of
the equations of the set for one of the
unknowns and to eliminate that variable from
the remaining equations by substitution.
22
Elimination of Unknowns Method
2.5x1 + 6.2x2 = 3.0
Given a 2x2 set of equations:
4.8x1 - 8.6x2 = 5.5
23
(Difference is due to the round-off errors).
Naive Gauss Elimination Method
24
Naive Gauss Elimination Method
• Consider the following system of n equations.
26
Naive Gauss Elimination Method (cont’d)
0 0 a33
a3n x 3 b3
0 0 an3 ann
xn bn
29
Pseudo-code of Naive Gauss Elimination Method
k,j
30
Naive Gauss Elimination Method
Example 1
6 –2 2 4 | 16
12 –8 6 10 | 26 R2-2R1
3 –13 9 3 | -19 R3-0.5R1
-6 4 1 -18 | -34 R4-(-R1)31
Naive Gauss Elimination Method
Example 1 (cont’d)
Step 1: Forward elimination
32
Naive Gauss Elimination Method
Example 1 (cont’d)
Step 2: Back substitution
Find x4 x4 =(-3)/(-3) = 1
Find x3 x3 =(-9+5*1)/2 = -2
Find x2 x2 =(-6-2*(-2)-2*1)/(-4) = 1
Find x1 x1 =(16+2*1-2*(-2)-4*1)/6= 3
33
Naive Gauss Elimination Method Example 2
(Using 6 Significant Figures)
3.0 x1 - 0.1 x2 - 0.2 x3 = 7.85
0.1 x1 + 7.0 x2 - 0.3 x3 = -19.3 R2-(0.1/3)R1
0.3 x1 - 0.2 x2 + 10.0 x3 = 71.4 R3-(0.3/3)R1
34
Naive Gauss Elimination Method Example 2
(cont’d)
Step 2: Back substitution
x3 = 7.00003
x2 = -2.50000
x1 = 3.00000
Exact solution:
x3 = 7.0
x2 = -2.5
x1 = 3.0
35
Pitfalls of Gauss Elimination Methods
1. Division by zero
2 x2 + 3 x3 = 8
a11 = 0
4 x1 + 6 x2 + 7 x3 = -3
(the pivot element)
2 x1 + x2 + 6 x3 = 5
36
Pitfalls of Gauss Elimination (cont’d)
3. Ill-conditioned systems
x1 + 2x2 = 10
x1 + 2x2 = 10
38
Techniques for Improving Solutions
1. Use of more significant figures to solve for the
round-off error.
2. Pivoting. If a pivot element is zero, elimination step
leads to division by zero. The same problem may arise,
when the pivot element is close to zero. This Problem
can be avoided by:
Partial pivoting. Switching the rows so that the
largest element is the pivot element.
Complete pivoting. Searching for the largest element
in all rows and columns then switching.
3. Scaling
Solve problem of ill-conditioned system.
Minimize round-off error
39
Partial Pivoting
40
Use of more significant figures to solve for the
round-off error :Example.
2 2.0001 3(2 / 3)
x2 x1
3 0.0003
Significant
Figures x2 x1
3 0.667 -3
4 0.6667 0.000
5 0.66667 0.3000
6 0.666667 0.33000
7 0.6666667 0.333000
42
Pivoting: Example
Now, solving the pervious example using the partial
pivoting technique:
1.0000 x1+ 1.0000 x2 = 1.000
0.0003 x1+ 3.0000 x2 = 2.0001
The pivot is 1.0
1.0000 x1+ 1.0000 x2 = 1.000
2.9997 x2 = 1.9998
x2 = 0.6667 & x1=0.3333
Checking the effect of the # of significant digits:
# of dig x2 x1 Ea% in x1
44
Scaling: Example (cont’d)
B) Using the scaling algorithm to solve:
2 x1+ 100,000 x2 = 100,000
x1 + x2 = 2.0
Scaling the first equation by dividing by 100,000:
0.00002 x1+ x2 = 1.0
x1+ x2 = 2.0
Rows are pivoted:
x1 + x2 = 2.0
0.00002 x1+ x2 = 1.0
Forward elimination yield:
x1 + x2 = 2.0
x2 = 1.00
Solve: x2 = 1.00 & x1 = 1.00
The exact solution is x1 = 1.00002 & x2 = 0.99998 45
Scaling: Example (cont’d)
C) The scaled coefficient indicate that pivoting is necessary.
We therefore pivot but retain the original coefficient to give:
x1 + x2 = 2.0
2 x1+ 100,000 x2 = 100,000
46
Determinate Evaluation
1
( n 1) p
D a a a a
' ''
11 22 33 nn
where:
p represents the number of times that rows are pivoted
47
Example: Gauss Elimination
2x 2 x 4 0
2x 1 2x 2 3x 3 2x 4 2
4x 1 3x 2 x 4 7
6x 1 x 2 6x 3 5x 4 6
a) Forward Elimination
0 2 0 1 0 6 1 6 5 6
2 2 3 2 2 2 2 3 2 2
R 4
R 1
4 3 0 1 7 4 3 0 1 7
6 1 6 5 6 0 2 0 1 0
48
Example: Gauss Elimination (cont’d)
6 1 6 5 6
2 2 3 2 2 R 2 0.33333 R 1
4 3 0 1 7 R 3 0.66667 R 1
0 2 0 1 0
6 1 6 5 6
0 1.6667 5 3.6667 4
R 2 R 3
0 3.6667 4 4.3333 11
0 2 0 1 0
6 1 6 5 6
0 3.6667 4 4.3333 11
0 1.6667 5 3.6667 4
0 2 0 1 0
Example: Gauss Elimination (cont’d)
6 1 6 5 6
0 3.6667 4 4.3333 11
0 1.6667 5 3.6667 4 R 3 0.45455 R 2
0 2 0 1 0 R 4 0.54545 R 2
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 2.1818 3.3636 5.9999 R 4 0.32000 R 3
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 0 1.5600 3.1199
50
Example: Gauss Elimination (cont’d)
6 1 6 5 6
0 3.6667 4 4.3333 11
0 0 6.8182 5.6364 9.0001
0 0 0 1.5600 3.1199
b) Back Substitution
3.1199
x4 1.9999
1.5600
9.0001 5.6364 1.9999
x3 0.33325
6.8182
11 4.3333 1.9999 4 0.33325
x2 1.0000
3.6667
6 5 1.9999 6 0.33325 1 1.0000
x1 0.50000 51
6
Gauss-Jordan Elimination
0 2 0 1 0 1 0.16667 1 0.83335 1
R 4
2 2 3 2 2 R 1
2 2 3 2 2
4 3 0 1 7 R 4 / 6.0 4 3 0 1 7
6 1 6 5 6 0 2 0 1 0
1 0.16667 1 0.83335 1
2 2 3 2 2 R 2 2 R 1
4 3 0 1 7 R 3 4 R 1
0 2 0 1 0
53
1 0.16667 1 0.83335 1
0 1.6667 5 3.6667 2
0 3.6667 4 4.3334 7
0 2 0 1 0
Dividing the 2nd row by 1.6667 and reducing the second column.
(operating above the diagonal as well as below) gives:
1 0 0 0 0.49999
0 1 0 0 1.0001
0 0 1 0 0.33326
0 0 0 1 1.9999