Numerical Analysis

Simultaneous Linear
Equations

Lecture 4
Representing Linear Algebra
 In this chapter we look at the solution of n linear
algebraic equations in n unknowns.

A system of algebraic In matrix notation the

equations has the form equations are written as
a11 x1  a12 x 2  a13 x 3  b1 a11 a12 a13 x1  b1 
    
a21 x1  a22 x 2  a23 x 3  b2 a21 a22 a23 x 2  b2 
a31 x1  a32 x 2  a33 x 3  b3     
a33 
a31 a32 x 3  b3 

 [A]{x}  {b} 2
Uniqueness of Solution
 A system of n linear equations in n unknowns
has a unique solution, provided that the
determinant of the coefficient matrix is
nonsingular, that is, if |A| ≠ 0.
 If the coefficient matrix is singular, the
equations may have an infinite number of
solutions, or no solutions at all, depending
on the constant vector.

3
Iterative Methods
 If systems of linear equations are very large, direct methods
that produce the exact solution in a fixed number of steps,
require excessive computation time and computer memory.
 For systems that have coefficient matrices with the
appropriate structure – especially large, sparse systems
(many coefficients whose value is zero) – iterative
techniques may be preferable
 Three common classical iterative techniques for linear
systems
 The Jacobi method
 Gauss-Seidel method
 Successive Over Relaxation (SOR) method

4
Iterative solution procedure
 Write the system Ax=b in an equivalent form
x  Cx  d(construct updating formula )
 Starting with x0, generate a sequence of approximations {xk}
iteratively by
( i 1)

x  Cx (i )
 d
Representation of C and d depends on the type of the
method used
 But for every method C and d are obtained from A and b,
but in a different way

5
Convergence
 As k, the sequence {xk} converges to the
solution vector under some conditions on C
matrix
 This imposes different conditions on A matrix for
different methods
 For the same A matrix, one method may
converge while the other may diverge
 Therefore for each method the relation
between A and C should be found to decide
on the convergence

6
Jacobi method
 This method makes two assumptions:
1. The system given by:
a11 x1  a12 x2  a13 x3  b1
a21 x1  a22 x2  a23 x3  b2
a31 x1  a32 x2  a33 x3  b3

2. The coefficient matrix A has no zeros on its main

diagonal.
Notes:
 If any of the diagonal entries are zero, then rows or columns must be
interchanged to obtain a coefficient matrix that has nonzero entries on the
main diagonal.
 By using determinants, we can easily check that the solution of the given
system of linear equation exits and it is unique. 7
Basic Idea on Jacobi method
1. To begin the Jacobi method, solve the first equation for x1, the second
equation for x2, and so on, as follows.

a11 x1  a12 x2  a13 x3  b1 x1  (b1  a12 x2  a13 x3 ) / a11

2. a21 x1  a22updating
Construct x2  a23 xformula
3  b2 
as x2  (b2  a21 x1  a23 x3 ) / a22
a31 x1  a32 x2  a33 x3  b3 x3  (b3  a31 x1  a32 x2 ) / a33

x1( i 1)  (b1  a12 x2( i )  a13 x3( i ) ) / a11

x2( i 1)  (b2  a21 x1( i )  a23 x3( i ) ) / a22
x3( i 1)  (b3  a31 x1( i )  a32 x2( i ) ) / a33
8
Basic Idea on Jacobi method

3. Starts with initial guesses x1(0), x2(0), x3(0)

4. Iteratively compute x1(i+1), x2(i+1), x3(i+1) based on
the updating formula for i = 0, 1, …, until the x's
converge or we reach to maximum no. of
iterations

9
Stopping Criteria
• When the estimated percentage relative
error of every x's is less than the
acceptable error.

( j 1)
x x ( j)
 a ,i  i
( j 1)
i
100%   s , for all i
x i

10
Jacobi method : Example 1
1
x y3
• Consider the two-by-two system 2 x  y  6 2
x  2y  6 1
y   x3
• Start with x (1)  y (1)  1 / 2 2

• First iteration
( 2) 1 (1) 1 11
x   y 3  3
2 4 4
1 1 11
y ( 2)   x (1)  3    3 
2 4 4
• Second iteration
1 ( 2) 11 13
x (3)   y 3  3
2 8 8
1 11 13
y (3)   x ( 2)  3    3 
2 8 8 11
Jacobi method : Example 2
 Consider the three-by-three system

2 x1  x2  x3  1 AX=b

x1  2 x2  x3  6
x1  x2  2 x3  3
x1  0.5 x2  0.5 x3  0.5
x2  0.5 x1  0.5 x3  3.0 X=CX+d
x3  0.5 x1  0.5 x2  1.5

 x1( k )   0.0 0.5  0.5  x1( k 1)   0.5

 (k )     ( k 1)   
x
 2     0 . 5 0 .0 0. 5 x
  2 
  3 .0 
 x ( k )   0.5 0.5 0.0   x ( k 1)    1.5 
 3Start 
 with   3   

x (0)  (0,0,0)
Jacobi method : Example 3
 Use the Jacobi method to approximate the solution of the
following system of linear equations.

Continue the iterations until two successive approximations are

identical when rounded to three significant digits.

13
Jacobi method : Example 3

14
 Jacobi method : Example 4
Consider the following set of equations.
10 x1  x2  2 x3 6
 x1  11x2  x3  3 x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15
Convert the set Ax = b in the form of x = Cx + d.
1 1 3
x1  x2  x3 
10 5 5
1 1 3 25
x2  x1  x3  x4 
11 11 11 11
1 1 1 11
x3   x1  x2  x4 
5 10 10 10
3 1 15
x4   x2  x3 
8 8 8
15
 Jacobi method : Example 4
(1) 1 (0) 1 (0) 3
x1  x2  x3 
10 5 5
(1) 1 (0) 1 (0) 3 (0) 25
x2  x1  x3  x4 
11 11 11 11
(1) 1 (0) 1 (0) 1 (0) 11
x3   x1  x2  x4 
5 10 10 10
(1) 3 (0) 1 (0) 15
x4   x2  x3 
8 8 8
(0) (0) (0) (0)
x1  0, x2  0, x3  0 and x4  0.

(1)
x1  0.6000,
(1)
x2  2.2727,
(1)
x3  1.1000
(1)
x4  1.8750
16
 Jacobi method : Example 4
(2) 1 (1) 1 (1) 3
x1  x2  x3 
10 5 5
(2) 1 (1) 1 (1) 3 (1) 25
x2  x1  x3  x4 
11 11 11 11
( 2) 1 (1) 1 (1) 1 (1) 11
x3   x1  x2  x4 
5 10 10 10
(2) 3 (1) 1 (1) 15
x4   x2  x3 
8 8 8

(k) 1 ( k 1) 1 ( k 1) 3

x1  x2  x3 
10 5 5
(k ) 1 ( k 1) 1 ( k 1) 3 ( k 1) 25
x2  x1  x3  x4 
11 11 11 11
(k ) 1 ( k 1) 1 ( k 1) 1 ( k 1) 11
x3   x1  x2  x4 
5 10 10 10
(k ) 3 ( k 1) 1 ( k 1) 15
x4   x2  x3 
8 8 8

17
 Jacobi method : Example 2
Results:
iteration 0 1 2 3

(k ) 0.0000 0.6000 1.0473 0.9326

x1
(k ) 0.0000 2.2727 1.7159 2.0530
x2
(k ) 0.0000 -1.1000 -0.8052 -1.0493
x3
0.0000 1.8750 0.8852 1.1309

18
function x=Jacobi_1(A,b,xo,tol,maxitr)
Jacobi Method
% A Coefficient matrix % b Constant matrix
% xo Initial solution
% tol tolerance % maxitr maximum number of iterations
[n m]=size(A)
C=-A;
for i=1:n
C(i,:)=C(i,:)/A(i,i); % C matrix formulation
C(i,i)=0;
d(i,1)=b(i)/A(i,i); % d matrix formulation
end
itr=1
while(itr<=maxitr)
x=C*xo+d;
if abs((x-xo)./x)*100<=tol
disp('The Jacobi Method is Converged')
return;
else
xo=x;
end
itr=itr+1;
end
disp('The Jacobi Method is Not Converged')

19
function x=Jacobi_2(A,b,xo,tol,maxitr)
Jacobi Method
% A Coefficient matrix % b Constant matrix
% xo Initial solution
% tol tolerance % maxitr maximum number of iterations
[n m]=size(A);
C=-A;
for i=1:n
C(i,:)=C(i,:)/A(i,i); % C matrix formulation
C(i,i)=0;
d(i,1)=b(i)/A(i,i); % d matrix formulation
end

for i=1: maxitr

x=C*xo+d;
if abs((x-xo)./x)*100<=tol
disp('The Jacobi Method is Converged')
return;
else
xo=x;
end
end
disp('The Jacobi Method is Not Converged')

20
Jacobi method : Example 5
An Example of Divergence
- Apply the Jacobi method to the system
using the initial approximation (X1,X2)=(0,0)and show that the
method diverges.

Solution: by rewriting the given system in the form

21
Diagonally Dominant
We set up the Jacobi iteration but did not ask the
question “when will the Jacobi iteration converge?”

Definition: n
A matrix A is diagonally dominant if aii   aij
j1
ji
Theorem:
If the matrix A is diagonally dominant then Ax=b has a
unique solution x and the Jacobi iteration produces a
sequence which converges to  x for any initial guess

22