Simultaneous Linear Algebraic Equations

Jean Paul Nsabimana,

Department of Mathematics
University of Rwanda
Email: [email protected]

December 24, 2020

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
1 / 34
 Outline

1 Simultaneous Linear Algebraic Equations

Direct Method
Gauss Elimination Method
Gauss - Jordan Method
Iterative Method
Jacobi Method of iteration
Gauss-Sidel Iteration Method

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
2 / 34
 Simultaneous Linear Algebraic Equations

Introduction

Simultaneous linear algebraic equations are very common in various fields of

engineering and science. We use matrix inversion method or cramer’s rule
to solve these equations in general.
To solve the equations contain a large number of unknowns; Particularly
suited for computer operations.
These are of two types:
1 Direct Method;
2 Iterative methods.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
3 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Gauss Elimination Method

In this method, the unknowns are eliminated successively by transforming

the given matrix into an equivalent system with upper triangular coefficient
matrix (i.e a matrix in which all the elements below the principal diagonal
are zero) by means of elementary row operations, from which the unknowns
are found by back substitution.
Here, we shall explain it by considering of three equations in three
unknowns.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
4 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Gauss Elimination Method

Consider a system

a1 x + b1 y + c1 z = d1

a2 x + b2 y + c2 z = d2 (1)

a3 x + b3 y + c3 z = d3


where x, y , z are unknowns. The system in matrix form is

AX = B,

where
     
a1 b1 c1 x d1
A = a2 b2 c2  , X = y  and B = d2  .
a3 b3 c13 z d3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
5 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Gauss Elimination Method

Consider the augmented matrix [A|b],

 
a1 b1 c1 d1
[A|B] = a2 b2 c2 d2  (2)
a3 b3 c13 d3

Now, Equation (2) is to be reduced to an upper triangular matrix, let a1 6= 0.

Then
 
( a1 b1 c1 d1
R2 ⇒ R2 − aa21 R1
a3
⇔  0 b20 c20 d20  (3)
R3 ⇒ R3 − a1 R1 0 b0 c 0 d 0 3 13 3

Here, a1 is called the first pivot and b20 , c20 , b30 , c30 , d20 , d20 are transformed elements.
Now take b20 as the pivot (b 0 2 6= o), then

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
6 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Gauss Elimination Method

 
a1 b1 c1 d1
b30
R 3 ⇒ R3 − 0 R2 ⇔  0 b20 c20 d20  (4)
b2 00
0 0 c13 d300
Now, if c300 6= 0, from Equation (4). The given system of linear equation is
equivalent to

a1 x + b1 y + c1 z = d1

0 0 0
b2 y + c2 z = d2 (5)
 00
 00
c 3 z = d3 .
Using back substitution,
 00
d
z = c300


 3  0 00 
0 00
y = b01c 00 d2 c3 − c2 d3 (6)
 2 3  
x = 10 00 d1 b 0 c 00 − b1 d 00 c 00 + b1 c 0 d 00 − b 0 c1 d 00


a b c 1 2 3
2 3 2 3 2 3 2 3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
7 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Note and example

0 00
Note: This method fails if any one of the pivots a1 , b2 or c3 becomes zero.
In such case, by interchanging the rows we can get the non-zero pivots.
Example
Solve the system of equations

3x + y − z = 3

2x − 8y + z = −5 ,

x − 2y + 9z = 8


using Gauss elimination method.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
8 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Solution

The given system is equivalent to

    
3 1 −1 x 3
2 −8 1  y  = −5
1 −2 9 z 8

The augmented matrix is

 
3 1 −1 3
[A|B] = 2 −8 1 −5
1 −2 9 8

Now we will make A as upper triangular choosing 3 as a pivot,

 
(
2 3 1 −1 3
R 2 ⇒ R2 − 3 R1
⇔ 0 − 26 3
5
3 −7
R3 ⇒ R3 − 13 R1 0 −3 7 28
7
3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations
Decemberof
24, Rwanda
2020 Em
9 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Solution Cont...

Now choosing − 26
3 as the pivot from the second column, we get
 
3 1 −1 3
7
R3 ⇒ R 3 − R2 ⇔ 0 − 26 3
5
3 −7 
26 693 231
0 0 78 26

From this, we get

 
3x + y − z = 3
 z = 1

− 26 5
3 y + 3 z = −7
⇒ − 26 5 26
3 y = −7 + 3 = − 3 ⇒ y = 1
 693
 231
 1 1
78 z = 26 x = 3 (3 − y + z) = 3 (3 − 1 + 1) = 1



z = 1

∴ y =1

x =1


Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
10 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Exercise IV.1

1 Solve the system of equations


28x + 4y − z = 32

x + y + 10z = 24 By Gauss elimination method.

2x + 17y + 4z = 35


2 Using Gauss elimination method. Solve the system of equations


3.15x − 1.96y
 + 3.85z = 12.95
2.13x + 5.12y − 2.89z = −8.61

5.92x + 3.05y + 2.15z = 6.88


3 Solve the system of equations



 x1 + x2 + x3 + x4 = 2

x + x + 3x − 2x = −6
1 2 3 4
2x1 + 3x2 − x3 + 2x4 = 7


x1 + 2x2 + x3 − x4 = −2


Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
11 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Gauss - Jordan Method

This method is a modified form of Gauss elimination method. In this

method, the coefficient matrix A of AX = B is reduced to a diagonal
matrix or unit matrix by making all the elements above and below to the
principal diagonal of A zero.
The labor of back substitution is saved here even though it involves
additional computation.

Example
Solve the system of equations

10x + y + z = 12

2x + 10y + z = 13 by Gauss - Jordan method.

x + y + 5x = 7


Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
12 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Solution

The given system in matrix form is

      
10 1 1 x 12 10 1 1 12
 2 10 1 y  = 13 ∴ [A|B] =  2 10 1 13 .
1 1 5 z 7 1 1 5 7
 
1 −8 −44 −51
R1 → R1 − 9R3 ⇔ 2 10 1 13 
1 1 5 7
 
( 1 −8 −44 −51
R2 → R2 − 2R1
⇔ 0 26 89 115 
R3 → R 3 − R1 0 9 49 58
 
1 −8 −44 −51
R2 → −(R2 − 3R3 ) ⇔ 0 1 58 59 
0 9 49 58

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
13 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Solution

 
( 1 0 420 421
R1 → R1 + 8R2
⇔ 0 1 58 59 
R3 → R3 − 9R2 0 0 −473 −473
 
1 0 420 421
1
R3 → − R3 ⇔ 0 1 58 59 
473
0 0 1 1
 
( 1 0 0 1
R1 → R1 − 420R3
⇔ 0 1 0 1
R2 → R2 − 58R3 0 0 1 1
Therefore, the system AX = B reduced to the form

x = 1
    
1 0 0 x 1 
0 1 0 y  = 1 ∴ y =1
0 0 1 z 1

z =1


Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
14 / 34
 Simultaneous Linear Algebraic Equations Direct Method

Exercise IV.2

1 Solve the system of equations


10x1 + x2 + x3 = 12,

x1 + 10x2 − x3 = 10, by Gauss - Jordan method.

x1 − 2x2 + 10x3 = 9,


2 Solve the following system of equations by Gauss - Jordan method



 x + 2y + z − w = −2,

2x + 3y − z + 2w = 7,


 x + y + 3z − 2w = −6,
x + y + z + w = 2.


Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
15 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Introduction

The method of iteration is not applicable to all systems of equations. For

this, each equation of the system must contain one large coefficient
and large coefficient must be attached to a different unknown in that
equation.
The solution to a system of linear equations will exist by iterative procedure
if the absolute value of largest coefficient is greater than the sum of the
absolute values of all remaining coefficients in each equation. (i.e. This is
the condition for convergence).
Now let us see some of such method in details:

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
16 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Jacobi Method of iteration

This method is also known as Gauss-Jacobi method.

a1 x + b1 y + c1 z = d1
a2 x + b2 y + c2 z = d2 (7)
a3 x + b3 y + c3 z = d3

Let |a1 | > |b1 | + |c1 |; |b2 | > |a2 | + |c2 |; |c3 | > |a3 | + |b3 |.
That is, in each equation the coefficients of diagonal terms are large. Hence
the system (8) is ready for iteration. Solving for x, y and z respectively, we
get
1
x = (d1 − b1 y − c1 z)
a1
1
y = (d2 − a2 x − c2 z) (8)
b2
1
z = (d3 − a3 x − b3 y )
c3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
17 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Jacobi Method of iteration

Let x0 , y0 and z0 be the initial approximations of x, y and z. Substituting these

values on RHS of Equation (9), the first approximations:
1
x1 = (d1 − b1 y0 − c1 z0 )
a1
1
y1 = (d2 − a2 x0 − c2 z0 ) (9)
b2
1
z1 = (d3 − a3 x0 − b3 y0 )
c3
Substituting the values of x1 , y1 and z1 in the RHS of Equation (9), the second
approximations are given by
1
x2 = (d1 − b1 y1 − c1 z1 )
a1
1
y2 = (d2 − a2 x1 − c2 z1 ) (10)
b2
1
z2 = (d3 − a3 x1 − b3 y1 )
c3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
18 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Jacobi Method of iteration

Proceeding in the same way, if xr , yr , zr are the r th iterates, then

1
xr +1 = (d1 − b1 yr − c1 zr )
a1
1
yr +1 = (d2 − a2 xr − c2 zr )
b2
1
zr +1 = (d3 − a3 xr − b3 yr ) (11)
c3
The process continued till convergence is secured.
Note: In the absence of any better estimates, the initial approximations are taken
as x0 = y0 = z0 = 0.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
19 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Example

Solve by Jacobi iteration method the system

8x − 3y + 2z = 20;
6x + 3y + 12z = 35;
4x + 11y − z = 33.

Solution: Consider the given system as

8x − 3y + 2z = 20;
4x + 11y − z = 33;
6x + 3y + 12z = 35.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
20 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

So that the diagonal elements are dominant in the coefficient matrix. Now we
write the equations in the form
1
x = (20 + 3y − 2z)
8
1
y = (33 − 4x + z)
11
1
z = (35 − 6x − 3y ) (12)
12
We start from an approximations x0 = y0 = z0 = 0. Substituting these values on
RHS of equation (12). First approximation is given as:
1
x1 = (20 + 3y0 − 2z0 ) = 2.5
8
1
y1 = (33 − 4x0 + z0 ) = 3
11
1
z1 = (35 − 6x0 − 3y0 ) = 2.9166667
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
21 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

Second approximation: substituting x1 , y1 , z1 on RHS of Equations (12), we get

1
x2 = (20 + 3y1 − 2z1 ) = 2.895833
8
1
y2 = (33 − 4x1 + z1 ) = 2.3560606
11
1
z2 = (35 − 6x1 − 3y1 ) = 0.9166666
12
Third approximation: Substituting x2 , y2 , z2 on RHS of Equation (12), we get

1
x3 = (20 + 3y2 − 2z2 ) = 3.1543561
8
1
y3 = (33 − 4x2 + z2 ) = 2.030303
11
1
z3 = (35 − 6x2 − 3y2 ) = 0.8797348
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
22 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

Proceeding in the same way, we get the following results

x4 = 3.0419299 x5 = 3.0168733 x6 = 3.0104409 x7 = 3.0157694
y4 = 1.9329373 y5 = 1.9696539 y6 = 1.9859295 y7 = 1.9885503
z4 = 0.8319128 z5 = 0.9127173 z6 = 0.9158165 z7 = 0.9149638
Until
x11 = 3.0167368 x12 = 3.0167424
y11 = 1.9858681 y12 = 1.9858987
z11 = 0.9118326 x12 = 0.9118312
th th
Therefore, from 11 and 12 approximations, the values of x, y , z are same
correct to four decimal places. Stopping at this stage, we get
x = 3.0167;
y = 1.9858;
z = 0.9118.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
23 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Exercise IV.3

Solve the following system of equations by Jacobi iteration method:

3x + 4y + 15z = 54.8;
x + 12y + 3z = 39.66;
10x + y − 2z = 7.74.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
24 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Gauss-Sidel Iteration Method

This is a modification of Gauss - Jacobi method. As before, the system of linear

equations

a1 x + b1 y + c1 z = d1
a2 x + b2 y + c2 z = d2
a3 x + b3 y + c3 z = d3 (13)

is written as
1
x = (d1 − b1 y − c1 z) (a)
a1
1
y = (d2 − a2 x − c2 z) (b)
b2
1
z = (d3 − a3 x − b3 y ) (c) (14)
c3
We start with the initial approximations x0 , y0 , z0 . Substituting y0 and z0 in
equation (a),
Jean Paul Nsabimana, Department
Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
25 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Gauss-Sidel Iteration Method

We get
1
x1 = (d1 − b1 y0 − c1 z0 ).
a1
Now substituting x = x1 , z = z0 in Equation (b), we get

1
y1 = (d2 − a2 x1 − c2 z0 ).
b2

Now substituting x = x1 , y = y1 in Equation (c), we get

1
z1 = (d3 − a3 x1 − b3 y1 ).
c3

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
26 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Gauss-Sidel Iteration Method

This process is continued till the values of x, y , z are obtained to the desired
degree of accuracy. The general algorithm is as follows:
1
xk+1 = (d1 − b1 yk − c1 zk )
a1
1
yk+1 = (d2 − a2 xk+1 − c2 zk )
b2
1
zk+1 = (d3 − a3 xk+1 − b3 yk+1 ) (15)
c3
Since the current values of the unknowns at each stage of iteration are used in
proceeding to the next stage of iteration, this method is more rapid in
convergence than Gauss- Jacobi Method.

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
27 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Example

Solve by Gauss-Seidal iteration method the system

8x − 3y + 2z = 20;
4x + 11y − z = 33;
6x + 3y + 12z = 35.

So that the diagonal elements are dominant in the coefficient matrix. Now we
write the equations in the form
1
x = (20 + 3y − 2z) (a)
8
1
y = (33 − 4x + z) (b)
11
1
z = (35 − 6x − 3y ) (c)
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
28 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution
20
Putting y = 0, z = 0 in RHS of (a), we get x1 = 8 = 2.5. Putting
x = 2.5, z = 0 in RHS of (b), we get
1
y1 = (33 − 4(2.5)) = 2.0909091.
11
Putting x = 2.5, y = 2.0909091 in RHS of (c), we get
1
z1 = (35 − 6(2.5) − 3(2.0909091)) = 1.1439394.
12
For second approximation:
1
x2 = (20 + 3y1 − 2z1 ) = 2.9981061
8
1
y2 = (33 − 4x2 + z1 ) = 2.0137741
11
1
z2 = (35 − 6x2 − 3y2 ) = 0.9141701
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
29 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

For third approximation:

1
x3 = (20 + 3y2 − 2z2 ) = 3.0266228
8
1
y3 = (33 − 4x3 + z2 ) = 1.9825163
11
1
z3 = (35 − 6x3 − 3y3 ) = 0.9077262
12
For fourth approximation:
1
x4 = (20 + 3y3 − 2z3 ) = 3.0165121
8
1
y4 = (33 − 4x4 + z3 ) = 1.9856071
11
1
z4 = (35 − 6x4 − 3y4 ) = 0.9120088
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
30 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

For fifth approximation:

1
x5 = (20 + 3y4 − 2z4 ) = 3.0166005
8
1
y5 = (33 − 4x5 + z4 ) = 1.9859643
11
1
z5 = (35 − 6x5 − 3y5 ) = 0.9118753
12
For sixth approximation:
1
x6 = (20 + 3y5 − 2z5 ) = 3.0167678
8
1
y6 = (33 − 4x6 + z5 ) = 1.9858913
11
1
z6 = (35 − 6x6 − 3y6 ) = 0.9118099
12

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
31 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Solution

For seventh approximation:

1
x7 = (20 + 3y6 − 2z6 ) = 3.0167568
8
1
y7 = (33 − 4x7 + z6 ) = 1.9858894
11
1
z7 = (35 − 6x7 − 3y7 ) = 0.9118159
12
Since at the sixth and seventh approximations, the values of x, y , z are the same,
correct to four decimal places; We can stop the iteration process. Therefore

x = 3.0167
y = 1.9858
z = 0.9118

We find that 12 iterations are necessary in Gauss - Jacobi method to get the
same accuracy as achieved by 7 iterations in Gauss - Seidel method.
Jean Paul Nsabimana, Department
Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
32 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Exercise IV.4

1 Solve by Gauss- Seidel method the following system of equations

28x + 4y − z = 32
x + 3y + 10z = 24
2x + 17y + 4z = 35

2 Using Gauss-Seidel iteration method, find the solution of the following

system of equations correct to four decimal places.

10x − 2y − z − w = 3
−2x + 10y − z − w = 15
−x − y + 10z − 2w = 27
−x − y − 2z + 10w = −9

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
33 / 34
 Simultaneous Linear Algebraic Equations Iterative Method

Exercise IV.4

3. Solve the following system of linear equations by

1 Gauss-Jacobi method
2 Gauuss-seidel iteration method
(i)

2x + y + z = 4
x + 2y + z = 4
x + y + 2z = 4

(ii)

8x + y + z = 8
2x + 4y + z = 4
x + 3y + 5z = 5

Jean Paul Nsabimana, Department

Simultaneous
of Linear
Mathematics
Algebraic
University
Equations of 2020
December 24, Rwanda Em
34 / 34