1

MTS 202
NUMERICAL ANALYSIS 1
Contents
o Error analysis
o Approximation
o Transcendental equations
o Systems of linear equations

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
2

1.0 ERROR ANALYSIS

Most of the numerical methods developed for large numbers of calculation are too complex
therefore their implementation depends largely on the use of calculating aids which is capable
of obtaining result to the desired level of accuracy. Before we start implementing any
numerical method for computation, we shall discuss a few of the errors committed in
computational mathematics that may have adverse effect on the results.
1.1. TYPES OF ERRORS
1. Round off Error: this is the error committed when a number is abbreviated or some of its
digits are chopped off.
E.g.1: a)0.23456789 ≈ 0.235 (to 3-decimal places)
b) 84219 ≈ 84200 (to the nearest hundred)
2. Generated and Propagated Error: generated error is the error committed due to the use of
computing aids such as calculator and computers. The further use of this result with
generated error in computation produces another error known as the propagated error.
E.g.2 𝑢 = 5𝑣 2 + 6𝑣, given that 𝑣 = √2𝑏 and 𝑏 = 6.2
Solution
𝑣 = √2(6.2) = √12.4 = 3.521363372 ≈ 3.521 (To 3 decimal places)
Generated error = 3.521363372 − 3.521 = 0.00063372
𝑢∗ = 5(3.521)2 + 6(3.521) = 82.7265
2
𝑢 = 5(√2𝑏) + 6(√2𝑏) = 10𝑏 + 6√2𝑏 = 10(6.2) + 6(√12.4) = 83.12818023
𝑝𝑟𝑜𝑝𝑎𝑔𝑎𝑡𝑒𝑑 𝑒𝑟𝑟𝑜𝑟 = 83.12818023 − 82.7265 = 0.401680234
3. Truncation Error: this is the error committed due to the truncation of a series when trying
to simplify the amount of function evaluated in the series.
1
E.g.3: use the binomial expansion of (1 − 𝑥) ⁄2 to expand √0.98 at 𝑥 = 0.02
Solution
1 1 1 1 5 4
(1 − 𝑥) ⁄2 = 1 − 𝑥 − 𝑥 2 − 𝑥 3 − 𝑥 −⋯
2 8 16 128
2
Suppose we truncate at 𝑥 by neglecting powers above 2 we have
1 1 1
(1 − 𝑥) ⁄2 = 1 − 𝑥 − 𝑥 2 + 𝐸
2 8
Where E is the truncation error
1 3 5 4
𝐸= 𝑥 − 𝑥 −⋯
16 128
1 1 1 1
√0.98 = (0.98) ⁄2 = (1 − 0.02) ⁄2 = 1 − (0.02) − (0.02)2
2 8
= 1 − 0.01 − 5 × 10−5 = 0.98995
But √0.98 = 0.989949493 which is the calculator value
Truncation error 𝐸 = 0.989949493 − 0.98995 = 5.063 × 10−7
4. Relative Error: from E.g.1 above, the error in u can be relative depending on the evaluation
of v. the error may be reduced if u is evaluated directly as follows
2
𝑢 = 5(√2𝑏) + 6(√2𝑏) = 10𝑏 + 6√2𝑏 = 10(6.2) + 6(√12.4) = 83.12818023
1.2 USEFULL FORMULARS
 𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝐸𝑟𝑟𝑜𝑟 = |𝑓(𝑥) − 𝑓 ∗ (𝑥)|
𝑓(𝑥)−𝑓∗ (𝑥)
 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐸𝑟𝑟𝑜𝑟 = | 𝑓(𝑥)
|
𝑓(𝑥)−𝑓∗ (𝑥)
 𝑃𝑒𝑐𝑒𝑛𝑡𝑎𝑔𝑒 𝐸𝑟𝑟𝑜𝑟 = | 𝑓(𝑥) | × 100%

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
3

1.3 SOURCES OF ERROR

 Inherent Error: this is the error which is already present in the given data
 Computational Error: this is the error committed due to the use of computing aids. It is also
known as round off error.
 Human Error: this is the error committed in the process of measurement, collection and
computation of data.
2.0. APPROXIMATION
Some methods of approximation are as follows:

2.1 Binomial Expansion:

This is a series use for expanding a function of the form (𝑎 + 𝑏)𝑛 where 𝑛 ∈ 𝑅. The binomial
expansion for a function (𝑎 + 𝑏)𝑛 is:
𝑎𝑛 𝑏0 𝑛𝑎𝑛−1 𝑏1 𝑛(𝑛 − 1)𝑎𝑛−2 𝑏2 𝑛(𝑛 − 1) ⋯ (𝑛 − 𝑛 + 1)𝑎0 𝑏 𝑛
(𝑎 + 𝑏)𝑛 = + + + ⋯+
0! 1! 2! 𝑛!
1
E.g. Using binomial expansion, approximate the function 𝑓(𝑥) = at x=4. Calculate the
(𝑥−1)1⁄2
relative error.

Solution

1
𝑓(𝑥) = = (𝑥 − 1)−1⁄2
(𝑥 − 1)1⁄2

(−1)0 −1⁄2 (− 1⁄2)(−1)1 −3⁄4 (− 1⁄2)(− 3⁄2)(−1)2 −5⁄2

= 𝑥 + 𝑥 + 𝑥 +⋯
0 1! 2!
1 3 5
= 𝑥 −1⁄2 + 𝑥 −3⁄4 + 𝑥 −5⁄2 + 𝑥 −7⁄2 + ⋯
2 8 16
Then at x=4, we have

1 3 5
𝑓 ∗ (4) = (4)−1⁄2 + (4)−3⁄4 + (4)−5⁄2 + (4)−7⁄2 + ⋯
2 8 16
901
= = 0.577194106
1561
1 1
𝑓(4) = 1 ⁄2
= = 0.577350269
(4 − 1) √3

𝑓(4) − 𝑓 ∗ (4) 0.577350269 − 0.577194106

𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐸𝑟𝑟𝑜𝑟 = | |=| |
𝑓(4) 0.577350269

= 2.70 × 10−4

EXERCICES

Using the first five terms of binomial expansion of:

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
4

1. 𝑓(𝑥) = √𝑥 + 1 evaluate √7,

1 1
2. 𝑔(𝑦) = 1+𝑦 Evaluate 1.2
.

Also, calculate their respective relative error.

2.2 Taylor Series:

This is a series for expressing functions which have no polynomial in a polynomial form. The
Taylor series expansion for the function 𝑓(𝑥) about a point 𝑥 = 𝑎is given as:

𝑓 ′′ (𝑎) 𝑓 ′′′ (𝑎) 𝑓 𝑛 (𝑎)

𝑓(𝑥) = 𝑓(𝑎) + (𝑥 − 𝑎)𝑓 ′ (𝑎) + (𝑥 − 𝑎)2 + (𝑥 − 𝑎)3 + ⋯ + (𝑥 − 𝑎)𝑛
2! 3! 𝑛!
Where 𝑓 𝑛 (𝑎) is the nth derivative of 𝑓(𝑥)𝑎𝑡 𝑥 = 𝑎.

2.3 Maclaurin’s Series:

This is a division of Taylor series expansion only about the point 𝑥 = 0. The
Maclaurin’s series expansion for the function 𝑓(𝑥) is given as:
𝑓 ′′ (0) 2 𝑓 ′′′ (0) 3 𝑓 𝑛 (0) 𝑛
𝑓(𝑥) = 𝑓(0) + 𝑥𝑓 ′ (0) + 𝑥 + 𝑥 + ⋯+ 𝑥
2! 3! 𝑛!

E.g. Evaluate 𝑓(5) using the first four terms of Maclaurin’s series expansion of 𝑓(𝑥) = 𝐼𝑛(1 + 𝑥).
Also calculate relative error.

Solution

𝑓′′ (0) 2 𝑓′′′ (0) 3 𝑓𝑛 (0) 𝑛

𝑓(𝑥) = 𝑓(0) + 𝑥𝑓 ′ (0) + 𝑥 + 𝑥 + ⋯+ 𝑥 ………. (1)
2! 3! 𝑛!

𝑓(𝑥) = 𝐼𝑛(1 + 𝑥) 𝑓(0) = 𝐼𝑛(1) = 0.

1 1
𝑓 ′ (𝑥) = 𝑓 ′ (0) = =1
(1+𝑥) (1)

1 1
𝑓 ′′ (𝑥) = − (1+𝑥)2 𝑓 ′′ (0) = − (1)2 = −1

2 2
𝑓 ′′′ (𝑥) = (1+𝑥)3 𝑓 ′′′ (0) = (1)3 = 2

Hence equation (1) becomes

(−1) 2 (2) 3
𝑓(𝑥) = (0) + 𝑥(1) +
𝑥 + 𝑥
2! 3!
1 1
= 𝑥 − 𝑥2 + 𝑥3
2 3
𝑔𝑖𝑣𝑒𝑛
1 1 205
𝑓 ∗ (5) = 5 − (5)2 + (5)3 = = 34.1666666667
2 3 6
Then 𝑓(5) = 𝐼𝑛 (1 + 5) = 𝐼𝑛 6 = 1.791759469
𝑓(5) − 𝑓 ∗ (5) 1.791759469 − 34.1666666667
𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐸𝑟𝑟𝑜𝑟 = | |=| | = 18.06877974
𝑓(5) 1.791759469

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
5

EXERCISES

Using the first five terms of Maclaurin’s series of:

a. 𝑓(𝑥) = sin(5 + 𝑥), evaluate sin 5.4

b. 𝑓(𝑥) = 𝑒 2+𝑥 , evaluate 𝑒 4.1.

Also evaluate their respective relative error.

3.0. TRANSCEDENTAL EQUATIONS

These are equations which contain polynomial, trigonometry, logarithm and exponential functions.
Examples are: tan 𝑥 − 𝑒 𝑥 , sin 𝑥 − 𝑥𝑒 2𝑥 ,etc. These equations may have one of more roots and the
solutions are also known as the zeros of the equation.

Solutions to transcendental equations

1. Direct/ Analytical method

There is no direct method for solving transcendental equations or higher order algebraic equations.

2. Numerical methods

The numerical methods are:

a. Two point method

i) Bisection /Bolzano/Bracketing interval method
ii) False position/ Regular Falsi method
b. One point method
i) Newton’s/Newton Raphson’s method
ii) Iteration/Fixed point iteration method

3.1 BISECTION METHOD

Theorem: suppose the function 𝑓(𝑥) is continuous and defined in an interval I=[a,b]. We also
suppose that at least one of the roots of 𝑓(𝑥) lie within this interval. Then the value of 𝑓(𝑥) at x=a
and x=b must have positive signs such that

𝑓(𝑎). 𝑓(𝑏) < 0

Given the interval I=[a, b], we calculate the value of 𝑓(𝑥) at the midpoint x0 of the interval
𝑎+𝑏
i.e. 𝑥0 = 2

Conditions for bisection method

 If 𝑓(𝑎). 𝑓(𝑏) < 0, solution lies within the interval,

 If 𝑓(𝑎). 𝑓(𝑏) > 0, solution does no lie within the interval,
 Iteration continues until

Example (Tutorial sheet b -Q3a)

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
6

Use bisection method to approximate the root of the following

𝑓(𝑥) = 2𝑥 3 − 5

Try the starting the interval [1, 2]

Solutions

𝑓(𝑥) = 2𝑥 3 − 5

𝑓(1) = 2(1)3 − 5 = −3
𝑓(2) = 2(2)3 − 5 = 11
Since𝑓(1)𝑓(2) = −33 < 0, then the root lies between the interval [1, 2]
𝑎+𝑏
𝑥1 = , 𝑎 = 1 𝑎𝑛𝑏 𝑏 = 2
2
1+2
=> 𝑥1 = = 1.5
2
𝑓(1.5) = 2(1.5)3 − 5 = 1.75
F(x)

Since𝑓(1)𝑓(1.5) = −5.25 < 0, then the root lies between the interval [1, 1.5]
1 + 1.5
𝑥2 = = 1.25
2
3
𝑓(1.25) = 2(1.25) − 5 = −1.09375

Since 𝑓(1.25)𝑓(1.5) − 1.91406 < 0, then the root lies between the interval [1.25, 1.5]
1.25 + 1.5
𝑥3 = = 1.375
2
𝑓(1.375) = 2(1.375)3 − 5 = 0.1992
Since 𝑓(1.25)𝑓(1.375) = −0.2179 < 0, then the root lies between the interval [1.25, 1.375]

Iteration continues till

1. The midpoint 𝑥𝑛 , coincides with the root i.e. 𝑓(𝑥𝑛 ) = 0 OR

2. The length of search interval is less than some prescribed value called tolerance
𝑖. 𝑒. |𝑋𝑛+1 − 𝑋𝑛 | ≤ 𝑡𝑜𝑙𝑒𝑟𝑎𝑛𝑐𝑒

3.2 FALSE POSITION/REGULAR FALSI

Derivation of formula

Given 𝑓(𝑥) = 0 and [a, b]

Recall the equation 𝑓(𝑥) = 𝑚𝑥 + 𝑐 ……….3.0

When x=a, 𝑓(𝑎) = 𝑎𝑚 + 𝑐 ……….3.1

When x=b 𝑓(𝑏) = 𝑏𝑚 + 𝑐 ………3.2

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
7

Equation (3.2)-(3.1) gives 𝑓(𝑏) − 𝑓(𝑎) = 𝑚(𝑏 − 𝑎)

𝑓(𝑏)−𝑓(𝑎)
𝑚= 𝑏−𝑎
………3.3

𝑓(𝑎)−𝑐
From (3.1), we have 𝑚= 𝑎
………3.4

𝑓(𝑏)−𝑐
And from (3.2), we have 𝑚= 𝑏
………3.5

𝑓(𝑎)−𝑐 𝑓(𝑏)−𝑐
Equating 3.4 and 3.5, we have 𝑎
= 𝑏

𝑏(𝑎) − 𝑏𝑐 = 𝑎𝑓(𝑏) − 𝑎𝑐

𝑏𝑓(𝑎) − 𝑎𝑓(𝑏) = 𝑐(𝑏 − 𝑎)

𝑏𝑓(𝑎)−𝑎𝑓(𝑏)
𝑐= 𝑏−𝑎
……..3.6

Putting 𝑓(𝑥0 ) = 0 𝑖𝑛𝑡𝑜 3.0, we have 𝑚𝑥0 + 𝑐 = 0 =>

𝑐
𝑥0 = − 𝑚 ………3.7

Putting 3.3 and 3.6 into 3.7, we have

𝑏𝑓(𝑎) − 𝑎𝑓(𝑏) 𝑏−𝑎

𝑥0 = − ( )( )
𝑏−𝑎 𝑓(𝑏) − 𝑓(𝑎)

𝑎𝑓(𝑏) − 𝑏𝑓(𝑎)
𝑥0 =
𝑓(𝑏) − 𝑓(𝑎)

Example (Tutorial sheet b -Q4a)

Use the False position method to approximate the root of the following

a) 𝑓(𝑥) = 2𝑥 3 − 5
Try the starting the interval [1,2]

Solution

a) 𝑓(𝑥) = 2𝑥 3 − 5
𝑎𝑓(𝑏)−𝑏𝑓(𝑎)
Using 𝑥𝑜 = 𝑓(𝑏)−𝑓(𝑎)
where a=1 and b=2

𝑓(𝑎) = 𝑓(1) = 2(1)3 − 5 = −3

𝑓(𝑏) = 𝑓(2) = 2(2)3 − 5 = 11

Since 𝑓(1)𝑓(2) = −33 < 0, then the root lies between the interval [1,2]
(1)(11) − (2)(−3) 11 + 6
𝑥𝑜 = = = 1.21429
11 − (−3) 14

𝑓(𝑥𝑜) = 𝑓(1.2142) = 2(1.21429)3 − 5 = −1.41906

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
8

Since 𝑓(1.21429)𝑓(2) < 0, then the root lies between the interval [1.21429,2]

(1.21429)(11) − (2)(−1.41906) 13.35719 + 2.83812

𝑥1 = = = 1.304069
11 − (−1.41906) (12.41906)

𝑓(𝑥1 ) = 𝑓(1,304069) = 2(1.304069)3 − 5 = −0.564611

Since 𝑓(1.304069)𝑓(2) < 0, then the root lies between the interval [1.304069, 2]

(1.304069)(11) − (2)(−0.564611) 14.344759 + 1.129222

𝑥2 = = = 1.338046
11— (−0.564611) 11.564611

𝑓(𝑥2 ) = 𝑓(1.338046) = 2(1.338046)3 − 5 = −0.208813

Since 𝑓(1.338046)𝑓(2) < 0, then the root lies between the interval [1.338046, 2]
(1.338046)(11) − (2)(−0.208813) 14.718506 + 0.417626
𝑥3 = = = 1.350378
11— (−0.208813) 11.208813
𝑓(𝑥3 ) = 𝑓(1.350378) = 2(1.350378)3 − 5 = −0.075115

Since 𝑓(1.350378)𝑓(2) < 0, then the root lies between the interval [1.350378, 2]

(1.350378)(11) − (2)(−0.075115) 14.854158 + 0.15023

𝑥4 = = = 1.354784
11— (−0.075115) 11.075115

𝑓(𝑥3 ) = 𝑓(1.354784) = 2(1.354784)3 − 5 = −0.026751

Since 𝑓(1.354784)𝑓(2) < 0, then the root lies between the interval [1.350378, 2]

Iteration continues till

1. The midpoint coincides with the root OR

2. The length of search interval is less than some prescribed value called tolerance
𝒊. 𝒆. |𝑋𝑛+1 − 𝑋𝑛 | ≤ 𝑡𝑜𝑙𝑒𝑟𝑎𝑛𝑐𝑒

3.3 NEWTON’S/NEWTON RAPHSON’S METHOD

A more poplar iteration developed initially by Newton and later modified by Raphson. This method is
only applicable to a differentiable function and it is a more efficient procedure for evaluating the root
of equation 𝑓(𝑥) = 0

Derivation of the method

Given 𝑓(𝑥) = 0

Where 𝑓(𝑥) is differentiable about 𝑥0 , then the Taylor series expansion for 𝑓(𝑥) is given as:

𝑓 ′′ (𝑥0 ) 𝑓 𝑛 (𝑥0 )
𝑓(𝑥) = 𝑓(𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ (𝑥0 ) + (𝑥 − 𝑥0 )2 + ⋯ + (𝑥 − 𝑥0 )𝑛
2! 𝑛!

Assume that (𝑥 − 𝑥0 ) is small, then we neglect order 2 and above. Approximating 𝑓(𝑥) = 0, we have

𝑓(𝑥) = 𝑓(𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ (𝑥0 ) = 0

𝑓(𝑥0 ) + 𝑥𝑓 ′ (𝑥0 ) − 𝑥0 𝑓 ′ (𝑥0 ) = 0

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
9

𝑥𝑓 ′ (𝑥0 ) = 𝑥0 𝑓 ′ (𝑥0 ) − 𝑓(𝑥0 )

𝑓(𝑥0 )
𝑥 = 𝑥0 −
𝑓 ′ (𝑥0 )

Generally,

𝑓(𝑥𝑛 )
𝑥𝑛+1 = 𝑥𝑛 −
𝑓 ′ (𝑥𝑛 )

Example (Tutorial sheet B –Q5)

Use the Newton-Raphson’s method to obtain the root of the equation 𝑥 3 − 𝑥 2 − 𝑥 − 1 = 0 within the
interval 1.6 < 𝑥 < 2.1

Solution

𝑓(𝑥) = 𝑥 3 − 𝑥 2 − 𝑥 − 1

𝑓(1.6) = (1.6)3 − (1.6)2 − (1.6) − 1 = −1.064

𝑓(2.1) = (2.1)3 − (2.1)2 − (2.1) − 1 = 1.751

Since 𝑓(1.6). 𝑓(2.1) < 0, then the root lies within the interval (1.6, 2.1)

2.1 + 1.6
𝑥0 = = 1.85
2

𝑓(1.85) = (1.85)3 − (1.85)2 − (1.85) − 1 = 0.059125

𝑓 ′ (𝑥) = 3𝑥 2 − 2𝑥 − 1

𝑓 ′ (1.85) = 3(1.85)2 − 2(1.85) − 1 = 5.5675

𝑓(𝑥0 )
𝑥1 = 𝑥0 −
𝑓 ′ (𝑥0 )

0.059125
𝑥1 = 1.85 − = 1.83938
5.5675
Iteration continues till

1. The midpoint 𝑥𝑛 , coincides with the root i.e. 𝑓(𝑥𝑛 ) = 0 OR

2. The length of search interval is less than some prescribed value called tolerance
𝑖. 𝑒. |𝑋𝑛+1 − 𝑋𝑛 | ≤ 𝑡𝑜𝑙𝑒𝑟𝑎𝑛𝑐𝑒

3.4 FIXED POINT ITERATION

THEOREM: Let 𝑥 = 𝑥0 be the zero (solution) of f(x) =0 and I be an interval containing the point 𝑥 =
𝑥0 . Let 𝜑(𝑥)𝑎𝑛𝑑 𝜑′ (𝑥) be define and continuous in the Interval I, the arrangement of f(x) =0 be

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
10

defined as 𝑥𝑖 = 𝜑𝑖(𝑥) , (𝑖 = 1,2,3 … . ) then if | 𝜑(𝑥)| < 𝑘 for all𝑥 ∈ 𝐼, 𝑘 < 1, the sequence {xk}
converges to the point x0.

EXAMPLE (Tutorial sheet B –Q7)

Solve 𝑥 2 − 2𝑥 − 8 = 0 using Fixed Iteration Method if the starting value is 𝑥0 = 5

Solution

We get the possible arrangement by making the x in 𝑥 2 − 2𝑥 − 8 = 0 subject of the formula i.e.

1. 2𝑥 = 𝑥 2 − 8
𝒙𝟐 −𝟖
𝒙𝟏 = 𝟐 = 𝝋𝟏 (𝒙)
2
2. 𝑥 = 2𝑥 + 8
𝒙𝟐 = √𝟐𝒙 + 𝟖 = 𝝋𝟐 (𝒙)
3. 𝑥 2 = 2𝑥 + 8 (divide both sides by x)
𝟐𝒙−𝟖
𝒙𝟑 = 𝒙
= 𝝋𝟑 (𝒙)

4. 𝑥(𝑥 − 2) = 8
𝟖
𝒙𝟒 = 𝒙−𝟐 = 𝝋𝟒 (𝒙)
5. 𝑥 2 − 2𝑥 = 8
𝑥 2 − 2𝑥 + 𝑥 = 8 + 𝑥

𝑥2 − 𝑥 = 8 + 𝑥

𝑥(𝑥 − 1) = 8 + 𝑥
𝟖+𝒙
𝒙𝟓 = (𝒙−𝟏) = 𝝋𝟓 (𝒙)
Putting x=5 into the arrangements
𝟓𝟐 −𝟖 𝟐𝟓−𝟖
1. 𝜑1 (𝑥) = 𝟐
= 𝟐
= 8.5
2. 𝜑2 (𝑥)=√2(5) + 8 = √18 = 4.2426
2(5)−8
3. 𝜑3 (𝑥) = 5
= 0.25
8
4. 𝜑4 (𝑥) = 5−2
= 2.6667
8+5
5. 𝜑5 (𝑥) = (5−1)
= 0.75

4.0. SYSTEM OF LINEAR EQUATIONS

Linear equations are equations of the form:

𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ 𝑎1𝑛 𝑥𝑛 = 𝑏1

𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 + ⋯ 𝑎24 𝑥𝑛 = 𝑏2

⋮ ⋮ ⋮ ⋮ ⋮

𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + 𝑎𝑚3 𝑥3 + ⋯ 𝑎𝑚𝑛 𝑥𝑛 = 𝑏𝑚

The example above is a system of m linear equation with n unknowns (where 𝑛, 𝑚 ∈ 𝑅) and it can be
written in mayrix form as:

𝐴𝑋 = 𝐵

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
11

Where; A= matrix of the coefficients 𝑎𝑖𝑗 (𝑖 = 1,2,3, … , 𝑛 & 𝑗 = 1,2,3, … , 𝑚)., X= matrix of the
unknown 𝑥𝑖 (𝑖 = 1,2,3, … , 𝑛) and B= matrix of the constants 𝑏𝑗 (𝑗 = 1,2,3, … , 𝑚),

Then we have
𝑎11 𝑎12 𝑎13 … 𝑎1𝑛 𝑥1 𝑏1
𝑎21
( … 𝑎22 𝑎23 …
… … 𝑎 𝑥 𝑏
2𝑛 ) ( 2 ) = ( 2 )
… … … …
𝑎𝑚1 𝑎𝑚2 𝑎𝑚3 . . 𝑎𝑚𝑛 𝑥3 𝑏𝑚

4.1. USEFULL HINTS ABOUT MATRIX

Matrix A is said to be:

 Singular or invertible if the determinant of A denoted by |𝐴| = 0

0, 𝑓𝑜𝑟 𝑖 < 𝑗
 Lower triangular if 𝐴 = { 1, 𝑓𝑜𝑟 𝑖 = 𝑗 , that is
𝑎𝑖𝑗 , 𝑓𝑜𝑟 𝑖 > 𝑗
The lower triangular matrix for a 3 × 3 matrix is
1 0 0
(𝑎21 1 0)
𝑎31 𝑎32 1
0, 𝑓𝑜𝑟 𝑖 < 𝑗
 Upper triangular matrix if 𝐴 = {
𝑎𝑖𝑗 , 𝑓𝑜𝑟 𝑖 ≥ 𝑗
The upper triangular matrix for a 3 × 3 matrix is
𝑎11 𝑎12 𝑎13
( 0 𝑎22 𝑎23 )
0 0 𝑎33

 The equations is said to be homogenous if B=0

4.2. SOLUTIONS TO SYSTEM OF LINEAR EQUATIONS

A. Direct method
a. Matrix inversion
b. Crammers’ rule
c. Gauss’ or Gaussian elimination method
d. Gauss Jordan method
e. Choleski’s method (L-U decomposition)
B. Indirect or iterative method
a. Jacobi (Gauss-Jacobi) method
b. Gauss-Siedel method
A. DIRECT METHOD

4.2.1 MATRIX INVERSION

Given a system of equations which has been expressed in the form

𝐴𝑋 = 𝐵

We are required to solve for values of the unknowns Xi,

We have that 𝑋 = 𝐴−1 𝐵

To determine A-1, we follow the following processes;

 Determine the determinant of matrix A i.e. |𝐴|

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
12

 Determine the minor matrix of A

 Determine the cofactor of A
 Determine the adjoint of A
1
 𝐴−1 = |𝐴| × Adjoint of A

EXAMPLE (Tutorial sheet C, Question 3)

By matrix inversion method, solve the following system of equation

𝑥1 + 2𝑥2 − 6𝑥3 = 5

2𝑥1 − 4𝑥2 + 5𝑥3 = 6

3𝑥1 − 16𝑥2 + 9𝑥3 = 4

Solution

Rewriting the equations in a matrix format following AX=B

1 2 −6 𝑥1 5
(2 −4 5 ) (𝑥2 ) = (6)
3 −16 9 𝑥3 4

1 2 −6
Det A = |𝐴| = [2 −4 5 ] = (−36 + 80) − 2(18 − 15) − 6(−32 + 12) = 44 − 6 + 120
3 −16 9
= 158
−36 + 80 18 − 5 −32 + 12 44 3 −20
𝑚𝑖𝑛𝑜𝑟 𝑜𝑓 𝐴 = ( 18 − 96 9 + 18 −16 − 6 ) = (−78 27 −22)
10 − 24 5 + 12 −4 − 4 −14 17 −8
44 3 −20 + − + 44 −3 −20
𝑐𝑜𝑓𝑎𝑐𝑡𝑜𝑟 𝑜𝑓 𝐴 = (−78 27 −22) (− + −) = ( 78 27 22 )
−14 17 −8 + − + −14 −17 −8
44 78 −14
𝑎𝑑𝑗𝑜𝑖𝑛𝑡 𝑜𝑓 𝐴 = ( −3 27 −17)
−20 22 −8
22 39 7
−
79 79 79
1 1 44 78 −14 3 27 17
𝐴−1 = × Adjoint of A = × ( −3 27 −17) = − −
|𝐴| 158 158 158 158
−20 22 −8
10 11 4
−
( 79 −
79 79 )
22 39 7 110 + 234 − 28
− 4
79 79 79 5 79 1
−1 3 27 17 6 −15 + 162 − 68
𝑋=𝐴 𝐵= − − ( )= = 2
158 158 158 4 158 20
10 11 4 −50 + 66 − 36
( − 79 − (79)
79 79 ) ( 79 )
1 20
𝑥1 4 , 𝑥2 = , 𝑥3 =
2 79
4.2.2. CRAMMERS’ RULE

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
13

EXAMPLE (Tutorial sheet C -question six)

By using Crammer’s rule, solve the following system of equation

2𝑥1 + 𝑥2 − 5𝑥3 + 𝑥4 = 8

𝑥1 − 3𝑥2 − 6𝑥4 = 9

2𝑥2 − 𝑥3 + 2𝑥4 = −5

𝑥1 + 4𝑥2 − 7𝑥3 + 6𝑥4 = 0

Solution

Rewriting the equations in a matrix form following AX=B

2 1 −5 1 𝑥1 8
𝑥
−6 ) ( 2 ) = ( 9 )
(1 −3 0
𝑥3
0 2 −1 2 −5
1 4 −7 6 𝑥4 0
2 1 −5 1
∆= |𝐴| = |1 −3 0 −6|
0 2 −1 2
1 4 −7 6
−3 0 −6 1 0 −6 1 −3 −6 1 −3 0
∆= 2 | 2 −1 2 | − |0 −1 2 | − 5 |0 2 2 | − |0 2 −1|
4 −7 6 1 −7 6 1 4 6 1 4 −7
= 2[−3(−6 + 14) − 0 − 6(−6 + 14)]— [(−6 + 14) − 0 − 6(0 + 1)]
− 5[(12 − 8) + 3(0 − 2) − 6(0 − 2)] − [(−14 + 4) + 3(0 + 1) + 0]

= 2(−24 + 60) − (8 − 6) − 5(4 − 6 + 12) − (−10 + 3)

= 72 − 2 − 50 + 7

∆= 27

Replace the first column of matrix A by matrix B and calculate the determinant of the resulting matrix
∆𝑥1
8 1 −5 1
∆𝑥1 = | 9 −3 0 −6|
−5 2 −1 2
0 4 −7 6
−3 0 −6 9 0 −6 9 −3 −6 9 −3 0
∆𝑥1 = 8 | 2 −1 2 | − |−5 −1 2 | − 5 |−5 2 2 | − |−5 2 −1|
4 −7 6 0 −7 6 0 4 6 0 4 −7
= 8[−3(−6 + 14) − 0 − 6(−6 + 14)] − [9(−6 + 14) − 0 − 6(35 − 0)]
− 5[9(12 − 8) + 3(−30 − 0) − 6(−20 − 0)] − [9(−14 + 4) + 3(35 − 0) + 0]

= 8(−24 + 60) − (72 − 210) − 5(36 − 90 + 120) − (−90 + 105)

= 288 + 138 − 330 − 15

∆𝑥1 = 81

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
14

Replace the second column of matrix A by matrix B and calculate the determinant of the resulting
matrix ∆𝑥2

2 8 −5 1
∆𝑥2 = |1 9 0 −6|
0 −5 −1 2
1 0 −7 6
9 0 −6 1 0 −6 1 9 −6 1 9 0
= 2 |−5 −1 2 | − 8 |0 −1 2 | − 5 | 0 −5 2 | − |0 −5 −1|
0 −7 6 1 −7 6 1 0 6 1 0 −7
= 2[9(−6 + 14) − 0 − 6(35 − 0)] − 8[(−6 + 14) − 0 − 6(0 + 1)]

−5[(−30 − 0) − 9(0 − 2) − 6(0 + 5)] − [(35 − 0) − 9(0 + 1) + 0]

= 2(72 − 210) − 8(8 − 6) − 5(−30 + 18 − 30) − (35 − 9)

= −276 − 16 + 210 − 26

∆𝑥2 = −108

Replace the third column of matrix A by matrix B and calculate the determinant of the resulting
matrix ∆𝑥3
2 1 8 1
∆𝑥3 = |1 −3 9 −6|
0 2 −5 2
1 4 0 6
−3 9 −6 1 9 −6 1 −3 −6 1 −3 9
= 2 | 2 −5 2 | − |0 −5 2 | + 8 |0 2 2 | − |0 2 −5|
4 0 6 1 0 6 1 4 6 1 4 0
2[−3(−30 − 0) − 9(12 − 8) − 6(0 + 20)]— [(−30 − 0) − 9(0 − 2) − 6(0 + 5)]

+8[(12 − 8) + 3(0 − 2) − 6(0 − 2)] − [(0 + 20) + 3(0 + 5) + 9(0 − 2)]

= 2(90 − 36 − 120)— (−30 + 18 − 30) + 8(4 − 6 + 12) − (20 + 15 − 18)

= −132 + 42 + 80 − 17

∆𝑥3 = −27

Replace the fourth column of matrix A by matrix B and calculate the determinant of the resulting
matrix ∆𝑥4
2 1 −5 8
∆𝑥4 = |1 −3 0 9|
0 2 −1 −5
1 4 −7 0
−3 0 9 1 0 9 1 −3 9 1 −3 0
= 2 | 2 −1 −5| − |0 −1 −5| − 5 |0 2 −5| − 8 |0 2 −1|
4 −7 0 1 −7 0 1 4 0 1 4 −7
= 2[−3(0 − 35) − 0 + 9(−14 + 4)] − [(0 − 35) − 0 + 9(0 + 1)]
− 5[(0 + 20) + 3(0 + 5) + 9(0 − 2)] − 8[(−14 + 4) + 3(0 + 1) + 0]

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
15

= 2(105 − 90)— (−35 + 9) − 5(20 + 15 − 18) − 8(−10 + 3)

30 + 26 − 85 + 56

∆𝑥4 = 27
∆𝑥1 81
𝑥1 = = =3
∆ 27
∆𝑥2 108
𝑥2 = =− = −4
∆ 27
∆𝑥3 27
𝑥3 = =− = −1
∆ 27
∆𝑥4 27
𝑥4 = = =1
∆ 27
4.2.3. GAUSS’ (GAUSSIAN ELIMINATION) METHOD

This method is used to solve a system of linear equations by transforming it to an upper triangular
matrix using elementary row operations. The resulting matrix is then solved using the backward
substitution method. The elementary row operations are:

 Interchange any two row

 Addition of a scalar multiple of one row to another
 Division of a row by a non-zero scalar
EXAMPLE (Tutorial sheet C – Q9)

Solve the system of equations by using Gaussian Elimination Method

5𝑥1 − 2𝑥2 + 3𝑥3 = 0

22𝑥1 + 1𝑥2 − 4𝑥3 = −7

3𝑥1 − 8𝑥2 + 4𝑥3 = 9

Solution

5 −2 3 ⋮ 0 𝑅1 5 −2 3 ⋮ 0 𝑅1 5 −2 3 ⋮ 0
(22 1 −4 ⋮ −7) ~ 5𝑅2 − 2𝑅1 ( 0 9 −26 ⋮ 35 ) ~ 𝑅2 ( 0 9 −26 ⋮ −35 )
3 −8 4 ⋮ 9 5𝑅3 − 3𝑅1 0 −34 11 ⋮ 45 9𝑅3 + 34𝑅2 0 0 −785 ⋮ −785

Using back substitution,

−785𝑥3 = −785

𝑥3 = 1

9𝑥2 − 26𝑥3 = −35

−35 + 26
𝑥2 = = −1
9
5𝑥1 − 2𝑥2 + 3𝑥3 = 0
−2 − 3
𝑥1 = = −1
5

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
16

4.2.4. GAUSS JORDAN METHOD

This is an improvement of Gaussian elimination method and it provides an easy way of obtaining
results of system of equations. It reduces the matrix A of the system AX=B to a diagonal matrix by
using the echelon reduction process

EXAMPLE (Tutorial sheet C -question twelve a)

Use Gauss-Jordan method to solve the following system of equations

𝑥1 + 2𝑥2 + 𝑥3 = 8
2𝑥1 + 3𝑥2 + 4𝑥3 = 20

4𝑥1 + 3𝑥2 + 2𝑥3 = 16

Solution

1 2 1 ⋮ 8 𝑅1 1 2 1 ⋮ 8
(2 3 4 ⋮ 20) ~ 2𝑅1 − 𝑅2 (0 1 −2 ⋮ −4 )
4 3 2 ⋮ 16 4𝑅1 − 𝑅3 0 5 2 ⋮ 16

𝑅1 1 2 1 ⋮ 8 𝑅1 1 2 1 ⋮ 8
~ 𝑅2 (0 1 −2 ⋮ −4) ~ 𝑅 2 (0 1 −2 ⋮ −4)
𝑅3 − 5𝑅2 0 0 12 ⋮ 36 𝑅3 /12 0 0 1 ⋮ 3

𝑅1 − 𝑅3 1 2 0 ⋮ 5 𝑅1 − 2𝑅2 1 0 0 ⋮ 1
~ 𝑅2 + 2𝑅2 (0 1 0 ⋮ 2) ~ 𝑅2 (0 1 0 ⋮ 2)
𝑅3 0 0 1 ⋮ 3 𝑅3 0 0 1 ⋮ 3

Using forward substitution,

𝑥1 = 1, 𝑥2 = 2 and 𝑥3 = 3

4.2.5. CHOLESKI’S METHOD

This method is sometimes called L-U decomposition (i.e. lower-upper triangular matrix
decomposition). In this method, we

 Decompose matrix A in the system AX=B into the products of lower triangular matrix L and
upper triangular matrix U, using
𝐿𝑈 = 𝐴
 Now solve the system of triangular matrices
𝐿𝑌 = 𝐵 𝑎𝑛𝑑 𝑈𝑋 = 𝑌

EXAMPLE (Tutorial sheet c –Q14)

Use Choleski’s method to solve the following system of equations

0.610𝑥1 + 1.23𝑥2 + 17.2𝑥3 = 0.72

1.02𝑥1 + 2.15𝑥2 − 5.51𝑥3 = 12.0

−4.34𝑥1 + 112𝑥2 − 4.25𝑥3 = 16.3

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
17

Solution

0.610 1.23 17.2 𝑥1 0.792

( 1.02 2.15 −5.51) (𝑥2 ) = ( 12.0 )
−4.34 11.2 −4.25 𝑥3 16.3

1 0 0 𝑈11 𝑈12 𝑈13 0.610 1.23 17.2

(𝐿21 1 0) ( 0 𝑈22 𝑈23 ) = ( 1.02 2.15 −5.51)
𝐿31 𝐿32 1 0 0 𝑈33 −4.34 11.2 −4.25

𝑈11 𝑈12 𝑈13 0.610 1.23 17.2

(𝐿21 𝑈11 𝐿21 𝑈12 + 𝑈22 𝐿21 𝑈13 + 𝑈23 ) = ( 1.02 2.15 −5.51)
𝐿31 𝑈11 𝐿31 𝑈12 + 𝐿32 𝑈22 𝐿31 𝑈13 + 𝐿32 𝑈23 + 𝑈33 −4.34 11.2 −4.25

Comparing the two matrices we have

𝑈11 = 0.610, 𝑈12 = 1.23, 𝑈13 = 17.2

1.02
𝐿21 𝑈11 = 1.02 => 𝐿21 = = 1.67213
0.610

𝐿21 𝑈12 + 𝑈22 = 2.15 => 𝑈22 = 2.15 − (1.67213 × 1.23) = 0.09328

𝐿21 𝑈13 + 𝑈23 = −5.51 => 𝑈23 = −5.51 − (1.67213 × 17.2) = −34.27064
4.34
𝐿31 𝑈11 = −4.34 => 𝐿31 = − 0.610 = −7.11475

11.2−(−7.11475×1.23)
𝐿31 𝑈12 + 𝐿32 𝑈22 = 11.2 => 𝐿32 = 0.09328
= 213.88446

𝐿31 𝑈13 + 𝐿32 𝑈23 + 𝑈33 = −4.25

=> 𝑈33 = −4.25 − (7.11475 × 17.2) − (213.88466 × −34.27064) = 7448.087887

1 0 0 0.610 1.23 17.2

𝐿 = ( 1.67213 1 0) 𝑈=( 0 0.09328 −34.27064 )
−7.11475 213.88446 1 0 0 7448.087887

𝐿𝑌 = 𝐵

1 0 0 𝑦1 0.792
( 1.67213 1 0) (𝑦2 ) = ( 12.0 )
−7.11475 213.88446 1 𝑦3 16.3

𝑦1 = 0.792

1.67213𝑦1 + 𝑦2 = 12.0

=> 𝑦2 = 12.0 − (1.67213 × 0.792)

𝑦2 = 10.67567

−7.11475𝑦1 + 213.88446𝑦2 + 𝑦3 = 16.3

=> 𝑦3 = 16.3 + 7.11475(0.792) − 213.88446(10.67567)

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
18

𝑦3 = −2261.42503

0.792
𝑇ℎ𝑎𝑡 𝑖𝑠 𝑌 = ( 10.67567 )
−2261.42503

𝐴𝑙𝑠𝑜 𝑈𝑋 = 𝑌

0.610 1.23 17.2 𝑥1 0.792

( 0 0.09328 −34.27064 ) (𝑥2 ) = ( 10.67567 )
0 0 7448.087887 𝑥3 −2261.42503

Adopting back substitution, we have that

7448.087887𝑥3 = −2261.42503

2261.42503
𝑥3 = − = −0.30363
7448.087887

0.09328𝑥2 − 34.27064𝑥3 = 10.67567

10.67567 − (−34.27064 × 0.30363)

𝑥2 = = 2.89532
0.09328

0.610𝑥1 + 1.23𝑥2 + 17.2𝑥3 = 0.792

0.792 − 17.2(−0.30363) − 1.23(2.89532)

𝑥1 = 4.02163
0.610
4.02
𝑇ℎ𝑒𝑟𝑒𝑓𝑜𝑟𝑒 𝑋 = (2.90)
0.30

B. INDIRECT OR ITERATIVE METHOD

4.2.6 JACOBI (GAUSS-JACOBI) METHOD

Given the system of equations:

𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ 𝑎1𝑛 𝑥𝑛 = 𝑏1

𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 + ⋯ 𝑎24 𝑥𝑛 = 𝑏2

⋮ ⋮ ⋮ ⋮ ⋮

𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + 𝑎𝑚3 𝑥3 + ⋯ 𝑎𝑚𝑛 𝑥𝑛 = 𝑏𝑚

To solve this using the Jacobi method,

 Interchange any two rows,

 Write the equations in their normal form by making 𝑥1 the subject in equation 1, 𝑥2 the
subject in equation 2… as follows:
1
𝑥1 = {𝑏 − (𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ 𝑎1𝑛 𝑥𝑛 )}
𝑎11 1

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
19

1
𝑥2 = {𝑏 − (𝑎21 𝑥1 + 𝑎23 𝑥3 + ⋯ 𝑎24 𝑥𝑛 )}
𝑎22 2

⋮ ⋮ ⋮ ⋮ ⋮
1
𝑥𝑛 = {𝑏𝑚 − (𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + 𝑎𝑚3 𝑥3 + ⋯ 𝑎(𝑚−1)(𝑛−1) 𝑥𝑛 )}
𝑎𝑚𝑛

 Substitute 𝑥1 (0) = 0, 𝑥2 (0) = 0 … 𝑥𝑛 (0) = 0 which forms the first set of the solution
 The second set of solution (𝑥1 (1) , 𝑥2 (1) , … , 𝑥𝑛 (1) ) will be substituted into the normal form
 The iteration goes on till:
o The required number of iteration
o |𝑥𝑛+1 − 𝑥𝑛 | < 𝐸
o The solutions converge
EXAMPLE (Tutorial sheet C – Q18)

Solve by Gauss-Jacobi method, the system of equations

17𝑥1 + 6𝑥2 + 𝑥3 = 85

6𝑥1 + 18𝑥2 + 2𝑥3 = 72

𝑥1 + 𝑥2 + 22𝑥3 = 110

Solution

Writing the equations in their normal form

From (1),
1
𝑥1 = (85 − 6𝑥2 − 𝑥3 ) (18.1)
17
1
𝑥2 = (72 − 6𝑥1 + −2𝑥3 ) (18.2)
18
1
𝑥3 = (110−𝑥1 − 𝑥2 ) (18.3)
22

Putting (𝑥1 (0) , 𝑥2 (0) , 𝑥3 (0) )= (0, 0, 0) into (18.1) – (18.3), we have
1 86
𝑥1 (1) = (85 − 0 − 0) = =5
17 17

1 72
𝑥2 (1) = (72 − 0 − 0) = =4
18 18
1 110
𝑥3 (1) = (110 − 0 − 0) = = 5
22 22

Putting (𝑥1 (1) , 𝑥2 (1) , 𝑥3 (1) ) = (5,4,5) into (18.1) – (18.3), we have

1 56
𝑥1 (2) = [85 − 6(4) − 5] = = 3.29412
17 17

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
20

1 32
𝑥2 (2) = [72 − 6(5) − 2(5)] = = 1.77778
18 18
1 101
𝑥3 (2) = (110 − 5 − 4) = = 4.59091
22 22

Putting (𝑥1 (2) , 𝑥2 (2) , 𝑥3 (2) ) = (3.29412,1.77778,4.59091) into (18.1) – (18.3), we have

1
𝑥1 (3) = [85 − 6(1.77778) − 4.59091] = 4.10249
17
1
𝑥2 (3) = [72 − 6(3.29412) − 2( 4.59091)] = 2.39186
18
1
𝑥3 (3) = (110 − 3.29412 − 1.77778) = 4.76946
22

Putting (𝑥1 (3) , 𝑥2 (3) , 𝑥3 (3) ) = (4.10249, 2.39186, 4.76946) into (18.1) – (18.3), we have

1
𝑥1 (4) = [85 − 6(2.39186) − 4.76946] = 3.87526
17
1
𝑥2 (4) = [72 − 6(4.10249) − 2( 4.76946)] = 2.10256
18
1
𝑥3 (4) = (110 − 4.10249 − 2.39186) = 4.70480
22

Putting (𝑥1 (4) , 𝑥2 (4) , 𝑥3 (4) ) = (3.87526, 2.10256, 4.70480) into (18.1) – (18.3), we have

1
𝑥1 (5) = [85 − 6(2.10256) − 4.70480] = 3.98117
17
1
𝑥2 (5) = [72 − 6(3.87526) − 2( 4.70480)] = 2.18549
18
1
𝑥3 (5) = (110 − 3.87526 − 2.10256) = 4.72828
22
It continues to the 9th iteration

N 𝑥1 (𝑛) 𝑥2 (𝑛) 𝑥3 (𝑛)

0 0.00000 0.00000 0.00000

1 5.00000 4.00000 5.00000

2 3.29412 1.77778 4.59091

3 4.10249 2.39186 4.76946

4 3.87526 2.10256 4.70480

5 3.98117 2.18549 4.72828

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
21

6 3.95052 2.14758 4.71970

7 3.96440 2.15375 4.72281

8 3.96028 2.15378 4.72168

Since the of 𝑥 (7) 𝑎𝑛𝑑 𝑥 (8) are equal, then we stop at the 9th iteration and we pick

𝑥1 = 3.960, 𝑥2 = 2.154 𝑎𝑛𝑑 𝑥3 = 4.722 (To 3 decimal places)

4.2.6 GAUSS-SIEDEL METHOD

Given the system of equations:

𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ 𝑎1𝑛 𝑥𝑛 = 𝑏1

𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 + ⋯ 𝑎24 𝑥𝑛 = 𝑏2

⋮ ⋮ ⋮ ⋮ ⋮

𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + 𝑎𝑚3 𝑥3 + ⋯ 𝑎𝑚𝑛 𝑥𝑛 = 𝑏𝑚

To solve this using the Jacobi method,

 Interchange any two rows,

 Write the equations in their normal form by making 𝑥1 the subject in equation 1, 𝑥2 the
subject in equation 2… as follows:

1
𝑥1 = {𝑏 − (𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ 𝑎1𝑛 𝑥𝑛 )}
𝑎11 1
1
𝑥2 = {𝑏 − (𝑎21 𝑥1 + 𝑎23 𝑥3 + ⋯ 𝑎24 𝑥𝑛 )}
𝑎22 2

⋮ ⋮ ⋮ ⋮ ⋮
1
𝑥𝑛 = {𝑏 − (𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + 𝑎𝑚3 𝑥3 + ⋯ 𝑎(𝑚−1)(𝑛−1) 𝑥𝑛 )}
𝑎𝑚𝑛 𝑚

 Substitute 𝑥2 (0) = 0 … 𝑥𝑛 (0) = 0 into the first normal equation and obtain 𝑥1 (1)
 Substitute 𝑥1 (1) , 𝑥3 (0) = 0 … 𝑥𝑛 (0) = 0 into the first normal equation and obtain 𝑥2 (1) i.e.
use the updated value of each 𝑥 𝑖 .
 Continue till:
o The required number of iteration
o |𝑥𝑛+1 − 𝑥𝑛 | < 𝐸
o The solutions converge
NOTE: Gauss-Siedel method uses the updated value of each 𝑥 𝑖 .

EXAMPLE (Tutorial sheet C- question twenty)

Use gauss-Siedel scheme to solve the system of equations

5𝑥1 + 0.2𝑥2 − 0.15𝑥3 = 8

0.05𝑥1 + 4𝑥2 − 0.2𝑥3 = 6

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
22

0.03𝑥1 + 0.08𝑥2 + 6𝑥3 = 2

Solution

Writing the equations in their normal form

1
𝑥1 = (8 − 0.2𝑥2 + 0.15𝑥3 ) (20.1)
5
1
𝑥2 = (6 − 0.05𝑥1 + 0.2𝑥3 ) (20.2)
4
1
𝑥3 = (2 − 0.03𝑥1 − 0.08𝑥2 ) (20.3)
6
Putting (𝑥1 (0) , 𝑥2 (0) , 𝑥3 (0) )= (0, 0, 0) into (20.1) – (20.3), we have

1 8
𝑥1 (1) = (8 − 0 + 0) =
5 5
𝑥1 (1) = 1.6
1 5.92
𝑥2 (1) = (6 − 0.05(1.6) + 0) =
4 4
𝑥2 (1) = 1.48
1 1.8336
𝑥3 (1) = (2 − 0.03(1.6) − 0.08(1.48)) =
6 6
𝑥3 (1) = 0.3056

Putting (𝑥1 (1) , 𝑥2 (1) , 𝑥3 (1) ) = (1.6, 1.48,0.3056) into (20.1) – (20.3), we have

1
𝑥1 (2) = [8 − 0.2(1.48) + 0.15(0.3056)] = 1.54997
5
1
𝑥2 (2) = [6 − 0.05(1.54997) + 0.2(0.3056)] = 1.49591
4
1
𝑥3 (2) = [2 − 0.03(1.54997) − 0.08(1.49591)] = 0.30564
6

Putting (𝑥1 (2) , 𝑥2 (2) , 𝑥3 (2) ) = (1.54997,1.49591,0.30564) into (20.1) – (20.3), we have

1
𝑥1 (3) = [8 − 0.2(1.49591) + 0.15(0.30564)] = 1.54933
5
1
𝑥2 (3) = [6 − 0.05(1.54933) + 0.2(0.30564)] = 1.49592
4

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
23

1
𝑥3 (3) = [2 − 0.03(1.54933) − 0.08(1.49592)] = 0.30564
6

It continues to the desired or required result

n 𝑥1 (𝑛) 𝑥2 (𝑛) 𝑥3 (𝑛)

0 0.00000 0.00000 0.00000

1 1.60000 1.48000 0.3056

2 1.54997 1.49591 0.30564

3 1.54933 1.49592 0.30564

Since the value converges to four decimal places, then we conclude that

𝑥1 =, 𝑥2 = 𝑎𝑛𝑑 𝑥3 = (To 4 decimal places)

CURVE FITTING

Reading recorded values from a test or experiment normally include error of variouski9nd and
therefore, the point plotted from these data are scattered about the position they should ideally occupy.
Unless very few reading are taken, it can be assumed that the inherent error will be of a random nature
resulting in some of the value being slightly too low or too high.

Therefore, curve fitting deals with creation of functions from observed data. The least square
methods is a popular method for fitting curves.

METHOD OF LEAST SQUARE

The method of least square determine the best straight line entries by calculation, using the set
recorded results. The underlying idea of the least square approximation is to fit a polynomial function

𝑝(𝑥) To a set of data points (𝑥𝑖 , 𝑦𝑗 )

Having an exact solution

𝑦 = 𝑓(𝑥)

The set of data in the given observation determine the method to be used. The available solution are:

i. Discrete case
ii. Continuous case
DISCRETE CASE

Suppose the set of data satisfying the theoretical exact solution

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
24

𝑦 = 𝑓(𝑥) 𝐸1

Are

(𝑥1 , 𝑦1 ), (𝑥2 , 𝑦2 ), (𝑥3 , 𝑦3 ), … , (𝑥𝑛 , 𝑦𝑛 )

We are to fit a polynomial 𝑝(𝑥) to the observation 𝑦𝑖 , 𝑖 = 1,2,3, … , 𝑛 if the difference between
𝑓(𝑥𝑖 ) 𝑎𝑛𝑑 𝑝(𝑥𝑖 ), , 𝑖 = 1,2,3, … , 𝑛 is least. That is, the sum of the differences

𝑦𝑖 − 𝑝(𝑥), 𝑖 = 1,2,3, … , 𝑛

Should be minimum. Consequently, the required equation for the sum of square error to be minimized
is
𝑛
2
𝑆 = ∑(𝑦𝑖 − 𝑝(𝑥)) 𝐸2
𝑖=1

Where

𝑝(𝑥) = 𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 … + 𝑎𝑛 𝑥𝑖 𝑛 𝐸3

To minimize x, we obtain the stationary value of S with respect to each constant a𝑖 , 𝑖 = 1,2, … , 𝑛.
𝑛

𝑆 = ∑(𝑦𝑖 − (𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 … + 𝑎𝑛 𝑥𝑖 𝑛 ))2 𝐸4

𝑖=1

Hence differentiating S partially w.r.t. each constant 𝑎𝑖 and equating each to zero, we have:
𝑛
𝜕𝑆 2
= −2𝑛 ∑(𝑦𝑖 − (𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 + ⋯ + 𝑎𝑛 𝑥𝑖 𝑛 )) = 0 𝐸5
𝜕𝑎0
𝑖=1

𝑛
𝜕𝑆
= −2 ∑(𝑦𝑖 − (𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 + ⋯ + 𝑎𝑛 𝑥𝑖 𝑛 ))𝑥𝑖 𝐸6
𝜕𝑎1
𝑖=1

𝑛
𝜕𝑆
= −2 ∑(𝑦𝑖 − (𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 + ⋯ + 𝑎𝑛 𝑥𝑖 𝑛 ))𝑥𝑖 2 = 0 𝐸7
𝜕𝑎2
𝑖=1

⋮ ⋮ ⋮
𝑛
𝜕𝑆
= −2 ∑(𝑦𝑖 − (𝑛𝑎0 + 𝑎1 𝑥𝑖 + 𝑎2 𝑥𝑖 2 + ⋯ + 𝑎𝑛 𝑥𝑖 𝑛 ))𝑥𝑖 𝑛 = 0 𝐸8
𝜕𝑎𝑛
𝑖=1

If we expand, and re-arrange, we have:

∑ 𝑦𝑖 = 𝑛𝑎0 + 𝑎1 ∑ 𝑥𝑖 + 𝑎2 ∑ 𝑥𝑖 2 + … + 𝑎𝑛 ∑ 𝑥𝑖 𝑛 𝐸9

∑ 𝑥𝑖 𝑦𝑖 = 𝑎0 ∑ 𝑥𝑖 + 𝑎1 ∑ 𝑥𝑖 2 + 𝑎2 ∑ 𝑥1 3 + … + 𝑎𝑛 ∑ 𝑥𝑖 𝑛+1 𝐸10

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
25

∑ 𝑥𝑖 2 𝑦𝑖 = 𝑎0 ∑ 𝑥𝑖 2 + 𝑎1 ∑ 𝑥1 3 + 𝑎2 ∑ 𝑥1 4 + … + 𝑎𝑛 ∑ 𝑥𝑖 𝑛+2 𝐸1

⋮ ⋮ ⋮

∑ 𝑥𝑖 𝑛 𝑦𝑖 = 𝑎0 ∑ 𝑥𝑖 𝑛 + 𝑎1 ∑ 𝑥1 𝑛+1 + 𝑎2 ∑ 𝑥1 𝑛+2 + … + 𝑎𝑛 ∑ 𝑥𝑖 2𝑛 𝐸10

Solving equations E9-E12 to determine 𝑎0 , 𝑎1 , 𝑎2 , … , 𝑎𝑛 and substituting back to equation E3 gives

the best fitted curve to E1. Equations E9-E12 are called normal equations

we can put the system of n+1 equations into matrix form as follow:

𝑛 ∑ 𝑥𝑖 ∑ 𝑥𝑖 2 … ∑ 𝑥𝑖 𝑛 ∑ 𝑦𝑖
𝑎0
∑ 𝑥𝑖 ∑ 𝑥𝑖 2 ∑ 𝑥1 3 … ∑ 𝑥𝑖 𝑛+1 𝑎1 ∑ 𝑥𝑖 𝑦𝑖
𝑎2 =
∑ 𝑥𝑖 2 ∑ 𝑥1 3 ∑ 𝑥1 4 … ∑ 𝑥𝑖 𝑛+2 .. ∑ 𝑥𝑖 2 𝑦𝑖
… … … … … 𝑎
( 𝑛) …
𝑛 𝑛
(∑ 𝑥𝑖 ∑ 𝑥𝑖 𝑛+1 ∑ 𝑥𝑖 𝑛+2 … ∑ 𝑥𝑖 2𝑛 ) (∑ 𝑥𝑖 𝑦𝑖 )

The matrix equation can be solved for 𝑎0 , 𝑎1 , 𝑎2 , … , 𝑎𝑛 using any known methods.

Example (Tutorial manual sheet F –Q3)

The table below gives the readings from a laboratory experiment.

Time(t) 2 3 5 6 9
Reading(x) 7 17 49 71 161

Fit: A. linear function; and

B. quadratic polynomial to the above data by least square3 method and determine which of the
two is better approximation
Solution

𝑖 1 2 3 4 5 ∑
𝑡𝑖 2 3 5 6 9 25
𝑡𝑖 2 4 9 25 36 81 155
𝑡𝑖 3 8 27 125 216 729 1105
𝑡𝑖 4 16 81 625 1296 6561 8579
𝑥𝑖 7 17 49 71 161 305
𝑥𝑖 𝑡𝑖 14 51 245 426 1449 2185
𝑥𝑖 𝑡𝑖 2 28 171 1225 2556 13041 17021

A. For a linear function, 𝑝(𝑥) = 𝑎0 + 𝑎1 𝑥

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
26

𝑛 ∑ 𝑡𝑖 𝑎0 ∑ 𝑥𝑖
( ) (𝑎 ) = ( )
1
∑ 𝑡𝑖 ∑ 𝑡1 2 ∑ 𝑥𝑖 𝑡𝑖

5 25 𝑎0 305
( )( ) = ( )
25 155 𝑎1 2185

5𝑎0 + 25𝑎1 = 305 … .1

25𝑎0 + 155𝑎1 = 2185 … .2

From equation 1, 𝑎0 = 61 − 5𝑎1

25(61 − 5𝑎1 ) + 155𝑎1 = 2185

1525 − 125𝑎1 + 155𝑎1 = 2185

30𝑎1 = 2185 − 1525

30𝑎1 = 660

660
𝑎1 = = 22
30

𝑎0 = 61 − 5(22) = −49

Then the required linear equation will be

𝑝(𝑥) = −49 + 22𝑥

B. For the quadratic equation, the equation will be of the form

𝑝(𝑥) = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥 2

And the corresponding matrix will be

𝑛 ∑ 𝑡𝑖 ∑ 𝑡1 2 ∑ 𝒙𝒊
𝑎0
∑ 𝑡𝑖 ∑ 𝑡1 2 ∑ 𝑡1 3 (𝑎1 ) = ∑ 𝑥𝑖 𝑡𝑖
𝑎2
2
(∑ 𝑡 ∑ 𝑡1 3 ∑ 𝑡4 ) 2
(∑ 𝑥𝑖 𝑡 )

5 25 155 𝑎0 305
( 25 155 1105) (𝑎1 ) = ( 2185 )
155 1105 8579 𝑎2 17021
𝑎0 5 25 155 −1 305
(𝑎1 ) = ( 25 155 1105) ( 2185 )
𝑎2 155 1105 8579 17021
𝑎0 5.0333 −0.2 0.1667 305
(𝑎1 ) = ( −2 0.8736 −0.0796) ( 2185 )
𝑎2 0.1667 −0.076 6.944 17021

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
27

𝑎0 −1
(𝑎1 ) = ( 0 )
𝑎2 2
Then the required quadratic equation becomes

𝑝(𝑥) = −1 + 2𝑥 2

CONTINOUS CASE

To find a least square approximation defined over a closed interval [𝑎, 𝑏], then the summation sign in
equation 2must be replaced by integral sign so that the function to be minimized becomes:
𝑏
𝑆 = ∫ [𝑓(𝑥) − 𝑝(𝑥)]2 𝑑𝑥
𝑎

Where y=f(x) is a continuous function and p(x) is an approximating function.

EXAMPLE (TUTORIAL SHEET F- Q4)

Find the least square straight line that provides the best fit to the curve 𝑦 = √𝑥 over the interval
0≤𝑥≤1

SOLUTION

Let the line be 𝑦 = 𝑎 + 𝑏𝑥, then we must minimize

𝑏
𝑆 = ∫ [𝑓(𝑥) − 𝑝(𝑥)]2 𝑑𝑥
𝑎

1 𝑏
𝑆 = ∫ [√𝑥 − (𝑎 + 𝑏𝑥)]2 𝑑𝑥 = ∫ [√𝑥 − 𝑎 − 𝑏𝑥]2 𝑑𝑥
0 𝑎

1`
𝑆 = ∫ (𝑥 − 𝑎√𝑥 − 𝑏𝑥√𝑥 − 𝑎√𝑥 + 𝑎2 + 𝑎𝑏𝑥 − 𝑏𝑥√𝑥 + 𝑎𝑏𝑥 + 𝑏 2 𝑥 2 )𝑑𝑥
0

1
𝑆 = ∫ (𝑥 − 2𝑎√𝑥 − 2𝑏𝑥√𝑥 + 2𝑎𝑏𝑥 + 𝑏 2 𝑥 2 + 𝑎2 )𝑑𝑥
0

Differentiating with respect to the constant a and b we have,

1
1 4 3 4 5 1
𝑆 = [ 𝑥 2 − 𝑎𝑥 ⁄2 − 𝑏𝑥 ⁄2 + 𝑎𝑏𝑥 2 + 𝑎2 𝑥 + 𝑏 2 𝑥 3 ]
2 3 5 3 0

𝑑𝑆 4
= − + 𝑏 + 2𝑎 = 0
𝑑𝑎 3

6𝑎 + 3𝑏 = 4 …………………1

𝑑𝑆 4 2
=− +𝑎+ 𝑏 =0
𝑑𝑎 5 3

15𝑎 + 10𝑏 = 12 …………………2

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit
28

From equation 1 we have that:

4 − 3𝑏
𝑎= ……………………3
6
Putting (3) into (2) we have that

4 − 3𝑏
15 ( ) + 10𝑏 = 12
6

20 − 15𝑏 + 20𝑏 = 24

4
𝑏=
5
4 12 20 − 12
4 −3( ) 4 − 8 1
𝑎= 5 = 5 = 5 = ×
6 6 6 5 6
4
𝑎=
15

For more info, call: 08139279193 or 090331701334 or visit RCF FUTA, academic unit