Ambo University

Department of Mathematics

Lecture Note

Numerical Analysis

Tadesse Nigisa

May 3, 2017
Contents

1 Basic Concepts in Error Estimation 3

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Theory of Errors or Error Analysis . . . . . . . . . . . . . . . 3
1.3 Common types of errors: . . . . . . . . . . . . . . . . . . . . . 4
1.4 Basic sources of Errors . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Approximations of Errors . . . . . . . . . . . . . . . . . . . . 6
1.6 Propagation of Errors and Instability . . . . . . . . . . . . . . 8

2 Solutions of non-linear Equations 9

2.1 Locating roots . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Bisection (Bolzano) method . . . . . . . . . . . . . . . . . . . 14
2.3 Method of False Position . . . . . . . . . . . . . . . . . . . . . 16
2.4 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5 Newton-Raphson Method . . . . . . . . . . . . . . . . . . . . 22
2.6 General Iteration Method . . . . . . . . . . . . . . . . . . . . 26

3 System of equation 32
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Direct Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.1 Gauss Elimination Method . . . . . . . . . . . . . . . 35
3.2.2 Gauss-Jordan Method . . . . . . . . . . . . . . . . . . 41
3.2.3 Matrix Decomposition . . . . . . . . . . . . . . . . . . 43
3.3 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3.1 Gauss-Jacobi Iteration Method . . . . . . . . . . . . . 50
3.3.2 Gauss-Seidel Iteration Method . . . . . . . . . . . . . . 55
3.4 Solving System of non-linear Equations using Newtons Method 60

4 Curve Fitting 63
4.1 Least squares Method for Fitting . . . . . . . . . . . . . . . . 63
4.1.1 Linear least squares Regression . . . . . . . . . . . . . 63
4.1.2 power least square regression . . . . . . . . . . . . . . . 66

1
 4.1.3 Polynomial least square regression . . . . . . . . . . . . 68
4.2 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.1 Interpolation with equally spaced intervals . . . . . . . 75
4.2.2 Interpolation with unequal interval . . . . . . . . . . . 81

5 Numerical Differentiation and Integration 91

5.1 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . 91
5.1.1 Derivatives Using Newtons Forward Difference Formula 91
5.1.2 Derivatives using Newton’s Backward Interpolation For-
mula . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . 94
5.2.1 Trapezium Rule . . . . . . . . . . . . . . . . . . . . . . 95
5.2.2 Simpsons 1/3 Rule . . . . . . . . . . . . . . . . . . . . 100
5.2.3 Simpsons 3/8 Rule . . . . . . . . . . . . . . . . . . . . 102
5.2.4 Gauss-Quadrature Formula . . . . . . . . . . . . . . . 104
5.2.5 Romberg Integration . . . . . . . . . . . . . . . . . . . 108

6 First Order Initial Value Problems 113

6.1 Definition of ODE and examples . . . . . . . . . . . . . . . . . 115
6.2 Order of a differential equation, linear and nonlinear ODE . . 116
6.3 Nature of Solutions of ODE: particular and general solutions . 118
6.4 Initial value problem . . . . . . . . . . . . . . . . . . . . . . . 119
6.5 Existence of a unique solution (Picards theorem) . . . . . . . 121
6.6 Method of separable of variables . . . . . . . . . . . . . . . . 124
6.7 Homogeneous equations . . . . . . . . . . . . . . . . . . . . . 125
6.8 Exact equations, non-exact equations and integrating factor . 127
6.9 Orthogonal trajectories . . . . . . . . . . . . . . . . . . . . . . 131

7 Numerical Methods for Initial Value Problems 133

7.1 Taylors method of order n . . . . . . . . . . . . . . . . . . . . 134
7.2 Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.3 Eulers modified method . . . . . . . . . . . . . . . . . . . . . 142

2
Chapter 1

Basic Concepts in Error

Estimation

1.1 Introduction
What is Numerical Analysis?
The aim of the field numerical analysis is to provide convenient methods
for obtaining useful solutions to mathematical problems. Such mathemat-
ical problems can be formulated, for example, in terms of an algebraic or
transcendental equation, an ordinary or partial differential equation, or an
integral equation, or in terms of a set of such equations.

Example 1.1.1 sinx + x2 ex + lnx = 0

The solution of this in general cannot be obtained so easily by known exact

methods. The numerical analysis will provide solutions which are not exact
but approximate. However, the approximate solutions should be close enough
to exact solution.

1.2 Theory of Errors or Error Analysis

The study of errors is a central concern of numerical analysis. Most numeri-
cal methods give solutions that are only approximate to the desired solutions
and it is important to understand and able to estimate or found the resulting
errors.

Error = T rueV alue − ApproximateV alue

3
1.3 Common types of errors:
Inherent errors:

• Errors which are already present in the statement of a problem before

its use.

• Arise either due to the given data being approximate or due to the lim-
itations of mathematical tables, calculators, or the digital computers.

• Can be minimized by taking better data or by using high precision

computing aids.

Rounding errors:

• Arise from the process of rounding off the numbers during the compu-
tation.

• Are unavoidable in most of the calculations due to the limitations of

the computing aids. However, rounding errors can be reduced.

Truncation errors:

• Are caused by using approximate results or on replacing an infinite

process by a finite one.

Example 1.3.1 If we use a decimal computer having a fixed word length of

four digitals, then

• Rounding off of 13.658 gives 13.66; and

• Truncation of 13.658 gives 13.65.

Hence,
Rounding error=13.66-13.658=0.002;and
Truncation error=13.658-13.65=0.008.

Absolute, Relative and Percentage Errors

If X is the true value of a quantity and X is its approximate value, then

• Absolute error, Ea = |X − X|;

4
 Ea |X−X|
• Relative error, Er = |X|
= |X|
; and

• Percentage error, Ep = Er × 100.

If X be such a number that |X − X| ≤ X, then Xis an upper limit on the

magnitude of absolute error and measures the absolute accuracy.
Remarks:

1. The relative and percentage errors are independent of the units used.

2. If a number is correct to n decimal places, then the error(absolute) =

1
2
× 10−n .

1.4 Basic sources of Errors

• Mathematical statements rarely give exact picture of actual phenom-
ena. Most of the cases are idealized models (Mathematical modeling).
This is the source of error (errors of the problem).

• Consider the function

x3 x5
sinx = x − + ···
3! 5!
In this function, one has to consider a few terms to make computational
practice. Hence, we are forced to stop at some term of the sequence and
consider it to be an approximation. Such a termination of the process
gives rise to an error. This error is called residual error or truncation
error.

• Consider π = 22/7 = 3.141516 · · ·

We may have a non-terminating and a non-repeating decimal. It is ob-
vious that we can use only a finite number of digits in our computation.
This is the source of error called rounding error.

• When we are performing computational with approximate numbers we

naturally carry (to some extent) the errors of the original data into
final result. This is the source of error called inherent error.

5
1.5 Approximations of Errors
Errors in the Approximation of a Function Let y = f (x1 , x2 ) be a
function of two variables x1 and x2 . If δx1 and δx2 be the errors in x1 and
x2 , respectively, then the error δy in y is given by

y + δy = f (x1 + δx1 , x2 + δx2 ) (1.1)

Expanding the right hand side by Taylors series, we get:

∂f ∂f
y+δy = f (x1 , x2 )+ δx1 + δx2 + terms involving higher powers of δx1 and δx2 · · · (i).
∂x1 ∂x2
Neglecting the higher powers of δx1 and δx2
We obtain
∂f ∂f
δy ≈ δx1 + δx2 (1.2)
∂x1 ∂x2
In general, the error δy in the function y = f (x1 , x2 , · · · , xn ) corresponding
to the error δxi in xi (i = 1, 2, · · · , n) is given by:
∂y ∂y ∂y
δy ≈ δx1 + δx2 + · · · + δxn (1.3)
∂x1 ∂x2 ∂xn
and the relative error in y is given by:
δy ∂y δx1 ∂y δx2 ∂y δxn
Er = ≈ + + ··· + (1.4)
y ∂x1 y ∂x2 y ∂xn y
2 3
Example 1.5.1 : Let u = 4xz4y and errors in x, y, z be 0.001. Then compute
the maximum relative error in u when x = y = z = 1.
8xy 3 ∂u 12x2 y 2 ∂u 2 3
Solution: Since ∂u
∂x
= z4
, ∂y = z4
, ∂z = − 16xz5 y , we have

∂u ∂u ∂u 8xy 3 12x2 y 2 16x2 y 3

δu ≈ δx + δy + δz = 4 δx + δy + − δz.
∂x ∂y ∂z z z4 z5
As errors δx, δy and δz may be positive or negative, we take the absolute
values of the terms on the right hand side, giving
8xy 3 12x2 y 2 16x2 y 3
δumax ≈ | δx| + | )δy| + | − δz|
z4 z4 z5
= 8(0.001) + 12(0.001) + 16(0.001) = 0.036
Hence, the maximum relative error in u is

Ermax = δu/u ≈ 0.036/4 = 0.009.

6
Example 1.5.2 Find the maximum relative error in the function y = axm 1 m2 mn
1 x2 · · · xn ,
where a, m1 , m2 , · · · , mn are constants.
Solution: ln(y) = ln(a) + m1 ln(x1 ) + m2 ln(x2 ) + · · · + mn ln(xn )
∂y 1 1 ∂y ∂y
Therefore, y1 ∂x 1
=m ,
x1 y ∂x2
=m
x2
2
· · · , y1 ∂xn
=mn
xn
.
∂y δx1 ∂y δx2 ∂y δxn
Hence, Er ≈ ∂x1 y
+ ∂x2 y
+ ··· + ∂xn y
.
δx1 δx2 δxn
= m1 + m2 + · · · + mn .
x1 x2 xn
Since the errors δx1 , δx2 , · · · , δxn may be positive or negative, we take the
absolute values of the terms on the right side. This gives:
δx1 δx2 δxn
(Er )max = |m1 | + |m2 | + · · · + |mn |.
x1 x2 xn
Remark: Taking a = 1, m1 = m2 = · · · = mn = 1, we have y = x1 x2 · · · xn .
Then
δx1 δx2 δxn
Er = + + ··· + .
x1 x2 xn
Thus the relative error of a product of n numbers is approximately equal to
the algebraic sum of their relative errors.

Errors in a Series Approximation

The Taylors series for f(x) at x=a with a remainder after n terms is
(x − a)2 00 (x − a)n−1 (n−1)
f (x) = f (a+x−a) = f (a)+(x−a)f 0 (a)+ f (a)+· · ·+ f (a)+Rn (x)
2! (n − 1)!
(1.5)
n
where Rn (x) = (x−a)
n!
f n (θ), a < θ < x.
If the series is convergent, then Rn (x) → 0 as n → ∞ and hence if f (x) is
approximated by the first n terms of this series, then the maximum error will
be given by the remainder term Rn (x). Moreover, if the accuracy required
in a series approximation is pre-assigned, then we can find n, the number of
terms which would yield the desired accuracy.
Example 1.5.3 : Find the number of terms of the exponential series such
that their sum gives the value of ex correct to six decimal places at x = 1.
Solution: The Taylors series for ex at x = 0 is

x2 x 3 xn−1
ex = 1 + x + + + ··· + + Rn (x) · · · (i)
2! 3! (n − 1)!

7
 n
where Rn (x) = xn eθ , 0 < θ < x. Therefore, the maximum absolute error (at
n n
θ = x) = xn! ex ; and the maximum relative error = xn!
Hence, (Er )max at x=1 = n!1 .
For a six decimal place accuracy at x=1, we have
1 1
(Er )max = < × 10−6
n! 2
⇒ n! > 2 × 106
Thus the minimum integer value of n which satisfy this condition is n=10.
Thus we need 10 terms of the series (i) in order that its sum is correct to 6
decimal places.

1.6 Propagation of Errors and Instability

Algorithm:
An algorithm is a procedure that describes a finite sequence of steps to be
performed in specified order to solve a given problem.

Propagated error:

Propagated error is an error in the succeeding step of an algorithm due

to an error in the initial step.

• If the error E0 is introduced at initial stage of the algorithm and after

’n’ subsequent operation of the algorithm, En is the error resulted, then
this is the propagated error.

Stability:

• If errors are magnified continuously as the algorithm continues, even-

tually they will over shadow the true value and hence destroying the
validity of the algorithm. If this is so, the algorithm is instable.

• If errors made at initial step die out (decrease) as the algorithm con-
tinuous, the algorithm is said to be stable.

8
Chapter 2

Solutions of non-linear
Equations

2.1 Locating roots

Introduction:
A problem of great importance in science and engineering is that of deter-
mining the roots/zeros of an equation of the form

f (x) = 0

A polynomial equation of the form

f (x) = pn (x) = an xn + an−1 xn−1 + · · · + a1 x + a0 = 0 (2.1)

is called an algebraic equation. An equation which contains exponential func-

tions logarithmic functions, trigonometric functions etc are called a transcen-
dental equation. For example
3x3 − 2x2 − x − 5 = 0 ,x4 − 3x2 + 1 = 0, x2 − 3x + 1 = 0
Are algebraic (polynomial) equations, and xex − 1 = 0 , cosx − xe2x = 0 ,
tanx = x are transcendental equations.

Definition 2.1 1. A number α ,for which f (α) = 0 is called a root of the

equation f (x) = 0 is called a root of the equation f (x) = 0, or a zero of
f (x) = 0. Geometrically , a root of an equation f (x) = 0 is the value
of x at which the graph of the equation y = f (x) intersects the x-axis.

2. A number α is a simple root of f (x) = 0 if f (α) = 0 and f 0 (α) 6= 0 ,

Then we can write f (x) as f (x) = (x − α)g(x), g(α) 6= 0
For example , since x − 1 is a factor of f (x) = x3 + x − 2 = 0 we can

9
 Figure 2.1: geometrical representation

write f (x) = (x − 1)(x2 + x + 2) = (x − 1)g(x), g(1) = 4 6= 0 Alterna-

tively, we find f (1) = 0,and f 0 (1) = 4 6= 0 .Hence ,x = 1 is a simple
root of f (x) = x3 + x − 2 = 0 .

3. A number α is a multiple root, of multiplicity m , of f (x) = 0 if

f (α) = 0, f 0 (α) = 0 ,f 00 (α) = 0 · · · f m−1 (α) = 0 and f m (α) 6= 0.
Then we can write f (x) as f (x) = (x − α)m g(x), g(α) 6= 0.

For example: consider the equation f (x) = x3 − 3x2 + 4 = 0 .

we find f (2) = 8 − 12 + 4 = 0 f 0 (x) = 3x2 − 6x, f 0 (2) = 12 − 12 = 0

f 00 (x) = 6x − 6, f 00 (2) = 6 6= 0.

Hence x = 2 is a multiple root of multiplicity 2(double root) of

f (x) = x3 − 3x2 + 4 = 0
We can write f (x) = (x − 2)2 (x + 1) = (x − 2)2 g(x), g(2) = 3 6= 0

In this chapter , we shall be considering the case of simple roots only.

Remark 2.1 A polynomial equation of degree n has exactly n roots , real or

complex ,simple or multiple ,where as a transcendental equation may have
one root, infinite number of roots or no root.

We shall derive methods for finding only the real roots. The methods for
finding the roots are classified as (i) direct methods ,and (ii) iterative methods
Direct Methods: These methods give the exact value of all the roots in

10
a finite number of steps( disregarding the round off errors).Therefore , for
any direct method , we can give the total number of operations ( addition,
subtraction divisions and multiplications ) This number is the operational
count of the method.
For example ,the roots of the quadratic equation ax2 + bx + c = 0, a = 0 ,
can be obtained using the method
1 √
x = [−b ± b2 − 4ac]
2a
For this method we can give the count of total number of operations. There
are direct methods for finding the roots of a cubic and fourth degree polyno-
mial. However these methods are difficult to use. Direct methods for finding
the roots of polynomial equations of degree greater than 4 or transcendental
equations are not available in literature
Iterative Methods: These methods are based on the idea of successive
approximations. We start with one or two initial approximations to the root
and obtain a sequence of approximations x0 , x1 , x2 , · · · , xk , · · · which in the
limit as k → ∞ , converge to the exact root α. An iterative method for
finding a root of the equation f (x) = 0 can be obtained as
xk+1 = ϕ(xk ), k = 0, 1, 2, . . .
This method uses one initial approximation to the root x0 . The sequence of
approximations is given by
x1 = ϕ(x0 ), x2 = ϕ(x1 ), x3 = ϕ(x2 ), · · · .
The function ϕ is called the iteration function and x0 is called the initial
approximation.
If the method uses two initial approximations x0 , x1 to the root , then we can
write the method as xk+1 = ϕ(xk−1 , xk ) k = 1, 2, · · · Convergence of iterative
methods : The sequence of iterates ,xk , is said to converge to the exact root
α , if
limk→∞ xk = α
Remark 2.2 Given one or two initial approximations to the root, we re-
quire a suitable iteration function α for a given function f (x) , such that the
sequence of iterates,{xk } , converge to the exact root α.
Further , we also require a suitable criterion to terminate the iteration.

Criterion to terminate iteration procedure;

Since we cannot perform infinite number of iterations, we need a criterion to
stop the iterations. We use one or both of the following criterion:

11
 1. The equation f (x) = 0 is satisfied to a given accuracy or f (xk ) is
bounded by an error tolerance . |f (xk )| ≤ .

2. The magnitude of the difference between two successive iterates is

smaller than a given accuracy or an error bound .

|xk+1 − xk | ≤ .

Generally, we use the second criterion. In some very special problems, we

require to use both the criteria.
For example , if we require two decimal place accuracy , then we iterate until
|xk+1 − xk | ≤ 0.005.
If we require three decimal place accuracy, then we iterate until
|xk+1 − xk | ≤ 0.0005.
As we have seen earlier , we require a suitable iteration function and
suitable initial approximation(s) to start the iteration procedure. Initial ap-
proximation for an iterative procedure
Descartes Rule of iterative procedure : This gives the bound for the
number of positive and negative real root for polynomial equations f (x) = 0;

i) We count the number of changes of signs is the coefficients of pn (x) for

the equation f (x) = pn (x) = 0 . The number of positive roots can not
exceed the number of changes of signs.
For example , If there are 4 changes of signs, then the equation may
have 4 positive roots or two(2) positive roots or no positive roots.
If there are 3 changes of signs , then the equation may have 3 positive
roots or definitely one positive root.

ii) We write the equation f (−x) = pn (−x) = 0 and count the number of
changes of signs in the coefficients of pn (−x). The number of negative
roots can not exceed the number of changes of signs. For example ,
if there are 4 changes of signs, then the equation may have 4 negative
roots or 2. If there are 3 changes of signs , then the equation may have
3 negative roots or definitely one negative root.

We use the following theorem of calculus to determine an initial approxima-

tion. It is also called the intermediate value theorem.

12
 Theorem 1

If f (x) is continuous on some interval [a, b] and f (a)f (b) < 0, then the
equation f (x) = 0 has at least one real root or an odd number of real
roots in the interval (a, b).

Example 2.1.1 Determine the maximum number of positive and negative

roots and intervals of length one unit in which the real roots lie for the fol-
lowing equations.

i. 8x3 − 12x2 − 2x + 3 = 0

ii. 3x3 − 2x2 − x − 5 = 0

Solutions: Let f (x) = 8x3 − 12x2 − 2x + 3 = 0 the number of changes in

signs of the coefficients (8, −12, −2, 3) is 2 . Therefore , the equation has 2
positive roots or no positive roots.

i. Now f (−x) = −8x3 − 12x2 + 2x + 3 = 0 The number of changes in

signs of the coefficient (−8, −12, 2, 3) is one . Therefore the equation
has one negative root. We have the following table of values for f (x)

x -2 -1 0 1 2 3
f (x) -105 -15 3 -3 15 105

Since f (−1)f (0) < 0 , there is a root in the interval (−1, 0)

f (0)f (1) < 0 there is a root in the interval (0, 1)
f (1)f (2) < 0 there is a root in the interval (1, 2)
Therefore there are 3 real roots (2 positive and 1 negative ) and the
root lies in the
intervals (−1, 0) , (0, 1) and (1, 2).

ii. Let f (x) = 3x3 − 2x2 − x − 5 = 0 ,The number of changes in signs

of the coefficients (3, −2, −1, −5) is 1 .Therefore the equation has one
positive root.Now f (−x) = −3x3 − 2x2 + x − 5 = 0 the number of
change in signs of the coefficients (−3, −2, 1, −5) is two. Therefore the
equation has 2 negative roots or no negative root.
The table of values for f(x).

x -3 -2 0 1 2 3
f (x) -101 -35 -5 -5 9 55

13
From the table there is 1 positive real root in the interval (1, 2). The equation
has no negative real root .
Example 2.1.2 Locate the smallest positive real root of the equations
i. xex = cosx
ii. tanx = 2x
Solution

i. Let f (x) = xex − cosx = 0 since f (0) = −1 and

f (1) = e − cos(1) = 2.178,
there is a root in the interval (0, 1).
ii. Let f (x) = tanx − 2x = 0 Then we have the following function values
f (0) = 0 , f (0.1) = −0.0997 , f (0.5) = −0.4537 f (1) = −0.4426
f (1.1) = −0.2352 , f (1.2) = 0.1722, since f (1.1)f (1.2) < 0 ,the root
lies in the interval (1.1, 1.2)

2.2 Bisection (Bolzano) method

This method is based on the repeated application of the intermediate value
theorem. Let the function f (x) be continuous on [a, b]. For definiteness let
f (a) < 0 and f (b) > 0 then the first approximation to the root.
a+b
x1 =
2
If f (x1 ) = 0 , then x1 is a root of f (x) = 0 .Otherwise, the root lies in
(a, x1 ) or (x1 , b) according as f (x1 ) is positive or negative. Then we bisect
the interval as before and continue the process until the root is found to
desired accuracy. In the figure (2.4), f (x1 ) > 0, so that the root lies in the
interval (a, x1 ) . Then the second approximation to the root is
a + x1
x2 =
2
If f (x1 ) < 0) , then the root lies in (x2 , x1 ) .Then the third approximation
to the root is
x1 + x2
x3 =
2
and so on

14
 Figure 2.2: Bisection method

Remark 2.3 Since the new interval containing the root is exactly half the
length of the previous one , the interval width is reduced by a factor of 21 at
each step. Therefore at the end of the nth step , the interval will be of length
b−a
(2.2)
2n
If on repeating this process n times , the latest interval is as small as given
ε ,then
b−a ln( b−a )
2n ≤ ε or n ≥ ln2
ε

This gives the number of iterations required for achieving an accuracy ε . In

particular , the minimum number of iterations required for converging to a
root in the interval (0, 1) for a given ε are as under;

ε 10−2 10−3 10−4

n 7 10 14

Example 2.2.1 Find a root of the equation x3 −4x−9 = 0 using the bisection
method correct to 3 decimal places.

Solution: Let f (x) = x3 − 4x − 9 = 0. Since f (2)f (3) < 0, the root

lies in the interval (2, 3). Therefore, the first approximation to the root is
x1 = 2+32
= 2.5 and f (x1 ) = f (2.5) = −3.375.
Since f (2.5)f (3) < 0, the root lies in the interval (2.5, 3). Therefore, the
2nd approximation to the root is x2 = 2.5+3 2
= 2.75 and f (x2 ) = f (2.75) =
0.7969. Since f (2.5)f (2.75) < 0, the root lies in the interval (2.5, 2.75).
Therefore, the 3rd approximation to the root is x3 = 2.5+2.75 2
= 2.625 and
f (x3 ) = f (2.625) = −1.4121.
Since f (2.625)f (2.75) < 0, the root lies in the interval (2.625, 2.75). There-
fore, the 4th approximation to the root is x4 = 2.625+2.75
2
= 2.6875.
Repeating this process, the successive approximations are

15
x5 = 2.71875, x6 = 2.70313
x7 = 2.71094, x8 = 2.70703
x9 = 2.70508, x10 = 2.70605
x11 = 2.70654
Since |x11 − x10 | = 0.00012 < 0.0005, the required root is x ≈ x11 = 2.70654.

2.3 Method of False Position

The method is also called linear interpolation method or regula-falsi method.
At the start of all iterations of the method, we require the interval in which
the root lies. Let the root of the equation f (x) = 0 lie in the interval
(xk−1 , xk ), that is, fk−1 fk < 0, where f (xk−1 ) = fk−1 , and f (xk ) = fk . Then
P (xk−1 , fk−1 ), Q(xk , fk ) are points on the curve f (x) = 0. Draw a straight
line joining the points P and Q From figure (2.3). The line P Q is taken

Figure 2.3: Metod of false postion

as an approximation of the curve in the interval [xk−1 , xk ]. The equation of

−fk−1
y−fk
the line P Q is given by x−x k
= xfkk −x k−1
. The point of intersection of this line
P Q with the x-axis is taken as the next approximation to the root. Setting
y = 0, and solving for x, we get
xk − xk−1
x = xk − ( )fk .
fk − fk−1
The next approximation to the root is
xk − xk−1
xk+1 = xk − ( )fk .
fk − fk−1
Simplifying, we can also write the approximation as
xk (fk − fk−1 ) − (xk − xk−1 )fk (xk−1 fk − xk fk−1 )
xk+1 = = , k = 1, 2, . . .
fk − fk−1 fk − fk−1
Therefore, starting with the initial interval (x0 , x1 ) in which the root lies, we
compute
x 0 f 1 − x1 f 0
x2 = .
f1 − f0

16
Now, if f (x0 )f (x2 ) < 0 , then the root lies in the interval (x0 , x2 ). Otherwise,
the root lies in the interval (x2 , x1 ). The iteration is continued using the
interval in which the root lies, until the required accuracy criterion is satisfied.
Remark: At the start of each iteration, the required root lies in an interval,
whose length is decreasing. Hence, the method always converges.
Remark: The method of false position has a disadvantage. If the root
lies initially in the interval (x0 , x1 ), then one of the end points is fixed for
all iterations. For example, in Fig.1.2a, the left end point x0 is fixed and
the right end point moves towards the required root. Therefore, in actual
computations, the method behaves like
x0 f k − xk f 0
xk+1 = , k = 1, 2, . . .
fk − f0
In Fig.1.2b, the right end point x1 is fixed and the left end point moves
towards the required root. Therefore, in this case, in actual computations,
the method behaves like
x1 f k − xk f 1
xk+1 = , k = 2, 3, . . .
fk − f1
Example 2.3.1 Locate the intervals which contain the positive real roots of
the equation x3 − 3x + 1 = 0.
Obtain these roots correct to three decimal places, using the method of false
position.

Solution: We form the following table of values for the function f(x).

x 0 1 2 3
f (x) 1 -1 3 19

There is one positive real root in the interval (0, 1) and another in the interval
(1, 2). There is no real root for x > 2 as f (x) > 0 for all x > 2. First, we
find the root in (0, 1). We have

x0 = 0, x1 = 1, f0 = f (x0 ) = f (0) = 1, f1 = f (x1 ) = −1

x0 f 1 − x1 f 0 0(−1) − 1
x2 = = = 0.5, f2 = f (x2 ) = f (0.5) = −0.375 < 0.
f1 − f0 −1 − 1
Since f (0)f (0.5) < 0 ,the root lies in the interval (0, 0.5)

x 0 f 2 − x2 f 0 0 − 0.5
x3 = = = 0.36364.
f2 − f0 −0.375 − 1

17
and
f3 = f (x3 ) = f (0.36364) = −0.04283 < 0
Since f (0)f (0.36364) < 0, the root lies in (0, 0.36364)
x0 f 3 − x3 f 0 0 − 0.36364
x4 = = = 0.34870
f3 − f0 −0.04283 − 1
and f4 = f (x4 ) = f (0.34870) = −0.00370 < 0
Since f (0)f (0.34870) < 0 the root lies in (0, 0.34870).
x0 f 4 − x4 f 0 0 − 0.34870(1)
x5 = = = 0.34741
f4 − f0 −0.00370 − 0
and f5 = f (x5 ) = f (0.34741) = −0.00030 < 0
since f (0)f (0.34741) < 0 , the root lies in (0, 0.34741)
x0 f 5 − x5 f 0 0 − 0.34741
x6 = = = 0.347306
f5 − f0 −0.00030 − 1
Now |x6 − x5 | = |0.347306 − 0.34741| = 0.000104 < 0.0005
Since the root has been computed correct to 3 decimal places the required
roots can be taken as x ≈ x6 = 0.347306 , We may give the result as 0.347 ,
even though x6 is more accurate .
Note: The left end point x = 0 is fixed for all iterations. Now we compute
the root in (1, 2).
we have x0 = 1, x1 = 2 , f0 = f (x0 ) = f (1) = −1 and f1 = f (x1 ) = f (2) = 3
Therefore
x0 f 1 − x1 f 0 3 − 2(−1)
x2 = = = 1.25
f1 − f0 3 − (−1)
f2 = f (x2 ) = f (1.25) = −0.796875
Since f (1.25)f (2) < 0, the root lies in (1.25, 2)
x2 f 1 − x1 f 2 1.25(3) − 2(−0.796875)
x3 = = = 1.407407
f1 − f2 3 − (−0.796875)
f3 = f (x3 ) = f (1.407407) = −0.434437 Since f (1.407407)f (2) < 0, the root
lies in (1.407407, 2)
x3 f 1 − x1 f 3 1.407407(3) − 2(−0.434437)
x4 = = = 1.482367
f1 − f3 3 − (−0.434437)
f4 = f (x4 ) = f (1.482367) = −0.189730
Since f (1.482367)f (2) < 0 , the root lies in the interval (1.482367, 2)
x4 f 1 − x1 f 4 1.482367(3) − 2(−0.189730)
x5 = = = 1.513156
f1 − f4 3 − (−0.189730)

18
and
f5 = f (x5 ) = f (1.513156) = −0.074884
Since f (1.513156)f (2) < 0 , the root lies in the interval (1.513156, 2)

x5 f 1 − x1 f 5 1.513156(3) − 2(−0.074884)
x6 = = = 1.525012
f1 − f5 3 − (−0.074884)
and
f6 = f (x6 ) = f (1.525012) = −0.028374
Since f (1.525012)f (2) < 0 ,the root lies in the interval (1.525012, 5)

x6 f 1 − x1 f 6 1.525012(3) − 2(−0.028374)
x7 = = = 1.529462
f1 − f6 3 − (−0.028374)

f (x7 ) = f (1.529462) = −0.010586

Since, f (1.529462)f (2) < 0, the root lies in the interval (1.529462, 2).

x7 f 1 − x1 f 7 1.529462(3) − 2(−0.010586)
x8 = = = 1.531116
f1 − f7 3 − (−0.010586)

f (x8 ) = f (1.531116) = −0.003928

Since, f (1.531116)f (2) < 0, the root lies in the interval (1.531116, 2).

x8 f 1 − x1 f 8 1.531116(3) − 2(−0.003928)
x9 = = = 1.531729
f1 − f8 3 − (−0.003928)
f (x9 ) = f (1.531729) = −0.001454.
Since, f (1.531729)f (2) < 0, the root lies in the interval (1.531729, 2).

x9 f 1 − x 1 f 9 1.531729(3) − 2(−0.001454)
x10 = = = 1.531956
f1 − f9 3 − (−0.001454)

Now, |x10 − x9 | = |1.531956 − 1.53179| ≈ 0.000227 < 0.0005.

The root has been computed correct to three decimal places. The required
root can be taken as xx10 = 1.531956. Note that the right end point x = 2
is fixed for all iterations.

Example 2.3.2 Find the root correct to two decimal places of the equation
xex = cosx, using the method of false position.

19
Solution Define f (x) = cosx − xex = 0. There is no negative root for the
equation.
We have f (0) = 1, f (1) = cos1 − e = −2.17798.
A root of the equation lies in the interval (0, 1). Let x0 = 0, x1 = 1. Using
the method of false position, we obtain the following results.
x0 f 1 − x1 f 0 0 − 1(1)
x2 = = = 0.31467
f1 − f0 −2.17798 − 1
and
f (x2 ) = f (0.31467) = 0.51986
Since, f (0.31467)f (1) < 0, the root lies in the interval (0.31467, 1).
x2 f 1 − x 1 f 2 0.31467(−2.17798) − 1(0.51986)
x3 = = = 0.44673
f1 − f2 −2.17798 − 0.51986
f (x3 ) = f (0.44673) = 0.20354.
Since, f (0.44673)f (1) < 0, the root lies in the interval (0.44673, 1).
x3 f 1 − x1 f 3 0.44673(2.17798) − 1(0.20354)
x4 = = = 0.49402
f1 − f3 −2.17798 − 0.20354
f (x4 ) = f (0.49402) = 0.07079.
Since, f (0.49402)f (1) < 0, the root lies in the interval (0.49402, 1).
x4 f 1 − x 1 f 4 0.49402(−2.17798) − 1(0.07079)
x5 = = = 0.50995
f1 − f4 −2.17798 − 0.07079
f (x5 ) = f (0.50995) = 0.02360.
Since, f (0.50995)f (1) < 0, the root lies in the interval (0.50995, 1).
x5 f 1 − x1 f 5 0.50995(−2.17798) − 1(0.0236)
x6 = = = 0.51520
f1 − f5 −2.17798 − 0.0236
f (x6 ) = f (0.51520) = 0.00776.
Since, f (0.51520)f (1) < 0, the root lies in the interval (0.51520, 1).
x6 f 1 − x1 f 6 0.5152(−2.17798) − 1(0.00776)
x7 = = = 0.51692.
f1 − f6 −2.17798 − 0.00776
Now, |x7 − x6 | = |0.51692 − 0.51520| ≈ 0.00172 < 0.005.
The root has been computed correct to two decimal places. The required
root can be taken as x ≈ x7 = 0.51692.
Note that the right end point x = 2 is fixed for all iterations
Exercises 2.1 Find a root of the equation x3 − 4x − 9 = 0 using regula-falsi
method correct to 3 decimal places.

20
2.4 Secant Method
This method does not require the condition f (x0 )f (x1 ) < 0 of the method.see
fig(4.3) Here also the graph of the function y = f (x) is approximated by a
secant line but at each iteration ,two most recent approximations to the root
are used to find the next approximation.Also it is not necessary that the
interval must contain the root. Taking x0 and x1 as the intial limits of the

Figure 2.4: Secant method

interval the equation of the secant line through (x0 , f (x0 )) and (x1 , f (x1 )) is

f (x1 ) − f (x0 )
y − f (x1 ) = (x − x1 )
x1 − x0
Then the abscissa of the point where it crosses the x-axis (y=0) is given by
x1 − x0
x = x1 − ( )f (x1 )
f (x1 ) − f (x0 )

hence the next approximation is given by

x1 − x0
x2 = x1 − ( )f (x1 ).
f (x1 ) − f (x0 )

Therefore ,the general formula for successive approximations is given by

xk − xk−1
xk+1 = xk − ( )f (xk ) , k = 1, 2, 3, . . .
f (xk ) − f (xk−1 )

Remark 2.4 If f (xn ) = f (xn−1 ) ,then this method fails and shows that it
does not converge necessarily. This is a drowback of secant method . But if
the secant method once converges,its rate of convergence is faster than that
of the method of false position.

21
Example 2.4.1 Find the root of the equation cosx = xex using the secant
method correct to 4 decimal places .

Solutions: Let f (x) = cosx − xex = 0 Taking the initial approximations

x0 = 0 , x1 = 1 so that f (x0 ) = 1, f (x1 ) = cos1 − e = −2.17798 Then by
secant method we have
x1 − x0
x2 = x1 − ( )f (x1 ) = 0.31467
f (x1 ) − f (x0 )

f (x2 ) = f (0.31467) = 0.51987

x 2 − x1
x3 = x2 − ( )f (x2 ) = 0.44673 , f (x3 ) = f (0.44673) = 0.20354
f (x2 ) − f (x1 )
x3 − x2
x4 = x3 − ( )f (x3 ) = 0.53171
f (x3 ) − f (x2 )
Repeating the process the successive approximations are

x5 = 0.51690 , x6 = 0.51775 , x7 = 0.51776, etc

Hence the root is x ≈ 0.51776 correct to 4 decimal places

2.5 Newton-Raphson Method

This method is also called Newton’s method. This method is also a chord
method in which we approximate the curve near a root, by a straight line.
Let x0 be an initial approximation to the root of f (x) = 0. Then, P (x0 , f0 ),
where f0 = f (x0 ), is a point on the curve.
Draw the tangent to the curve at P, (Fig.2.4). We approximate the curve
in the neighborhood of the root by the tangent to the curve at the point
P. The point of intersection of the tangent with the x-axis is taken as the
next approximation to the root. The process is repeated until the required
accuracy is obtained. The equation of the tangent to the curve y = f (x) at
the point P (x0 , f0 ) is given by

y − f (x0 ) = (x − x0 )f 0 (x0 )

where f 0 (x0 ) is the slope of the tangent to the curve at p.

y = f (x) at the point P (x0 , f0 ) is given by

y − f (x0 ) = (x − x0 )f (x0 )

22
 Figure 2.5: Newton Raphson Method

where f (x0 ) is the slope of the tangent to the curve at P. Setting y = 0 and
solving for x, we get
f (x0 ) 0
x = x0 − , f (x0 ) 6= 0
f 0 (x0 )
The next approximation to the root is given by
f (x0 )
x1 = x0 − , f 0 (x0 ) 6= 0
f 0 (x0 )
We repeat the procedure. The iteration method is defined as
f (xk )
xk+1 = xk − , f 0 (xk ) 6= 0
f 0 (xk )
This method is called the Newton-Raphson method or simply the Newtons
method. The method is also called the tangent method.

Alternate derivation of the method

Let xk be an approximation to the root of the equation f (x) = 0. Let ∆x

be an increment in x such that xk +∆x is the exact root, that is f (xk +∆x) =
0.
Expanding in Taylors series about the point xk , we get

0 (∆x)2 00
f (xk ) + ∆xf (xk ) + f (xk ) + . . . = 0
2!
Neglecting the second and higher powers of ∆x, we obtain

f (xk ) + ∆xf 0 (xk ) ≈ 0,

or
f (xk )
∆x = −
f 0 (xk )

23
Hence, we obtain the iteration method

f (xk )
xk+1 = xk + ∆x = xk − , f 0 (xk ) 6= 0 , k = 0, 1, 2, . . .
f 0 (xk )
which is same as the method derived earlier.

Remark 2.5 Convergence of the Newtons method depends on the initial ap-
proximation to the root. If the approximation is far away from the exact root,
the method diverges (see Example 2.5.1). However, if a root lies in a small
interval (a, b) and x0 ∈ (a, b), then the method converges.

Remark 2.6 From Eq.(1.14), we observe that the method may fail when
f 0 (x) is close to zero in the neighborhood of the root. Later, in this section,
we shall give the condition for convergence of the method.

Remark 2.7 The computational cost of the method is one evaluation of the
function f (x) and one evaluation of the derivative f 0 (x), for each iteration.

Example 2.5.1 Derive the Newtons method for finding N1 , where N > 0.
1
Hence, find 17 ,using the initial approximation as (i) 0.05, (ii) 0.15. Do the
iterations converge ?

Solution: Let x = N1 or 1
x
=N Define f (x) = 1
x
− N.
Then ,f 0 (x) = − x12 .
Newtons method gives
1
f (xk ) xk
−N
xk+1 = xk − 0 = xk − = xk + (xk − N x2k ) = 2xk − N x2k
f (xk ) − x12
k

(i) With N = 17, and x0 = 0.05, we obtain the sequence of approximations

x1 = 2x0 − N x20 = 2(0.05) − 17(0.05)2 = 0.0575

x2 = 2x1 − N x21 = 2(0.0575) − 17(0.0575)2 = 0.058794.

x3 = 2x2 − N x22 = 2(0.058794) − 17(0.058794)2 = 0.058823.
x4 = 2x3 − N x23 = 2(0.058823) − 17(0.058823)2 = 0.058823.
Since, |x4 − x3 | = 0, the iterations converge to the root.
The required root is 0.058823.
(ii) With N = 17, and x0 = 0.15, we obtain the sequence of approximations

x1 = 2x0 − N x20 = 2(0.15) − 17(0.15)2 = −0.0825.

24
 x2 = 2x1 − N x21 = 2(−0.0825) − 17(−0.8025)2 = −0.280706.
x3 = 2x2 − N x22 = 2(−0.280706) − 17(0.280706)2 = −1.900942.
x4 = 2x3 − N x23 = 2(−1.900942) − 17(−1.900942)2 = −65.23275.
We find that xk → −∞ as k increases.
Therefore, the iterations diverge very fast. This shows the importance of
choosing a proper initial approximation.
Example 2.5.2 Derive the Newtons method for finding the q th root of a
positive number N, N 1/q , where N > 0, q > 0.
Hence, compute 171/3 correct to four decimal places, assuming the initial
approximation as x0 = 2.
Solution : Let x = N 1/q , or xq = N. Define f (x) = xq − N.

Then , f 0 (x) = qxq−1 .

Newtons method gives the iteration
xqk − N qxqk − (xqk − N ) (q − 1)xqk + N
xk+1 = xk − = =
qxq−1
k qxq−1
k qxq−1
k
1
For computing 17 3 , we have q = 3 and N = 17. Hence, the method becomes
(3 − 1)x3k + 17 2x3k + 17
xk+1 = = k = 0, 1, 2, 3, ...
3x3−1
k 3x2k

With x0 = 2, we obtain the following results.

2x30 + 17 2(23 ) + 17
x1 = = = 2.75
3x20 3(2)2

2x31 + 17 2(2.75)3 + 17
x2 = = = 2.582645
3x21 3(2.75)2
2x32 + 17 2(2.582645)3 + 17
x3 = = = 2.571332
3x22 3(2.582645)2
2x33 + 17 2(2.571332)3 + 17
x4 = = = 2.571282.
3x23 3(2.571332)2
Now,
|x4 − x3 | = |2.571282 − 2.571332| = 0.00005.
We may take x ≈ 2.571282 as the required root correct to four decimal places

25
Example 2.5.3 Perform four iterations of the Newtons method to find the
smallest positive root of the equation f (x) = x3 − 5x + 1 = 0.

Solution: We have f (0) = 1, f (1) = 3. Since, f (0)f (1) < 0 , the smallest
positive root lies in the interval (0, 1). Applying the Newtons method, we
obtain
x3k − 5xk + 1 2x3k − 1
xk+1 = xk − = , k = 0 , 1 , 2 , ...
3x2k − 5 3x2k − 5

Let x0 = 0.5. We have the following results.

2x30 − 1 2(0.5)3 − 1
x1 = = = 0.176471,
3x20 − 5 3(0.5)2 − 5

2x31 − 1 2(0.176471)3 − 1
x2 = = = 0.201568,
3x21 − 5 3(0.176471)2 − 5
2x32 − 1 2(0.201568)3 − 1
x3 = = = 0.201640,
3x22 − 5 3(0.201568)2 − 5
2x33 − 1 2(0.201640)3 − 1
x4 = = = 0.201640.
3x23 − 5 3(0.201640)2 − 5
Therefore, the root correct to six decimal places is x ≈ 0.201640.

Exercises 2.2 Using Newton-Raphson method solve xlog10 x = 12.34 with

x0 = 10 correct to four decimal places.

ans:x ≈ 11.594854

2.6 General Iteration Method

The method is also called iteration method or method of successive approx-
imations or fixed point iteration method. The first step in this method is to
rewrite the given equation f (x) = 0 in an equivalent form as

x = ϕ(x). (1)

There are many ways of rewriting f (x) = 0 in this form. For example,
f (x) = x3 − 5x + 1 = 0, can be rewritten in the following forms.
r
x3 + 1 1 5x − 1
x= , x = (5x − 1) 3 , x = , etc. (2)
5 x

26
Now, finding a root of f (x) = 0 is same as finding a number α such that
α = ϕ(α), that is, a fixed point of ϕ(x). A fixed point of a function ϕ is a
point α such that α = ϕ(α). This result is also called the fixed point theorem.
Using Eq.(1), the iteration method is written as

xk+1 = ϕ(xk ), k = 0, 1, 2, ... (3)

The function ϕ(x) is called the iteration function. Starting with the initial
approximation x0 , we compute the next approximations as

x1 = ϕ(x0 ), x2 = ϕ(x1 ), x3 = ϕ(x2 ), ...

The stopping criterion is same as used earlier. Since, there are many ways
of writing f (x) = 0 as x = ϕ(x), it is important to know whether all or at
least one of these iteration methods converges.

Remark 2.8 Convergence of an iteration method xk+1 = ϕ(xk ), k = 0, 1, 2, ...,

depends on the choice of the iteration function ϕ(x), and a suitable initial
approximation x0 , to the root.

Consider again, the iteration methods given in Eq.(2), for finding a root of
the equation f (x) = x3 − 5x + 1 = 0. The positive root lies in the interval
(0, 1).
x3 + 1
(i) xk+1 = k k = 0, 1, 2, ... (4)
5
With x0 = 1, we get the sequence of approximations as

x1 = 0.4, x2 = 0.2128, x3 = 0.20193, x4 = 0.20165, x5 = 0.20164.

The method converges and x ≈ x5 = 0.20164 is taken as the required ap-

proximation to the root.
1
(ii) xk+1 = (5xk − 1) 3 , k = 0, 1, 2, ... (5)

With x0 = 1, we get the sequence of approximations as

x1 = 1.5874, x2 = 1.9072, x3 = 2.0437, x4 = 2.0968, ...

which does not converge to the root in (0, 1).

r
5xk − 1
xk+1 = , k = 0, 1, 2, ... (6)
kk

27
With x0 = 1, we get the sequence of approximations as

x1 = 2.0, x2 = 2.1213, x3 = 2.1280, x4 = 2.1284, ...

which does not converge to the root in (0, 1). Now, we derive the condition
that the iteration function ϕ(x) should satisfy in order that the method con-
verges.

Condition of convergence

Theorem 1

If α be exact root of f (x) = 0 ,I be any interval containing the points

x = α and |φ0 (x)| < 1 for all x ∈ I , then the sequence of approximations
x0 , x1 , x2 , . . . , xk , . . . will converge to the exact root α provided that the
initial approximation x0 is chosen in I.

proof:
The iteration method for finding a root of f (x) = 0, is written as

xk+1 = ϕ(xk ) , k = 0, 1, 2, ... (2.3)

Let α be the exact root. That is,

α = ϕ(α) (2.4)

We define the error of approximation at the k th iterate as

εk = xk ,

k = 0, 1, 2,... Subtracting (2.4) from (2.3), we obtain

xk+1 − α = ϕ(xk ) − ϕ(α) = (xk − α)ϕ0 (tk ) (2.5)

28
(using the mean value theorem) (2.5) or

εk+1 = ϕ(tk )εk , xk < tk < α.

Setting k = k − 1, we get

εk = ϕ0 (tk−1 )εk−1 , xk−1 < tk−1 < α.

Hence,
εk+1 = ϕ0 (tk )ϕ0 (tk−1 )εk−1 .
Using (2.5) recursively, we get

εk+1 = ϕ0 (tk )ϕ0 (tk−1 ) . . . ϕ0 (t0 )ε0 .

The initial error ε0 is known and is a constant. We have

|εk+1 | = |ϕ0 (tk )||ϕ(tk1 )|...|ϕ0 (t0 )||ε0 |.

Let |ϕ0 (tk )| ≤ c, k = 0, 1, 2, . . . Then,

|εk+1 | ≤ ck+1 |ε0 |. (2.6)

For convergence, we require that |εk+1 | → 0 as k → ∞. This result is

possible, if and only if c < 1. Therefore, the iteration method (2.3) converges,
if and only if
|ϕ(xk )| ≤ c < 1, k = 0, 1, 2, . . .
or |ϕ0 (x)| ≤ c < 1, for all x in the interval (a, b). (2.6) We can test this
condition using x0 , the initial approximation, before the computations are
done.
Let us now check whether the methods (4), (5), (6) converge to a root in
(0, 1) of the equation f (x) = x3 − 5x + 1 = 0.
3 2 2
i. We have ϕ(x) = x 5+1 , ϕ0 (x) = 3x5 and |ϕ0 (x)| = 3x5 ≤ 1, for all x in
0 < x < 1. Hence, the method converges to a root in (0, 1).
1
ii. We have ϕ(x) = 5x − 1) 3 , ϕ0 (x) = 5
3(5x−1)2/3

Now |ϕ0 (x)| < 1, when x is close to 1 and |ϕ(x)| > 1 in the other part
of the interval. Convergence is not guaranteed.
q
iii. We have ϕ(x) = 5x−1 x
, ϕ0 (x) = 3 1 1 .
2x 2 (5x−1) 2

Again, |ϕ0 (x)| < 1, when x is close to 1 and |ϕ0 (x)| > 1 in the other
part of the interval. Convergence is not guaranteed.

29
Example 2.6.1 Find the smallest positive root of the equation x3 − x − 10 =
0, using the general iteration method.

Solution: We have

f (x) = x3 − x − 10, f (0) = −10, f (1) = −10,

f (2) = 8 − 2 − 10 = −4, f (3) = 27 − 3 − 10 = 14.

Since, f (2)f (3) < 0, the smallest positive root lies in the interval (2, 3). Write
1
x3 = x + 10, and x = (x + 10) 3 = ϕ(x). We define the iteration method as
1
xk+1 = (xk + 10) 3 .

We obtain
1
ϕ0 (x) = 2
3(x + 10) 3
We find |ϕ0 (x)| < 1 for all x in the interval (2, 3). Hence, the iteration
converges. Let x0 = 2.5. We obtain the following results.
1 1
x1 = (12.5) 3 = 2.3208, x2 = (12.3208) 3 = 2.3097,
1 1
x3 = (12.3097) 3 = 2.3090, x4 = (12.3090) 3 = 2.3089.
Since, |x4 − x3 | = |2.3089 − 2.3090| = 0.0001, we take the required root as
x ≈ 2.3089.
Example 2.6.2 Find the smallest negative root in magnitude of the equation
3x4 + x3 + 12x + 4 = 0, using the method of successive approximations.
Solution: We have

f (x) = 3x4 + x3 + 12x + 4 = 0, f (0) = 4, f (−1) = 3 − 1 − 12 + 4 = −6.

Since, f (−1)f (0) < 0, the smallest negative root in magnitude lies in the
interval (−1, 0). Write the given equation as

x(3x3 + x2 + 12) + 4 = 0,

and
4
x=− = ϕ(x)
3x3
+ x2 + 12
The iteration method is written as
4
xk+1 = −
3x3k + x2k + 12

30
We obtain
4(9x2 + 2x)
ϕ0 (x) =
(3x3 + x2 + 12)2
We find |ϕ0 (x)| < 1 for all x in the interval (−1, 0). Hence, the iteration
converges. Let x0 = −0.25. We obtain the following results.
4 4
x1 = − =− = −0.33290,
3x30 2
+ x0 + 12 3(−0.25) + (−0.25)2 + 12
3

4 4
x2 = − =− = −0.33333,
3x31 2
+ x1 + 12 3(−0.33290) + (−0.33290)2 + 12
3

4 4
x3 = − =− = −0.33333.
3x32 2
+ x2 + 12 3(−0.33333) + (−0.33333)2 + 12
3

The required approximation to the root is x ≈ −0.33333.

We conclude that both fixed point iteration and regula-falsi methods con-
verge slowly as they have only linear rate of convergence. Farther, Newton’s
method converges at least twice as fast as the fixed point iteration.

31
Chapter 3

System of equation

3.1 Introduction
Consider a system of n linear algebraic equations in n unknowns

a11 x1 + a12 x2 + . . . + a1n xn = b1

a21 x1 + a22 x2 + . . . + a2n xn = b2
..
.
an1 x1 + an2 x2 + . . . + ann xn = bn
where aij , i = 1, 2, · · · , n, j = 1, 2, · · · , n, are the known coefficients, bi , i =
1, 2, · · · , n, are the known right hand side values and xi , i = 1, 2, · · · n are
the unknowns to be determined.
In matrix notation we write the system as
Ax = b (3.1)
     
a11 a12 a13 . . . a1n x1 b1
 a21 a22 a23 . . . a2n   x2   b2 
Where A =  .. ..  , x =  ..  and b =  .. 
     
.. ..
 . . . .  . .
an1 an2 an3 . . . ann xn bn
The matrix [A|b], obtained by appending the column b to the matrix A
is called the augmented matrix. That is
 
a11 a12 . . . a1n b1
 a21 a22 . . . a2n b2 
[A|b] =  ..
 
.. .. .. .. 
 . . . . . 
an1 an2 . . . ann bn

32
We define the following.
i. The system of equations (3.1) is consistent (has at least one solution),
if rank(A) = rank[A|b] = r. If r = n, then the system has unique
solution. If r < n, then the system has (n − r) parameter family of
infinite number of solutions.
ii. The system of equations (3.1) is inconsistent (has no solution) if
rank(A) 6= rank[A|b].
We assume that the given system is consistent. The methods of solution of
the linear algebraic system of equations (3.1) may be classified as direct and
iterative methods.
(a) Direct methods produce the exact solution after a finite number of steps
(disregarding the round-off errors). In these methods, we can determine
the total number of operations (additions, subtractions, divisions and
multiplications). This number is called the operational count of the
method.
(b) Iterative methods are based on the idea of successive approximations.
We start with an initial approximation to the solution vector x = x0 ,
and obtain a sequence of approximate vectors x0 , x1 , ·, xk , · which in the
limit as k → ∞ converge to the exact solution vector x.
Now, we derive some direct methods.

3.2 Direct Methods

If the system of equations has some special forms, then the solution is ob-
tained directly We consider two such special forms.
(a) Let A be a diagonal matrix, A = D. That is, we consider the system
of equations Dx = b as

a11 x1 = b1
a22 x2 = b2
... ..
.
ann xn = bn
This system is called a diagonal system of equations. Solving directly,
we obtain
bi
xi = , aii 6= 0, i = 1, 2, ..., n.
aii

33
 (b) Let A be an upper triangular matrix, A = U . That is, we consider the
system of equations U x = b as
a11 x1 + a12 x2 + · · · + a1n xn = b1
a22 x2 + · · · + a2n xn = b2
... ..
.
an−1,n−1 xn−1 + an−1,n xn = bn−1
ann xn = bn
This system is called an upper triangular system of equations. Solving for
the unknowns in the order xn , xn−1 , ..., x1 , we get
xn = abnn
n

bn−1 −an−1,n xn
xn−1 = an−1 ,n−1
··· P···
(b1 − nj=2 a1j xj )
x1 = a11

The unknowns are obtained by back substitution and this procedure is called
the back substitution method. Therefore, when the given system of equa-
tions is one of the above two forms, the solution is obtained directly.

Elementary row transformations (operations)

The following operations on the rows of a matrix A are called the elementary
row transformations (operations).
(i) Interchange of any two rows. If we interchange the ith row with the j th
row, then we usually denote the operation as Ri ↔ Rj .
(ii) Division/multiplication of any row by a non-zero number p. If the
ith row is multiplied by p, then we usually denote this operation as
pRi → Ri .
(iii) Adding/subtracting a scalar multiple of any row to any other row. If
all the elements of the j th row are multiplied by a scalar p and added
to the corresponding elements of the ith row, then, we usually denote
this operation as Ri + pRj → Ri .
These row operations change the form of A, but do not change the row-rank
of A. The matrix B obtained after the elementary row operations is said
to be row equivalent with A. In the context of the solution of the system
of algebraic equations, the solution of the new system is identical with the
solution of the original system. The above elementary operations performed
on the columns of A (column C in place of row R) are called elementary
column transformations (operations). However, we shall be using only the
elementary row operations.

34
3.2.1 Gauss Elimination Method
The method is based on the idea of reducing the given system of equations
Ax = b, to an upper triangular system of equations U x = z, using elementary
row operations
This reduced system U x = z, is then solved by the back substitution method
to obtain the solution vector x.
We illustrate the method using the 3 × 3 system

a11 x1 + a12 x2 + a13 x3 = b1

a21 x1 + a22 x2 + a23 x3 = b2 (3.2)
a31 x1 + a32 x2 + a33 x3 = b3

We write the augmented matrix [A|b] and reduce it to the following form

[A|b]−−
Gauss −−−−−−−−−→[U |z]
−−−−−elimination
The augmented matrix of the system (3.2) is
 
a11 a12 a13 b1
 a21 a22 a23 b2  (3.3)
a31 a32 a33 b3

First stage of elimination We assume a11 6= 0. This element a11 in the 1 × 1

position is called the first pivot. We use this pivot to reduce all the elements
below this pivot in the first column as zeros. Multiply the first row in (3.3)
by aa11
21
and aa11
31
respectively and subtract from the second and third rows.
That is, we are performing the elementary row operations R2 − aa21 11
R1 and
a31
R3 − a11 R1 respectively. We obtain the new augmented matrix as

a11 a12 a13 b1

 
 0 a(1)
22
(1) (1)
a23 b2  (3.4)
(1) (1) (1)
0 a32 a33 b3

where
(1) a21 (1) a21 (1) a21
a22 = a22 − a12 , a23 = a23 − a13 , b2 = b2 − b1
a11 a11 a11
(1) a31 (1) a31 (1) a31
a32 = a32 − a12 , a33 = a33 − a13 , b3 = b3 − b1
a11 a11 a11

35
Second stage of elimination
(1) (1)
We assume a22 6= 0 . This element a22 in the 2 × 2 position is called
the second pivot. We use this pivot to reduce the element below this pivot
(1)
a32
in the second column as zero. Multiply the second row in (3.4) by (1) and
a22
subtract from the third row. That is, we are performing the elementary row
(1)
a32
operation R3 − ( (1) )R2 . We obtain the new augmented matrix as
a22

a11 a12 a13 b1

 
 0 a(1)
22
(1) (1)
a23 b2  (3.5)
(2) (2)
0 0 a33 b3

Where
(1) (1)
(2) (1) a32 (1) (2) (1) a32 (1)
a33 = a33 − a
(1) 23
, b 3 = b3 − b
(1) 2
a22 a22
(2)
The element a33 6= 0 is called the third pivot. This system is in the required
upper triangular form [U |z]. The solution vector x is now obtained by back
substitution. (2)
b
From the third row, we get x3 = 3(2)
a33
(1) (1)
(b2 −a23 x3 )
From the second row, we get x2 = (1) .
a(22)
(b1 −a12 x2 −a13 x3 )
From the first row, we get x1 = a11
.

In general, using a pivot, all the elements below that pivot in that column
are made zeros. Alternately, at each stage of elimination, we may also make
the pivot as 1, by dividing that particular row by the pivot.

Remark 3.1 When does the Gauss elimination method as described above
fail? It fails when any one of the pivots is zero or it is a very small number,
as the elimination progresses.

If a pivot is zero, then division by it gives over flow error, since division
by zero is not defined. If a pivot is a very small number, then division by it
introduces large round-off errors and the solution may contain large errors.
In the second stage of elimination, the second column is searched for the
largest element in magnitude among the n − 1 elements leaving the first
element, and this element is brought as the second pivot by interchanging
the second row of the augmented matrix with the later row having the largest
element in magnitude. This procedure is continued until the upper triangular

36
system is obtained. Therefore, partial pivoting is done after every stage of
elimination. There is another procedure called complete pivoting. In this
procedure, we search the entire matrix A in the augmented matrix for the
largest element in magnitude and bring it as the first pivot.
This requires not only an interchange of the rows, but also an interchange
of the positions of the variables. It is possible that the position of a variable
is changed a number of times during this pivoting. We need to keep track
of the positions of all the variables. Hence, the procedure is computationally
expensive and is not used in any software.

Remark 3.2 Gauss elimination method is a direct method. Therefore, it is

possible to count the total number of operations, that is, additions, subtrac-
tions, divisions and multiplications. Without going into details, we mention
that the total number of divisions and multiplications (division and multi-
2
plication take the same amount of computer time) is n( n +3n−1
3
). The total
number of additions and subtractions (addition and subtraction take the same
amount of computer time) is n( (n−1)(2n+5)
6
).

Remark 3.3 When the system of algebraic equations is large, how do we

conclude that it is consistent or not, using the Gauss elimination method? A
way of determining the consistency is from the form of the reduced system
(3.5). We know that if the system is inconsistent then rank(A) 6= rank[A|b].
By checking the elements of the last rows, conclusion can be drawn about the
consistency or inconsistency.
(2) (2)
Suppose that in (3.5), a33 6= 0 and b3 6= 0. Then, rank(A) = rank[A|b] =
3. The system is consistent and has a unique solution. Suppose that we
obtain the reduced system as
a11 a12 a13 b1
 
 0 a(1)22
(1) (1)
a23 b2 
(2)
0 0 0 b3
Then, rank(A) = 2, rank[A|b] = 3 and rank(A) 6= rank[A|b]. Therefore,
the system is inconsistent and has no solution. Suppose that we obtain the
reduced system as  
a11 a12 a13 b1
 0 a(1) a(1) b(1) 
22 23 2
0 0 0 0
Then, rank(A) = rank[A|b] = 2 < 3. Therefore, the system has 3 − 2 = 1
parameter family of infinite number of solutions.

37
Example 3.2.1 Solve the system of equations
x1 + 10x2 − x3 = 3
2x1 + 3x2 + 20x3 = 7
10x1 − x2 + 2x3 = 4
using the Gauss elimination with partial pivoting.
Solution: We have the augmented matrix as
 
1 10 −1 3
 2 3 20 7 
10 −1 2 4
We perform the following elementary row transformations and do the elimi-
nations.  
10 −1 2 4
R1 ↔ R3 :  2 3 20 7 
1 10 −1 3
R2 − (R1 /5) → R2 , R3 − (R1 /10) → R3 :
 
10 −1 2 4
 0 3.2 19.6 6.2 
0 10.1 −1.2 2.6
R2 ↔ R3 :  
10 −1 2 4
 0 10.1 −1.2 2.6 
0 3.2 19.6 6.2
R3 − (3.2/10.1)R2 → R3 :
 
10 1 2 4
 0 101 12 26 
0 0 19.98020 5.37624
Back substitution gives the solution.
5.37624
Third equation gives x3 = 19.98020 = 0.26908

Second equation gives

1 1
x2 = (2.6 + 1.2x3 ) = (2.6 + 1.2(0.26908)) = 0.28940
10.1 10.1

First equation gives

1 1
x1 = (4 + x2 − 2x3 ) = (4 + 0.2894 − 2(0.26908)) = 0.37512.
10 10

38
Example 3.2.2 Solve the system of equations

2x1 + x2 + x3 − 2x4 = −10

4x1 + 2x3 + x4 = 8
3x1 + 2x2 + 2x3 = 7
x1 + 3x2 + 2x3 − x4 = −5
using the Gauss elimination with partial pivoting.

Solution: The augmented matrix is given by

 
2 1 1 −2 −10
 4 0 2 1 8 
 
 3 2 2 0 7 
1 3 2 −1 −5

We perform the following elementary row transformations and do the

eliminations.  
4 0 2 1 8
 2 1 1 −2 −10 
R1 ↔ R2 : 
 3 2

2 0 7 
1 3 2 −1 −5

 
4 0 2 1 8
 0 1 0 − 25 −14 
R2 −(1/2)R1 → R2 , R3 −(3/4)R1 → R3 , R4 −(1/4)R1 → R4 :  1 3

 0 2
2
− 4
1 
3 5
0 3 2
− 4 −7
 
4 0 2 1 8
 0 3 3 − 5 −7 
R2 ↔ R4 :  2
 0 2 1 −3
4 
2 4
1 
0 1 0 − 25 −14
 
4 0 2 1 8
 0 3 3 − 5 −7 
R3 − (2/3)R2 → R3 , R4 − (1/3)R2 → R4 :   0 0 −1
2
1
4
17


2 12 3
0 0 − 12 − 25 2
− 35
3
 
4 0 2 1 8
 0 3 3 − 5 −7 
R4 − R3 → R4 , :  0 0 −1
2
1
4
17 

2 12 3
0 0 0 − 13 6
− 52
3

39
Using back substitution, we obtain
52 6 17 1 17 1
x4 = (− )(− ) = 8 , x3 = −2( − )x4 = −2( − ( )8) = −10
3 13 3 12 3 12
1 3 5 1 3 5
x2 = [−7 − ( )x3 + ( )x4 ] = [−7 − ( )(−10) + ( )8] = 6
3 2 4 3 2 4
1 1
x1 = [8 − 2x3 − x4 ] = [8 − 2(−10) − 8] = 5.
4 4
Exercises 3.1 Solve the system of equations

3x1 + 3x2 + 4x3 = 20

2x1 + x2 + 3x3 = 13
x1 + x2 + 3x3 = 6
using the Gauss elimination method.

Example 3.2.3 Test the consistency of the following system of equations

x1 + 10x2 x3 = 3

2x1 + 3x2 + 20x3 = 7

9x1 + 22x2 + 79x3 = 45
using the Gauss elimination method.

Solution: We have the augmented matrix as

 
1 10 1 3
 2 3 20 7 
9 22 79 45
We perform the following elementary row transformations and do the elimi-
nations.
 
1 10 1 3
R2 − 2R1 → R2 , R3 − 9R1 → R3 :  0 17 22 1 
0 68 88 18
.  
1 10 1 3
R3 − 4R2 → R3 :  0 17 22 1 
0 0 0 14
Now, rank [A] = 2, and rank[A|b] = 3. Therefore, the system is inconsistent
and has no solution.

40
3.2.2 Gauss-Jordan Method
The method is based on the idea of reducing the given system of equations
Ax = b, to a diagonal system of equations Ix = d, where I is the identity
matrix, using elementary row operations. We know that the solutions of both
the systems are identical. This reduced system gives the solution vector x.
This reduction is equivalent to finding the solution as x = A−1 b.

[A|b]−Gauss-Jordan
−−−−−−−−−−−− method
−−−−→[I|X]
In this case, after the eliminations are completed, we obtain the aug-
mented matrix for a 3 × 3 system as
 
1 0 0 d1
 0 1 0 d2  (3.6)
0 0 1 d3

and the solution is xi = di , i = 1, 2, 3.

Elimination procedure
The first step is same as in Gauss elimination method, that is, we make the
elements below the first pivot as zeros, using the elementary row transfor-
mations. From the second step onwards, we make the elements below and
above the pivots as zeros using the elementary row transformations. Lastly,
we divide each row by its pivot so that the final augmented matrix is of the
form (3.6). Partial pivoting can also be used in the solution. We may also
make the pivots as 1 before performing the elimination.

Example 3.2.4 Solve the following system of equations

x1 + x2 + x3 = 1

4x1 + 3x2 − x3 = 6
3x1 + 5x2 + 3x3 = 4
using the Gauss-Jordan method (i) without partial pivoting, (ii) with partial
pivoting.
Solution: We have the augmented matrix as
 
1 1 1 1
 4 3 −1 6 
3 5 3 4

41
(i) We perform the following elementary row transformations and do the
eliminations.
 
1 1 1 1
R2 − 4R1 → R2 , R3 − 3R1 → R3 :  0 −1 −5 2 
0 2 0 1
 
1 0 −4 3
R1 + R2 → R1 , R3 + 2R2 → R3 :  0 −1 −5 2 
0 0 −10 5
 
1 0 0 1
4 5
R1 − R3 → R1 , R2 − R3 → R2 :  0 −1 0 − 12 
10 10
0 0 −10 5
R3
Now, making the pivots as 1, ((−R2 ), −10 ) we get
 
1 0 0 1
 0 1 0 1 
2
0 0 1 − 12

Therefore, the solution of the system is x1 = 1, x2 = 21 , x3 = − 12 .

(ii) We perform the following elementary row transformations and do the

elimination.  
4 3 −1 6
R1 ↔ R2 :  1 1 1 1 
3 5 3 4

1 43 − 14 32
 
R1
→ R1 :  1 1 1 1 
4
3 5 3 4
1 34 − 14 3
 
2
R2 − R1 → R2 , R3 − 3R1 → R3 :  0 41 5
4
− 12 
0 114
15
4
− 12
1 43 − 41 3
 
4R2 15
2
2 
→ R2 :  0 1 11 − 11
11
0 41 54 − 12
14 18
 
1 0 − 11
3 1 11
R1 − R2 → R1 , R3 − R2 → R3 :  0 1 15 11
2
− 11 
4 4
0 0 10 11
5
− 11

42
 14 18
 
1 0 − 11 11
R3 /(10/11) → R3 :  0 1 15
11
2 
− 11
1
0 0 1 −2
 
1 0 0 1
R1 + (14/11)R3 → R1 , R2 − (15/11)R3 → R2 :  0 1 0 12 
0 0 1 − 21
Therefore, the solution of the system is x1 = 1, x2 = 12 , x3 = − 12 .

Inverse of a Matrix by Gauss-Jordan Method

We start with the augmented matrix of A with the identity matrix I of the
same order. When the Gauss-Jordan procedure is completed, we obtain
−1
[A|I]−Gauss-Jordan
−−−−−−−−−−−− method
−−−−→[I|A ]
since, AA−1 = I.

Remark 3.4 Partial pivoting can also be done using the augmented matrix
[A|I]. However, we cannot first interchange the rows of A and then find the
inverse. Then, we would be finding the inverse of a different matrix.

Exercises 3.2 Find the inverse of the matrix

 
1 1 1
 4 3 −1 
3 5 3

using the Gauss-Jordan method (i) without partial pivoting, and (ii) with
partial pivoting.

3.2.3 Matrix Decomposition

LU decomposition
Suppose we have the system of equations

AX = B.

The motivation for an LU decomposition is based on the observation that

systems of equations involving triangular coefficient matrices are easier to
deal with.

43
Definition 3.1 An LU decomposition of a matrix A is the product of a lower
triangular matrix and an upper triangular matrix that is equal to A.
Thus write the matrix A as
A = LU
where  
L11 0 0 ··· 0

 L21 L22 0 ··· 0 

L=
 L31 L32 L33 ··· 0 

 .. .. .. .. .. 
 . . . . . 
Ln1 Ln2 Ln3 ··· Lnn
 
U11 U12 U13 ··· U1n

 0 U22 U23 ··· U2n 

U =
 0 0 U33 ··· U3n 

 .. .. .. .. .. 
 . . . . . 
0 0 0 ··· Unn
Provided that Lii = 1 i = 1, 2, 3, · · · , n Doolittle’s method and
Uii = 1 i = 1, 2, 3, · · · , n Crout’s method

Note: One can consider Lii and uii any other value.
For the sack of simplicity we need only to consider
Lii = 1
for all i = 1, 2, 3, · · · n.

Remark 3.5 What ever the assumption one can take ,the result is not changed
Here is an example, let
 
1 2 4
A =  3 8 14  = LU
2 6 13
   
1 0 0 U11 U12 U13
where L =  L21 1 0  and U =  0 U22 U23  Multiplying out
L31 L32 1 0 0 U33
LU and setting the answer equal to A gives
   
U11 U12 U13 1 2 4
LU =  L21 U11 L21 U12 + U22 L21 U13 + U23  =  3 8 14 
L31 U11 L31 U12 + L32 U22 L31 U13 + L32 U23 + U33 2 6 13

44
Now we have to use this to find the entries in L and U. Fortunately this is
not nearly as hard as it might at first seem. We begin by running along the
top row to see that

U11 = 1 , U12 = 2 , U13 = 4.

Now consider the second row

L21 U11 = 3 ⇒ L21 × 1 = 3 ∴ L21 = 3,

L21 U12 + U22 = 8 ⇒ 3 × 2 + U22 = 8 ∴ U22 = 2,

L21 U13 + U23 = 14 ⇒ 3 × 4 + U23 = 14 ∴ U23 = 2
Notice how, at each step, the equation in hand has only one unknown in it,
and other quantities that we have already found. This pattern continues on
the last row
L31 U11 = 2 ⇒ L31 × 1 = 2 ⇒ L31 = 2,
L31 U12 + L32 U22 = 6 ⇒ 2 × 2 + L32 × 2 = 6 ⇒ L32 = 1,
L31 U13 + L32 U23 + U33 = 13 ⇒ (2 × 4) + (1 × 2) + U33 = 13 ⇒ U33 = 3.
We have shown that
    
1 2 4 1 0 0 1 2 4
A =  3 8 14  =  3 1 0   0 2 2 
2 6 13 2 1 1 0 0 3

and this is an LU decomposition of A.

Using an LU decomposition to solve systems of equations

Once a matrix A has been decomposed into lower and upper triangular parts
it is possible to obtain the solution to AX = B in a direct way. The proce-
dure can be summarised as follows

• Given A, find L and U so that A = LU . Hence LU X = B.

• Let D = U X so that LD = B. Solve this triangular system for D .

• Finally solve the triangular system U X = D for X.

The benefit of this approach is that we only ever need to solve triangular
systems. The cost is that we have to solve two of them.

45
 Figure 3.1: Lu decomposition
    
x1 1 2 4 x1
Example 3.2.5 Find the solution of X =  x2  of  3 8 14   x2  =
  x3 2 6 13 x3
3
 13 
4
Solution
The first step is to calculate the LU decomposition of the coefficient matrix
on the lefthand side. In this case that job has already been done since this
is the matrix we considered earlier. We found that
   
1 0 0 1 2 4
L= 3 1 0  , U = 0 2 2 
2 1 1 0 0 3
 
d1
The next step is to solve LD = B for the vector D =  d2  That is we
d3
consider     
1 0 0 d1 3
LD =  3 1 0   d2  =  13  = B
2 1 1 d3 4
which can be solved by forward substitution. From the top equation we see
that d1 = 3. The middle equation states that 3d1 +d2 = 13 and hence d2 = 4.
Finally the bottom line says that 2d1 + d2 + d3 = 4 from which we see that
d3 = −6. Now that we have found D we finish the procedure by solving

46
U X = D for X. That is we solve
    
1 2 4 x1 3
U X =  0 2 2   x2  =  4  = D
0 0 3 x3 −6

by using back substitution. Starting with the bottom equation we see that
3x3 = −6 so clearly x3 = −2. The middle equation implies that 2x2 +2x3 = 4
and it follows that x2 = 4. The top equation states that x1 + 2x2 + 4x3 = 3
and consequently x1 = 3. Therefore we have found that the solution to the
system of simultaneous equations
      
1 2 4 x1 3 3
 3 8 14   x2  =  13  is X =  4 
2 6 13 x3 4 −2

QR Decomposition
If A is a m × n matrix with linearly independent columns, then A can be
decomposed as A = QR, where Q is a m × n matrix whose columns form an
orthonormal basis for the column space of A and R is an nonsingular upper
triangular matrix.

Theorem 1

If A is a m × n matrix with linearly independent columns, then A can be

decomposed as A = QR , where Q is a m×n matrix whose columns form
an orthonormal basis for the column space of A and R is an non-singular
upper triangular matrix.

Proof : Suppose A = [u1 | u2 | . . . | un ] and rank(A) = n. Apply the Gram-

Schmidt process to {u1 , u2 , ..., un } and the orthogonal vectors v1 , v2 , . . . , vn
are

hui , v1 i hui , v2 i hui , vi−1 i

vi = ui − v 1 − v2 − · · · − vi−1
kv1 k2 kv2 k2 kvi−1 k2
Let qi = kvvii k for i = 1, 2, . . . , n. Thus q1 , q2 , . . . , qn form a orthonormal
basis for the column space of A.
Now,

hui , v1 i hui , v2 i hui , vi−1 i

ui = vi + 2
v1 + 2
v2 + · · · + vi−1
kv1 k kv2 k kvi−1 k2

47
 ⇒ ui = kvi kqi + hui , q1 iq1 + hui , q2 iq2 + · · · + hui , qi−1 iqi−1
i.e. ui ∈ span{v1 , v2 , v3 , . . . , vi } = span{q1 , q2 , q3 , . . . , qi }
Thus ui is orthogonal to qj for j > i

u1 = kv1 kq1
u2 = kv2 kq2 + hu2 , q1 iq1
u3 = kv3 kq3 + hu3 , q1 iq1 + hu3 , q2 iq2
..
.
un = kvn kqn + hun , q1 iq1 + hun , q2 iq2 + · · · + hun , qn−1 iqn−1
Let Q = [q1 q2 ... qn ] , so Q is a m × n matrix whose columns form an
orthonormal basis for the column space of A . Now
 
kv1 k hu2 , q1 i hu3 , q1 i . . . hun , q1 i
 0
 kv2 k hu3 , q2 i . . . hun , q2 i  
A = [u1 u2 . . . un ] = [q1 q2 q3 · · · qn ]  0
 0 kv3 k . . . hun , q3 i  
 .. .. .. .. .. 
 . . . . . 
0 kvn k
0 0 ...
(3.7)
Thus A can be decomposed as A = QR, where R is an upper triangular and
nonsingular matrix.

Example 3.2.6 Find the QR decomposition of

 
1 −1 −1
 1 0 0 
A= 1 −1 0 


0 0 −1

Solution: Applying Gram-Schmidt process of computing QR decompo-

sition
Step 1 √
r11 = ka1 k = 3
 1 
√
3
1 √1
 
q1 = a1 = 
 3 
ka1 k √1

3
 
0

48
step 2
2
r12 = q1T a2 = − √
3
step 3
 1  
√
− 31
  
−1 3
 √1   2 
 − (− √2 ) 
 0 
qb2 = a2 − q1 q1T a2 = a2 − q1 r12 =   √13  =  3 

 −1  1 
3  3
  − 3
0 0 0
r
2
r22 = kqb2 k =
3
 √1 
− 6
q
1  2 
q2 = qb2 =  3 
 
kqb2 k  −√ 1
6
0
step 4
1
r13 = q1T a3 = − √
3
step 5
1
r23 = q2T a3 = √
6
step 6
 1 
−2
T T
qb3 = a3 − q1 q1 a3 − q2 q2 a3 = a3 − r13 q1 − r23 q2 =  0 12 
−1
√
6
r33 = kqb3 k =
2
 
− √16
1  0 
q3 = qb3 =  √1 
 
kqb3  6 
− √26
Therefore, A = QR
√1 − √16 − √16  √
   
1 −1 −1 3 3 − √2 − √1

 1 0 0   √1 √2 0  3 3
3 6 √2 √1
 =  0 
 1 −1 0   √1 − √6 √16
1 6 √6
 
 3  6
0 0 −1 0 0 − √26 0 0 2

49
Uses: QR decomposition is widely used in computer codes to find the eigen-
values of a matrix, to solve linear systems, and to find least squares approx-
imations.

3.3 Iterative Methods

General linear iterative method for the solution of the system of equations
Ax = b, can be written in matrix form as
x(k+1) = Hx(k) + c, k = 0, 1, 2, . . .
respectively. H is called the iteration matrix, which depends on A and c is
a column vector, which depends on A and b. When to stop the iteration?
We stop the iteration procedure when the magnitudes of the differences be-
tween the two successive iterates of all the variables are smaller than a given
accuracy or error tolerance or an error bound , that is,
|xk+1
i − xki | ≤ , for all i (3.8)
Now, we derive two iterative methods for the solution of a 3 × 3 system of
algebraic equations
a11 x1 + a12 x2 + a13 x3 = b1
a21 x1 + a22 x2 + a23 x3 = b2 (3.9)
a31 x1 + a32 x2 + a33 x3 = b3

3.3.1 Gauss-Jacobi Iteration Method

Sometimes, the method is called Jacobi method. We assume that the pivots
aii 6= 0, for all i. Write the equations as
a11 x1 = b1 − (a12 x2 + a13 x3 )
a22 x2 = b2 − (a21 x1 + a23 x3 )
a33 x3 = b3 − (a31 x1 + a32 x2 )
The Jacobi iteration method is defined as
(k+1) 1
x1 = [b
a11 1
− (a12 xk2 + a13 xk3 )]
(k+1) 1
x2 = [b
a22 2
− (a21 xk1 + a23 xk3 )] k = 0, 1, 2, ... (3.10)
(k+1) 1
x3 = [b
a33 3
− (a31 xk1 + a32 xk2 )]

Since, we replace the complete vector x(k) in the right hand side of (3.10) at
the end of each iteration, this method is also called the method of simulta-
neous displacement.

50
Remark 3.6 A sufficient condition for convergence of the Jacobi method is
that the system of equations is diagonally dominant, that is, the coefficient
matrix A is diagonally dominant. We can verify that
n
X
|aii | ≥ |aij |.
j=1,i6=j

If the system is not diagonally dominant, we may exchange the equations, if

possible, such that the new system is diagonally dominant and convergence
is guaranteed. This implies that convergence may be obtained even if the
system is not diagonally dominant.

Remark 3.7 If the system is diagonally dominant, then the iteration con-
verges for any initial solution vector. If no suitable approximation is avail-
able, we can choose x = 0, that is xi = 0 for all i. Then, the initial approxi-
mation becomes xi = abiii , for all i.

Example 3.3.1 Solve the system of equations

4x1 + x2 + x3 = 2
x1 + 5x2 + 2x3 = −6
x1 + 2x2 + 3x3 = −4

using the Jacobi iteration method. Use the initial approximations as

(i) xi = 0, i = 1, 2, 3 , (ii) x1 = 0.5, x2 = −0.5, x3 = −0.5.
Perform five iterations in each case.

Solution: Note that the given system is diagonally dominant. Jacobi method
gives the iterations as
(k+1) (k) (k)
x1 = 0.25[2 − (x2 + x3 )]
(k+1) (k) (k)
x2 = 0.2[−6 − (x1 + 2x3 )]
(k+1) (k) (k)
x3 = 0.33333[−4 − (x1 + 2x2 )], k = 0, 1, ...
We have the following results.
(0) (0) (0)
(i) x1 = 0, x2 = 0, x3 = 0
First iteration
(1) (0) (0)
x1 = 0.25[2 − (x2 + x3 )] = 0.5

51
 (1) (0) (0)
x2 = 0.2[−6 − (x1 + 2x3 )] = 1.2
(1) (0) (0)
x3 = 0.33333[−4 − (x1 + 2x2 )] = 1.33333.
Second iteration
(2) (1) (1)
x1 = 0.25[2 − (x2 + x3 )] = 0.25[2 − (−1.2 − 1.33333)] = 1.13333
(2) (1) (1)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(0.5+2(−1.33333))] = −0.76668,
(2) (1) (1)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(0.5+2(−1.2))] = −0.7.
Third iteration
(3) (2) (2)
x1 = 0.25[2 − (x2 + x3 )] = 0.25[2 − (−0.76668 − 0.7)] = 0.86667,
(3) (2) (2)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(1.13333+2(−0.7))] = −1.14667,
(3) (2) (2)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(1.13333+2(−0.76668))] = −1.19998.
Fourth iteration
(4) (3) (3)
x1 = 0.25[2 − (x2 + x3 )] = 0.25[2(1.146671.19999)] = 1.08666,
(4) (3) (3)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(0.86667+2(−1.19998))] = −0.89334,
(4) (3) (3)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(0.86667+2(−1.14667))] = −0.85777.
Fifth iteration
(5) (4) (4)
x1 = 0.25[2−(x2 +x3 )] = 0.25[2−(−0.89334−0.85777)] = 0.93778,
(5) (4) (4)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(1.08666+2(−0.85777))] = −1.07422,
(5) (4) (4)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(1.08666+2(−0.89334))] = −1.09998.
It is interesting to note that the iterations oscillate and converge to the
exact solution
x1 = 1.0, x2 = 1, x3 = −1.0.
(0) (0) (0)
(ii) x1 = 0.5, x2 = −0.5, x3 = −0.5.
First iteration
(1) (0) (0)
x1 = 0.25[2 − (x2 + x3 )] = 0.25[2 − (−0.5 − 0.5)] = 0.75,
(1) (0) (0)
x2 = 0.2[−6 − (x1 + 2x3 )] = 0.2[−6 − (0.5 + 2(−0.5))] = −1.1,
(1) (0) (0)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(0.5+2(−0.5))] = −1.16667.

52
 Second iteration
(2) (1) (1)
x1 = 0.25[2 − (x2 + x3 )] = 0.25[2 − (−1.1 − 1.16667)] = 1.06667,
(2) (1) (1)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(0.75+2(−1.16667))] = −0.88333,
(2) (1) (1)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(0.75+2(−1.1))] = −0.84999.
Third iteration
(3) (2) (2)
x1 = 0.25[2−(x2 +x3 )] = 0.25[2−(−0.88333−0.84999)] = 0.93333,
(3) (2) (2)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(1.06667+2(−0.84999))] = −1.07334,
(3) (2) (2)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(1.06667+2(−0.88333))] = −1.09999.
Fourth iteration
(3) (3)
x1 (4) = 0.25[2−(x2 +x3 )] = 0.25[2−(−1.07334−1.09999)] = 1.04333,
(4) (3) (3)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(0.93333+2(−1.09999))] = −0.94667,
(4) (3) (3)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(0.93333+2(−1.07334))] = −0.92887.
Fifth iteration
(5) (4) (4)
x1 = 0.25[2−(x2 +x3 )] = 0.25[2−(−0.94667−0.92887)] = 0.96889,
(5) (4) (4)
x2 = 0.2[−6−(x1 +2x3 )] = 0.2[−6−(1.04333+2(−0.92887))] = −1.03712,
(5) (4) (4)
x3 = 0.33333[−4−(x1 +2x2 )] = 0.33333[−4−(1.04333+2(−0.94667))] = −1.04999.

Example 3.3.2 Solve the system of equations

26x1 + 2x2 + 2x3 = 12.6

3x1 + 27x2 + x3 = −14.3
2x1 + 3x2 + 17x3 = 6.0

using the Jacobi iteration method. Obtain the result correct to three decimal
places.

Solution: The given system of equations is strongly diagonally dominant.

Hence, we can expect faster convergence. Jacobi method gives the iterations
as
(k+1) (k) (k)
x1 = [12.6 − (2x2 + 2x3 )]/26
(k+1) (k) (k)
x2 = [−14.3 − (3x1 + x3 )]/27

53
 (k+1) (k) (k)
x3 = [6.0 − (2x1 + 3x2 )]/17 k = 0, 1, ...
Choose the initial approximation as
(0) (0) (0)
x1 = 0, x2 = 0, x3 = 0.

We obtain the following First iteration

(1) 1 (0) (0) 1

x1 = [12.6 − (2x2 + 2x3 )] = [12.6] = 0.48462
26 26
(1) 1 (0) (0) 1
x2 = [−14.3 − (3x1 + x3 )] = [−14.3] = −0.52963,
27 27
(1) 1 (0) (0) 1
x3 = [6.0 − (2x1 + 3x2 )] = [6.0] = 0.35294.
17 17
Second iteration
(2) 1 (1) (1) 1
x1 = [12.6−(2x2 +2x3 )] = [12.6−2(−0.52963+0.35294)] = 0.49821,
26 26
(2) 1 (1) (1) 1
x2 = [−14.3−(3x1 +x3 )] = [−14.3−(3(0.48462)+0.35294)] = −0.59655,
27 27
(2) 1 (1) (1) 1
x3 = [−6.0−(2x1 +3x2 )] = [6.0−(2(0.48462)+3(−0.52963))] = 0.38939
17 17
Third iteration
(3) 1 (2) (2) 1
x1 = [12.6−(2x2 +2x3 )] = [12.6−2(−0.59655+0.38939)] = 0.50006,
26 26
(3) 1 (2) (2) 1
x2 = [−14.3−(3x1 +x3 )] = [−14.3−(3(0.49821)+0.38939)] = −0.59941,
27 27
(3) 1 (2) (2) 1
x3 = [−6.0−(2x1 +3x2 )] = [6.0−(2(0.49821)+3(−0.59655))] = 0.39960
17 17
Fourth iteration
(4) 1 (3) (3) 1
x1 = [12.6−(2x2 +2x3 )] = [12.6−2(−0.59941+0.39960)] = 0.50000
26 26
(4) 1 (3) (3) 1
x2 = [−14.3−(3x1 +x3 )] = [−14.3−(3(0.50006)+0.39960)] = −0.59999
27 27
(4) 1 (3) (3) 1
x3 = [−6.0−(2x1 +3x2 )] = [6.0−(2(0.50006)+3(−0.59941))] = 0.39989
17 17
We find
(4) (3)
|x1 − x1 | = |0.5 − 0.50006| = 0.00006,

54
 (4) (3)
|x2 − x2 | = | − 0.59999 + 0.59941| = 0.00058,
(4) (3)
|x3 − x3 | = |0.39989 − 0.39960| = 0.00029.
Three decimal places of accuracy have not been obtained at this iteration.
Fifth iteration
(5) 1 (4) (4) 1
x1 = [12.6−(2x2 +2x3 )] = [12.6−2(−0.59999+0.39989)] = 0.50001,
26 26
(5) 1 (4) (4) 1
x2 = [−14.3−(3x1 +x3 )] = [−14.3−(3(0.50000)+0.39989)] = −0.60000
27 27
(5) 1 (4) (4) 1
x3 = [−6.0−(2x1 +3x2 )] = [6.0−(2(0.50000)+3(−0.59999))] = 0.40000
17 17
. We find
(5) (4)
|x1 − x1 | = |0.50001 − 0.5| = 0.00001
(5) (4)
|x2 − x2 | = | − 0.6 + 0.59999| = 0.00001,
(5) (4)
|x3 − x3 | = |0.4 − 0.39989| = 0.00011.
Since, all the errors in magnitude are less than 0.0005, the required solu-
tion is
x1 = 0.5, x2 = 0.6, x3 = 0.4.

3.3.2 Gauss-Seidel Iteration Method

we use the updated values of x1 , x2 , ..., xi−1 in computing the value of the
variable xi . We assume that the pivots aii 6= 0, for all i.
We write the equations as

a11 x1 = b1 − (a12 x2 + a13 x3 )

a22 x2 = b2 − (a21 x1 + a23 x3 )
a33 x3 = b3 − (a31 x1 + a32 x2 )

The Gauss-Seidel iteration method is defined as

(k+1) (k) (k)
x1 = a111 [b1 − (a12 x2 + a13 x3 )]
(k+1) (k+1) (k)
x2 = a122 [b2 − (a21 x1 + a23 x3 )] k = 0, 1, 2, ... (3.11)
(k+1) (k+1) (k+1)
x3 = a133 [b3 − (a31 x1 + a32 x2 )]

This method is also called the method of successive displacement.

If both the Gauss-Jacobi and Gauss-Seidel methods converge, then Gauss-
Seidel method converges at least two times faster than the Gauss-Jacobi
method.

55
Example 3.3.3 Find the solution of the system of equations

45x1 + 2x2 + 3x3 = 58

−3x1 + 22x2 + 2x3 = 47
5x1 + x2 + 20x3 = 67
correct to three decimal places, using the Gauss-Seidel iteration method.
Solution: The given system of equations is strongly diagonally dominant.
Hence, we can expect fast convergence. Gauss-Seidel method gives the iter-
ation
(k+1) 1 (k) (k)
x1 = (58 − 2x2 − 3x3 ),
45
(k+1) 1 (k+1) (k)
x2 = (47 + 3x1 − 2x3 ),
22
(k+1) 1 (k+1) (k+1)
x3 = (67 − 5x1 − x2 ).
20
(0) (0) (0)
Starting with x1 = 0, x2 = 0, x3 = 0, we get the following results.

First iteration
(1) 1 (0) (0) 1
x1 = (58 − 2x2 − 3x3 ) = (58) = 1.28889,
45 45
1
(1) (1) (0) 1
x2 = (47 + 3x1 − 2x3 ) = (47 + 3(1.28889) − 2(0)) = 2.31212,
22 22
(1) 1 (1) (1) 1
x3 = (675x1 − x2 ) = (67 − 5(1.28889) − (2.31212)) = 2.91217
20 20
Second iteration
(2) 1 (1) (1) 1
x1 = (58 − 2x2 − 3x3 ) = (58 − 2(2.31212) − 3(2.91217)) = 0.99198,
45 45
(2) 1 (2) (1) 1
x2 = (47 + 3x1 − 2x3 ) = (47 + 3(0.99198) − 2(2.91217)) = 2.00689,
22 22
(2) 1 (2) (2) 1
x3 = (67 − 5x1 − x2 ) = (67 − 5(0.99198) − (2.00689)) = 3.00166.
20 20
Third iteration
(3) 1 (2) (2) 1
x1 = (58 − 2x2 − 3x3 ) = (58 − 2(2.00689) − 3(3.00166) = 0.99958,
45 45
(3) 1 (3) (2) 1
x2 = (47 + 3x1 − 2x3 ) = (47 + 3(0.99958) − 2(3.00166)) = 1.99979,
22 22
56
 (3) 1 (3) (3) 1
x3 = (67 − 5x1 − x2 ) = (67 − 5(0.99958) − (1.99979)) = 3.00012.
20 20
Fourth iteration
(4) 1 (3) (3) 1
x1 = (58 − 2x2 − 3x3 ) = (58 − 2(1.99979) − 3(3.00012)) = 1.00000,
45 45
(4) 1 (4) (3) 1
x2 = (47 + 3x1 − 2x3 ) = (47 + 3(1.00000) − 2(3.00012)) = 1.99999,
20 22
(4) 1 (4) (4) 1
x3 = (67 − 5x1 − x2 ) = (67 − 5(1.00000) − (1.99999)) = 3.00000.
20 20
We find
(4) (3)
|x1 − x1 | = |1.00000 − 0.99958| = 0.00042,
(4) (3)
|x2 − x2 | = |1.99999 − 1.99979| = 0.00020,
(4) (3)
|x3 − x3 | = |3.00000 − 3.00012| = 0.00012.
Since, all the errors in magnitude are less than 0.0005, the required solu-
tion is
x1 = 1.0, x2 = 1.99999, x3 = 3.0.
Rounding to three decimal places, we get x1 = 1.0, x2 = 2.0, x3 = 3.0.

Example 3.3.4 Computationally show that Gauss-Seidel method applied to

the system of equations

3x1 − 6x2 + 2x3 = 23

−4x1 + x2 − x3 = −8
x1 − 3x2 + 7x3 = 17
diverges. Take the initial approximations as x1 = 0.9, x2 = −3.1, x3 = 0.9.
Interchange the first and second equations and solve the resulting system by
the Gauss-Seidel method. Again take the initial approximations as x1 =
0.9, x2 = 3.1, x3 = 0.9, and obtain the result correct to two decimal places.
The exact solution is x1 = 1.0, x2 = 3.0, x3 = 1.0.
Solution: Note that the system of equations is not diagonally dominant.
Gauss-Seidel method gives the iteration

(k+1) 1 (k) (k)

x1 = [23 + 6x2 − 2x3 )]
3
(k+1) (k+1) (k)
x2 = [−8 + 4x1 + x3 ]

57
 1
(k+1) (k+1) (k+1)
x3 = [17 − x1 + 3x2 ].
7
Starting with the initial approximations x1 = 0.9, x2 = −3.1, x3 = 0.9, we
obtain the following results.

First iteration
(1) 1 (0) (0) 1
x1 = [23 + 6x2 − 2x3 ] = [23 + 6(−3.1) − 2(0.9)] = 0.8667,
3 3
(1) (1) (0)
x2 = [−8 + 4x1 + x3 ] = [−8 + 4(0.8667) + 0.9] = −3.6332
(1) 1 (1) (1) 1
x3 = [17 − x1 + 3x2 ] = [17 − (0.8667) + 3(−3.6332)] = 0.7477
7 7
Second iteration
(2) 1 (1) (1) 1
x1 = [23 + 6x2 − 2x3 ] = [23 + 6(−3.6332) − 2(0.7477)] = −0.0982,
3 3
(2) (2) (1)
x2 = [−8 + 4x1 + x3 ] = [−8 + 4(−0.0982) + 0.7477] = −7.6451,
(2) 1 (2) (2) 1
x3 = [17 − x1 + 3x2 ] = [17 + 0.0982 + 3(−7.6451)] = −0.8339.
7 7
Third iteration
(3) 1 (2) (2) 1
x1 = [23 + 6x2 − 2x3 ] = [23 + 6(−7.6451) − 2(−0.8339)] = −7.0676,
3 3
(3) (3) (2)
x2 = [−8 + 4x1 + x3 ] = [−8 + 4(−7.0676) − 0.8339] = −37.1043,
(3) 1 (3) (3) 1
x3 = [17 − x1 + 3x2 ] = [17 + 7.0676 + 3(−37.1043)] = −12.4636.
7 7
It can be observed that the iterations are diverging very fast.
Now, we exchange the first and second equations to obtain the system

−4x1 + x2 − x3 = −8

3x1 − 6x2 + 2x3 = 23

x1 − 3x2 + 7x3 = 17.
The system of equations is now diagonally dominant. Gauss-Seidel method
gives iteration
(k+1) 1 (k) (k)
x1 = [8 + x2 − x3 ]
4
(k+1) (k+1) (k)
x2 = −[23 − 3x1 − 2x3 ]

58
 1
(k+1) (k+1) (k+1)
x3= [17 − x1 + 3x2 ]
7
Starting with the initial approximations x1 = 0.9, x2 = 3.1, x3 = 0.9, we
obtain the following results. First iteration
(1) 1 (0) (0) 1
x1 = [8 + x2 − x3 ] = [8 − 3.1 − 0.9] = 1.0,
4 4
(1) −1 (1) (0) −1
x2 = [23 − 3x1 2x3 ] = [23 − 3(1.0) − 2(0.9)] = −3.0333,
6 6
(1) 1 (1) (1) 1
x3 = [17 − x1 + 3x2 ] = [17 − 1.0 + 3(−3.0333)] = 0.9857
7 7
Second iteration
(2) 1 (1) (1) 1
x1 = [8 + x2 − x3 ] = [8 − 3.0333 − 0.9857] = 0.9953,
4 4
(2) 1 (2) (1) 1
x2 = − [23 − 3x1 − 2x3 ] = − [23 − 3(0.9953) − 2(0.9857)] = −3.0071,
6 6
(2) 1 (2) (2) 1
x3 = [17 − x1 + 3x2 ] = [17 − 0.9953 + 3(−3.0071)] = 0.9976.
7 7
Third iteration
(3) 1 (2) (2) 1
x1 = [8 + x2 − x3 ] = [8 − 3.0071 − 0.9976] = 0.9988,
4 4
(3) 1 (3) (2) 1
x2 = − [23 − 3x1 − 2x3 ] = − [23 − 3(0.9988) − 2(0.9976)] = −3.0014,
6 6
(3) 1 (3) (3) 1
x3 = [17 − x1 + 3x2 ] = [17 − 0.9988 + 3(−3.0014)] = 0.9996.
7 7
Fourth iteration
(4) 1 (3) (3) 1
x1 = [8 + x2 − x3 ] = [8 − 3.0014 − 0.9996] = 0.9998,
4 4
(4) 1 (4) (3) 1
x2 = − [23 − 3x1 − 2x3 ] = − [23 − 3(0.9998) − 2(0.9996)] = −3.0002,
6 6
(4) 1 (4) (4) 1
x3 = [17 − x1 + 3x2 ] = [17 − 0.9998 + 3(−3.0002)] = 0.9999.
7 7
We find
(4) (3)
|x1 − x1 | = |0.9998 − 0.9988| = 0.0010,
(4) (3)
|x2 − x2 | = | − 3.0002 + 3.0014| = 0.0012,
(4) (3)
|x3 − x3 | = |0.9999 − 0.9996| = 0.0003.
Since, all the errors in magnitude are less than 0.005, the required solution is
x1 = 0.9998, x2 = −3.0002, x3 = 0.9999. Rounding to two decimal places,
we get x1 = 1.0, x2 = −3.0, x3 = 1.0.

59
3.4 Solving System of non-linear Equations
using Newtons Method
Now we consider the solution of simultaneous non-linear equations by New-
tons method.
Consider the system
f (x, y) = 0
(3.12)
f (x, y) = 0
involving two non-linear equations. Let (x0 , y0 ) be an initial approximation
to the root of the system, and (x0 + h, y0 + k) be the root of the system given
by (3.12). Then we must have

f (x0 + h, y0 + k) = 0
(3.13)
g(x0 + h, y0 + k) = 0

Let us assume that f and g are differentiable expanding (3.13) by Taylors

series, we obtain
∂f ∂f
f (x0 + h, y0 + k) = f0 + h ∂x + k ∂y + ... = 0
∂g
0
∂g
0
(3.14)
g(x0 + h, y0 + k) = g0 + h ∂x0 + k ∂y0 + . . . = 0

neglecting, the second and higher order terms and retaining only the linear
terms of (3.14), we obtain
∂f ∂f
h ∂x + k ∂y = −f0
∂g
0
∂g
0
(3.15)
h ∂x0 + k ∂y0 = −g0
∂f
where f0 = f (x0 , y0 ), ∂x0
= ( ∂f )
∂x x=x0
∂f
; ∂y 0
= ( ∂f )
∂y y=y0
solving (3.14) for h and
k, the next approximation of the root is given by

x1 = x0 + h

and
y1 = y0 + k
The above process is repeated to desired degree of accuracy.
Example 3.4.1 Solve
x2 − y 2 = 4
and
x2 + y 2 = 16
By NewtonRaphson Method

60
Solution: To obtain the initial approximation we replace the first equation
by its asymptote y = x, which gives
√
2x2 = 16 ⇒ x = 2 2
√ √
let x0 = 2 2, y0 = 2 2, and (x0 , y0 ) be the initial approximation to the
root of the system.
We have f = x2 − y 2 − 4 ⇒ f0 = −4 and g = x2 + y 2 − 16 ⇒ g0 = 0
differentiating partially, we obtain
∂f ∂f
= 2x , = −2y
∂x ∂y
∂g ∂g
= 2x , = 2y
∂x ∂y
∂f
√ ∂f
√
so that ∂x0
= 2x0 = 4 2 , ∂y 0
= −4 2

∂g √ ∂g √
= 2x0 = 4 2 , = 2y0 = 4 2
∂x0 ∂y0
The system of linear equations can be written as
∂f ∂f √ √
h +k = −f0 ⇒ h(4 2) − k(4 2) = −(−4)
∂x0 ∂y0
⇒ h − k = 0.7072
and
∂g ∂g √ √
h +k = −g0 ⇒ h(4 2) + k(4 2) = 0
∂x0 ∂y0
⇒ h+k =0
so that
hk = 0.7072 (i)
h+k =0 (ii)
solving we get h = 0.3536, k = 0.3536
The second approximation to the root is given by
√
x1 = x0 + h = 2 2 + 0.3536 = 3.1820
√
y1 = y0 + k = 2 2 − 0.3536 = 2.4748
The process can be repeated.

61
Example 3.4.2 Solve

f (x, y) = x2 + y − 20x + 40 = 0

g(x, y) = x + y 2 − 20y + 20 = 0

Solution: Let x0 = 0, y0 = 0 be the initial approximation to the root

∂f ∂f
f = x2 + y − 20x + 40 ⇒ = 2x − 20 , =1
∂x ∂y
∂g ∂g
g = x + y 2 − 20y + 20 ⇒ =1 , = 2y − 20
∂x ∂y
and f0 = 40, g0 = 20

So that
∂f ∂f
= −20, =1
∂x0 ∂y0
∂g ∂g
=1 , = −20
∂x0 ∂y0
the linear equations are
∂f ∂f
h +k = −f0 ⇒ 20h + k = −40
∂x0 ∂y0
∂g ∂g
h +k = −g0 ⇒ h − 20k = −20
∂x0 ∂y0
Solving, we get

h = 2.055, k = 1.103
The next approximation is given by

x1 = x0 + h = 2.055
y1 = y0 + k = 1.103.

62
Chapter 4

Curve Fitting

Definition 4.1 Curve fitting is a procedure in which a mathematical formula

is used to best fit a given set of data points .

The function does not have to give the exact value at any single point ,but
fits the data will overall.

4.1 Least squares Method for Fitting

curve fitting with a linear equation is the method to find a linear function

y = a1 x + a0

that fits the data points overall. To determine a1 and a0 we use the definition
of best fit which is given by the following formula.i.e
n
X
E= ri2 = minimum
i=1

where
ri = [yi − a1 x − a0 ]

4.1.1 Linear least squares Regression

It is a procedure in which a1 and a0 of y = a1 x + a0 are determined such that
the function has the best fit is a given set of data points
Give set of n data points (xi , yi )
n
X
E= [yi − a1 x − a0 ]2 = minimum (4.1)
i=1

63
Figure 4.1: (a) Data exhibiting significant error. (b) Polynomial fit oscillating
beyond the range of the data. (c) More satisfactory result using the least-
squares fit.

Figure 4.2: The residual in linear regression represents the vertical distance
between a data point and the straight line.

where E is a function of a1 and a0 by extreme value theorem E has a min

64
value when
∂E
=0
∂a1
∂E
=0
∂a0
∂E X ∂
= (yi − a1 x − a0 )2 = 0
∂a1 ∂a1
X
⇒ 2(−xi )(yi − a1 xi − a0 ) = 0
X X X
⇒ a1 x2i + a0 xi = xi y i (i)
∂E
=0
∂a0
∂E X ∂
= (yi − a1 x − a0 )2 = 0
∂a0 ∂a0
X
⇒ 2(−1)(yi − a1 xi − a0 ) = 0
X X X
⇒ a1 x i + a0 1= yi
X X
⇒ a1 xi + na0 = yi (ii)
Using (i) and (ii) we have a system of linear equation in a1 and a0
P 2 P P
a1 xP i + a0 xi =P xi yi
(4.2)
a1 xi + na0 = yi
 P 2 P    P 
x i x i a 1 x i y i
P = P (4.3)
xi n a0 yi

Example 4.1.1 Given the following data

x -1 0 1 2 3 4 5 6
y 10 9 7 5 4 3 0 -1

i. Using linear least squares regression to determine the function

y = ax + b

that best fit the data .

ii. Using equation (4.1) determine the over all error

65
 x y xy x2
-1 10 -10 1
0 9 0 0
1 7 7 1
2 5 10 2
3 4 12 9
4 3 12 16
5 0 0 25
6 -1 -1 36
20 37 25 92 sum
Then  P 2 P    P 
x i x i a x i yi
P = P (4.4)
xi n b yi
    
92 20 a 25
⇒ = (4.5)
20 8 b 37
Using Cramer’s Rule

25 20
37 8
a= = −1.607
92 20
20 8
92 25
20 37
b= = 8.643
92 20
20 8
Therefore y = ax + b
⇒ y = −1.607x + 8.643

4.1.2 power least square regression

Given
y = axm
⇒ e = y − axm
e2 is minimum i.e.
P
Suppose
X
E= (yi − axm )2

66
is minimum by extreme value theorem.
∂E
=0
∂a
X ∂
⇒ (yi − axm 2
i ) = 0
∂a
X
⇒ 2(−xm m
i )(yi − axi ) = 0
X X
⇒a x2m
i = xmi yi
P m
x yi
⇒ a = P i2m
xi
Example 4.1.2 Given the following data ,that experimentally represents the
relation d = 12 gt2 where d, dependent variable and t ,independent variable g,
constant.Find the value of g

t 0.2 0.4 0.6 0.8 1.0

d 0.1960 0.7850 1.7665 3.1405 4.9075

solution
Let a = 12 g ,y = d and x = t

⇒ y = ax2 herem = 2

then P 2
x yi
a = P i4
xi
xi yi x2i x2i yi x4i
0.2 0.1960 0.4 0.0078 0.0016
0.4 0.7850 0.16 0.1256 0.256
0.6 1.7665 0.36 0.6359 0.1296
0.8 3.1405 0.64 2.0099 0.4096
1.0 4.9075 1.0 4.9075 1.0000
2.56 7.6863 1.5664 sum
P 2
x yi 7.6863
a = P i4 = = 4.9070
xi 1.5664
But a = 12 g ⇒ g = 2a = 2(4.9070) = 9.8140

67
4.1.3 Polynomial least square regression
consider a second degree polynomial

y = a2 x 2 + a1 x + a0

⇒ e = y − a2 x 2 − a1 x − a0
X
E= e2
X
⇒E= (y − a2 x2 − a1 x − a0 )2 minimum
∂E
=0
∂a2
X ∂
⇒ (y − a2 x2 − a1 x − a0 )2 = 0
∂a2
X
⇒ 2(−x2 )(y − a2 x2 − a1 x − a0 ) = 0
X X X X
⇒ a2 x4i + a1 x3i + a0 x2i = xi y i (i)
∂E
=0
∂a1
X X X X
⇒ a2 x3i + a1 x2i + a0 xi = xi y i (ii)
∂E
=0
∂a0
X X X
⇒ a2 x2i + a1 x i + a0 n = yi (iii)
combining the three (i.e (i),(ii),(iii) ) equation we have a system of three
equations in a2 , a1 , and a0 .
 P 4 P 3 P 2    P 2 
x i x
P 3 P 2 P i x i a 2 P xi y i
P x2i P xi xi a1 = Pxi yi
     
xi xi n a0 yi

Example 4.1.3 Determine the coefficient s of the polynomial y = a2 x2 +

a1 x + a0 that best fit the data given below

x 1 1.5 2 2.5
i.
y 3.8 4.9 5.2 5.4

x -3 0 2 4
ii.
y 3 1 1 3

68
4.2 Interpolation
Introduction:

In this section, we discuss the problem of approximating a given func-

tion by polynomials. There are two main uses of these approximating poly-
nomials. The first use is to reconstruct the function f (x) when it is not
given explicitly and only values of f (x) and or its certain order derivatives
are given at a set of distinct points called nodes or tabular points. The
second use is to perform the required operations which were intended for
f (x), like determination of roots, differentiation and integration etc. can
be carried out using the approximating polynomial P (x). The approximat-
ing polynomial P (x) can be used to predict the value of f (x) at a non
tabular point. The deviation of P (x) from f (x), that is f (x) − P (x), is
called the error of approximation. Let f (x) be a continuous function defined
on some interval [a, b], and be prescribed at n + 1 distinct tabular points
x0 , x1 , . . . , xn such that a = x0 < x1 < x2 < ... < xn = b. The distinct
tabular points x0 , x1 , ..., xn may be non-equispaced or equispaced, that is
xk+1 − xk = h, k = 0, 1, 2, . . . , n − 1. The problem of polynomial approxima-
tion is to find a polynomial Pn (x), of degree ≤ n, which fits the given data
exactly, that is,
Pn (xi ) = f (xi ), i = 0, 1, 2, . . . , n. (4.6)
The polynomial Pn (x) is called the interpolating polynomial. The conditions
given in (4.6) are called the interpolating conditions

Remark 4.1 Through two distinct points, we can construct a unique poly-
nomial of degree 1 (straight line). Through three distinct points, we can
construct a unique polynomial of degree 2(parabola) or a unique polynomial
of degree 1 (straight line). That is, through three distinct points, we can
construct a unique polynomial of degree ≤ 2.
In general, through n + 1 distinct points, we can construct a unique poly-
nomial of degree ≤ n. The interpolation polynomial fitting a given data is
unique. We may express it in various forms but are otherwise the same
polynomial

Finite Differences
Introduction

69
 Numerical Analysis is a branch of mathematics which leads to approx-
imate solution by repeated application of four basic operations of Algebra.
The knowledge of finite differences is essential for the study of Numerical
Analysis. In this section we introduce few basic operators.

Shift Operator
Let y = f (x) be function of x and x, x + h, x + 2h, x + 3h, . . . , etc., be the
consecutive values of x, then the operator E is defined as
Ef (x) = f (x + h),
E is called shift operator. It is also called displacement operator.
Note: E is only a symbol but not an algebraic sum. E 2 f (x) means the
operator E is applied twice on f (x), i.e.,
E 2 f (x) = E[Ef (x)]
= Ef (x + h)
= f (x + 2h)
Similarly
E n f (x) = f (x + nh)
and
E −n f (x) = f (x − nh).
The operator E has the following properties:
1. E(f1 (x) + f2 (x) + + fn (x)) = Ef1 (x) + Ef2 (x) + + Efn (x)
2. E(cf (x)) = cEf (x) (where c is constant)
3. E m (E n f (x)) = E n (E m f (x)) = E m+n f (x) where m, n are positive
integers
4. If n is positive integer E n [E −n f (x)] = f (x)
Alternative notation If y0 , y1 , y2 , . . . , yn , . . . , etc are consecutive values of
the function y = f (x) corresponding to equally spaced values x0 , x1 , x2 , . . . , xn ,
etc., of x then in alternative notation
Ey0 = y1
Ey1 = y2
...
2
E y0 = y2
...
n
and in general E y0 = yn .

70
Forward Difference Operator
Let y = f (x) be any function given by the values y0 , y1 , y2 , . . . , yn , which it
takes for the equidistant values x0 , x1 , x2 , . . . , xn , of the independent variable
x, then y1 − y0 , y2 − y1 , . . . , yn − yn1 are called the first differences of the
function y. They are denoted by y0 , y1 , . . . , etc. We have

∆y0 = y1 − y0

∆y1 = y2 − y1
...
∆yn−1 = yn − yn−1
The symbol ∆ is called the difference operator. The differences of the first
differences denoted by ∆y and ∆2 y0 , ∆2 y1 , . . . , ∆2 yn are called second differ-
ences, where
∆2 y0 = ∆[∆y0 ]
= ∆[y1 − y0 ]
= ∆y1 − ∆y0
= y2 − 2y1 + y0
Similarly
∆r yk = ∆r−1 yk+1 − ∆r−1 yk
Alternative Notation
Let the functions y = f (x) be given at equal spaces of the independent
variable x, say at x = a, a + h, a + 2h, . . . , etc., and the corresponding values
of f (a), f (a + h), f (a + 2h), . . . , etc. The independent variable x is often
called the argument and the corresponding value of the dependent variable
is of the function at x = a, and is denoted by ∆f (a). Thus we have

∆f (a) = f (a + h) − f (a),

writing the above definition we can write

∆f (a + h) = f (a + h + h) − f (a + h) = f (a + 2h) − f (a + h)
Similarly

∆2 f (a) = ∆(∆f (a)) = ∆(f (a + h) − f (a)) = f (a + 2h) − 2f (a + h) + f (a)

Note: The operator ∆ is called forward difference operator and in general it

is defined as
∆f (x) = f (x + h) − f (x),

71
where h is called the interval of differencing.

Difference Table
It is a convenient method for displaying the successive differences of a func-
tion. The following table is an example to show how the differences are
formed.
x y ∆y ∆2 y ∆3 y ∆4 y ∆5 y
x0 y 0
∆y0
x1 y 1 ∆2 y0
∆y1 ∆3 y0
x2 y 2 ∆2 y1 ∆4 y
3
∆y2 ∆ y1 ∆5 y0
x3 y 3 ∆2 y2 ∆4 y1
∆y3 ∆3 y1
2
x4 y 4 ∆ y3
∆y4
x5 y 5
The above table is called a diagonal difference table. The first term in the ta-
ble is y0 . It is called the leading term. The differences ∆y0 , ∆2 y0 , ∆3 y0 , . . . ∆n y,
are called the leading differences. The differences ∆n yn with a fixed subscript
are called forward differences. In forming such a difference table care must
be taken to maintain correct sign.
Properties of the Operator

1. If c is a constant then ∆c = 0.

2. ∆ is distributive, i.e., ∆[f (x) ± g(x)] = ∆f (x) ± ∆g(x).

3. If c is a constant then ∆cf (x) = c∆f (x).

4. If m and n are positive integers then ∆m ∆n f (x) = ∆m+n f (x).

5. ∆[f1 (x) + f2 (x) + . . . + fn (x)] = ∆f1 (x) + ∆f2 (x) + . . . + ∆fn (x).

6. ∆f (x)g(x) = f (x)∆g(x) + g(x)∆f (x).

7. ∆[ fg(x)
(x)
]= g(x)∆f (x)−f (x)∆g(x)
g(x)g(x+h)
.

Example 4.2.1 Construct a difference table for y = f (x) = x3 + 2x + 1 for

x = 1, 2, 3, 4, 5.

72
Solution:
x y ∆y ∆2 y ∆3 y
1 4
9
2 13 12
21 6
3 34 18
39 6
4 73 24
63
5 136

Theorem 1

: The nth forward differences of a polynomial of the nth degree are

constant when the values of independent variable are at equal intervals.
i.e ∆f (x) = n!hn where f (x) is nth degree polynomial.

Proof Exercise
Example 4.2.2 By constructing a forward difference table and taking the
second order forward differences as constant find the sixth term of the series
8, 12, 19, 29, 42, . . . .
Solution: Let K be the sixth term of the series. The difference table is
x y ∆ ∆2
1 8
4
1 12 3
7
3 19 3
10
4 29 3
13
5 42 k − 55
k−4
6 k
The second differences are constant.
⇒ K − 55 = 3

73
 ⇒ K = 58.
The sixth term of the series is 58.

Backward Differences
Let y = f (x) be a function given by the values y0 , y1 , . . . yn which it takes
for the equally spaced values x0 , x1 , . . . , xn of the independent variable x.
Then y1 − y0 , y2 − y1 , . . . , yn − yn−1 are called the first backward differences
of y = f (x). They are denoted by 5y0 , 5y1 , . . . , 5yn respectively. Thus we
have
y1 − y0 = 5y1
y2 − y1 = 5y2
..
.
yn − yn−1 = 5yn
where 5 is called the backward difference operator.

x y 5 52 y 53 y 54 y
x0 y0
5y1
x1 y1 52 y 2
5y2 53 y 3
2
x2 y2 5 y3 54 y4
5y3 53 y 4
2
x3 y3 5 y4
5y4
x4 y4

Note: In the above table the differences 5n y with a fixed subscript i, lie
along the diagonal upward sloping.
Alternative notation: Let the function y = f (x) be given at equal spaces
of the independent variable x at x = a, a + h, a + 2h, . . . then we define
5f (a) = f (a) − f (a − h) where 5 is called the backward difference operator,
h is called the interval of differencing.
In general we can define

5f (x) = f (x) − f (x − h).

We observe that

5f (x + h) = f (x + h) − f (x) = ∆f (x)

74
 5f (x + 2h) = f (x + 2h) − f (x + h) = ∆f (x + h)
...
5f (x + nh) = f (x + nh) − f (x + (n − 1)h) = ∆f (x + (n − 1)h)
Similarly we get

52 f (x + 2h) = 5[5f (x + 2h)]

= 5[∆f (x + h) = ∆[∆f (x)] = ∆2 f (x)

...
5 f (x + nh) = ∆n f (x).
n

Relation between E and 5 :

5f (x) = f (x) − f (x − h) = f (x) − E −1 f (x)

⇒ 5 = 1 − E −1
or
E−1
5=
E

4.2.1 Interpolation with equally spaced intervals

Newton Interpolation Formula(forward and backward difference
formulas)
Newton’s Forward Interpolation Formula

Let y = f (x) be a function which takes the values y0 , y1 , y2 , . . . , yn corre-

sponding to the (n + 1) values x0 , x1 , x2 , . . . , xn of the independent variable
x. Let the values x be equally spaced, i.e.,

xr = x0 + rh, r = 0, 1, 2, . . . ,

where h is the interval of differencing. Let φ(x) be a polynomial of the nth

degree in x taking the same values as y corresponding to x = x0 , x1 , . . . , xn ,
then, φ(x) represents the continuous function y = f (x) such that f (xr ) =
φ(xr ) for r = 0, 1, 2, . . . , n and at all other points f (x) = φ(x) + R(x) where
R(x) is called the error term (Remainder term) of the interpolation formula.
Ignoring the error term let us assume

f (x) ≈ φ(x) ≈ a0 +a1 (x−x0 )+a1 (x−x0 )(x−x1 ) . . . an (x−x0 )(x−x1 ) . . . (x−xn−1 )
(4.7)

75
 the constants a0 , a1 , a2 , . . . , an can be determine as follows. Putting x =
x0 in (4.7) we get
f (x0 ) ≈ φ(x0 ) = a0
⇒ y0 = a0
putting x = x1 in (4.7) we get
f (x1 ) ≈ φ(x1 ) = a0 + a1 (x1 − x0 ) = y0 + a1 h
∴ y 1 = y 0 + a1 h
y1 − y0 ∆y0
⇒ a1 = =
h h
Putting x = x2 in (4.7) we get
f (x2 ) ≈ φ(x2 ) = a0 + a1 (x2 − x0 ) + a2 (x2 − x0 )(x2 − x1 )
∆y0
∴ y2 = a0 + a1 (2h) + a2 (2h)(h) = y0 + (2h) + a2 (2h)(h)
h
⇒ y2 = y0 + 2(y1 − y0 ) + a2 (2h2 )
y2 − 2y1 + y0 ∆2 y0
⇒ a2 = =
2h2 2!h2
Similarly by putting x = x3 , x = x4 , . . . , x = xn in (4.7) we get
∆3 y0 ∆4 y0 ∆n y0
a3 = , a 4 = , . . . , a n =
3!h3 4!h4 n!hn
putting the values of a0 , a1 , . . . , an in (4.7) we get
∆y0 ∆2 y0 ∆n y0
f (x) ≈ φ(x) = y0 + (x−x0 )+ (x−x 0 )(x−x 1 )+· · ·+ (x−x0 )(x−x1 ) . . . (x−xn−1 )
h 2!h2 n!hn
(4.8)

x−x0
Writing p = h
, we get x − x0 = ph
x − x1 = x − x0 − (x1 − x0 ) = ph − h = (p − 1)h
x − x2 = x − x1 + x1 − x2 = (x − x1 ) − (x2 − x1 ) = (p − 1)h − h = (p − 2)h
Similarly
x − xn−1 = (p − n + 1)h
Equation (4.1) can be written as
p p(p − 1) 2 p(p − 1)(p − 2) . . . (p − n + 1) n
φ(x) = y0 + ∆y0 + ∆ y0 +· · ·+ ∆ y0
1! 2! n!
The above formula is called Newtons forward interpolation formula.
Note:

76
 1. Newton forward interpolation formula is used to interpolate the values
of y near the beginning of a set of tabular values.

2. y0 may be taken as any point of the table, but the formula contains
only those values of y which come after the value chosen as y0 .

Example 4.2.3 Evaluate y = e2x for x = 0.05 using the following table

x 0.00 0.10 0.20 0.30 0.40

2x
y=e 1.000 1.2214 1.4918 1.8221 2.255

Solution The difference table is

x y = e2x ∆y ∆2 y ∆3 y ∆4 y
0.000 1.0000
0.2214
0.10 1.2214 0.0490
0.2704 0.0109
0.20 1.4918 0.0599 0.0023
0.3303 0.0132
0.30 1.8221 0.0731
0.4034
0.4 2.2255

We have x0 = 0.00, x = 0.05, h = 0.1.

x − x0 0.5 − 0.00
∴p= = = 0.5
h 0.1
Using Newtons forward formula

p p(p − 1) 2 p(p − 1)(p − 2) 3 p(p − 1)(p − 2)(p − 3) 4

f (x) = y0 + ∆y0 + ∆ y0 + ∆ y0 + ∆ y0
1! 2! 3! 4!
0.5(0.5 − 1) 0.5(0.5 − 1)(0.5 − 2)
f (0.05) = 1.0000+0.5×0.2214+ (0.0490)+ (0.0109)
2 6
0.5(0.5 − 1)(0.5 − 2)(0.5 − 3)
+ (0.0023)
24
= 1.000 + 0.1107 − 0.006125 + 0.000681 − 0.000090 = 1.105166
∴ f (0.05) ≈ 1.052.

77
Exercises 4.1 In an examination the number of candidates who obtained
marks between certain limits were as follows:
M arks 30 − 40 40 − 50 50 − 60 60 − 70 70 − 80
N o.of Students 31 42 51 35 31
Find the number of candidates whose scores lie between 45 and 50.
Solution: Fist of all we construct a cumulative frequency table for the given
data.

U pper limits of the class intervals 40 50 60 70 80

Cumulative f requency 31 73 124 159 190
let x = U pper limits of the class intervals(marks) and y = Cumulative f requency
The difference table is
x y ∆y ∆2 y ∆3 y ∆4 y
40 31
42
50 73 9
51 −25
60 124 −16 37
35 12
70 159 −4
31
80 190
we have x0 = 40, x = 45, h = 10
x − x0 45 − 40
p= = = 0.5
h 10
and
y0 = 73, ∆y0 = 42, ∆2 y0 = 9, ∆3 y0 = −25, ∆4 y0 = 37
From Newtons forward interpolation formula
p p(p − 1) 2 p(p − 1)(p − 2) 3 p(p − 1)(p − 2)(p − 3) 4
f (x) = y0 + ∆y0 + ∆ y0 + ∆ y0 + ∆ y0
1! 2! 3! 4!
0.5(0.5 − 1) 0.5(0.5 − 1)(0.5 − 2) 0.5(0.5 − 1)(0.5 − 2)(0.5 − 3)
= 73+0.5(42)+ (9)+ (−25)+ (37)
2! 3! 4!
= 31 + 21 − 1.125 − 1.5625 − 1.4452 = 47.8673
= 48 (approximately)
The number of students who obtained marks less than 45 = 48, and the
number of students who scored marks between 45 and 50 = 73 - 48 = 25.

78
Exercises 4.2 A second degree polynomial passes through the points (1, -1),
(2, -1), (3, 1), (4, 5). Find the polynomial.

Newton’s Backward Interpolation Formula

Newtons forward interpolation formula cannot be used for interpolating a

value of y near the end of a table of values. For this purpose, we use another
formula known as Newtonbackward interpolation formula. It can be derived
as follows. Let y = f (x) be a function which takes the values y0 , y1 , y2 , . . . , yn
corresponding to the values x0 , x1 , x2 , . . . , xn of the independent variable x.
Let the values of x be equally spaced with h as the interval of differencing,
i.e.,
Let xr = x0 + rh, r = 0, 1, 2, . . . , n
Let φ(x) be a polynomial of the nth degree in x taking the same values
as y corresponding to x = x0 , x1 , . . . , xn , i.e., φ(x) represents y = f (x) such
that f (xr ) = φ(xr ), r = 0, 1, 2, . . . , we may write φ(x) as

f (x) ≈ φ(x) = a0 +a1 (x−xn )+a2 (x−xn )(x−xn−1 )+· · ·+an (x−xn )(x−xn−1 ) . . . (x−x1 )
(4.9)
Putting x = xn is (4.9) we get

f (xn ) ≈ φ(xn ) = a0 .

⇒ y n = a0 .
Putting x = xn−1 in (4.9) we get

f (xn−1 ≈ φ(x) = a0 + a1 (xn−1 − xn )

⇒ yn−1 = yn + a1 (−h)
⇒ a1 h = yn − yn−1
yn − yn−1 5yn
⇒ a1 = =
h 1!h
Putting x = xn−2 , we get

f (xn−2) ≈ φ(xn−2 ) = a0 + a1 (xn−2 − xn ) + a2 (xn−2 − xn )(xn−2 − xn−1 )

yn − yn−1
⇒ yn−2 = yn + (−2h) + a2 (−2h)(−h)
h
⇒ yn−2 = yn − 2yn + 2yn−1 + 2(h)2 a2
yn − 2yn−1 + yn−2 52 yn
⇒ a2 = =
2h2 2!h2
79
similarly putting x = xn−3 , x = xn−4 , x = xn−5 , . . . we get
53 yn 54 yn 5n y n
a3 = , a 4 = , . . . , a n =
3!h3 4!h4 n!hn
substituting these values in (4.9)
5yn 52 y n 5n yn
f (x) ≈ φ(x) = yn + (x−xn )+ (x−x n )(x−x n−1 )+. . .+ (x−xn )(x−xn−1 ) . . . (x−x1 )
1!h 2!h2 n!hn
(4.10)
x−xn
writing p = h
we get x − xn = ph
∴ x − xn−1 = x − xn + xn − xn−1 = ph + h = (p + 1)h
⇒ x − xn−2 = (p + 2)h, . . . , (x − x1 ) = (p + n − 1)h
∴ The equation (4.10) may be written as
5yn P (p + 1) 2 p(p + 1)(p + 2) 3
f (x) ≈ φ(x) = yn + p + 5 yn + 5 yn
1! 2! 3!
p(p + 1)(p + 2) . . . (p + n − 1) n
+··· + 5 yn
n!
The above formula is known as Newtons backward interpolation formula.
Example 4.2.4 The following data gives the melting point of an alloy of
lead and zinc, where t is the temperature in degrees c and P is the percentage
of lead in the alloy. P is the percentage of lead in the alloy.
P 40 50 60 70 80 90
t 180 204 226 250 276 304
Find the melting point of the alloy containing 84 percent lead.
Solution The value of 84 is near the end of the table, therefore we use the
Newtons backward interpolation formula. The difference table is
x y 5y 52 y 53 y 54 y 55 y
40 184
20
50 204 2
22 0
60 226 2 0
24 0 0
70 250 2 0
26 0
80 276 2
28
90 304

80
We have xn = 90, x = 84, h = 10 , tn = yn = 304, 5 tn = 5yn =
28, 52 yn = 2
53 yn = 54 yn = 55 yn = 0
x − xn 84 − 90
p= = = −0.6
h 10
From Newtons backward formula

5tn P (p + 1) 2
f (84) = tn + p + 5 tn + . . .
1! 2!
(−0.6)(−0.6 + 1)
f (84) = 304 + (−0.6)(28) + (2)
2!
= 304 − 16.8 − 0.24 = 286.96.

Exercises 4.3 Calculate the value of f (7.5) using table below

x 1 2 3 4 5 6 7 8
f (x) 1 8 27 64 125 216 343 512

4.2.2 Interpolation with unequal interval

Linear Interpolation
Given two points (x0 , y0 ) and (x1 , y1 ), the linear polynomial passing through
the two points is the equation of the line passing through the points. One
way to write its formula is
x1 − x x − x0
P1 (x) = y0 + y1
x1 − x0 x 1 − x0
Example 4.2.5 For the data points (2, 3) and (5, 7) find P1 (x).

Solution: let x0 = 2, y0 = 3 , x1 = 5 , y1 = 7
x1 − x x − x0 2−x x−2 7
P1 (x) = y0 + y1 = 3( ) + 7( ) = (5 − x) + (x − 2)
x1 − x0 x1 − x0 5−2 5−2 3
In general, if y0 = f (x0 ) and y1 = f (x1 ) for some function f , then
−x
P1 (x) = f (x0 )( xx11−x0
) + f (x1 )( xx−x 0
1 −x0
) is a linear approximation of f (x) for all
x ∈ [x0 , x1 ].

81
Lagrange’s Interpolation formula
Let the data

x x0 x1 x2 ... xn
(4.11)
f (x) f (x0 ) f (x1 ) f (x2 ) . . . f (xn )

be given at distinct unevenly spaced points or non-uniform points x0 , x1 , . . . , xn.

This data may also be given at evenly spaced points. For this data, we can
fit a unique polynomial of degree ≤ n. Since the interpolating polynomial
must use all the ordinates f (x0 ), f (x1 ), . . . , f (xn ), it can be written as a lin-
ear combination of these ordinates. That is, we can write the polynomial
as
Pn (x) = l0 (x)f (x0 ) + l1 (x)f (x1 ) + ... + ln (x)f (xn )
(4.12)
= l0 (x)f0 + l1 (x)f1 + ... + ln (x)fn
where f (xi ) = fi and li (x), i = 0, 1, 2, . . . , n are polynomials of degree n.
This polynomial fits the data given in (4.11) exactly.
At x = x0 , we get

f (x0 ) = Pn (x0 ) = l0 (x0 )f (x0 ) + l1 (x0 )f (x1 ) + . . . + ln (x0 )f (xn ).

This equation is satisfied only when l0 (x0 ) = 1 and li (x0 ) = 0, i 6= 0. At a gen-

eral point x = xi , we get f (xi ) = Pn (xi ) = l0 (xi )f (x0 )+. . .+li (xi )f (xi )+. . .+
ln (xi )f (xn ). This equation is satisfied only when li (xi ) = 1 and lj (xi ) = 0, ij.
Therefore, li (x), which are polynomials of degree n, satisfy the conditions

0 if i 6= j
li (xj ) = (4.13)
1 if i = j

Since, li (x) = 0 at x = x0 , x1 , . . . , xi−1 , xi+1 , . . . , xn , we know that

(x − x0 ), (x − x1 ), . . . , (x − xi−1 ), (x − xi+1 ), . . . , (x − xn )

are factors of li (x). The product of these factors is a polynomial of degree n.

Therefore, we can write

li (x) = C(x − x0 )(x − x1 )...(x − xi−1 )(x − xi+1 )...(x − xn )

where C is a constant.
Now, since li (xi ) = 1, we get

li (xi ) = 1 = C(xi − x0 )(xi − x1 )...(xi − xi−1 )(xi − xi+1 )...(xi − xn )

82
Hence
1
C=
(xi − x0 )(xi − x1 )...(xi − xi−1 )(xi − xi+1 )...(xi − xn )
Therefore,
(x − x0 )(x − x1 )...(x − xi−1 )(x − xi+1 )...(x − xn )
li (x) = (4.14)
(xi − x0 )(xi − x1 )...(xi − xi−1 )(xi − xi+1 )...(xi − xn )
Note that the denominator on the right hand side of li (x) is obtained by set-
ting x = xi in the numerator. The polynomial given in (4.12) where li (x) are
defined by (4.13) is called the Lagrange interpolating polynomial and li (x)
are called the Lagrange fundamental polynomials.
We can write the Lagrange fundamental polynomials li (x) in a simple nota-
tion.
Denote
w(x) = (x − x0 )(x − x1 ) . . . (x − xn )
which is the product of all factors. Differentiating w(x) with respect to x
and substituting x = xi we get

w0 (xi ) = (xi − x0 )(xi − x1 ) . . . (xi − xi−1 )(xi − xi+1 ) . . . (xi − xn )

since all other terms vanish. Therefore, we can also write li (x) as
w(x)
li (x) = .
(x − xi )(w0 (x)
Linear interpolation

For n = 1, we have the data

x x0 x1
f (x) f (x0 ) f (x1 )

The Lagrange fundamental polynomials are given by

x − x1
l0 (x) = ,
x0 − x1
x − x0
l1 (x) =
x 1 − x0
The Lagrange linear interpolation polynomial is given by

P1 (x) = l0 (x)f (x0 ) + l1 (x)f (x1 ).

83
Quadratic interpolation

For n = 2, we have the data

x x0 x1 x2
f (x) f (x0 ) f (x1 ) f (x2 )

Figure 4.3: A visual depiction of the rationale behind the Lagrange polyno-
mial. This figure shows a second-order case. Each of the three terms in Eq.
(4.14) passes through one of the data points and is zero at the other two. The
summation of the three terms must, therefore, be the unique second-order
polynomial p1 (x) that passes exactly through the three points.

The Lagrange fundamental polynomials are given by

(x − x1 )(x − x2 )
l0 (x) = ,
(x0 − x1 )(x0 − x2 )

(x − x0 )(x − x2 )
l1 (x) =
(x1 − x0 )(x1 − x2
(x − x0 )(x − x1 )
l2 (x) =
x2 − x0 )(x2 − x1 )

84
The Lagrange quadratic interpolation polynomial is given by

P1 (x) = l0 (x)f (x0 ) + l1 (x)f (x1 ) + l2 (x)f (x2 ). (4.15)

Example 4.2.6 Use Lagranges formula, to find the quadratic polynomial

that takes the values

x 0 1 3
y 0 1 0

Solution Since f0 = 0 and f2 = 0, we need to compute l1 (x) only. We have

(x − x0 )(x − x2 ) x(x − 3) 1
l1 (x) = = = (3x − x2 )
(x1 − x0 )(x1 − x2 ) (1)(−2) 2

The Lagrange quadratic polynomial is given by

1 1
p2 (x) = l1 (x)f (x1 ) = (3x − x2 )(1) = (3x − x2 )
2 2
Example 4.2.7 Construct the Lagrange interpolation polynomial for the data

x −1 1 4 7
f (x) −2 0 63 342
Hence, interpolate at x = 5.

Solution The Lagrange fundamental polynomials are given by

(x − x1 )(x − x2 )(x − x3 ) (x − 1)(x − 4)(x − 7)

l0 (x) = =
(x0 − x1 )(x0 − x2 )(x0 − x3 ) (−1 − 1)(−1 − 4)(−1 − 7)
1 3
=− (x − 12x2 + 39x − 28).
80
(x − x0 )(x − x2 )(x − x3 ) (x + 1)(x − 4)(x − 7)
l1 (x) = =
(x1 − x0 )(x1 − x2 )(x1 − x3 ) (1 + 1)(1 − 4)(1 − 7)
1 3
= (x − 10x2 + 17x + 28).
36
(x − x0 )(x − x1 )(x − x3 ) (x + 1)(x − 1)(x − 7)
l2 (x) = =
(x2 − x0 )(x2 − x1 )(x2 − x3 ) (4 + 1)(4 − 1)(4 − 7)
1 3
=− (x − 7x2 − x + 7).
45
85
 (x − x0 )(x − x1 )(x − x2 ) (x + 1)(x − 1)(x − 4)
l3 (x) = =
(x3 − x0 )(x3 − x1 )(x3 − x2 ) (7 + 1)(7 − 1)(7 − 4)
1
= (x3 − 4x2 − x + 4).
144
Note that we need not compute l1 (x) since f (x1 ) = 0. The Lagrange inter-
polation polynomial is given by
P3 (x) = l0 (x)f (x0 ) + l1 (x)f (x1 ) + l2 (x)f (x2 ) + l3 (x)f (x3 )
1 3 1 1
=− (x −12x2 +39x−28)(−2)− (x3 −7x2 −x+7)(63)+ (x3 −4x2 −x+4)(342)
80 45 144
= x3 − 1.
Hence, f (5) ≈ P3 (5) = 53 − 1 = 124
Exercises 4.4 Using Lagranges interpolation formula find a polynomial which
passes the points (0, 12), (1, 0), (3, 6), (4, 12). Hence find the value of y at
x = 6.

Newton’s Divided Difference Interpolation

Divided Difference
Lagrange formula has the drawback that if another interpolation value were
inserted ,then the interpolation coefficients are required to be recalculated.This
labour of recomputing the interpolation coefficients is saved by using Newtons
general interpolation formula which employs what are called divided differ-
ences.Before deriving this formula we shall first define these differences. If
(x0 , y0 ), (x1 , y1 ), (x2 , y2 ), . . . be given points ,then the first divided difference
for the arguments x0 , x1 is defined by the relation
y1 − y0
[x0 , x1 ] =
x1 − x0
Similarly
y2 − y1 y3 − y2
[x1 , x2 ] = and [x2 , x3 ] = etc.
x2 − x1 x3 − x2
The second divided difference for x0 , x1 , x2 is defined as
[x1 , x2 ] − [x0 , x1 ]
[x0 , x1 , x2 ] = .
x2 − x0
The third divided difference for x0 , x1 , x2 , x3 is defined as
[x1 , x2 , x3 ] − [x0 , x1 , x2 ]
[x0 , x1 , x2 , x3 ] = and so on.
x3 − x0

86
Remark 4.2 i. The divided difference are symmetrical in their argu-
ments,i.e. independent of the order of the arguments.For it easy to
write
y1 − y0 y0 − y1
[x0 , x1 ] = = = [x1 , x0 ]
x1 − x0 x0 − x1
Similarly
[x0 , x1 , x2 ] = [x1 , x2 , x0 ]or [x2 , x1 , x0 ]

ii. The nth divided differences of a polynomial of the nth degree are con-
stant.

proof:
Let the arguments be equal spaced i.e.

x1 − x0 = x2 − x1 = . . . = xn − xn−1 = h

Then
y1 − y0 ∆y0
[x0 , x1 ] = =
x 1 − x0 h
[x1 , x2 ] − [x0 , x1 ]
[x0 , x1 , x2 ] =
x2 − x0
1 ∆y1 ∆y0 1
= [ − ]= ∆2 y0
2h h h 2!h2
and in general
1
∆n y0
[x0 , x1 , . . . , xn ] =
n!hn
If the tabulated function is nth degree polynomial,then ∆n y0 will be constant
.Hence ,then nth divided differences will also be constant.

Newton’s Divided Difference Formula

Let y0 , y1 , . . . , yn values of y = f (x) corresponding to the arguments x0 , x1 , x2 , . . . , xn .
Then from the definition of divided differences,we have
y − y0
[x, x0 ] =
x − x0
⇒ y = y0 + (x − x0 )[x, x0 ] (4.16)
[x, x0 ] − [x0 , x1 ]
Again [x, x0 , x1 ] =
x − x1
⇒ [x, x0 ] = [x0 , x1 ] + (x − x1 )[x, x0 , x1 ]

87
substituting this values of [x, x0 ] in (4.16) we get

y = y0 + (x − x0 )[x0 , x1 ] + (x − x0 )(x − x1 )[x, x0 , x1 ] (4.17)

[x, x0 , x1 ] − [x0 , x1 , x2 ]
Also [x, x0 , x1 , x2 ] =
x − x2
⇒ [x, x0 , x1 ] = [x0 , x1 , x2 ] + (x − x2 )[x, x0 , x1 , x2 ]
Substituting this value of [x, x0 , x1 ] in (4.17). we obtain

y = y0 + (x − x0 )[x0 , x1 ] + (x − x0 )(x − x1 )[x0 , x1 , x2 ]

+(x − x0 )(x − x1 )(x − x2 )[x, x0 , x1 , x2 ]

Proceeding in this manner ,we get
y = y0 + (x − x0 )[x0 , x1 ] + (x − x0 )(x − x1 )[x0 , x1 , x2 ]
+(x − x0 )(x − x1 )(x − x2 )[x0 , x1 , x2 , x3 ]
(4.18)
+(x − x0 )(x − x1 )(x − x2 )(x − x3 )[x0 , x1 , x2 , x3 , x4 ]
+ . . . + (x − x0 )(x − x1 ) . . . (x − xn−1 )[x, x0 , x1 , x2 , . . . , xn ]

Example 4.2.8 Given the values

x 5 7 11 13 17
f (x) 150 392 1452 2366 5202
Evaluate f (9) using
i. Newton’s divided difference formula

ii. Lagrange’s formula

solution:
i. The divided difference table is

x y 1st d.d. 2nd d.d. 3rd d.d. 4th d.d.

5 150
121
7 392 24
265 1
11 1452 32 0
457 1
13 2366 42
709
17 5202

88
 Taking x = 9 in the Newton’s divided difference formula ,we obtain

f (9) = 150 + (9 − 5) × 121 + (9 − 5)(9 − 7)(24) + (9 − 5)(9 − 7)(9 − 11)(1)

= 150 + 484 + 192 − 16 = 810

ii. using Lagrange’s formula here x0 = 7, x1 = 7, x2 = 11, x3 = 13, x4 = 17

and
y0 = 150, y1 = 392, y2 = 1452, y3 = 2366, y4 = 5202
putting x = 9 and substituting the above values in Lagrange’s for-
mula,we get
9 − 7)(9 − 11)(9 − 13)(9 − 17) 9 − 5)(9 − 11)(9 − 13)(9 − 17)
f (9) ≈ ×150+ ×392+
(5 − 7)(5 − 11)(5 − 13)(5 − 17) (7 − 5)(7 − 11)(7 − 13)(7 − 17)
(9 − 5)(9 − 7)(9 − 13)(9 − 17) (9 − 5)(9 − 7)(9 − 11)(9 − 17)
×1452+ ×2366
(11 − 5)(11 − 7)(11 − 13)(11 − 17) (13 − 5)(13 − 7)(13 − 11)(13 − 17)
(9 − 5)(9 − 7)(9 − 11)(9 − 13)
× 52023
(17 − 5)(17 − 7)(17 − 11)(17 − 13)
50 3136 3872 2366 578
=− + + − +
3 15 3 3 5
= 810

Example 4.2.9 Determine f (x) as a polynomial in x for the following data

x -4 -1 0 2 5
f(x) 1245 33 5 9 1335

solution:
The divided difference table is

x y 1st d.d 2nd d.d 3rd d.d 4th d.d.

-4 1245
-404
-1 33 94
-28 -14
0 5 10 3
2 13
2 9 88
442
5 1335

89
Applying Newton’s divided difference formula

f (x) ≈ y0 + (x − x0 )[x0 , x1 ] + (x − x0 )(x − x1 )[x0 , x1 , x2 ]

= 1245+(x+4)(−404)+(x+4)(x+1)(94)+(x+4)(x+1)(x)(−14)+(x+4)(x+1)(x)(x−2)(3)
= 3x4 − 5x3 + 6x2 − 14x + 5

90
Chapter 5

Numerical Differentiation and

Integration

5.1 Numerical Differentiation

5.1.1 Derivatives Using Newtons Forward Difference
Formula
Consider the data (xi , f (xi )) given at equispaced points xi = x0 + ih, i =
0, 1, 2, . . . , n where h is the step length. The Newtons forward difference
formula is given by

∆y0 ∆2 y0 ∆n y0
f (x) = y0 + (x−x0 )+ (x−x 0 )(x−x 1 )+· · ·+ (x−x0 )(x−x1 ) . . . (x−xn−1 )
h 2!h2 n!hn
Set x = x0 + ph. we get

f (x) = f (x0 + ph)

p(p−1) 2 (5.1)
= y0 + p
1!
∆y0 + 2!
∆ y0+ · · · + p(p−1)(p−2)...(p−n+1)
n!
∆n y0

Differentiating (5.1) with respect to x, we get

df df dp 1 df
= =
dx dp dx h dp
dp 1
Since dx
= h

df 1 1 1 1
= [∆f0 + (2p−1)∆2 f0 + (3p2 −6p+2)∆3 f0 + (4p3 −18p2 +22p−6)∆4 f0 +. . .]
dx h 2 6 24
(5.2)

91
At x = x0 , that is, at s = 0, we obtain the approximation to the derivative
f 0 (x) as
1 1 1 1
f 0 (x) = [∆f0 − ∆2 f0 + ∆3 f0 − ∆4 f0 + . . .]
h 2 3 4
Differentiating (5.2) with respect to x, we get
d2 f d df dp
= h1 dp
dx2
( dp ) dx
1 1
(5.3)
= h2 [∆ f0 + (p − 1)∆ f0 + 12 (6p2 − 18p + 11)∆4 f0 + · · · ]
2 3

putting p = 0 in (5.3) we have

d2 y 1 11
( 2
)x=x0 = 2 [∆2 f0 − ∆3 f0 − ∆4 f0 + · · · ] (5.4)
dx h 12
and so on similarly we get
d3 y 1 3
(
3
)x=x0 = 3 [∆3 f0 + ∆4 f0 + · · · ] (5.5)
dx h 2
Example 5.1.1 Find dy/dx at x = 1 from the following table of values
x 1 2 3 4
f (x) 1 8 27 64
Solution We have the following forward difference table.
Forward difference table

x y ∆y ∆2 y ∆3 y
1 1
7
2 8 12
9 6
3 27 18
37
4 64
We have h = 1, x0 = 1, and x = x0 + sh = 1 + s. For x = 1, we get s = 0.
Therefore,
dy 1 1 1
(1) = (∆f0 − ∆2 f0 + ∆3 f0 )
dx h 2 3
1 1
= 7 − (12) + (6) = 3
2 3
Exercises 5.1 The following data gives the velocity of a particle for 8 sec-
onds at an interval of 2 seconds. Find the initial acceleration using the entire
data.
time(sec) 0 2 4 6 8
velocity(m/sec) 0 172 1304 4356 10288

92
5.1.2 Derivatives using Newton’s Backward Interpola-
tion Formula
Consider the Newtons backward interpolation formula

y = yn + p 5y1!
n
+ P (p+1)
2!
52 yn + p(p+1)(p+2)
3!
53 yn
(5.6)
+ · · · + p(p+1)(p+2)...(p+n−1)
n!
5n y n
dp
where p = x−xh
n
⇒ dx = h1 h being the interval of differencing
Differentiating (5.6) w.r.t. x we get

df df dp 1 df
= =
dx dp dx h dp

df
dx
= h1 [5fn + 21 (2p + 1) 52 fn + 61 (3p2 + 6p + 2) 53 fn + 1
24
(4p3 + 18p2 + 22p + 6) 54 fn

1
+ 120 (5p4 + 40p3 + 105p2 + 100p + 24) 54 fn + · · · ]
(5.7)
At x = xn , we get s = 0. Hence, we obtain the approximation to the first
derivative f (xn ) as
1 1 1 1 1
f 0 (xn ) = [5fn + 52 fn + 53 fn + 54 fn + 55 fn + · · · ]
h 2 3 4 5
Differentiating (5.15) with respect to x again, we get

d2 f
   
1 d df dp 1 d df
= = 2
dx2 h dp dp dx h dp dp
d2 f 1 1
dx2
= h2
[52 fn + (p + 1) 53 fn + 12
(6p2 + 18p + 11) 54 fn
(5.8)
1
+ 24 (4p3 2 4
+ 24p + 42p + 20) 5 fn + · · · ]
At x = xn , that is, at s = 0, we obtain the approximation to the second
derivative f 00 (x) as

f 00 (xn ) = 1
h2
[52 fn + 53 f n + 11
12
54 f n
(5.9)
+ 56 54 fn + 137
180
55 f n + · · · ]

Remark We use the backward difference formulas for derivatives, when we

need the values of the derivatives near the end of table of values.

93
Example 5.1.2 Find f 0 (3) using the Newtons backward difference formula,
for the data

x 1.0 1.5 2.0 2.5 3.0

f (x) −1.5 −2.875 −3.5 −2.625 0.5
Solution: The step length is h = 0.5 and x = xn + sh = 3.0 + s(0.5). For
x = 3.0, we get s = 0. We have the following backward difference table.
Backward difference table
x y 5y 52 y 53 y 54 y
1.0 −1.5
−1.375
1.5 −2.875 0.75
−0.625 0.75
2.0 −3.5 1.5 0
0.875 0.75
2.5 −2.625 2.25
3.125
3.0 0.5
From the formula
1 1 1 1 1
f 0 (xn ) = [5fn + 52 fn + 53 fn + 54 fn + 55 fn + · · · ]
h 2 3 4 5
we obtain
1 1 1
f 0 (3.0) = [3.125 + (2.25) + (0.75)] = 9
0.5 2 3

5.2 Numerical Integration

Introduction
The problem of numerical integration is to find an approximate value of the
integral Z b
I= w(x)f (x)dx
a
where w(x) > 0 in (a, b) is called the weight function. The function f (x) may
be given explicitly or as a tabulated data. We assume that w(x) and f (x) are
integrable on [a, b]. The limits of integration may be finite, semi-infinite or
infinite. The integral is approximated by a linear combination of the values
of f (x) at the tabular points as
Z b Xn
I= w(x)f (x)dx = λk f (xk )
a k=0

94
Integration Rules Based on Uniform Mesh Spacing
When w(x) = 1 and the nodes xk s are prescribed and are equispaced with
x0 = a, xn = b, where h = (b − a)/n, the methods are called Newton-Cotes
integration rules. The weights λ0k s are called Cotes numbers. We shall now
derive some Newton-Cotes formulas. That is, we derive formulas of the form
Z b n
X
I= f (x)dx = λk f (xk ) (5.10)
a k=0

Rb
We note that, a f (x)dx defines the area under the curve y = f (x), above
the x-axis, between the lines x = a, x = b.

5.2.1 Trapezium Rule

This rule is also called the trapezoidal rule. Let the curve y = f (x), a ≤ x ≤
b, be approximated by the line joining the points P (a, f (a)), Q(b, f (b)) on
the curve (see 5.2).
Using the Newtons forward difference formula, the linear polynomial approx-
imation to f (x), interpolating at the points P (a, f (a)), Q(b, f (b)), is given by

Figure 5.1: Trapizium rule

95
 1
f (x) = f (x0 ) + (x − x0 )∆f (x0 )
h
where x0 = a, x1 = b and h = b − a. Substituting in
we obtain
Z b Z x1 Z x1
1 x1
Z
I= f (x)dx = f (x)dx = f (x0 ) dx + [ (x − x0 )dx]∆f0
a x0 x0 h x0

1 1
= (x1 − x0 )f (x0 ) + [ (x − x0 )2 ]xx10 ∆f (x0 )
h 2
1
= (x1 − x0 )f (x0 ) + [f (x1 ) − f (x0 )](x1 x0 )2
2h
h
= hf (x0 ) + [f (x1 ) − f (x0 )]
2
h (b − a)
= [f (x1 ) + f (x0 )] = [f (b) + f (a)].
2 2
The trapezium rule is given by
Z b
h b−a
I= f (x)dx = [f (x1 ) + f (x0 )] = [f (b) + f (a)] (5.11)
a 2 2
Hence, the trapezium rule integrates exactly polynomial of degree ≤ 1, and
the method is of order 1.
Composite trapezium rule
Let the interval [a, b] be subdivided into N equal parts of length h.

b−a
That is, h = N
. The nodal points are given by

a = x0 , x1 = x0 + h, x2 = x0 + 2h, ..., xN = x0 + N h = b.

We write
Z b Z xN Z x1 Z x2 Z xN
f (x)dx = f (x)dx = f (x)dx + f (x)dx + · · · + f (x)dx
a x0 x0 x1 xN −1

There are N integrals. Using the trapezoidal rule to evaluate each integral,
we get the composite trapezoidal rule as
Rb
a
f (x)dx = h2 [{f (x0 ) + f (x1 )} + {f (x1 ) + f (x2 )} + · · · + {f (xN −1 ) + f (xN )}]
= h2 [f (x0 ) + 2{f (x1 ) + f (x2 ) + ... + f (xN −1 )} + f (xN )].
(5.12)

96
 Figure 5.2: Composite Trapizium rule

The composite trapezium rule is also of order 1.

The formula (5.23) is called Trapezoidal rule for numerical integration. The
error committed in this formula is given by

(b − a) 2 00 (b − a)3 00
E≈− h f (ξ) = − f (ξ), (5.13)
12 12n2
where
a = x0 < ξ < xn = b.
R1 1
Example 5.2.1 Find the approximate value of I = 0 1+x dx , using the
trapezium rule with 2, 4 and 8 equal subintervals. Using the exact solution,
find the absolute errors.

Solution With N = 2, 4 and 8, we have the following step lengths and nodal
points.
N = 2, h = b−a
N
= 12 the nodes are 0, 0.5, 1.0
N = 4, h = b−a
N
= 14 the nodes are 0, 0.25, 0.5, 0.75, 1.0
N = 8, h = b−a
N
= 18 the nodes are 0, 0.125, 0.25, 0.375, 0.5, 0.675, 0.75, 0.875, 1.0
We have the following tables of values.

N =2:x 0 0.5 1.0

f (x) 1.0 0.666667 0.5

97
 N = 4: We require the above values. The additional values required are
the following:
x 0.25 0.75
f (x) 0.8 0.571429
N = 8: We require the above values. The additional values required are the
following:

x 0.125 0.375 0.625 0.875

f (x) 0.888889 0.727273 0.615385 0.533333

Now, we compute the value of the integral

h
N =2: I1 = [f (0)+2f (0.5)+f (1.0)] = 0.25[1.0+2(0.666667)+0.5] = 0.708334.
2
h
N =4: I2 = [f (0) + 2f (0.25) + f (0.5) + f (0.75) + f (1.0)]
2
= 0.125[1.0+20.8 + 0.666667 + 0.571429+0.5] = 0.697024.
h
N =8: I3 = [f (0) + 2(f (0.125) + f (0.25) + f (0.375) + f (0.5)
2
+ f (0.625) + f (0.75) + f (0.875)) + f (1.0)]
= 0.0625[1.0+2(0.888889+0.8+0.727273+0.666667+0.615385
+ 0.571429 + 0.533333) + 0.5] = 0.694122.
The exact value of the integral is I = ln2 = 0.693147. The errors in the
solutions are the following:

|Exact − I1 | = |0.693147 − 0.708334| = 0.015187

|Exact − I2 | = |0.693147 − 0.697024| = 0.003877

|Exact − I3 | = |0.693147 − 0.694122| = 0.000975.
R2 1
Example 5.2.2 Evaluate I = 1 5+3x dx with 4 and 8 subintervals using the
trapezium rule. Compare with the exact solution and find the absolute errors
in the solutions. Comment on the magnitudes of the errors obtained. Find
the bound on the errors.

98
Solution: With N = 4 and 8, we have the following step lengths and nodal
points. N = 4 : h = b−a
N
= 14 . The nodes are 1, 1.25, 1.5, 1.75, 2.0.

N = 8 : h = b−aN
= 18 . The nodes are 1, 1.125, 1.25, 1.375, 1.5, 1.675, 1.75, 1.875, 2.0.
We have the following tables of values.
N = 4:
x 1.0 1.25 1.5 1.75 2.0
f (x) 0.125 0.11429 0.10526 0.09756 0.09091
N = 8: We require the above values. The additional values required are
the following

x 1.125 1.375 1.625 1.875

f (x) 0.11940 0.10959 0.10127 0.09412

Now, we compute the value of the integral.

h
N =4: I1 = [f (1) + 2(f (1.25) + f (1.5) + f (1.75)) + f (2.0)]
2
= 0.125[0.125+2(0.11429+0.10526+0.09756)+0.09091] = 0.10627.
h
N =8: I2 = [f (1) + 2(f (1.125) + f (1.25) + f (1.375) + f (1.5)
2
+ f (1.625) + f (1.75) + f (1.875)) + f (2.0)]
= 0.0625[0.125 + 2(0.11940 + 0.11429 + 0.10959 + 0.10526 + 0.10127
+ 0.09756 + 0.09412) + 0.09091] = 0.10618.
The exact value of the integral is
1 1
I = [5 + 3x]21 = [ln(11) − ln(8)] = 010615.
3 3
The errors in the solutions are the following:

|Exact − I1 | = |0.10615 − 0.10627| = 0.00012.

|Exact − I2 | = |0.10615 − 0.10618| = 0.00003.

R1 dx
Exercises 5.2 Using the trapezium rule, evaluate the integral I = 0 x2 +6x+10 ,
with 2 and 4 subintervals. Compare with the exact solution. Comment on
the magnitudes of the errors obtained.

99
Exercises 5.3 The velocity of a particle which starts from rest is given by
the following table.
t(sec) 0 2 4 6 8 10 12 14 16 18 20
v(f t/sec) 0 16 29 40 46 51 32 18 8 3 0
Evaluate using trapezium rule, the total distance travelled in 20 seconds.

Note: Trapezoidal rule can be applied to any number of subintervals odd or

even.

5.2.2 Simpsons 1/3 Rule

In the previous section, we have shown that the trapezium rule of integra-
tion integrates exactly polynomials of degree 1, that is, the order of the
formula is 1. In many science and engineering applications, we require meth-
ods which produce more accurate results. One such method is the Simp-
sons 1/3 rule. Let the interval [a, b] be subdivided into two equal parts
with step length h = b−a 2
. We have three abscissas x0 = a, x1 = a+b 2
, and
x2 = b. Then, P (x0 , f (x0 )), Q(x1 f (x1 )), R(x2 , f (x2 )) are three points on the
curve y = f (x). We approximate the curve y = f (x), axb, by the parabola
joining the points P, Q, R, that is, we approximate the given curve by
a polynomial of degree 2. Using the Newtons forward difference formula,
the quadratic polynomial approximation to f(x), interpolating at the points
P (x0 , f (x0 )), Q(x1 f (x1 )), R(x2 , f (x2 )), is given by
1 1
f (x) = f (x0 ) + (x − x0 )∆f (x0 ) + 2 (x − x0 )(x − x1 )∆f (x0 ).
h 2h
Substituting in (5.10), we obtain
Z b Z x2 Z x2
1 1
f (x)dx = f (x)dx = [f (x0 )+ (x−x0 )∆f (x0 )+ 2 (x−x0 )(x−x1 )∆f (x0 )]dx
a x0 x0 h 2h
(5.14)
Composite Simpsons 1/3 rule
We note that the Simpsons rule derived earlier uses three nodal points.
Hence, we subdivide the given interval [a, b] into even number of subintervals
of equal length h. That is, we obtain an odd number of nodal points. We
take the even number of intervals as 2N. The step length is given by h = b−a 2N
.
The nodal points are given by a = x0 , x1 = x0 + h, x2 = x0 + 2h, . . . , x2N =
x0 + 2N h = b. The given interval is now written as
Z b Z x2N Z x2 Z x4 Z x6 Z x2n
f (x)dx = f (x)dx = f (x)dx+ f (x)dx+ f (x)dx+. . .+ f (x)dx
a x0 x0 x2 x4 x2N −2

100
Note that there are N integrals. The limits of each integral contain three
nodal points. Using the Simpsons 1/3 rule to evaluate each integral, we get
the composite Simpsons 1/3 rule as
Rb
a
f (x)dx = h3 [{f (x0 ) + 4f (x1 ) + f (x2 )} + {f (x2 ) + 4f (x3 ) + f (x4 )}
+ . . . + {f (x2N −2 ) + 4f (x2N −1 ) + f (x2 N )}]
= h3 [f (x0 ) + 4{f (x1 ) + f (x3 ) + . . . . + f (x2N −1 )}
+2{f (x2 ) + f (x4 ) + . . . + f (x2N −2 )} + f (x2N )]
(5.15)
The composite Simpsons 1/3 rule is also of order 3.
Remark 5.1 We have noted that the Simpson 1/3 rule and the composite
Simpsons 1/3 rule are of order 3. If f(x) is a polynomial of degree ≤ 3, then
f (4) (x) = 0. This result implies that error is zero and the composite Simpsons
1/3 rule produces exact results for polynomials of degree≤ 3.
R1 1
Example 5.2.3 Find the approximate value of I = 0 1+x dx , using Simp-
son’s 1/3 rule with 2, 4 and 8 equal subintervals. Using the exact solution,
find the absolute errors.

Solution:
With N = 2, 4 and 8, we have the following step lengths and nodal points.
N = 2, h = b−a
N
= 12 the nodes are 0, 0.5, 1.0
N = 4, h = N = 14 the nodes are 0, 0.25, 0.5, 0.75, 1.0
b−a

N = 8, h = b−a
N
= 18 the nodes are 0, 0.125, 0.25, 0.375, 0.5, 0.675, 0.75, 0.875, 1.0
We have the following tables of values.
N =2:x 0 0.5 1.0
f (x) 1.0 0.666667 0.5
N = 4: We require the above values. The additional values required are the
following:
x 0.25 0.75
f (x) 0.8 0.571429
N = 8: We require the above values. The additional values required are the
following:
x 0.125 0.375 0.625 0.875
f (x) 0.888889 0.727273 0.615385 0.533333
Now, we compute the value of the integral
h
N =2: I1 = [f (0) + 4f (0.5) + f (1.0)]
3
101
 1
= [1.0 + 4(0.666667) + 0.5] = 0.674444.
6
h
N =4: I2 = [f (0) + 4{f (0.25) + f (0.75)} + 2f (0.5) + f (1.0)]
3
1
= [1.0+4{0.8+0.571429}+2(0.666667)+0.5]
12
= 0.693254.
h
N =8: I3 = [f (0) + 4{f (0.125) + f (0.375) + f (0.625) + f (0.875)}
3
+ 2{f (0.25) + +f (0.5) + f (0.75)} + f (1.0)]
1
= [1.0 + 4{0.888889 + 0.727273 + 0.615385 + 0.533333}
24
+ 2{0.571429 + 0.8 + 0.666667} + 0.5] = 0.693155.
The exact value of the integral is I = ln(2) = 0.693147. The errors in the
solutions are the following:

|Exact − I1 | = |0.693147 − 0.694444| = 0.001297.

|Exact − I2 | = |0.693147 − 0.693254| = 0.000107.

|Exact − I3 | = |0.693147 − 0.693155| = 0.000008.

5.2.3 Simpsons 3/8 Rule

Putting n = 3 in 5.10 and neglecting all differences above third order ,we get
Z x0 +3h
3 3 1
f (x)dx = 3h(y0 + ∆y0 + ∆2 y0 + ∆3 y0 )
x0 2 4 8

3
= h(y0 + 3y1 + 3y2 + y3 )
8
similarly Z x0 +6h
3h
f (x)dx = (y3 + 3y4 + 3y5 + y6 )
x0 +3h 8
Composite Simpson’s 3/8 rule
Z x0 +nh
3h
f (x)dx = [y0 +3(y1 +y2 +y4 +y5 +· · ·+yn−1 )+2(y3 +y6 +· · ·+yn−3 )+yn ]
x0 8
where n is a multiple of three which is the number of subintervals.

102
Remark 5.2 Simpsons 3/8 rule has some disadvantages. They are the fol-
lowing:
i. The number of subintervals must be divisible by 3.

ii. It is of the same order as the Simpsons 1/3 rule, which only requires
that the number of nodal points must be odd.

iii. The error constant c in the case of Simpsons 3/8 rule is c = 3/80,
which is much larger than the error constant c = 1/90, in the case of
Simpsons 1/3 rule. Therefore, the error in the case of the Simpsons
3/8 rule is larger than the error in the case Simpson 1/3 rule. Due to
these disadvantages, Simpsons 3/8 rule is not used in practice.
R2 1
Example 5.2.4 Using the Simpsons 3/8 rule, evaluate I = 1 5+3x dx with
3 and 6 subintervals. Compare with the exact solution.

Solution: With n = 3N = 3 and 6, we have the following step lengths and

nodal points.
b−a 1
n = 3N = 3 : h= = . The nodes are 1, 4/3, 5/3, 2.0.
3N 3

b−a 1
n = 3N = 6 : h= = The nodes are 1, 7/6, 8/6, 9/6, 10/6, 11/6, 2.0
3N 6
We have the following tables of values.
n = 3N = 3:
x 1.0 4/3 5/3 2.0
f (x) 0.125 0.11111 0.10000 0.09091

n = 3N = 6 : We require the above values. The additional values required

are the following.

x 7/6 9/6 11/6

f(x) 0.11765 0.10526 0.09524

Now, we compute the value of the integral. n = 3N = 3:

3h
I1 = [f (1) + 3f (4/3) + 3f (5/3) + f (2.0)]
8

= 0.125[0.125 + 3{0.11111 + 0.10000} + 0.09091] = 0.10616.

103
n = 3N = 6 :
3h
I2 = [f (1) + 3{f (7/6) + f (8/6) + f (10/6) + f (11/6)} + 2{f (9/6)} + f (2.0)]
8
1
= [0.125+3{0.11765+0.11111+0.10000+0.09524}+2(0.10526)+0.09091] = 0.10615.
16
The exact value of the integral is I = 31 [log11 − log8] = 0.10615.
The magnitude of the error for n = 3 is 0.00001 and for n = 6 the result
is correct to all places.

5.2.4 Gauss-Quadrature Formula

We have determined the value of the function at equal intervals of the argu-
ments x by employing Trapezoidal and Simpson’s rules.Then we fit a poly-
nomial through these functional values and integrate this polynomial.Gauss
noted that when we compute the values of the function at unequal interval of
the argument x and we approximate the function by a polynomial through
these points then, we get a significantly lower error for the integral.Now we
consider Z b
I= f (x)dx (5.16)
a
R1
Gauss Formula requires that the integral should be in the form of −1
f (x)dx
Now we assume a new variable y is related to x linearly .

x = p + qy
Lower limit x = a corresponding to y = −1
Upper limit x = b corresponding to y = 1 .
Thus
Z b Z 1 Z 1
I= f (x)dx = f (p + qy)qdy = q f (p + qy)dy (5.17)
a −1 −1

where x = (p + qy) and dx = qdy

To determine p and q
At x = a y = −1; so we have a = p − q
At x = b y = 1; so we have b = p + q
Solving these two equations for p and q we have
a+b b−a
p= q=
2 2

104
Therefore
1
x = [a + b + (b − a)y]
2
From (5.24 ) we have
b 1 1
b−a a + b + (b − a)y b−a
Z Z Z
I= f (x)dx = f[ ]dy = φ(y)dy
a 2 −1 2 2 −1
(5.18)
where φ(y) = f [ a+b+(b−a)y
2
]
The general form for the Gauss Quadrature formula is
n
b−aX
I= Ri φ(yi ) (5.19)
2 i=1

where R1 , R2 , . . . , Rn are constants called weight functions and y1 , y2 , . . . , yn

are values of y where we determine the transformed φ(y) and n denotes the
number of points for determining the function .
The values of R1 , R2 , . . . , Rn and y1 , y2 , . . . , yn are shown in the table
given below for up to n=5 points

No.of Weight yi order of truncation

points n Ri error
2 R1 = 1.000 y1 = −0.5773503 O[f 4 (y0 )]
R2 = 1.000 y2 = −y1 p
3 R1 = 0.555556 = 59 y1 = 0.77459677 = − 2/5 O[f 6 (y0 )]
R2 = 0.88888891 = 89 y2 = 0
R3 = R1 y3 = −y1
4 R1 = 0.3478549 y1 = −0.8611363 O[f 8 (y0 )]
R2 = 0.6521452 y2 = −0.3399811
R3 = R2 y3 = −y2
R4 = R1 y4 = −y1
5 R1 = 0.2369269 y1 = −09061799 O[f 10 (y0 )]
R2 = 0.4786287 y2 = −0.5384693
R3 = 0.56888889 y3 = o
R4 = R2 y4 = −y2
R5 = R1 y5 = −y1

Where −1 ≤ y0 ≤ 1 and values of Ri0 s are rounded off to eight decimal digit.
R4
Example 5.2.5 Evaluate 2 (x2 + 2x)dx by Gauss Quadrature formula .

105
Solution: Here a = 2, b = 4, f (x) = x2 + 2x
Let us transform the interval [2, 4] to [−1, 1]by putting
1
x = [(a + b) + (b − a)y]
2
1
x = [(2 + 4) + (4 − 2)y] = 3 + y
2
when x = 2, y = −1 and x = 4, y = 1

dx = dy

and Z 4 Z 1
I= f (x)dx = φ(y)dy
2 −1

where
φ(y) = φ(3 + y)
By taking n = 2 in the Gauss formula
n
b−aX
I= Ri φ(yi ),
2 i=1

we have
I = [R1 φ(y1 ) + R2 φ(y2 )] (5.20)
Where R1 = 1.0 R2 = 1.0, y1 = −0.5773503, y2 = −y1

φ(y) = f (3 + y) = (3 + y)2 + 2(3 + y) = y 2 + 8y + 15

Now

φ(y1 ) = (−0.5773503)2 + 8(−0.5773503) + 15 = 0.3333333 − 4.618802 + 15

= 10.714531
φ(y2 ) = (−y1 )2 +8(−y1 )+15 = (0.5773503)2 +8(0.5773503)+15 = 19.952135
From 5.26

I = 1.0 × 10.714531) + (1.0 × 19.952135) = 30.666666667

I = 30.6666667
R 1.5 −x2
Example 5.2.6 Evaluate 0.2 e dx using Gauss Quadrature formula with
n = 3.

106
Solution:
2
Here a = 0.2, b = 1.5, f (x) = e−x
Let us transform the interval [0.2,1.5] to [-1,1] by putting
1
x = [(a + b) + (b − a)y]
2
Therefore
1
x = [(0.2 + 1.50) + (1.5 − 0.2)]] = 0.85 + 0.65y
2
when x = 0.2, ay = −1, and x = 1.5, y = 1
dx = 0.65dy
and
Z 1.5 Z 1
I= f (x)dx = 0.65 φ(y)dy, whereφ(y) = f (0.85 + 0.65y)
0.2 −1
2
So φ(y) = f (0.85 + 0.65y) = e−(0.85+0.65y)
By taking n = 3 in the Gauss formula
n
b−aX
I= Ri φ(yi ),
2 i=1

we have
I = 0.65[R1 φ(y1 ) + R2 φ(y2 ) + R3 φ(y3 )] (5.21)
Where R1 = 0.5555556, R2 = 08888889, R3 = R1
y1 = −0.7745967, y2 = 0, y3 = −y1
Now
2 2
φ(y1 ) = e−[0.85+0.65(−0.7745967)] = e−[0.85+0.5034878]
2
e−(0.3465121) = e−0.1200706 = 1.1275765
2
φ(y2 ) = e−(0.85+0) = e− 0.7225 = 2.059595757
2 2
φ(y3 ) = e−(0.85+0.65(0.7745967) = e−(1.3534878) = 6.2459259
From 5.27
I = 0.65[0.5555556(1.1275765) + 0.888889(2.0595757) + 0.555556(6.2459259)
= 0.65[0.6264314 + 1.830734 + 3.4699591] = 3.8526309 ≈ 3.8526
R 12
Exercises 5.4 compute the integral 5 x1 dx using Gauss Quadrature for-
mula.

107
5.2.5 Romberg Integration
We modify the Trapezoidal rule to find a better approximation to the value
of an integral. We know that the truncation error in the trapezoidal rule is
nearly proportional to h2 an interval of size h. The error in the Trapezoidal
rule
b − a 00
E=− y (ξ)h2 (5.22)
12
where a < ξ < b If we put
b − a 00
c= y (ξ)
12
then the error in the Trapezoidal rule = ch2 . y 00 (ξ), the second derivative,
is reasonably constant c may be taken to be constant. Consider the evaluation
of the integral Z b
I= ydx
a
by the Trapezoidal rule with two different intervals say h1 , h2 . Let I1 and
I2 denote the approximate values with the corresponding errorsE1 and E2
respectively.
I = I1 + ch21
Then
I = I2 + ch22
and We get
I1 + ch21 = I2 + ch22
or
I1 − I2
⇒c=
h22 − h21
I1 − I2 2
I = I1 + h
h22 − h21 1
I1 h22 − I2 h21
⇒I= (5.23)
h22 − h21
This will be a better approximation to I than I1 or I2 . The above method
is called Richardsons method. If we take h = h1 and h2 = 12 h in (7)
we get
2
I1 h4 − I2 h 4I1 − I2
I = h2 =
4
−h2 − 34

108
Therefore
I2 − I1
I = I2 + (5.24)
3
If we apply the Trapezoidal rule several times successively halving h, every
time the error is reduced by a factor 14 . Let A1 , A2 , A3 , · · · denote the results.
Let the formula (5.24) be applied to each pair of Ai s and denote the results
by B1 , B2 , B3 , · · · etc.
Applying formula (5.24) to each pair of Bi s we get next results C1 , C2 , · · ·
in this process the following array of results is obtained.
A1 A2 A3 A4 ···
B1 B2 B3 ···
C1 C2 ···
The above computation is continued with two successive values are very
close to each other. This refinement of Richardsons method is known as
Romberg integration. The values of the integral, in Romberg integration can
be tabulated as follows.
I(h)
I(h, h2 )
I(h/2) I(h, h/2, h/4)
I(h/2, h/4) I(h, h/2, h/4, h/8)
I(h/4) I(h/2, h/4, h/8)
I(h/4, h/8)
I(h/8)
Where
h 1 h
I(h, ) = [4I( − I(h)]
2 3 2
h h 1 h h
I( , ) = [4I( ) − I( )]
2 4 3 4 2
1
I(h, h/2, h/4) = [4I(h/2, h/4) − I(h, h/2)]
3
1
I(h/2, h/4, h/8) = [4I(h/4, h/8) − I(h/2, h/4)]
3
1
I(h, h/2, h/4, h/8) = [4I(h/2, h/4, h/8) − I(h, h/2, h/4)]
3
b−a
Alternatively since hi = ni . where ni is the number of intervals.
Replacing hi for i = 1, 2 by b−a
ni
in (5.23) we get

n21 I1 − n22 I2
I=
n21 − n22

109
 R 1.2 1
Example 5.2.7 Using Rombergs method compute I = 0 1+x
dx correct to
4 decimal places.

Solution:
Solution Here
1
f (x) =
1+x
We can take h = 0.6, 0.3, 0.15
h h
i.e. h = 0.6 , = 0.3, = 0.15
2 4
x 0 0.15 0.30 0.40 0.60 0.75 0.90 1.05 1.20
f(x) 1 0.8695 0.7692 0.6896 0.6250 0.5714 0.5263 0.48780 0.4545
Using Trapezoidal rule with h = 0.6 we get
0.6
I(h) = I(0.6) = I1 = (1 + 0.4545 + 2 × 0.62560) = 0.8113
2
0.6
with h = 2
= 0.3 we get

0.3
I(h/2) = I(0.3) = I2 = [(1+0.4545)+2(0.7692+0.625+0.5263)] = 0.7943,
2
0.6
with h = 4
= 0.15 we get

0.15
I(h/4) = I(0.15) = I3 = [(1 + 0.4545) + 2(0.8695 + 0.7692 + 0.6896)]
2
0.15
+ [2(0.6250 + 0.5714 + 0.5263 + 0.4878)] = 0.7899.
2
Now I(h, h/2) = I(0.6, 0.3)
1
∴ I(0.6, 0.3) = [4 × I(0.3) − I(0.6)]
3
1
= [4 × 0.7943 − 0.8113] = 0.7886,
3
Similarly I(h/2, h/4) = I(0.3, 0.15)
1
∴ I(0.3, 0.15) = [4 × I(0.15) − I(0.3)]
3
1
= [4 × 0.7899 − 0.7943] = 0.7884
3
110
We get I(h, h/2, h/4) = I(0.6, 0.3, 0.15)
1
∴ I(0.6, 0.3, 0.15) = [4 × I(0.15, 0.3) − I(0.3, 0.6)
3
1
= [4 × 0.7884 − 0.7886] = 0.7883
3
The table of these values is
0.8113
0.7886
0.7948
0.7883
0.7884
0.7899
Z 1.2
1
∴I= dx = 0.7883
0 1+x
simpson’s Rule

If we use n1 number of intervals

I ≈ I1

with error = − (b−a)

90
h4 f (4) (ξ),for a < ξ < b

(b − a)5 (4)
=− f (ξ)
90n41

(b − a)5 (4)
⇒ n41 (I − I1 ) = − f (ξ) (5.25)
90
Similarly for n2 number of intervals

(b − a)5 (4)
⇒ n42 (I − I2 ) = − f (ξ) (5.26)
90
From (5.25) and (5.26)

n41 (I − I2 ) = n42 (I − I2 )

n41 I1 − n42 I2
I= (5.27)
n41 − n42
The result is the same using simpson’s 1/3 and 3/8 rule

111
 R 1.2 1
Example 5.2.8 Using Rombergs method compute I = 0 1+x
dx correct to
4 decimal places, by simpson’s rule with h=0.6 and h=0.3

Solution:
1
since f (x) = 1+x
and h1 = 0.6 , h2 = 0.3

1.2 − 0 1.2 − 0
⇒ n1 = = 2 and n2 = =4
0.6 0.3
x 0 0.30 0.60 0.90 1.20
f(x) 1 0.769231 0.6250 0.526316 0.454545
h1 0.6
I1 = [f (x0 ) + 4f (x1 ) + f (x2 )] = [1 + 4 × (0.6250) + 0.454545]
3 3
= 0.790909
h2
I2 = [f (x0 ) + 4(f (x1 ) + f (x3 )) + 2f (x2 ) + f (x4 )]
3
0.3
= [1 + 4 × (0.526316 + 0.769231) + 2 × 0.6250 + 0.454545]
3
= 0.7886733
Now applying Ruberg’s Integration formula we have

n41 I1 − n42 I2
I=
n41 − n42

24 × 0.790909 − 44 × 0.7886733
I=
24 − 44
12.654544 − 201.900365
=
16 − 256
= 0.78852425
R1 1
Exercises 5.5 Using Rombergs method prove that 0 1+x dx = 0.6931.
Rπ
Exercises 5.6 Apply Rombergs method to show that 0
2
sinx dx = 1 .

112
Chapter 6

First Order Initial Value

Problems
dy
INTRODUCTION The derivative dx of a function y = φ(x) is itself an-
0
other function φ (x) found by an appropriate rule. The exponential function
2
y = e0.1x is differentiable on the interval (−∞, ∞), and, by the Chain Rule,
2 2
its first derivative is dy/dx = 0.2xe0.1x . If we replace e0.1x on the right-hand
side of the last equation by the symbol y, the derivative becomes
dy
= 0.2xy (1)
dx
Now imagine that a friend of yours simply hands you equation (1)
you have no idea how it was constructed and asks, What is the function
represented by the symbol y? You are now face to face with one of the basic
problems in this course:
How do you solve such an equation for the functiony = φ(x)?
The equation that we made up in (1) is called a differential equation. Before
proceeding any further, let us consider a more precise definition of this con-
cept.

DEFINITION: Differential Equation

An equation containing the derivatives of one or more unknown functions (or
dependent variables), with respect to one or more independent variables, is
said to be a differential equation (DE). To talk about them, we shall classify
differential equations according to type, order, and linearity.
Classification by Type:
If a differential equation contains only ordinary derivatives of one or more
unknown functions with respect to a single independent variable, it is said
to be an ordinary differential equation (ODE). An equation involving partial

113
derivatives of one or more unknown functions of two or more independent
variables is called a partial differential equation (PDE). Our first example
illustrates several of each type of differential equation.

Example 6.0.9 Types of Differential Equations

a The equations

dy d2 y dy dx dy
+5y = ex , − +6y = 0, and, + = 2x+y (2)
dx dx2 dx dt dt
are examples of ordinary differential equations.

b The following equations are partial differential equations:

∂ 2u ∂ 2u ∂ 2u ∂ 2u ∂u ∂u ∂v
+ = 0, = −2 , =− (3)
∂x2 ∂y 2 ∂x 2 ∂t 2 ∂t ∂y ∂x
Notice in the third equation that there are two unknown functions and
two independent variables in the PDE. This means u and v must be
functions of two or more independent variables.

Classification by Order:
The order of a differential equation (either ODE or PDE) is the order of the
highest derivative in the equation. For example,

is a second-order ordinary differential equation. In Example 6.1, the first

and third equations in (2) are first-order ODEs, whereas in (3) the first two
equations are second-order PDEs. First-order ordinary differential equations
are occasionally written in differential form M (x, y)dx + N (x, y)dy = 0. For
example, if we assume that y denotes the dependent variable in (y − x)dx +
4xdy = 0, then y 0 = dy/dx, so by dividing by the differential dx, we get
the alternative form 4xy 0 + y = x. In symbols we can express an nth -order
ordinary differential equation in one dependent variable by the general form

F (x, y, y 0 , ..., y (n) ) = 0 (4)

114
 where F is a real-valued function of n + 2 variables: x, y, y 0 , ..., y (n) . For
both practical and theoretical reasons we shall also make the assumption
hereafter that it is possible to solve an ordinary differential equation in the
form (4) uniquely for the highest derivative y (n) in terms of the remaining
n + 1 variables. The differential equation

dn y/dxn = f (x, y, y 0 , ..., y (n−1) ) (5)

where f is a real-valued continuous function, is referred to as the normal form

of (4).
Thus when it suits our purposes, we shall use the normal forms

dy/dx = f (x, y)

and d2 y/dx2 = f (x, y, y 0 ) to represent general first- and second-order ordinary

differential equations.

6.1 Definition of ODE and examples

DEFINITION: An ordinary differential equation(ODE) is a relation be-
tween a function y(t), its derivatives y 0 , y 00 , · · · , y (m) where m denotes the
order of the highest derivative, and the variable t upon which the function y
and all its derivatives depend. The most general form of an ODE is given by

Φ(t, y, y 0 , y 00 , · · · , y (m) ) = 0. (6.1)

Example 6.1.1 The following DE are an examples of an ordinary differen-

tial equation

dy d2 y dy d3 x d2 x
+ 5y = ex , 2
− + 6y = 0, + 5 + 3x = sin(t)
dx dx dx dt3 dt2

115
6.2 Order of a differential equation, linear
and nonlinear ODE
Definition: The order of the differential equation (DE) is the order of its
highest derivative.
Definition: The degree of the DE is the degree of the derivative of the
highest order after the equation has been rationalized in derivatives. An
ODE is of two types namely linear and non-linear.
Definition:An ODE is said to be linear if no product of the dependent
variable y(t) with itself or any one of its derivatives occur. Otherwise, it is
non-linear. A linear DE of order m can be expressed in the form
m
X
Φp (t)y p (t) = r(t)
p=0

in which Φp (t) are known functions. If the general non-linear DE (6.1) of

order m can be written as
y (m) = F (t, y, y 0 , . . . , y (m−1) ) (6.2)
then the equation in (6.2) is called a canonical (or natural) representation of
the DE (6.1). In such form, the highest order derivative is expressed in terms
of the function y, the lower order derivatives and the independent variable t.
Example 6.2.1 A second order differential equation is an equation of the
form
F (x, y, y 0 , y 00 ) = 0.
An nth-order ordinary differential equation (4) is said to be linear if F is
linear in y, y 0 , . . . , y (n) . This means that an nth-order ODE is linear when
(4) is an (x)y (n) + an−1 (x)y (n−1) + . . . + a1 (x)y 0 + a0 (x)y − g(x) = 0 or
dn y d(n−1) y dy
an (x) n
+ a n−1 (x) (n−1)
+ . . . + a1 (x) + a0 (x)y = g(x). (6)
dx dx dx
Two important special cases of (6) are linear first-order (n = 1) and linear
second- order (n = 2) DEs:
dy
a1 (x) + a0 (x)y = g(x)
dx
and
d2 y dy
a2 (x) 2
+ a1 (x) + a0 (x)y = g(x)
dx dx
In the additive combination on the left-hand side of equation (6) we see that
the characteristic two properties of a linear ODE are as follows:

116
 • The dependent variable y and all its derivatives y 0 , y 00 , . . . , y (n) are of
the first degree, that is, the power of each term involving y is 1.

• The coefficients a0 , a1 , . . . , an of y, y 0 , . . . , y (n) depend at most on the

independent variable x.

A nonlinear ordinary differential equation is simply one that is not linear.

Nonlinear functions of the dependent variable or its derivatives, such as siny
0
or ey , cannot appear in a linear equation.

Example 6.2.2 Linear and Nonlinear ODEs

(a) The equations

d3 y dy
(y − x)dx + 4xydy = 0, y 00 − 2y 0 + y = 0, x3 3
+ x − 5y = ex
dx dx
are, in turn, linear first-, second-, and third-order ordinary differential equa-
tions. We have just demonstrated that the first equation is linear in the
variable y by writing it in the alternative form 4xy 0 + y = x.
(b) The equations

are examples of nonlinear first-, second-, and fourth-order ordinary differen-

tial equations, respectively.

Example 6.2.3 A linear differential equations are

a. a(x)y 0 + b(x)y = c(x) is first order ,

b. a(x)y 00 + b(x)y 0 + c(x)y = d(x) is second order,

where a(x), b(x), c(x), d(x) are known functions of x

Example 6.2.4 Some non-linear differential equations

a. (y 00 )2 = xy

b. y 0 y = −x

c. y 0 = siny

117
6.3 Nature of Solutions of ODE: particular
and general solutions
Definition 6.1 Any function φ, defined on an interval I and possessing at
least n derivatives that are continuous on I, which when substituted into an
nth-order ordinary differential equation reduces the equation to an identity,
is said to be a solution of the equation on the interval.

In other words, a solution of an nth-order ordinary differential equation (4)

is a function φ that possesses at least n derivatives and for which

F (x, φ(x), φ0 (x), . . . , φ(n) (x)) = 0 for all x in I.

We say that φ satisfies the differential equation on I. For our purposes

we shall also assume that a solution φ is a real-valued function. In our
2 dy
introductory discussion we saw that y = e0.1x is a solution of dx = 0.2xy on
the interval (−∞, ∞).
Occasionally, it will be convenient to denote a solution by the alternative
symbol y(x).
Families of Solutions: The study of differential equations is similar to
that of integral calculus. In some texts a solution φ is sometimes referred
to as an integral of the equation, and its graph is called an integral curve.
When evaluating an anti-derivative or indefinite integral in calculus, we use
a single constant c of integration.Analogously, when solving a first-order dif-
ferential equation F (x, y, y 0 ) = 0, we usually obtain a solution containing
a single arbitrary constant or parameter c. A solution containing an ar-
bitrary constant represents a set G(x, y, c) = 0 of solutions called a one-
parameter family of solutions. When solving an nth-order differential equa-
tion F (x, y, y 0 , . . . , y (n) ) = 0, we seek an n-parameter family of solutions

G(x, y, c1 , c2 , . . . , cn ) = 0.

This means that a single differential equation can possess an infinite num-
ber of solution corresponding to the unlimited number of choices for the
parameter(s). A solution of a differential equation that is free of arbitrary
parameters is called a particular solution.

Example 6.3.1 The one-parameter family y = cx − xcosx is an explicit

solution of the linear first-order equation xy 0 − y = x2 sinx on the interval
(−∞, ∞). (Verify.)

118
 Figure 6.3.1 shows the graphs of some particular solu-
tions in this family for various choices of c. The solution
y = −xcosx, the blue graph in the figure, is a particular
solution corresponding to c = 0.
Figure 6.1: Solution of
DE in Example 6.3.1

6.4 Initial value problem

INTRODUCTION We are often interested in problems in which we seek a
solution y(x) of a differential equation so that y(x) also satisfies certain pre-
scribed side conditionsthat is, conditions that are imposed on the unknown
function y(x) and its derivatives at a point x0 . On some interval I contain-
ing x0 the problem of solving an nth-order differential equation subject to n
side conditions specified at x0 :
dn y
Solve: n
= f (x, y, y 0 , . . . , y (n−1) ) (1)
dx
Subject to: y(x0 ) = y0 , y 0 (x0 ) = y1 , . . . , y (n−1) (x0 ) = yn−1
where y0 , y1 , . . . , yn−1 are arbitrary real constants, is called an nth-order
initial-value problem (IVP). The values of y(x) and its first n − 1 deriva-
tives at x0 , y(x0 ) = y0 , y 0 (x0 ) = y1 , . . . , y(n − 1)(x0 ) = yn−1 are called initial
conditions (IC). Solving an nth-order initial-value problem such as (1) fre-
quently entails first finding an n-parameter family of solutions of the given
differential equation and then using the initial- conditions at x0 to determine
the n constants in this family. The resulting particular solution is defined on
some interval I containing the initial point x0 .
Geometric Interpretation of IVPs: The cases n = 1 and n = 2 in (1),
dy
Solve: = f (x, y) (2)
dx
Subject to: y(x0 ) = y0
and
d2 y
Solve: = f (x, y, y 0 ) (3)
dx2
Subject to: y(x0 ) = y0 , y 0 (x0 ) = y1
are examples of first and second-order initial-value problems, respectively.
These two problems are easy to interpret in geometric terms. For (2) we are

119
Figure 6.2: Solution curve of first-orde Figure 6.3: Solution curve of second-
IVP orde IVP

seeking a solution y(x) of the differential equation y 0 = f (x, y) on an interval

I containing x0 so that its graph passes through the specified point (x0 , y0 ).
A solution curve is shown in blue in Figure 6.1. For (3) we want to find
a solution y(x) of the differential equation y 00 = f (x, y, y 0 ) on an interval I
containing x0 so that its graph not only passes through the point (x0 , y0 ) but
the slope of the curve at this point is the number y1 . A solution curve is
shown in blue in Figure 6.2. The words initial conditions derive from physical
systems where the independent variable is time t and where y(t0 ) = y0 and
y 0 (t0 ) = y1 represent the position and velocity, respectively, of an object at
some beginning, or initial, time t0 .
dy 2
−cosxsinx
Example 6.4.1 Solve dx = xy y(1−x 2 , y(0) = 2.
SOLUTION:
By writing the differential equation in the form

(cosxsinx − xy 2 )dx + y(1 − x2 )dy = 0,

we recognize that the equation is exact because

∂M ∂N
= −2xy = .
∂y ∂x
∂f
Now = y(1 − x2 )
∂y
y2
f (x, y) = (1 − x2 ) + h(x)
2
∂f
= −xy 2 + h0 (x) = cosxsinx − xy 2 .
∂x
The last equation implies that h0 (x) = cosxsinx. Integrating gives
Z
1
h(x) = − (cosx)(−sinx)dx = − cos2 x.
2

120
 y2 1
Thus (1−x2 )− cos2 x = c1 or y 2 (1−x2 )−cos2 x = c (7)
2 2
where 2c1 has been replaced by c. The initial condition y = 2 when x = 0
demands that 4(1) − cos2 (0) = c, and so c = 3. An implicit solution of the
problem is then
y 2 (1 − x2 ) − cos2 x = 3.

The solution curve of the IVP is the curve drawn in blue

in Figure 2.4.1; it is part of an interesting family of curves.
The graphs of the members of the one parameter family
of solutions given in (7) can be obtained in several ways,
two of which are using software to graph level curves (as
discussed in Section 6.6) and using a graphing utility to
carefully graph the explicit functions obtained for various
Figure 6.4: Solution values of c by solving
curves of DE in Example (c + cos2 x)
6.4.1 y2 = for y.
(1 − x2 )

6.5 Existence of a unique solution (Picards

theorem)
The general first-order ODE is
y 0 = F (x, y) , y(x0 ) = y0 (1)
We are interested in the following questions:
(i) Under what conditions can we be sure that a solution to (1) exists?
(ii) Under what conditions can we be sure that there is a unique solution
to (1)?
Here are the answers.

Theorem 1

(Existence).

121
 Suppose that F (x, y) is a continuous function defined in some region

R = {(x, y) : x0 − δ < x < x0 + δ, y0 −  < y < y0 + }

containing the point (x0 , y0 ). Then there exists a number δ1 (possibly

smaller than δ) so that a solution y = f (x) to (1) is defined for

x 0 − δ1 < x < x 0 + δ1 .

Theorem 2

(Uniqueness)
Suppose that both F (x, y) and ∂F
∂y
(x, y) are continuous functions defined
on a region R as in Theorem 1. Then there exists a number δ2 (possibly
smaller than δ1 ) so that the solution y = f (x) to (1), whose existence
was guaranteed by Theorem 1, is the unique solution to (1) for

x 0 − δ2 < x < x 0 + δ2

Figure 6.5: Region containing the solution

For a real number x and a positive value δ, the set of numbers x satisfying
x0 − δ < x < x0 + δ is called an open interval centered at x0 .
Example 6.5.1 Consider the ODE
y 0 = x − y + 1, y(1) = 2.

Solution: In this case, both the function F (x, y) = x − y + 1 and its partial

122
derivative ∂F
∂y
(x, y) = −1 are defined and continuous at all points (x, y). The
theorem guarantees that a solution to the ODE exists in some open interval
centered at 1, and that this solution is unique in some (possibly smaller)
interval centered at 1. In fact, an explicit solution to this equation is

y(x) = x + e1−x .

(Check this for yourself.) This solution exists (and is the unique solution to
the equation) for all real numbers x. In other words, in this example we may
choose the numbers δ1 and δ2 as large as we please.

Example 6.5.2 Discuss the existence and unique solution for the IVP
2y
y0 = , y(x0 ) = y0
x
Solution: Here f (x, y) = 2y/x and ∂f /∂y = 2/x. Clearly both of these
exist and bounded around (x0 , y0 ) if x0 = 0. Hence, unique solution exists
in a interval about x0 for x0 = 0. For x0 = 0, nothing can be said from
the existence and uniquness theorem. Fortunately, we can solve the actual
problem and find y = Ax2 to be the general solution. When x0 = 0, there
exists no solution when y0 = 0. If y0 = 0, then we have infinite number of
solutions y = αx2 (α any real number) that satisfy the IVP

y = 2y/x, y(0) = 0.

Theorem 3

Picards Existence and Uniqueness Theorem

Consider the Initial Value Problem (IVP)

y 0 = f (x, y), y(x0 ) = y0

. Suppose f (x, y) and ∂f∂y

(x, y) are continuous functions in some open
rectangle R = {(x, y) : a < x < b, c < y < d} that contains the point
(x0 , y0 ). Then the IVP has a unique solution in some closed interval
I = [x0 − h, x0 + h], where h > 0. Moreover, the Picard iteration defined
by Z x
yn+1 (x) = y0 + f (t, yn (t))dt
x0

produces a sequence of functions {yn (x)} that converges to this solution

123
 uniformly on I.

Example 6.5.3 Apply Picard’s iteration for the IVP

y 0 = 2x(1 − y) , y(0) = 2

Solution: Here y0 (x) = 2 Now

Z x Z x
y1 (x) = y0 + f (t, y0 (t))dt = 2 + 2t(1 − 2)dt = 2 − x2
0 0

x x
x4
Z Z
y2 (x) = y0 + f (t, y1 (t))dt = 2 + 2t(1 − (2 − t2 ))dt = 2 − x2 +
0 0 2
x x
x2 x4 x6
Z Z
y3 (x) = y0 + f (t, y2 (t))dt = 2+ 2t(1−(2−x2 + ))dt = 2−x2 + −
0 0 2 2 3!
x
x4 x6 x8
Z
y4 (x) = 2 + f (t, y3 (t))dt = 2 − x2 + − +
0 2 3! 4!
By induction it can be shown that

x4 x6 x2n
2 − x2 + − + . . . + (−1)n
2 3! n!
2 2
Hence ,yn (x) → 1+e−x as x → ∞. Now y(x) = 1+e−x is the exact solution
of the given IVP. Thus ,the Picard’s iteration converges to unique solution
of the given IVP.
Comment: Picard iteration has more theoretical value than practical
value. It is used in the proof of existence and uniqueness theorem. On
the other hand, finding approximate solution using this method is almost
impractical for complicated function f (x, y)

6.6 Method of separable of variables

Many practically useful ODEs can be reduced to the form

g(y)y 0 = f (x) (1)

by purely algebraic manipulations. Then we can integrate on both sides with

respect to x,obtaining
Z Z
0
g(y)y dx = f (x)dx + c. (2)

124
On the left we can switch to y as the variable of integration. By calculus,
y 0 dx = dy , so that
Z Z
g(y)dy = f (x)dx + c. (3)

If f and g are continuous functions, the integrals in (3) exist, and by

evaluating them we obtain a general solution of (1). This method of solving
ODEs is called the method of separating variables, and (1) is called a sep-
arable equation, because in (3) the variables are now separated: x appears
only on the right and y only on the left.

Example 6.6.1 Separable ODE

The ODE y 0 = 1 + y 2 is separable because it can be written
dy
= 1+y 2 By integration, arctan y = x+c or y = tan (x+c).
dx
It is very important to introduce the constant of integration immediately when
the integration is performed. If we wrote arctany = x, then y = tanx, and
then introduced c, we would have obtained y = tanx + c, which is not a
solution (when c 6= 0 ). Verify this.

Example 6.6.2 The ODE y 0 = (x + 1)e−x y 2 is separable; we obtain

y −2 dy = (x + 1)e−x dx.
1
By integration, − y −1 = −(x + 2)e−x + c, ⇒ y=
(x + 2)e−x − c

6.7 Homogeneous equations

If a function f possesses the property f (tx, ty) = tα f (x, y) for some real
number α , then f is said to be a homogeneous function of degree α. For
example, f (x, y) = x3 + y 3 is a homogeneous function of degree 3, since

f (tx, ty) = (tx)3 + (ty)3 = t3 (x3 + y 3 ) = t3 f (x, y),

whereas f (x, y) = x3 + y3 + 1 is not homogeneous. A first-order DE in

differential form

M (x, y)dx + N (x, y)dy = 0 (1)

125
 is said to be homogeneous if both coefficient functions M and N are ho-
mogeneous functions of the same degree. In other words, (1) is homogeneous
if
M (tx, ty) = tα M (x, y) and N (tx, ty) = tα N (x, y).
In addition, if M and N are homogeneous functions of degree α, we can also
write
y
M (x, y) = xα M (1, u) and N (x, y) = xα N (1, u), where u = (2)
x
, and

M (x, y) = y α M (v, 1) and N (x, y) = y α N (v, 1), (3)

where v = xy . Properties (2) and (3) suggest the substitutions that can
be used to solve a homogeneous differential equation. Specificall , either
of the substitutions y = ux or x = vy, where u and v are new dependent
variables, will reduce a homogeneous equation to a separable first-orde dif-
ferential equation. To show this, observe that as a consequence of (2) a
homogeneous equation M (x, y)dx + N (x, y)dy = 0 can be rewritten as

xα M (1, u)dx + xα N (1, u)dy = 0 or M (1, u)dx + N (1, u)dy = 0,

where u yc or y = ux. By substituting the differential dy = udx + xdu into the

last equation and gathering terms, we obtain a separable DE in the variables
u and x:
M (1, u)dx + N (1, u)[udx + xdu] = 0
[M (1, u) + uN (1, u)]dx + xN (1, u)du = 0

dx N (1, u)du
or + =0
x M (1, u) + uN (1, u)
At this point we offer the same advice as in the preceding sections: Do not
memorize anything here (especially the last formula); rather, work through
the procedure each time. The proof that the substitutions x = vy and
dx = vdy + ydv also lead to a separable equation follows in an analogous
manner from (3).

Example 6.7.1 Solve (x2 + y 2 )dx + (x2 − xy)dy = 0.

Solution: Inspection of M (x, y) = x2 + y 2 and N (x, y) = x2 − xy shows

that these coefficients are homogeneous functions of degree 2. If we let y = ux,
then dy = udx + xdu, so after substituting, the given equation becomes

126
 (x2 + u2 x2 )dx + (x2 − ux2 )[udx + xdu] = 0
x2 (1 + u)dx + x3 (1 − u)du = 0
1−u dx
du + =0
1+u x
2 dx
[−1 + ]du + = 0. ← long division
1+u x
After integration the last line gives

−u + 2 ln|1 + u| + ln|x| = ln|c|

y y y
− + 2 ln |1 + | + ln|x| = ln|c| ← resubstituting u =
x x x
Using the properties of logarithms, we can write the preceding solution as

(x + y)2 y
ln| |= or (x + y)2 = cxey/x
cx x
.

Although either of the indicated substitutions can be used for every homoge-
neous differential equation, in practice we try x = vy whenever the function
M (x, y) is simpler than N (x, y). Also it could happen that after using one
substitution, we may encounter integrals that are difficult or impossible to
evaluate in closed form; switching substitutions may result in an easier prob-
lem.

6.8 Exact equations, non-exact equations and

integrating factor
We recall from calculus that if a function u(x, y) has continuous partial
derivatives, its differential (also called its total differential) is

∂u ∂u
du = dx + dy
∂x ∂y

From this it follows that if u(x, y) = c =const, then du = 0. For example, if

u = x + x2 y 3 = c, then

du = (1 + 2xy 3 )dx + 3x2 y 2 dy = 0

127
or
dy 1 + 2xy 3
y0 = =
dx 3x2 y 2
an ODE that we can solve by going backward. This idea leads to a powerful
solution method as follows. A first-order ODE M (x, y) + N (x, y)y 0 = 0,
written as (use dy = y 0 dx as in Sec. 1.3)

M (x, y)dx + N (x, y)dy = 0 (1)

is called an exact differential equation if the differential form M (x, y)dx +

N (x, y)dy is exact, that is, this form is the differential
∂u ∂u
du = dx + dy (2)
∂x ∂y
of some function u(x, y). Then (1) can be written du = 0. By integration
we immediately obtain the general solution of (1) in the form

u(x, y) = c. (3)

This is called an implicit solution, in contrast to a solution y = h(x) , which

is also called an explicit solution, for distinction. Sometimes an implicit
solution can be converted to explicit form. (Do this for x2 + y 2 = 1. ) If
this is not possible, your CAS may graph a figure of the contour lines (3) of
the function u(x, y) and help you in understanding the solution. Comparing
(1) and (2), we see that (1) is an exact differential equation if there is some
function u(x, y) such that
∂u ∂u
(a) ∂x
=N (b) ∂y
=M (4)
From this we can derive a formula for checking whether (1) is exact or not,
as follows. Let M and N be continuous and have continuous first partial
derivatives in a region in the xy-plane whose boundary is a closed curve
without self-intersections. Then by partial differentiation of (4) ,
∂M ∂ 2u
=
∂y ∂y∂x
∂N ∂ 2u
=
∂y ∂x∂y
By the assumption of continuity the two second partial derivaties are
equal. Thus
∂M ∂N
= (5)
∂y ∂x

128
This condition is not only necessary but also sufficient for (1) to be an exact
differential equation. If (1) is exact, the function u(x, y) can be found by in-
spection or in the following systematic way. From (4a) we have by integration
with respect to x
Z
u = M dx + k(y) (6)

in this integration, y is to be regarded as a constant, and k(y) plays the role

of a constant of integration. To determine k(y) , we derive ∂u/∂y from (6),
use (4b) to get dk/dy , and integrate dk/dy to get k. (See Example 1, below.)
Formula (6) was obtained from (4a). Instead of (4a) we may equally well use
(4b). Then, instead of (6), we first have by integration with respect to y
Z
u = N dy + l(x) (6∗ )

To determine l(x) , we derive ∂u/∂x from (6∗ ), use (4a) to get dl/dx , and
integrate. We illustrate all this by the following typical examples.

Example 6.8.1 Solve

cos(x + y)dx + (3y 2 + 2y + cos(x + y))dy = 0. (7)
Solution:

• Step 1. Test for exactness. Our equation is of the form (1) with
M = cos(x + y), N = 3y 2 + 2y + cos(x + y).
Thus
∂M ∂N
= −sin(x + y) , = −sin(x + y)
∂y ∂x
From this and (5) we see that (7) is exact.
• Step 2. Implicit general solution. From (6) we obtain by integration

Z Z
u= M dx+k(y) = cos(x+y)dx+k(y) = sin(x+y)+k(y). (8)

To find k(y) , we differentiate this formula with respect to y and use

formula (4b), obtaining
∂u dk
= cos(x + y) + = N = 3y 2 + 2y + cos(x + y).
∂y dy

129
 Hence dk/dy = 3y 2 + 2y. By integration, k = y 3 = y 2 + c ∗ . Inserting
this result into (8) and observing (3), we obtain the answer

u(x, y) = sin(x + y) + y 3 + y 2 = c.

• Step 3. Checking an implicit solution. We can check by differentiating

the implicit solution u(x, y) = c implicitly and see whether this leads to
the given ODE (7):
∂u ∂u
du = dx+ dy = cos(x+y)dx+(cos(x+y)+3y 2 +2y)dy = 0. (9)
∂x ∂y
This completes the check.

Example 6.8.2 The simplest non-exact equation:

y 0 = y or equivalently − ydx + dy = 0
We check simply
∂M ∂N
= −1 6= 0 =
∂y ∂x
But it can be easily solved!
• Approach 1: Remember that (ex )0 = ex itself. Then realize after a
while that this is also true for Cex for any constant C. As we have one
arbitrary constant now, the general solution

y = cex

• Approach 2: Write the equation as

y0 − y = 0

Recall that
(e−x y)0 = −e−x y + e−x y 0 = e−x (y 0 − y)
Therefore multiplying the equation by e−x makes the left hand side a total
derivative:

(e−x y)0 = e−x (y 0 − y) ⇒ e−x y = C ⇒ y = Ce−x

It is clear that the 2nd approach is much more systematic

integrating factor:

130
Definition 6.2 If the differential equation M (x, y)dx + N (x, y)dy = 0 is not
exact but the differential equation

µ(x, y)M (x, y)dx + µ(x, y)N (x, y)dy = 0

is exact,then the multiplicative function µ(x, y) is called an integrating factor

of the DE.

Example 6.8.3 The differential equation ydx + 2xdy = 0 is not exact.But

if we multiply this equation by y the equation changed to exact equation.That
is
y 2 dx + 2xydy
is exact since
∂y 2 ∂2xy
= 2y =
∂y ∂x

6.9 Orthogonal trajectories

An important type of problem in physics or geometry is to find a family
of curves that intersects a given family of curves at right angles. The new
curves are called orthogonal trajectories of the given curves (and conversely).
Examples are curves of equal temperature (isotherms) and curves of heat
flow, curves of equal altitude (contour lines) on a map and curves of steepest
descent on that map, curves of equal potential (equipotential curves, curves of
equal voltagethe ellipses in Fig.6.6) and curves of electric force (the parabolas
in Fig. 6.6). Here the angle of intersection between two curves is defined to
be the angle between the tangents of the curves at the intersection point.
Orthogonal is another word for perpendicular. In many cases orthogonal
trajectories can be found using ODEs. In general, if we consider G(x, y, c) = 0
to be a given family of curves in the xy-plane, then each value of c gives a
particular curve. Since c is one parameter, such a family is called a one-
parameter family of curves. In detail, let us explain this method by a family
of ellipses
1 2
x + y2 = c (c > 0) (1)
2
and illustrated in Fig. 6.6. We assume that this family of ellipses represents
electric equipotential curves between the two black ellipses (equipotential
surfaces between two elliptic cylinders in space, of which Fig. 6.6 shows a
cross-section). We seek the orthogonal trajectories, the curves of electric
force. Equation (1) is a one-parameter family with parameter c. Each value

131
of (c > 0) corresponds to one of these ellipses. Step 1. Find an ODE for
which the given family is a general solution. Of course, this ODE must no
longer contain the parameter c. Differentiating (1), we have x + 2yy 0 = 0.
Hence the ODE of the given curves is
x
y 0 = f (x, y) = − (2)
2y
Step 2. Find an ODE for the orthogonal trajectories yy = y(x). This ODE

Figure 6.6: Electrostatic field between two ellipses (elliptic cylinders in space):
Elliptic equipotential curves (equipotential surfaces) and orthogonal trajectories
(parabolas)

is
1 2y
y0 = − =+ (3)
f (x, y) x
with the same f as in (2). Why? Well, a given curve passing through a point
(x0 , y0 ) has slope f (x0 , y0 ) at that point, by (2). The trajectory through
(x0 , y0 ) has slope − f (x01,y0 ) by (3). The product of these slopes is −1 , as we
see. From calculus it is known that this is the condition for orthogonality
(perpendicularity) of two straight lines (the tangents at (x0 , y0 ) ), hence of
the curve and its orthogonal trajectory at (x0 , y0 ). Step 3. Solve (3) by
separating variables, integrating, and taking exponents:
dy dx
=2 , ln|y| = 2 ln x + c, y = c ∗ x2 .
y x
This is the family of orthogonal trajectories, the quadratic parabolas along
which electrons or other charged particles (of very small mass) would move
in the electric field between the black ellipses (elliptic cylinders).

132
Chapter 7

Numerical Methods for Initial

Value Problems

INTRODUCTION
The most general form of an ordinary differential equation of nth order is
given by
dy d2 y dn y
φ(x, y, , 2 , . . . , n ) = 0. (1)
dx d x d x
A general solution of an ordinary differential equation such as (1) is a relation
between y, x and n arbitrary constants which satisfies the equation and it is
of the form

f (x, y, c1 , c2 , . . . , cn ) = 0 (2)

If particular values are given to the constants c1 , c2 , . . . , cn , then the resulting

solution is called a Particular solution. To obtain a particular solution from
the general solution given by (2), we must be given n conditions so that the
constants can be determined. If all the n conditions are specified at the same
value of x, then the problem is termed as initial value problem. Though
there are many analytical methods for finding the solution of the equation
form given by (1), there exist large number of ordinary differential equations,
whose solution cannot be obtained by the known analytical methods. In such
cases we use numerical methods to get an approximate solution of a given
differential equation under the prescribed initial condition. In this chapter we
restrict ourselves and develop the numerical methods for findings a solution
of an ordinary differential equation of first order and first degree which is of
the form
dy
= f (x, y) (3)
dx

133
with the initial condition y(x0 ) = y0 , which is called initial value problem.
The general solutions of equation (3) will be obtained in two forms:

(1) the values of y as a power series in independent variable x and

(2) as a set of tabulated values of x and y.

We shall now develop numerical methods for solution of the initial value
problem of the form given by (3). We partition the interval [a, b] on which
the solution is derived in finite number of sub- intervals by the points

a = x0 < x1 , < x2 , . . . < xn = b.

The points are called Mesh Points. We assume that the points are spaced
uniformly with the relation

xn = x0 + nh.

The existence of uniqueness of the solution of an initial value problem in

[x0 , b] depends on the theorem due to Lipschitz, which states that:

1. If f (x, y) is a real function defined and continuous in [x0 , b], y ∈ (−∞, +∞),
where x0 , and b are finite.

2. There exists a constant L > 0 called Lipschitzs constant such that for
any two values y = y1 and y = y2

|f (x, y1 ) − f (x, y2 )| < L|L1 − L2 |

where x ∈ [x0 , b], then for any y(x0 ) = y0 , the initial value problem
(3), has unique solution for x ∈ [x0 , b].

Now we shall discuss the Taylors series method and the Eulers method.

7.1 Taylors method of order n

Let y = f (x), be a solution of the equation
dy
= f (x, y)
dx
with y(x0 ) = y0 . Expanding it by Taylors series about the point x0 , we get

x − x0 0 (x − x0 )2 00
f (x) = f (x0 ) + f (x0 ) + f (x0 ) + · · ·
1! 2!
134
this may be written as

x − x0 0 (x − x0 )2 00
y = f (x) = y0 + y0 + y0 + · · ·
1! 2!
Putting x = x1 = x0 + h, we get

h 0 h2 00 h3
f (x1 ) = y1 = y0 + y + y + + ··· (4)
1! 0 2! 0 3!
Similarly we obtain

h 0 h2 h3
yn+1 = yn + yn + yn00 + + ··· (5)
1! 2! 3!
Equation (5) may be written as

h 0 h2
yn+1 = yn + yn + yn00 + O(h3 ) (6)
1! 2!
where O(h3 ) means that all the succeeding terms containing the third and
higher powers of h. If the terms containing the third and higher powers of h
are neglected then the local truncation error in the solution is kh3 where k
is a constant. For a better approximation terms containing higher powers of
h are considered.

Note: Taylors series method is applicable only when the various deriva-
tives of f (x, y) exist and the value of (x − x0 ) in the expansion of y = f (x)
near x0 must be very small so that the series converge.
dy
Example 7.1.1 Solve dx = x + y, y(1) = 0, numerically up to x = 1.2,
with h = 0.1.
Solution: We have x0 = 1, y0 = 0 and
dy
= y 0 = x + y ⇒ y00 = x0 + y0 = 1 + 0 = 1
dx
d2 y
= y 00 = 1 + y 0 ⇒ y000 = 1 + y00 = 1 + 1 = 2
dx2
d3 y
= y 000 = y 00 ⇒ y0000 = y000 = 2
dx3
d4 y (4)
4
= y (4) = y 000 ⇒ y0 = y0000 = 2
dx

135
 d5 y
= y (5) = y (4) ⇒ y (5) = y (4) = 2
dx5
..
.
Substituting the above values in

h 0 h2 00 h3 000 h4 (4) h5 (5)

y1 = y0 + y + y + y0 + y0 + y + · · ·
1! 0 2! 0 3! 4! 5!
we get

0.1 (0.1)2 (0.1)3 (0.1)4 (0.1)5

y1 = 0 + 1+ 2+ 2+ 2+ 2 + ···
1 2 6 24 120
⇒ y1 = 0.11033847
∴ y1 = y(0.1) ≈ 0.110
Now
x1 = x0 + h = 1 + 0.1 = 1.1
we have
y10 = x1 + y1 = 1.1 + 0.110 = 1.210
y100 = 1 + y10 = 1 + 1.21 = 2.21
y1000 = y100 = 2.21
(4)
y1 = y1000 = 2.21
(5) (4)
y1 = y1 = 2.21
..
.
Substituting the above values in (1), we get

h 0 h2 00
y2 = y1 + y + y + ···
1! 1 2! 1
0.1 (0.1)2 (0.1)3 (0.1)4 (0.1)5
⇒ y2 = 0.11 + 1.21 + 2.21 + + 2.21 + 2.21 + · · ·
1 2 6 24 120
∴ y2 = 0.24205
∴ y(0.2) = 0.24205

136
 dy
Example 7.1.2 Given dx = 1 + xy with the initial condition that y = 1, x =
0. Compute y(0.1) correct to four places of decimal by using Taylors series
method.
dy
Solution: Given dx
= 1 + xy and y(0) = 1

∴ y 0 (0) = 1 + 0 × 1 = 1

Differentiating the given equation w.r.t. x, we get

d2 y dy
= y + x
dx2 dx
y000 = 1 + 0 × 1 = 1 + 0 = 1
Similarly
d3 y dy dy d2 y dy d2 y
= + + x = 2 + x
dx3 dx dx dx2 dx dx2
⇒ y0000 = 2
and
d4 y dy 3 d2 y
= x + 3
dx4 dx3 dx2
(4)
⇒ y0 = 3
from Taylors series method, we have
h2 00 h3 000 h4 (4)
y1 = 1 + hy00 + y + y0 + y0 + · · ·
2 0 6 24
0.12 0.13 0.14
∴ y(0.1) = 1 + (0.1)1 + (1) + (2) + (3) + · · · = 1.1053425
2 6 24
∴ y(0.1) = 1.1053
correct to four decimal places.

Example 7.1.3 Apply the Taylors series method to find the value of y (1.1)
dy 1
and y (1.2) correct to three decimal places given that dx = xy 3 , y(1) = 1
taking the first three terms of the Taylors series expansion.

Solution Given
dy 1
= xy 3 , y0 = 1, x0 = 1 and h = 0.1
dx
1 1
⇒ y00 = x0 y03 = (1)(1) 3 = 1

137
differentiating the given equation w.r.t. x we get

d2 y 1 −2 dy 1

2
= xy 3 + y3
dx 3 dx
1 −2 1 1 1 −1 1
= xy 3 (xy 3 ) + y 3 = x2 y 3 + y 3
3 3
1 4
⇒ y000 = (1)(1) + 1 =
3 3
Taking the first three terms of the Taylors formula we get

h2 00
y1 = y0 + hy00 + y (7)
2 0
substituting the values in (7)

0.12 4
y1 = y(1.1) = 1 + (0.1)(1) + = 1.1066
2 3
∴ y(1.1) = 1.1066
x1 = x0 + h = 1 + 0.1 = 1.1
1 1
⇒ y10 (1.1) = xy 3 = (1.1)(1.1066) 3 = 1.138
1 −1 1
y100 = x21 y1 3 + y13
3
1 1 1
y100 (1.1) = (1.1)2 (1.1066)− 3 + (1.1066) 3
3
= 1.4249
Substituting in
h2 00
y2 = y1 + hy10 + y
2 1
we get
0.12
y2 = y(1.2) = 1.1066 + 0.11138 + × 1.4249
2
= 1.2275245 ≈ 1.228

∴ y2 = y(1.2) = 1.228
dy 1
Exercises 7.1 Given dx = x2 +y
, y(4) = 4 find y(4.2) by Taylors series
method, taking h = 0.1.

138
 dy
Exercises 7.2 Solve dx = 3x + y 2 , y = 1, when x = 0, numerically up to
x = 0.1 by Taylors series method.

Exercises 7.3 Apply Taylors algorithm to y 0 = x2 + y 2 , y(0) = 1. Take

h = 0.5 and determine approximations to y(0.5). Carry the calculations up
to 3 decimals.

Exercises 7.4 Solve y 0 = ysinx + cosx, subject to x = 0, y = 0 by Taylors

series method.

7.2 Eulers Method

Consider the first order and first degree differential equation
dy
= f (x, y) refer (3)
dx
with the condition that y(x0 ) = y0 . Suppose we want to find the approximate
value of y say yn when x = xn . We divide the interval [x0 , xn ] into nsubin-
tervals of equal length say h, with the division point x0 , x1 , . . . , xn , where
xr = x0 + rh, (r = 1, 2, . . . , n). Let us assume that b g d

f (x, y) ≈ f (xr−1 , yr−1 )

in [xr−1 , xr ]. Integrating (3) in [xr−1 , xr ], we get

Z xr Z xr
dy = f (x, y)dx
xr−1 xr−1
Z xr
⇒ [yr − yr−1 ] = f (x, y)dx
xr−1
Z xr
⇒ yr ≈ yr−1 + f (xr−1 , yr−1 ) dx
xr−1

⇒ yr ≈ yr−1 + f (xr−1 , yr−1 )(xr − xr−1 )

∴ yr ≈ yr−1 + hf (xr−1 , yr−1 ) (8)
Equation (8) is called Eulers iteration formula. Taking r = 1, 2, ..., n in
(8), we get the successive approximately of y as follows

y1 = y(x1 ) = y0 + hf (x0 , y0 )

y2 = y(x2 ) = y1 + hf (x1 , y1 )

139
 ..
.
yn = y(xn ) = yn−1 + hf (xn−1 , yn−1 )
Note: Eulers method has limited usage because of the large error that is
accumulated as the process proceeds. The process is very slow and to obtain
reasonable accuracy with Eulers method we have to take a smaller value of h.
Further, the method should not be used for a larger range of x as the values
found by this method go on becoming farther and farther away from the true
values. To avoid this difficulty one can choose Eulers modified method to
solve the equation (3).
dy
Example 7.2.1 Solve the equation dx = 1 − y, with the initial condition
x = 0, y = 0, using Eulers algorithm and tabulate the solutions at x =
0.1, 0.2, 0.3.
dy
Solution Given dx = 1 − y , with the initial condition x = 0, y = 0

∴ f (x, y) = 1 − y

h = 0.1
we have
∴ x0 = 0, y0 = 0
x1 = x0 + h = 0 + 0.1 = 0.1
x2 = 0.2, x3 = 0.3
Taking n = 0 in
yn+1 = yn + hf (xn , yn )
y1 = y0 + hf (x0 , y0 ) = 0 + 0.1(1 − 0) = 0.1
we get ∴ y1 = 0.1 , i . e ., y(0.1) = 0.1,

y2 = y1 + hf (x1 , y1 )
y2 = 0.1 + 0.1(1 − 0.1) = 0.19
∴ y2 = y(0.2) = 0.19
y3 = y2 + hf (x2 , y2 )
⇒ y3 = 0.19 + (0.1)(1 − y2 )
= 0.19 + (0.1)(0.81) = 0.271
∴ y3 = y(0.3) = 0.271.

140
 x Solution by Eulers method
0 0
0.1 0.1
0.2 0.19
0.3 0.271
dy
Example 7.2.2 Given dx
= x3 + y, y(0) = 1, compute y(0.2) by Eulers
method taking h = 0.01.

Solution:Given
dy
= x3 + y
dx
with the initial condition y(0) = 1. We have
f (x, y) = x3 + y
x0 = 0, y0 = 1, h = 0.01
x1 = x0 + h = 0 + 0.01 = 0.01,
x2 = x0 + 2h = 0 + 2(0.01) = 0.02.
Applying Eulers formula we get
y1 = y0 + hf (x0 , y0 )
y1 = 1 + (0.01)(x30 + y0 )
= 1 + (0.01)(03 + 1) = 1.01
∴ y1 = y(0.01) = 1.01

y2 = y1 + hf (x1 , y1 )
= 1.01 + (0.01)[x31 + y1 ]
= 1.01 + (0.01)[(0.01)3 + 1.01] = 1.0201

∴ y2 = y(0.02) = 1.0201
Exercises 7.5 Solve by Eulers method the following differential equation x
dy
= 0.1 correct to four decimal places dx = y−x
x−y
with the initial condition y(0)
= 1.
Exercises 7.6 Find y(1) by Eulers method from the differential equation
dy −y
dx
= − 1+x , when y(0.3) = 2. Convert up to four decimal places taking step
length h = 0.1.
Exercises 7.7 Solve y 0 = x − y 2 , by Eulers method for x = 0.2 to 0.6 with
h = 0.2 initially x = 0 , y = 1.

141
7.3 Eulers modified method
From Eulers iteration formula we h known that

yr ≈ yr−1 + hf (xr−1 , yr−1 ) (9)

Let y(xr ) = yr denote the initial value using (9) an approximate value of yr0
can be calculated Z xr
(0)
yr = yr−1 + f (x, y)dx
xr−1

⇒ yr(0) ≈ yr−1 + hf (xr−1 , yr−1 )

replacing f (x, y) by f (xr−1 , yr−1 ) in xr−1 ≤ x < xr using Trapezoidal rule in
[xr−1 , xr ], we can write
h
yr(0) = yr−1 + [f (xr−1 , yr−1 ) + f (xr , yr )].
2
(0)
Replacing f (xr , yr ) by its approximate value f (xr , yr ) at the end point of
the interval [xr−1 , xr ], we get
h
yr(1) = yr−1 + [f (xr−1 , yr−1 ) + f (xr , yr(0) )].
2
(1)
where yr is the first approximation to yr = y(xr ) proceeding as above we
get the iteration formula
h
yr(n) = yr−1 + [f (xr−1 , yr−1 )+f (xr , yr(n−1) )]. (11)
2
where yrn denoted the nth approximation to yr we have
h
yr ≈ yr(n) = yr−1 + [f (xr−1 , yr−1 ) + f (xr , yr(n−1) )].
2
Example 7.3.1 Solve the Eulers modified method the following differential
dy
equation for x = 0.02 by taking h = 0.01 given dx = x2 + y , y = 1, when
x = 0.

Solution Here we have

f (x, y) = x2 + y
h = 0.01, x0 = 0, y0 = y(0) = 1
x1 = 0.01, x2 = 0.02

142
we get
(0)
∴ y1 = y0 + hf (x0 , y0 )
= 1 + 0.01[x20 + y0 ] = 1 + 0.01[02 + 1] = 101
(0)
y1 = 1.01
Applying Eulers modified formula we get

(1) h (0)
y1 = y0 + [f (x0 , y0 ) + f (x1 , y1 )]
2
0.01 2
=1+ [0 + 1 + 0.012 + 1.01] = 1.01005
2
(1)
∴ y1 = 1.01005
(2) 2 (1)
y1 = y0 + [f (x0 , y0 ) + f (x1 , y1 ]
2
0.01 2
=1+ [0 + 1 + 0.012 + 1.01005] = 1.01005
2
(2)
⇒ y1 = 1.01005
(1) (2)
∴ y1 = y1 = 1.01005
(0)
y2 = y1 + hf (x1 , y1 )
= 1.01005 + 0.01(x21 + y1 )
= 1.01005 + 0.01((0.01)2 + 1.01005) = 1.02015
(1) h (0)
y2 = y1 + [f (x1 , y1 ) + f (x2 , y2 )]
2
0.01
= 1.01005 + [(0.01)2 + 1.01005 + (0.02)2 + 1.02015]
2
= 1.020204
(2) h (1)
y2 = y1 + [f (x1 , y1 ) + f (x2 , y2 )]
2
0.01
= 0.01 + [(0.01)2 + 1.01005 + (0.02)2 + 1.020204] = 1.020204
2
∴ y2 = y(0.02) = 1.020204

Exercises 7.8 Find y(4.4), by Eulers modified method taking h = 0.2 from
dy 2
the differential equation dx = 2−y
5x
, y = 1 when x = 4.

143
Exercises 7.9 Using Eulers modified method, solve numerically the equation
dy √
dx
= x + | y| with boundary condition y = 1 at x = 0 for the range 0 < x <
0.4 in steps of 0.2.
dy
Exercises 7.10 solve dx = 1y with y = 0 when x = 0 in the range 0 < x <
0.2 by taking h = 0.1 (apply Eulers modified formula).

144