MULLER METHOD

Working Procedure
2. Using the location theorem, we find ( x 0 , x 2 ) in which a root of the equation f(x)
= 0 lies. We may assume x0 , x1 , x 2 as the initial approximations to the root of
x  x2
f(x)=0 where x1  0 .
2
2. Compute f i  f ( xi ), i  0,1,2
2. Compute d i  xi 1  xi , i  0,1 . For the above choice of x0 , x1 , x 2 ; di = 0.5,
otherwise d 0  d1 .
2. Compute g i  ( f i 1  f i ) / d i , i = 0,1
2. Set i = 0
f ( xi  2, xi 1 )  f ( xi 1, xi ) ( g i 1  g i )
2. Compute hi  f ( xi  2, xi 1, xi ) = =
xi  2  xi d i 1  d i
2. Compute ci  g i 1  d i 1 hi
 2 f i2
2. Compute d i  2  choosing the proper sign in the denominator so
ci  ci2  4 f i  2 hi
as to make its numerical value greater.
2. Set xi 3  xi  2  d i  2
2. Compute f i 3  f ( xi 3 ) and g g i  2  ( f i 3  f i  2 ) / d i  2
2. set i  i  1 and repeat steps 5-10, until the required accuracy is got. i.e
xi 1  xi  .

Problem: Find the negative root of the equation x 3  4 x  1  0 , that lies between –3
and –2, correct to 4 places of decimals, using Muller’s method.

Solution: Let f ( x)  x 3  4 x  1  0, x0  3 and x 2  2 Hence x1  2.5

f 0  f (3)  - 14; f 1  4.625; f 2  1; d 0  0.5; d 1  0.5
f1  f 0
g0   18.75;
d0
f  f1
g1  2  11.25;
d1
gi  g0
h0   7.5;
d1  d 0
c0  g1  d 1 h0  7.5
2 f2 2 2
d2      0.11914
c 0  c 02  4 f 2 h0 7.5  9.28709 16.78709
x3  x 2  d 2  2.11914
f 3  f ( 2.11914 ) =-0.03998
g 2  ( f 3  f 2 ) / d 2  8.72906
we proceed to the second iteration
g  g1
h1  2  6.61907;
d 2  d1
c1  g 2  d 2 h1  9.51766
2 f3 0.07996
d3     0.00421
c1  c12  4 f 3 h1 9.51766  9.46189.
x 4  x3  d 3  2.11493
f 4  f (2.11493)  0.00021
g 3  ( f 4  f 3 ) / d 3  9.44656

we proceed to the third iteration

g3  g 2
h2   6.24293;
d3  d2
c 2  g 3  d 3 h2  9.18373
2 f4 0.00042
d4     0.00002
c 2  c  4 f 4 h2
2
2
9.18373  9.18344.
x5  x 4  d 4  2.11491
since x 4  2.11493 and x5  2.11491 , the required root, correct to 4 places of
decimals. x =  2.1149

BIRGE VIETA METHOD

Using this method, we can find a real root of a polynomial equation f(x)=0.

We will start with an initial approximation r0 to r and use Newton Raphson method to
improve the value of r such that R(r )=0.
If r1 is a close approximation to the root, then
f (r )
r1  r0  1 0
f ( r0 )
In B-V-M we will not compute f (r0 ) and f 1 ( r0 ) directly using f(x) and f’(x), but
compute them using synthetic division as

r0 a0 a1 a2  an2 a n 1 an
r0 b0 r0 b1  r0 bn 3 r0 bn  2 r0 bn 1
b0 b1 b2  b n2 bn 1 bn
r0 c 0 r0 c1  r0 c n 3 r0 c n  2
c0 c1 c2  cn2 c n 1
 Using r1 in the place of ro, proceeding as before, we get a new set of values for bn
and c n-1 and hence a better approximation for r is given by
b
r2  r1  n
c n 1
Thus the Birge Vieta formula provides the iterative formula
b where bn and c n-1 are obtained from above corresponding to rk.
rn 1  rk  n
c n 1
Problem:
Find the root of the equation x 3  2 x 2  10 x  20  0 that is near 1, using Birge Vieta
method , correct to 4 places of decimals.

Let us take r0 = 1

1 1 2 10  20
1 3 13
1 1 3 13  7  bn
1 4
1 4 17  c n 1

bn 7
r1  r0   1 =1.4118
c n 1 17

1.4118 1 2 10  20
1.4118 4.8168 20.9184
1.4118 1 3.4118 14.8168 0.9184  bn
1.4118 6.8100
1 4.8236 21.6268  c n 1

bn 0.9184
r2  r1   1.4118   1.3693
c n 1 21.6268

1.3693 1 2 10  20
1.3693 4.6136 20.0104
1.3693 1 3.3693 14.6136 0.0104  bn
1.3693 6.4886
1 4.7386 21.1022  c n 1

bn 0.0104
r3  r2   1.3693   1.3688
c n 1 21.1022
 1.3688 1 2 10  20
1.3688 4.6112 19.9998
1.3693 1 3.3693 14.6136  0.0002  bn
1.3688 6.4848
1 4.7376 21.0960  c n 1

bn 0.0002
r4  r3   1.3693   1.3688
c n 1 21.0960

r3 = r4 correct to 4 places of decimals, the required root = 1.3688.

GRAEFFE’S ROOT SQUARING METHOD

Using this method, we can find all the roots of a polynomial equation with real
coefficients,
n 1 n2
(i.e) the equation of the form a 0 x  a1 x  a 2 x  ....  a n  0
n
…..(1)

a0 a1 a2 a3 ….. an

a 02 a12 a 22 a 32 …. a n2
-2 a 0 a 2 -2 a1 a 3 -2 a3 a 4 ……
+2 a 0 a 4 2 a1 a5
-2 a 0 a 6 …….
b0 b1 b2 b3 ….. bn

If this procedure is repeated m times, we get the equation

B0 x n  B1 x n 1  B2 x n  2  ....  Bn  0 …. (2)
whose roots are R1 , R2 ,...., Rn which are the 2 th power of the roots of equation (1)
m
m
with sign changed. (ie) Ri   i2 i=1,2,…,n.
Case1:
Suppose we assume  1   2  .....   n
Then R1  R2  .....  Rn
If the roots of (1) differ in magnitude, then the 2mth power of the roots are separted
widely for higher values of m.
 B1
Therefore,  Ri 
B0
B
 Ri R j  R1 R2  B2
0
  B3
R R i j Rk  R1 R2 R3 
B0
(1) n Bn
 Rk 
B0
Bi 2m
Ri   i
Bi 1
Therefore, log  i  2 (log Bi  log Bi 1 ) ,
m 1
i=1,2,3,..,m
From this we can find the values of  i or -  i in (1), we can determine the sign of
the roots of the equation.

Case 2:
After a few squaring process, if the magnitude of the coefficients Bi is half the square of
the magnitude of corresponding coefficient in the previous equation, then this indicates
that  i is a double root.
Bi
Since Ri  ,
Bi 1
Bi 1
Ri 1 
Bi
Bi 1
and Ri Ri 1  Ri 
2

Bi
2 m 1 Bi
Ri2   i 
Bi 1
From this we can get  i double root. The sign of it can got as before by substitution in
(1).
Case 3:
If  k ,  k 1 are two complex conjugate roots, then this would make the coefficients of
x n  k in the successive squaring to fluctuate both in magnitude and sign..

If  k ,  k 1   k (cos  k  i sin  k ) is the complex pair of roots, then coefficients will

fluctuate in magnitude and sign by the quantity 2  km cos m k .
A complex pair is located only by such oscillation. If m is large,  k can be got from
B B
 k22  k 1 and  is got from 2  km cos m k = k 1
m

Bk 1 Bk 1
If the equation possesses only two complex roots possesses only two complex roots p 
iq, we have  1   2  ...   k 1  2 p   k  2  ...   n   a1
This gives the value of p
Since  k  p 2  q 2 and  k is known already q is known from this relation.
2

Problem:
Solve : x 3  x 2  x  2, using Graeffe’s method.
Solution.:
m 2m
0 1 1 -1 -1 -2
1 1 1 4
2 -4
1 2 1 3 -3 4
1 9 9 16
6 –24 1

2 4 1 15 -15 16
1 225 225 256
30 -480
3 8 1 225 -225 256
1 65025 65025 65536
510 -130560
4 16 1 65535 -65535 65536
1 4.2948362*109 4.2948362*10 4.2949673*109
9

131070 -8.5898035*109
5 32 1 4.2949673*109 -4.2949673*109 4.2949673*109

From above, we see that the magnitude of the coefficient B1 has become constant (upto
few decimals), while the magnitude of the coefficient B2 oscillates. Hence

SIMULTANEOUS ALGEBRAIC LINEAR EQUATIONS

Solving the system of simultaneous algebraic linear equations are two types (i) Direct
method (ii) Indirect methods or iterative methods.
Direct methods consist of Gauss Elimination method,Gauss – Jordan method, Method of
Triangularization, Crout’s method.
Iterative methods consist Jacobi method of iteration or Gauss – Jacobi method, Gauss-
Seidel method , Relaxation method.

The system,
a11 x1  a12 x 2  a13 x3  b1
a 21 x1  a 22 x 2  a 23 x3  b2
a31 x1  a32 x 2  a33 x3  b3
will be solvable by this method if
a11  a12  a13
a 22  a 21  a 23
a33  a 31  a 32
The system is call it as diagonally dominates.
Note: This condition is sufficient but not necessary.
GAUSS JACOBI METHOD:
Consider the system of equations

The system,
a1 x  b1 y  c1 z  d1
a 2 x  b2 y  c 2 z  d 2 ……….. (1)
a3 x  b3 y  c3 z  d 3
Let us assume
a1  b1  c1
b2  a 2  c 2
c3  a 3  b3
Then, iterative method can be used for the system (1).

1
x (d1  b1 y  c1 z )
a1
1
y (d 2  a 2 x  c 2 z )
b2
1
z (d 3  a3 x  b3 y )
c3
If x ( 0 ) , y ( 0 ) , z ( 0 ) are the initial values of x, y, z respectively, then
1
x (1)  (d 1  b1 y ( 0 )  c1 z ( 0 ) )
a1
1
y (1)  (d 2  a 2 x ( 0)  c 2 z ( 0 )
b2
1
z (1)  (d 3  a3 x ( 0 )  b3 y ( 0 ) )
c3
proceeding in the same way, if the rth iterates are x ( r ) , y ( r ) , z ( r ) , the iteration scheme
reduces to

1
x ( r 1)  (d1  b1 y ( r )  c1 z ( r ) )
a1
1
y ( r 1)  (d 2  a 2 x ( r )  c 2 z ( r ) )
b2
1
z ( r 1)  ( d 3  a 3 x ( r )  b3 y ( r ) )
c3
The procedure is continued till the convergence is assured.

GAUSS-SEIDEL METHOD OF ITERATION

 1
x (d1  b1 y  c1 z )
a1
1
y (d 2  a 2 x  c 2 z )
b2
1
z (d 3  a3 x  b3 y )
c3
We start with initial values y ( 0 ) , z ( 0 ) for y and z and get x (1) from the first equation.
(i.e)

1
x (1)  (d1  b1 y ( 0 )  c1 z ( 0 ) )
a1
1
y (1)  ( d 2  a 2 x (1)  c 2 z ( 0 )
b2
1
z (1)  (d 3  a3 x (1)  b3 y (1) )
c3
In finding the values of the unknowns, we use the latest available values on the right hand
side. If x ( r ) , y ( r ) , z ( r ) are the rth iterates, then the iteration scheme will be
1
x ( r 1)  (d1  b1 y ( r )  c1 z ( r ) )
a1
1
y ( r 1)  (d 2  a 2 x ( r 1)  c 2 z ( r ) )
b2
1
z ( r 1)  (d 3  a3 x ( r 1)  b3 y ( r 1) )
c3
This process of iteration is continued until the convergence is assured.

Note 1: For all systems of equations, this method will not work. It converges only for
special systems of equations.
2. Iteration method is self-correcting method.
3. The iteration is stopped when the values of x,y, z start repeating with the required
degree of accuracy.

Solve the following system by Gauss-Jacobi and Gauss-seidel methods:

10x-5y-2z =3; 4x-10y+3z=-3; x+6y+10z=-3.

Solution: since 10   5   2 , - 10  4  3 and 10  1  6

Therefore the given system is diagonally dominant.

Solving for x, y, z we have

 1
x (3  5 y  2 z ), (1)
10
1
y (3  4 x  3 z ), (2)
10
1
z (3  x  6 y ) (3)
10
Let the initial values be (0,0,0). Using these initial values in (1),(2), (3), we get
1
x (1)  (3  5(0)  2(0))  0.3
10
1
y (1)  (3  4(0)  3(0))  0.3
10
1
z (1)  ( 3  (0)  6(0))  0.3
10
proceeding in this way
1
x (9 )  (3  5(0.2852214)  2(0.505))  0.3
10
1
y (9 )  (3  4(0.34167891)  3(0.50519319))  0.285113607
10
1
z (9 )  (3  (0.34167891)  6(0.2852214))  0.505300731
10
Hence correct to 3 decimal places, the values are x= 0.342, y =0.285, z =-0.505

GAUSS- SEIDEL METHOD :

Initial values: y =0 , z =0
First iteration:

1
x (1)  (3  5(0)  2(0))  0.3
10
1
y (1)  (3  4(0.3)  3(0))  0.42
10
1
z (1)  (3  (0.3)  6(0.42))  0.582
10
proceeding in this way,

1
x (7)  (3  5(0.285039017)  2( 0.5051728))  0.3414849
10
1
x (7)  (3  4(0.3414849)  3( 0.5051728))  0.28504212
10
1
z (7)  ( 3  (0.3414849)  6(0.28504212))  0.5051737
10

The values correct to 3 decimal places are x=0.342, y=0.285, z=-0.505.

Problem: Solve the following system by Gauss-Jacobi and Gauss-seidel methods:

8x-3y+2z =20; 4x+11y-z=33; 6x+3y+12z=-35

FINITE DIFFERENCE OPERATOR:.

FINITE DIFFERENCES 2
Forward, backward, central difference notations

There are several different notations for the single set of finite differences,
described in the preceding Step. We introduce each of these three notations in
terms of the so-called shift operator, which we will define first.

1. The shift operator E

Let be a set of values of the

function f(x) The shift operator E is defined by:

Consequently,

and so on, i.e.,

where k is any positive integer. Moreover, the last formula can be

extended to negative integers, and indeed to all real values of j and k, so
that, for example,

and

2. The forward difference operator Q

If we define the forward difference operator Q by

,
 then

which is the first-order forward difference at xj. Similarly, we find that

is the second-order forward difference at xj, and so on. The forward

difference of order k is

where k is any integer.

3. The backward difference operator

If we define the backward difference operator by

then

which is the first-order backward difference at xj. Similarly,

is the second-order backward difference at xj, etc. The backward

difference of order k is

where k is any integer. Note that .

4. The central difference operator

If we define the central difference operator by

,
 then

which is the first-order central difference at xj. Similarly,

is the second-order central difference at xj, etc. The central difference

of order k is

where k is any integer. Note that .

5. Differences display

The role of the forward, central, and backward differences is displayed

by the difference table:
Although forward, central, and backward differences represent
precisely the same data:

1. Forward differences are useful near the start of a table, since

they only involve tabulated function values below xj ;
2. Central differences are useful away from the ends of a table,
where there are available tabulated function values above and
below xj;
3. Backward differences are useful near the end of a table, since
they only involve tabulated function values above xj.

Checkpoint
4. What is the definition of the shift operator?
5. How are the forward, backward, and central difference
operators defined?
 6. When are the forward, backward, and central difference
notations likely to be of special use?

EXERCISES
7. Construct a table of differences for the polynomial

for x = 0(1)4. Use the table to obtain the values of :

1. ;
2. ;
3. .
8. For the difference table of f (x) = ex for x = 0.1(0.05)0.5
determine to six significant digits the quantities (taking x0 = 0.1 ):
1. ;
2. ;
3. ;
4. ;
5. ;
9. Prove the statements:
1. ;
2. ;
3. ;
4. .

FINITE DIFFERENCES 3
Polynomials

Since polynomial approximations are used in many areas of Numerical

Analysis, it is important to investigate the phenomena of differencing
polynomials.

1. Finite differences of a polynomial

Consider the finite differences of an n-th degree polynomial

tabulated for equidistant points at the tabular interval h.

Theorem: The n-th difference of a polynomial of degree n is a constant
proportional to n
and higher order differences are zero.

Proof: For any positive integer k, the binomial expansion

yields

Omitting the subscript of x, we find

In passing, the student may recall that in the Differential Calculus the increment
is related to the derivative of f (x) at the point x.

2. Example

3
Construct for f (x) = x with x = 5.0(0.1)5.5 the difference table:
Since in this case n = 3, an =1, h = 0.1, we find

Note that round-off error noise may occur; for example, consider the tabulation of f(x) = x3 for
5.0(0.1)5.5, rounded to two decimal places:

3. Approximation of a function by a polynomial

Whenever the higher differences of a table become small (allowing for round-off noise), the function
represented may be approximated well by a polynomial. For example, reconsider the difference table
of 6D for f (x ) = ex with x = 0.1(0.05)0.5:
Since the estimate for round-off error at (cf. the table in STEP 12), we say that third
differences are constant within round-off error, and deduce that a cubic approximation is appropriate
for ex over the range 0.1 < x < 0.5. An example in which polynomial approximation is inappropriate
occurs when f(x) = 10x for x = 0(1)4, as is shown by the next table:

Although the function f(x) = 10x is `smooth', the large tabular interval (h = 1) produces large higher
order finite differences. It should also be understood that there exist functions that cannot usefully be
tabulated at all, at least in certain neighbourhoods; for example, f(x) = sin(1/x) near the origin x = 0.
Nevertheless, these are fairly exceptional cases.

Finally, we remark that the approximation of a function by a polynomial is fundamental to the

widespread use of finite difference methods.
 Checkpoint
1. What may be said about the higher order (exact) differences of a polynomial?
2. What is the effect of round-off error on the higher order differences of a polynomial?
3. When may a function be approximated by a polynomial?

EXERCISES

1. Construct a difference table for the polynomial f(x) = x4 for x = 0(0.1)1 when
a.
b. the values of f are exact;
c. the values of f have been rounded to 3D.
d. Compare the fourth difference round-off errors with the estimate +/-6.
e.
2. Find the degree of the polynomial which fits the data in the table:

INTERPOLATION 1
Linear and quadratic interpolation

Interpolation is the art of reading between the lines in a table. It may be

regarded as a special case of the general process of curve fitting. More
precisely, interpolation is the process whereby untabulated values of a function,
given only at certain values, are estimated on the assumption that the function
has sufficiently smooth behavior between tabular points, so that it can be
approximated by a polynomial of fairly low degree.

Interpolation is not as important in Numerical Analysis as it has been, now that

computers (and calculators with built-in functions) are available, and function
values may often be obtained readily by an algorithm (probably from a standard
subroutine). However,

1. interpolation is still important for functions that are available only in

tabular form (perhaps from the results of an experiment); and
2. interpolation serves to introduce the wider application of finite
differences.
we have observed that, if the differences of order k are constant (within round-
off fluctuation), the tabulated function may be approximated by a polynomial
of degree k. Linear and quadratic interpolation correspond to the cases k = 1
and k = 2, respectively.

1. Linear interpolation

When a tabulated function varies so slowly that first differences are

approximately constant, it may be approximated closely by a straight
line between adjacent tabular points. This is the basic idea of linear
interpolation. In Fig. 10, the two function points (xj, fj) and (xj+1, fj+1) are
connected by a straight line. Any x between xj and xj+1 may be defined by
a value of  such that

If f (x) varies only slowly in the interval, a value of the function at x is

approximately given by the ordinate to the straight line at x. Elementary
geometrical considerations yield

so that

FIGURE 10. Linear interpolation.

In analytical terms, we have approximated f (x) by

the linear function of x which satisfies

As an example, consider the following difference table, taken from a 4D

table of e-x:

The first differences are almost constant locally, so that the table is
suitable for linear interpolation. For example,

2. Quadratic interpolation

As previously indicated, linear interpolation is appropriate only for

slowly varying functions. The next simple process is quadratic
interpolation, based on a quadratic approximating polynomial; one
might expect that such an approximation would give better accuracy for
functions with larger variations.

Given three adjacent points xj, xj+1 = xj and xj+2 = xj + 2h, suppose that f
(x) can be approximated by

,.
where a, b, and c are chosen so that

Thus,

whence

Setting , we obtain the quadratic interpolation formula:

We note immediately that this formula introduces a second term

(involving ), not included in the linear interpolation formula.

As an example, we determine the second-order correction to the value

of f (0.934) obtained above using linear interpolation. The extra term is

so that the quadratic interpolation formula yields

(In this case, the extra term -0.0024/200 is negligible!)

Checkpoint
1. What process obtains an untabulated value of a function?
2. When is linear interpolation adequate?
3. When is quadratic inteipolation needed and adequate?
 EXERCISES
4. Obtain an estimate of sin(0.55) by linear interpolation of f (x) =
sin x over the interval [0.5, 0.6] using the data:

Compare your estimate with the value of sin(0.55) given by your

calculator.

5. The entries in a table of cos x are:

Obtain an estimate of cos(80? 35') by means of

1. Linear interpolation,
2. quadratic interpolation.
6. The entries in a table of tan x are:

Is it more appropriate to use linear or quadratic interpolation?

Obtain an estimate of tan(80? 35').

INTERPOLATION 2
Newton interpolation formulae

The linear and quadratic interpolation formulae are based on first and
second degree polynomial approximations. Newton has derived general
forward and backward difference interpolation formulae, corresponding for
tables with constant interval h. (For tables with variable interval, we can use an
interpolation procedure in involving divided differences.)

1. Newton's forward difference formula

Consider the points xj, xj + h, xj + 2h, . . ., and recall that

where  is any real number. Formally, one has (since )

 ,

which is Newton's forward difference formula. The linear and

quadratic (forward) interpolation formulae correspond to first and
second order truncation, respectively. If we truncate at n-th order,
we obtain

which is an approximation based on the values fj, fj+1,. . . , fj+n. It will be

exact if (within round-off errors)

which is the case if f is a polynomial of degree n.

2. Newton's backward difference formula

Formally, one has (since Newton's backward difference

formula. The linear and quadratic (backward) interpolation formulae
correspond to truncation at first and second order, respectively. The
approximation based on the fj-n, fj-1, . . . , fj-n is

3. Use of Newton's interpolation formulae

Newton's forward and backward difference formulae are wel1 suited

for use at the beginning and end of a difference table, respectively.
(Other formulae which use central differences may be more convenient
elsewhere.)

As an example, consider the difference table of f (x) = sin x for x = 0?

( 10?)50?:
 .

Since the fourth order differences are constant, we conclude that a

quartic approximation is appropriate. (The third-order differences are
not quite constant within expected round-offs, and we anticipate that a
cubic approximation is not quite good enough.) In order to determine
sin 5? from the table, we use Newton's forward difference formula (to
fourth order); thus, taking xj = 0, we find and

Note that we have kept a guard digit (in parentheses) to minimize

accumulated round-off error.

In order to determine sin 45? from the table, we use Newton's

backward difference formula (to fourth order); thus, taking xj = 40, we
find and
 4. Uniqueness of the interpolating polynomial

Given a set of values f(x0), f(x1), . . , f(xn) with xj = x0 + jh, we have two
interpolation formulae of order n available:

Clearly, Pn and Qn are both polynomials of degree n. It can be verified)

that Pn(xj) = Qn(xj) = f(xj) for j = 0,1, 2, . . . , n, which implies that Pn - Qn
is a polynomial of degree n which vanishes at (n + 1 ) points. In turn, this
implies that Pn - Qn  0, or Pn  Qn. In fact, a polynomial of degree n
through any given (n + 1) (distinct but not necessarily equidistant) points
is unique, and is called the interpolating polynomial.

5. Analogy with Taylor series

If we define for an integer k

the Taylor series about xj becomes

Setting we have formally

 A comparison with Newton's interpolation formula

shows that the operator (applied to functions of a continuous

variable) is analogous to the operator E (applied to functions of a
discrete variable).

Checkpoint
1. What is the relationship between the forward and backward
linear and quadratic interpolation formulae (for a table of
constant interval h) and Newton's interpolation formulae?
2. When do you use Newton's forward difference formula?
3. When do you use Newton's backward difference formula?

EXERCISES
4. From a difference table of f (x) = ex to 5D for x = 0.10(0.05)0.40,
estimate:
1. e0.14 by means of Newton's forward difference formula;
2. e0.315 by means of Newton's backward difference
formula.
5. Show that for j = 0, 1, 2, . . .,

6. Derivc the equation of the interpolating polynomial for the data.

INTERPOLATION 3
Lagrange interpolation formula

The linear and quadratic interpolation formulae correspond to first and second
degree polynomial approximations, respectively.We have discussed
Newton's forward and backward interpolation formulae and noted that
higher order interpolation corresponds to higher degree polynomial
approximation. In this Step we consider an interpolation formula attributed to
Lagrange, which does not require function values at equal intervals.
Lagrange's interpolation formula has the disadvantage that the degree of the
approximating polynomial must be chosen at the outset; an alternative
approach is discussed in the next Step. Thus, Lagrange's formula is mainly of
theoretical interest for us here; in passing, we mention that there are some
important applications of this formula beyond the scope of this book - for
example, the construction of basis functions to solve differential equations
using a spectral (discrete ordinate) method.

1. Procedure

Let the function f be tabulated at (n + 1), not necessarily equidistant

points xj, j = 1, 2,…., n and be approximated by the polynomial

of degree at most n, such that

Since for k = 0,1, 2, . . , n

is a polynomial of degree n which satisfies

then:

is a polynomial of degree n which satisfies

Hence,

is a polynomial of degree (at most) n such that

 ,

i.e., the (unique) interpolating polynomial. Note that for x = xj all terms
in the sum vanish except the j-th, which is fj; Lk(x) is called the k-th
Lagrange interpolation coefficient, and the identity

(established by setting f(x)  1) may be used as a check. Note also that

with n = 1 we recover the linear interpolation formula:

2. Example

We will use Lagrange's interpolation formula to find the

interpolating polynomial P3 through the points (0, 3), (1, 2), (2, 7), and
(4, 59), and then find the approximate value P3(3).

The Lagrange coefficients are:

(The student should verify that Hence, the

required polynomial is
Consequently, However, note that, if the
explicit form of the interpolating polynomial were not required, one
would proceed to evaluate P3(x) for some value of x directly from the
factored forms of Lk(x). Thus, in order to evaluate P3(3), one has

3. Notes of caution

In the case of the Newton interpolation formulae, considered in the

preceding Step, or the formulae to be discussed in the next Step, the
degree of the required approximating polynomial may be determined
merely by computing terms until they no longer appear to be significant.
In the Lagrange procedure, the polynomial degree must be chosen at
the outset! Also, note that

1. a change of degree involves recomputation of all terms; and

2. for a polynomial of high degree the process involves a large
number of multiplications, whence it may be quite slow.

Lagrange interpolation should be used with considerable caution. For

example, let us employ it to obtain an estimate of from the points (0,
0), (1,1), (8, 2), (27, 3), and (64, 4) on . We find

so that which is not very close to the correct value 2.7144!

A better result (i.e., 2,6316) can be obtained by linear interpolation
between (8, 2) and (27, 3). The problem is that the Lagrange method
yields no indication as to how well is represented by a quartic.
In practice, therefore, Lagrange interpolation is used only rarely.
 Checkpoint
1. When is the Lagrange interpolation formula used in practical
computations?
2. What distinguishes the Lagrange formula from many other
interpolation formulae?
3. Why should the Lagrange formula be used in practice only with caution?

EXERCISE

Given that f (-2) = 46, f (-1 ) = 4, f ( 1 ) = 4, f (3) = 156, and f (4) = 484, use
Lagrange's interpolation formula to estimate the value of f(0).

INTERPOLATION 4*
Divided differences*

We have noted that the Lagrange interpolation formula is mainly of

theoretical interest, because, at best, it involves, in practice, very considerable
computation and its use can be quite risky. It is much more efficient to use
divided differences to interpolate a tabulated functions (especially, if the
arguments are unequally spaced); moreover, its use is relatively safe, since the
required degree of the interpolating polynomial can be decided upon at the
start. An allied procedure, due to Aitken, is also commonly used in practice.

1. Divided differences

Again, let the function f be tabulated at the (not necessarily equidistant)

points [x0, x1, . . . , xn]. We define the divided differences between points
as follows:

As an example, we will construct from the data:

the divided difference table:

We note that the third divided differences are constant. In Section 3, we

shall use the table to interpolate by means of Newton's divided
difference formula and determine the corresponding interpolating
cubic.

2. Newton's divided difference formula

According to the definitions of divided differences, we find

Multiplying the second equation by (x - x0), the third by (x - x0)(x - x1),

etc., and adding the results yields Newton's divided difference
forrnula, suitable for computer implementation (PSEUDO CODE)

where

Note that the remainder term R vamishes at x0, x1, . . . , xn, whence we
infer that the other terms on the right-hand side constitute the
interpolating polynomial or, equivalently, the Lagrange polynomial. If
the required degree of the interpolating polynomial is not known in
advance, it is customary to arrange the points x1, . . . , xn, according to
their increasing distance from x and add terms until R is small enough.

3. Example

From the tabulated function in Section 1, we will estimate f (2) and f(4),
using Newton's divided difference formula and find the corresponding
interpolating polynomials.

The third divided difference being constant, we can fit a cubic through
the five points. By Newton's divided difference formula, using x0 = 0,
x1 = 1, x2 = 3, and x3 = 6, the interpolation cubic becomes:

so that

Obviously, the interpolating polynomial is

In order to estimate the value of f(4), we identify x0 =1, x1 = 3, x2 = 6, x3

=10, whence

and

As expected, the two interpolating polynomials are the same cubic, i.e.,
x3 - 8x + 1.

4. Errors in interpolating polynomials

In Section 2, we have seen that the error in an interpolating polynomial

of degree n was given by

.
As it stands, this expression is not very useful, because it involves the
unknown quantity However, it may be shown (cf., for
example, Conte and de Boor (1980)) that, if
and f is (n + 1)-times differentiable on (a, b), then
there exists a such that

whence it follows that

This formula may be useful when we know the function generating the
data and wish to find lower and upper error bounds. For example, let
there be given sin 0 = 0, sin(0.2) = 0.198669, and sin(0.4) = 0.389418 to
6D (where the arguments in the sine function are in radians). Then we
can form the divided difference table:

Thus, the quadratic approximation to sin(0.1) is:

. .

Since n = 2, the magnitude of the error in the approximation is given by

where 0 <  0.4. For f(x) = sin x, one has , so that
It then follows that

.
The absolute value of the actual error is 0.000492, which is within
these bounds.

5. Aitken's method

In practice, Aitken's procedure is often adopted; it yields systematically

and successively better interpolation polynomials (corresponding to
successively higher order truncation of Newton's divided difference
formula). Thus, one finds

and, obviously,

Next, since

one finds

noting that

etc. In passing, we may note that

At first sight, this procedure may look complicated, but it is systematic,

and therefore computationally straightforward. It is represented by the
scheme:
 .

One of its major advantages is that the accuracy may be gauged by

comparing successive steps (corresponding, of course, to gauging the
appropriate truncation of Newton's divided difference formula.) As in the
case of Newton's divided difference formula, as a rule, the points x0, x1,
x2, . . . are ordered such that x0 - x, x - x, x2 - x, . . . form a sequence with
increasing magnitude. Finally, we remark that although the derivation of
Aitken's method emphasizes its relationship to the Newton formula, it
is notable that it ultimately does not involve at all divided differences!

As an example, we will estimate f (2) by Aitken's method from the

function, tabulated.

We have x = 2, so that we choose x0 = 1, x1 = 3, x2 = 0, x3 = 6, and x4 =

10: thus Aitken's scheme yields

The computations proceed from the left, row by row, with an

appropriately divided cross multiplication of the respective entries with
those in the (xk -- x) column on the right hand side. Thus,
 The entry -7 (in circle) appears twice successively along the diagonal,
so that one may conclude that f (2) = -7.

Checkpoint
1. What major practical advantage has Newton's divided
difference interpolation formula over Lagrange's formula?
2. How are the tabular points usually ordered for interpolation by
Newton's divided difference formula or Aitken's method?
3. Are divided differences actually used in interpolation by
Aitken's method?

EXERCISES
1. Use Newton's divided difference formula to show that it is
quite invalied to interpolate from the points
.
2.
3. Given that use Newton's divided
difference formula to estimate the value of e0.25. Find lower and
upper bounds on the magnitude of the error and verify that the actual
magnitude is within the calculated bounds.
4. Given that f(-2) = 46, f(-1) = 4, f(1) = 4, f(3) = 156, and f(4) =
484, estimate the value of f (0) from
a. Newton's divided difference formula, and
 b. Aitken's method.
Comment on the validity of this interpolation.
c. Given that f (0) = 2.3913, f( 1 ) = 2.3919, f (3) = 2.3938, and f
(4) = 2.3951, use Aitken's method to estimate the value of f(2).

INTERPOLATION 5*
Inverse interpolation *

Instead of the value of a function f (x) for a certain x, one might seek the value
of x which corresponds to a given value of f (x), a process referred to as inverse
interpolation. For example, the reader may have contemplated the possibility
of obtaining roots of f (x) = 0 by inverse interpolation.

1. Linear inverse interpolation

An obvious elementary procedure is to tabulate the function in the

neighbourhood of the given value at an interval so small that linear
inverse interpolation may be used. Reference to Step 21 yields

where

is the linear approximation. (Note that if f (x) = 0, we recover the

method of false position - Step 8).

For example, one finds from a 4D table of f (x) = ex that f (0.91) =

0.4025, f (0.92) = 0.3985, so that f (x) = 0.4 corresponds to

In order to obtain an immediate check, we will use direct interpolation

to recover f (x) = 0.4. Thus,
 .

2. Iterative inverse interpolation

As undoubtedly the reader may appreciate, it may be preferable to adopt

(at least approximately) an interpolating polynomial of degree higher
than one rather than seek to tabulate at a small enough interval to permit
linear inverse interpolation. The degree of the approximating polynomial
may be decided implicitly by an iterative (successive approximation)
method.

For example, Newton's forward difference formula may be rearranged

as follows:

Since terms involving second and higher differences may be expected to

decrease fairly quickly, we obtain successive approximations (  i) to 
given by

Similar iterative procedures may be based on other interpolation

formulae such as Newton's backward difference formulae.

In order to illustrate this statement, consider the first table of f (x) = sin x
in STEP 22 and let us seek the value of x for which f (x) = 0.2.
Obviously, 10? > x
(Note that it is unnecessary to carry many digits in the first estimates of
.) Consequently,

which yields x = 11.539?.

A check, either by the usual method of direct interpolation or in this case

directly, yields sin(11.539?) = 0.2000.

3. Divided differences

Since divided differences are suitable for interpolation with tabular

values which are unequally spaced, they may also be used for inverse
interpolation. Consider again

f(x) = sin x for x=l0? (l0?)50?

and determine the value of x for which f(x) = 0.2. Ordering according to
increasing distance from f(x) = 0.2, one finds the divided difference table
(entries multiplied by 100);
 ,

Hence,

Aitken's scheme could also have been used here! However, by either
method, we note that any advantage in accuracy gained by the use of
iterative inverse interpolation may not justify the additional
computational demand.

Checkpoint
1. Why may linear inverse interpolation be either tedious or
impractical?
2. What is the usual method for checking inverse interpolation?
3. What is the potential advantage of inverse interpolation, using
either divided differences or Aitken's scheme, compared with
the iterative method? What is a likely disadvantage?

EXERCISES
4. Use linear inverse interpolation to find the root of x + cos x = 0
correct to 4D.
5. Solve 3xex =1 to 3D.
6. Given a table of values of a cubic f without knowledge of its
specific form:
 find x for which f(x) = 10, 20 and 40, respectively. Check your
answers by (direct) interpolation. Finally, obtain the equation of
the cubic and use it to recheck your answers.

NUMERICAL DIFFERENTIATION AND INTEGRATION:

Newton forward differentiation:

dy 1 1 1
 (y 0  2 y 0  3 y 0  ......)
dx h 2 3

d2y 1 1 11
2
 2 (2 y 0  3 y 0  4 y 0  ......)
dx h 2 12

Newton backward differentiation:

dy 1 1 1
 (y n   2 y n   3 y n  ......)
dx h 2 3

d2y 1 2 11
2
 ( y n   3 y n   4 y n  ......)
dx h 12

d3y 1 3
3
 3 ( 3 y n   4 y n  ......)
dx h 2

1
Find first two derivatives of x 3 at x=50 and x=56 given the table below:

x 50 51 52 53 54 55 56
Y= x
1 3.6840 3.7084 3.7325 3.7563 3.7798 3.8030 3.8259
3

Solution: since we require f’(x) at x=50 we use Newton’s forward formula and to get
f’(x) at x=56 we use Newton’s backward formula..
 x y y 2 y 3 y
50 3.6840
51 3.7084 0.0244  0.0003 0
52 3.7325 0.0241  0.0003 0
53 3.7563 0.0238  0.0003 0
54 3.7798 0.0235  0.0003 0
55 3.8030 0.0232  0.0003
56 3.8259 0.0229

By Newton’s forward formula

dy 1 1 1
 (y 0  2 y 0  3 y 0  ......)
dx h 2 3

1 1 1
= [0.0244  ( 0.003)  (0)]
1 2 3

= 0.02455

d2y 1 1 11
2
 2 (2 y 0  3 y 0  4 y 0  ......)
dx h 2 12

= 1[-0.0003]=-0.0003

By Newton backward difference formula,

dy 1 1 1
 (y n   2 y n   3 y n  ......)
dx h 2 3

dy 1 1
( ) x 56  (0.0229  (0.0003)  0)
dx 1 2

=0.02275

d2y 1 2 11
2
 ( y n   3 y n   4 y n  ......)
dx h 12

d2y 1 11
( )
2 x 56
 ( 2 y n   3 y n   4 y n  ......)
dx h 12

=[-0.0003]

Problem: Given the following data, find y’(6) and the maximum value of y.

X: 0 2 3 4 7 9
 Y: 4 26 58 112 466 922

Solution: Since the arguments ar ot equally spaced, we will use Newton’s divided
difference formula (or even Lagrange’s formula)

x f ( x) f ( x ) 2 f ( x ) 3 f ( x ) 4 f ( x )
0 4
2 26 11 7 1
3 58 32 11 1 0
4 112 54 16 1 0
7 466 118 22
9 922 228

By Newton’s divided difference formula,

Y= f(x) =f(x0)+(x-x0)f(x0,x1)+(x-x0)(x-x1)f(x0,x1,x2)+….

= 4+(x-0)11+(x-0)(x-2)7+(x-0)(x-2)(x-3).1

= x3+2x2+3x+4

Therefore, y’(x) =3x2+4x+3

Y’(6) =135.

Y(x) is maximum if y’(x)=0 Therefore 3x2+4x+3=0. But the roots are imaginary.
Therefore, there is no extremum value in the range. In fact, it is an increasing curve.

NUMERICAL INTEGRATION 1
The trapezoidal rule

It is often either difficult or impossible to evaluate by analytical methods

definite integrals of the form

so that numerical integration or quadrature must be used.

It is well known that the definite integral may be interpreted as the area under
the curve y = f (x) for and may be evaluated by subdivision of the
interval and summation of the component areas. This additive property of the
definite integral permits evaluation in a piecewise sense. For any subinterval
 of the interval , we may approximate f (x) by the
interpolating polynomial Pn(x). Then we obtain the approximation

which will be a good approximation, provided n is chosen so that the error |f

(x) - Pn(x)| in each tabular subinterval is sufficiently
small. Note that for n > 1 the error is often alternately positive and negative in
successive subintervals and considerable cancellation of error occurs; in
contrast with numerical differentiation, quadrature is inherently accurate!
It is usually sufficient to use a rather low degree, polynomial approximation
over any subinterval .

1. The trapezoidal rule

Perhaps the most straightforward quadrature is to subdivide the interval

into N equal strips of width h by the points

such that b = a + Nh. Then one can use the additive property

and the linear approximations, involving

to obtain the trapezoidal rule, which is suitable for computer

implementation (cf. Pseudo-Code
Integration by the trapezoidal rule therefore involves computation of a
finite sum of values of the integrand f, whence it is very quick. Note that
this procedure can be interpreted geometrically (Figure 16) as the sum of
the areas of N trapezoids of width h and average height (fj + fj+1)/2.

FIGURE 15 The trapezoidal rule

2. Accuracy

The trapezoidal rule corresponds to a rather crude polynomial

approximation (a straight line) between successive points xj and xj+1 = xj
+ h, and hence can only be accurate for sufficiently small h. An
approximate (upper) bound on the error may be derived as follows: The
Taylor expansion

yields the trapezoidal form:

while f (x) may be in xj  x  xj+1 as

to arrive at the exact form:

 Comparison of these two forms shows that the truncation error is

(STEP 2 regarding the concept of truncation error.) Ignoring higher-order

terms, one arrives at an approximate bound on this error when using the
trapezoidal rule over N subintervals:

Whenever possible, we will choose h small enough to make this error

negligible. In the case of hand computations from tables, this may not be
possible. On the other hand, in a computer program in which f (x) may
be generated anywhere in , the interval may be resubdivided
until sufficient accuracy is achieved. (The integral value for successive
subdivisions can be compared, and the subdivision process terminated
when there is adequate agreement between successive values.)

3. Example

Obtain an estimate of the integral

using the trapezoidal rule and the data in STEP 20:. If we use T(h) to
denote the approximation with strip width h, we obtain

Since we may observe that the error sequence

-0.00081, -0.00020, -0.00005 decreases with h?, as expected.
 Checkpoint
1. Why is quadrature using a polynomial approximation for the
integrand likely to be satisfactory, even if the polynomial is of low
degree?
2. What is the degree of the approximating polynomial
corresponding to the trapezoidal rule?
3. Why is the trapezoidal rule well suited for implementation on a
computer?

EXERCISES
4. Estimate the value of the integral

using the trapezoidal rule and the data given in Exercise 2 of the
preceding Step.

5. Use the trapezoidal rule with h = 1,0.5, and 0.25 to estimate the
value of the integral

Answers
last next
Simpson's Rule

If it is undesirable (for example, when using tables) to increase the subdivision

of an interval , in order to improve the accuracy of a quadrature, one
alternative is to use an approximating polynomial of higher degree. The
integration formula, based on a quadratic (i.e., parabolic) approximation is
called Simpson's Rule. It is adequate for most purposes that one is likely to
encounter in practice.

1. Simpson's Rule

Simpson's Rule gives for

 .

A parabolic arc is fitted to the curve y = f(x) at the three tabular points
Hence, if N - (b - a) is even, one obtains Simpson's Rule:

where

Integration by Simpson's Rule involves computation of a finite sum of

given values of the integrand f, as in the case of the trapezoidal rule.
Simpson's Rule is also effective for implementation on a computer; a
single direct application in a hand calculation usually gives sufficient
accuracy.

2. Accuracy

For a given integrand f, it is quite appropriate to computer program

increased interval subdivision, in order to achieve a required accuracy,
while for hand calculations an error bound may again be useful.

Let the function f(x) have in the Taylor expansion

then

.
One may reformulate the quadrature rule for by replacing
fj+2 = f (j+1 + h) and fj = f (xj+1 - k) by its Taylor series; thus

A comparison of these two versions shows that the truncation error is

Ignoring higher order terms, we conclude that the approximate bound

on this error while estimating

by Simpson's Rule with N/2 subintervals of width 2h is

Note that the error bound is proportional to h4, compared with h2 for the
cruder trapezoidal rule. Note that Simpson's rule is exact for cubics!

3. Example

We shall estimate the value of the integral

using Simpson's rule and the data in Exercise 2 of STEP29. If we choose

h = 0.15 or h = 0.05, there will be an even number of intervals. Denoting
the approximation with strip width h by S(h), we obtain
 and

respectively. Since f(4)(x) = -15x-7/2/16, an approximate truncation error

bound is

whence it is 0.000 000 8 for h = 0.15 and 0.000 000 01 for h = 0.05.
Note that the truncation error is negligible; within round-off error, the
estimate is 0.32148(6).

Checkpoint
1. What is the degree of the approximating polynomial corresponding to
Simpson's Rule?
2. What is the error bound for Simpson's rule?
3. Why is Simpson's Rule well suited for implementation on a computer?

EXERCISES
1. Estimate by numerical integration.the value of the integral

to 4D.

2. Use Simpson's Rule with N = 2 to estimate the value of

Estimate to 5D the resulting error, given that the true value of the
integral is 0.26247.
 ORDINARY DIFFERENTIAL EQUATIONS 1
Single-step methods

In pure mathematics courses, a lot of attention is paid to the properties of

differential equations and analytical techniques for solving them.
Unfortunately, many differential equations, including nearly all non-linear ones,
encountered in the real world are not amenable to analytic solution. Even the
apparently simple task of solving the problem

involves considerable effort before the unwieldy solution

is obtained. Even then a lot more effort is required to extract the value of y,
corresponding to one value of x. In such situations, it is preferable to use a
numerical approach from the start!

Partial differential equations are beyond the scope of this text, but in this and
the next Step we shall take a brief look at some methods for solving the single
first-order ordinary differential equation

for a given initial value y(x0) = y0. The first-order differential equation and the
given initial value constitute a first-order initial value problem. The numerical
solution of this problem involves an estimation of the values of y(x) at, as a
rule, equidistant points x1, x2,..., xN. For the sake of convenience, we shall
assume that these points are indeed equidistant and use h to denote the constant
step length. In practice, it is sometimes desirable to adjust the step length as
the numerical method proceeds. For instance, one may wish to use a smaller
step length when a point is reached at which the derivative is especially large.

The numerical methods for first-order initial value problems may be used
(in a slightly modified form) to solve higher-order differential equations. A
simple (optional) introduction to this topic.

1. Taylor series

We have already encountered one technique available for this problem;

we can estimate the value of y(x1) by means of a Taylor sequence of
order p, the particular value of p depending on the magnitude of h and
the accuracy required:

where y(x0) is given and y'(x0) can be found by substitution of x = x0 and

y = y0 into the differential equation, while the evaluation of y"(x0), . . . ,
y(p)(x0) requires differentiation of f(x,y), which can be messy. Note that y1,
y2,.. , yN will be used to denote the estimates of y(x1), y(x2),..., y(xN2).

Once yl has been computed, we can estimate y(x2) by a Taylor series,

based either on x1, when the error in y1 will be propagated, or on x0, when
p may have to be increased. In the local approach, yn+1 is computed
from a Taylor series, based on xn, while in the global approach yl,
y2, . . ., yN are all computed from the Taylor series based on x0. The local
approach is more commonly used in practice. The Taylor series method
based on the local approach is referred to as a single-step method, since
yn+1 depends only on the preceding approximation yn. Only single-step
methods will be discussed in this Step; multi-step methods are the topic
of the next Step.

One way of avoiding differentiation of f(x, y) is to fix p = 1 and

compute:

This is Euler's method. However, unless the step length h is very small,
the truncation error will be large and the results inaccurate.

2. Runge-Kutta methods

A popular way of avoiding differentiation of f(x, y) without sacrificing

accuracy involves estimation of yn+1 from yn and weighted averaging of
values of f (x, y), chosen so that the truncation error is comparable to
that of a Taylor series of order p. Details of the derivation of this
approach lie beyond the scope of this book, but we will quote two of the
simpler Runge-Kutta methods, which are suitable for computer
implementation.( PSEUDO-CODE).

The first has the same order of accuracy as the Taylor series method with
p = 2 and is usually presented in three steps:
 .

The second is the fourth-order method:

We see that neither method involves evaluations of derivatives of f(x, y);

instead, f(x, y) itself is evaluated twice in the second-order method,
four times in the fourth-order method.

3. Example

It is instructive to compare some of the methods given above by

application to a very simple problem. For example, we will estimate the
value of y(0.5), given that

The exact solution of this problem is whence y(0.5) =

1.797 44. We shall use a fixed step length h = 0.1 and work to 5D:

a. Euler's method (first order):

whence

a result which is not even accurate to 1D; in fact, the error is

approximately 0.08.
 b. Taylor series (fourth order):

Since

we find

Thus,

which is accurate to 4D, the approximate error being 0.000 01.

c. Runge-Kutta (second order):

and

:
 .

which is accurate to 2D, the error being approximately 0.003.

As we might have expected, the fourth-order method is obviously

superior, the first-order method is clearly inferior, and the second-order
method lies in between.

Checkpoint
1. For each of the two types of method outlined in this Step, state the main
disadvantage?
2. Why might we expect higher order methods to be more accurate?

EXERCISES
1. Obtain estimates of y(0.8) of the solution of the initial value problem

y' = x + y with y(0) = 1,

considered in the preceding section, by performing three more steps of:

a. Euler's method,
b. the fourth-order Taylor series method,
c. the second-order Runge-Kutta method with h = 0.1.
d. Compare the accuracy of the three methods.
2. Use Euler's method with step length h = 0.2 to estimate the value of
y(1), given that y' = -xy2 and y(0) = 2.
 ORDINARY DIFFERENTIAL EQUATIONS 2
Multi-step methods

As has been mentioned earlier, the methods covered in the last Step are
classified as single-step methods, because the only value of the approximate
solution used in the construction.of yn+I is yn, the result of the preceding step. In
contrast, multi-step methods use earlier values like yn-1, yn-2, . . . , in order to
reduce the number of times that f (x, y) or its derivatives have to be evaluated.

1. Introduction

Among the multi-step methods which can be derived by integration of

interpolating polynomials, we have (using fn to denote f(xn, yn)):

1. The midpoint method (second order):

2. Milne's Method (fourth order):

3. The family of Adams-Bashforth Methods, among which the second-

order formula is given by:

and the fourth-order formula by:

4. The family of Adams-Moulton Methods, among which the second-

order formula is given by:

often referred to as the trapezoidal method, and the fourth-order

formula by:
 .

Note that the family of Adams-Moulton methods requires evaluation of

fn+1=f(xn+1,.yn+1). Because f(xn+1,.yn+1) is therefore involved on the left-hand
as well as the right-hand sides of the expressions, such methods are
known as implicit methods. On the other hand, since yn+1 appears only
on the left-hand side in all the families listed under 1 - 3, they are called
explicit methods. Implicit methods have the disadvantage that one
usually requires some numerical technique, in order to solve for yn+1.
However, it is common to use explicit and implicit methods together
and, thus, to produce a predictor-corrector method, an approach which
is discussed in more advanced texts such as Mathews (1992).

We will not go into the various ways in which multi-step methods are
used, but it is obvious that we will need more than one starting value,
which may be obtained by first using a single-step method (cf the
preceding Step). One advantage of a multi-step method is that we need
to evaluate only once f(x, y) to obtain yn+1, since fn-1, fn-2, . . ., will already
have been computed. In contrast, any Runge-Kutta method (i.e.,
single-step method) involves more than one function evaluation at each
step, which in the case of complicated functions f (x, y) can be
computationally expensive. Thus, the comparative efficiency of multi-
step methods is often attractive, but such methods may be numerically
unstable.

2. Stability

Numerical stability is discussed in depth in more advanced texts such

as Burden and Faires (1993). In general, a method is unstable, if any
errors introduced into the computation are amplified as computation
progress. It turns out that the Adams-Bashforth and Adams-Moulton
families of methods have good stability properties.

As an example of a multi-step method with poor stability properties,

let us apply the mid-point method, given above, with h = 0.1 to the
initial value problem for the differential equation y' = -5y, y(0) = 1. The
correct solution to this problem is y(x) = e-5x. We will introduce an error
by taking y1 to be the value obtained by rounding the true value e-0.5 to
5D, i.e., by setting y1 = 0.606 53. The resulting method is then given by:

.
 Working to 5D, we construct the following table which allows us to
compare the consequent estimates yn with the analytic values:

It is seen that the estimates get worse as n increases. Not only do the
approximations alternate in sign after x6, but their magnitudes also grow.
Further calculations shows that y20 has the value 77.824 55 with an error
which is more than a million times larger than the analytic value!

Checkpoint
1. What distinguishes an explicit multi-step method from an implicit
method?
2. State one advantage of multi-step methods.

EXERCISES
1. Apply the second-order Adams-Bashforth Method with h = 0.1 to the
problem: y' = -5y, y(0) = 1 and obtain the approximations y2, . . ., y10,
using y1 = 0.606 53. Confirm that the approximations do not exhibit the
unstable behaviour of the mid-point method as shown by the table of
Section 2.
2. Retaining up to six digits, use the second-order Adams-Bashforth
Method with step length h=0.1 to estimate y(0.5), given that y' = x + y,
y(0) =1. Use y2 =1.11 as second starting value.

ORDINARY DIFFERENTIAL EQUATIONS 2

Multi-step methods

As has been mentioned earlier, the methods covered in the last Step are
classified as single-step methods, because the only value of the approximate
solution used in the construction.of yn+I is yn, the result of the preceding step. In
contrast, multi-step methods use earlier values like yn-1, yn-2, . . . , in order to
reduce the number of times that f (x, y) or its derivatives have to be evaluated.

1. Introduction

Among the multi-step methods which can be derived by integration of

interpolating polynomials, we have (using fn to denote f(xn, yn)):

1. The midpoint method (second order):

2. Milne's Method (fourth order):

3. The family of Adams-Bashforth Methods, among which the second-

order formula is given by:

and the fourth-order formula by:

4. The family of Adams-Moulton Methods, among which the second-

order formula is given by:

often referred to as the trapezoidal method, and the fourth-order

formula by:

Note that the family of Adams-Moulton methods requires evaluation of

fn+1=f(xn+1,.yn+1). Because f(xn+1,.yn+1) is therefore involved on the left-hand
as well as the right-hand sides of the expressions, such methods are
known as implicit methods. On the other hand, since yn+1 appears only
on the left-hand side in all the families listed under 1 - 3, they are called
explicit methods. Implicit methods have the disadvantage that one
usually requires some numerical technique, in order to solve for yn+1.
However, it is common to use explicit and implicit methods together
and, thus, to produce a predictor-corrector method, an approach which
is discussed in more advanced texts such as Mathews (1992).

We will not go into the various ways in which multi-step methods are
used, but it is obvious that we will need more than one starting value,
which may be obtained by first using a single-step method (cf the
preceding Step). One advantage of a multi-step method is that we need
to evaluate only once f(x, y) to obtain yn+1, since fn-1, fn-2, . . ., will already
have been computed. In contrast, any Runge-Kutta method (i.e.,
single-step method) involves more than one function evaluation at each
step, which in the case of complicated functions f (x, y) can be
computationally expensive. Thus, the comparative efficiency of multi-
step methods is often attractive, but such methods may be numerically
unstable.

2. Stability

Numerical stability is discussed in depth in more advanced texts such

as Burden and Faires (1993). In general, a method is unstable, if any
errors introduced into the computation are amplified as computation
progress. It turns out that the Adams-Bashforth and Adams-Moulton
families of methods have good stability properties.

As an example of a multi-step method with poor stability properties,

let us apply the mid-point method, given above, with h = 0.1 to the
initial value problem for the differential equation y' = -5y, y(0) = 1. The
correct solution to this problem is y(x) = e-5x. We will introduce an error
by taking y1 to be the value obtained by rounding the true value e-0.5 to
5D, i.e., by setting y1 = 0.606 53. The resulting method is then given by:

Working to 5D, we construct the following table which allows us to

compare the consequent estimates yn with the analytic values:
 .

It is seen that the estimates get worse as n increases. Not only do the
approximations alternate in sign after x6, but their magnitudes also grow.
Further calculations shows that y20 has the value 77.824 55 with an error
which is more than a million times larger than the analytic value!

Checkpoint
1. What distinguishes an explicit multi-step method from an implicit
method?
2. State one advantage of multi-step methods.

EXERCISES
1. Apply the second-order Adams-Bashforth Method with h = 0.1 to the
problem: y' = -5y, y(0) = 1 and obtain the approximations y2, . . ., y10,
using y1 = 0.606 53. Confirm that the approximations do not exhibit the
unstable behaviour of the mid-point method as shown by the table of
Section 2.
2. Retaining up to six digits, use the second-order Adams-Bashforth
Method with step length h=0.1 to estimate y(0.5), given that y' = x + y,
y(0) =1. Use y2 =1.11 as second starting value.

Higher Order Differential Equations

In the preceding two Steps, we have discussed numerical methods for solving
the first-order initial value problem y' = f(x, y), y(x0) = y(x0). However,
ordinary differential equations which arise in practice are often of higher
order. For example, a more realistic differential equation for the motion of a
pendulum is given by
which may have to be solved subject to given values of y(x0) = y0 and y'(x0) =
y'0. (In order to ensure notational consistency with the preceding two Steps, we
have changed the variables from  and t to y and x, respectively.) In general, an
initial value problem for an n-th order differential equation may assume the
form:

We shall see how this n-th order initial value problem may be rewritten as a
system of first-order initial value problems, which leads us to numerical
procedures for the solution of the general initial value problem which turn
out to be extensions of the numerical methods considered in the preceding
two Steps.

1. Systems of first-order initial value problems

If we set wj = y(j-1) for j = 1, 2, . . ., then the n-th order differential

equation

becomes y(n)=g(x,y,y',y",???,y(n-2),y(n-1)). Moreover, since w'j = wj+1 for j =

1, 2, . . . , we have an equivalent system of n first-order differential
equations, which is subject to the n initial conditions

For computational purposes, we choose to regard this hierarchy as a

system of n first-order initial value problems. Thus, for example, if the
initial conditions for the simple pendulum are y(0) = 0 and y'(0) = 1, then
one abtains the system of two first-order initial value problems

Note that an initial value problem for a more general system of n first-
order differential equations is given by:

for j =1, 2, . . ., n.
 2. Numerical methods for first-order systems

For the sake of simplicity, we shall consider only the case n = 2, when
the second-order initial value problem

yields the first-order system:

In order to extend any of the numerical methods, considered in the

preceding two Steps, we simply apply the methods to each of these two
initial value problems. The easiest way to see how this may be done is to
write the system in vector form. Setting:

we see that the problem becomes:

Now we recall that Euler's method for solving y' = f(x, y) was

with the given starting (initial) value y(x0) = y0. The analogous method
for solving the system is given by

with w(x0) = w0 given; again, the convention of the preceding steps by

which subscripts denote the components xn = x0 + nh has been adopted;
moreover, we shall denote the components of wn by wl,n and w2,n. These
are the approximations to w1(xn) = y(xn) and w2(xn) = y(xn), respectively.
If we write out the separate components in these vectors, we find

and
 .

As another illustration, we recall the Runge-Kutta method:

The analogous method of solving the second-order initial value problem

becomes

In component form, the equations are:

3. Numerical example

If we use Euler's method to solve the pendulum problem:

the resulting equations are

and

With w = 1 and h = 0.2, we obtain the values in the table:

 .

If we use the Runge-Kutta method, given in the preceding section, we

obtain the following values:

Since this Runge-Kutta method is second-order and the Euler method

is only first-order, we might expect the values in this table to be more
accurate than those displayed in the preceding table. By obtaining very
accurate approximations using much smaller values of h, it may be
verified that this is indeed the case.

Checkpoint
1. How may an n-th order initial value problem be written as a system of
n first-order initial value problems?
2. How may a numerical method for solving a first-order initial value
problem be extended to solve a system of two first-order initial value
problems?

EXERCISE

Apply Euler's method with step length h = 0.2 to obtain approximations y(1)
and y'(1) for the second-order initial value problem: