CONTENTS

➢ Numerical Differentiation
a) Finite Difference Approximations
b) Richardson Extrapolation
 NUMERICAL DIFFERENTIATION

Numerical differentiation deals with the following problem: We are given the
function y = f (x) and wish to obtain one of its derivatives at the point x = xk.

The term “given” means that we either have an algorithm for computing the
function, or we possess a set of discrete data points (xi , yi ), i = 0, 1, . . . , n.

In either case, we have access to a finite number of (x, y) data pairs from which
to compute the derivative.

Numerical differentiation is related to interpolation, one means of finding the

derivative is to approximate the function locally by a polynomial and then to
differentiate it. An equally effective tool is the Taylor series expansion of f (x)
about the point xk , which has the advantage of providing us with information
about the error involved in the approximation.
Figure 1: Illustration of how small data errors are
Numerical differentiation is not a particularly accurate process. It suffers from a amplified by numerical differentiation: (a) data with no
error, (b) data modified slightly, (c) the resulting
conflict between round off errors (caused by limited machine precision) and
numerical differentiation of curve (a), and (d) the
errors inherent in interpolation. For this reason, a derivative of a function can resulting differentiation of curve (b) manifesting
never be computed with the same precision as the function itself. increased variability. In contrast, the reverse operation of
integration [moving from (d) to (b) by taking the area
under (d)] tends to attenuate or smooth data errors.
➢ Finite Difference Approximations

• Taylor series expansions derive finite difference approximations of derivatives.

• It developed forward, backward and centered difference approximations of first and higher derivatives.
• High- accuracy divided-difference formulas can be generated by including additional terms from the Taylor series expansion.

(a)

• It can be solved for first derivative.

(b)

• It is referred to as the «first forward difference» and h is called the step size, that is , the length of the interval over which the
approximation is made.
• It is termed a ‘’forward’’ difference because it utilizes data at i and i+1 to estimate the derivative.
• The Taylor series can be expanded backward to calculate previous values on the basis of a present value, as in

(c)

• It is referred to as the «first backward difference ».

• A third way to approximate the first derivative is called «centered difference».

(d)
Finite divided difference can be represented generally as;

(e)

or

(f)

where Δfi is referred to as the first forward difference and h is called the step size, that is, the length of the interval over which the
approximation is made. It is termed a “forward” difference because it utilizes data at i and i + 1 to estimate the derivative (Fig. 2a).
The entire term Δf/h is referred to as a first finite divided difference.

This forward divided difference is but one of many that can be developed from the Taylor series to approximate derivatives
numerically. For example, backward and centered difference approximations of the first derivative can be developed in a fashion
similar to the derivation . The former utilizes values at xi−1 and xi (Fig. 2b), whereas the latter uses values that are equally spaced
around the point at which the derivative is estimated (Fig. 2c). More accurate approximations of the first derivative can be
developed by including higher-order terms of the Taylor series.
Backward Difference Approximation of the First Derivative: The Taylor series can be expanded backward to calculate a previous
value on the basis of a present value, as in

(1)

Truncating this equation after the first derivative and rearranging yields

(2)

∇ f1 is referred to as the first backward difference. See Fig. 2b for a graphical representation

Centered Difference Approximation of the First Derivative: A third way to approximate the first
derivative is to subtract Eq. (1) from the forward Taylor series expansion:

(3)

to yield

Figure 2 :Graphical depiction of (a) forward, (b)

backward, and (c) centered finite-divided-
difference approximations of the first derivative.
which can be solved for

Or

(4)

Equation (4) is a centered difference representation of the first derivative. Notice that the truncation error is of the order of h2 in
contrast to the forward and backward approximations that were of the order of h. Consequently, the Taylor series analysis yields
the practical information that the centered difference is a more accurate representation of the derivative (Fig. 2.c). For example, if
we halve the step size using a forward or backward difference, we would approximately halve the truncation error, whereas for the
central difference, the error would be quartered.
Example 1: Use forward, backward and a centered difference approximation to estimate the first derivative of function:

at x = 0.5 using a step size h = 0.5. Also repeat the computation using h = 0.25 and compare the accuracy.

Solution:
Note that the derivative can be calculated directly as below and can be used to compute the true value as f’(0.5)=−0.9125

For h = 0.5, the function can be employed to determine

... x i-2 x i-1 xi x i+1 x i+2 ...

.... x-2.h x-h x x+h x+2.h ...

These values can be used to compute the forward divided difference [Eq. (e)],

the backward divided difference [Eq. (c)],

and the centered divided difference [Eq. (4)],

For h = 0.25,

which can be used to compute the forward divided difference,

the backward divided difference,

and the centered divided difference,

For both step sizes, the centered difference approximation is more accurate than forward or backward differences. Also, as predicted
by the Taylor series analysis, halving the step size approximately halves the error of the backward and forward differences and
quarters the error of the centered difference.
Finite Difference Approximations of Higher Derivatives: Besides first derivatives, the Taylor series expansion can be used to derive
numerical estimates of higher derivatives. To do this, we write a forward Taylor series expansion for f (xi+2) in terms of f (xi):

(5)

Equation (3) can be multiplied by 2 and subtracted from Eq. (5) to give

which can be solved for

(6)

This relationship is called the second forward finite divided difference. Similar manipulations can be employed to derive a backward
version

and a centered version

(7)

As was the case with the first-derivative approximations, the centered case is more accurate. Notice also that the centered version can
be alternatively expressed as

Thus, just as the second derivative is a derivative of a derivative, the second divided difference approximation is a difference of two
first divided differences.
Example 2: Recall that in Example 1 we estimated the derivative of

at x = 0.5 using finite divided differences and a step size of h = 0.25,

where the errors were computed on the basis of the true value of −0.9125. Repeat this computation, but employ the high-accuracy
formulas from Figs.3 through 5.
Solution: The data needed for this example is
... x i-2 x i-1 xi x i+1 x i+2 ...

.... x-2.h x-h x x+h x+2.h ...

The forward difference

The backward difference

The centered difference

Example 3: Use forward, backward difference approximations of O(h) and O(h2) ; a centered difference approximation of O(h2) and
O(h4) to estimate the first derivative of
f(x)= 0.1x5-0.2x3+0.1x-0.2

at x = 0.1 Compare numerical solution at h=0.1.

Solution: The data needed for this example is

xi+2 x+2h = 0.3 f(xi+2) f(0.3) = - 0.17516
xi+1 x+h = 0.2 f(xi+1) f(0.2) = - 0.181568
xi x = 0.1 f(xi) f(0.1) = - 0.190199
xi-1 x-h =0 f(xi-1) f(0) = -0.2
xi-2 x-2h = -0.1 f(xi-2) f(-0.1) = - 0.2098

The forward difference;

𝑓 0.2 −𝑓(0.1) − 0.181568−(−0.190199)

f’(0.1)= = =0.08631
ℎ 0.1

−𝑓 0.3 +4𝑓 0.2 −3𝑓(𝑥) 0.17516 +4(− 0.181568 )−3(−0.190199)

f’(0.1)= = =0.0993
2ℎ 0.2
The backward difference of ;

f 0.1 −f(0) − 0.190199−(−0.2)

f’(0.1)= = =0.09801
h 0.1

3f 0.1 −4f 0 +f(−0.1) 0.17516 +4(− 0.181568 )−3(−0.190199)

f’(0.1)= =
2h 0.2

=0.098015

The centered difference of accuracy ;

f 0.2 −f(0) − 0.181568−(−0.2)

f’(0.1)= = =0.09216
2h 0.2

−f 0.3 +8f 0.2 −8f 0 +f(−0.1)

f ’(0.1)=
12h
− 0.17516 +8(− 0.181568 )−8(−0.2)− 0.2098
= =0.09401
1.2
Example 4: Using the same function as in Example 2, estimate the first derivative at x = 0.5 employing step sizes of h1 = 0.5 and
h2 = 0.25. Then use Eq. (16) to compute an improved estimate with Richardson extrapolation. Recall that the true value is −0.9125.

Solution: The first-derivative estimates can be computed with centered differences as

f xi+2 −f(xi−2 )
D(xi)=
(xi+2 )−(xi−2 )
And

f xi+1 −f(xi−1 )
D(xi-1)=
(xi+1 )−(xi−1 )

The improved estimate can be determined by applying Eq. (16) to give

which for the present case is a perfect result.

REFERENCES

• Jaan Kiusalaas, “Numerical Methods in Engineering with Python 3”,3rd Edition, Cambridge, NY, 2013
• S.C. Chapra and R.P. Canale, “Numerical Methods for Engineers”, 6th ed., McGraw-Hill,, NY, 2010