APMA 0160 (A.

Yew) Spring 2011

Numerical differentiation: finite differences

The derivative of a function f at the point x is defined as the limit of a difference quotient:

f (x + h) − f (x)
f 0 (x) = lim
h→0 h
f (x + h) − f (x)
In other words, the difference quotient is an approximation of the derivative f 0 (x), and
h
this approximation gets better as h gets smaller.
How does the error of the approximation depend on h?
Taylor’s theorem with remainder gives the Taylor series expansion

f 00 (ξ)
f (x + h) = f (x) + hf 0 (x) + h2 where ξ is some number between x and x + h.
2!
Rearranging gives
f (x + h) − f (x) f 00 (ξ)
− f 0 (x) = h ,
h 2
f (x + h) − f (x)
which tells us that the error is proportional to h to the power 1, so is said to be a
h
“first-order” approximation.
If h > 0, say h = ∆x where ∆x is a finite (as opposed to infinitesimal) positive number, then

f (x + ∆x) − f (x)
∆x
is called the first-order or O(∆x) forward difference approximation of f 0 (x).
If h < 0, say h = −∆x where ∆x > 0, then

f (x + h) − f (x) f (x) − f (x − ∆x)

=
h ∆x
is called the first-order or O(∆x) backward difference approximation of f 0 (x).
By combining different Taylor series expansions, we can obtain approximations of f 0 (x) of various orders.
For instance, subtracting the two expansions

f 00 (x) f 000 (ξ1 )

f (x + ∆x) = f (x) + ∆x f 0 (x) + ∆x2 + ∆x3 , ξ1 ∈ (x, x + ∆x)
2! 3!
f 00 (x) f 000 (ξ2 )
f (x − ∆x) = f (x) − ∆x f 0 (x) + ∆x2 − ∆x3 , ξ2 ∈ (x − ∆x, x)
2! 3!
f 000 (ξ1 ) + f 000 (ξ2 )


0
gives f (x + ∆x) − f (x − ∆x) = 2∆x f (x) + ∆x 3 , so that
6
000 000


f (x + ∆x) − f (x − ∆x) 0 2 f (ξ1 ) + f (ξ2 )
− f (x) = ∆x
2∆x 12
f (x + ∆x) − f (x − ∆x)
Hence is an approximation of f 0 (x) whose error is proportional to ∆x2 . It is
2∆x
called the second-order or O(∆x2 ) centered difference approximation of f 0 (x).
If we use expansions with more terms, higher-order approximations can be derived, e.g. consider

f 00 (x) f 000 (x) f (4) (x) f (5) (ξ1 )

f (x + ∆x) = f (x) + ∆x f 0 (x) + ∆x2 + ∆x3 + ∆x4 + ∆x5
2! 3! 4! 5!
00
f (x) 000
f (x) (4)
f (x) (5)
f (ξ2 )
f (x − ∆x) = f (x) − ∆x f 0 (x) + ∆x2 − ∆x3 + ∆x4 − ∆x5
2! 3! 4! 5!
00
f (x) 000
f (x) (4)
f (x) f (5) (ξ3 )
f (x + 2∆x) = f (x) + 2∆x f 0 (x) + 4∆x2 + 8∆x3 + 16∆x4 + 32∆x5
2! 3! 4! 5!
f 00 (x) f 000 (x) f (4) (x) f (5) (ξ )
4
f (x − 2∆x) = f (x) − 2∆x f 0 (x) + 4∆x2 − 8∆x3 + 16∆x4 − 32∆x5
2! 3! 4! 5!
Taking 8 × (first expansion − second expansion) − (third expansion − fourth expansion) cancels out the
∆x2 and ∆x3 terms; rearranging then yields a fourth-order centered difference approximation of f 0 (x).
Approximations of higher derivatives f 00 (x), f 000 (x), f (4) (x) etc. can be obtained in a similar manner.
For example, adding
f 00 (x) f 000 (x) f (4) (ξ1 )
f (x + ∆x) = f (x) + ∆x f 0 (x) + ∆x2 + ∆x3 + ∆x4 ···
2! 3! 4!
f 00 (x) f 000 (x) f (4) (ξ2 )
f (x − ∆x) = f (x) − ∆x f 0 (x) + ∆x2 − ∆x3 + ∆x4 ···
2! 3! 4!
(4) (ξ ) + f (4) (ξ )


f 1 2
gives f (x + ∆x) + f (x − ∆x) = 2f (x) + ∆x2 f 00 (x) + ∆x4 , so that
24
(4) (ξ ) + f (4) (ξ )


f (x + ∆x) − 2f (x) + f (x − ∆x) 00 2 f 1 2
− f (x) = ∆x
∆x2 24
f (x + ∆x) − 2f (x) + f (x − ∆x)
Hence is a second-order centered difference approximation of the sec-
00
∆x2
ond derivative f (x).

Here are some commonly used second- and fourth-order “finite difference” formulas for approximating
first and second derivatives:
O(∆x2 ) centered difference approximations:
f 0 (x) : f (x + ∆x) − f (x − ∆x) /(2∆x)


f 00 (x) : f (x + ∆x) − 2f (x) + f (x − ∆x) /∆x2

O(∆x2 ) forward difference approximations:

f 0 (x) : −3f (x) + 4f (x + ∆x) − f (x + 2∆x) /(2∆x)


f 00 (x) : 2f (x) − 5f (x + ∆x) + 4f (x + 2∆x) − f (x + 3∆x) /∆x3

O(∆x2 ) backward difference approximations:

0

f (x) : 3f (x) − 4f (x − ∆x) + f (x − 2∆x) /(2∆x)
f 00 (x) : 2f (x) − 5f (x − ∆x) + 4f (x − 2∆x) − f (x − 3∆x) /∆x3

O(∆x4 ) centered difference approximations:

f 0 (x) : −f (x + 2∆x) + 8f (x + ∆x) − 8f (x − ∆x) + f (x − 2∆x) /(12∆x)


f 00 (x) : −f (x + 2∆x) + 16f (x + ∆x) − 30f (t) + 16f (x − ∆x) − f (x − 2∆x) /(12∆x2 )

In science and engineering applications it is often the case that an exact formula for f (x) is not known.
We may only have a set of data points (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ) available to describe the functional
dependence y = f (x). If we need to estimate the rate of change of y with respect to x in such a situation,
we can use finite difference formulas to compute approximations of f 0 (x). It is appropriate to use a
forward difference at the left endpoint x = x1 , a backward difference at the right endpoint x = xn , and
centered difference formulas for the interior points.