Computer Science 117

Numerical Methods: Numerical Differentiation

Olive R. Cawiding

Department of Mathematics and Computer Science

College of Science, University of the Philippines
Gov. Pack Road, Baguio City 2600 Philippines
[email protected]
Computer Science 117

Recall.

Definition.
The derivative of a function f at x0 is given by

f (x0 + h) − f (x0 )
f 0 (x0 ) = lim (0.1)
h→ h

Goal.
To approximate f 0 (x0 ) using different techniques
Computer Science 117

Numerical Differentiation

Suppose x0 ∈ (a, b), where f ∈ C 2 [a, b], and that x1 = x0 + h, for

some h 6= 0 that is sufficiently small to ensure that x1 ∈ [a, b].
Then,
f (x0 + h) − f (x0 ) 2(x − x0 ) − h 00
f 0 (x) = + f (ξ(x))
h 2
(x − x0 )(x − x0 − h)
+ Dx (f 00 (ξ(x))).
2

If x = x0 , then
f (x0 + h) − f (x0 ) h 00
f 0 (x0 ) = − f (ξ(x0 )). (0.2)
h 2
Computer Science 117

Numerical Differentiation

f (x0 + h) − f (x0 )
For small values of h, the difference quotient
h
can be used to approximate f 0 (x0 ) with an error bounded by
M |h|
, where M is a bound on |f 00 (x)| for x ∈ [a, b].
2
This formula is known as the forward-difference formula if h > 0
and the backward-difference formula if h < 0.
Computer Science 117

Numerical Differentiation
Computer Science 117

Numerical Differentiation

EXAMPLE.
Use the forward difference formula to approximate f 0 (1.8) with
h = 0.1. Find a bound for its error.
Computer Science 117

Numerical Differentiation

We obtain general derivative approximation formulas using a

theorem from our discussion on Lagrange polynomials.

Suppose that x0 , x1 , . . . , xn are (n + 1) distinct numbers in some

interval I and that f ∈ C n+1 (I). Then,
n
X (x − x0 ) . . . (x − xn ) (n+1)
f (x) = f (xk )Lk (x) + f (ξ(x)),
(n + 1)!
k=0
(0.3)
for some ξ(x) ∈ I.
Computer Science 117

Numerical Differentiation

Differentiation leads to
n  
0
X (x − x0 ) . . . (x − xn ) (n+1)
f (x) = f (xk )L0k (x) + Dx f (ξ(x))
(n + 1)!
k=0

(x − x0 ) . . . (x − xn )
+ Dx [f (n+1) (ξ(x))].
(n + 1)!
Computer Science 117

Numerical Differentiation

If x = xj , then
n n
X f (n+1)(ξ(xj )) Y
f 0 (xj ) = f (xk )L0k (x) + (xj − xk ). (0.4)
(n + 1)!
k=0 k=0,k6=j

This is called an (n+1)-point formula to approximate f 0 (xj ).

Computer Science 117

Three-Point Formulas

Consider the points x0 , x1 , and x2 . A three-point formula for

f 0 (xj ) is given by
   
0 2xj − x1 − x2 2xj − x0 − x2
f (xj ) = f (x0 ) + f (x1 )
(x0 − x1 )(x0 − x2 ) (x1 − x0 )(x1 − x2 )
  2
2xj − x0 − x1 1 (3) Y
+ f (x2 ) + f (ξj ) (xj − xk )
(x2 − x0 )(x2 − x1 ) 6
k=0,k6=j

for each j = 0, 1, 2, where the notation ξj indicates that this point

depends on xj .
Computer Science 117

Three-Point Formulas

We consider the case where these three nodes are equally spaced,
i.e.,

x1 = x0 + h and x2 = x0 + 2h, for some h 6= 0.

Computer Science 117

Three-Point Endpoint Formula

1 h2
f 0 (x0 ) = [−3f (x0 ) + 4f (x0 + h) − f (x0 + 2h)] + f (3) (ξ0 )
2h 3
where ξ0 lies between x0 and x0 + 2h
Computer Science 117

Three-Point Midpoint Formula

1 h2
f 0 (x0 ) = [f (x0 + h) − f (x0 − h)] − f (3) (ξ1 )
2h 6
where ξ1 lies between (x0 − h) and (x0 + h).

Remark.
The error term for both three-point formulas is O(h2 ).
Computer Science 117

Five-Point Formulas

Five-Point Midpoint Formula

1
f 0 (x0 ) = [f (x0 − 2h) − 8f (x0 − h) + 8f (x0 + h) − f (x0 + 2h)]
12h
h4 (5)
+ f (ξ)
30
where ξ lies between x0 − 2h and x0 + 2h.
Computer Science 117

Five-Point Formulas

Five-Point Endpoint Formula

1
f 0 (x0 ) = [−25f (x0 ) + 48f (x0 + h) − 36f (x0 + 2h)]
12h
h4
+ 16f (x0 + 3h) − 3f (x0 + 4h)] + f (5) (ξ)
5
where ξ lies between x0 and x0 + 4h.
This formula can be used for left-endpoint approximations with
h > 0 and for right-endpoint approximations with h < 0.

Remark.
The error term for both five-point formulas is O(h4 ).
Computer Science 117

Numerical Differentiation

EXAMPLE.
The values for f (x) = xex are given in the following table. Use
applicable three-point and five-point formulas to approximate
f 0 (2.0).
Computer Science 117

Numerical Differentiation

The actual value is given by f 0 (2.0) = (2 + 1)e2 = 22.167168.

Results of the different formula are given in the following table.
Formula h Approximation to f 0 (2.0) Error
Three-point endpoint 0.1 22.032310 1.35 × 10−1
Three-point endpoint −0.1 22.054525 1.13 × 10−1
Three-point midpoint 0.1 22.228790 −6.16 × 10−2
Three-point midpoint 0.2 22.414163 −2.47 × 10−1
Five-point midpoint 0.1 22.166999 1.69 × 10−4
Computer Science 117

Approximations of Higher Derivatives

Second Derivative Midpoint Formula

1 h2 (4)
f 00 (x0 ) = [f (x 0 − h) − 2f (x 0 ) + f (x 0 + h)] − f (ξ) (0.5)
h2 12
for some ξ, where x0 − h < ξ < x0 + h.

EXAMPLE.
Use the table of values in the previous example to approximate
f 00 (2.0).
Computer Science 117

Approximations of Higher Derivatives

Approximations using the second derivative midpoint formula is

given in the following table.
h Approximation to f 00 (2.0) Error
0.1 29.593200 −3.70 × 10−2
0.2 29.704275 −1.48 × 10−1

The actual value is given by f 00 (2.0) = 29.556224.

Computer Science 117

The Instability of Round-off Errors

We examine the three-point midpoint formula

1 h2
f 0 (x0 ) = [f (x0 + h) − f (x0 − h)] − f (3) (ξ1 )
2h 6
. Suppose that in evaluating f (x0 + h) and f (x0 − h) we
encounter round-off errors e(x0 + h) and e(x0 − h), i.e.,

f (x0 + h) = f˜(x0 + h) + e(x0 + h) and (0.6)

f (x0 − h) = f˜(x0 − h) + e(x0 − h) (0.7)
Computer Science 117

The Instability of Round-off Errors

Hence, the total error in approximation is

f˜(x0 + h) − f˜(x0 − h) e(x0 + h) − e(x0 − h) h2 (3)

f 0 (x0 )− = − f (ξ1 ).
2h 2h 6
(0.8)
Suppose that the round off errors e(x0 ± h) are bounded by some
number  > 0 and that the third derivative of f is bounded by a
number M > 0, then
 

 0 f˜(x0 + h) − f˜(x0 − h)   h2
f (x0 ) −  ≤ + M. (0.9)
 2h  h 6
Computer Science 117

The Instability of Round-off Errors

Problem.
We want to minimize the error bound for the three-point midpoint
formula given by
 h2
E(h) = + M. (0.10)
h 6
At what value of h is the error bound minimized?
Computer Science 117

The Instability of Round-off Errors

EXAMPLE.
Consider the following values of f (x) = sin x using different values
of x. Use these values to approximate f 0 (0.900).
Computer Science 117

The Instability of Round-off Errors

The formula
f (0.900 + h) − f (0.900 − h)
f 0 (0.900) ≡ (0.11)
2h
with different values of h gives the following approximations.
Computer Science 117

The Instability of Round-off Errors

The true value for f 0 (0.900) is given by cos(0.900) = 0.62161.

The optimal choice for h appears to lie between 0.005 and 0.05.
We can verify this by checking the minimum error bound.
Computer Science 117

Seatwork.

Let f (x) = 3xex − cos x. Approximate f 0 (1.3) and f 00 (1.3) with

h = 0.1 and h = 0.01 using all formulas discussed. Compare your
results to the actual values for f 0 (1.3) and f 00 (1.3).