Numerical Dierentiation

Mg. Dennis Quispe Sánchez

March 23, 2024

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 1 / 22

Outline

1 Introduction

2 Forward and Backward dierence formula

3 (n+1)-Point Formula to approximate f ′ (xj )

Three-Point Formula
Five-Point Formula

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 2 / 22

Introduction

Introduction

The derivative approximation formulas are used extensively for

approximating the solutions to ordinary and partial dierential equations.
The derivative of the function f at x0 is dened as:
f (x0 + h) − f (x0 )
f ′ (x0 ) = lim
h→0 h
This formula gives an obvious way to generate an approximation to f ′ (x0 );
simply compute
f (x0 + h) − f (x0 )
h
for small values of h. Although this may be obvious, it is not very
successful, due to our old nemesis, round-o error. But it is certainly the
place to start.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 3 / 22

To approximate f ′ (x0 ), suppose rst that x0 ∈ (a, b), where f ∈ C 2 [a, b],
and that x1 = x0 + h for some h > 0 that is suciently small to ensure
that x1 ∈ [a, b]. We construct the rst Lagrange polynomial, P0,1 , for f
determined by x0 and x1 with its error term.
(x − x0 )(x − x1 ) ′′
f (x) = P0,1 (x) + f (ξ(x))
2!
f (x0 )(x − x0 − h) f (x0 + h)(x − x0 ) (x − x0 )(x − x0 − h) ′′
= + + f (ξ(x))
−h h 2

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 4 / 22

Let's consider some ξ(x) between x0 and x1 . Dierentiating gives:
 
′ f (x0 + h) − f (x0 ) (x − x0 )(x − x0 − h) ′′
f (x) = + Dx f (ξ(x))
h 2

f (x0 + h) − f (x0 ) 2(x − x0 ) − h ′′ (x − x0 )(x − x0 − h)

= + ·f (ξ(x))+ ·Dx (f ′′ (ξ
h 2 2
Deleting the terms involving ξ(x) gives:
f (x0 + h) − f (x0 )
f (x) ≈
h

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 5 / 22

One diculty with this formula is that we have no information about
Dx f ′′ (ξ(x)), so the truncation error cannot be estimated. When x is x0 ,
however, the coecient of Dx f ′′ (ξ(x)) is 0, and the formula simplies to
f (x0 + h) − f (x0 ) h ′
f (x0 ) = − f (ξ).
h 2
For small values of h, the dierence quotient f (x0 +h)−f
h
(x0 )
can be used to
approximate f (x0 ) with an error bounded by 2 , where M is a bound on
′ M|h|

|f (x)| for x between x0 and x0 + h. This formula is known as the

forward-dierence formula if h > 0 and the backward-dierence formula if
h < 0.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 6 / 22

Simulation

Figure: Numerical aproximation.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 7 / 22

Example
Example

Use the forward-dierence formula to approximate the derivative of

f (x) = ln x at x0 = 1.8 using h = 0.1, h = 0.05, and h = 0.01, and
determine bounds for the approximation errors.

f (1.8 + h) − f (1.8)
h
With h = 0.1 gives:
ln 1.9 − ln 1.8 0.64185389 − 0.58778667
= = 0.5406722.
0.1 0.1
Because f (x) = − x12 and 1.8 < ξ < 1.9, a bound for this approximation
error is:
|hf ′′ (ξ)| |h| 0.1
= 2 < = 0.0154321.
2 2ξ 2(1.8)2
The approximation and error bounds when h = 0.05 and h = 0.01 are
found in a similar manner andNumerical
Mg. Dennis Quispe Sánchez
the results are shown in theMarch
Dierentiation
table.
23, 2024 8 / 22
Simulation

Figure: Numerical aproximation.

Since f ′ (x) = x1 , the exact value of f (1.8) is 0.555, and in this case, the
error bounds are quite close to the true approximation error.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 9 / 22

To obtain general derivative approximation formulas, suppose that
{x0 , x1 , . . . , xn } are (n + 1) distinct numbers in some interval I and that
f ∈ C n+1 (I ). From Theorem of Lagrange:
n
X (x − x0 ) . . . (x − xn ) (n+1)
f (x) = f (xk )Lk (x) + f (ξ(x)),
(n + 1)!
k=0

for some ξ(x) in I , where Lk (x) denotes the k th Lagrange coecient

polynomial for f at x0 , x1 , . . . , xn . Dierentiating this expression gives

n  
(x − x0 ) . . . (x − xn )
f (n+1) (ξ(x)).
X
f ′ (x) = f (xk )L′k (x) + Dx
(n + 1)!
k=0

(x − x0 ) . . . (x − xn ) h (n+1) i
+ Dx f (ξ(x)) .
(n + 1)!

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 10 / 22

We again have a problem estimating the truncation error unless x is one of
the numbers xj . In this case, the term multiplying Dx [f (n+1) (ξ(x))] is 0,
and the formula becomes

f (n+1) (ξ(xj )) Y
n
X n
′
f (xj ) = f (xk )L′k (xj ) + (xj − xk ),
(n + 1)!
k=0 k=0
k̸=j

which is called an (n + 1)-point formula to approximate f ′ (xj ). In general,

using more evaluation points produces greater accuracy, although the
number of functional evaluations and growth of round-o error discourages
this somewhat. The most common formulas are those involving three and
ve evaluation points. We rst derive some useful three-point formulas and
consider aspects of their errors.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 11 / 22

Because
(x − x1 )(x − x2 )
L0 (x) = ,
(x0 − x1 )(x0 − x2 )
we have
2x − x1 − x2
L′0 (x) = .
(x0 − x1 )(x0 − x2 )
Similarly,
2x − x0 − x2
L′1 (x) =
(x1 − x0 )(x1 − x2 )
and
2x − x0 − x1
L′2 (x) = .
(x2 − x0 )(x2 − x1 )
Hence,
2xj − x1 − x2 2xj − x0 − x2
f ′ (xj ) = f (x0 ) · + f (x1 ) ·
(x0 − x1 )(x0 − x2 ) (x1 − x0 )(x1 − x2 )
2
2xj − x0 − x1 1 (3) Y
+f (x2 ) · + f (ξj ) (xj − xk ).
(x2 − x0 )(x2 − x1 ) 6
k=0
k̸=j

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 12 / 22

for each j = 0, 1, 2, where the notation ξj indicates that this point depends
on xj .
When x1 = x0 + h and x2 = x0 + 2h, for some h ̸= 0. We will assume
equally-spaced nodes. Using the last equation with xj = x0 , x1 = x0 + h,
and x2 = x0 + 2h gives
3 1 1 h2
 
′
f (x0 ) = − f (x0 ) + 2f (x1 ) − f (x2 ) + f (3) (ξ0 ).
h 2 2 3
Doing the same for xj = x1 gives
1 1 1 h2
 
f ′ (x1 ) = − f (x0 ) + f (x2 ) − f (3) (ξ1 ),
h 2 2 6
and for xj = x2 ,
1 1 3 h2
 
′
f (x2 ) = f (x0 ) − 2f (x1 ) + f (x2 ) + f (3) (ξ2 ).
h 2 2 3

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 13 / 22

Since x1 = x0 + h and x2 = x0 + 2h, these formulas can also be expressed as

1 3 1 h2
 
′
f (x0 ) = − f (x0 ) + 2f (x0 + h) − f (x0 + 2h) + f (3) (ξ0 ),
h 2 2 3

1 1 1 h2
 
′
f (x0 + h) = − f (x0 ) + f (x0 + 2h) − f (3) (ξ1 ),
h 2 2 6
and

1 1 3 h2
 
f (x0 + 2h) =
′
f (x0 ) − 2f (x0 + h) + f (x0 + 2h) + f (3) (ξ2 ).
h 2 2 3
As a matter of convenience, the variable substitution x0 for x0 + h is used in
the middle equation to change this formula to an approximation for f ′ (x0 )

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 14 / 22

A similar change, x0 for x0 + 2h, is used in the last equation. This gives
three formulas for approximating f (x0 ):
1 h2
f ′ (x0 ) = [−3f (x0 ) + 4f (x0 + h) − f (x0 + 2h)] + f ′′ (ξ0 ),
2h 3
1 h2
f ′ (x0 ) = [−f (x0 − h) + f (x0 + h)] − f ′′ (ξ1 ),
2h 6

and

1 h2
f ′ (x0 ) = [f (x0 − 2h) − 4f (x0 − h) + 3f (x0 )] + f ′′ (ξ2 ).
2h 3
Finally, note that the last of these equations can be obtained from the rst
by simply replacing h with −h, so there are actually only two formulas:

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 15 / 22

Three-Point Endpoint Formula

1 h2
f ′ (x0 ) = [−3f (x0 ) + 4f (x0 + h) − f (x0 + 2h)] + f ′′′ (ξ0 ),
2h 3
where ξ0 lies between x0 and x0 + 2h

Three-Point Midpoint Formula

1 h2
f (x0 ) =[f (x0 + h) − f (x0 − h)] − f ′′′ (ξ1 ),
2h 6
where ξ1 lies between x0 − h and x0 + h.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 16 / 22

 Figure: Three-Point Formula.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 17 / 22

Five-Point Midpoint Formula

1 h4
f ′ (x0 ) = [f (x0 − 2h) − 8f (x0 − h) + 8f (x0 + h) − f (x0 + 2h)]+ f (5) (ξ)
12h 30
where ξ lies between x0 − 2h and x0 + 2h.

Five-Point Endpoint Formula

1
f ′ (x0 ) = [−25f (x0 ) + 48f (x0 + h) − 36f (x0 + 2h)
12h
h4
+16f (x0 + 3h) − 3f (x0 + 4h)] + f (5) (ξ)
5
where ξ lies between x0 and x0 + 4h.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 18 / 22

Example 2

Values for f (x) = x · e x are given in Table 2. Use all the applicable
three-point and ve-point formulas to approximate f ′ (2.0).
x f (x)
1.8 10.889365
1.9 12.703199
2.0 14.778112
2.1 17.148957
2.2 19.855030

The data in the table permit us to nd four dierent three-point

approximations. We can use the endpoint formula with h = 0.1 or with
h = −0.1, and we can use the midpoint formula with h = 0.1 or with
h = 0.2.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 19 / 22

Using the endpoint formula with h = 0.1 gives
1 1
[−3f (2.0)+4f (2.1)−f (2.2)] = [−3(14.778112)+4(17.148957)−19.85503
0.2 5
= 22.032310. and with h = −0.1 gives 22.054525.
Using the midpoint formula with h = 0.1 gives
1 1
[f (2.1) − f (1.9)] = (17.148957 − 12.703199) = 22.228790,
0.2 5
and with h = 0.2 gives 22.414163.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 20 / 22

The only ve-point formula for which the table gives sucient data is the
midpoint formula (4.6) with h = 0.1. This gives
1
[f (1.8) − 8f (1.9) + 8f (2.1) − f (2.2)] =
1.2
1
[10.889365 − 8(12.703199) + 8(17.148957) − 19.855030] = 22.166999.
1.2
If we had no other information we would accept the ve-point midpoint
approximation using h = 0.1 as the most accurate, and expect the true
value to be between that approximation and the three-point midpoint
approximation that is in the interval [22.166, 22.229].
The true value in this case is f ′ (2.0) = (2 + 1)e 2 = 22.167168, so the
approximation errors are actually:
- Three-point endpoint with h = 0.1: 1.35 × 10−1 ;
- Three-point endpoint with h = −0.1: 1.13 × 10−1 ;
- Three-point midpoint with h = 0.1: −6.16 × 10−2 ;
- Three-point midpoint with h = 0.2: −2.47 × 10−1 ;
- Five-point midpoint with h = 0.1: 1.69 × 10−4 .
Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 21 / 22
 Figure: Code.

Mg. Dennis Quispe Sánchez Numerical Dierentiation March 23, 2024 22 / 22