Lecture 7

Numerical Differentiation

First order derivatives

High order derivatives
Richardson Extrapolation

1
Motivation
How do you evaluate the Time Displacement
(second) (meters)
derivative of a tabulated
function. 0 30.1

How do we determine the 5 48.2

velocity and acceleration
from tabulated 10 50.0

measurements.
15 40.2

2
Recall
df f ( x + h) − f ( x )
= lim
dx h→0 h

Taylor Theorem :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3
f ( x + h) = f ( x ) + f ' ( x ) h + + + O(h 4 )
2! 3!
n
E = O(h ) ⇒ ∃ real , finite C , such that : E ≤ C h
n

E is of order h n ⇒ E is approaching zero at rate similar to h n

3
Three Formula
df ( x) f ( x + h) − f ( x)
Forward Difference : =
dx h

df ( x) f ( x) − f ( x − h)
Backward Difference : =
dx h

df ( x) f ( x + h) − f ( x − h)
Central Difference : =
dx 2h

Which method is better? How do we judge them?

4
The Three Formulas

5
Forward/Backward Difference Formula
Forward Difference : f ( x + h) = f ( x) + f ' ( x) h + O ( h 2 )
⇒ f ' ( x ) h = f ( x + h) − f ( x ) + O ( h 2 )
f ( x + h) − f ( x )
⇒ f ' ( x) = + O ( h)
h
_______________________________________________________
Backward Difference : f ( x − h) = f ( x ) − f ' ( x ) h + O ( h 2 )
⇒ f ' ( x ) h = f ( x ) − f ( x − h) + O ( h 2 )
f ( x ) − f ( x − h)
⇒ f ' ( x) = + O ( h)
h

6
Central Difference Formula
Central Difference :
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x + h) = f ( x ) + f ' ( x ) h + + + + ...
2! 3! 4!
f ( 2 ) ( x ) h 2 f ( 3) ( x ) h 3 f ( 4 ) ( x ) h 4
f ( x − h) = f ( x ) − f ' ( x ) h + − + + ...
2! 3! 4!
f ( 3) ( x ) h 3
f ( x + h) − f ( x − h) = 2 f ' ( x ) h + 2 + ...
3!
f ( x + h) − f ( x − h )
⇒ f ' ( x) = + O (h 2 )
2h

7
The Three Formula (Revisited)
df ( x) f ( x + h) − f ( x)
Forward Difference : = + O ( h)
dx h

df ( x) f ( x) − f ( x − h)
Backward Difference: = + O ( h)
dx h

df ( x) f ( x + h) − f ( x − h)
Central Difference : = + O(h 2 )
dx 2h

Forward and backward difference formulas are comparable in accuracy.

Central difference formula is expected to give a better answer.

8
Higher Order Formulas
f ( 2) ( x)h 2 f (3) ( x)h3 f ( 4) ( x)h 4
f ( x + h) = f ( x ) + f ' ( x ) h + + + + ...
2! 3! 4!
f ( 2) ( x)h 2 f (3) ( x)h3 f ( 4) ( x)h 4
f ( x − h) = f ( x ) − f ' ( x ) h + − + + ...
2! 3! 4!
f ( 2) ( x ) h 2 f ( 4) ( x ) h 4
f ( x + h) + f ( x − h ) = 2 f ( x ) + 2 +2 + ...
2! 4!
f ( x + h) − 2 f ( x ) + f ( x − h)
⇒ f ( 2) ( x ) = 2
+ O ( h 2
)
h

f ( 4) (ξ )h 2
Error = −
12
9
Other Higher Order Formulas
f ( x + h) − 2 f ( x) + f ( x − h)
f ( 2) ( x ) =
h2
f ( x + 2h ) − 2 f ( x + h ) + 2 f ( x − h ) − f ( x − 2h )
f (3) ( x ) =
2h 3
f ( x + 2 h ) − 4 f ( x + h ) + 6 f ( x ) − 4 f ( x − h ) + f ( x − 2h )
f ( 4) ( x ) =
h4

Central Formulas with Error = O ( h 2 )

Other formulas for f ( 2) ( x ), f (3) ( x )... are also possible.
You can use Taylor Theorem to prove them and obtain the error order.

10
Example
Use forward, backward and centered difference
approximations to estimate the first derivative of:
f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2
at x = 0.5 using step size h = 0.5 and h = 0.25
Note that the derivative can be obtained directly:
f’(x) = –0.4x3 – 0.45x2 – 1.0x – 0.25
The true value of f’(0.5) = -0.9125
In this example, the function and its derivative are
known. However, in general, only tabulated data
might be given.
11
Solution with Step Size = 0.5
f(0.5) = 0.925, f(0) = 1.2, f(1.0) = 0.2
Forward Divided Difference:
f’(0.5) ≈ (0.2 – 0.925)/0.5 = -1.45
|εt| = |(-0.9125+1.45)/-0.9125| = 58.9%
Backward Divided Difference:
f’(0.5) ≈ (0.925 – 1.2)/0.5 = -0.55
|εt| = |(-0.9125+0.55)/-0.9125| = 39.7%
Centered Divided Difference:
f’(0.5) ≈ (0.2 – 1.2)/1.0 = -1.0
|εt| = |(-0.9125+1.0)/-0.9125| = 9.6%
12
Solution with Step Size = 0.25
f(0.5)=0.925, f(0.25)=1.1035, f(0.75)=0.6363
Forward Divided Difference:
f’(0.5) ≈ (0.6363 – 0.925)/0.25 = -1.155
|εt| = |(-0.9125+1.155)/-0.9125| = 26.5%
Backward Divided Difference:
f’(0.5) ≈ (0.925 – 1.1035)/0.25 = -0.714
|εt| = |(-0.9125+0.714)/-0.9125| = 21.7%
Centered Divided Difference:
f’(0.5) ≈ (0.6363 – 1.1035)/0.5 = -0.934
|εt| = |(-0.9125+0.934)/-0.9125| = 2.4%
13
Discussion
For both the Forward and Backward difference,
the error is O(h)
Halving the step size h approximately halves the
error of the Forward and Backward differences
The Centered difference approximation is more
accurate than the Forward and Backward
differences because the error is O(h2)
Halving the step size h approximately quarters
the error of the Centered difference.

14
Richardson Extrapolation
f ( x + h) − f ( x − h)
Central Difference : f ' ( x) = + O (h 2 )
2h

Can we get a better formula?

Hold f ( x) and x fixed :

f ( x + h) − f ( x − h )
φ ( h) =
2h
φ (h) = f ' ( x) − a2 h 2 − a4 h 4 − a6 h 6 − ...

15
Richardson Extrapolation
To get a better formula:

Hold f ( x) and x fixed :

f ( x + h ) − f ( x − h)
φ ( h) =
2h
φ (h) = f ' ( x) − a2 h 2 − a4 h 4 − a6 h 6 − ...
2 4 6Use two derivative
h h h h
φ ( ) = f ' ( x) − a2   − a4   − a6   − ... estimates to
2 2 2 2 compute a third,
h 3 15
φ (h) − 4φ   = −3 f ' ( x) − a4 h 4 − a6 h 6 − ... more accurate
2 4 16 approximation
4 1
⇒ f ' ( x ) = φ ( h / 2) − φ ( h ) + O ( h 4 )
3 3
16
Richardson Extrapolation Example
Use the function:

f(x) = –0.1x4 – 0.15x3 – 0.5x2 – 0.25x + 1.2

Starting with h1 = 0.5 and h2 = 0.25, compute

an improved estimate of f’(0.5) using
Richardson Extrapolation

Recall the true value of f’(0.5) = -0.9125

17
Solution
The first-derivative estimates can be computed
with centered differences as:
f ( 0 .5 + h ) − f ( 0 .5 − h )
φ ( h) = at x = 0.5
2h
f (1) − f (0) 0.2 − 1.2
φ (0.5) = = = −1.0, | ε t | = 9.6%
1 1
f (0.75) − f (0.25)
φ (0.25) = = −0.934375, | ε t | = 2.4%
0 .5
The improved estimate can be obtained by applying :
4 1 4 1
f ' (0.5) ≅ φ (h / 2) − φ (h) = (−0.934375) − (−1) = −0.9125
3 3 3 3
which produces the exact result for this example
18
 Higher Order
4 1
f ' ( x) = φ (h / 2) − φ (h) + O(h 4 ) = ρ (h) + a4 h 4 + a6 h 6 + ...
3 3
4 1
where ρ (h) = φ (h / 2) − φ (h)
3 3
1
= [8 f ( x + h / 2) − 8 f ( x − h / 2) − f ( x + h) + f ( x − h)]
12h
4 6
h h h
ρ (h) = f ' ( x) − a4 h − a6 h − ... ; ρ ( ) = f ' ( x) − a4   − a6   − ...
4 6

2 2 2
h 3
ρ (h) − 16 ρ   = −15 f ' ( x) − a6 h 6 − ...
2 4
1  h 
⇒ f ' ( x) = 16 ρ   − ρ (h) + O(h 6 ) 19
15  2 
 Richardson Extrapolation Table
Repeating this operation, one can obtain the following table:

D(0,0)=Φ(h)

D(1,0)=Φ(h/2) D(1,1)

D(2,0)=Φ(h/4) D(2,1) D(2,2)

D(3,0)=Φ(h/8) D(3,1) D(3,2) D(3,3)

Error Level O(h 2 ) O(h 4 ) O(h 6 ) O(h8 )

20
Richardson Extrapolation Table
 h 
First Column : D ( n,0) = φ  n 
2 

Others :
4m 1
D ( n, m ) = m D ( n, m − 1) − m D ( n − 1, m − 1)
4 −1 4 −1

21
Example
Evaluate numerically the derivative of :
f ( x) = x cos( x ) at x = 0.6

Use Richardson Extrapolation with h = 0.1

Obtain D(2,2) as the estimate of the derivative .

TRUE VALUE : 1.091570709288434

22
Example
First Column
f ( x + h)-f ( x-h)
φ ( h) =
2h
f (0.7)-f (0.5)
φ (0.1) = = 1.08483
0.2
f (0.65)-f (0.55)
φ (0.05) = = 1.08988
0.1
f (0.625)-f (0.575)
φ (0.025) = = 1.09115
0.05
23
Example
Richardson Table
D (0,0) = 1.08483 , D(1,0) = 1.08988, D(2,0) = 1.09115

4m 1
D ( n, m ) = m D ( n, m − 1) − m D ( n − 1, m − 1)
4 −1 4 −1
4 1
D (1,1) = D (1,0) − D (0,0) = 1.09156
3 3
4 1
D ( 2 ,1) = D ( 2 ,0) − D (1,0) = 1.09157
3 3
16 1
D ( 2 ,2) = D ( 2 ,1) − D (1,1) = 1.09157
15 15
24
Example
Richardson Table

1.08483 This is the best

1.08988 1.09156 estimate of the
derivative of the
1.09115 1.09157 1.09157 function.

All entries of the Richardson table are estimates of the

derivative of the function.
The first column are estimates using the central
difference formula with different step size h.

25