Numerical Differentiation

Numerical and Computing Methods

Numerical differentiation

Suppose that we are given the

following data and we want to
x y dy/dx
find the derivatives 0 0 ?
1 0.7 ?
2 1.8 ?
So how we are going to solve it? 3 3.4 ?
4 5.1 ?
…
Numerical Differentiation

⚫ Forward finite divided difference

⚫ Backward finite divided difference
⚫ Center finite divided difference
⚫ All based on the Taylor Series

f ' ' ( xi ) 2
f ( xi +1 ) = f ( xi ) + f ' ( xi )h + h + .........
2!
Taylor Series

⚫ Taylor series is of great importance in

numerical analysis, especially in evaluating the
truncation error in numerical methods
⚫ Generally, Taylor series provides a means to
predict a function value at one point in terms of
the function value and its derivatives at
another point
⚫ The theorem states that any smooth function
can be approximated by a polynomial
Taylor series
⚫ For mathematical function of single variable, the
Taylor series can be expressed by the following
form:
f ' ' (xi ) 2 f (3) (xi ) 3 f ( n) (xi ) n
f (xi +1 ) = f (xi ) + f ' (xi )h + h + h + ... + h + Rn
2! 3! n!

where h is a step size h = xi +1 − xi include to account

for terms from n+1
to infinity

f ( n +1) ( ) n +1
and Rn is the remainder term Rn = h = O(h n +1 )
(n + 1)!
Forward Finite Difference

f ' ' (xi ) 2

f (xi +1 ) = f (xi ) + f ' (xi )h + h + .........
2!

f (xi +1 ) − f (xi ) f ' ' (xi )

which can be solved for
f ' (xi ) =
h
−
2
( )
h + O h2

By excluding the second and higher order derivatives terms:

f (xi +1 ) − f (xi )
f ' (xi ) = + O ( h) Forward difference
h approximation
Forward Divided Difference

f ( xi + 1 ) − f ( xi ) f i
f ' ( xi ) = + O( xi + 1 − xi ) = + O( h)
xi + 1 − xi h

f(x)

What is derivative at this

(xi, yi) point?

x
Forward Divided Difference

f ( xi + 1 ) − f ( xi ) f i
f ' ( xi ) = + O( xi + 1 − xi ) = + O( h)
xi + 1 − xi h

f(x)
(x i+1,y i+1)

(xi, yi) Determine a second point

base on Dx (h)

x
Forward Divided Difference

f ( xi + 1 ) − f ( xi ) f i
f ' ( xi ) = + O( xi + 1 − xi ) = + O( h)
xi + 1 − xi h

f(x)
How does (x i+1,y i+1)
this compare
to the actual (xi, yi)
first derivative
at xi?
x
Forward Divided Difference

f ( xi + 1 ) − f ( xi ) f i
f ' ( xi ) = + O( xi + 1 − xi ) = + O( h)
xi + 1 − xi h

f(x)
(x i+1,y i+1)

(xi, yi)

x
Forward Divided Difference
f i
f ' ( xi ) = + O( h) Error is proportional to
h the step size

first forward divided difference

O(h2) error is proportional to the square of the step size

O(h3) error is proportional to the cube of the step size

Backward Difference Approximation of the
First Derivative

Expand the Taylor series backwards

f ' ' (xi ) 2

f (xi −1 ) = f (xi ) − f ' (xi )h + h − .....
2!
f (xi ) − f (xi −1 ) f i
f ' (xi )  =
h h

The error is still O(h)

Backward Divided Difference
f(x)

(xi,yi)

(xi-1,yi-1)

x
Centered Difference Approximation of the
First Derivative

Subtract backward Taylor series from the forward Taylor series

f (3) (xi ) 3
f ( xi +1 ) = f (xi −1 ) + 2 f ' (xi )h + h +
3!
f (xi +1 ) − f (xi −1 )
f ' ( xi ) =
2h
( )
− O h2
 f ( xi +1 ) − f ( xi −1 )
f(x) f ' ( xi ) = − O(h 2 )
2h

(xi+1,yi+1)
(xi,yi)

(xi-1,yi-1)

x
f(x) f(x)

forward
true derivative
finite divided
difference approx.

x x
f(x) f(x)

backward centered
finite divided finite divided
difference approx. difference approx.

x x
Deriving more accurate version of
the divided difference formula
Let’s return back to the derivation of the forward difference formula

f (xi +1 ) − f (xi ) f ' ' (xi )

f ' (xi ) =
h
−
2
h + O h2( )

By substituting the f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
following approximation for f " ( x) = + O ( h)
the second derivative: h2

f (xi +1 ) − f (xi ) f (xi + 2 ) − 2 f ( xi ) + f ( xi )

The corresponding
equation becomes f ' (xi ) =
h
−
2h 2
h + O h 2
( )

− f ( xi + 2 ) + 4 f ( xi +1 ) − 3 f ( xi )
Or by collecting terms f ' (xi ) = + O( h 2 )
2h
Forward finite-divided-difference
formulas
Backward finite-divided-difference
formulas
Centered finite-divided-difference
formulas
Example 1
Given the following function, use finite divided difference
(with error of O(h2)) to estimate the derivative at x = 2.2
using step size h = 0.6 , h=0.2 Forward , backward
and centre, then calculate true percentage error.
H=0.2 F(x) H=0.6 F(x)
f(x) = 50cos 3x 2
Xi-2 + ln 3x 1.8 -46.15 1.0 -48.4010
Xi-1 2 43.984 1.6 10.2239
xi 2.2 -16.7904 2.2 -16.7904

Xi+1 2.4 2.036 2.8 0.03610

Xi+2 2.6 9.049 3.4 -47.3030
FWD 123.66721 81.50054
BWD -681.14925 -116.389775
CTR -104.871 -8.493565
Example 2
x y dy/dx
0 0
Find dy/dx at all data points by
using finite divided difference 1 0.7
of at least O(h2) 2 1.8
3 3.4
4 5.1
5 6.5
6 7.3
7 8.0
8 8.4
…
Derivatives of Unequally Spaced
Data
⚫ Common in data from experiments or field studies
⚫ Fit a second order Lagrange interpolating polynomial
to each set of three adjacent points, since this
polynomial does not require that the points be equi-
spaced
⚫ Differentiate analytically

2 x − xi − xi +1 2 x − xi −1 − xi +1
f ' ( x) = f ( xi −1 ) + f ( xi )
( xi −1 − xi )( xi −1 − xi +1 ) ( xi − xi −1 )( xi − xi +1 )
2 x − xi − xi −1
+ f ( xi +1 )
( xi +1 − xi )( xi +1 − xi −1 )
Derivatives of Unequally Spaced
Data
⚫ Refer to example problem 23.3 from the
text book
…..end of lecture