Numerical Differentiation 

Numerical Differentiation
a d teg at o /
and Integration/
Numerical Differentiation

By
Dr Ali Jawarneh
Dr Ali Jawarneh 1
 Ch.23:
Ch 23: Numerical Differentiation and
Integration/ Numerical Differentiation

23.1- High-Accuracy Differentiation

Formulas
23 2 Richardson
23.2- Ri h d Extrapolation
E t l ti
23.3- Derivatives of Unequally
Spaced Data

Dr Ali Jawarneh 2
NUMERICAL DIFFERENTIATION
High-Accuracy Differentiation Formulas
Taylor series
It provides a mean to predict a function value at one point in terms of the
function value and it’s derivative at another point. Taylor series expansions
are used to derive finite-divided-difference approximations of derivatives.
True derivative
f (x)
h=xi+1-xi=xi-xi-1
forward
backward centered

h h

x
x i −1 xi x i +1
(old) (present) (new)
Dr Ali Jawarneh 3
Taylor ∞
f ( x) = ∑ am ( x − x 0 ) n 1 (n )
where a n = f ( x 0 )
Series n=0
n= n!
(x − x0 ) ( x − x 0 )2 ( x − x 0 )3
f ( x) = f ( x 0 ) + f ' (xo ) + f " (xo ) + f " ' ( x o ) + ...
1! 2! 3!
n: is order of the series
Forward Taylor
f "(xi )
f ( x i +1 ) = f ( x i ) + f ' ( x i )( x i +1 − x i ) + ( x i +1 − x i )2 +
2!
(n) h = x i +1 − x i
f "' (xi ) f (xi )
( x i +1 − x i ) 3 + ... + ( x i + 1 − x i )n
3! n!
Backward Taylor
f "(xi )
f ( x i −1 ) = f ( x i ) − f ' ( x i )( x i − x i −1 ) + ( x i − x i −1 ) 2 −
2!
(n )
h = x i − x i −1
f "' (xi ) f (xi )
( x i − x i −1 ) 3 + ... − ... + ( x i − x i −1 )n
3! n!
Dr Ali Jawarneh 4
 Forward Taylor
f " (xi ) 2 f "' (xi ) 3 f (n ) (xi ) n
f ( x i +1 ) = f ( x i ) + f ' ( x i )h + h + h + ... + h (1)
2! 3! n!
Zero order ((n=0): ) f ( x i+ i + 1 ) = f ( x i ) + O(h ) O(h): Truncation
error of order h
1st order (n=1): f ( x i +1 ) = f ( x i ) + f ' ( x i )h + O(h 2 )
f " (xi ) 2
2nd order (n=2): f ( x i +1 ) = f ( x i ) + f ' ( x i )h + h + O( h 3 )
2!
Backward Taylor
f " (xi ) 2 f "' (xi ) 3 f (n ) (xi ) n
f ( x i −1 ) = f ( x i ) − f ' ( x i )h + h − h + ... − ... + h (2)
2! 3! n!
Zero order: f ( x i −1 ) = f ( x i ) + O(h )
1st order: f ( x i −1 ) = f ( x i ) − f ' ( x i )h + O(h 2 ) Dr Ali Jawarneh

2nd order: f " (xi ) 2

f ( x i −1 ) = f ( x i ) − f ' ( x i )h + h + O( h 3 )
2! 5
 Forward Approximation
f "(xi ) 2 (3)
from (1) f ( x i +1 ) = f ( x i ) + f ' ( x i )h + h + ...
2!
f ( x i +1 ) − f ( x i ) f " ( x i )
f ' (xi ) = + h + ...
h 2!
f ( x i +1 ) − f ( x i )
f ' (xi ) = + O( h ) Forward , 1st derivative of order h
h
f "(xi ) ((4))
Similar to (1) f ( x i + 2 ) = f ( x i ) + f ' ( x i )( 2h ) + ( 2h ) 2 + ...
2!
Multiplication Eq.(3) with 2, then subtract the result from (4), you will get:

f ( x i + 2 ) − 2f ( x i +1 ) + f ( x i ) Forward
F d , 2nd
2 d derivative
d i ti
f "(xi ) = 2
+ O(h ) of order h
h

Dr Ali Jawarneh 6
from (1) f " (xi ) 2 f "' (xi ) 3
f ( x i +1 ) = f ( x i ) + f ' ( x i )h + h + h + ...
2! 3!
Use also Forward , 2nd derivative of order h

f ( x i +1 ) − f ( x i ) f " ( x i ) h 2
f ' (xi ) = − + O( h 2 )
h 2! h
f ( x i +1 ) − f ( x i ) [f ( x i + 2 ) − 2f ( x i +1 ) + f ( x i )] h 2
= − 2
+ O ( h 2
)
h h 2! h
Collecting terms:

− f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i ) Forward , 1st derivative

f ' (xi ) = + O( h 2 ) of order h2
2h

Dr Ali Jawarneh 7
 Backward Approximation
From (2) f ( x i −1 ) = f ( x i ) − f ' ( x i )h +
f " (xi ) 2 f "' (xi ) 3
h − h + ... (5)
2! 3!
f ( x i ) − f ( x i −1 )
f ' (xi ) = + O(h ) Backward , 1stt derivative of order h
h
f " (xi )
Similar to (2)
f ( x i − 2 ) = f ( x i ) − f ' ( x i )( 2h ) + ( 2h ) 2 − ... (6)
2!
Multiplication Eq.(5) with 2, then subtract the result from (6), you will get:
f ( x i ) − 2f ( x i −1 ) + f ( x i − 2 ) b k
backwardd , 2nd
2 d derivative
d i ti off
f " (xi ) = = O ( h )
h2 order h

Dr Ali Jawarneh 8
from (2) f " (xi ) 2 f "' (xi ) 3
f ( x i −1 ) = f ( x i ) − f ' ( x i )h + h − h +
2! 3!
Use also Backward , 2nd derivative of order h
f ( x i ) − f ( x i −1 ) f ( x i ) − 2f ( x i −1 ) + f ( x i − 2 ) h 2 1
f ' (xi ) = +[ 2
]
h h 2! h
Collecting terms:

3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 )
f ' (xi ) = 2 + O( h 2 )
h

Dr Ali Jawarneh 9
 Centered Approximation

Subtract (1) from (2)

f ( x i + 1 ) − f ( x i −1 )
f ' (xi ) = + O( h 2 )
2h

Dr Ali Jawarneh 10
Summary of Finite-Divided-Difference Formulas
(1) Forward Finite-Divided-Difference
Finite Divided Difference Formulas
f ( x i +1 ) − f ( x i )
1st derivative: f ' (xi ) = + O( h )
h
− f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i )
f ' (xi ) = + O( h 2 ) More accurate
2h
2nd derivative:

f ( x i + 2 ) − 2f ( x i +1 ) + f ( x i )
f "(xi ) = 2
+ O( h )
h

− f ( x i + 3 ) + 4f ( x i + 2 ) − 5f ( x i +1 ) + 2f ( x i )
f "(xi ) = 2
+ O ( h 2
)
h
More accurate
Dr Ali Jawarneh 11
(2) Backward Finite-Divided-Difference
Finite Divided Difference Formulas
f ( x i ) − f ( x i −1 )
1st derivative: f ' (xi ) = + O( h )
h
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 )
f ' (xi ) = + O( h 2 ) More accurate
2h
2nd derivative:

f ( x i ) − 2f ( x i −1 ) + f ( x i − 2 )
f "(xi ) = 2
+ O( h )
h

2f ( x i ) − 5f ( x i −1 ) + 4f ( x i − 2 ) − f ( x i − 3 )
f "(xi ) = 2
+ O ( h 2
)
h
More accurate
Dr Ali Jawarneh 12
 (3) Centered Finite-Divided-Difference
Finite Divided Difference Formulas
f ( x i + 1 ) − f ( x i −1 )
1st derivative: f ' (xi ) = + O( h 2 )
2h
− f ( x i + 2 ) + 8f ( x i +1 ) − 8f ( x i −1 ) + f ( x i − 2 ) More accurate
f ' (xi ) = + O( h )
4

12h
2nd derivative:

f ( x i +1 ) − 2f ( x i ) + f ( x i −1 )
f "(xi ) = 2
+ O ( h 2
)
h

− f ( x i + 2 ) + 16f ( x i +1 ) − 30f ( x i ) + 16f ( x i −1 ) − f ( x i − 2 )

f "(xi ) = 2
+ O ( h 4
)
12h
More accurate
Dr Ali Jawarneh 13
 Two ways to improve derivative
estimates when employing
finite divided differences

(1) Decrease the step size

(2) Use a higher-order formula
that employs more points

Dr Ali Jawarneh 14
 Example: Find the first derivative of y= cos(x) at x=π/4
using a step size of h= π/12.
/12 Use forward and backward
approximations of O(h) & O(h2) , and central difference
approximation of O(h2) and O(h4). Also, estimate the true
l ti error εt for
percentt relative f each h approximation.
i ti
Solution f ( x)
Forward f ' ( x i ) = f ( x i +1 ) − f ( x i ) + O(h )
h
h
− f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i )
f ' (xi ) = + O( h 2 )
2h x
Backward f ( x i ) − f ( x i −1 ) x i − 2 x i −1 xi x i +1 x i + 2
f ' (xi ) = + O( h )
h

f ' (xi ) =
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 )
+ O( h 2 ) π π π π 5π
2h
12 6 4 3 12
Centered f ( x i + 1 ) − f ( x i −1 )
f ' (xi ) = + O( h 2 )
2h
− f ( x i + 2 ) + 8f ( x i +1 ) − 8f ( x i −1 ) + f ( x i − 2 ) Dr Ali Jawarneh
f ' (xi ) = + O( h 4 ) 15
12h
 π π
x f(x)  (radian) f ' ( ) |true = − sin( ) = −0.70710678
4 4
Xi‐2 π/12 0 965925826
0.965925826 (radian
f ( x i +1 ) − f ( x i )
Forward f ' ( x i ) = + O( h )
Xi‐1 π/6 0.866025404 h

Xi π/4 0.707106781 π cos(( π / 3) − cos(( π / 4)

f '( ) = = −0.79108963
Xi+1 π/3 0.5 4 π / 12
− f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i )
Xi+2 5π//12 0.258819045 f ' (xi ) = + O( h 2 )
2h

− cos(5π / 12) + 4 cos( π / 3) − 3 cos( π / 4)

f ' ( π / 4) = = −0.72601275
2π / 12
f ( x i ) − f ( x i −1 ) cos( π / 4) − cos( π / 6)
Backward f ' ( x i ) = + O( h ) f ' ( π / 4) = = −0.60702442
h π / 12
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 )
f ' (xi ) = + O( h 2 )
2h
3 cos( π / 4) − 4 cos( π / 6) + cos( π / 12)
f ' (xi ) = = −0.71974088
2π / 12
2π

Dr Ali Jawarneh 16
 Centered
f ( x i + 1 ) − f ( x i −1 ) cos( π / 3) − cos( π / 6)
f ' (xi ) = + O( h 2 ) f ' ( π / 4) = = −0.69905703
2h 2π / 12
− f ( x i + 2 ) + 8f ( x i +1 ) − 8f ( x i −1 ) + f ( x i − 2 )
f ' (xi ) = + O( h 4 )
12h

− cos(5π / 12) + 8 cos( π / 3) − 8 cos( π / 6) + cos( π / 12)

f ' ( π / 4) = = −0.70699696
12π / 12

First Order Second Order
Forward ‐0.79108963 ‐0.72601275
εt=11.877% εt=2.67%
Backward ‐0.60702442 ‐0.71974088
εt=14.154% εt=1.787%
Centered ‐0.69905703 ‐0.70699696
εt=1.138% εt=0.016%
Dr Ali Jawarneh 17
 Example: The following data was collected for the
distance traveled versus time of a rocket
rocket. Use
numerical differentiation of order O(h2) to estimate the
rocket’s velocity (V) and acceleration (a) at each time
t, s 0 25 50 75 100 125
y, km 0 32 58 78 92 100
Solution − f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i )
Forward f ' (xi ) = + O( h 2 ) Forward Backward
2h
Centered
− f ( x i + 3 ) + 4f ( x i + 2 ) − 5f ( x i +1 ) + 2f ( x i )
f " (xi ) = 2
+ O( h 2 )
h
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 )
B k
Backward
d f ' (xi ) = + O( h 2 )
2h
2f ( x i ) − 5f ( x i −1 ) + 4f ( x i − 2 ) − f ( x i − 3 )
f " (xi ) = 2
+ O( h 2 )
h
Centered f ' ( x ) = f ( x i +1 ) − f ( x i −1 ) + O(h 2 ) f " ( x ) = f ( x i +1 ) − 2f ( x i ) + f ( x i −1 ) + O(h 2 )
i i 2
2h h
18
Dr Ali Jawarneh
 Do not forget that our question is of order O(h2)
t=0 The only option is Forward, O(h) and O(h2)
− f ( x i + 2 ) + 4f ( x i +1 ) − 3f ( x i ) − 58 + 4( 32) − 3(0)
Forward f ' (xi ) = + O( h 2 ) f ' ( 0 ) = = 1.4 km / s
2h 2( 25)

− f ( x i + 3 ) + 4f ( x i + 2 ) − 5f ( x i +1 ) + 2f ( x i ) − 78 + 4(58) − 5( 32) + 2(0)

f " (xi ) = 2
+ O ( h 2
) f " ( 0 ) = 2
= −0.0096 km / s 2
h 25

t=25
2 You have
Y h many options:
(1) Forward, O(h) and O(h2)
ti

(2) Backward of order O(h) only for 1st derivative

(3) Centered of order O(h2) for 1st and 2ndd derivatives
Centered Order of O(h4) is not allowable
f ( x i + 1 ) − f ( x i −1 ) 58 − 0
f ' (xi ) = + O( h 2 ) f ' ( 25) = = 1.16 km / s
2h 2( 25)
f ( x i + 1 ) − 2f ( x i ) + f ( x i −1 ) 58 − 2( 32) + 0
f " (xi ) = 2
+ O( h 2 ) f " ( 25) = = − 0 . 0096 km / s 2
h 25 2
At interior points centered is preferable
Dr Ali Jawarneh 19
 t=50 You have many options:
(1) Forward, O(h) and O(h2)
(2) Backward except order O(h2) for 2nd derivative
(3) Centered, O(h2) and O(h4)
C t
Centered
d
78 − 32
f ( x i + 1 ) − f ( x i −1 ) f ' (50) = = 0.92 km / s
f ' (xi ) = + O( h 2 ) 2( 25)
2h
f ( x i + 1 ) − 2f ( x i ) + f ( x i −1 ) 78 − 2(58) + 32
f " (xi ) = 2
+ O( h 2 ) f " ( 25) = = − 0.0096 km / s 2
h 25 2

t=75 You have many options:

(1) Forward, except order O(h2) for 2nd derivative
(2) Backward, O(h) and O(h2)
Centered ((3)) Centered, O(h ( 2) and O(h ( 4)
f ( x i + 1 ) − f ( x i −1 ) 92 − 58
f ' (xi ) = + O( h 2 ) f ' (78) = = 0.68 km / s
2h 2( 25)
f ( x i + 1 ) − 2f ( x i ) + f ( x i −1 ) 92 − 2(78) + 58
f " (xi ) = + O( h 2 )
h 2 f " ( 25) = = − 0. 0096 km / s 2

25 2
Dr Ali Jawarneh 20
 t=100 You have many options:
(1) Forward, Only 1 derivative of O(h) stt

(2) Backward, O(h) and O(h2)

(3) Centered, only 1st and 2nd derivatives of O(h2)
C t
Centered
d
100 − 78
f ( x i + 1 ) − f ( x i −1 ) f ' (100) = = 0.44 km / s
f ' (xi ) = + O( h 2 ) 2( 25)
2h
f ( x i + 1 ) − 2f ( x i ) + f ( x i −1 ) 100 − 2(92) + 78
f " (xi ) = 2
+ O( h 2 ) f " (100) = = − 0 .0096 km / s 2
h 25 2

t=125 You have one option:

(1) Backward, O(h) and O(h2)
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 ) 3(100) − 4(92) + (78)
Backward f ' (xi ) = + O(h 2 ) f ' (125) = = 0.2 km / s
2h 2( 25)

2f ( x i ) − 5f ( x i −1 ) + 4f ( x i − 2 ) − f ( x i − 3 )
f " (xi ) = 2
+ O( h 2 )
h
2(100) − 5(92) + 4(78) − (58)
f " (125) = 2
= −0.0096 km / s 2
25
Dr Ali Jawarneh 21
 t y V a
0 0 1.4 ‐0.0096
0.0096
25 32 1.16 ‐0.0096
50 58 0 92
0.92 ‐0.0096
0 0096
75 78 0.68 ‐0.0096
100 92 0.44 ‐0.0096
125 100 0.20 ‐0.0096
0.0096

Dr Ali Jawarneh 22
Example: for the following available data:
(1) What is the least accurate value of f’(1
f’(1.399)
399)
(2)What is the value of f”(1.399)
(3)What is the best available estimate value of f’(1.400)

x 1.390 1.398 1.399 1.400 1.404

f( )
f(x) 0 686 0.705
0.686 0 705 0
0.706
706 0.711
0 711 0
0.721
721
9x10-3 5x10-3
Solution
8x10-3 1x10-3 1x10-3 4x10-3

x 1.390 1.398 1.399 1.400 1.404

f(x) 0.686 0.705 0.706 0.711 0.721
Dr Ali Jawarneh 23
 (1) What is the least accurate value of f’(1.399)
Backward
f ( x i ) − f ( x i −1 ) 0.706 − 0.686
f ' (xi ) = + O( h ) f ' (1.399) = −3
= 2.222
h 9x10
(2) What is the value of f”(1.399)
Centered
Do you know why?
f ( x i +1 ) − 2f ( x i ) + f ( x i −1 ) 0.711 − 2(0.706) + (0.705)
f "(xi ) = + O ( h 2
) f " (1 . 399 ) = = 4000
h2 (1x10 − 3 ) 2

(3) What is the best available estimate value of f’(1.400)

Backward
3f ( x i ) − 4f ( x i −1 ) + f ( x i − 2 ) Do you know why?
f ' (xi ) = + O( h 2 )
2h

3(0.711) − 4(0.706) + 0.705

f ' (1.4) = −3
=7
2(1x10 )
Dr Ali Jawarneh 24
 Richardson Extrapolation
Use two derivative estimates to compute
a third, more accurate approximation
It is similar to Richardson extrapolation for integral

4 1 h1
D ≅ D(h 2 ) − D(h1 ) h2 =
3 3 2
For centered difference approximation with
O(h2), the application of this formula will yield
a new derivative estimate O(h ( 4)

Dr Ali Jawarneh 25
 Three ways to improve derivative
estimates when employing finite
f
divided differences

(1) Decrease the step size

(2) Use a higher-order formula
that employs more points
(3) Richardson Extrapolation

Dr Ali Jawarneh 26
 Example: Use Richardson extrapolation to estimate 
th fi t d i ti
the first derivative of y= cos(x) at x=π/4 using step 
f ( ) t /4 i t
sizes of π/3 and π/6. Employ centered differences of 
O(h2) for the initial estimates.
) for the initial estimates
f (x ) − f (x )
Solution: f ' (x ) = i
2h
i +1
+ O( h )
i −1 2

cos(7 π / 12) − cos( − π / 12) h1 h1 = π / 3

D( π / 3) = = −0.58477 h1
2( π / 3)
x i −1 x i x i + 1
− π / 12 π / 4 7π
7 / 12
cos(5π / 12) − cos( π / 12)
D( π / 6) = = −0.67524
2( π / 6)

D≅
4 1
D(h 2 ) − D(h1 ) h2 h2 h2 = π / 6
3 3
x i −1 xi x i +1
4 1
D≅ ( −0.67524) − ( −0.58477) = −0.70530 π / 12 π / 4 5π / 12
3 3

True derivative=-0.70710678 Dr Ali Jawarneh

27
 Derivatives of Unequally Spaced Data
Finite-divided-
Fi it di id d differences
diff : The
Th data
d t had
h d to
t be
b evenlyl spaced
d
Richardson-Extrapolation: The data had to be evenly spaced
and generally for successively halves intervals.

In case of unequal intervals:

Use 2nd order Lagrange interpolating (3 points)
x − x1 x − x 2 x − x0 x − x2 x − x 0 x − x1
f 2 ( x) = f (xo ) + f ( x1 ) + f (x 2 )
x 0 − x1 x 0 − x 2 x1 − x 0 x1 − x 2 x 2 − x 0 x 2 − x1

Differentiate:
2x − x i − x i + 1 2x − x i −1 − x i + 1 2x − x i −1 − x i
f ' ( x ) = f ( x i −1 ) + f (xi ) + f ( x i +1 )
( x i −1 − x i )( x i −1 − x i +1 ) ( x i − x i −1 )( x i − x i +1 ) ( x i +1 − x i −1 )( x i +1 − x i )

Where x is the value at which you to estimate the derivative

Dr Ali Jawarneh 28
Example: The heat flux at the oil-air interface can be
computed with Fourier law: dT
q |z = 0 = −kρc |z = 0
dz
Where: q
q= heat flux (W/m
( 2)

k= coefficient of thermal diffusivity in soil (=3.5x10-7 m2/s)

ρ= soil density (=1800 kg/m3)
c= soil specific heat (=840 J/kg.oC)

Use numerical differentiation to evaluate the temperature

gradient (dT/dz) at z=0 and estimate the heat flux

T(oC) 13.5 12 10
z 0 1.25 3.75

Dr Ali Jawarneh 29
 Solution: x i −1 = 0
x i = 1.25
x i +1 = 3.75
x i −1 xi x i +1
2x − x i − x i + 1 2x − x i −1 − x i + 1 2x − x i −1 − x i
f ' ( x ) = f ( x i −1 ) + f (xi ) + f ( x i +1 )
( x i −1 − x i )( x i −1 − x i +1 ) ( x i − x i −1 )( x i − x i +1 ) ( x i +1 − x i −1 )( x i +1 − x i )

dT 2(0) − 1.25 − 3.75 2(0) − 0 − 3.75

|z = 0 = f ' (0) = 13.5 + 12 +
dz (0 − 1.25)(0 − 3.75) (1.25 − 0)(1.25 − 3.75)
2(0) − 0 − 1.25
10 = −133.3333 oC / m
( 3.75 − 0)( 3.75 − 1.25)

dT
q |z = 0 = − kρc |z = 0
dz
q |z = 0 = −( 3.5x10 −7 )(1800)(840)( −133.3333) = 70.56 W / m 2
Dr Ali Jawarneh 30