Chapter 23

~Numerical Differentiation and

Integration ~
Numerical Differentiation

“These notes are only to be used in class presentations”

1
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Numerical Differentiation and Integration
• Calculus is the mathematics of change. Since engineers
continuously deal with systems and processes that change, calculus
is an essential tool of engineering.
• Standing at the heart of calculus are the concepts of:
Differentiation Integration
y f ( xi  x)  f ( xi )
 b
x x I   f ( x)dx
dy f ( xi  x)  f ( xi ) a
 lim
dx x0 x

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Where do we use numerical techniques?
– A complicated continuous function that is difficult or
impossible to differentiate or integrate directly
– When we have a sampling of points representing a
function as with the experiments results

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Numerical Differentiation

• To evaluate numerical derivatives we’ll use Finite

Divided Difference Methods
– Forward
– Backward
– Central

4
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Forward Divided Difference method uses the value of points in
front of and at the point where the derivative is calculated.

5
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
First derivative with forward divided difference
Consider a function f(xi+1) which can be expanded in a Taylor
series in the neighborhood of a point xi
h  xi 1  xi : step size

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ( n 1) () ( n 1)
Rn  h  O(hn+1),
(n  1)!

If we truncate the series after the 1st derivative term

f ' ( xi )
f ( xi 1 )  f ( xi )  h  R1
1!

6
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Solve for the 1st derivative

' f ( xi 1 )  f ( xi )
f ( xi )   O ( h) 2 pt. formula
h
f ( xi 1)  f ( xi )  f ( xi )
1st forward divided difference

' f ( xi )
f ( xi )   O
 ( h)
 h
order of the 1st order accurate
1st derivative truncation error
estimate

7
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
First derivative with backward divided difference

Backward Divided Difference method uses the value of points

behind of and at the point where the derivative is calculated.

8
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Consider a function f(xi-1) which can be expanded in a Taylor
series in the neighborhood of a point xi and
h  xi  xi 1 : step size

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
Truncate the series after the 1st derivative term and solve for the 1st
derivative
' f ( xi )  f ( xi 1 )
f ( xi )   O ( h) 2 pt. formula
h

f ( xi )  f ( xi 1)  f ( xi ) 1st backward divided difference

' f ( xi )
f ( xi )   O
 ( h)
 h 1st order accurate
order of the
1st derivative truncation error 9
estimate
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 First derivative with central divided difference

xi 1  xi 1  2h

Central Divided Difference uses the value of points both in

front and behind of the point where the derivative is
calculated.
10
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Subtract backward difference Taylor series expansion from
forward difference Taylor series expansion

f ' ( xi ) f ( 3) ( xi ) 3 f ( n ) ( xi ) n
f ( xi 1 )  f ( xi 1 )  2 h2 h  2 h  Rn
1! 3! n!
n is odd.

Truncate the series after the 1st derivative term and solve for the
1st derivative

f ( xi 1 )  f ( xi 1 )  2 f ( xi )h  R1 ' f (3) () 3

R1  h
(3)!

11
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 2 pt. formula

f ( xi 1 )  f ( xi 1 )
f ( xi ) 
'
 O(h 2 ) 2nd order accurate
2h

f ( xi 1 )  f ( xi 1 )  f ( xi ) 1st central divided difference

f ( xi )
f ( xi ) 
'
 O(h 2 )

2 h
order of the
1st derivative truncation error
estimate

Central method is usually more accurate than forward &

backward methods 12
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Example 23.1 : Determine the 1st derivative of
f ( x)  0.1x 4  0.15x3  0.5x 2  0.25x  1.2

at x=0.5 using finite divided differences and a step size of h=0.25.

Forward divided difference

f (0.5)  1.1547
Backward divided difference
f (0.5)  0.7141
Central divided difference

f (0.5)  0.9344
13
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Higher order derivatives
Second derivative with forward divided difference

Consider functions f(xi+1) and f(xi+2) which can be expanded in

Taylor series in the neighborhood of a point xi

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) f ( n)
( xi )
f ( xi  2 )  f ( xi )  2h  ( 2h)   
2
(2h) n  Rn
1! 2! n!

xi 1  xi  h
xi  2  xi  2h 14
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Truncate the series after the 2nd derivative term

f ' ( xi ) f " ( xi ) 2
f ( xi 1 )  f ( xi )  h h  R2
1! 2!
f ' ( xi ) f " ( xi )
f ( xi  2 )  f ( xi )  2h  (2h) 2  R2
1! 2!
Multiply f(xi+1) by 2 and subtract that from f(xi+2)

f ( xi  2 )  2 f ( xi 1 )   f ( xi )  f ( xi )h 2  O(h3 )

15
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Solve for the 2nd derivative
f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f '' ( xi )   O
 ( h)
2
h
  order of the
2 nd derivative truncation error
estimate

f ( xi  2 )  2 f ( xi 1 )  f ( xi )  2 f ( xi ) 2nd forward divided difference

2
 f ( xi )
f ( xi )   O ( h) 1st order accurate
h2
For the nth derivative
 f ( xi ) n
 f ( xi )
n nth forward divided
f ( xi ) 
(n)
n
 O (h ) difference
h
16
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 General equation for nth order forward divided difference

j n
n
 f ( xi )   (1)   f ( xi  (n  j )h)
n

j 0  j

 fi  3 fi 1  3 fi 2  fi 3
Example:
f ' ' ' x   3
 O h 
h

17
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Second derivative with backward divided difference

Consider functions f(xi-1) and f(xi-2) which can be expanded in

Taylor series in the neighborhood of a point xi

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) 2 f ( n)
( xi )
f ( xi  2 )  f ( xi )  2h  ( 2h)    (2h n )  Rn
1! 2! n!

xi  xi 1  h
xi  xi  2  2h 18
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Truncate the series after the 2nd derivative term

f ' ( xi ) f " ( xi ) 2
f ( xi 1 )  f ( xi )  h h  R2
1! 2!
f ' ( xi ) f " ( xi )
f ( xi  2 )  f ( xi )  2h  (2h) 2  R2
1! 2!
Multiply f(xi-1) by 2 and subtract that from f(xi-2)

f ( xi  2 )  2 f ( xi 1 )   f ( xi )  f ( xi )h 2  O(h3 )

19
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Solve for the 2nd derivative
'' f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi )   O
 ( h)
2
h
  order of the
2 nd derivative truncation error
estimate

f ( xi  2 )  2 f ( xi 1 )  f ( xi )   2 f ( xi ) 2nd backward divided difference

 2 f ( xi )
f ( xi )   O ( h) 1st order accurate
h2
For the nth derivative
n
 f ( xi )
 O(h)  f ( xi )
n nth backward divided
( n)
f ( xi )  difference
n
h
20
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
General equation for nth order backward divided difference

j n
n
 f ( xi )   (1)   f ( xi  jh)
n

j 0  j

Example: f '''
x    fi 3  3 fi  23  3 fi 1  fi
 Oh 
h

21
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Second derivative with central divided difference

f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!
f ' ( xi ) f " ( xi ) 2 f ( n) ( xi ) n
f ( xi 1 )  f ( xi )  h h  h  Rn
1! 2! n!

Add these terms and truncate the series after the 2nd derivative term
f ( xi 1 )  f ( xi 1 )  2 f ( xi )  f " ( xi )h 2  O(h 4 )

" f ( xi 1 )  2 f ( xi )  f ( xi 1 )
f ( xi )   O(h 2 ) 2nd order accurate
h2
22
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 2 f ( xi )  f ( xi 1 )  2 f ( xi )  f ( xi 1 )
 2
f ( xi )
f " ( xi )  2
 O ( h 2
)
h

For the nth derivative

 n f ( xi )
f ( n) ( xi )   O(h 2 )
hn

23
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 General equation for nth order central divided difference

j n
n
n
 f ( xi )   (1)   f ( xi  (  j )h)
n

j 0  j 2
h h
When n is odd,  f ( xi  )   f ( xi  )
n n
2 2
2
• Write the equation for δn(x) and then substitute x-(h/2) and
x+(h/2) for x and take the average.

Example: 1 1
f i  2  f i 1  f i 1  f i  2
f ' ' ' x   2
h3
2  O h 2
 
24
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 23.2: Use the following data to find velocity and
acceleration. Use the appropriate divided difference equations
which give the most accurate result.

Time (s) 0 1 2 3 4 5 6 7 8
Position (m) 0 0.7 1.8 3.4 5.1 6.5 7.3 8.0 8.4

Answer:

velocity 0.7 0.9 1.35 1.65 1.55 1.1 0.75 0.55 0.4
(m/s)
acceleration 0.4 0.4 0.5 0.1 -0.3 -0.6 -0.1 -0.3 -0.3
(m/s2)
25
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 High Accuracy Differentiation Equations
• High-accuracy divided-difference formulas can be generated by
including additional terms from the Taylor series expansion.
1st derivative with forward divided difference

f ( xi ) 2
f ( xi 1 )  f ( xi )  f ( xi )h  h  R2
2
f ( xi 1 )  f ( xi ) f ( xi )
f ( xi )   h  O(h 2 )
h 2

f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi ) 
h2

26
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 f ( xi 1 )  f ( xi ) f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi )   2
h  O ( h 2
)
h 2h

 f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )
f ( xi )   O(h 2 )
2h

• Inclusion of the 2nd derivative term has improved the

accuracy to O(h2).

27
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 1st derivative with backward divided difference

f ( xi ) 2
f ( xi 1 )  f ( xi )  f ( xi )h  h  R2
2
f ( xi )  f ( xi 1 ) f ( xi )

f ( xi )   h  O(h 2 )
h 2
f ( xi  2 )  2 f ( xi 1 )  f ( xi )
f ( xi ) 
h2

f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )
f ( xi )   O(h 2 )
2h

28
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Coefficient Tables for Calculating Derivatives with an Error Order of O(h2)
Forward Divided Differences

fi fi+1 fi+2 fi+3 fi+4 fi+5 

-3/2 4/2 -1/2


hf'(x)

h2f''(x) 2 -5 4 -1 O h
2
 
h3f'''(x) -5/2 18/2 -24/2 14/2 -3/2


h4fıv(x) 3 -14 26 -24 11 -2 

Backward Divided Differences

fi-5 fi-4 fi-3 fi-2 fi-1 fi 

hf'(x) 1/2 -4/2 3/2


h2f''(x) -1 4 -5 2 O h
2
 
h3f'''(x) 3/2 -14/2 24/2 -18/2 5/2


h4fıv(x) -2 11 -24 26 -14 3 
29
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 1st derivative with central divided difference

 f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi  2 )
f ( xi )   O(h 4 )
12h
Drive the formula for the central divided difference for the 1st
derivative and show that it’s 4th order accurate.
Central Divided Differences

fi+3 fi+2 fi+1 fi fi-1 fi-2 fi-3

hf'(x) -1/12 8/12 -8/12 1 


h2f''(x) -1/12 16/12 -30/12 16/12 -1/12
O h 
4

h3f'''(x) -1/8 1 -13/8 13/8 -1 1/8 


h4fıv(x) -1/6 2 39/6 56/6 -39/6 2 1/6 

30
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Richardson Extrapolation

Richardson extrapolation, uses two derivative estimates to arrive at a better

approximation
D : true but unknown value of the derivative
D(h1) : approximation based on the step size h1
D(h2) : approximation based on the step size h2
E(h1) : truncation error based on the step size h1
E(h2) : truncation error based on the step size h2

D  D(h )  E (h )
D(h1 )  E (h1 )  D(h2 )  E (h2 )

31
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 2
E (h1 ) h 2
 h1 
 1
2
 E (h1 )  E (h2 ) 
E (h2 ) h 2  h2 
D(h2 )  D(h1 )
D(h1 )  E (h2 )h1 / h2  2
 D(h2 )  E (h2 )  E (h2 ) 
h1 / h2 2  1
D  D(h2 )  E (h2 )

D  D(h2 ) 
1
D(h2 )  D(h1 )
h1 / h2   1
2

If (h2  h1 / 2)  4 1
D  D(h2 )  D(h1 )
3 3
Improved estimate of the
derivative It can be shown that the
error of this estimate is O(h4).
32
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 23.3 : Determine the 1st derivative of
f ( x)  2 x sin x

at x=1.05 using Richardson extrapolation with h1=0.4 and

h2=0.2

Answer:

f (01 .05)  2.2741

'

33
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Derivatives of Unequally Spaced Data

• For finite divided difference approximation the data have to

be evenly spaced.
• For the Richardson extrapolation the data have to be evenly
spaced and generated for succesively halved intervals.
• But data from experiments or field studies are often
collected at unequal intervals.

• So one way to handle unequally spaced data fit a 2nd order

Lagrange interpolating polynomial to each set of 3 adjacent
points. (xi-1,xi,xi+1)

34
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 f ( x)  f ( xi 1 )
x  xi x  xi 1   f ( x ) x  xi 1 x  xi 1   f ( x ) x  xi 1 x  xi 
xi 1  xi xi 1  xi 1  i
xi  xi 1 xi  xi 1  i 1
xi 1  xi 1 xi 1  xi 

Take the derivative

2 x  xi  xi 1 2 x  xi 1  xi 1 2 x  xi 1  xi
f ( x)  f ( xi 1 )  f ( xi )  f ( xi 1 )
xi1  xi xi 1  xi 1  xi  xi 1 xi  xi 1  xi 1  xi 1 xi1  xi 

x: the value where you want to estimate the derivative

• These equations can be used to estimate the derivative anywhere

within the range prescribed by 3 points.
•Points don’t have to be equally spaced.
• Derivative estimate is of the same accuracy as the central divided
difference. 35
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 23.4 : Use the following data to find f ’(0.8)

x 0.6 0.8 0.9

f(x) 0.825 0.697 0.622

f (0.8)  0.715

36
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.