ME 310

Numerical Methods

7. Differentiation

These presentations cannot be used and/or modified without the permission of the authors.

Mechanical Engineering Department

Middle East Technical University

Ankara, Turkey

[email protected]
 Numerical Differentiation

Measured velocity of a car during

a crash test.

X 100
Velocity, Vx (inches/s)

Problem: What is the

maximum acceleration
of the car?

Time (s)
2
Data and video: http://rockyroer.blogspot.com.tr/2011/11/crash-test-data.html
Derivative: Rate of change of a dependent variable with respect to an independent variable.

f(x)

df f ( x i + Dx ) - f ( x i ) tangent
f = f (x ) ® = lim
dx Xi Dx ®0 Dx slope = f ¢(xi)

x
xi

which can be written approximately as a difference equation

f(x)

df Df f ( x i + Dx ) - f ( x i ) Df
» =
dx Xi Dx Dx
Dx
x
xi

• Numerical differentiation is considered if

• the function cannot be differentiated analytically
• the function is known at discrete points only
• the differentiation is to be automated in an algorithm.
 Finite Divided Difference Formulas using TSE
Formulas for the first derivative
• Forward differencing (use 1st order TSE of f(xi+1) around xi. Call this TSE1)
f ¢¢(x) 2
f ( x i + 1 ) = f ( x i ) + f ¢( x i ) h + h where h = xi+1 - xi
2!

f (x i + 1 ) - f (x i ) f ¢¢(x) f (x i + 1 ) - f (x i )
f ¢( x i ) = - h = - O(h)
h 2! h

• Backward differencing (use 1st order TSE of f(xi-1) around xi. Call this TSE2)

f ¢¢(x) 2
f ( x i - 1 ) = f ( x i ) - f ¢( x i ) h + h where h = xi – xi-1
2!

f (x i ) - f (x i - 1 ) f ¢¢(x) f (x i ) - f (x i - 1 )
f ¢( x i ) = + h = + O(h)
h 2! h

• Centered differencing (use TSE1 – TSE2. But consider 2nd order terms also.)

f ¢¢(x i ) 2
f (x i + 1 ) = f (x i ) + f ¢(x i ) h + h + O(h 3 ) f (x i + 1 ) - f (x i - 1 )
2! f ¢( x i ) = + O(h2 )
f ¢¢(x i ) 2 2h
f ( x i - 1 ) = f ( x i ) - f ¢( x i ) h + h - O(h 3 )
2!
• Forward and backward difference formulas are first order, O(h), accurate. That is the error drops
approximately by a factor of 2 as the step size h drops to h/2.
• Centered difference formula is second order, O(h2). Error drops by a factor of 4 as h drops to h/2.
• Centered difference formula uses the same number of arithmetic operations as forward and
backward formulas, and it offers better accuracy. Therefore it is more efficient.

Example 1: Position of a body moving in a straight path is shown below. Find its velocity.

• Use forward differencing at t = 0.0

t x v
x(0.1) - x(0.0) 0.15 - 0.00
0.0 0.00 1.50 v(0.0) = = = 1.50
h 0.1
0.1 0.15
0.2 0.47 2.35 • Use centered differencing at t = 0.1, 0.2, 0.3 and 0.4
0.3 0.62 x(0.3) - x(0.1) 0.62 - 0.15
v(0.2) = = = 2.35
0.4 0.84 2h 0.2
0.5 0.98 1.40 • Use backward differencing at t = 0.5

x(0.5) - x(0.4) 0.98 - 0.84

v(0.5) = = = 1.40
h 0.1

Exercise 1: Complete the above table.

Higher order formulas for the first derivative

• To derive them use proper combinations of TSE of f(xi+1), f(xi-1), f(xi+2), f(xi-2)

- f ( x i + 2 ) + 4 f ( x i + 1 ) - 3f ( x i )
• Forward differencing f ¢( x i ) = - O(h2 )
2h

3f ( x i ) - 4 f ( x i - 1 ) + 3f ( x i - 2 )
• Backward differencing f ¢( x i ) = + O(h2 )
2h

- f (x i + 2 ) + 8f (x i + 1 ) - 8f (x i - 1 ) + f (x i - 2 )
• Centered differencing f ¢( x i ) = + O(h 4 )
12 h

• See Section 23.1 in Chapra and Canale (2010) for even more higher order formulas.

Exercise 2: Derive the above formulas.

Exercise 3: Solve the previous example using the above formulas. Note that for t=0.0 and 0.1
forward differencing must be used. For t=0.4 and 0.5 backward differencing is suitable. Centered
differencing can only be used for t=0.2 and 0.3.
Formulas for the second derivative

f (x i + 2 ) - 2f (x i + 1 ) + f (x i )
• Forward differencing f ¢¢(x i ) = 2
+ O(h)
h

f (x i ) - 2f (x i - 1 ) + f (x i - 2 )
• Backward differencing f ¢¢(x i ) = + O(h)
h2

f (x i + 1 ) - 2f (x i ) + f (x i - 1 )
• Centered differencing f ¢¢(x i ) = + O(h2 )
2
h

• See Section 23.1 for formulas for the 3rd and 4th derivatives.

Exercise 4: Derive the above formulas. Use proper combinations of TSE of f(xi+1), f(xi-1), f(xi+2)
and f(xi-2).

Exercise 5: Use the table given for the first example to calculate the acceleration of the particle.
You can either use the second derivative formulas, or use the first derivative formulas with the
previously calculated velocities. Compare and comment on the results.
How can we improve the derivative estimates?
• Use smaller h values.
• Use higher order approximations.

Exercise 6: The following two tables are for the function f(x) = ex. They use step sizes of 0.2
and 0.1. Calculate the first derivative of the function using O(h2) estimates. Calculate true errors.
Compare the results.

x f(x) f¢(x) et x f(x) f¢(x) et

0.4 1.4918247 0.4 1.4918247
0.6 1.8221188 0.5 1.6487213
0.8 2.2255409 0.6 1.8221188
1.0 2.7182818 0.7 2.0137527
0.8 2.2255409
0.9 2.4596031
1.0 2.7182818

• Another alternative to improve derivative estimates is to use Richardson Extrapolation. It

combines two estimates obtained with different h values to get a better estimate (similar to what
we have learned in integration).
Richardson’s Extrapolation
D: Exact derivative (usually not known)
D1: Estimated derivative using h=h1. E1: Error of the estimation D1. ® D = D1 + E1
D2: Estimated derivative using h=h2. E2: Error of the estimation D2. ® D = D2 + E2

D1 + E1 = D2 + E2

If we used second-order differentiation to get D1 and D2, than E1 = O(h12) , E2 = O(h22)

2
E1 h12 æh ö
» 2 ® E1 » E2 çç 1 ÷÷
E2 h2 è h2 ø
2
æh ö
Substitute this into the first equation D1 + E2 çç 1 ÷÷ » D 2 + E2
è h2 ø
D 2 - D1
Solve for E2 E2 »
( h1 h2 )
2
- 1

D 2 - D1
Combine this with D2 D » D2 +
( h1 h2 )
2
- 1

This estimate is or order O(h4), obtained by two O(h2) estimates.

D 2 - D1 4 1
A special case of h2 = h1/2 results in D » D2 + = D 2 - D1
3 3 3
Example 2:

Use Richardson Extrapolation to estimate the first derivative of sin(x) at x=p/4 using step sizes of
h1=p/3 and h2=p/6. Use centered differences of O(h2).

sin(p/4 + p/3) - sin(p/4 - p/3)

Using h1=p/3 D1 = f ¢(p/4) » = 0.584772601
2 ( p / 3)
|et| = 17.3 %

sin(p/4 + p/6) - sin(p/4 - p/6)

Using h2=p/6 D2 = f ¢(p/4) » = 0.675237237
2 ( p / 6) |et| = 4.5 %

4 1
Apply Richardson Extrapolation f ¢(p/4) » 0.675237237 - 0.584772601 = 0.70539215
3 3
|et| = 0.24 %

Romberg Algorithm
Apply multiple Richardson Extrapolation one after the other until the error falls below a specified
tolerance.
Exercise 6: In the above example use h3=p/12 to calculate D3. Combine D1 with D2, and D2 with
D3 to get two O(h4) estimates. Then combine these two estimates to get an estimate of order
O(h6).
Partial Derivatives
The formulation is similar to ordinary derivatives.

!" " # + Δ#, ( − " # − Δ#, (

=
!# 2Δ#

¶f f ( x, y + Dy ) - f ( x, y - Dy )
=
¶y 2 Dy

Mixed partial derivative:

¶f ¶f
( x + Dx, y ) - ( x - Dx, y )
¶2 f ¶ æ ¶f ö ¶y ¶y
= ç ÷=
¶x¶y ¶x è ¶y ø 2 Dx

Substitute the partial derivative formula for !"/!y

¶2 f f ( x + Dx, y + Dy ) - f ( x + Dx , y - Dy ) - f ( x - Dx , y + Dy ) + f ( x - Dx , y - Dy )
=
¶x¶y 4 Dx Dy
Differentiating and Integrating Data with Errors

Figure taken from Chapra and Canale 2010