2025-01-21

ESOF 3558
Numerical Methods & Modeling

Part 5
Chapters 21: Numerical Differentiation

All images copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Chapter Objectives
• High-accuracy numerical differentiation formulas
• Numerical differentiation of equally spaced data
• Numerical differentiation of unequally spaced data
• Richardson extrapolation of numerical differentiation
• Sensitivity of numerical differentiation to data error
• MATLAB diff and gradient differentiation functions

1
 2025-01-21

Numerical Differentiation
• Numerical differentiation formulas from
Chapter 4:
• Forward difference:
𝑓 𝑥 𝑓 𝑥
𝑓 𝑥 𝑂 ℎ
ℎ
• Backward difference:
𝑓 𝑥 𝑓 𝑥
𝑓 𝑥 𝑂 ℎ
ℎ
• Centered difference:
𝑓 𝑥 𝑓 𝑥
𝑓 𝑥 𝑂 ℎ
2ℎ

High-Accuracy Forward Difference Formula

• Derive high-accuracy forward difference formulas by incorporating
additional terms from Taylor series
• Expand 𝑓 𝑥 and 𝑓 𝑥 in terms of 𝑓 𝑥
𝑓 𝑥 𝑓 𝑥 𝑓 𝑥
𝐀 : 𝑓 𝑥 𝑓 𝑥 𝑓 𝑥 ℎ ℎ 𝑂 ℎ 𝑓 𝑥
2!
𝑓 𝑥
𝐁 : 𝑓 𝑥 𝑓 𝑥 𝑓 𝑥 2ℎ 2ℎ 𝑂 ℎ ℎ ℎ
2!
𝑥 𝑥 𝑥
• Eliminate the 2nd derivative term
𝐁 𝟒 𝐀 : 𝑓 𝑥 4𝑓 𝑥 3𝑓 𝑥 2𝑓 𝑥 ℎ 𝑂 ℎ

• High accuracy Forward difference formula

𝑓 𝑥 4𝑓 𝑥 3𝑓 𝑥
𝑓 𝑥 𝑂 ℎ
2ℎ

2
 2025-01-21

Forward Finite-Difference

Backward Finite-Difference

3
 2025-01-21

Centered Finite-Difference

High-Accuracy Differentiation: Example

• Example 21.1: Estimate 𝑓′ 𝑥 at x = 0.5 using a step size h = 0.25:
𝑓 𝑥 0.1𝑥 0.15𝑥 0.5𝑥 0.25𝑥 1.2
• Solution:
𝑥 0 𝑓 𝑥 1.2
𝑥 0.25 𝑓 𝑥 1.1035156
𝑥 0.5 𝑓 𝑥 0.925
𝑥 0.75 𝑓 𝑥 0.6363281
𝑥 1 𝑓 𝑥 0.2

h h h=0.25 h
𝑥 𝑥 𝑥 0.5 𝑥 𝑥
8

4
 2025-01-21

High-Accuracy Differentiation: Example (cont)

• Forward difference of accuracy 𝑶 𝒉𝟐 :
𝑓 𝑥 4𝑓 𝑥 3𝑓 𝑥 0.2 4 0.6363281 3 0.925
𝑓 𝑥 0.5 0.859375
2ℎ 0.5
ε 5.82%

• Backward difference of accuracy 𝑶 𝒉𝟐 :

3𝑓 𝑥 4𝑓 𝑥 𝑓 𝑥 3 0.925 4 1.1035156 1.2
𝑓 𝑥 0.5 0.878125
2ℎ 0.5
ε 3.77%

• Centered difference of accuracy 𝑶 𝒉𝟒 :

𝑓 𝑥 8𝑓 𝑥 8𝑓 𝑥 𝑓 𝑥 0.2 8 0.6363281 8 1.1035156 1.2
𝑓 𝑥 0.5
12ℎ 12 0.25
0.9125
ε 0%
9

Richardson Extrapolation
• Richardson extrapolation combines two derivative estimates to produce a more
accurate estimate.
• Combining two 𝑂 ℎ estimates, and halving the interval (ℎ ℎ /2), produces
an 𝑂 ℎ estimate:
4 1
𝐷 𝐷 ℎ 𝐷 ℎ
3 3
• Combining two 𝑂 ℎ estimates, and halving the interval (ℎ ℎ /2), produces
an 𝑂 ℎ estimate:
16 1
𝐷 𝐷 ℎ 𝐷 ℎ
15 15
• Combining two 𝑂 ℎ estimates, and halving the interval (ℎ ℎ /2), produces
an 𝑂 ℎ estimate:
64 1
𝐷 𝐷 ℎ 𝐷 ℎ
63 63 10

5
 2025-01-21

Richardson Extrapolation: Example

• Example 21.1: compute an improved estimate of 𝑓′ 𝑥 at x = 0.5 by first using
step sizes h1 = 0.5 and h2 = 0.25.
𝑓 𝑥 0.1𝑥 0.15𝑥 0.5𝑥 0.25𝑥 1.2
• Use low-accuracy centered difference to estimate 1st derivatives:
𝑓 𝑥 𝑓 𝑥
𝑓 𝑥 𝑂 ℎ h h
2ℎ 𝑥 𝑥 0.5 𝑥
𝑓 𝑥 ℎ 𝑓 𝑥 ℎ 𝑓 0.5 ℎ 𝑓 0.5 ℎ
𝑓 𝑥
2ℎ 2ℎ
𝑓 1 𝑓 0 0.2 1.2
𝐷 ℎ 0.5 1.0 ε 9.6%
1 1.0
𝑓 0.75 𝑓 0.25 0.6363281 1.1035156
𝐷 ℎ 0.25 0.934375 ε 2.4%
0.5 0.5
11

Richardson Extrapolation: Example

• Then use Richardson extrapolation formula to compute an improved estimate:

𝐷 𝐷 ℎ 𝐷 ℎ

𝐷 0.934375 1.0 0.9125

• Note: the true value of 𝑓′ 0.5 = 0.9125

12

6
 2025-01-21

Derivative of Empirically Derived Data

• Techniques discussed focus on finding the 𝑣 𝑡
𝑣 𝑡 𝑣 𝑡

derivative of a given function.

h h
• They apply to determining the derivatives from 𝑡 𝑡 𝑡

evenly-spaced empirical data. 𝑣 𝑡

𝑣 𝑡 𝑣 𝑡
2ℎ

• Empirical information from experiments often

has unevenly collected data. 𝑣 𝑡
𝑣 𝑡
• These methods are not suitable for derivatives in 𝑣 𝑡
𝑣 𝑡
𝑣 𝑡

such cases.
𝑡 𝑡 𝑡 𝑡 𝑡

Example: empirically derived

information

13

Derivative of Unequally Spaced Data

• Calculating Derivatives for Unequally Spaced Data Fitted 2nd degree Lagrange
polynomial for data points
• One approach is to fit a polynomial to the data and
compute its derivative at a specific point. 𝑓 𝑥 𝑓 𝑥
𝑓 𝑥

• Example:
• Fit a second-degree Lagrange polynomial to three data x1 x2 x3
points and compute its derivative to obtain:
𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥
𝑓 𝑥 𝑓 𝑥 𝑓 𝑥 𝑓 𝑥
𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥

2𝑥 𝑥 𝑥 2𝑥 𝑥 𝑥 2𝑥 𝑥 𝑥
𝑓′ 𝑥 𝑓 𝑥 𝑓 𝑥 𝑓 𝑥
𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥 𝑥

14

7
 2025-01-21

Derivatives and Integrals for Data with Errors

• Numerical differentiation amplifies
data errors, while integration Error-free data Empirical data
smoothens them.
• Differentiating smooth error-free data
yields a smooth result; empirical data
with errors may not.
• One approach for derivative calculation
on data with errors is to fit a smooth
function and then find its derivative.

15

Partial Derivatives
• Partial derivative 𝑓 𝑥, 𝑦 along a single dimension
• Definition using calculus:
𝜕𝑓 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥, 𝑦
lim
𝜕𝑥 → Δ𝑥
• Numerically:
Numerical Technique Formula
Forward Difference 𝜕𝑓 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥, 𝑦
𝜕𝑥 Δ𝑥

Backward Difference 𝜕𝑓 𝑓 𝑥, 𝑦 𝑓 𝑥 Δ𝑥, 𝑦

𝜕𝑥 Δ𝑥

Centred Difference 𝜕𝑓 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥 Δ𝑥, 𝑦

𝜕𝑥 2Δ𝑥 16

8
 2025-01-21

Partial Derivatives
• Partial derivative 𝑓 𝑥, 𝑦 using centred differences:

𝜕𝑓 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥 Δ𝑥, 𝑦 𝜕𝑓 𝑓 𝑥, 𝑦 Δ𝑦 𝑓 𝑥, 𝑦 Δ𝑦
𝜕𝑥 2Δ𝑥 𝜕𝑦 2Δ𝑦

• For mixed partial derivative, take the partial derivative with respect to
each variable.
𝜕 𝑓 𝜕 𝜕𝑓 𝜕 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥 Δ𝑥, 𝑦
𝜕𝑦𝜕𝑥 𝜕𝑦 𝜕𝑥 𝜕𝑦 2Δ𝑥

𝜕 𝜕
𝜕 𝑓 𝑓 𝑥 Δ𝑥, 𝑦 𝑓 𝑥 Δ𝑥, 𝑦
𝜕𝑦 𝜕𝑦
𝜕𝑦𝜕𝑥 2Δ𝑥
17

Partial Derivatives (cont)

𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦

𝜕 𝑓 2Δ𝑦 2Δ𝑦
𝜕𝑦𝜕𝑥 2Δ𝑥

𝜕 𝑓 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦 𝑓 𝑥 Δ𝑥, 𝑦 Δ𝑦

𝜕𝑦𝜕𝑥 4Δ𝑥Δ𝑦

18

9
 2025-01-21

Numerical Differentiation with MATLAB

• MATLAB has two built-in functions to take derivatives:
• diff(x)
• Returns the difference between adjacent elements in vector x (that is x).
• diff(y)./diff(x)
• Returns a vector division of y’s by x’s.
x = [x1 x2 x3 x4 ]
y = [y1 y2 y3 y4 ]
diff(x)= x = [x1x2 x2x3 x3x4]
diff(y)= y = [y1y2 y2y3 y3y4]
diff(y)./diff(x)=[y1/x1 y2/x2 y3/x3]

19

MATLAB Function: gradient

• fx = gradient(f,h)
• Determines the derivative of the data in vector 𝑓 at each of the points.
• Uses forward difference for the first point, backward difference for the last
point, and centered difference for the interior points.
• ℎ is the spacing between points; if omitted ℎ 1.
f = [f1 f2 f3 f4 ]

• Gradient can also be used to find partial derivatives for matrices:

[fx, fy] = gradient(f,h)

20

10
 2025-01-21

Visualization
• MATLAB can generate contour plots of functions as well as vector
fields.
• Assuming x and y represent a mesh grid of x and y values and z
represents a function of x and y,
• contour(x, y, z) can be used to generate a contour plot
• [fx, fy]=gradient(z,h) can be used to generate partial derivatives
and
• quiver(x, y, fx, fy) can be used to generate vector fields

21

11