Numerical Analysis 0905201

Numerical Differentiation

1
Numerical Analysis 0905201
Chapter 18

ND – Part(1)
2
Introduction

Numerical differentiation can be very useful in the following cases:

1) To obtain the derivative using algebraic operations.

2) To understand how the computers find such derivatives.

3) To obtain some higher derivative without going through the lower derivatives.

4) To find the derivative(s) for some data points without having to do interpolation.
1) Forward Difference Approximation of the 1st Derivative

The Taylor series is:

1) Forward Difference Approximation of the 1st Derivative

By arranging the equation:

Which is:
Example x y

1 2
Given that
5 3

9 5

14 10

Find the value of 𝑓′(5) using the forward difference approximation of the 1st derivative.
Solution

𝒇(𝒙𝒊+𝟏 ) − 𝒇(𝒙𝒊 ) 𝒇(𝒙𝒊+𝟏 ) − 𝒇(𝒙𝒊 )

𝒇′ 𝒙𝒊 = =
𝒉 𝒙𝒊+𝟏 − 𝒙𝒊

𝒇(𝟖) − 𝒇(𝟓) 𝟒 − 𝟏
𝒇′ 𝟓 = = =𝟏
𝟖−𝟓 𝟖−𝟓
2) Backward Difference Approximation of the 1st Derivative

→ ℎ = 𝑥𝑖 − 𝑥𝑖−1
 Example

Given that

x y
14 -1
20 3
26 12

Find the value of 𝑓′(26) using the backward difference approximation of the 1st derivative.
Solution

𝑓(𝑥𝑖 ) − 𝑓(𝑥𝑖−1 ) 𝑓(𝑥𝑖 ) − 𝑓(𝑥𝑖−1 )

𝑓′ 𝑥𝑖 = =
ℎ 𝑥𝑖 − 𝑥𝑖−1

′
𝑓(26) − 𝑓(20) 12 − 3
𝑓 26 = = = 1.5
26 − 20 6
3) Centered Difference Approximation of the 1st Derivative

→ ℎ = 𝑥𝑖+1 − 𝑥𝑖 𝑜𝑟 ℎ = 𝑥𝑖 − 𝑥𝑖−1
Example

Given that

x y
14 -1
20 3
26 12

Find the value of 𝑓′(20) using the centered difference approximation of the 1st derivative.
Solution

𝑓(𝑥𝑖+1 ) − 𝑓(𝑥𝑖−1 ) 𝑓(𝑥𝑖+1 ) − 𝑓(𝑥𝑖−1 )

𝑓′ 𝑥𝑖 = =
2ℎ 2(𝑥𝑖 − 𝑥𝑖−1 )
or

𝑓(𝑥𝑖+1 ) − 𝑓(𝑥𝑖−1 )
=
2(𝑥𝑖+1 − 𝑥𝑖 )

𝑓(26) − 𝑓(14) 12 − −1 13
∴ 𝑓′ 20 = = =
2(26 − 20) 12 12
 Approximation of the 1st Derivative

′ 𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖 )
Forward approximation : 𝑓 𝑥𝑖 = + 𝑂(ℎ) ℎ = 𝑥𝑖+1 − 𝑥𝑖
ℎ

𝑓(𝑥𝑖 )−𝑓(𝑥𝑖−1 )
Backward approximation : 𝑓′ 𝑥𝑖 = +𝑂 ℎ ℎ = 𝑥𝑖 − 𝑥𝑖−1
ℎ

𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖−1 )
Centered approximation : 𝑓′ 𝑥𝑖 = +𝑂(ℎ2 ) ℎ = 𝑥𝑖 − 𝑥𝑖−1
2ℎ

ℎ = 𝑥𝑖+1 − 𝑥𝑖
Relative True Error 𝜺𝒕

Relative True Error:

𝑻𝒓𝒖𝒆 𝒗𝒂𝒍𝒖𝒆 − 𝑨𝒑𝒑𝒓𝒐𝒙𝒊𝒎𝒂𝒕𝒆 𝒗𝒂𝒍𝒖𝒆

𝜺𝒕 = . 𝟏𝟎𝟎 %
𝑻𝒓𝒖𝒆 𝒗𝒂𝒍𝒖𝒆
 Example
Use the forward, backward and centered difference approximations of the 1st
derivative to estimate the value of 𝑓′(2) for the following function:
𝑓 𝑥 = 𝑥3 + 3 𝑥 − 1

Given that the true value of 𝑓′(2) = 15

Calculate when:

a) h = 1

b) h = 0.5
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

a) When h = 1

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1 𝑓 1 =3

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 3 𝑓 3 = 35

𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖 ) 𝑓(3)−𝑓(2) 35−13

Forward approximation : 𝑓 ′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = = 22
ℎ ℎ 1

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −22

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 46.6 %
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

a) When h = 1

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1 𝑓 1 =3

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 3 𝑓 3 = 35

𝑓(𝑥𝑖 )−𝑓(𝑥𝑖−1 ) 𝑓(2)−𝑓(1) 13−3

Backward approximation : 𝑓 ′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = = 10
ℎ ℎ 1

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −10

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 33.3 %
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

a) When h = 1

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1 𝑓 1 =3

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 3 𝑓 3 = 35

𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖−1 ) 𝑓(3)−𝑓(1)

Centered approximation : 𝑓′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = 16
2ℎ 2(1)

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −16

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 6.66 %
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

b) When h = 0.5

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1.5 𝑓 1.5 = 6.875

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 2.5 𝑓 2.5 = 22.125

𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖 ) 𝑓(2.5)−𝑓(2)

Forward approximation : 𝑓 ′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = 18.25
ℎ ℎ

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −18.25

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 21.66 %
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

b) When h = 0.5

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1.5 𝑓 1.5 = 6.875

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 2.5 𝑓 2.5 = 22.125

𝑓(𝑥𝑖 )−𝑓(𝑥𝑖−1 ) 𝑓(2)−𝑓(1.5)

Backward approximation : 𝑓 ′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = 12.25
ℎ ℎ

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −12.25

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 18.33 %
 Solution 𝒇 𝒙 = 𝒙𝟑 + 𝟑 𝒙 − 𝟏 Find 𝒇′(𝟐)

b) When h = 0.5

𝑥𝑖−1 = 𝑥𝑖 − ℎ = 1.5 𝑓 1.5 = 6.875

𝑥𝑖 = 2 𝑓 2 = 13

𝑥𝑖+1 = 𝑥𝑖 + ℎ = 2.5 𝑓 2.5 = 22.125

𝑓(𝑥𝑖+1 )−𝑓(𝑥𝑖−1 ) 𝑓(2.5)−𝑓(1.5)

Centered approximation : 𝑓 ′ 𝑥𝑖 = ⇒ 𝑓′ 2 = = 15.25
2ℎ 2ℎ

𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 −𝐴𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒 𝑣𝑎𝑙𝑢𝑒 15 −15.25

𝜀𝑡 = . 100 % ⇒ 𝜀𝑡 = . 100 %
𝑇𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 15

𝜀𝑡 = 1.66 %
Solution

h=1 h = 0.5
value 𝜺𝒕 value 𝜺𝒕

Forward Approximation 22 46.6 % 18.25 21.66 %

Backward Approximation 10 33.3 % 12.25 18.33 %

Centered Approximation 16 6.66 % 15.25 1.66 %

𝑻𝒓𝒖𝒆 𝒗𝒂𝒍𝒖𝒆 𝒐𝒇 𝒇′ 𝟐 = 𝟏𝟓
Richardson Extrapolation

This method improves the estimation of the derivative obtained from

the forward, backward and centered approximations.

This method utilizes two estimations of different step sizes (𝒉𝟏 , 𝒉𝟐 )

where 𝒉𝟐 < 𝒉𝟏 , in order to produce a new, more accurate
estimation.

𝟒 𝟏
𝑫= 𝑫 𝒉𝟐 − 𝑫(𝒉𝟏 )
𝟑 𝟑
Example

Using the results from example 4 from the previous lecture, use the
approximations obtained from the centered method with Richardson
extrapolation to find the 𝑓′(2), given that the true value of 𝒇′ 𝟐 = 𝟏𝟓
𝒇 𝒙 = 𝒙𝟑 + 𝟑𝒙 − 𝟏
Solution

ℎ1 = 1 , ℎ2 = 0.5

4 1
𝐷 = 𝐷 ℎ2 − 𝐷 ℎ1
3 3
4 1
𝐷 = (15.25) − (16)
3 3
𝐷 = 15
𝑻𝒓𝒖𝒆 𝒗𝒂𝒍𝒖𝒆 −𝑨𝒑𝒑𝒓𝒐𝒙𝒊𝒎𝒂𝒕𝒆 𝒗𝒂𝒍𝒖𝒆 𝟏𝟓 −𝟏𝟓
𝜺𝒕 = . 𝟏𝟎𝟎 % ⇒ 𝜺𝒕 = . 𝟏𝟎𝟎 %
𝑻𝒓𝒖𝒆 𝒗𝒂𝒍𝒖𝒆 𝟏𝟓

𝜺𝒕 = 0 %