NUMERICAL METHODS WITH

APPLICATIONS
(MEC500)
Assoc. Prof. Dr. Wan Nurshazwani WZ
School of Mechanical Engineering
Office: T1-A11-14A
e-mail: [email protected]
 Learning outcome for Chapter 5
Other than Newton Cotes formula, two techniques that are designed to
integrate functions / discrete data:
• Romberg integration (based on Richardson extrapolation)
• Gauss quadrature

Differentiation > Finding rate of change of one quantity with respect to

another. Methods
• Finite divided difference
• Higher accuracy finite divided difference
• Richardson Extrapolation
 Integration of equation
Functions to be integrated numerically are in two forms:
• Table of values (discrete data);
• limited number of points are available.
Function;
• use the function to generate table of values
• can generate as many data values required to attain acceptable
accuracy!
• composite Simpson 1/3 rule can be used to integrate a function BUT
more efficient method is available
Two techniques that are designed to integrate functions:
• Romberg integration (based on Richardson extrapolation)
• Gauss quadrature
 Romberg Integration
Is based on successive application of the trapezoidal rule to attain
efficient numerical integrals of functions.
• Use Richardson extrapolation to trapezoidal rule.
• Gives better approximation of the integral >> lower the true error
faster.

Richardson’s Extrapolation
• Uses two estimates of an integral to compute a third and more accurate
approximation
 Romberg Integration

The estimate and error associated with a multiple-application trapezoidal

rule can be represented generally as:
𝑏−𝑎
𝐼 = 𝐼 ℎ + 𝐸(ℎ) (1) where ℎ=
𝑛
Comparing 2 application of trapezoidal rule at different step size
𝐼 ℎ! + 𝐸(ℎ! )=𝐼 ℎ" + 𝐸(ℎ" ) (2)

𝐸 ℎ! ℎ!" 𝑏−𝑎 $ 𝑏−𝑎 " Assumed constant

≅ " where E# = ̅
𝑓" = ℎ 𝑓"̅
𝐸 ℎ" ℎ" 12𝑛 " 12 regardless of step size

ℎ!"
𝐸 ℎ! ≅ 𝐸 ℎ" (3) I = exact value of integral
ℎ"" I(h) = the approximation from an n segment application
of trapezoidal rule with step size h
E(h) = the truncation error
 Romberg Integration

Insert Eq (3) in Eq (2) Insert Eq (4) in Eq (1)

%! " 𝐼 ℎ" − 𝐼 ℎ!
𝐼 ℎ! + 𝐸 ℎ" %"
≅ 𝐼 ℎ" + 𝐸(ℎ" ) 𝐼 = 𝐼 ℎ" + "
ℎ!
−1
" ℎ"
ℎ!
𝐸 ℎ" − 𝐸 ℎ" ≅ 𝐼 ℎ" − 𝐼 ℎ!
ℎ" Improved estimate of the integral

"
ℎ!
𝐸 ℎ" − 1 ≅ 𝐼 ℎ" − 𝐼 ℎ!
ℎ"

𝐼 ℎ" − 𝐼 ℎ!
𝐸 ℎ" ≅ "
(4)
ℎ!
−1
ℎ"
 Romberg Integration
• For the cases where there are two 𝑂(ℎ!) estimates 𝐼(ℎ") and 𝐼(ℎ!) using step sizes of ℎ" and
ℎ!, respectively, an improved 𝑂(ℎ#) estimate may be formed for the special case where the
$
interval is halved (ℎ! = !!), this becomes:
4 1
𝐼= 𝐼 ℎ" − 𝐼 ℎ!
3 3
$
• For the cases where there are two 𝑂(ℎ#) estimates and the interval is halved (ℎ% = !"), an
improved 𝑂(ℎ&) estimate may be formed using:
16 1
𝐼= 𝐼& − 𝐼
15 15 '
$
• For the cases where there are two 𝑂(ℎ&) estimates and the interval is halved (ℎ% = !"), an
improved 𝑂(ℎ') estimate may be formed using:

64 1
𝐼= 𝐼& − 𝐼
63 63 '
 Romberg Integration
Example 1

Example: Single and multiple applications of trapezoidal rule yielded;

Segment h I (𝑂(ℎ" ) ) 𝜀# (%) I (𝑂(ℎ( ) ) 𝜀# (%) I (𝑂(ℎ) ) ) 𝜀# (%)

1 0.8 0.1728 89.5

1.3675 16.6
2 0.4 1.0688 34.9 1.6405 0.0
1.6235 1.0
4 0.2 1.4848 9.5

The estimates for one and two segments can be combined to give;
4 1
𝐼= 1.0688 − 0.1728 = 1.367467 𝜀* = 16.6%
3 3
The estimates for two and four segments can be combined to give;
4 1
𝐼 = 1.4848 − 1.0688 = 1.623467 𝜀* = 1.0%
3 3
The estimates can then be used for further iteration to give;
16 1
𝐼= 1.623467 − 1.367467 = 1.640533 𝜀* = 0.0%
15 15
 Example 2
Integrate 𝒇 𝒙 = 𝟎. 𝟐 + 𝟐𝟓𝒙 − 𝟐𝟎𝟎𝒙𝟐 + 𝟔𝟕𝟓𝒙𝟑 − 𝟗𝟎𝟎𝒙𝟒 +
𝟒𝟎𝟎𝒙𝟓 from 𝒂 = 𝟎 to 𝒃 = 𝟎. 𝟖. Calculate the true error, 𝜀% (True value
1.640533)

f(0) = 0.2
f(0.4) = 2.456
f(0.8) = 0.232
 Romberg Integration
Algorithm
• Note that the weighting factors for the Richardson extrapolation add up to 1
and that as accuracy increases, the approximation using the smaller step size is
given greater weight.
• In general,
4-.! 𝐼+/!,-.! − 𝐼+,-.!
𝐼+,- =
4-.! − 1

• where 𝑖"#$,&'$ and 𝑖",&'$ are the more and less accurate integrals, respectively,
and 𝑖",& the new approximation. 𝑘 is the level of integration and j is used to
determine which approximation is more accurate.
• The process by which lower-level integrations are combined to produce more
accurate estimates
• In general, Romberg integration is more efficient than the trapezoidal and
Simpson’s rule. Simpson’s 1/3 requires 256-segment to yield I = 1.640533
 Gauss Quadrature
• Gauss quadrature describes a class of techniques
for evaluating the area under a straight line by
joining any two points on a curve rather than simply
choosing the endpoints.
• The key is to choose the line that balances the
positive and negative errors.
• The integral estimates are of the form:
𝐼 ≅ 𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥! + ⋯ + 𝑐1.! 𝑓(𝑥1.! )

• where the 𝑐& and 𝑥& are calculated to ensure that

the method exactly integrates up to 2𝑛 − 1 %'
order polynomials over the interval from -1 to 1.
 Gauss Quadrature
Gauss Legendre Formula

Derivation of the Two-Point Gauss-Legendre Formula

𝐼 ≅ 𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥!
• The objective of Gauss quadrature is to determine the equations of the
form
• However, in contrast to trapezoidal rule that uses fixed end points a
and b, the function arguments 𝑥( and 𝑥) are not fixed end points but
unknowns.
• Thus, four unknowns to be evaluated require four conditions.
• First two conditions are obtained by assuming that the above eqn. for I
fits the integral of a constant and a linear function exactly.
• The other two conditions are obtained by extending this reasoning to a
parabolic and a cubic functions.
 Gauss Quadrature
Gauss Legendre Formula

!
𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥! = B 1 𝑑𝑥 = 2
.!
!
𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥! = B 𝑥 𝑑𝑥 = 0
.!
!
2
𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥! = B 𝑥 " 𝑑𝑥 =
.! 3
!
𝑐0 = 𝑐! = 1
𝑐0 𝑓 𝑥0 + 𝑐! 𝑓 𝑥! = B 𝑥 $ 𝑑𝑥 = 0
.! 1 1
𝑥0 = − 𝑥! =
3 3
1 1
𝐼=𝑓 − +𝑓
3 3
 Gauss Quadrature
Gauss Legendre Formula

• The integration limit is between -1 through 1. This was done to simplify

the mathematics.
• A simple change of variable can be used to translate other limits of
integration into this form. This is accomplished by assuming that a new
variable 𝒙𝒅 is related to the original variable 𝒙 in a linear fashion.
b+a
𝑥 = 𝑎0 + 𝑎! 𝑥 𝑎0 =
2
𝑥=𝑎 → 𝑥2 = −1 b−a
𝑎! =
𝑥=𝑏 → 𝑥2 = 1 2
b + a + b − a 𝑥2
𝑎 = 𝑎0 + 𝑎! (−1) 𝑥=
2
𝑏 = 𝑎0 + 𝑎! (1) 𝑏−𝑎
𝑑𝑥 = 𝑑𝑥2
2
Gauss Quadrature
Gauss Legendre Formula
 Gauss Quadrature
Example
Integrate 𝒇 𝒙 = 𝟎. 𝟐 + 𝟐𝟓𝒙 − 𝟐𝟎𝟎𝒙𝟐 + 𝟔𝟕𝟓𝒙𝟑 − 𝟗𝟎𝟎𝒙𝟒 +
𝟒𝟎𝟎𝒙𝟓 from 𝒂 = 𝟎 to 𝒃 = 𝟎. 𝟖. Calculate the true error, 𝜀% (True value
1.640533)
• Before integrating the function, we must perform a change of variable
so that the limits are from −1 to +1. To do this, we substitute a = 0 and
b = 0.8 into equation below:

b + a + b − a 𝑥2
𝑥= = 0.4 + 0.4x3
2

• Both of these can be substituted into the original equation to yield

𝑏−𝑎
𝑑𝑥 = 𝑑𝑥2 = 0.4𝑑𝑥2
2
• The transformed function can be evaluated at −1/ 3 and 1/ 3
0.5
0.2 + 25(0.4 + 0.4𝑥2 ) − 200(0.4 + 0.4𝑥2 )" + 675(0.4 + 0.4𝑥2 )$
B 0.4𝑑𝑥2
0 −900(0.4 + 0.4𝑥2 )( + 400(0.4 + 0.4𝑥2 )6
 Question 1
For the following integral:
$
B 𝑥𝑒 7 𝑑𝑥
0

1.Use order of ℎ+ Romberg integration to evaluate the integral and

calculate the true error, 𝜀% .
2.Use the two-point Gauss quadrature approach to estimate the
integral and calculate the true error, 𝜀% .
 Numerical Differentiation
Differentiation:
• Process of finding derivative (dy/dx) - rate of change of a dependent
variable with respect to an independent variable.
• The mathematical definition of a derivative begins with a difference
approximation:
∆𝑦 𝑓 𝑥: + ∆𝑥 − 𝑓 𝑥:
=
∆𝑥 ∆𝑥
• and as Δx is allowed to approach zero, the difference becomes a
derivative:

𝑑𝑦 𝑓 𝑥: + ∆𝑥 − 𝑓 𝑥:
= lim
𝑑𝑥 ∆7→0 ∆𝑥
 Numerical Differentiation

Refresher … Taylor Series

< ## (7$ ) < % 7$
𝑓 𝑥:/! = 𝑓 𝑥: + 𝑓 ; (𝑥: )ℎ + ℎ" + ⋯+ ℎ1 + 𝑅1
"! 1!

𝑓 1/! 𝜉 1/! 𝑥:/! − 𝑥: = ℎ

𝑅1 = ℎ
𝑛+1 ! 𝑥:.! − 𝑥: = −ℎ
In general, the 𝑛%' order Taylor series expansion will be exact for an 𝑛%'
order polynomial.
In other cases, the remainder term 𝑅, is of the order of ℎ,-) , meaning:
• The more terms are used, the smaller the error, and
• The smaller the spacing, the smaller the error for a given number of
terms.
 Numerical Differentiation

Truncate TSE after the first order gives,

𝑓(𝑥:/! ) = 𝑓(𝑥: ) + 𝑓 ; (𝑥: )ℎ + Θ(ℎ)

Rearrange

𝑓(𝑥:/! ) − 𝑓(𝑥: )
Forward difference 𝑓 ; (𝑥: ) = + Θ(ℎ)
ℎ

Backward difference 𝑓(𝑥: ) − 𝑓(𝑥:.! )

𝑓 ; (𝑥: ) = + Θ(ℎ)
ℎ

𝑓 𝑥:/! − 𝑓 𝑥:.!
Centered difference 𝑓 ; (𝑥: ) = + Θ(ℎ" )
2ℎ
 Numerical Differentiation
• Notice that the error for the central different method is of the order of h2
as compared to h for the forward and backward difference methods.
• Therefore, central difference method gives better approximation of the
derivative than the other two methods.
 Higher Accuracy Numerical Differentiation
• More accurate differentiation formulas (i.e. finite divided difference) can
be developed by including more terms from the TSE.
• This is achieved using linear algebra to combine the expansion around
several points.
• Three categories for the formula include forward finite-difference,
backward finite-difference, and centered finite-difference.
 Higher Accuracy Numerical Differentiation

𝑓 ;; 𝑥: "
𝑓 𝑥:/! = 𝑓 𝑥: + 𝑓;
𝑥: ℎ + ℎ + Θ(ℎ" )
2!
;;
𝑓 𝑥:/! − 𝑓 𝑥: 𝑓 𝑥:
𝑓 ; 𝑥: = + ℎ + Θ(ℎ" ) (1)
ℎ 2!
𝑓 𝑥:/" − 2𝑓 𝑥:/! + 𝑓 𝑥:
𝑓 ;; 𝑥: = + Θ(ℎ) (2)
ℎ"

Substitute (2) into (1)

𝑓 𝑥:/! − 𝑓 𝑥: 𝑓 𝑥:/" − 2𝑓 𝑥:/! + 𝑓 𝑥:
𝑓; 𝑥: = + "
ℎ + Θ(ℎ" )
ℎ 2ℎ
−𝑓 𝑥:/" + 4𝑓 𝑥:/! − 3𝑓 𝑥:
𝑓; 𝑥: = + Θ(ℎ" ) Need extra points
2ℎ

Inclusion of the 2nd derivative term has improved the accuracy

Higher Accuracy Numerical Differentiation
Higher Accuracy Numerical Differentiation
Higher Accuracy Numerical Differentiation
 Question 2
Compute forward and backward difference approximations of 𝑂(ℎ)
and 𝑂(ℎ. ) and central difference approximations of 𝑂(ℎ. ) and
𝑂(ℎ/ ) for the first derivative of
𝑦 = 𝑥 $ + 4𝑥 − 15
At 𝑥 = 3 with ℎ = 0.5 and ℎ = 0.25. Then, calculate the percent
relative true error, 𝜀% .
 Higher Accuracy Numerical Differentiation

Three ways to improve derivative estimate using finite divided

differences;
1.Decrease step size (more computation >> round off?)
2.Use higher order formula (need more points)
3.Use Richardson Extrapolation

Richardson Extrapolation
• Combine two lower-accuracy estimates of the derivative to produce
a higher-accuracy estimate.
 Richardson Extrapolation
'@
• For two 𝑂(ℎ. ) estimates and ℎ. = > an improved 𝑂(ℎ/ ) estimate
.
may be formed using:
4 1
𝐷= 𝐷 ℎ" − 𝐷 ℎ!
3 3
'@
• For two 𝑂(ℎ/ ) estimates and ℎ. = > an improved 𝑂(ℎ+ ) estimate
.
may be formed using:
16 1
𝐷= 𝐷 ℎ" − 𝐷 ℎ!
15 15
'@
• For two 𝑂(ℎ+ ) estimates and ℎ. = > an improved 𝑂(ℎ0 ) estimate
.
may be formed using:
64 1
𝐷= 𝐷 ℎ" − 𝐷 ℎ!
63 63
 Question 3
Determine the estimate of first derivative at 𝑥 = 3 using Richardson
Extrapolation based on the values calculated in Question 2. Then,
calculate the percent relative true error, 𝜀% .

𝑦 = 𝑥 $ + 4𝑥 − 15