CIEN301 - Numerical Integration and Differentiation
CIEN301 - Numerical Integration and Differentiation
1
• Differentiation and integration are core parts of calculus.
• Differentiation – Definition:
A difference approximation of differentiation is:
∆𝑦 𝑓 𝑥𝑖 + ∆𝑥 − 𝑓 𝑥𝑖
=
∆𝑥 ∆𝑥
• If ∆𝑥 is allowed to approach zero, the difference becomes a derivative:
𝑑𝑦 𝑓 𝑥𝑖 + ∆𝑥 − 𝑓 𝑥𝑖
= lim
𝑑𝑥 ∆𝑥→0 ∆𝑥
• The second derivative – commonly known as the curvature - represents the
derivative of the first derivative, which indicates how fast the slope is changing.
𝑑2 𝑦 𝑑 𝑑𝑦
2
=
𝑑𝑥 𝑑𝑥 𝑑𝑥
2
The graphical definition of a derivative: as ∆𝑥 approaches
zero, the difference becomes a derivative
3
The Partial Derivative
• The partial derivative is equivalent to taking the derivative of the
function at a point with all but one variable held constant. For a
function 𝑓(𝑥, 𝑦) dependent on both 𝑥 and 𝑦, the partial derivative of
f with respect to 𝑥 at an arbitrary point (𝑥, 𝑦) is defined as:
𝜕𝑓 𝑓 𝑥 + ∆𝑥, 𝑦 − 𝑓 𝑥, 𝑦
= lim
𝜕𝑥 ∆𝑥→0 ∆𝑥
• Similarly, the partial derivative of 𝑓 with respect to 𝑦 is defined as:
𝜕𝑓 𝑓 𝑥, 𝑦 + ∆𝑦 − 𝑓 𝑥, 𝑦
= lim
𝜕𝑦 ∆𝑦→0 ∆𝑦
• The partial derivative corresponds to slopes of a point on a surface in
the 𝑥 or 𝑦 directions.
4
Integration
• Integration is the inverse process to
differentiation.
• Mathematically, integration is
represented by:
𝑏
𝐼 = න 𝑓 𝑥 𝑑𝑥
𝑎
where, 𝑓(𝑥) is referred to as the
integrand and 𝑎 & 𝑏 are the limits of
integration.
• The integral 𝐼 represents the area under
the curve of 𝑓(𝑥) between 𝑥 = 𝑎 and 𝑏.
5
Differentiation vs. Integration
• Given that 𝑦 𝑡 and 𝑣 𝑡
represent position and
velocity of a given object at
time 𝑡, respectively, then,
• 𝑣 𝑡 is the derivative of 𝑦 𝑡
with respect to 𝑡.
𝑑
𝑣 𝑡 = 𝑦 𝑡
𝑑𝑡
• Conversly, the position of the
object 𝑦(𝑡) can be
determined as integration of
𝑣(𝑡).
𝑡
𝑦 𝑡 = න 𝑣 𝑡 𝑑t
0
6
Mathematical Background
7
Mathematical Background
8
Integration
• Two main approaches of numerical integration will be discussed:
• Integration methods based on replacing a complicated function or
tabulated data with a simple polynomial – Newton-Cotes formulas.
These includes: trapezoidal rule, Simpson’s 1/3 rule and Simpson’s
3/8 rule.
• Integration methods designed to integrate known equations and
functions:
• Romberg Integration
• Adaptive Integration
• Gauss Quadrature
9
Newton-Cotes Integration Formulas
• The Newton-Cotes formulas are the most common numerical integration
schemes.
• They are based on the strategy of replacing a complicated function or
tabulated data with an approximating function that is easy to integrate.
• The approach involves approximating the integration of a function 𝑓(𝑥)
with a polynomial 𝑓𝑛 𝑥 , i.e.
𝑏 𝑏
𝐼 = න 𝑓 𝑥 𝑑𝑥 ≅ න 𝑓𝑛 𝑥 𝑑𝑥
𝑎 𝑎
Where the polynomial of order 𝑛, 𝑓𝑛 𝑥 , is expressed as:
𝑓𝑛 𝑥 = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥 2 + ⋯ + 𝑎𝑛−1 𝑥 𝑛−1 + 𝑎𝑛 𝑥 𝑛
10
Approximations of a function integration with a that of a
polynomial
• (a) First-order
(trapezoidal)
• (b) Single parabola
(Simpons’s 1/3 rule)
11
Approximating an integral with a series of polynomials
• Integrals can be
approximated using a series
of polynomials applied
piecewise to the function or
data over equal-length
segments.
12
Closed vs. Open forms of the Newton-Cotes formulas
13
The Trapezoidal Rule
• The trapezoidal rule is the first of the
Newton-Cotes integration formulas.
• The data or the function is approximated
with a first-order polynomial, i.e.
𝑏 𝑏
𝐼 = න 𝑓 𝑥 𝑑𝑥 ≅ න 𝑓1 𝑥 𝑑𝑥
𝑎 𝑎
• This integration can be approximated by
the following formula – known as the
trapezoidal rule:
𝑓 𝑎 +𝑓 𝑏
𝐼 = 𝑏−𝑎
2
14
The Truncation Error of the Trapezoidal Rule
• The truncation error of the trapezoidal rule is:
1 ′′
𝐸𝑡 = − 𝑓 𝜉 𝑏 − 𝑎 3
12
where 𝜉 is an intermediate value between a and b.
• The truncation error of the trapezoidal rule is of the order of ℎ3 ,
where
ℎ =𝑏−𝑎
15
Example of a Single Application of the Trapezoidal Rule
• Use the trapezoidal rule to numerically integrate
the following function from 𝑎 = 0 to 𝑏 = 0.8
(the exact value of the integral can be calculated
analytically and is found to be 1.640533)
𝑓 𝑥 = 0.2 + 25𝑥 − 200𝑥 2 + 675𝑥 3 − 900𝑥 4 + 400𝑥 5
𝑓 𝑎 +𝑓 𝑏
The trapezoidal rule: 𝐼 = 𝑏 − 𝑎
2
𝑓 𝑎 = 0 = 0.2; 𝑓 𝑏 = 0.8 = 0.232
Substitute into the trapezoidal rule:
0.2 + 0.232
𝐼 ≅ 0.8 − 0 = 0.1728
2
The actual error can be calculated as:
𝐸𝑡 = 1.640533 − 0.1728 = 1.467733
𝜖 𝑡 = 89.5%
16
The Multiple-Application Trapezoidal Rule
• The accuracy of the trapezoidal rule can be improved by dividing
the integration interval from a to b into equal number of
segments and apply the method to each segment.
• The integral would then be equal to the sum of the individual
segments. The resulting equations are called multiple-
application, or composite, integration formulas.
• A general equation for the trapezoidal rule can be expressed as
𝑓 𝑥0 + 2 σ𝑛−1
𝑖=1 𝑓 𝑥𝑖 + 𝑓 𝑥𝑛
𝐼=ℎ
2
𝑏−𝑎
• Where ℎ = is the segment size
𝑛
• And the truncation error
𝑛
𝑏−𝑎 3 𝑏 − 𝑎 3 𝑏 − 𝑎 3
𝐸𝑡 = − 𝑓 ′′ 𝜉 ≅ − 𝑛 𝑓 ҧ′′ = − 𝑓 ҧ′′
12𝑛3 12𝑛3 12𝑛2
𝑖=1
Thus, if the number of segments is doubled, the truncation error
will be quartered.
17
Example of a Multiple-Application Trapezoidal Rule
• Use the two-segment trapezoidal rule to numerically integrate the following function from
𝑎 = 0 to 𝑏 = 0.8 (Recall from the previous example that the exact value of the integral
1.640533)
𝑓 𝑥 = 0.2 + 25𝑥 − 200𝑥 2 + 675𝑥 3 − 900𝑥 4 + 400𝑥 5
The general trapezoidal rule:
𝑓 𝑥0 + 2 σ𝑛−1
𝑖=1 𝑓 𝑥𝑖 + 𝑓 𝑥𝑛
𝐼=ℎ
2
Substitute into the trapezoidal rule for 𝑛 = 2:
0.8 − 0 𝑓 0 + 2 × 𝑓 0.4 + 𝑓 0.8 0.2 + 2 × 2.456 + 0.232
𝐼= × = 0.4 = 1.0688
2 2 2
The actual error can be calculated as:
𝐸𝑡 = 1.640533 − 1.0688 = 0.57173
𝜖 𝑡 = 34.9%
18
Truncation error diminishes as ℎ gets smaller
The following results are obtained from applying
multiple-application trapezoidal rule to the
previous example with decreasing segment size.
19
Simpson’s Rules
• A more accurate estimate of the integral can be obtained by using higher-order
polynomials to connect the points.
• For example, if there is an extra point midway between f(a) and f(b), the three
points can be connected with a parabola (Simpson’s 1/3 rule).
• If there are two points equally spaced between f(a) and f(b), the four points can
connected with a third-order polynomial (Simpson’s 3/8 rule).
20
Segments (n)
Comparing Newton-Cotes Formulas (single application)
Trapezoidal 𝑓 𝑥0 + 𝑓 𝑥1 𝑏−𝑎 𝑓 𝑥0 + 𝑓 𝑥1
1 2 Rule ℎ
1
𝑏−𝑎
2 2
Simpson’s 1/3 1 𝑏−𝑎 𝑓 𝑥0 + 4𝑓 𝑥1 + 𝑓 𝑥2
2 3 rule 3
ℎ 𝑓 𝑥0 + 4𝑓 𝑥1 + 𝑓 𝑥2
2
𝑏−𝑎
6
Simpson’s 3/8 3 𝑏−𝑎 𝑓 𝑥0 + 3𝑓 𝑥1 + 3𝑓 𝑥2 + 𝑓 𝑥3
3 4 rule 8
ℎ 𝑓 𝑥0 + 3𝑓 𝑥1 + 3𝑓 𝑥2 + 𝑓 𝑥3
3
𝑏−𝑎
8
𝐼 = න 𝑓 𝑥 𝑑𝑥 + න 𝑓 𝑥 𝑑𝑥 + ⋯ + න 𝑓 𝑥 𝑑𝑥
𝑥0 𝑥2 𝑥𝑛−2
23
The Multiple-Application Simpson’s 1/3 Rule
• Substituting Simpson’s 1/3 rule for the individual
integral yields:
𝑓 𝑥0 +4𝑓 𝑥1 +𝑓 𝑥2 𝑓 𝑥2 +4𝑓 𝑥3 +𝑓 𝑥4
• I ≅ 2ℎ + 2ℎ +⋯+
6 6
𝑓 𝑥𝑛−2 +4𝑓 𝑥𝑛−1 +𝑓 𝑥𝑛
2ℎ
6
24
Example of a Multiple-Application Simpson’s 1/3 Rule
• Use the multiple-application Simpson’s 1/3 rule with 𝑛 = 4 to numerically integrate the
same function from previous examples (𝑎 = 0 to 𝑏 = 0.8) (recall from the previous
example that the exact value of the integral 1.640533)
𝑓 𝑥 = 0.2 + 25𝑥 − 200𝑥 2 + 675𝑥 3 − 900𝑥 4 + 400𝑥 5
𝑏−𝑎 0.8−0
For 𝑛 = 4: ℎ = = = 0.2
4 4
25
Comparing Truncation Errors of Newton-Cotes Formulas
(single application)
Segments (n)
Points
Newton-Cotes Formula
Name h Truncation Errors
𝑏−𝑎 3
𝑓 𝑥0 + 𝑓 𝑥1 𝑏−𝑎
1 2 Trapezoidal Rule ℎ
1
𝐸𝑡 = − 𝑓 ′′ 𝜉
2 12
1 𝑏−𝑎 𝑏−𝑎 5 4
2 3 Simpson’s 1/3 rule
3
ℎ 𝑓 𝑥0 + 4𝑓 𝑥1 + 𝑓 𝑥2
2
𝐸𝑡 = − 𝑓 𝜉
2880
3 𝑏−𝑎 𝑏−𝑎 5 4
3 4 Simpson’s 3/8 rule
8
ℎ 𝑓 𝑥0 + 3𝑓 𝑥1 + 3𝑓 𝑥2 + 𝑓 𝑥3
3
𝐸𝑡 = − 𝑓 𝜉
6480
𝜖 𝑡 = 7.4%
28
Example of using Simpson’s 3/8 Rule with Simpson’s 1/3 Rule
• Use the Simpson’s 3/8 rule in conjunction with Simpson’s 1/3 rule to numerically integrate
the same function from previous examples (𝑎 = 0 to 𝑏 = 0.8) for five segments (recall from
the previous example that the exact value of the integral 1.640533)
𝑓 𝑥 = 0.2 + 25𝑥 − 200𝑥 2 + 675𝑥 3 − 900𝑥 4 + 400𝑥 5
𝑏−𝑎 0.8
The data needed for a five-segment application (ℎ = = = 0.16
𝑁 5
29
Example of using Simpson’s 3/8 Rule with Simpson’s 1/3 Rule
For the last three segments (𝑥2 𝑡𝑜 𝑥3 ;𝑥3 𝑡𝑜 𝑥4 ;𝑥4 𝑡𝑜 𝑥5 ), the 3/8 rule can be used to
obtain
3
ℎ 𝑓 𝑥2 + 3𝑓 𝑥3 + 3𝑓 𝑥4 + 𝑓 𝑥5 =
8
3
0.16 𝑓 0.32 + 3𝑓 0.48 + 3𝑓 0.64 + 𝑓 0.8 =
8
3
0.16 1.743393 + 3 3.186015 + 3 3.181929 + 0.232 = 1.264754
8
The total integral is computed by summing the two results:
𝐼 = 0.3803237 + 1.264753 = 1.645077
𝐸𝑡 = 1.640533 − 1.645077 = −0.00454383
𝜖 𝑡 = −0.28%
The truncation error is much lower than those of earlier applications.
30
Integration with Unequal Segments
• The previous formulas discussed earlier apply only for segments of
equal size, i.e. equally spaced data points.
• One approach of numerically integrating unevenly spaced data is to
apply the trapezoidal data to each segment and sum the results:
𝑓 𝑥0 + 𝑓 𝑥1 𝑓 𝑥1 + 𝑓 𝑥2 𝑓 𝑥𝑛−1 + 𝑓 𝑥𝑛
𝐼 = ℎ1 + ℎ2 + ⋯ + ℎ𝑛
2 2 2
where ℎ𝑖 =the width of segment 𝑖.
• If some adjacent segments are equal then they could be evaluated
using Simpson’s rules.
31
Example: Trapezoidal Rule with Unequal Segments
• Consider the following table generated with unequally
x f(x)
spaced values based on the polynomial used in the previous 0.00 0.200000
examples. 0.12 1.309729
𝑓 𝑥 = 0.2 + 25𝑥 − 200𝑥 2 + 675𝑥 3 − 900𝑥 4 + 400𝑥 5 0.22 1.305241
0.32 1.743393
• Determine based on the trapezoidal rule the integral for this
data. (recall that correct answer is 1.640533) 0.36 2.074903
0.40 2.456000
• Applying the trapezoidal rule yields: 0.44 2.842985
𝐼 = 0.12
0.2 + 1.309729
+ 0.10
1.309729 + 1.305241
+ ⋯ + 0.1
2.363 + 2.363 0.54 3.507297
2 2 2
= 0.090584 + 0.130749 + ⋯ + 0.12975 = 1.594801 0.64 3.181929
0.70 2.363000
which represents an absolute percent relative error of 𝜀𝑡 = 2.8%.
0.80 0.232000
32
Example: Inclusion of Simpson’s Rules in the Evaluation of
Uneven Data
• Recompute the integral for the same data, but use x f(x)
Simpson’s rules for those segments where they are 0.00 0.200000
appropriate.
0.12 1.309729
• The first segment is evaluated with the trapezoidal 0.22 1.305241
rule:
0.2 + 1.309729 0.32 1.743393
𝐼 = 0.12 − 0.0 = 0.09058376
2 0.36 2.074903
• The next two segments (from x=0.12 to 0.32) are of 0.40 2.456000
equal length. Consequently their integral can be 0.44 2.842985
computed with a single application of Simpson’s 1/3
rule: 0.54 3.507297
1.309729 + 4 1.305241 + 1.743393 0.64 3.181929
𝐼 = 0.32 − 0.12
6 0.70 2.363000
= 0.2726863
0.80 0.232000
33
Example: Inclusion of Simpson’s Rules in the Evaluation of
Uneven Data – contd.
• The next three segments (from x=0.32 to 0.44) are of
x f(x)
equal length. Consequently their integral can be 0.00 0.200000
computed with a single application of Simpson’s 3/8 0.12 1.309729
rule: 0.22 1.305241
𝑓 0.32 + 3𝑓 0.36 + 3𝑓 0.40 + 𝑓 0.44 0.32 1.743393
𝐼 = 0.44 − 0.32 0.36 2.074903
8
1.743393 + 3 2.074903 + 3 2.456 + 2.842985 0.40 2.456000
𝐼 = 0.12
8 0.44 2.842985
= 0.2726863
0.54 3.507297
0.64 3.181929
0.70 2.363000
0.80 0.232000
34
Example: Inclusion of Simpson’s Rules in the Evaluation of
Uneven Data – contd.
• The next two segments (from 𝑥 = 0.44 𝑡𝑜 0.64) are
x f(x)
of equal length. Consequently their integral can be 0.00 0.200000
computed with single application of Simpson’s 1/3 0.12 1.309729
rule: 0.22 1.305241
𝑓 0.44 + 4𝑓 0.54 + 𝑓 0.64 0.32 1.743393
𝐼 = 0.64 − 0.44 0.36 2.074903
6
2.842985 + 4 3.507297 + 3.181929 0.40 2.456000
𝐼 = 0.64 − 0.44 0.44 2.842985
6
= 0.6684701 0.54 3.507297
0.64 3.181929
0.70 2.363000
0.80 0.232000
35
Example: Inclusion of Simpson’s Rules in the Evaluation of
Uneven Data – contd.
• Finally, the last two segments, which are of unequal
x f(x)
length, can be evaluated with the trapezoidal rule to 0.00 0.200000
give values of 0.1663479 and 0.1297500, respectively. 0.12 1.309729
0.22 1.305241
• The area of these individual segments can be
summed to yield a total integral: 0.32 1.743393
0.36 2.074903
0.09058376 + 0.2758029 + 0.2726863
0.40 2.456000
+ 0.6684701 + 0.1663479 + 0.1297500
= 1.603641 0.44 2.842985
0.54 3.507297
Which represents an error of 𝜀𝑡 = 2.2%, which is 0.64 3.181929
superior to the result using the trapezoidal rule in the 0.70 2.363000
previous example.
0.80 0.232000
36
Example: Inclusion of Simpson’s Rules in the Evaluation of
Uneven Data – contd.
37
Multiple Integrals
• Multiple integrals – widely common in
engineering - involve more than one variable.
• For example, a general equation to compute
the average of a two-dimensional function can
be written as:
𝑑 𝑏
𝑥 𝑓 𝑎 𝑐, 𝑦 𝑑𝑥 𝑑𝑦
𝑓ҧ =
𝑑−𝑐 𝑏−𝑎
The numerator is called a double integral.
• From calculus (order of integration is not
important
𝑑 𝑏 𝑏 𝑑
න න 𝑓 𝑥, 𝑦 𝑑𝑥 𝑑𝑦 = න න 𝑓 𝑥, 𝑦 𝑑𝑦 𝑑𝑥
𝑐 𝑎 𝑎 𝑐
39
Example: Using double integral to determine average
temperature
• The temperature of a rectangular
heated plate is described by the
following function:
𝑇 𝑥, 𝑦 = 2𝑥𝑦 + 2𝑥 − 𝑥 2 − 2𝑦 2 + 72
• Compute the average temperature
given that the plate is 8-m long (𝑥
dimension) and 6-m wide (𝑦
dimension).
• Note that the simple average is 47.33
• The true value can be obtained
analytically and is equal to 58.66667
40
Example: Using double integral to determine average
temperature
Solution:
• Use two-segment applications of the trapezoidal rule in each dimension.
𝑓 𝑥0 +2 σ𝑛−1
𝑖=1 𝑓 𝑥𝑖 +𝑓 𝑥𝑛
𝐼= ℎ (general multiple trapezoidal))
2
𝑏−𝑎 𝑓 𝑥0 +2𝑓 𝑥1 +𝑓 𝑥2
For two-segment trapezoidal rule 𝐼 =
2 2
• Apply the trapezoidal rule along the 𝑥 dimension for each y values. Then
these values are integrated along the 𝑦 dimension to give a final result equal
to 2688.
• Dividing this number by the area yields the average temperature as
2688Τ 6 × 8 = 56.
41
42
Example: Using double integral to determine average
temperature
• A single-segment Simpson’s 1/3 rule can be applied in the same
fashion.
• This results in an integral of 2816 and average of 58.66667, which is
exact.
• This occurs because Simpson’s 1/3 rule is based on 2nd order
polynomial and will therefore produce a perfect match for 2nd order
(cubic) polynomials.
• Since the highest order term in the function is 2nd order, the same
exact result occurs for present case.
43