Numerical Methods (CENG 3081) Chapter 6

CHAPTER 6
NUMERICAL DIFFERENTIATION & INTEGRATION
6.1 Introduction
Calculus is mathematics of change. Because engineers must continuously deal with
systems and processes that change, calculus is an essential tool of the engineering
profession. Standing at the heart of calculus are the related mathematical concepts of
differentiation and integration. Mathematically, the derivative represents the rate of
change of a dependent variable with respect to an independent variable. If y is the
dependent variable and x is the independent variable, the first derivative of y = f(x) w.r.t.
to x, represented by dy/dx, is given by

dy f (xi + x ) − f (xi )
= lim (6.1)
dx x→0 x

The inverse process to differentiation in calculus is integration. Mathematically,

integration is represented by
b
I =  f (x )dx (6.2)
a

Which stands for the integral of the function f(x) w.r.t. the independent variable x,
evaluated between the limits x = a to x = b. The function f(x) is referred to as the
integrand.
Non-Computer Methods for Differentiation and Integration
The function to be differentiated or integrated will typically be in one of the following
three forms:
1. A simple continuous function such as a polynomial, an exponential, or a
trigonometric function.
2. A complicated continuous function that is difficult or impossible to differentiate
or integrate directly.
3. A tabulated function where the values of x and f(x) are given at a number of
discrete points, as is often the case with experimental or field data.
In the first case, the derivative or integral of a simple function may be evaluated
analytically using calculus. For the second case, analytical solutions are often impractical,
and sometimes impossible, to obtain. In these instances, as well as in the third case of
discrete data, approximate methods must be employed.
A non-computer method for determining derivatives from data is called equal-area
graphical differentiation. In this method, the (x, y) data are tabulated and, for each
interval, a simple divided difference Δy/Δx is employed to estimate the slope. Then these
values are plotted as a stepped curve versus x. Next a smooth curve is drawn that
attempts to approximate the area under the stepped curve. That is, it is drawn so that

School of Civil and Environmental Engineering 2023/24 1

Numerical Methods (CENG 3081) Chapter 6

visually, the positive and the negative areas are balanced. The rates at given values of x
can then be read from the curve.
In the same spirit, visually oriented approaches were employed to integrate tabulated data
and complicated functions in the pre-computer era. A simple intuitive approach is to plot
the function on a grid and count the number of boxes that approximate the area. This
number multiplied by the area of each box provides a rough estimate of the total area
under the curve. This estimate can be refined, at the expense of additional effort, by using
a finer grid.
Another commonsense approach is to divide the area into vertical segments, or strips,
with a height equal to the function value at the midpoint of each strip. The area of the
rectangles can be then calculated and summed to estimate the total area. In this approach,
it is assumed that the value at the midpoint provides a valid approximation of the average
height of the function for each strip. As with the grid method, refined estimates are
possible by using more (and thinner) strips to approximate the integral.

6.2 Numerical differentiation

6.2.1 Lower Order Methods
By truncating the second- and higher-derivatives in the Taylor series

f (xi ) 2
f (xi +1 ) = f (xi ) + f (xi )h + h + .... (6.3)
2

The following formula for approximating the first derivative, called the first forward
difference, is obtained
f (xi +1 ) − f (xi ) f
f (x ) = + O(h ) = i + O(h ) (6.4)
h h

Where f i is referred to as the first forward difference and h is called the step size. The
entire term f h is referred to as a first finite divided difference.

Similarly, by truncating the Taylor series between xi −1 and x i ,

f (xi ) 2
f (xi ) = f (xi −1 ) − f (xi )h + h − ... (6.5)
2!

The following formula for approximating the first derivative is obtained

f (xi −1 ) − f (xi ) f
f (xi ) = + O(h ) = i + O(h ) (6.6)
h h

Where f i is referred to as the first backward difference.

School of Civil and Environmental Engineering 2023/24 2

Numerical Methods (CENG 3081) Chapter 6

A third way to approximate the first derivative is to subtract backward Taylor series
expansions (between xi −1 and x i ) from the forward Taylor series expansion (between x i
and xi +1 ) resulting in

f 3 ( xi ) 3
f (xi +1 ) = f (xi −1 ) + 2 f (xi )h + h + ... (6.7)
3!
From which we obtain
f (xi +1 ) − f (xi −1 )
f (xi ) =
2h
( ) − O h2 (6.8)
The above equation is a centered difference representation of the first derivative.

Example Consider the function

Compute numerical approximations f’[1] for the
derivative, using step sizes,

--------------------------

School of Civil and Environmental Engineering 2023/24 3

Numerical Methods (CENG 3081) Chapter 6

6.2.2 High-Accuracy Differentiation Formulas

High-accuracy divided difference formulas can be generated by including additional
terms from the Taylor series expansion. For example, the forward Taylor series
expansion can be written as

f (xi ) 2
f (xi +1 ) = f (xi ) + f (xi )h + h + ... (6.9)
2

which can be solved for

f (xi +1 ) − f (xi ) f (xi ) 2

f (xi ) =
h
−
2
h + O h2 ( ) (6.10)

The result can be truncated by excluding the second- and higher-derivative terms and
were thus left with a final result of

f (xi +1 ) − f (xi )
f (xi ) = + O(h ) (6.11)
h

We can retain the second-derivative term by substituting the following approximation of

the second derivative

f (xi + 2 ) − 2 f (xi +1 ) + f (xi )

f (xi ) = + O(h ) (6.12)
h2

Into Eq. 6.10 to yield

f (xi +1 ) − f (xi ) f (xi + 2 ) − 2 f (xi +1 ) + f (xi )

f (xi ) =
h
−
2h 2
( )
h + O h2 (6.11)

Or, collecting terms,

− f (xi + 2 ) + 4 f (xi +1 ) − 3 f (xi )

f (xi ) =
2h
( )
+ O h2 (6.12)

Notice that the inclusion of the second-derivative term has improved the accuracy to
O (h 2 ) . Similar improved versions can be developed for the backward and centered
formulas as well as for the approximations of the higher derivatives. The formulas are
given below.

School of Civil and Environmental Engineering 2023/24 4

Numerical Methods (CENG 3081) Chapter 6

Forward Finite-divided Difference Formulas

First Derivative Error
f (xi +1 ) − f (xi )
f (xi ) = O(h )
h
− f (xi + 2 ) + 4 f (xi +1 ) − 3 f (xi )
f (xi ) = ( )
O h2
2h

Second Derivative
f (xi + 2 ) − 2 f (xi +1 ) + f (xi )
f (xi ) =
h2
( )
Oh
− f (xi +3 ) + 4 f (xi + 2 ) − 5 f (xi +1 ) + 2 f (xi )
f (xi ) = ( )
O h2
h2

Third Derivative
f (xi +3 ) − 3 f (xi + 2 ) + 3 f (xi +1 ) − f (xi )
f (xi ) = O(h )
h3
− 3 f (xi + 4 ) + 14 f (xi +3 ) − 24 f (xi + 2 ) + 18 f (xi +1 ) − 5 f (xi )
f (xi ) = ( )
O h2
2h 3

Fourth Derivative
f (xi + 4 ) − 4 f (xi +3 ) + 6 f (xi + 2 ) − 4 f (xi +1 ) + f (xi )
f ( 4 ) ( xi ) = O(h )
h4
− 2 f (xi +5 ) + 11 f (xi + 4 ) − 24 f (xi +3 ) + 26 f (xi + 2 ) − 14 f (xi +1 ) + 3 f (xi )
f ( 4 ) ( xi ) = ( )
O h2
h4

Backward Finite-divided Difference Formulas

First Derivative Error
f (xi ) − f (xi −1 )
f (xi ) = O(h )
h
3 f (xi ) − 4 f (xi −1 ) + f (xi −2 )
f (xi ) = ( )
O h2
2h

Second Derivative
f (xi ) − 2 f (xi −1 ) + f (xi −2 )
f (xi ) = O(h )
h2
2 f (xi ) − 5 f (xi −1 ) + 4 f (xi −2 ) − f (xi −3 )
f (xi ) = ( )
O h2
h2
Third Derivative
f (xi ) − 3 f (xi −1 ) + 3 f (xi −2 ) − f (xi −3 )
f (xi ) = O(h )
h3

School of Civil and Environmental Engineering 2023/24 5

Numerical Methods (CENG 3081) Chapter 6

5 f (xi ) − 18 f (xi −1 ) + 24 f (xi −2 ) − 14 f (xi −3 ) + 3 f (xi −4 )

f (xi ) = ( )
O h2
2h 3

Fourth Derivative
f (xi ) − 4 f (xi −1 ) + 6 f (xi −2 ) − 4 f (xi −.3 ) + f (xi −4 )
f ( 4 ) ( xi ) = O(h )
h4
3 f (xi ) − 14 f (xi −1 ) + 26 f (xi −2 ) − 24 f (xi −3 ) + 11 f (x x −41 ) − 2 f (xi −5 )
f ( 4 ) ( xi ) = ( )
O h2
h4

Centered Finite-divided Difference Formulas

First Derivative
f (xi +1 ) − f (xi −1 )
f (xi ) = ( )
O h2
2h
− f (xi + 2 ) + 8 f (xi +1 ) − 8 f (xi −1 ) + f (xi −2 )
f (xi ) = ( )
O h4
12h

Second Derivative
f (xi +1 ) − 2 f (xi ) + f (xi −1 )
f (xi ) = ( )
O h2
h2
− f (xi + 2 ) + 16 f (xi +1 ) − 30 f (xi ) + 16 f (xi −1 ) − f (xi −2 )
f (xi ) = ( )
O h4
12h 2

Third Derivative
f (xi + 2 ) − 2 f (xi +1 ) + 2 f (xi −1 ) − f (xi −2 )
f (xi ) = ( )
O h2
2h 3
− f (xi +3 ) + 8 f (xi + 2 ) − 13 f (xi +1 ) + 13 f (xi −1 ) − 8 f (xi −2 ) + f (xi −3 )
f (xi ) = ( )
O h4
8h 3

Fourth Derivative
f (xi + 2 ) − 4 f (xi +1 ) + 6 f (xi ) − 4 f (xi −1 ) + f (xi −2 )
f ( 4 ) ( xi ) = 4
O (h 2 )
h
− f ( x ) + 12 f ( x ) − 39 f (xi +1 ) + 56 f (xi ) − 39 f (xi −1 ) + 12 f (xi −2 ) + f (xi −3 )
f ( 4 ) ( xi ) = i +3 i+2

6h 4
O (h 4 )
6.3 Newton-Cotes integration formulas
The Newton-Cotes formulas are the most common numerical integration schemes. They
are based on the strategy of replacing a complicated function or tabulated data with an
approximating function that is easy to integrate:

b b
I =  f (x )dx   f n (x )dx (6.13)
a a

School of Civil and Environmental Engineering 2023/24 6

Numerical Methods (CENG 3081) Chapter 6

Where f n (x ) = a polynomial of the form

f n (x ) = a0 + a1 x + ... + an−1 x n−1 + an x n (6.14)

Where n is the order of the polynomial.

The integral can be approximated by one polynomial or using a series of polynomials
applied piecewise to the function or date over segments of constant length.
Closed and open forms of the Newton-Cotes formulas are available. The closed forms are
those where the data points at the beginning and end of the limits of integration are
known. The open forms have integration limits that extend beyond the range of the data.
In this sense, they are akin to extrapolation. Open Newton-Cotes formulas are not
generally used for definite integration. However, they are utilized for evaluating improper
integrals and for the solution of ordinary differential equations.

6.3.1 The Trapezoidal Rule

The trapezoidal rule is the first of the Newton-Cotes closed integration formulas. It
corresponds to the case where the polynomial in Eq. 6.13 is first-order:

b b
I =  f (x )dx   f1 (x )dx (6.15)
a a

The straight line passing through the two points (a, f(a)) and (b, f(b)) is given by

f (b ) − f (a )
f1 (x ) = f (a ) + (x − a ) (6.16)
b−a

The area under this straight line is an estimate of the integral of f(x) between the limits a
and b:

 f (b ) − f (a )
(x − a )dx
b
I =   f (a ) + (6.17)
a 
b−a 

The result of the integration is

f (a ) + f (b )
I = (b − a ) (6.18)
2

Which is called the trapezoidal rule.

Error of the Trapezoidal Rule

School of Civil and Environmental Engineering 2023/24 7

Numerical Methods (CENG 3081) Chapter 6

When we employ the integral under a straight line to approximate the integral under the
curve, we obviously can incur an error that may be substantial. An estimate of the local
truncation error of a single application of the trapezoidal rule is

f ( )(b − a )
1
Et = −
3
(6.19)
12

where ξ lies somewhere in the interval from a to b.

Multiple-Application of Trapezoidal Rule

One way to improve the accuracy of the trapezoidal rule is to divide the integration
interval from a to b into a number of segments and apply the method to each segment.
The areas of the individual segments can then be added to yield the integral for the entire
interval. The resulting equations are called multiple-application, or composite,
integration formulas.
Considering n + 1 equally spaced base points (x0, x1, x2, ..., xn), and n segments of equal
width:

b−a
h= (6.20)
n

If a and b are designated as x0 and xn, respectively, the total integral can be represented as
x1 x2 xn

I=  f (x )dx +  f (x )dx + ... +  f (x )dx

x0 x1 xn −1
(6.21)

Substituting the trapezoidal rule for each interval yields

f (x0 ) + f (x1 ) f (x1 ) + f (x 2 ) f (xn −1 ) + f (xn )

I =h +h + ... + h (6.22)
2 2 2

or, grouping terms,

h n −1

I=  f ( x 0 ) + 2  f (xi ) + f (x n ) (6.23)
2 i =1 

or, using Eq. 5.20 to express Eq. 5.23 in the following form,
n −1
f ( x 0 ) + 2 f ( x i ) + f ( x n )
I = (b − a ) i =1
(6.24)
2n
Width Average Height

School of Civil and Environmental Engineering 2023/24 8

Numerical Methods (CENG 3081) Chapter 6

Because the summation of the coefficients of f(x) in the numerator divided by 2n is equal
to 1, the average height represents a weighted average of the function values. According
to Eq. 6.24, the interior points are given twice the weight of the two end points f(x0) and
f(xn).
An error for the multiple-application trapezoidal rule can be obtained by summing the
individual errors for each segment to give
(b − a ) n
Et = −
12n 3 i =1
i  f ( ) (6.25)

where f"(ξi) is the second derivative at a point ξi located in segment i. This result can be
simplified by estimating the mean or the average value of the second derivative for the
entire interval as
n

 f ( ) i
f   i =1
(6.26)
n

Therefore  f ( )  nf  and Eq. 6.25 can be rewritten as

i

Ea =−
(b − a)
3
f  (6.27)
12n 2

Thus, if the number in the bracket is doubled, the truncation error will be quartered. Note
that Eq. 6.27 is an approximate error because the approximate nature of Eq. 6.26.
NB: In the multiple-application of the trapezoidal formula, the error decreases as the
number of segments n increases. However, the rate of decrease of error is gradual.
(Doubling the number of segments quarters the error.) In the subsequent sections, we will
develop higher-order formulas that are more accurate and that converge more quickly on
the true integral as the segments are increased.

Example Numerically approximate the integral

by using the trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

For m = 1

For m = 2

School of Civil and Environmental Engineering 2023/24 9

Numerical Methods (CENG 3081) Chapter 6

For
m=4

For m = 8

For m = 16

6.3.2 Simpson's Rules

Aside from applying the trapezoidal rule with finer segmentation, another way to obtain a
more accurate estimate of an integral is to use higher-order polynomial to connect the
points. For example, if there is an extra point midway between f(a) and f(b), the three
points can be connected with a parabola. If there are two points equally spaced between
f(a) and f(b), the four points can be connected with a third-order polynomial. The
formulas that result from taking the integrals under these polynomials are called
Simpson's rules.

i) Simpson's 1/3 Rule

Simpson's 1/3 rule results when a second-order interpolating polynomial is substituted

into Eq. 6.13:

b b
I =  f (x )dx   f 2 (x )dx (6.28)
a a

School of Civil and Environmental Engineering 2023/24 10

 Numerical Methods (CENG 3081) Chapter 6

If a and b are designated as x0 and x2 and f2(x) is represented by a second-order Lagrange

polynomial, the intergral becomes
x2
 (x − x1 )(x − x 2 ) (x − x0 )(x − x2 ) (x − x1 )(x − x2 ) f (x )dx
I =  f (x0 ) + f (x1 ) +
(x0 − x1 )(x0 − x2 ) 2 
(6.29)
x 
(x0 − x1 )(x0 − x2 ) (x0 − x1 )(x0 − x2 )

After integration and algebraic manipulation, the following formula results

I
h
 f (x0 ) + 4 f (x1 ) + f (x2 ) (6.30)
3

Where, for this case, h = (b − a ) 2 . This equation is known as Simpson's 1/3 rule.

Simpson's 1/3 rules can also be expressed in the following format

f (x0 ) + 4 f (x1 ) + f (x2 )

I  (b − a ) (6.31)
6
Width Average Height

Where a = x0 , b = x 2 and x1 = the point midway between a & b, which is given

by (b + a ) 2 . Notice that, according to the above equation, the middle point is weighted
two-thirds and the two end points by one-sixth.

It can be shown that a single-segment application of Simpson's 1/3 rule has a truncation
error of

h f ( )
1 5 (4 )
Et = − (6.32)
90

Or, because h = (b − a ) 2 ,

Et =−
(b − a)
5
f (4 ) ( ) (6.33)
2880

Where ξ lies somewhere in the interval from a to b. Thus, Simpson's 1/3 rule is more
accurate that the trapezoidal rule.

Multiple Application of Simpson's 1/3 Rule

Just as with the trapezoidal rule, Simpson's rule can be improved by dividing the
integration interval into a number of segments of equal width:

School of Civil and Environmental Engineering 2023/24 11

Numerical Methods (CENG 3081) Chapter 6

b−a
h= (6.34)
n

The total integration can be represented as

x2 x4 xn

I=  f (x )dx +  f (x )dx + ... +  f (x )dx

x0 x2 xn − 2
(6.35)

Substituting Simpson's 1/3 rule for the individual integral yields

f (x0 ) + 4 f (x1 ) + f (x 2 ) f ( x 2 ) + 4 f ( x3 ) + f ( x 4 )
I  2h + 2h + ...
6 6
(6.36)
f (x n −2 ) + 4 f (x n −1 ) + f (x n )
+ 2h
6

or, combining terms, and using Eq. 6.34,

n −1 n−2
f (x0 ) + 4  f ( xi ) + 2  f (x ) + f (x )
j n

I  (b − a )
i =1, 3, 5 j = 2, 4,6
(6.37)
3n
Width Average Height

An error estimate for the multiple-application Simpson's rule is obtained in the same
fashion as for the trapezoidal rule by summing the individual errors for the segments and
averaging the derivative to yield

Ea = −
(b − a )5 f (4 ) (6.38)
4
180n

Where f (4 ) is the average fourth derivative for the interval.

Example Numerically approximate the integral

by using Simpson's rule with m = 1, 2, 4, and 8.

For m = 1

For m = 2

School of Civil and Environmental Engineering 2023/24 12

Numerical Methods (CENG 3081) Chapter 6

For m = 4

For m = 8

Simpson's 3/8 Rule

In a similar manner to the derivation of the trapezoidal and Simpson's 1/3 rule, a third-
order Lagrange polynomial can be fit to four points and integrated:

b b
I =  f (x )dx   f 3 (x )dx (6.39)
a a

to yield

3h f (x0 ) + 3 f (x1 ) + 3 f (x2 ) + f (x3 )

I (6.40)
8 8

where h = (b − a ) 3 . This equation is called Simpson's 3/8 rule because h is multiplied by

3/8. It is the third Newton-Cotes closed integration formula. The 3/8 rule can also be
expressed in the following form:
f (x0 ) + 3 f (x1 ) + 3 f (x 2 ) + f (x3 )
I  (b − a ) (6.41)
8
Width Average Height

Thus, the two interior points are given weight's are given weights of three-eighths,
whereas the end points are weighted with one-eighth. Simpson's 3/8 rule has an error of

Et =−
(b − a)
5
f (4 ) ( ) (6.42)
6480

School of Civil and Environmental Engineering 2023/24 13

Numerical Methods (CENG 3081) Chapter 6

Simpson's 1/3 rule is usually the method of preference because it attains third-order
accuracy with three points rather than the four points required for the 3/8 version.
However, the 3/8 rule has utility when the number of segments is odd.

Summary of the Newton-Cotes integration formulas is given below.

Example Numerically approximate the

integral by using Simpson's 3/8 rule with m = 1, 2, 4.

For m = 1

For m = 2

For m = 3

School of Civil and Environmental Engineering 2023/24 14