Numerical Methods (CENG 2002) Lecture Note

CHAPTER-V
NUMERICAL DIFFERENTIATION & INTEGRATION
5.1 Introduction
Calculus is mathematics of change. Because engineers must continuously deal with systems and
processes that change, calculus is an essential tool of the engineering profession. Standing at the
heart of calculus are the related mathematical concepts of differentiation and integration.
Mathematically, the derivative represents the rate of change of a dependent variable with respect
to an independent variable. If y is the dependent variable and x is the independent variable, the
first derivative of y = f(x) w.r.t. to x, represented by dy/dx, is given by

dy f xi  x   f xi 
 lim (5.1)
dx x0 x

The inverse process to differentiation in calculus is integration. Mathematically, integration is

represented by
b
I   f x dx (5.2)
a

which stands for the integral of the function f(x) w.r.t. the independent variable x, evaluated
between the limits x = a to x = b. The function f(x) is referred to as the integrand.
Non-Computer Methods for Differentiation and Integration
The function to be differentiated or integrated will typically be in one of the following three
forms:
1. A simple continuous function such as a polynomial, an exponential, or a trigonometric
function.
2. A complicated continuous function that is difficult or impossible to differentiate or
integrate directly.
3. A tabulated function where the values of x and f(x) are given at a number of discrete
points, as is often the case with experimental or field data.
In the first case, the derivative or integral of a simple function may be evaluated analytically
using calculus. For the second case, analytical solutions are often impractical, and sometimes
impossible, to obtain. In these instances, as well as in the third case of discrete data, approximate
methods must be employed.
A non-computer method for determining derivatives from data is called equal-area graphical
differentiation. In this method, the (x, y) data are tabulated and, for each interval, a simple divided
difference Δy/Δx is employed to estimate the slope. Then these values are plotted as a stepped
curve versus x. Next a smooth curve is drawn that attempts to approximate the area under the
stepped curve. That is, it is drawn so that visually, the positive and the negative areas are
balanced. The rates at given values of x can then be read from the curve.
In the same spirit, visually oriented approaches were employed to integrate tabulated data and
complicated functions in the pre-computer era. A simple intuitive approach is to plot the function
on a grid and count the number of boxes that approximate the area. This number multiplied by the

Academic year 2008/9 Instructor: Zerihun A. 1

Numerical Methods (CENG 2002) Lecture Note

area of each box provides a rough estimate of the total area under the curve. This estimate can be
refined, at the expense of additional effort, by using a finer grid.
Another commonsense approach is to divide the area into vertical segments, or strips, with a
height equal to the function value at the midpoint of each strip. The area of the rectangles can be
then calculated and summed to estimate the total area. In this approach, it is assumed that the
value at the midpoint provides a valid approximation of the average height of the function for
each strip. As with the grid method, refined estimates are possible by using more (and thinner)
strips to approximate the integral.

5.2 Numerical differentiation

5.2.1 Lower Order Methods
By truncating the second- and higher-derivatives in the Taylor series

f xi  2
f xi 1   f xi   f xi h  h  .... (5.3)
2

the following formula for approximating the first derivative, called the first forward difference, is
obtained
f xi 1   f xi  f
f x    Oh   i  Oh  (5.4)
h h

where f i is referred to as the first forward difference and h is called the step size. The entire
term f h is referred to as a first finite divided difference.

Similarly, by truncating the Taylor series between xi 1 and x i ,

f xi  2
f xi   f xi 1   f xi h  h  ... (5.5)
2!

the following formula for approximating the first derivative is obtained

f xi 1   f xi  f
f xi    Oh   i  Oh  (5.6)
h h

where f i is referred to as the first backward difference.

A third way to approximate the first derivative is to subtract backward Taylor series expansions
(between xi 1 and x i ) from the forward Taylor series expansion (between x i and xi 1 ) resulting
in

f 3  xi  3
f xi 1   f xi 1   2 f xi h  h  ... (5.7)
3!

Academic year 2008/9 Instructor: Zerihun A. 2

Numerical Methods (CENG 2002) Lecture Note

from which we obtain

f xi 1   f xi 1 
f xi  
2h
 
 O h2 (5.8)

The above equation is a centered difference representation of the first derivative.

Example Consider the function

Compute numerical approximations for the f’[1] derivative, using step sizes,

5.2.2 High-Accuracy Differentiation Formulas

High-accuracy divided difference formulas can be generated by including additional terms from
the Taylor series expansion. For example, the forward Taylor series expansion can be written as

f xi  2
f xi 1   f xi   f xi h  h  ... (5.9)
2

which can be solved for

Academic year 2008/9 Instructor: Zerihun A. 3

Numerical Methods (CENG 2002) Lecture Note

f xi 1   f xi  f xi  2

f xi  
h

2
h  O h2   (5.10)

The result can be truncated by excluding the second- and higher-derivative terms and were thus
left with a final result of

f xi 1   f xi 
f xi    Oh  (5.11)
h

We can retain the second-derivative term by substituting the following approximation of the
second derivative

f xi  2   2 f xi 1   f xi 

f xi    Oh  (5.12)
h2

into Eq. 5.10 to yield

f xi 1   f xi  f xi  2   2 f xi 1   f xi 

f xi  
h
 2
 
h  O h2 (5.11)
2h

or, collecting terms,

 f xi  2   4 f xi 1   3 f xi 

f xi  
2h
 
 O h2 (5.12)

Notice that the inclusion of the second-derivative term has improved the accuracy to O h 2 .  
Similar improved versions can be developed for the backward and centered formulas as well as
for the approximations of the higher derivatives. The formulas are given below.

Forward Finite-divided Difference Formulas

First Derivative Error
f xi 1   f xi 
f xi   Oh 
h
 f xi  2   4 f xi 1   3 f xi 
f xi  
2h
 
O h2

Second Derivative
f xi  2   2 f xi 1   f xi 
f xi  
h2
 
O h2
 f xi 3   4 f xi  2   5 f xi 1   2 f xi 
f xi  
h2
 
O h2

Third Derivative

Academic year 2008/9 Instructor: Zerihun A. 4

Numerical Methods (CENG 2002) Lecture Note

f xi 3   3 f xi  2   3 f xi 1   f xi 

f xi   Oh 
h3
 3 f xi  4   14 f xi 3   24 f xi  2   18 f xi 1   5 f xi 
f xi  
2h 3
 
O h2

Fourth Derivative
f xi  4   4 f xi 3   6 f xi  2   4 f xi 1   f xi 
f  4   xi   Oh 
h4
 2 f xi 5   11 f xi  4   24 f xi 3   26 f xi  2   14 f xi 1   3 f xi 
f  4   xi  
h4
 
O h2

Backward Finite-divided Difference Formulas

First Derivative Error
f xi   f xi 1 
f xi   Oh 
h
3 f xi   4 f xi 1   f xi 2 
f xi  
2h
O h2  
Second Derivative
f xi   2 f xi 1   f xi 2 
f xi   Oh 
h2
2 f xi   5 f xi 1   4 f xi 2   f xi 3 
f xi  
h2
 
O h2
Third Derivative
f xi   3 f xi 1   3 f xi 2   f xi 3 
f xi   Oh 
h3
5 f xi   18 f xi 1   24 f xi 2   14 f xi 3   3 f xi 4 
f xi  
2h 3
 
O h2

Fourth Derivative
f xi   4 f xi 1   6 f xi 2   4 f xi .3   f xi 4 
f  4   xi   Oh 
h4
3 f xi   14 f xi 1   26 f xi 2   24 f xi 3   11 f x x 41   2 f xi 5 
f  4   xi  
h4
 
O h2

Centered Finite-divided Difference Formulas

First Derivative
f xi 1   f xi 1 
f xi  
2h
 
O h2

 f xi  2   8 f xi 1   8 f xi 1   f xi 2 

f xi  
12h
 
O h4

Academic year 2008/9 Instructor: Zerihun A. 5

Numerical Methods (CENG 2002) Lecture Note

Second Derivative
f xi 1   2 f xi   f xi 1 
f xi  
h2
 
O h2

 f xi  2   16 f xi 1   30 f xi   16 f xi 1   f xi 2 

f xi  
12h 2
 
O h4

Third Derivative
f xi  2   2 f xi 1   2 f xi 1   f xi 2 
f xi  
2h 3
 
O h2
 f xi 3   8 f xi  2   13 f xi 1   13 f xi 1   8 f xi 2   f xi 3 
f xi  
8h 3
 
O h4

Fourth Derivative
f xi  2   4 f xi 1   6 f xi   4 f xi 1   f xi 2 
f  4   xi  
h 4
 
O h2
 f  x   12 f  x   39 f xi 1   56 f xi   39 f xi 1   12 f xi 2   f xi 3 
f  4   xi   i 3 i2

6h 4
 
O h4
5.3 Newton-Cotes integration formulas
The Newton-Cotes formulas are the most common numerical integration schemes. They are
based on the strategy of replacing a complicated function or tabulated data with an approximating
function that is easy to integrate:

b b
I   f x dx   f n x dx (5.13)
a a

where fn(x) = a polynomial of the form

f n x   a0  a1 x  ...  an1 x n1  an x n (5.14)

where n is the order of the polynomial.

The integral can be approximated by one polynomial or using a series of polynomials applied
piecewise to the function or date over segments of constant length.
Closed and open forms of the Newton-Cotes formulas are available. The closed forms are those
where the data points at the beginning and end of the limits of integration are known. The open
forms have integration limits that extend beyond the range of the data. In this sense, they are akin
to extrapolation. Open Newton-Cotes formulas are not generally used for definite integration.
However, they are utilized for evaluating improper integrals and for the solution of ordinary
differential equations.

5.3.1 The Trapezoidal Rule

The trapezoidal rule is the first of the Newton-Cotes closed integration formulas. It corresponds
to the case where the polynomial in Eq. 5.13 is first-order:

Academic year 2008/9 Instructor: Zerihun A. 6

Numerical Methods (CENG 2002) Lecture Note

b b
I   f x dx   f1 x dx (5.15)
a a

The straight line passing through the two points (a, f(a)) and (b, f(b)) is given by

f b   f a 
f1 x   f a   x  a  (5.16)
ba

The area under this straight line is an estimate of the integral of f(x) between the limits a and b:

 f b   f a 
x  a dx
b
I    f a   (5.17)
a
ba 

The result of the integration is

f a   f b 
I  b  a  (5.18)
2

which is called the trapezoidal rule.

Error of the Trapezoidal Rule
When we employ the integral under a straight line to approximate the integral under the curve, we
obviously can incur an error that may be substantial. An estimate of the local truncation error of a
single application of the trapezoidal rule is

f  b  a 
1
Et  
3
(5.19)
12

where ξ lies somewhere in the interval from a to b.

Multiple-Application of Trapezoidal Rule

One way to improve the accuracy of the trapezoidal rule is to divide the integration interval from
a to b into a number of segments and apply the method to each segment. The areas of the
individual segments can then be added to yield the integral for the entire interval. The resulting
equations are called multiple-application, or composite, integration formulas.
Considering n + 1 equally spaced base points (x0, x1, x2, ..., xn), and n segments of equal width:

ba
h (5.20)
n

If a and b are designated as x0 and xn, respectively, the total integral can be represented as

x1 x2 xn

I  f x dx   f x dx  ...   f xdx

x0 x1 xn 1
(5.21)

Academic year 2008/9 Instructor: Zerihun A. 7

Numerical Methods (CENG 2002) Lecture Note

Substituting the trapezoidal rule for each interval yields

f x0   f x1  f x1   f x2  f xn1   f xn 

I h h  ...  h (5.22)
2 2 2

or, grouping terms,

h n 1

I  f  x 0   2 f xi   f xn  (5.23)
2 i 1 

or, using Eq. 5.20 to express Eq. 5.23 in the following form,

n 1
f x0   2 f xi   f x n 
I  b  a  i 1
(5.24)
2n
Width Average Height

Because the summation of the coefficients of f(x) in the numerator divided by 2n is equal to 1, the
average height represents a weighted average of the function values. According to Eq. 5.24, the
interior points are given twice the weight of the two end points f(x0) and f(xn).
An error for the multiple-application trapezoidal rule can be obtained by summing the individual
errors for each segment to give
b  a  n
Et  
12n 3
 f  
i 1
i (5.25)

where f"(ξi) is the second derivative at a point ξi located in segment i. This result can be
simplified by estimating the mean or the average value of the second derivative for the entire
interval as
n

 f   i
f   i 1
(5.26)
n

Therefore  f    nf  and Eq. 5.25 can be rewritten as

i

Ea  
b  a 3 f  (5.27)
12n 2

Thus, if the number in the bracket is doubled, the truncation error will be quartered. Note that Eq.
5.27 is an approximate error because the approximate nature of Eq. 5.26.
NB: In the multiple-application of the trapezoidal formula, the error decreases as the number of
segments n increases. However, the rate of decrease of error is gradual. (Doubling the number of
segments quarters the error.) In the subsequent sections, we will develop higher-order formulas

Academic year 2008/9 Instructor: Zerihun A. 8

Numerical Methods (CENG 2002) Lecture Note

that are more accurate and that converge more quickly on the true integral as the segments are
increased.

Example Numerically approximate the integral

by using the trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

For m = 1

For m = 2

For m = 4

For m = 8

For m = 16

5.3.2 Simpson's Rules

Aside from applying the trapezoidal rule with finer segmentation, another way to obtain a more
accurate estimate of an integral is to use higher-order polynomial to connect the points. For
example, if there is an extra point midway between f(a) and f(b), the three points can be
connected with a parabola. If there are two points equally spaced between f(a) and f(b), the four
points can be connected with a third-order polynomial. The formulas that result from taking the
integrals under these polynomials are called Simpson's rules.

Academic year 2008/9 Instructor: Zerihun A. 9

 Numerical Methods (CENG 2002) Lecture Note

i) Simpson's 1/3 Rule

Simpson's 1/3 rule results when a second-order interpolating polynomial is substituted into
Eq. 5.13:

b b
I   f x dx   f 2 x dx (5.28)
a a

If a and b are designated as x0 and x2 and f2(x) is represented by a second-order Lagrange

polynomial, the intergral becomes

x2
 x  x1 x  x2  x  x0 x  x2  x  x1 x  x2    (5.29)
I   f x0   f x1  
x0  x1 x0  x2  2 
f x dx
x 
x0  x1 x0  x2  x0  x1 x0  x2 
After integration and algebraic manipulation, the following formula results

I
h
 f x0   4 f x1   f x2  (5.30)
3

where, for this case, h  b  a  2 . This equation is known as Simpson's 1/3 rule.

Simpson's 1/3 rule can also be expressed in the following format

f x0   4 f x1   f x2 

I  b  a  `(5.31)
6
Width Average Height

where a  x0 , b  x2 , and x1 = the point midway between a and b, which is given by

b  a  2 . Notice that, according to the above equation, the middle point is weighted two-thirds
and the two end points by one-sixth.

It can be shown that a single-segment application of Simpson's 1/3 rule has a truncation error of

h f  
1 5 4 
Et   (5.32)
90

or, because h  b  a  2 ,

Et  
b  a 5 f 4    (5.33)
2880

where ξ lies somewhere in the interval from a to b. Thus, Simpson's 1/3 rule is more accurate that
the trapezoidal rule.

Academic year 2008/9 Instructor: Zerihun A. 10

Numerical Methods (CENG 2002) Lecture Note

Multiple Application of Simpson's 1/3 Rule

Just as with the trapezoidal rule, Simpson's rule can be improved by dividing the integration
interval into a number of segments of equal width:

ba
h (5.34)
n

The total integration can be represented as

x2 x4 xn

I  f x dx   f x dx  ...   f xdx

x0 x2 xn  2
(5.35)

Substituting Simpson's 1/3 rule for the individual integral yields

f x0   4 f x1   f x 2  f  x 2   4 f  x3   f  x 4 
I  2h  2h  ...
6 6
(5.36)
f x n 2   4 f x n 1   f x n 
 2h
6

or, combining terms, and using Eq. 5.34,

n 1 n2
f x0   4  f  xi   2  f x   f x 
j n

I  b  a 
i 1, 3, 5 j  2, 4, 6
(5.37)
3n
Width Average Height

An error estimate for the multiple-application Simpson's rule is obtained in the same fashion as
for the trapezoidal rule by summing the individual errors for the segments and averaging the
derivative to yield

Ea  
b  a 5 f 4  (5.38)
180n 4

where f 4  is the average fourth derivative for the interval.

Example Numerically approximate the integral

by using Simpson's rule with m = 1, 2, 4, and 8.

For m = 1

Academic year 2008/9 Instructor: Zerihun A. 11

Numerical Methods (CENG 2002) Lecture Note

For m = 2

For m = 4

For m = 8

Simpson's 3/8 Rule

In a similar manner to the derivation of the trapezoidal and Simpson's 1/3 rule, a third-order
Lagrange polynomial can be fit to four points and integrated:

b b
I   f x dx   f 3 x dx (5.39)
a a

to yield

3h f x0   3 f x1   3 f x2   f x3 

I (5.40)
8 8

where h  b  a  3 . This equation is called Simpson's 3/8 rule because h is multiplied by 3/8. It
is the third Newton-Cotes closed integration formula. The 3/8 rule can also be expressed in the
following form:
f x0   3 f x1   3 f x2   f x3 
I  b  a  (5.41)
8
Width Average Height
Thus, the two interior points are given weight's are given weights of three-eighths, whereas the
end points are weighted with one-eighth. Simpson's 3/8 rule has an error of

Et 
b  a
5
f 4    (5.42)
6480

Academic year 2008/9 Instructor: Zerihun A. 12

Numerical Methods (CENG 2002) Lecture Note

Simpson's 1/3 rule is usually the method of preference because it attains third-order accuracy with
three points rather than the four points required for the 3/8 version. However, the 3/8 rule has
utility when the number of segments is odd.

Summary of the Newton-Cotes integration formulas is given below.

Segments Points Name Formula Truncation Error

(n)
x0   f x1 
1 2 Trapezoidal rule b  a  f  1 12h 3 f  
2
     
2 3 Simpson's 1/3 rule b  a  f x0  4 f x1  f x2  1 90h 5 f 4   
6
       
3 4 Simpson's 3/8 rule b  a  f x0  3 f x1  3 f x2  f x3  3 80h 5 f 4   
8

Example Numerically approximate the integral

by using Simpson's 3/8 rule with m = 1, 2, 4.

For m = 1

For m = 2

For m = 3

Academic year 2008/9 Instructor: Zerihun A. 13