10.

001: Numerical Integration

R. Sureshkumar

1 Introduction
The topic for today's discussion is numerical integration of functions. In particular, we
would like to obtain a numerical approximation to the integral I (f ), of a suciently
smooth, integrable function f (x), de ned in the closed interval a  x  b. i.e.,
Z b
I (f ) = f (x)dx: (1)
a
This problem arises in many engineering applications. In some cases, the function may be
too complicated to allow for an analytical evaluation of the integral. In some other cases,
the `function' itself may be available at a discrete set of points, either from experiment or
from simulation.

2 Numerical Methods
Numerical methods are developed based on the results of mathematical analyses. For
today's lecture, our understanding of elementary calculus suces. The class of numerical
integration techniques we discuss today can be compactly expressed as:
iX
=n

I (f )  i f (xi ): (2)
i=0
i.e., we express the numerical approximation to the integral I (f ) as a sum of a series of
nite number of terms, each term containing the value of the function at a given point
xi weighted by a coecient i . One way to determine the coecients i is to replace
the original function f (x) by an interpolating function which is simple enough, or in the
terminology of numerical analysts, to use a quadrature technique.
Let us explore this idea a little further:
1
2.1 Simple Quadrature Rules
Consider the function f (x) whose integral we wish to approximate over the interval a 
x  b. The simplest of the interpolating functions we can think of is a constant function,
i.e., we assume that for a  x  b, the function f (x) is approximated by the interpolant
function C (x) which is simply given by C (x) = f (a). We now proceed to compute the
integral as the area enclosed by the rectangle formed by y = 0, y = f (a), x = a, and
x = b in the x , y Cartesian plane. This gives
I (f )  (b , a)f (a); (3)
The interpolation rule given by Eq.3 is known as the Rectangle Rule.
Alternatively, we can use the constant interpolant C (x), given by the value of f (x)
at the midpoint of the interval, i.e., C (x) is now given by C (x) = f ( a b ). This results in
+
2
the Midpoint Rule given by
a+b
I (f )  (b , a)f (
2 ): (4)

Evidently, Eq.3 is exact for constant functions and Eq.4 is exact for constant or
linear functions of the type f (x) = x + (Verify this, why so?). However, for the
integrals of functions of higherR order and complexity, Eqs.3 and 4 in general o er poor
approximations. For instance, x dx = 1=3 when approximated with the midpoint rule
1
0
2

gives a value of 1=4. This error would amplify as the polynomial order is increased further.
This motivates us to look for higher order interpolant functions. We could use a
linear interpolant, i.e., we approximate the function f (x) as a straight line, say y (x),1
which passes passes through the points (a; f (a)) and (b; f (b)) (What is the equation
representing this line?). We now evaluate the integral as the area enclosed by the trapezoid
enclosed by y = 0, y = y (x), x = a and x = b. This results in the Trapezoidal Rule, given
1
by
b,a
I (f ) 
2 [f (a) + f (b)]: (5)

A more accurate quadrature rule can be obtained by using a quadratic interpolant

function, which passes through the points (a; f (a)), (b; f (b)) and ((a + b)=2; f ((a + b)=2)).
If we now evaluate the integral as the area enclosed by this parabola, the x axis, and the
lines x = a and x = b, we arrive at the Simpson's rule, given by
b,a a+b
I (f )  [ f (a) + 4f (
6 2 ) + f (b)]: (6)

2
 In a similar fashion, we can derive higher order quadrature rules through the use
of polynomials of arbitrary degree as interpolants. The general class of quadrature rules
which result from such exercises are called Newton-Cotes rules. However, this approach
has many drawbacks from mathematical as well as computational points of view. Math-
ematically, the convergence of the right hand side of Eq.2 is not guaranteed as n ! 1
even for in nitely di erentiable functions. Computationally, for n  8, we run into neg-
ative coecients ( i in Eq.2) resulting in unacceptably high roundo errors (We will
discuss roundo errors in $4:2). Hence, we seek an alternative approach, the one of using
composite formula as explained below.

2.2 Composite Rules

I will illustrate this concept using the midpoint rule de ned by Eq.4. The idea is to divide
the interval [a; b] into n subintervals, the ith subinterval being [xi, ; xi ]. The length of
1
the the ith subinterval, hi, is then given by hi = xi , xi, . Notice that x = a and xn = b.
1 0
Now the contribution to the integral from the ith subinterval, according to the midpoint
rule is given by hif ((xi, + xi )=2). Hence, adding up the elemental contributions from
1
each one of the n subintervals, we arrive at the Composite Midpoint Rule, given by
n x + xi
ICM = hi f ( i,
X
2 ); (7)
1

i =1

where the subscript CM denotes Composite Midpoint.

A similar procedure using Eq.5 and 6 results in the Composite Trapezoidal and the
Composite Simpson's rules respectively, given by
n
X hi
ICT =
i=1 2 [f (xi, ) + f (xi)] for the composite trapezoidal rule
1 (8)
and
n
X hi x + xi
ICS = [f (xi, ) + 4f ( i,
2 ) + f (xi)] for the Composite Simpson's rule: (9)
1

i=1 6 1

3 Writing C Programs to Use the Composite Rules

We would now like to develop a C program which would compute the integral of a user
de ned function using the composite trapezoidal rule using equally spaced points, i.e.,
3
hi = h = (b , a)=n for each subinterval. The program I wrote for this purpose is given
in /mit/10.001/Examples/ctrule.c. Let's analyze this little C program brie y. First of
all, we would need an algorithm based on the composite trapezoidal rule given by Eq.8.
A simple algorithm would be to compute the n elemental contributions (with h factored
out) to the integral and add them up. This would result in 2n function evaluations and
2n additions. However, let's expand Eq.8 to examine the CT rule more closely. When we
expand the series, we get
h
ICT = [f (a) + 2f (x1 ) + 2f (x2 ) +


+ 2f (xn,1 ) + f (b)] : (10)
2
The following algorithm can now be developed based on Eq. 10:

 Initialize Integral = (f (a) + f (b))=2.

 Within a for loop which runs from i = 2 to n , 1, let integral = Integral + f (xi).
 Multiply the nal answer with h.
This method requires only n function evaluations and n additions and hence would yield
answers in half the time as compared to the original algorithm which required 2n function
evaluations and 2n additions. This is the algorithm that has been implemented in the C
program ctrule.c in the /mit/10.001/Examples directory.
Further analysis of program ctrule.c shows that the function to be integrated is
de ned separately in a subprogram: double f(double x). This allows for the modularity of
the program, i.e., if we need to use this program to integrate another function, we only
need to change the function de nition in this subprogram.
We could write similar programs to implement the composite midpoint and the
composite Simpson's rules. In fact we could incorporate in one program, all the three
methods discussed above so that a given user can choose from any one of the given
options. It is evident that this exibility can be obtained through an if .. then construct
discussed in the previous lectures.

4 Testing the C program

Once we have written and successfully compiled the program, we need to test its perfor-
mance. The common practice is to solve a problem whose solution is known to us using
4
the new program. For instance, in the program ctrule.c mentioned above, we could put in
various functions, such as f (x) = 1; x; x ; :: etc. and check whether the program yields an
2

answer close to the known analytical result. I have used an exponential function which
may be thought of as a polynomial of in nite order.
Well, thinking of good test cases for your programs is Ran art in itself. For instance,
to test the program, we could try to evaluate the integral , xexp(x )dx using ctrule.c
1
1
2

since we know the analytical result is 0 (Why?).

4.1 Testing for Convergence

The concept of convergence is of cardinal importance in numerical analysis. This can be
explained as follows with our current example. If we use any of the equations 7 - 9 to
evaluate the integral, how fast does the numerically computed answer, say Ih, converge to
the exact answer I ? To quantify this, we could de ne di erent measures of convergence.
One of them is the absolute error, EA, de ned by EA = jI , Ihj and we will use this
measure for the purpose of our discussion.
Let's evaluate from the program ctrule.c EA for di erent values of h. If we do this
and plot EA vs. h in a log-log plot (see gure below), we see that the error decreases as h
is decreased and on a log-log plot we get a straight line of slope 2. This error behavior is
typical of quadratically convergent algorithms. It can be shown that both the CM and the
CT rules are quadratically convergent, whereas the CS rule converges as the fourth-power
(quartically, if you like) of the subinterval size h.
Once we have made such observations, we should try to reconcile them with relevant
theoretical analysis. For instance, why does the trapezoidal rule converge as the second-
power of h? A theoretical explanation of this requires the following theorem which arises
primarily from the concept of the Taylor series expansion of a smooth function, and
we simply state the theorem without proof and use it to see how we can explain the
convergence characteristics of the trapezoidal rule.
Polynomial Interpolation Theorem: Let f (x) be a function with n + 1 contin-
uous derivatives on an interval containing distinct points x < x < ::: < xn . If p(x) is
0 1
the unique polynomial of degree n or less such that p(xi ) = f (xi ), then for any value of
x in the interval [x ; xn ], we have
0

f (x) , p(x) =
(x , x )(x , x ):::(x , xn ) f n (z);
0 1 ( +1)
(11)
(n + 1)!
5
 Covergence Comparison: CM, CT and CS Rules.
1
’CT’
’CM’
’CS’
Absolute Error

1e-10

1e-20
0.001 0.01 0.1
h

where f n ( +1)
denotes the (n + 1)th derivative of f and z is some point in the interval
[x ; xn]. 0

Let's apply this theorem for the trapezoidal rule. Here, for each one of the n subin-
tervals, we use linear interpolation functions, so that the error in this approximation in
the ith subinterval is given by x,x ,1 x,x f (z). So the absolute value of the error in
( i
2
)( i) (2)

the integral evaluation from this subinterval is given by (verify this result)
EA;i = j
Z x
(x , xi, )(x , xi ) f (z)dxj
i
1 (2)

x +1 2 i

= (xi ,12xi, ) f (z)

3
1 (2)

h 3
= 12 f (z ): (12) (2)

Now, the if we add up the error contributions from all the n subintervals and replace
f z by its maximum value in [a,b], say M , and utilize the fact that nh = (b , a), we
(2( ))
2
arrive at
M
EA (h)  (b , a)h ; 2
(13)
2

12
which correctly describes the error behavior we saw in the computations, see Figure 6.
Similar analyses show that for the CM rule, EA(h)  M2 (b , a)h and for the CS rule, 24
2

6
EA (h)  2880
M4 (b , a)h4 , where M is the maximum value of the fourth derivative of the
4
integrand function. Try to reconcile the numerical results of Figure 6 with the theoretically
obtained error bounds for the three di erent integration rules.

4.2 Roundo Error

The analysis in the previous subsection addressed the discretization error resulting from
the numerical approximation. Another type of error which we have to guard our programs
against is the Roundo Error, arising purely due to the nite precision arithmetic of the
computer. For instance, the number 0:0 can be represented in the computer as a decimal
which has a nite number of digits after the decimal point as 0. In single precision,
this could be for instance 0.00000000*** where *** denotes an integer over which we
generally have little control. So if the rst non-zero digit is greater than or equal to 5, the
representation of 0 as a real number, correct to eight decimal digits on the computer is
0:00000001, or 10, . In double precision calculations, this representation is more accurate,
8

up to 10, , and in more accurate quadruple precision, up to 10, .

16 32

We see from Eq.13 that the absolute error arising from the numerical discretization
should go to 0 as h ! 0. However, the smaller h is, the larger the number of additions
and function evaluations the computer has to perform and the rounding error incurred
in each evaluation and addition adds up to a signi cant value. For the double precision
calculations we employ, the rounding error in each operation is of the order 10, . Then
16

if we do 10 evaluations/ additions, the total rounding error could be of the order 10, .
8 8

This reasoning is substantiated by the numerical results presented in a gure above. We

can see from there that for h = 10, , the error is greater than 10, .
7 10

7
 Effect of Rounding Error: Composite Trapezoidal Rule
0.01

0.001

0.0001

1e-05

1e-06
Absolute Error

1e-07

1e-08

1e-09

1e-10

1e-11

1e-12

1e-13
1e-07 1e-06 1e-05 0.0001 0.001 0.01 0.1 1
h