Math 563 Lecture Notes

Numerical integration (fundamentals)

Spring 2020

The point: Techniques for computing integrals are derived, using interpolation and piece-
wise constructions (composite formulas). In addition, the asymptotic error series for the
trapezoidal rule is introduced, enabling the use of Richardson extrapolation for integration.

Related reading: 9.1-9.4; 9.6 (there is a detailed proof for Newton-Cotes formulas that is
skipped here to derive the error term).

1 Integrals via interpolation

In this section we derive formulas to approximate a definite integral
Z b
f (x) dx
a

for a continuous function f given its values at a set of nodes, following the same interpolation
strategy employed for differentiation. The approach leads to Newton-Cotes formulas.

It will be useful to recall the mean value theorem in its two forms:

Mean value theorem (MVT), two forms:

• Differential form: Suppose f ∈ C([a, b]) and f 0 exists on (a, b). Then there is a point
ξ ∈ (a, b) such that
f (b) − f (a)
f 0 (c) = .
b−a
• Integral form: If f ∈ C([a, b]) and g is singled-signed and integrable on [a, b] then there
exists a point ξ ∈ (a, b) such that
Z b Z b
f (x)g(x) dx = f (ξ) g(x) dx (MVT)
a a

Important caveat:
R1 Note that
R 1g must be single signed for the integral version to apply.
For instance, −1 x · x dx 6= C −1 x dx for any constant C; the integral of x is zero.

1
1.1 Newton-Cotes formulas: setup
Given nodes x0 , x1 , · · · xn in
Qnthe interval (a, b), construct the interpolating polynomial in
Lagrange form (with ωn = j=0 (x − xj )):

f (x) = pn (x) + E(x)

n
X f (n+1) (ηx )
= f (xi )`i (x) + ωn (x).
i=0
(n + 1)!

Then integrate over [a, b] to obtain the formula and error:

Z b Xn Z b  Z b
f (x) dx = f (xi ) `i (x) dx + E(x) dx.
a i=0 a a

Since pn is a linear combination of the function values f (xi ), the formula has the form
Z b n
X
f (x) dx ≈ ci f (xi )
a i=0

for constants ci . The error has the form

Z b b
f (n+1) (ηx )
Z
E(x) dx = ωn (x) dx,
a a (n + 1)!
which takes some work to simplify into a nicer form.

Change of interval: To simplify the derivation, it is easiest to first ontain formulas for a
fixed interval like [0, 1] and then transform to [a, b] via

s ∈ [0, 1] → x ∈ [a, b], x = a + (b − a)s. (CoI)

The general formula can then be obtained from the fixed one by
Z b Z 1
f (x) dx = (b − a) g(s) ds, g(s) := f (a + (b − a)s)
a 0

Note that g (n+1) (ηs ) transforms into (b − a)n+1 f (n+1) (ξx ) due to the chain rule (for some ξx
in [a, b] depending on x).

1.2 First example: trapezoidal rule

The familiar trapezoidal rule from calculus is a Newton-Cotes formula (see ??).
Consider first integrating g(s) on the interval s ∈ [0, 1] with nodes s0 = 0, s1 = 1.
The Lagrange basis polynomials are

`0 (s) = 1 − s, `1 (s) = s

2
and the interpolation theorem says that
g 00 (ηs )
g(s) = g0 `0 (s) + g1 `1 (s) + x(x − 1).
2
Integrate this expression from 0 to 1 to get
Z 1 1
g 00 (ηs )
Z
g(x) dx = c0 g0 + c1 g1 + s(s − 1) ds (1)
0 0 2
where Z 1 Z 1
1 1
c0 = (1 − s) ds = . c1 = s ds = .
0 2 0 2
Now note that s(s − 1) ≤ 0 in the interval, so the mean value theorem for integrals (MVT)
applies and we may ‘move’ g 00 (ηs ) outside the integral (at the cost of a new arbitrary η):
Z 1 00
g 00 (η) 1
Z
g (ηs )
s(s − 1) ds = s(s − 1) ds
0 2 2 0

for some η in [0, 1]. Plugging the results into (1) gives the ‘trapezoidal rule’ on [0, 1],
Z 1
1 g 00 (η)
g(s) ds = (g(0) + g(1)) −
0 2 12
General case: Now consider an interval [a, b] and let h = b−a (the spacing between points).
Use the transform rule (CoI) to get a general formula,
Z b
h h f 00 (ξ) 3
f (x) dx = f (a) + f (b) − h.
a 2 2 12
Note the extra factors of h on the f 00 term (where do they come from?).

The error is O(h3 ), although the h → 0 limit is not so significant because we are more
interested in a fixed interval [a, b] as more points are added.

1.3 Second example: midpoint rule

Consider g(s) and s ∈ [0, 1] as before but take x0 = 1/2 as the only node; note that the
endpoints are not included. The interpolant is trivial (constant), and we have

g(s) = g(1/2) + g 0 (ηs )(s − 1/2).

Integrate to obtain Z 1 Z 1
1
g(s) ds = g(1/2) + g 0 (ηs )(s − ) ds, (2)
0 0 2
and then use the change of interval to obtain the midpoint rule in [a, b] (with x0 = (a+b)/2)
Z b Z b
f (x) dx = (b − a)f (x0 ) + f 0 (ξx )(x − x0 ) dx.
a a

3
Since x − x0 changes sign in the interval, the MVT cannot be applied. In fact, the error
term in the above is misleading. Observe that if f ∈ P1 then f 0 = const. so
Z 1 Z 1
error = C(s − 1/2) ds = C (s − 1/2) ds = 0.
0 0

Thus, the formula actually has degree of accuracy ≥ 1 (easily checked to be exactly one),
even though the interpolant has degree of accuracy zero.

Simplifying the error: There are two ways around the MVT issue. One is to use Taylor
series instead to derive the formula; just expand g(s) in a Taylor series around the midpoint:
Z 1 Z 1
1
g(s) ds = g(1/2) + g 0 (1/2)(s − 1/2) + g 00 (η)(s − 1/2)2 ds
0 0 2
Z 1
1 1 00
Z
0
= g(1/2) + g (1/2) (s − 1/2) ds + g (ηs )(s − 1/2)2 ds
2 0
|0 {z }
=0

Since (s − 1/2)2 ≥ 0 the mean value theorem applies so

Z 1 Z 1
00 1
g(s) ds = g(1/2) + g (η) (s − 1/2)2 ds = g(1/2) + g 00 (η).
0 0 24

It follows, after transforming to [a, b], that the midpoint rule formula with error is
b
f 00 (ξ) 3
Z
f (x) dx = hf (x0 ) + h
a 24

with x0 = (a + b)/2 and h = b − a. It has the same order/degree as the trapezoidal rule, but
the error has an extra factor of 1/2. Surprisingly, the midpoint rule ttypically has a smaller
error than the trapezoidal rule (same order, better constant), despite using only one point!

4
1.4 General Newton-Cotes formulas
The procedure can be generalized to interpolants of any degree. The two types are:
• Open Newton-cotes formulas, which use n + 1 equally spaced ‘interior’ points
(which excludes the endpoints), xi = a+(i+1)h for i = 0, · · · n with h = (b−a)/(n+1).
• Closed Newton-cotes formulas, which use n + 1 equally spaced points, including
endpoints, xi = a + ih for i = 0, · · · , n with h = (b − a)/(n − 1).

The formulas, using the transformation from s → x to simplify a bit, have the form
Z b n Z b (n+1) n
X
n+2 g (ηs ) Y
f (x) dx = h ci f i + h ωn (s) ds, ωn (s) := (s − si )
a i=0 a (n + 1)! i=0

where g(s) = f (x) and si = i/n and the coefficients are

Z b Z 1 Y n
s − sj
ci = `i (x) dx = ds
a 0 j=0, j6=i si − sj

Now plug in a polynomial of degree n + 1. It is true (check this!) that

Z 1 (
0 for odd n
ωn (s) ds = .
0 6= 0 for even n
One can show that the error must have the usual form, so it follows that for odd n, the
formula has degree exactly n and
Cf (n+1) (ξ) n+2
error = h
(n + 1)!
Here C denotes a constant that depends on n. Note that n + 1 factors of h come from the
ωn term, plus one more from the integral over [a, b].
R1
However, the error vanishes for f ∈ Pn+1 due to 0 ωn (s) ds = 0. The degree turns out
to be n + 1 exactly and instead the error has the form1
Cf (n+2) (ξ) n+3
error = h .
(n + 2)!
In the even case, the symmetry provides an extra factor of h and an extra degree. The key
point is that the formulas with symmetry gain an extra degree of accuracy, so they
are preferable when available (simpson’s rule below; the midpoint rule...)
1
The unpleasant proof involves integrating by parts to manipulate the error into a form where the MVT
applies, then use some technical results to simplify. See Section 9.3 of the textbook for the proof.

5
1.5 Taking the limit
Rb
The Newton-Cotes formulas suggest that one could estimate a f (x) dx to a desired accuracy
by increasing the number of points (n → large and h → small). However, the success of
the approach depends on the interpolant being well-behaved as the number of nodes
increases.

We know from Runge’s example that this is not always the case. In fact, the high-order
Newton-Cotes formulas have some unappealing properties and can be troublesome. One
would work through the details, but the point is that they are rarely used and other meth-
ods are preferred that avoid the issue entirely.

1.6 Simpson’s rule

The first closed formula with symmetry is Simpson’s rule (n = 2), which uses quadratics.
After some work (left as an exercise), one obtains the formula
Z b
h 1
f (x) dx = (f (x0 ) + 4f (x1 ) + f (x2 )) − h5 f (4) (ξ). (3)
a 3 90
Note that since n = 2, the Lagrange error formula gives the error as
Z b (3)
f (ηx )
ω2 (x) dx, ω2 (x) = (x − x0 )(x − x1 )(x − x2 )
a 3!
Rb
but a ω2 (x) dx = 0, so the error vanishes when f ∈ P3 .

Due to a symmetry, we have gained an extra order and degree of accuracy. Simpson’s rule
is a good building block for composite formulas that use piecewise functions (next section).

2 Composite formulas
As with splines, when integrating over an interval it is a good strategy to break it into small
pieces and use low-degree formulas in each. Such a scheme does not require f to be as
smooth as the Newton-Cotes formulas demand.

As a first example, we construct the (composite) trapezoidal rule (usually referred to

as the trapezoidal rule). With n + 1 points x0 , · · · , xn , use the trapezoidal rule in each
interval [xk , xk+1 ]:
Z b n−1 Z
X xk+1
f (x) dx = f (x) dx
a k=0 xk
n−1
X 1
≈ (xk+1 − xk )(f (xk ) + f (xk+1 )).
k=0
2

6
For the error, suppose for simplicity that the points are equally spaced (h = xk+1 − xk ).
Then, including the error term from before in each sub-interval,
Z b n−1
X h
f (x) dx = (f (xk ) + f (xk+1 )) + ET
a k=0
2

where, for some constants ηk ∈ [xk , xk+1 ],

n−1 (2)
X f (ηk )
ET = − h3 .
k=0
12

Also suppose that

|f (2) (x)| ≤ M for x ∈ [a, b].
The terms can all be bounded by M h3 /12, which do not depend on n, so it follows that

nM h3
|ET | ≤ .
12
Now one can write the error in terms of n or h:
M (b − a)3 M (b − a) 2
|ET | ≤ , |ET | ≤ h.
12n2 12
Thus the error is O(1/n2 ) as n → ∞ or alternatively O(h2 ) as h → 0.

Note that the error in each sub-interval is O(h3 ), and there are O(1/h) sub-intervals, so
the total error is O(h3 · 1/h) (a useful rule of a thumb). The total error is proportional
(roughly) to the interval width times the local error.

Precise simplification: The IVT trick use for derivatives can also be used to simplify
the error as well. Write
n−1
!
nh3 X 1 (2) nh3 (2)
ET = − f (ηk ) = − f (ξ)
12 k=0 n 12

since the weights 1/n are positive and sum to 1.

7
2.1 Composite Simpson’s rule
The error can be improved by upgrading the trapezoidal rule to Simpson’s rule (linear
to quadratic). Assume there are an even number of sub-intervals (n is even). Then use
Simpson’s rule (3) on the sub-intervals
[x2k , x2k+2 ], k = 0, · · · n/2 − 1
as shown below (here h = xk+1 − xk is now half the sub-interval width). The result is a
formula with an error
1
ES = − (b − a)h4 f (4) (η).
180
The formula is O(h4 ) if f ∈ C 4 .

Convergence rate: As an example, the convergence is shown for

Z 2
xex dx
0

using the composite trapezoidal rule and composite Simpson’s rule. The integral was com-
puted with spacing h = (b − a)2−n and the error is plotted vs. n. Since the error is O(hp ),
we expect the log-log plot to be linear, with slope −p log 2 (depending on the base used).

Reference lines that are Chp for p = −2 and p = −4 are plotted to show that the error
does indeed have the right slope. This is the ‘generic’ case for both rules when the function
f is smooth enough that the error bound holds (f ∈ C 2 for the trapezoidal rule and f ∈ C 4
for Simpson’s) and has no special properties (so it does not do better).

100
10 2

10 4

10 6
err.

10 8
trap.
10 10 simp.
p=-2
10 12
p=-4
101 102 103
n

8
2.2 Euler-Maclaurin formula
We would like to be able to apply Richardson extrapolation to integration formulas. To do
so, we need an asymptotic error formula. The error for the composite trapezoidal rule, as
it turns out, has such a form, called the Euler-Maclaurin formula.

Theorem: Suppose f ∈ C (2m+2) ([a, b]). Let a = x0 < x1 < · · · < xn = b, equally spaced and
n−1
1 X
Th f = (f (a) + f (b)) + hf (xn ). (4)
2 i=1

i.e. the composite trapezoidal rule where h is the spacing. Then

Z b m
X B2k 2k (2k−1)
(b) − f (2k−1) (a)


Th f ∼ f (x) dx + h f
a k=1
(2k)!

as h → 0, where B2k denotes a Bernoulli number. The error term, precisely, is given by
B2m+2 2m+2 (2m+1)
(b − a) h f (η) for some η ∈ (a, b).
(2m + 2)!

The first few Bernoulli numbers are B2 = 1/6, B4 = −1/30, B6 = 1/42.

That is the equally-spaced trapezoidal rule with spacing h has an asymptotic error series
Z b
Tn f ∼ f (x) dx + c2 h2 + c4 h4 + · · · , c2k = const.k · (f (2k−1) (b) − f (2k−1) (a)).
a
It is important to note that this is an asymptotic but not a convergent series. That is,
while the result holds in the asymptotic-to sense for any finite number of terms, the series
does not converge (in fact, the coefficients B2k grow quite rapidly).

Modified trapezoidal rule: With the first two terms,

Z b
h2
Th f = f (x) dx + (f 0 (b) − f 0 (a)) + O(h4 )
a 12
In particular, if the derivatives of f are known at the endpoints, we can define a ‘modified
(composite) trapezoidal rule’
h2 0
(f (b) − f 0 (a))
Mh f = Th f −
12
which improves the accuracy to O(h4 ) without much more work (if f 0 is easy to compute).
The formula has significant consequences for the error in the trapezoidal rule (see homework).

Preview: What happens if all the odd-order derivatives match? It is not true that the
error is zero (which would not make sense!). The question is better answered later with
Fourier series. The asymptotic series only says that the error decays faster than h2m for any
m (since that term in the series vanishes). There are small terms ‘past the sum’ in the ∼.

9
2.3 Romberg integration (Richardson extrapolation, again)
Define the composite trapezoidal rule Th f as in the previous section (eq. (4)).

Th f = trapezoidal rule applied to f with n + 1 equally spaced points, spacing h.

Rb
and let I = a f (x) dx. Then the Euler-Maclaurin formula says that

Th f = I + c2 h2 + c4 h4 + · · ·

assuming that f is smooth. Let

R0,j = trapezoidal rule with 2j sub-intervals

= Th f with h = (b − a)2−j .

Then by the same procedure as we saw for derivatives,

c2 2 c4 4
Th/2 f = I + h + h + ···
4 16
1
=⇒ (4Th/2 − Th ) = I + b4 h4 + · · ·
3
We then define the next iteration of R’s as this approximation:
1
R1,j = (4R0,j+1 − R0,j ).
3
The process is then continued via Richardson extrapolation (details: homework). Continuing
this process, we obtain
1
Ri,j := i (4i Ri−1,j+1 − Ri−1,j ).
4 −1
This process is called Romberg integration. Essentially, we are taking the trapezoidal
rule and employing Richardson extrapolation to get an accurate approximation.

A small improvement: The process can be optimized a bit by observing that half the
points of Th/2 are also in Th . We have that
n/2−1
X
Th f = 2Th/2 f − hf (a + (2i − 1)h)
i=1

so only half of the sum in the rule needs to be updated.

Rb
Each new column in the table is a formula for a f (x) dx using 2i subintervals; R0,0 is the
trapezoidal rule, R1,0 is Simpson’s rule and so on.

10
2.4 Adaptive integration (and another error estimate)
Suppose we want to design an ‘adaptive’ scheme to estimate
Z b
I= f (x) dx
a

that takes in f , the interval [a, b] and a value  and returns an approximation to I with error
at most (b − a). The idea is that the scheme should be able to assess its own accuracy and
do only work as needed. Essentially, we want to use a method like Romberg integration, but
only split intervals in half when the accuracy is needed.

Let S(a, b) denote Simpson’s rule in [a, b] with h = (b − a)/2:

h
S(a, b) := (f (a) + 4f (a + h) + f (b)).
3
A more accurate approximation can be computed by splitting the interval in half and ap-
plying Simpson’s rule to both halves. Let c = (a + b)/2 and

S1 = S(a, b), S2 = S(a, c) + S(c, b).

Both are approximations to I. We have that

f (4) (ξ1 ) 5
I = S(a, b) − h
90
| {z }
E
(4)
f (ξ2 ) f (4) (ξ3 )
I = S(a, c) + S(c, b) − (h/2)5 − (h/2)5
| 90 {z } | 90 {z }
≈E/32 ≈E/32

We wish to estimate the error E in Simpson’s rule. To do so, assume that the ξ’s are equal
(not true, but close) to get two equations for E and I:
1
I = S1 − E, I ≈ S2 − E.
16
Solving for E, we get
|E| ≈ 15|S2 − S1 |. (5)
The algorithm can then proceed recursively as follows, given , a, b and f :

• Compute the approximate integrals S1 = S(a, b) and S2 = S(a, c) + S(c, b)

• Estimate the error E in S1 using (5).
• If |E| < (b − a), return S2 (since it is likely more accurate than S1 ).
• If |E| > (b − a) then apply the algorithm in [a, c] and [c, b] (recursively).

11
Figure 1: Illustration of an adaptive scheme using Simpson’s rule (blue dots: points used).

Summary; The algorithm recursively splits the interval in half until a sub-interval is ac-
cepted. The result is a a collection of subintervals [aj , bj ] for which
Z bj
error in f (x) dx < (bj − aj )
aj

Once added up, the total error will be bounded by (b − a). The algorithm will use more
points in ’bad’ sub-intervals as it will sub-divide more. This leads to an approximation where
the points used are concentrated more in areas where f has rapid variation.

Note that it is not an absolute guarantee, since our error estimate relied on some rough
simplification; however, if f (4) does not vary too much the error estimate should be good.

Implementation note: It is usually most efficient to avoid writing the algorithm recursively
in practice, instead using a while loop and some careful bookkeeping.

12