Previous Home Next

Chapter 4

NUMERICAL DERIVATIVES AND INTEGRALS

The overall purpose of this chapter is to introduce you to methods that are appropri-
ate for estimating derivatives and integrals numerically. The first, and very import-
ant, topic is that of the discreteness of numerical data, discussed in Section 4.2.
This is related in Section 4.3 to the finite number of significant figures in numerical
computations and to the consequent problem of the increase in errors that may be
caused by subtractive cancellation.
With these precautions understood, we are prepared by Section 4.4 to investi-
gate various schemes for approximating derivatives and to develop a C program to
compute numerical derivatives, a topic that is explored further in Project 4A in Se-
tion 4.5. Numerical integration methods, including derivation of the trapezoid rule
and Simpson’s formula, are developed and tested in Section 4.6 and applied in Pro-
ject 4B (Section 4.7) on the electrostatic potential from a line charge. References
on numerical derivatives and integrals round out the chapter.
We must emphasize here a profound difference between differentiation and inte-
gration of functions. Analytically, for a function specified by a formula its deriva-
tives can almost always be found by using a few direct algorithms, such as those for
differentiation of products of functions and for differentiating a function of a func-
tion. By contrast, the same function may be very difficult (if not impossible) to inte-
grate analytically. This difference is emphasized in Section 3.5 of Wolfram’s book,
where he describes the types of integrals that Mathematica can and cannot do.
Numerically, the situation with respect to differentiation and integration is usual-
ly reversed. As becomes clear in Sections 4.4 and 4.5, the accuracy of numerical
differentiation is sensitive to the algorithms used and to the numerical noise in the
computer, such as the finite computing precision and subtractive cancellation errors
discussed in Section 4.3. On the other hand, almost any function that does not os-
cillate wildly can be integrated numerically in a way that is not extremely sensitive to
numerical noise, either directly or by simple transformations of the variables of inte-
gration.

99
100 NUMERICAL DERIVATIVES AND INTEGRALS

Other methods for differentiating and integrating discretized functions by using

splines are presented in Sections 5.4 and 5.5. The numerical methods developed
for differentiation in this chapter are useful for differential-equation solution in Sec-
tions 7.4 and 7.5 (first-order equations) and in Sections 8.3 - 8.5 (second-order
equations). In Section 10.4 we compute the convolution of a Gaussian distribution
with a Lorentzian distribution (the Voigt profile) by using a combination of analytical
methods and numerical integration.

4.1 THE WORKING FUNCTION AND ITS PROPERTIES

For purposes of comparing various methods, particularly those for numerical deriv-
atives and integrals, it is useful to have a working function and to systematize its an-
alytical properties. A polynomial, which can readily be differentiated and integrated
analytically to provide benchmark values, is more appropriate than the functions that
we investigated by power series in Section 3.2. Further, most of our approxima-
tions are based on the use of low-order polynomials, so by using such a polynomial
we can make direct analytical comparison between approximate and exact values.
The working function will stress the methods greatest if the polynomial oscillates
within the region where the derivatives and integrals are being performed, and such
oscillations can be designed into the polynomial by choosing its roots appropriately.
In this section we first derive some analytical and numerical properties of the
working function, then we develop a C function for efficient numerical evaluation of
polynomials by using Horner’s algorithm, then we include this in a program for the
working function.

Properties of the working function

We define the working function, yw(x), by a product expansion in terms of its

roots, defining it by the expression

(4.1)

which is a sixth-order polynomial having all real roots, and five of these are within
the range -0.5 O.5 over which we will explore the function properties. This
working function, yw is shown in Figure 4.1 for a slightly smaller range than this.
Outside the range of the graph, the function has another zero at x = -0.5 and one at
x = 1.0.
Although the product expansion (4.1) is convenient for controlling the positions
of roots of the working function, and thereby controlling its oscillations, it is not
convenient for analytical differentiation or integration. These can be performed by
expanding (4.1) into powers of x, then differentiating or integrating term by term.
This involves a little algebra and some numerical work, so why don’t you try it?
 4.1 THE WORKING FUNCTION AND ITS PROPERTIES 101

FIGURE 4.1 The working function (4.1), a pathological polynomial having six real roots in
the interval [-0.5, 1].

Exercise 4.1
(a) Show that the polynomial for the working function, (4.1), can be expanded
into

(4.2)

where the coefficients, wk, are given in Table 4.1,

(b) Starting with the expansion (4.2) with the given wk, evaluate the successive
derivatives of yW in terms of the expansion coefficients appearing in the formu-
las

(4.3)

in which for the i th derivative, i = 1, 2, . . . . 6, the coefficients are given in

Table 4.1.
(c) Perform a similar derivation for the integral of the working function yw ( x )
in order to show that its indefinite integral is given by

(4.4)

for which the coefficients Ik are given in the bottom row of Table 4.1. n
102 NUMERICAL DERIVATIVES AND INTEGRALS

TABLE 4.1 Expansion coefficients of working function (4.1) for the value, wk, for derivatives,
wi,k, and for the indefinite integral, Ik. Omitted values are zero.

0 1 2 3 4 6 7
0
6
6

0 0 1

This table, transformed into an array, is used in the program at the end of this sec-
tion. It may also be used for the programs in Sections 4.4, 4.5, and 4.6.
Derivatives of the working function, appropriately scaled, are shown in Fig-
ure 4.2. Notice that the derivatives grow steadily in magnitude, but steadily become
smoother with respect to x as the order of the derivative increases. Clearly, for our
polynomial of sixth order, derivatives past the sixth are zero, as the constancy of
y(6) in Figure 4.2 indicates.

-0.5 0

FIGURE 4.2 The working polynomial, (4.1) or (4.2), and its derivatives. Note that the deriva-
tives become larger but smoother as the order of the derivative increases.

It is useful to have available a program for computing the working function (or
other polynomial) and its derivatives. Since this function, its derivatives, and its in-
tegral are all polynomials, it is practical to have an efficient means of evaluating
them. Horner’s polynomial algorithm, which we now describe and for which we
provide a C function for its implementation, is efficient, accurate, and simple.
 4.1 THE WORKING FUNCTION AND ITS PROPERTIES 103

A C function for Horner’s algorithm

Polynomials occur very frequently in numerical mathematics, for example in any

Taylor expansion approximation to a function, as considered in Section 3.2. It is
therefore worthwhile to find an accurate and efficient way to evaluate polynomials
numerically. Horner’s method is one approach. Suppose that the polynomial to be
evaluated, y ( x ), is of order N in the variable x , so that

(4.5)

We can avoid the inefficient (and sometimes inaccurate) direct evaluation of powers
of x by starting with the highest-order term, then adding in the next-lowest coeffic-
ient plus the current value of the polynomial multiplied by x. For example, suppose
that we have the quadratic polynomial

(4.6)

We may evaluate this by successive stages as

(4.7)

(4.8)

(4.9)

In each of these stages the symbol means “assign the right side to the left side.”
After the last assignment, (4.9), we have completely evaluated the polynomial (4.5)
without computing powers of x explicitly. Clearly, this recursive procedure can be
generalized to any order of polynomial, N. The Homer method provides an exam-
ple of the effective use of repetition in computing, as we discuss in the Diversion in
Section 3.4.
Program 4.1, Horner Polynomials, implements the Horner algorithm for a
polynomial of order N, where N is restricted only by the size of the array, a[MAX ] ,
used to hold the values of the ak, k = 0, 1 , . . . . N. In this program the constant
term in the polynomial, a0, is stored in a[0] , and so on for the other coefficients.
Thus, we have an exception to our practice, stated in Section 1.2, of starting arrays
at element [1]. By breaking this rule, however, we have a direct correspondence
between the formula (4.5) and the coding.
The main program consists mainly of bookkeeping to input and test the order of
the polynomial N, to test its bounds (a negative value produces a graceful exit from
the program), then to input the polynomial coefficients. Once these are chosen by
the program user, the polynomial, y, can be evaluated for as many x values as de-
sired, except that input of x = 0 is used to signal the end of this polynomial.
104 NUMERICAL DERIVATIVES AND INTEGRALS

PROGRAM 4.1 A test program and function for evaluating an Nth-order polynomial by the
Horner algorithm.

#include <stdio.h>
#include <math.h>
#define MAX 10

/* Homer Polynomals */
double a[MAX];
double x,y;
int N,k;
double Horner_Poly();

printf("Horner Polynomials\n");
N = 1;
while(N>=O)
{
printf("\nNew polynomial; Input N (N<0 to end):");
scanf("%i",&N);
if(N<O)
{ printf("\nEnd Homer Polynomials"); exit(O); }
if(N>MAX-1)

printf("\n\n!! N=%i > maximum N=%i",N,MAX-1);

}
else

printf("\nInput a[k], k=O,...,%i (%i values)\n",N,N+l);

for ( k = 0; k <= N; k++ ) scanf("%lf",&a[k]);
x = 1;
while(x!=0)
{
printf("\n\nInput x (x=0 to end this polynomial):");
scanf("%lf",&x);
if ( x .!= 0 )
{
y = Horner_Poly(a,N,x);
printf("y(%lf) = %lf",x,y);
}
} /*end x loop*/

} /*end N loop*/
 4.1 THE WORKING FUNCTION AND ITS PROPERTIES 105

double Horner_Poly(a,N,x)
/* Horner Polynomial Expansion */
double a[],x;
int N;

double poly;
int k;

for ( k = N-l; k >= 0; k-- )

poly = a[k]+poly*x;

return poly;

The Horner polynomial is in function Horner-Poly, which is very compact.

Note that the for loop is not executed if N = 0, because the value of k is check-
ed before any instructions within the loop is executed. The function is therefore cor-
rect even for a zero-order polynomial.

Exercise 4.2
(a) Check the correctness of the Horner polynomial algorithm as coded in func-
tion Horner-Poly in Program 4.1 by using it to check the case N = 2 worked
in formulas (4.7) through (4.9).
(b) code program Horner Polynomials, then test it for a range of N values,
x values, and correctness for some low-order polynomials. n

Note that in the Horner algorithm the order of the polynomial, N, must be
known before the polynomial is evaluated. The algorithm is therefore of limited use
for computing power-series expansions (as in Chapter 3), unless one has decided
beforehand at what order to terminate the expansion.
Program 3.3 in Section 3.2 gives an example of this latter usage for the func-
tions CosPoly and SinPoly. On the other hand, if a convergence criterion (such
as a given fractional change in the series partial sum) is used, then the method of
generating successive terms by recurrence (Sections 3.4, 3.5) is appropriate. Pro-
gram 3.4 in Project 3 (Section 3.5) shows this usage in the functions for the six
power series.
The Horner-poly function should be of general use to you in future program
development. We apply it immediately to program the numerical evaluation of the
working function presented above.
106 NUMERICAL DERIVATIVES AND INTEGRALS

Programming the working function

The working function, described in Section 4.1, is used in Sections 4.4 and 4.5 to
test numerical derivatives and may be used in Section 4.6 for integrals. It would be
possible, but tedious (especially if you use a different polynomial), to evaluate the
numerical values by hand or with help from a pocket calculator. Also, if one wants
the derivatives of various orders or the integral this is also tedious and error-prone in
hand calculations. Here we provide a general-purpose program that, given the order
and coefficients of a polynomial, determines the polynomials that represent its non-
vanishing derivatives. Then the program allows the user to choose specific x values
for evaluating these quantities numerically.
Here we give a preliminary discussion of the program Working Function:
Value and Derivatives (henceforth abbreviated to Working Function). We
defer discussion of the functions for numerical derivatives (function names starting
with Der) to Sections 4.4 and 4.5. Working Function is structured as follows.
The main program outermost loop is over the order of the polynomial, Npoly.
Within this loop are input and testing sections for Npoly, a section for the input of
the polynomial coefficients, a section where derivative coefficients are calculated,
then a loop for choosing x values at which to evaluate the function and its first and
second derivatives.
The polynomials are evaluated using the Horner algorithm and a modified ver-
sion of the C function from the preceding subsection. Function Horner-Poly-2
has a two-dimensional array (rather than one-dimensional, as in Horner-Poly), and
an additional argument, i, to control the first index of the array. For the function
and its derivatives the array elements are the wk in (4.2) as the zeroth elements of the
array, and the Wi,k in (4.3) as the elements for the ith derivative. At the expense of a
slight awkwardness, this array is also two-dimensional, which allows the Horner-
polynomial program function to be used also to evaluate coefficients for the integral.
Note that the labeling of arrays from zero, rather than unity, in the C language is
a great advantage in Working Function. Therefore, to maintain a closeness be-
tween the analysis and the numerics, we break with our general rule (Section 1.2)
of starting arrays at unity (as is common practice among Fortran programmers).

Exercise 4.3
Code and test Working Function. For testing it may be useful to add an out-
put section for the coefficient elements w[i][k] and Int eg [0][k] . Test coef-
ficients for all these are given in Table 4.1 for the working function (4.2),
which is equivalent to (4.1). The latter provides x values at which the function,
but not its derivatives, should evaluate to zero. n
We now have a working function and a program for its numerical properties to
use in examples and tests of the numerical methods in this chapter. After listing the
program, we consider some limitations of numerical computation.
 4.1 THE WORKING FUNCTION AND ITS PROPERTIES 107

PROGRAM 4.2 The working function (4.1) or (4.2): its values and derivatives.

#include <stdio.h>
#include <math.h>
#define MAXDER 7

main ()
{
/* Working Function: Value and Derivatives */
double w[MAXDER][MAXDER];
double x,ywDeriv_i,h;
double ym2.ym1.y0,yp1,yp2,yf,yC,yCCD,y3CD,y5CD;
int Npoly,k,i;
char polyOK,choose_x;
double Horner_Poly_2(),Der_1F(),De_r1C(),
Der_2CCD(),Der_23CD(),Der25CD();

printf(Working Function: Value and Derivatives\n");

Npoly = 1;
while ( Npoly >= 0 )
{
printf("\nNew polynomial;Input order,Npoly (<0 to end): ");
scanf("%i",&Npoly);
if(Npoly<O)
{
printf("\nEnd Working Function Value and Derivatives");
exit (0);
}
polyOK = 'y';
if ( Npoly >= MAXDER )
{
printf("\n !! Order=%i>max_order=%i\n",Npoly,MAXDER-1);
polyOK = 'n';

else

printf("\nInput %i coefficients:\n", Npoly+l);

for ( k = 0; k <= Npoly; k++ )

scanf("%lf",&w[O][k]);
108 NUMERICAL DERIVATIVES AND INTEGRALS

/* Generate all derivative coefficients */

if (Npoly > O)

for ( i = 1; i <= Npoly; i++ ) /* derivative order */

{
for ( k = 0; k <= Npoly-i; k++ )/* polynomial terms */
{
W [i ] [k] = (k+l)*w[i-1][k+l];
}
}
}
}
if ( polyOK == 'y' )

/* Evaluate working function at input x values */

choose_x = 'y';
while ( choose_x == 'y' )

printf("\n\nWant an x ( y or n)? ");

scanf("%s",&choose_x);
if ( choose_x == 'y' )

printf("Input x and stepsize h: ");

scanf("%lf%lf",&x,&h);
for ( i = 0; i <= Npoly; i++ )

ywDeriv_i = Horner_Poly_2(w,Npoly-i,i,x);
if ( i == 0 )
printf("\nPolynomial value is %lf",ywDeriv_i);
else
printf("\ni=%i, derivative is %lf",i,ywDeriv_i);

/* Numerical derivatives; function values first */

ym2 = Horner_Poly_2(w,Npoly,0,x-2*h);
ym1 = Horner_Poly_2(w,Npoly,0,x-h);
y0 = Horner_Poly_2(w,Npoly,0,x);
yp1 = Horner_Poly_2(w,Npoly,0,x+h);
yp2 = Horner_Poly_2 (w,Npoly,0,x+2*h);
/* First derivatives: */
yF = Der_1F(y0,yp1,h); /* Forward */
yC = Der_1C(ym1,yp1,h); /* Central */
/* Second derivatives: */
yCCD = Der_2CCD(ym2,y0,yp2,h);/* Central */
y3CD = Der_23CD(ym1,y0,yp1,h);/* 3-point */
y5CD = Der_25CD(ym2,ym1,y0,yp1,yp2,h);/* 5-point */
 4.1 THE WORKING FUNCTION AND ITS PROPERTIES 109

printf("\nNumerical first derivatives:");

printf("yF = %lf, yC = %lf",yF,yC);
printf("\nNumerical second derivatives:");
printf("\nyCCD=%lf y3CD=%lf y5CD=%lf",yCCD,y3CD,y5CD);
}
} /* end while loop for x */
}
} /* end Npoly loop */
}

double Horner_Poly_2 (a,N,i,x)

/* Homer Polynomial Expansion */
/* Coefficients have two subscripts;first subscript is i */
double a[MAXDER][MAXDER],x;
int N,i;
{
double poly;
int k;
poly = a[i][N];
for ( k = N-l; k >= 0; k-- )
{
poly = a[i] [k]+poly*x;
}
return poly;
}

double Der_1F(y0,yp1,h)
/* Forward-difference first derivative */
double y0,yp1,h;
{
double y1F0;
y1F0 = (yp1-y0)/h;
return y1F0;
}

double Der_1C(ym1,yp1,h)
/* Central-difference first derivative */
double ym1,yp1,h;
{
double y1C0;
y1C0 = (yp1-ym1)/(2*h);
return y1C0;
}

double Der_2CCD(ym2,y0,yp2,h)
110 NUMERICAL, DERIVATIVES AND INTEGRALS

/* Central-difference first derivative */

double ym2,y0,yp2,h;

double y2CCD;
y2CCD = ( (yp2-y0)-(y0-ym2) ) / (4*h*h);
return y2CCD;
}

double Der_23CD(ym1,y0,yp1,h)
/* Three-point central-difference derivative */
double ym1,y0,yp1,h;
{
double y23CD;
y23CD = ( (yp1-y0)+(ym1-y0) ) / (h*h);
return y23CD;
}

double Der_25CD(ym2,ym1,y0,yp1,yp2,h)
/* Five-point central-difference derivative */
double ym2,ym1,y0,yp2,h;
{
double diff1,diff2,y25CD;
diffl = (yp1-y0) + (ym1-y0);
diff2 = (yp2-y0) + (ym2-y0);
y25CD = ( 4*diff1 - diff2/4 )/(3*h*h);
return y25CD;
}

4.2 DISCRETE DATA AND NUMERICAL MATHEMATICS

We wish to emphasize briefly the importance of the discreteness of data and of nu-
merical mathematics. Both topics place fundamental limits on the completeness of
our interpretation of quantitative information.

The discreteness of data

In experimental work and also in numerical calculations, data are always obtained at
discrete values of the independent variables, even though we may believe that the
quantity being measured is a smooth function of the parameters (independent vari-
ables) of the problem. Similarly, in numerical applications of mathematical analysis
the values are always obtained at finite parameter spacing. Such values are often,
quite appropriately, called “numerical data.” Subsequent operations on data, such as
approximations to derivatives and integrals of the presumed smooth function under-
lying them, will always be limited in accuracy by the discreteness of the data.
 4.3 NUMERICAL NOISE IN COMPUTING 111

Numerical mathematics

In this chapter, and in much of this book, we have a different emphasis than in pure
mathematics. The latter emphasizes the existence of mathematical entities, the logical
consistency of results, and the analytic solution and properties of quantities such as
derivatives and integrals. Numerical analysis, such as we are developing, must al-
ways be guided by pure mathematics, especially on questions of the existence and
convergence of quantities that we attempt to evaluate numerically. This viewpoint
explains why in Figure 1.1 we showed mathematical analysis as the foundation
upon which are built numerical methods and computer programming for various sci-
entific applications.
When evaluating a formula numerically, discrete values of the variables are al-
ways required, rather than the continuous range of values often allowed by the form-
ula definition. For practical reasons, such as minimizing computer storage and exe-
cution time, one often characterizes a function by calculating accurately as few val-
ues as possible with large stepsizes for the independent variables. Related properties
of the function, such as values at intermediate values of the variables (interpolation),
slopes (differentiation), and area under the function between two limits (integration),
are then estimated by numerical procedures.
One often refers to mathematics that emphasizes numerical properties as numeri-
cal analysis, and the process of replacing continuous functions by discrete values is
called discretization. They are to be distinguished from two other distinct branches
of mathematics, number theory and discrete mathematics.

4.3 NUMERICAL NOISE IN COMPUTING

In a digital computer, most numbers can be represented to only a finite number of

significant figures, unless they can be expressed exactly in binary (base-2) arithmetic
requiring fewer than the number of bits used for each number. For example, in a
32-bit computer word the largest integer is 232- 1 (allowing for a sign bit), and
there can be at most a precision of 1 part in 232, that is, less than 1 part in 1010. The
decimal fraction 0.5 is exactly represented, since 0.5 = 2-1, so that in base-2 arith-
metic 0.510 = 0.12. On the other hand, however, the precise decimal fraction
0.410 = 0.0110011001...2. which is an imprecise binary fraction.
The term “computer arithmetic,” in which the number of digits is finite, is often
used to make a distinction from formal or exact arithmetic. Note that errors in nu-
merical calculations introduced by computer arithmetic must be clearly distinguished
from random errors introduced in computations by computer hardware (or software)
malfunctions. The type of error that we discuss should be completely reproducible
over time, provided that you execute exactly the same program as previously. (In
the worst possible case of program bugs, this may be difficult to do unless the com-
puter is performing exactly the same from run to run.) Note that the random error
rate in the arithmetic hardware of a computer is typically less than 1 in 1012, which
is less than one error per year of continuous running.
112 NUMERICAL DERIVATIVES AND INTEGRALS

Two main methods of handling the least-significant digits after an arithmetic op-
eration are roundoff and truncation. We now discuss briefly these types of numeri-
cal noise.

Roundoff and truncation errors

When a computer uses roundoff of numbers the least-significant digit is rounded to

the nearest least-but-one significant digit. For example, 0.12345676 rounded to 7
significant figures is 0.1234568, while 0.12345674 rounded to 7 significant figures
is 0.1234567; the maximum error from such roundoff is therefore about 10-7. As
another example, the numbers 1.99999 and 2.00001 are both 2.0000 when rounded
to 5 significant figures.
In truncation, sometimes referred to as “chopping,” the least-significant digit is
dropped. For example, both 0.12345676 and 0.12345674 truncate to 0.1234567.
When converting floating-point variable values to integer-variable values, truncation
is always used by scientific programming languages. Thus, both 3.9999999 and
3.0000001 truncate to integer 3. For this reason, counting should never be pro-
grammed in terms of floating-point variables.
A simple program to determine both the number of significant figures carried in a
computer’s floating-point calculations, and also whether roundoff or truncation is
used, is given as Program 4.3. The program Significant Digits in Float-
ing Point will estimate the number of significant digits to a given base, and will
signal 'y' or 'n' depending on whether the final value was truncated or not. A
nonzero test number close to unity, value, is input, a given floating-point base
(input as base) is chosen, then a while loop is executed until value is unchang-
ed by adding in successively smaller powers. The number of digits is (approxi-
mately, for base 2) the number of significant digits carried.

Exercise 4.4
(a) Explain how the algorithm coded in Significant Digits in Floating
Point works.
(b) Code and run the program on your computer. Does the number of signifi-
cant binary digits and decimal digits agree with what is stated in your computer
manual? Is the floating-point arithmetic done by truncating or by rounding?
Note that you should run several nearly equal values through the test. If any of
them returns truncate = 'n', then your computer does rounding arithmetic.
Explain this remark. ■

For the double-precision variables declared in Program 4.3, my workstation

gave 63 binary bits and did truncating arithmetic. The 63 bits are presumably stored
in two 32-bit words, with one bit allowed for the sign. When using base-10 the
program declared that 19 decimal digits were carried: indeed 263 is just less than
1019, and 264 is just greater than 1019.
 4.3 NUMERICAL NOISE IN COMPUTING 113

PROGRAM 4.3 A program for estimating, for the computer on which it is run, the number of
significant digits to a given base.

#include <stdio.h>
#include <math.h>

main()
{
/* Significant Digits in Floating Point */
double value,power,newvalue;
int base,digits;
char truncate;

printf("Significant Digits in Floating Point\n");

value = 1;
while (value != . 0)

printf("\n\nInput test value near 1 (0 to end):");

scanf("%1f",&value);
if (value == 0)
{
printf("\nEnd Significant Digits in Floating-Point");
exit (0);
}
printf("\nInput floating-point base:");
scanf("%i",&base);
power = 1; digits = 0; newvalue = 0; truncate = 'y';
tile (value != newvalue)

power = power/base;
digits = digits+l;
newvalue = value+power;

if (newvalue > value+power*base) truncate = 'n';

printf("Number of digits is %i",digits-1);
printf("\nValue of truncate is '%c'",truncate);

The same algorithm can be used with a pocket calculator to determine its number of
significant digits and whether it uses roundoff or truncation. The most common
scheme in both computers and calculators is truncation, because this allows simpler
hardware to be used.
Now that you know the number of significant digits in your computer or calcula-
tor, it is interesting to see some consequences of the finite number of digits. This
exercise is probably easiest done on a pocket calculator rather than on a computer.
114 NUMERICAL DERIVATIVES AND INTEGRALS

Exercise 4.5
(a) Demonstrate by examples that finite-precision arithmetic does not satisfy the
associative law of addition: (a + b) + c = a + (b + c). Exhibit cases in which
the signs of a, b, and c differ.
(b) Prove that for addition the ordering that produces the smallest error from
loss of significant figures is to arrange the numbers by increasing order of mag-
nitude, that is, smallest numbers first.
(c) Demonstrate that the identity b(a/b) = a for b 0 is not always satisfied
in numerical arithmetic. For what ranges of a, b, and c are the relative errors lar-
gest in your arithmetic unit? n

Part (b) of this exercise suggests that convergence of power series, as considered in
Section 3.5, is not performed in the most accurate way. The Horner polynomial ex-
pansion, considered later in this section, which considers the highest powers of a
variable first, may often be preferred for numerical accuracy whenever feasible.

Unstable problems and unstable methods

The effects of the finite accuracy of computer arithmetic that we have just considered
arise partly from the use of unstable methods, which are sensitive to the accuracy of
the calculation steps. Examples are subtractive cancellation in calculating variances
or the solutions of quadratic equations, as we investigate in the next subsection.
Such unstable methods should be clearly distinguished from unstable problems, in
which the exact solution is very sensitive to slight variations in the data, indepen-
dently of the accuracy of the arithmetic.
An example of an unstable problem is the solution of the pair of linear equations

(4.10)

which are shown graphically in Figure 4.3 as the heavy solid and solid lines, re-
spectively. These have an exact and unique solution x = 1, y = 1, which is the
intersection point (solid circle) of the two lines in the figure.
The solution of a pair of equations that are very similar to (4.10) namely,

(4.11)

shown dotted in Figure 4.3, is at x = -1, y = 3, the intersection point (dotted

circle) in the figure. If one attempts to solve such an unstable problem numerically,
then the results will be very sensitive to numerical noise. Geometrically, we see
from Figure 4.3 that all three lines have almost the same slope, so that their intersec-
tion points are very sensitive to these slopes.
 4.3 NUMERICAL NOISE IN COMPUTING 115

FIGURE 4.3 Lines that represent (4.10) (heavy solid and solid lines) and (4.11) (heavy solid and
dotted lines). Determination of their intersections is an unstable problem, since the intersection is
at (1, 1) for (4.10) but at (1,3) for (4.11).

From the viewpoint of linear algebra and the numerical solution of matrix in-
verses, the determinant of the two-by-two matrices made from the left-hand sides in
(4.10) and (4.11) is very much less than the quantities on the right-hand sides of
these equations, so that the matrices are ill-conditioned.

Exercise 4.6
(a) Generalize the above example from the equation pair (4.10) or (4.11) to the
Pair.
(4.12)

Eliminate x between this pair of equations to show that

(4.13)

in which the denominator is the determinant from the left-hand side of the pair of
equations. Explain why values of b close to 1 produce y values that are very
sensitive to the value of b.
116 NUMERICAL DERIVATIVES AND INTEGRALS

(b) Check out (4.13) by first choosing a = 2, b = 1.OOOOO1, c = 2.001, to

show that y = 1000 and therefore x = -998. Second, choose a = 2, but
b = 0.999999, c = 2.000001, to show that now y = -1, x = 3, which is a
thousand-fold change in the y solution for a change of only two parts per mil-
lion in the b coefficient.
(c) Show that the problem cannot be alleviated by first solving for x, then using
the first equation in (4.12) to calculate y. First give an algebraic demonstration,
then rotate the book through a right angle and look at Figure 4.3 to devise a geo-
metrical explanation n

In many numerical problems differences in coefficients such as those in Exer-

cise 4.6 (b) may arise from roundoff or truncation errors in steps of the computa-
tion that generate the coefficients. Because there are so many numbers flying around
in the computer, we seldom inspect the intermediate results to check on this numeri-
cal malfunction, unlike the situation when most of the calculations are done using a
pocket calculator with manual input and output for many of the steps.
Therefore, numerical methods developed in the olden days of hand-cranked cal-
culators, when numerical errors could easily be monitored and sometimes con-
trolled, are often quite unsuitable for computer applications. Nowadays, the user of
a numerical recipe might not even know how the algorithm that is being used was
coded, and whether the method used has been tested for numerical stability. This
distinction between the suitability of numerical methods for hand calculation versus
computer calculation is discussed further in Section 3.6 in the diversion on comput-
ers and spline methods.
The difficulties with unstable problems and with unstable methods are related to
instability of equilibrium and to chaos in mechanical systems, as well as to noise am-
plification and feedback in electrical systems. These topics are lucidly discussed in
Chapter 9 of Pippard’s book on the physics of vibration, and are developed in more
detail in his book on response and stability. Unstable problems and unstable meth-
ods also occur when solving differential equations numerically, often because of ex-
treme sensitivity to boundary conditions. This is discussed in Sections 8.4 and 8.6
for the second-order Euler method and for stiff differential equations, respectively.
Such mechanical, electrical, and numerical instabilities are intimately related.

Errors from subtractive cancellation

Among other problems facing those who would compute numerically are errors aris-
ing from subtractive cancellation, which is the reduction in the number of significant
figures that may occur when two numbers (assuming they have the same signs) are
subtracted. For example, if x1 = 1000000 and x2 = 999999, then their differ-
ence x1- x 2 = 1, has only one significant figure, whereas x1 and x2 had about six
significant figures. We now examine two examples in which such perils of subtrac-
tive-cancellation errors may arise, variance calculations and the solution of quadratic
equations.
 4.3 NUMERICAL NOISE IN COMPUTING 117

Variance calculations (and thence standard deviations) among large numbers pro-
vide good examples of difficulties from subtractive cancellation. Suppose that the
average of a set of x values is calculated as

(4.14)

The variance among this set, V, is defined by

(4.15)

Analytically, this can be expanded to produce

(4.16)

Exercise 4.7
Verify the algebraic equivalence of these two expressions for the variance, V. n

Notice that in (4.15) all N of the xj values must be input and their average must be
calculated before their variance can be calculated, whereas in (4.16) the sum of
squares and the sum of values (needed to compute the average) may be accumulated
as soon as each x value is entered. If (4.15) is used for large N, more storage than
such devices as pocket calculators have available for data will often be required. So,
for calculators the second method is almost always used, because it requires storage
only for the running sums. Numerically, the outcome of using (4.16) can be com-
plete loss of significance, as you will discover if you work the following exercise.

Exercise 4.8
(a) Use a pocket calculator to calculate the average and variance for the three
data values x1 = 999999, x2 = 1000000, x3 = 1000001. Show that using
(4.16) you should get V = 2. What result does your calculator get, and why?
(b) Reduce the order of magnitude of the three data by successive powers of 10
until the two formulas agree. From this numerical experiment estimate the num-
ber of significant figures carried by your calculator. Does this agree with its in-
struction manual? n

By using my pocket calculator and the data in Exercise 4.8 (a), I obtained a vari-
ance of zero according to (4.16). With values of 99999, 100000, and 100001,
however, I obtained the correct variance by using either (4.15) or (4.16). My calcu-
lator manual advertises 10 digits for entry or display and 12 digits for each step of a
calculation: I now believe it, but I don’t like the subtractive cancellation effects. An
extensive discussion of roundoff errors in computers is given in Chapter 1 of the
textbook by Maron.
118 NUMERICAL DERIVATIVES AND INTEGRALS

Quadratic equations provide another example where subtractive cancellation can

be a severe problem with a numerical calculation as simple as finding their roots ac-
curately. Write the quadratic equation with two real roots as

(4.17)

The choice of unity for the first root, x = x 1 = 1, can always be achieved by ap-
propriate scaling of x, and is convenient when comparing the second root,
x = x2 = with the first, especially when is small in magnitude. By expanding
(4.17) into standard quadratic-equation form, we have

(4.18)

The usual way that this is solved is to write the solution as

(4.19)

Since we know the roots, we could simplify this analytically and obtain the two so-
lutions given above. Proceeding numerically, however, is quite another story. I us-
ed my pocket calculator to input then calculate the two numerical solutions in
(4.19), xn1 and xn2, as written. This calculator enters and displays 10 decimal digits
but it uses 12 digits at each step of a calculation. For values of between 10-l1 and
10-5 (and presumably for larger values) the first root, xn1, was exact, but the rela-
tive error in the second root, xn2, behaved quite erratically, as Table 4.2 shows.

TABLE 4.2 Relative errors in root of (4.18) calculated using (4.19) and a 10-digit calculator.

This example illustrates that when is less than the square root of the accuracy of
the calculator < 10-5 for a 10-digit calculator), then the smaller root cannot be
reliably computed, because the squared factor under the root in (4.19) is not
calculated accurately enough. Significant digits are then lost when the root is sub-
tracted from the first term in the denominator of (4.19).
When solving quadratic equations, how can we reduce this numerical noise from
subtractive cancellation? The easiest way is to avoid subtracting. To show how this
can be accomplished, and to cast the quadratic-equation problem in a more conven-
tional form, let us write

(4.20)
 4.3 NUMERICAL NOISE IN COMPUTING 119

The general analytical solution of this quadratic when a 0 is

(4.2 1)

Exercise 4.9
(a) Multiply numerator and denominator of (4.21) by factor in
order to show that the roots of a quadratic are equivalently given by

(4.22)

in which the denominator is assumed to be nonzero.

(b) In order to make the derivation of (4.22) less ad hoc, consider the following
transformation of the quadratic equation (4.20). Assume that the roots are non-
zero (c 0), so that (4.20) can be divided throughout by x 2. You now have a
quadratic equation in the variable l/x with the roles of a and c interchanged.
Write down the solution for this quadratic by using (4.21), then take its recipro-
cal. Viola! the solution (4.22) is obtained. n

Between the use of the two equations (4.21) and (4.22) for the roots, we can avoid
subtractive cancellation completely. For, if b > 0 (4.22) may be used with the +
sign and (4.21) may be used with the - sign. If b < 0 the opposite choice of signs
is appropriate.

Program for roots of quadratic equations

The program that we now provide uses the algorithm devised in Exercise 4.9. Our
Program 4.4, Quadratic Equation Roots, computes the roots of quadratic
equations (4.20) for which the coefficients a, b, and c are each real numbers.
Program Quadratic Equation Roots has a small driver program to control
program execution, and function quadroots to solve for the roots. The main pro-
gram reads a, b, and c; then if any one of them is nonzero it attempts a solution.
Most of the function quadroots is occupied with taking care of the case a = 0,
including the subcase that gives rise to inconsistent solutions, namely b = 0. In
this subcase if the coefficients of the powers of x in (4.20) are both zero, then the
roots are indeterminate if c = 0 and the equation is inconsistent if c 0.
If a is nonzero, then subtractive cancellation can be a problem only if the square
root is real, since there is no subtraction when the real and imaginary parts of a com-
plex number have opposite signs, as discussed in Section 2.1. Therefore, a test for
a positive discriminant is made before deciding whether to use formulas (4.21) and
(4.22) for the case of real roots. For complex roots the usual form of the solution,
(4.2 l), is used for the two roots, which are related through complex conjugation.
120 NUMERICAL DERIVATIVES AND INTEGRALS

PROGRAM 4.4 Quadratic equation roots avoiding subtractive cancellation.

#include <stdio.h>
#include <math.h>

main()

/* Quadratic Equation Roots */

double a,b,c,ReXp,ImXp,ReXm,ImXm;
int error;
void quadroots();

printf("Quadratic Equation Roots\n");

a = 1;
while ( a != 0 || b != 0 || c!= 0 )

printf("\nInput a,b,c (all zero to end) :\n");

scanf("%lf%lf%lf",&a,&b,&c);
if ( a == 0 && b == 0 && c == 0 )
{
printf("\nEnd Quadratic Equation Roots"); exit(O);
}
quadroots(a,b,c,%ReXp,&ImXp,&ReXm,&ImXm,&error);
printf("\nRoots are:");
printf("(%lf)+i(%lf) & (%lf)+i(%lf)",ReXp,ImXp,ReXm,ImXm);
printf("\nError flag = %i",error);

void quadroots(a,b,c,ReXp,ImXp,ReXm,ImXm,error)
/* Quadratic equation roots; Real and Imginary */
double a,b,c,*ReXp,*ImXp,*ReXm,*ImXm;
int *error;
{
double Disc;

*ReXp= *ImXp = *ReXm = *ImXm = 0;

*error = 0; /* no problems SO Far */
if ( a == 0 )
{
if (b == 0 )

if ( c != 0 ) *error = 2; /* inconsistent equation */

else *error = 1; /* roots indeterminate */
 4.3 NUMERICAL NOISE IN COMPUTING 121

else /* a is zero, b is not zero */

{
*ReXp = -c/b; *ReXm = *ReXp; /* degenerate */
}
}1
else /* a is not zero */

Disc = b*b-4*a*c; /* discriminant */

if ( Disc >= 0 ) /* real roots */
{
if (b>= 0)
{
*ReXp = -2*c/(b+sqrt(Disc));
*ReXm = (-b-sqrt(Disc))/(2*a);

else /* b is negative */

*ReXp = (-bsqrt (Disc))/(2*a);

*ReXm = -2*c/ (b-sqrt(Disc));

else /* complex roots */

&ReXp = -b/(2*a); *ReXm = *ReXp;

*ImXp = sqrt(-Disc)/(2*a); *ImXm = *ImXp;

After function quadroots has been executed, the two complex numbers
ReXp + i ImXp and ReXm + i ImXm are returned, together with the integer vari-
able error, which is 0 for no problem, 1 for indeterminate roots, and 2 for an in-
consistent equation. This value is output with the roots before the program requests
another set of coefficients. If all three coefficients are input as zero, then execution
of Quadratic Equation Roots terminates.

Exercise 4.10
(a) Write down the solution of (4.20) when a = 0, and also when both a and b
are zero. Check that these solutions agree with what is coded in the function
quadroots in the program Quadratic Equation Roots.
(b) Test the program with some simple (say integer) values of a, b, and c, in-
cluding the special cases of a = 0, as well as when both a and b are zero.
122 NUMERICAL DERIVATIVES AND INTEGRALS

(c) Verify that subtractive cancellation is not a problem with the present algor-
ithm. To do this, choose in Quadratic Equation Roots values of a = 1,
b = -(1 + c = where << 1, as in the examples in Table 4.2. n

Now that we have explored subtractive cancellation with the practical examples
of variance calculations and of solving quadratic equations, we should be prepared
to recognize the possibilities for subtractive-cancellation effects as a source of noise
in many numerical computations. These effects can be especially troublesome in the
numerical approximation of derivatives (Sections 4.4, 4.5) and in the related topic
of numerical solution of differential equations (Sections 7.4, 7.5, and 8.3 - 8.6).

4.4 HOW TO APPROXIMATE DERIVATIVES

For computing derivatives numerically it is important to find stable methods that are
insensitive to errors from subtractive cancellation. The possibilities of such errors
are implicit in the definition of derivatives. For the first derivative of y at the point
x, y(l), one has the definition

(4.23)

From this definition we see that numerical evaluation of derivatives is inherently an

unstable problem, as defined in Section 4.3. Our challenge is to find a method that
is insensitive to such instability.
We introduce an abbreviated notation in which the ith derivative evaluated n
steps (each of equal size h) away from the point x is written as

(4.24)

If i = 0, then we are dealing with y itself, so the superscript is usually omitted. In

the notation in (4.24) it is understood that both x and h are fixed during a given cal-
culation. The stepsize h is assumed to be positive, whereas the integer n may be of
either sign. The function y is assumed to be well-behaved, in the sense that for
whatever order of derivative we wish to estimate numerically there exists a corre-
sponding analytical derivative at that x value.
The general formulas we derive are polynomial approximations, and they are
often derived from the viewpoint of fitting a polynomial through a given number of
points (N + 1, say), then finding an analytical expression for the polynomial deriv-
atives. There is no direct way of estimating the error incurred in the various deriva-
tives in such a viewpoint. On the other hand, if we consider that we are making a
Taylor expansion of a function in the neighborhood of some point and that we are
 4.4 HOW TO APPROXIMATE DERIVATIVES 123

truncating the expansion after N + 1 terms, then we have an N th-order polynomial

approximation. Additionally, an estimate of the remainder term (3.7) serves to esti-
mate the error incurred. Here we use the first neglected Taylor term as an estimate
of the remainder, and therefore as an approximation to the error.
For our working function, yw (x), discussed in Section 4.1, the analytical deriv-
atives are obtained from the expansion coefficients in Table 4.1. If these derivatives
are evaluated at x = 0, they are just the coefficients in the k = 0 column. It is also
useful to record the derivatives divided by the factorial of the same order, since we
need these for our improved polynomial approximation below. These modified
derivatives are just the expansion coefficients wk with k = i from the first row of
Table 4.1, but we repeat them for convenience. Table 4.3 shows the coefficients
that are needed.

TABLE 4.3 Expansion coefficients for derivatives of the working function, evaluated at x = 0.

Now that we have notation established, and a working function with its check
values well understood, we investigate two simple schemes for approximating deriv-
atives numerically. We also discuss some programming aspects of the derivative
functions (those starting with Der) in Program 4.1.

Forward-difference derivatives

The most obvious estimate of the first derivative of y is just to consider the quantity
in the brackets on the right-hand side of (4.23), with h small but finite. Graphically,
we use the slope of the line segment FF in Figure 4.4. The function shown is ac-
tually our working function (4.1) and we have chosen x = 0 and h = 0.1. Thus
we have the forward-difference derivative estimate, given by

(4.25)

Note that the estimate clearly depends upon the value of h, but we hope that this de-
pendence is not too strong. We can check this out for the working function.
124 NUMERICAL DERIVATIVES AND INTEGRALS

FIGURE 4.4 Schemes for estimating first derivatives are shown for the working function, (4.1)
and Figures 4.1 and 4.2, near x = 0. The forward-difference derivative is taken along FF. The
central-difference derivative is taken along CC.

Exercise 4.11
(a) Use the working function (4.2), a more convenient form than (4.1) for this
purpose, to calculate its forward-difference derivative defined in (4.25) at x = 0
for the values of h in Table 4.4. If you have coded up and tested Program 4.2,
Working Function, you can get the forward-difference derivative estimate as
variable yF from the function Der_1F.
(b) Write down the first few terms of the Taylor expansion of any function
y (X + nh) in terms of the function y (x) and its derivatives at x in order to show
that the first additional term beyond the first derivative itself in the forward-dif-
ference derivative estimate (4.25) is y( 2 ) (x) h /2.
(c) Show that this neglected term is a fair estimate of the error in the numerical
derivatives in Table 4.4, especially for the smallest h value. From Table 4.3 or
the program Working Function, the analytical derivative at x = 0 has the
value -1. n

TABLE 4.4 Forward-difference first derivatives for the working function, evaluated at x = 0.
The exact first derivative has value -1.

From Exercise 4.11 and from Figure 4.4 we can see why the forward-differ-
ence derivative estimate is inaccurate for the h values that we used. The second de-
rivative is of the opposite sign to the first derivative, so its inclusion in the forward-
 4.4 HOW TO APPROXIMATE DERIVATIVES 125

difference estimate makes the magnitude of the slope too small. It is therefore ap-
propriate to make an estimate for the first derivative that is less sensitive to the pres-
ence of second derivatives.

Derivatives by central differences

In choosing a numerical estimator of the first derivative in the preceding subsection,

we could as well have used a backward difference. We would have run into similar
problems as for the forward difference. So why not make a more balanced ap-
proach, using central differences, as follows:

(4.26)

This scheme is shown graphically in Figure 4.4 by the line segment CC. It is
straightforward to extend the method of analysis made for the forward-derivative es-
timate.

Exercise 4.12
(a) Use the working function (4.2) to calculate its central-difference derivative
estimate defined by (4.26) at x = 0 for the values of h in Table 4.5. If you
have Program 4.2 (Working Function) running, you can get the central-dif-
ference derivative estimate as variable yC from the function Der_1C.
(b) Write down the first few terms of the Taylor expansion of a function
y (x + nh) in terms of the function y (x) and its derivatives at x. Thus show
that the first term beyond the first derivative itself in the central-difference deriva-
tive estimate (4.26) is y(3) (x) h2/6.
(c) Show that this neglected term is a very good estimate of the error in the nu-
merical derivatives in Table 4.5, especially for the smallest h value.
(d) Use the program Working Function to show that if one wants accuracy
of about 6 significant figures for the first derivative at x = 0, then the stepsize
for the forward-difference derivative must be about h = 10-6, whereas the same
accuracy in the central-difference derivative requires a stepsize of only about
h = 10-4. n

TABLE 4.5 Central-difference first derivatives for the working function, evaluated at x = 0.
The exact first derivative is -1.
126 NUMERICAL DERIVATIVES AND INTEGRALS

From these examples, especially from a comparison of Tables 4.4 and 4.5, we
see that the central-difference derivative estimate should be used whenever feasible.
It is less likely to run into problems from subtractive cancellation effects than is the
forward-difference method because its stepsize can be significantly larger for the
same accuracy. Occasionally, only the forward-difference (or backward-difference)
method can be used, as when the function is not defined for x values less than (or
greater than) the value at which the derivative is required. This may occur in starting
the numerical solution of differential equations or at the endpoints of runs of data
whose slopes we are trying to estimate.
In the remainder of this treatment of numerical derivatives we use only central-
difference estimates because of their superior accuracy.

Numerical second derivatives

The numerical estimation of second derivatives can be based on the analytical defini-
tion

(4.27)

Immediately we see, from the preceding discussion of first derivatives, that the nu-
merical estimation problems may be much more severe for second derivatives than
for first derivatives, especially if we take differences of numerical first derivatives
rather than doing as much of the calculation as possible by analytical methods.
One possibility is suggested by looking at Figure 4.5. Here we could estimate a
first central derivative at x + h (in the figure at x = 0.1 by the slope of C+C+),
then a similar derivative at x - h (at x = -0.1 by the slope of line C-C-).

FIGURE 4.5 Schemes for numerical second derivatives, illustrated for the working function
(4.1) and Figure 4.1 near x = 0. Consecutive central derivatives (CCD) are obtained from central-
difference first derivatives along C+C+ and C-C-. The 3CD estimate (4.35) and the 5CD estimate
(4.37) use the central 3 points and all 5 points, respectively.
 4.4 HOW TO APPROXIMATE DERIVATIVES 127

This method, consecutive central derivatives, or CCD, can be analyzed algebra-

ically, as follows. First, write the formula (4.26) for central derivatives of the func-
tion which is already the first derivative, thus

(4.28)

Next, use the central-difference estimates for the first derivative, (4.26), in this ex-
pression, to obtain

(4.29)

For optimum accuracy, minimizing subtractive cancellation, it is better to compute

the second-derivative estimate this way rather than from further algebraic simplifica-
tion of the expression.

Exercise 4.13
(a) For the working function (4.2) calculate its central-difference derivative esti-
mate (4.29) at x = 0 for the values of h in Table 4.6. In Program 4.2, Work-
ing Function, this second-derivative estimate is variable yCCD from the func-
tion Der_2CCD.
(b) Write down the first few terms of the Taylor expansion of a function
y (x + nh) with n = -2, 0, and 2, in terms of the function y (x) and its deriva-
tives at x. Thus show that the first additional term beyond the second derivative
itself in the central-difference derivative estimate (4.29) is y(4) (x) h2 / 2 .
(c) Show that this neglected term is only a fair estimate of the error in the nu-
merical derivatives in Table 4.6, except for the smallest h value. (You can ob-
tain the value of the fourth derivative at x = 0 from Table 4.3.) For this ex-
ample and values of h, why is the estimate of the error inaccurate?
(d) Use the program Working Function (or a program of your own devising)
to show that if one wants an accuracy of about 6 significant figures in the second
derivative at x = 0, then the stepsize required is about h = 10-4. Show that
this agrees with the error estimate derived in (b). n

TABLE 4.6 Central-difference second derivatives for the working function, evaluated at x = 0.
The exact second derivative is 6.
128 NUMERICAL DERIVATIVES AND INTEGRALS

Compare Table 4.6 for the second derivative with Table 4.5 for the central-dif-
ference first derivative, to note that about the same value of h is required for the
same accuracy in first and second derivatives. This is not a usual behavior. By in-
specting the derivative values in Table 4.3 you can see that, although the increasing
powers of h (which is < 1) bring down the error estimates, the steadily increasing
derivatives increase them. Our choice of the wiggly sixth-order polynomial, defined
by (4.1) and shown in Figures 4.1 and 4.2, is responsible for this pathological be-
havior. By contrast, the derivatives at x = 0 for the exponential function are all
equal to unity, and the convergence is therefore much more rapid, as we discover in
Section 4.5.

Better algorithms for second derivatives

The algorithm for numerical second derivatives, (4.29), can be significantly improv-
ed without increased complexity, as we now show. The method that we use also
shows how optimized formulas can be produced and how their errors can be esti-
mated from Taylor series. We derive two formulas, one using three points and the
other using five points, all having the x value of interest as the central value.
Suppose that we go n steps, each of size h, to the left or right of the point x at
which we want a numerical estimate of the second derivative. Let us use Taylor’s
theorem (3.6) for up to five points surrounding x, as in Figure 4.5. We then have

(4.30)

(4.3 1)

where the coefficients

(4.32)

contain the desired (and undesired) derivatives. For second derivatives, we need to
eliminate the term v1 and as many higher derivatives as possible. All odd-order de-
rivatives (all vi with i odd) can be eliminated by summing values that are symmetric
about the midpoint. Thus

(4.33)

(4.34)

From these two formulas, depending on how many points are available, we obtain
various estimates for the second derivative that is hiding in v2 .
 4.4 HOW TO APPROXIMATE DERIVATIVES 129

Exercise 4.14
By using (4.33) show that an estimate of the second derivative that uses three
points is y 3CD, given by

(4.35)

with an error, 3CD, estimated as

(4.36)

which is the first omitted term of the Taylor expansion. n

Notice that this formula is predicted to be a factor of 6 more accurate than our CCD
formula (4.29).
If we may use all five points centered on x, as in Figure 4.5, then the estimate
of the second derivative can be significantly improved.

Exercise 4.15
Use both (4.33) and (4.34) to eliminate the fourth-derivative term between them,
and thence to show that an estimate of the second derivative that uses five points
is y5CD, given by

(4.37)

with an error, 5CD, estimated as

(4.38)

which is the first omitted term of the Taylor expansion. n

Formula (4.37) predicts many improvements over the two other estimates of the sec-
ond derivative, in that the error depends upon the sixth derivative, which is usually
smaller than the fourth derivative (except for our pathological working function!),
and scales as h 4 rather than h,2 so it will become relatively much more accurate as h
is decreased.
Now that we have done much intricate algebra, let us try out these second-
derivative formulas on our working function. The example is again for the deriva-
tive at x = 0.
130 NUMERICAL DERIVATIVES AND INTEGRALS

Exercise 4.16
(a) For the working function (4.1) calculate its central-difference derivative es-
timate (4.29) at x = 0 for the values of h in Table 4.7. In Program 4.2, Work-
ing Function, this second-derivative estimate is variable yCCD from the func-
tion Der_2CCD.
(b) Show that if you want the second derivative of the working function (4.1) at
x = 0 accurate to about 6 significant figures, then you have to choose
h 2 x 1O-4 for the three-point second derivative, but only h 1 x 10-2 for
the five-point derivative. n

TABLE 4.7 Central-difference second derivatives and error estimates for the working function
evaluated at x = 0. The exact second derivative is 6.

We now have considerable experience with estimating first and second deriva-
tives numerically for the very badly behaved working function (Figures 4.1 and
4.2). In the next section we explore the derivatives of some functions that are better
behaved.

4.5 PROJECT 4A: COMPUTING DERIVATIVES NUMERICALLY

Although in Section 4.4 we acquired considerable experience with numerical first

and second derivatives of the badly behaved working function (Section 4.1), it is
now interesting to explore the numerics with functions that are more regular. In this
project we explore two common functions that are paradigms for many other func-
tions and data, the exponential function and the cosine function. These characterize
steadily increasing (or decreasing, depending on your point of view) and oscillatory
functions, respectively.

Derivatives of the exponential function

The exponential function y = ex is straightforward to work with, especially at

the origin x = 0, because there its derivatives have value unity.
 4.5 COMPUTING DERIVATIVES NUMERICALLY 131

Exercise 4.17
(a) Modify the program Working Function by replacing the sections for
polynomial evaluation (including the Horner-polynomial function) by the expo-
nential function, y = eX. Keep the five functions for estimating derivatives.
(Since all the derivatives of the exponential are just the function itself, a single
use of the function exp (x) will produce all the derivatives you need.) Add
program statements to compute the difference between these derivatives and
those obtained from the five formulas for first and second derivatives that are al-
ready programmed. Test this new program, called Exponential Function
Numerical Derivatives.
(b) Run this program with x = 0 and successively smaller values of stepsize h,
say h = 0.2, 0.1, 0.01, 0.001, calculating all five numerical derivatives and
their errors for each h. Make a log-log plot of the absolute value of the error
versus h, as in Figure 4.6.
(c) To the extent that the graph for the error in each method versus stepsize h is
a straight line on a log-log plot, argue that the error has a power-law dependence
on stepsize h, and that the overall scale of the error estimates the factor preceding
the power of h. Do the powers estimated for the five derivative methods from
your graph agree with those predicted from our Taylor series analyses? n

FIGURE 4.6 Errors in derivatives of the exponential function at x = 0 as a function of step-

size h for various derivative estimators. First-derivative estimators are forward (F) from (4.25) and
central (C) from (4.26). Second-derivative estimators are consecutive central derivatives (CCD)
from (4.29), three-point derivatives (3CD) from (4.35), and five-point derivatives (5CD) from
(4.37).
132 NUMERICAL DERIVATIVES AND INTEGRALS

Now that you have a working program for estimating numerical derivatives, by
replacing the exponential function in Exponential Function Numerical Deriv-
atives by the function of your choice, you can explore its application to various
other functions, such as the cosine function in the next subsection. Also the pro-
gram may be used directly with data (if they are equally spaced) to estimate slopes
and higher-order derivatives.

Differentiating the cosine function

The cosine function, y = cos X, is interesting to investigate because it characterizes

oscillatory behavior, such as we obtain from the numerical solution of differential
equations in Chapters 7 and 8, as well as from the Fourier expansions in Chap-
ters 9 and 10. Indeed, an understanding of numerical derivatives provides an es-
sential foundation for developing numerical solutions of differential equations.

FIGURE 4.7 Errors in derivatives of the cosine function at x = 0 as a function of stepsize h

for the first-derivative forward (F) from (4.25), and for second derivatives - consecutive central
derivatives (CCD) from (4.29) three-point derivatives (3CD) from (4.35) and five-point derivatives
(5CD) from (4.37). The errors in the central-difference first-derivative estimate (4.26) are zero.

The procedure for the second part of Project 4A is similar to that in the first part,
and it can be done almost effortlessly.

Exercise 4.18
(a) Modify either the program Working Function or (more readily) the pro-
gram Exponential Function by writing sections for evaluating the cosine
function, y = cos x, for its value, for its first derivative -sin x, and for its sec-
ond derivative -cos x. Keep the five functions for estimating derivatives. In-
clude program statements to compute the difference between these derivatives
and those obtained from the five formulas for first and second derivatives.that
are already programmed. Test this new program, called Cosine Function
Numerical Derivatives, for example against hand calculations.
 4.6 NUMERICAL INTEGRATION METHODS 133

(b) Run the program with x = 0 and successively smaller values of stepsize h,
say h = 0.2, 0.1, 0.01, 0.001, calculating all five numerical derivatives and
their errors for each h. Except for the central-difference first derivative (which
has zero error at x = 0), make a table and a log-log plot of the absolute value of
the error against h, as in Table 4.8 and Figure 4.7.
(c) Assuming that the graph for each method on a log-log plot is a straight line,
show that the error has a power-law dependence on stepsize h, and that the over-
all scale of the error estimates the factor preceding the power of h. How well do
the powers estimated for the five derivative methods from your graph agree with
those predicted from our Taylor-series analyses? n

TABLE 4.8 Central-difference first and second derivatives errors for the cosine function evaluated
at x = 0. The exact second derivative is -1.

Notice in Figure 4.7 that the most accurate numerical method, the five-point central-
difference second derivative, does not continue to improve as rapidly as predicted
when h becomes of order 10-3. For this small an h value the error in the numerical
approximation approaches my computer’s roundoff error of about 10-14.
Now that you have experience in numerical differentiating with the badly behav-
ed working function, with a monotonic exponential, and with the oscillatory cosine,
you should be confident to tackle many functions and the differentiation of numerical
data.
There are several other practical methods for numerical estimating derivatives.
For example, cubic-spline fitting through a set of points will also produce, as part of
the process, first through third derivatives, as described in Section 5.4. Another
very effective method is to consider the derivatives as functions of the stepsize h,
then to extrapolate the derivative values at finite h to the limit h 0. This method,
called Richardson extrapolation, is described in Section 7.2 of Maron’s book.

4.6 NUMERICAL INTEGRATION METHODS

We remarked at the beginning of this chapter that integrals of functions are much
less likely to be able to be calculated analytically than are derivatives, but that integ-
ration is usually a more stable problem (in the sense discussed in Section 4.3) than
134 NUMERICAL DERIVATIVES AND INTEGRALS

is differentiation. We therefore seek efficient numerical-integration methods because

we will probably have to do many such integrals (absent analytical results), but a
fairly simple numerical algorithm may perform surprisingly well.
The differentiation-integration question can be approached from another view-
point, namely that the integral of a function is formally the “antiderivative” of that
function. Since we saw in Section 4.4 that successively higher-order derivatives
become numerically more difficult, therefore (at least formally) successive integrals
should become numerically less difficult. As we discussed for derivatives at the be-
ginning of Section 4.4, we also base our integration formulas on Taylor expansions
of the integrand. If we truncate the expansion to a given order, N, we are using an
Nth-order polynomial approximation. The first neglected term in the Taylor expan-
sion serves to estimate the error in the polynomial approximation.
The working function, introduced at the beginning of this chapter as (4.1), is
used for tests and for comparisons of the methods we develop. Recall that this func-
tion was designed as a highly oscillatory but bounded function near x = 0, as
shown in Figures 4.1 and 4.2. The expansion of its integral is given by (4.4), us-
ing the coefficients Ik in Table 4.1. The coefficients of the derivatives of various
orders, also in this table, are useful for estimating errors in integral approximations.
We derive the two most common integration formulas, the trapezoid and Simp-
son approximations. The emphasis in our discussion is on understanding how to
derive these formulas and how to estimate their errors, with a view to indicating how
to generalize the procedure for such approximations. Our approach is also consis-
tent with that used in Section 4.4 for obtaining the derivative estimates.
Integration algorithms using constant stepsize h are readily derived by using a
Taylor expansion of the integrand within the region to be integrated. Suppose that
the integration is centered on x0, so that the appropriate Taylor expansion of an inte-
grand, y (x), through the nth power of (x - x0) can be obtained by using Taylor’s
theorem, (3.6), as

(4.39)

in which the derivative is evaluated at x0 and the remainder, Rn, is formally given by
(3.7), but it may be approximated by the next term in the power series, which is
what we shall use. The indefinite integral over x of y (x) is obtained by term-by-
term integration of the series (4.39), a procedure which is usually valid for integrals
of interest.

Exercise 4.19
Show that integration of the series terms in (4.39) leads to

(4.40)

for the indefinite integral. n

 4.6 NUMERICAL INTEGRATION METHODS 135

The presence of n derivatives in this formula, ignoring the remainder integral, re-
quires that y be known at (n + 1) values for each basic integral. We now show the
construction explicitly for n = 1 (trapezoid rule) and for n = 2 (Simpson rule).

Trapezoid formula and program for integration

Trapezoid rule formula. The lowest-order formula obtained using (4.40) is for
n = 1, but let us carry the expressions to n = 2 then neglect the integral over the re-
mainder. The n = 2 term will then be an estimate of the error in the n = 1 approx-
imation. Suppose that we wish to integrate from x = a to x = a + h. The most
rapid convergence of the series (4.40) will then probably be achieved by choosing
x 0 =a + h/2.

Exercise 4.20
(a) Make the indicated substitutions of variables in (4.40) in order to show that

(4.41)

in which derivatives of fourth and higher order have been neglected.

(b) Write out the Taylor expansions (4.39) of y (a) and y (a + h) about
x0 = a + h /2, then solve for y (a + h /2) in terms of these y values and the
second derivative at a + h/2. Thus show that (4.41) becomes

(4.42)

which, with neglect of the derivative term, is the basic trapezoid formula.
(c) Show that if y is linear in x, then the trapezoid rule is exact. n

This derivation might seem a bit long-winded, because all we have ended up do-
ing is to average the y values at the beginning and end of the integration interval,
then multiplying by the length of the interval, which is h. However, if we did not
use the more complete analysis that you have just made (haven’t you?), then we
would not be able to estimate the error in the approximation.
Notice that this error is better than you might have guessed, since the effects of
the first derivative at the midpoint of the interval has vanished, leaving the second
derivative as the leading term in the error estimate. By using the midpoint of the in-
tegration interval as the expansion point, all odd-order derivatives vanish from the
integration formula. Therefore, the next neglected term in the integral, and therefore
in our error estimate, involves y( 4 )h 5 .
136 NUMERICAL DERIVATIVES AND INTEGRALS

FIGURE 4.8 Integration formulas, illustrated for the working function (4.1) with stepsize
h = 0.1 in [-O.l, 0.1]. The trapezoid rule uses a straight-line segment in each panel. The Simp-
son rule uses the parabolic section (dashed) over two panels.

Graphical interpretation of the trapezoid rule clarifies the analysis. Figure 4.8
shows our working function, (4.1), near x = 0, the same region where we investi-
gate derivatives in Sections 4.4 and 4.5. The geometric interpretation of Exer-
cise 4.20 (c) is that we approximate the function between the tabulated values at
x = a and x = a + h by a straight-line segment. For example, in Figure 4.8 we
have a trapezoid between x = -0.1 and x = 0, with another between x = 0 and
x = 0.1.
Starting from the basic trapezoid formula (4.42), we may use additivity of integ-
ration to find an approximation to the integral between x = a and x = a + nh.
Thereby we produce the composite trapezoid formula

(4.43)

in which the error estimate is

(4.44)

Here the second derivative is not larger in magnitude than the maximum second de-
rivative in the entire interval. The error scales as the quantity ny(2) h 3 /y, so halv-
ing the stepsize h (while doubling n to cover the same x range) should produce
about a factor of 4 greater accuracy in the integral estimated by the trapezoid rule.
Trapezoid rule program. Programming the trapezoid function is quite straightfor-
ward. As usual, the code for the driver program to make the function testable is
longer for the code than for the function itself. Also, we anticipate including the
Simpson rule in the next subsection, so some code is included for this. Here we
emphasize parts relevant to the trapezoid rule.
 4.6 NUMERICAL, INTEGRATION METHODS 137

PROGRAM 4.5 Trapezoid and Simpson integrals, composite formulas, applied to the working
function (4.1).

#include <stdio.h>
#include <Math.h>

main()
{
/* Trapezoid and Simpson Integrals */
double a,h,IntAnalyt,IntByTrap,error,IntBySimp;
int n,NTrap,NSimp;
double SimpInt(),TrapInt(),y(),YWInt();

printf("Trapezoid and Simpson Integrals\n");

n = 2;
while(n!=O)

printf("\n\nInput n, a, h (n=O to end): ");

scanf("%i%lf%lf",&n,&a,&h);
if ( n == 0 )

printf("\nEnd Trapezoid and Simpson Integrals");

exit (0) ;
}
NTrap = n;
if ( n == 2*(n/2) ) NSimp = n;
else NSimp = O;/* to bypass Simpson for n odd */
IntAnalyt = YWInt(a+n*h)-YWInt(a);
printf("Analytica1 integral=%lg",IntAnalyt);
InByTrap = TrapInt(a,NTrap,h);/* trapezoid rule */
error = IntAnalyt-IntByTrap;
printf("\nTrapezoid integral=%lg",IntByTrap);
printf(" Error=%lg",error);
if ( NSimp != 0 )

IntBySimp = SimpInt(a,NSimp,h);/* Simpson rule */

error = IntAnalyt-IntBySimp;
printf("\nSimpson integral=%lg",IntBySimp);
printf(" Error=%lg",error);

} /*end n loop */
138 NUMERICAL DERIVATIVES AND INTEGRALS

double SimpInt(a,n,h)
/* Simpson integral */
double a,h;
int n;

double sum;
double y();
int n2,j;
n2 = n/2;
if ( (2*n2 != n) || (n2 == 1) )
{
printf("\n !! SimpInt has n=%i; not allowed",n);
return 0;
}
else

sum= 0;
for ( j = 1; j < n2; j++ )
{
sum = sum+y(a+2*j*h)+2*y(a+(2*j-l)*h);

return h*(y(a)+4*y(a+(n-l)*h)+y(a+n*h)+2*sum)/3;

double TrapInt(a,n,h)
/* Trapezoid integral */
double a,h;
int n;
{
double sum;
double y();
int j;
sum = ( y(a) + y(a+n*h) )/2;
for ( j = 1; j < n; j++ )
{
sum = sum+y(a+j*h);
}
return h*sum;
 4.6 NUMERICAL INTEGRATION METHODS 139

double y(x)
/* Working function value*/
double x;

double prod;
prod=12O*(x+0.5)*(x+0.25)*x*(x-(l.0/3.0))*(x-0.2)*(x-1);
return prod;

double YWInt (x)

/* Working function integral */
double x;

double sum;
sum=(-0.5+x+(23.0/4.0)*x*x-10.2*pow(x,3)
-(47.0/3.0)*pow(x,4)+(120.0/7.0)*pow(x,5))*x*x;
return sum;

The structure of Trapezoid and Simpson Integrals can be summarized as

follows. The program uses a while loop controlled by n, the number of steps in
the integration interval, with a zero for n being used to terminate program execu-
tion. The analytical value for the working-function integral, IntAnalyt, is obtained
from (4.4) and Table 4.1, which are coded in function YWInt. The trapezoid rule
is coded in function TrapInt and its value in the main program is IntByTrap.
The function TrapInt integrates the function y, which here is coded as the
working-function expression (4.1). In the C function for y, the roots are given ei-
ther as exact decimals or as ratios of (floating-point) integers. The roots are thereby
computed to machine accuracy. Similarly, maximum accuracy is gained in the C
function YWInt for the working-function integral.

Exercise 4.21
(a) Code and test the trapezoid parts of program Trapezoid and Simpson
Integrals. You can test the program by temporarily substituting for the work-
ing function any linear function of x, and substituting in the analytic-integral
function the corresponding integral. The safest way to code these temporary
changes is to add them just before the return statements in the two C func-
tions. By doing this you do not have to disturb the code for the working func-
tion. Integration of a linear function should produce results accurate to machine
roundoff error.
(b) Run the program for the integral from a = - 0.2 up to 0.2, which Fig-
ure 4.1 suggests has a very small value because of the zero of the function at
x = 0. Thus, for h = 0.1 you need to input n = 4, and so on for three suc-
cessively halved values of h. Check your results against mine in Table 4.9.
140 NUMERICAL DERIVATIVES AND INTEGRALS

(c) Use (4.44) to estimate the error in the trapezoidal rule for each choice of n
and h, using the second derivative given in Table 4.3, namely the value at
x = 0, as an estimator of the appropriate derivative. Show that this over-predicts
the actual error by a factor of about 6. Suggest why. n

TABLE 4.9 Trapezoid integrals for the working function (4.1) from -0.2 to 0.2 with various
stepsizes h. The analytical value of the integral is 9.9109 x 10-3.

Notice, from comparing Table 4.9 with Table 4.4, relatively how much more
accurate the simplest formula for numerical integration (trapezoid rule) is compared
with the simplest formula for numerical differentiation (central-difference formula).
This justifies our remarks at the start of this section about integrals being considered
as antiderivatives.
We return to the trapezoid rule and consider its application to functions that are
less pathological than our working function at the end of the next subsection, where
it is also compared with the Simpson rule.

Simpson formula and program for integrals

In the trapezoid integration formula we assumed straight-line segments between

pairs of “knots” (points where the function is to be evaluated), and this collapsed the
general integration formula to the basic trapezoid formula. If we instead assume
parabolic segments between triples of knots, as shown in Figure 4.8, we can derive
the Simpson formula and an estimate of its error.

Simpson rule formula. The method of deriving this formula is the same as for the
trapezoid formula (Exercise 4.20), except that having three values of the function
allows more of the (usually) unknown derivatives to be eliminated. Try it and see.

Exercise 4.22
(a) To derive the Simpson integration rule, start from the general integral ex-
pression (4.40), expand about x0 = a + h, then solve for the second derivative
in terms of the function values at x = a, x = a + h, and x = a + 2h. Keep
 4.6 NUMERICAL INTEGRATION METHODS 141

the term containing the fourth derivative. Thus derive the basic Simpson for-
mula for integration

(4.45)

(b) Show that if y is a cubic or lower-order polynomial, then the Simpson for-
mula is exact.
(c) Use additivity of integration and the basic integration formula (4.45) to de-
rive the composite Simpson rule for n intervals, where n must be even,

(4.46)

where n2 = n /2, and the error estimate for the composite Simpson rule is

(4.47)

in which the second derivative is bounded by its maximum value in the interval
x = a to x = a + nh. n

The parabolic shape that the Simpson formula assumes for the function in the basic
interval of width 2h is shown as the dashed curve in Figure 4.8 for our working
function (4.1). A different parabola is assumed for each triple of points, so that the
rather poor fit of the dashed to the solid curve outside the range [-O.1,O.l] is not of
immediate concern. The discontinuities of the parabolic curves between successive
panels of width 2h that occur in the Simpson formula are removed if one uses the
cubic-spline technique developed in Chapter 5.

Simpson rule program. The overall program structure for Trapezoid and Simp-
son Integrals was described in the preceding subsection on the trapezoid formula
and program, so here we describe only parts specific to the Simpson formula. On
input the value of n is checked; only if n is even is the Simpson algorithm used to
produce the value IntBySimp, which is compared with IntAnalyt to obtain the
value error for output.
The function SimpInt uses the composite Simpson rule (4.46) to estimate nu-
merically the integral of the function y. In case this function is to be used in another
program, there is a trap to ensure that the number of points, n, is even. For the
composite rule (4.46) n > 2 is also required by the program. (For n = 2 the
basic Simpson formula (4.45) may be used.) The check for even n in SimpInt
142 NUMERICAL DERIVATIVES AND INTEGRALS

can be tested by temporarily disabling the test made in the main program. Note that
the function is coded to resemble closely the formula (4.46), rather than to be very
efficient. If you need a speedy version, you should recode most of SimpInt.
Now that you know how the Simpson integration formula is supposed to work,
it’s time to plug and play.

Exercise 4.23
(a) Test the Simpson rule parts of the program Trapezoid and Simpson In-
tegrals. Temporarily substitute for the working function any function of x as
high as third order, and substitute in the analytic-integral function the corre-
sponding integral. Code these temporary changes just before the return
statements in the two C functions, so as not to disturb the code for the working
function. Integration of a cubic or lower-order polynomial should produce re-
sults accurate to machine roundoff error, according to (4.47).
(b) Run the program for the Simpson integral from a = -0 . 2 up to 0 . 2,
which has a very small value because of the zero of the working function at
x = 0, as seen in (4.1) and Figure 4.1. For h = 0.1 input n = 4, and so on
for three successively halved values of h. Check your results against those in
Table 4.10.
(c) Use (4.47) to estimate the error in the Simpson rule for each choice of n
and h, using the fourth derivative in Table 4.3 at x = 0 as an estimator of the
appropriate derivative. Show that this overpredicts the actual error by a factor
of about 2. Suggest why. n

TABLE 4.10 Simpson-rule integrals for the working function (4.1) from -0.2 to 0.2 with vari-
ous stepsizes h. The analytical integral has value 9.9109 x 10-3.

By comparing Tables 4.9 and 4.10 for the trapezoid and Simpson formulas, re-
spectively, notice how the latter improves much more rapidly than the former as h
decreases, since there is an h 5 dependence of the error in the Simpson rule, compar-
ed with an h 3 dependence in the trapezoid-rule error. This occurs in spite of the
much larger fourth derivative than second derivative (Table 4.3) for the very wiggly
polynomial that is our working function.
 4.6 NUMERICAL INTEGRATION METHODS 143

Integrals with cosines

Now that we have a working program that has been torture tested against a badly
behaved function, it is interesting to try a more-usual function such as the cosine.
Since the cosine has derivatives that are bounded in magnitude by unity, the error
estimates for the trapezoid and Simpson rules, (4.44) and (4.47), will be dominated
by their dependence on h. How about trying your integration program on the cosine
function?

Exercise 4.24
(a) Modify the program Trapezoid and Simpson Integrals by replacing in
the C function y ( x )
y

each h. Make a table and a log-log plot of the absolute value of the error against
h, as in Table 4.11 and Figure 4.9.
(b) Assume that the graph for each method (trapezoid and Simpson) on the log-
log plot is a straight line. Thus show that the error has a power-law dependence
on stepsize h, and that the overall scale of the error estimates the factor preceding
the power of h. Show that the trapezoid formula error is consistent with an h 3
dependence, as in the estimate (4.44), and with a value of the second derivative
of about -0.5, just half the maximum-magnitude value of -1 (at x = 0).
(c) Similarly, for the Simpson formula error show that there is an h 5 depen-
dence, consistent with estimate (4.47), and show that an average fourth deriva-
tive of about 0.5 (again half the maximum value) would make a good match
between prediction and calculation. n

TABLE 4.11 Trapezoid and Simpson integrals for the cosine function from -2 to 2 with various
stepsizes h. The analytical integral has value 2 sin (2) = 1.8 1859.

In Section 4.5 we investigated the numerical first derivative of the cosine func-
tion, and we showed (Table 4.8 and Figure 4.7) that an accuracy of a few parts in
105 is attained for a method comparable to the Simpson integrator (central deriva-
tives, CCD) only for h of order 10-2, whereas for integration h = 0.25 gives
about the same accuracy (Table 4.11). This contrast emphasizes the much greater
difficulty of computing accurate numerical derivatives than numerical integrals.
144 NUMERICAL DERIVATIVES AND INTEGRALS

FIGURE 4.9 Errors in the integral of the cosine from -2 to 2 for various stepsizes h. The
trapezoid-rule errors are indicated by the dashed line, and the Simpson-rule errors are shown by the
solid line.

Higher-order polynomial integration

The numerical-integration technique of approximating a function by a polynomial of

a given order, then evaluating the integral of this polynomial can be made quite gen-
eral. If one does not require an estimate of the error involved, then a Lagrange inter-
polating polynomial, as described in Sections 6.1 and 7.1 of Maron’s book, may be
useful. Various recipes for polynomial integration algorithms, with error estimates
of the kind that we derived, are derived in Chapter 4 of the text by Nakamura and
listed in the handbook of Abramowitz and Stegun. The spline method of integration
we describe in Section 5.5 allows a low-order polynomial to be used for each step-
size, but accuracy is achieved by requiring continuity of values and derivatives
across adjacent steps.
The technique of evaluating the integrals as a function of h, then taking the limit
as h 0 analytically, is called Romberg integration, and it can be very effective,
especially if the integrands are difficult to calculate, so that the number of integrand
values that one wishes to evaluate is to be minimized. Romberg integration is dis-
cussed in detail in Vandergraft’s text on numerical computation.
If unequal steps of the integrand are allowed and the integrand can be evaluated
to arbitrary accuracy at any points, then the technique of Gaussian quadrature is cap-
able of providing great accuracy efficiently. Chapter 4 of Nakamura’s text on ap-
plied numerical methods has a description of such integration algorithms, and Chap-
ter 4 of the numerical recipes book of Press et al. has a function that performs
Gaussian quadrature. A compendium of numerical-integration methods by Davis
and Rabinowitz includes modem techniques (such as the use of splines) and very
complete coverage of the whole spectrum of methods.
 4.7 ELECTROSTATIC POTENTIAL FROM A CHARGED WIRE 145

4.7 PROJECT 4B: ELECTROSTATIC POTENTIAL

FROM A CHARGED WIRE

It is interesting to compute the electrostatic potential from a charged wire because it

involves analysis, numerics, and application to a scientific problem. It is also simple
enough that you can understand the science without extensive physics background.
Suppose that we have electric charge distributed along a finite segment of the y axis,
as shown in the insert to Figure 4.10.
In the figure insert the charge is indicated as being uniformly distributed along
the wire, but we assume only that its distribution, (y), goes from y = -1 to
y = 1 and is zero for y beyond this and for any nonzero x. Our task is to compute
the potential at any point in space, (x, y), not including the wire itself. Such points
are called field points.
From Coulomb’s law of electrostatic interactions, a small element of charge on a
segment of length of wire located near y’ on the wire produces at (x, y) a po-
tential where r is the distance between the charge element and
the field point. (In various systems of electrical units there will be an overall factor
between charge and potential, which we ignore, but electricians should not.) The
total potential at this point is obtained by superimposing the potentials from all points
along the wire in the limit that the lengths of the segments shrink to zero. Thus, the
potential is the Riemann integral of (y‘)/r. Explicitly, the potential is given by

(4.48)

in which we used the theorem of Pythagoras to express r in terms of the coordinates.

FIGURE 4.10 Potential from a wire having a uniform distribution of charge along the wire, as
indicated in the insert. The potential is shown as a function of x distance from the wire for three
different heights, y, above the center of the wire.
146 NUMERICAL DERIVATIVES AND INTEGRALS

Now that the physics problem has been set up, we should concern ourselves
with evaluating this integral. Suppose that the charge density has a simple power-
law dependence on position along the wire as

(4.49)

where p is any integer that is not negative. The potential formula can readily be
transformed so that it is expressed in terms of some simpler integrals.

Exercise 4.25
(a) Make a change of variables in (4.48) and (4.49) in order to simplify the de-
nominator in (4.48), namely, set

(4.50)

(x = 0 doesn’t occur in this problem), then show that the scaled potential,Vs, is
given by

(4.5 1)

where the binomial expansion is used to expand (y + XZ)P. The units of

and therefore of Vs, depend upon the exponent p. The integrals Ik are defined
by

(4.52)

in which k is an integer. The potential problem has been replaced by the actual
problem of evaluating these integrals.
(b) Show (by your own analytical skills, by consulting a table of integrals, or
by using a computer-algebra system) that the indefinite integrals in (4.52) for
k = 0 and 1 are given by

(4.53)

(4.54)

(c) To derive the analytical form of the integral for higher values of k, write the
k-1
factor z k in (4.52) as z x z . Then use integration by parts, the result for I1
in (4.54), and some algebraic manipulations, to derive the recurrence relation
 4.7 ELECTROSTATIC POTENTIAL FROM A CHARGED WIRE 147

(4.55)

Check this result by differentiating both sides with respect to z in order to verify
that you recover the integrand in (4.52). n

From the results of this exercise we have the component integrals that can be com-
bined by using (4.51), after inserting the limits in (4.52), to produce the total scaled
potential Vs at any field point (x, y).
We now have a choice of methods for calculating the electrostatic potential, ana-
lytical or numerical. The former method, although it has the elegant results from Ex-
ercise 4.25 at its command, is restricted to integer powers p. On the other hand any
positive p may be used with numerical integration of (4.48), provided that |y'|,
rather than y', is used in (4.49). However analytical integration is useful for testing
the accuracy of the numerical methods, so we start with it.

Potentials by analytical integration

The simplest analytical potentials are obtained if the electric charge is distributed uni-
formly along the wire. Then in the above integrals we have k = p = 0 only. The
scaled potential is then given by

(4.56)

This potential is plotted in Figure 4.10 as a function of x for three values of y, half-
way up the wire (y = 0.5), above the level of the wire (y = 1.5), and far above the
wire (y = 2.5). As you would guess from Coulomb’s law of the inverse-distance
dependence of the potential, Vs dies away rapidly as the field point is moved farther
away from the source of the charge on the wire.
If the electric charge is distributed linearly along the wire (which would have to
be made of a nonconducting material, so that the charges didn’t flow and neutralize
each other), with positive charges along the top half and negative charges along the
bottom half, then the scaled potential can be calculated from the integral expansion
(4.5 1).

Exercise 4.26
(a) Write a small C function to evaluate the analytical potential, Vs (x,y), for the
linear charge distribution (p = 1) in terms of the integrals Ik obtained in Exer-
cise 4.25 and the expression (4.51) for Vs.
(b) Make a plot similar to Figure 4.10 for the potential due to this linear distrib-
ution of charge. n

With two check cases in hand, we are ready to develop the numerical applications.
148 NUMERICAL DERIVATIVES AND INTEGRALS

Potentials by numerical-integration methods

In the preceding subsection we showed that analytical expressions for the potentials
from a line charge with various (positive-integer) power-law distributions for the
charge, given by (4.49), can be derived. In the numerical-integration methods we
use these as test cases, then explore the situation with noninteger power laws.
First we need a way to handle the numerics. It is straightforward to modify Pro-
gram4.5, Trapezoid and Simpson Integrals, to do this. The resulting pro-
gram, Electrostatic Potentials by Numerical Integration,is given in the
following program.

PROGRAM 4.6 Electrostatic potentials by numerical integration, trapezoidal and Simpson

formulas, for a uniform charge distribution.

#include <stdio.h>
#include <math.h>

main()
{
/* Electrostatic Potentials
by Numerical Integration */
double X,Y,zmin,zmax,h,VAnl,VByTrap,error,VBySimp;
int n;
double Vs(),SimpInt(),TrapInt(),y();

printf("Potentials by Numerical Integration\n");

x = 1;
while(X!=O)

printf("\n\nInput X, Y, n (X=0 to end): ");

scanf("%lf%lf%i",&X,&Y,&n) ;
if (X == 0)
{
printf("\nEnd Potentials by Numerical Integration");
exit (0) ;

n = 2*(n/2); /* truncate n to nearest even value */

zmin = (X-1)/Y; zmax = (X+1)/Y;
h = (zmax-zmin)/n;
VAnl = Vs(X,Y);
printf("Analytica1 potential=%lg",VAnl);
VByTrap = TrapInt(zmin,n,h);/* trapezoid rule */
error = VAnl-VByTrap;
printf("\nTrapezoid integral=%lg",VByTrap);
printf(" Error=%lg",error);
VPySimp = SimpInt(zmin,n,h);/* Simpson rule */
 4.7 ELECTROSTATIC POTENTIAL FROM A CHARGED WIRE 149

error = VAnl-VBySimp;
printf("\nSimpson integral=%lg",VBySimp);
printf(" Error=%lg",error);
} /*end x loop*/

double SimpInt(a,n,h)
/* Simpson integral */
double a,h;
int n;

double sum;
double y();
int n2,j;
n2 = n/2;
if ( ( 2*n2 != n ) || ( n2 == 1) )
{
printf("\n !! SimpInt has n=%i; not allowed",n);
return 0;
}
else

sum= 0;
for ( j = 1; j < n2; j++ )

sum = sum+y(a+2*j*h)+2*y(a+(2*j-l)*h);

return h*(y(a)+4*y(a+(n-l)*h)+y(a+n*h)+2*sum)/3;

double TrapInt(a,n,h)
/* Trapezoid integral */
double a,h;
int n;

double sum;
double y*();
int j;
sum = ( y(a) + y(a+n*h) )/2;
for ( j = 1; j < n; j++ )

sum = sum+y(a+j*h);
}
return h*sum;
150 NUMERICAL DERIVATIVES AND INTEGRALS

double y(x)
/* Potential integrand value*/
double x;

double prod;
prod = l/sqrt(x*x+l);
return prod;

double Vs(x,y)
/* Analytical potential integral */
double x,y;

double sum;
sum = -log(sqrt(pow((x+l)/y,2)+1)-(x+1)/y)
-log(sqrt(pow((x-1)/y,2)+l)+(x-l)/y);
return sum;

In Program 4.6 there has been some renaming of variables in the main program,
slight modifications of input, but no changes to integrator, functions SimpInt and
TrapInt. These two functions may therefore be copied from their parent, Prog-
ram 4.5. The price paid for this is that the variable names do not agree with those
used in formulating our electrostatic-potential problem, so care is needed if the pro-
grams are modified. Note that the method of signalling termination of program exe-
cution, input x = 0, is convenient rather than necessary. Provided that |y| > 1, so
the field point is not touching the wire (how shocking!), the potential is well-defined
on the x axis. If you don’t like this way of stopping execution, devise your own.
The functions for the potential integrand, y (x) , and for the analytical potential,
Vs (x , y ) , have been coded to agree with (4.48) and (4.56), respectively, for the
case p = 0. Now you should be ready to code and test the potential program.

Exercise 4.27
(a) Code and test Electrostatic Potentials by Numerical Integra-
tion. The program can be tested by coding below the formulas in y (x) and
Vs (x, y ) an integrand and an analytic integral, respectively, that can easily be
checked by hand, such as a low-order polynomial.
(b) Run the program for the coded example of the uniform charge density and a
reasonable stepsize, say h = 0.1, to check your results against those displayed
in Figure 4.10 at y = 0.5, 1.5, and 2.5 for x > 0.
(c) If you have a program that displays contours, modify Electrostatic Po-
tentia1s to compute a mesh of( x , y )points fine enough that smooth con-
tours (equipotentials) are produced by the Simpson method. Thus generate the
contours for the potential from a uniformly charged wire. Do the contours make
physical sense, in that electrostatic energy would be constant along a contour?
 REFERENCES ON NUMERICAL DERIVATIVES AND INTEGRALS 151

(d) Modify the program to allow positive integer powers, p, for the charge
density (4.49). This requires only minor changes to the potential-integrand
function y (x) , but substantial changes are needed to the potential integral func-
tion, Vs (x, y ) , to include the recurrence formula (4.56) and the summation
(4.51). (You may have already done this in Exercise 4.26.) Then this function
has to be used for both limits of integration. The numerical integration functions
should be scarcely modified, and thereby they serve to identify any errors in
your analytical results. As p is increased, the charges should pile up (positive
and negative) at the two ends of the wire, and the potential should resemble that
from two point charges of opposite sign at the two ends of the wire. Is this what
you find?
(e) As a final topic of exploration, turn off the analytical solution and try the nu-
merical integrations with noninteger values of p, such as p = 0.5, 0.75, and a
charge distributed as |y|p rather than yp, in order to avoid complex results. Your
results should roughly interpolate between those for adjacent integer values of p,
such as p = 0, 1. To make such a comparison you will need to modify the
solutions to accommodate the change in the charge-distribution formula (4.49)
from yp to |y|p. For example, just split the integral into contributions from
above and below the origin. n

With the experience in numerical integration that you have gained from this ap-
plied-physics project and from the exercises in Sections 4.4 - 4.6, you should be
confident to compute many kinds of numerical derivatives and integrals.

REFERENCES ON NUMERICAL DERIVATIVES AND INTEGRALS

Abramowitz, M., and I. A. Stegun, Handbook of Mathematical Functions, Dover,

New York, 1964.
Davis, P. J., and P. Rabinowitz, Methods of Numerical Integration, Academic
Press, Orlando, Florida, second edition, 1984.
Maron, M. J., Numerical Analysis, Macmillan, New York, second edition, 1987.
Nakamura, S., Applied Numerical Methods with Software, Prentice Hall, Engle-
wood Cliffs, New Jersey, 1991.
Pippard, A. B., Response and Stability, Cambridge University Press, Cambridge,
England, 1985.
Pippard, A. B., The Physics of Vibration, Cambridge University Press, Cambridge,
England, 1989.
Press, W. H., B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical
Recipes in C, Cambridge University Press, New York, 1988.
Vandergraft, J. S., Introduction to Numerical Computations, Academic Press, New
York, 1978.
Wolfram, S., Mathematica: A System for Doing Mathematics by Computer, Addi-
son-Wesley, Redwood City, California, second edition, 199 1.
152