Module

for

The Improved Newton Method

Introduction

Newton's method is used to locate roots of the equation . The Newton-

Raphson iteration formula is:

(1)

Given a starting value , the sequence is computed using:

(2) for

provided that .

If the value is chosen close enough to the root p, then the sequence generated in
(2) will converge to the root p. Sometimes the speed at which converges is
fast (quadratic) and at other times it is slow (linear). To distinguish these two cases we
make the following definitions.

Definition 1 (Order of Convergence) Assume that converges to p, and

set . If two positive constants exist, and

then the sequence is said to converge to p with order of convergence R. The

number A is called the asymptotic error constant. The cases are given
special consideration.

(3) If , and

then the convergence of is called quadratic.

(4) If , and

then the convergence of is called linear.

The mathematical characteristic for determining which case occurs is the "multiplicity"
of the root p.

Definition 2 (Order of a Root) If can be factored as

(5) where m is a positive integer,

and is continuous at and , then we say that has a root of

order m at .

A root of order is often called a simple root, and if it is called a multiple

root. A root of order is sometimes called a double root, and so on.

Theorem 1 (Convergence Rate for Newton-Raphson Iteration) Assume that

Newton-Raphson iteration (2) produces a sequence that converges to the
root p of the function .

(6) If p is a simple root, then convergence is quadratic and

for k sufficiently large.

(7) If p is a multiple root of order m, then convergence is linear and

for k sufficiently large.

Method A. Accelerated Newton-Raphson

Suppose that p is a root of order . Then the accelerated Newton-Raphson

formula is:

(8) .

Let the starting value be close to p, and compute the sequence

iteratively;
(9) for

Then the sequence generated by (9) will converge quadratically to p.

Proof Accelerated & Modified Newton-Raphson Accelerated & Modified

Newton-Raphson

On the other hand, if then one can show that the

function has a simple root at .

Derivation

Using in place of in formula (1) yields Method B.

Method B. Modified Newton-Raphson

Suppose that p is a root of order . Then the modified Newton-Raphson

formula is:

(10)

Derivation

Let the starting value be close to p, and compute the sequence

iteratively;

(11) for

Then the sequence generated by (11) converges quadratically to p.

Limitations of Methods A and B

Method A has the disadvantage that the order m of the root must be known a
priori. Determining m is often laborious because some type of mathematical analysis
must be used. It is usually found by looking at the values of the higher derivatives
of . That is, has a root of order m at if and only if

(12) .

Dodes (1978, pp. 81-82) has observed that in practical problems it is unlikely that we
will know the multiplicity. However, a constant m should be used in (8) to speed up
convergence, and it should be chosen small enough so that does not shoot to the
wrong side of p. Rice (1983, pp. 232-233) has suggested a way to empirically
find m. If is a good approximation to p and , and somewhat distant
from then m can be determined by the calculation:

Method B has a disadvantage, it involves three functions

. Again, the laborious task of finding the formula for could detract from
using Method B. Furthermore, Ralston and Rabinowitz (1978, pp. 353-356) have
observed that will have poles at points where the zeros of are not roots
of . Hence, may not be a continuous function.

The New Method C. Adaptive Newton-Raphson

The adaptive Newton-Raphson method incorporates a linear search method with

formula (8). Starting with , the following values are computed:

(13) for .

Our task is to determine the value m to use in formula (13), because it is not known a
priori. First, we take the derivative of in formula (5), and obtain:

(14)

When (5) and (14) are substituted into formula (1) we

have which can be simplified to get

.
This enables us to rewrite (13) as

(15) .

We shall assume that the starting value is close enough to p so that

(16) , where .

The iterates in (15) satisfy the following:

(17) for

If we subtract p from both sides of (17) then

and the result after simplification is:

(18) .

Since , the iterates get closer to p as j goes from , which is

manifest by the inequalities:

(19) .

The values are shown in Figure 1.

Notice that if the iteration (15) was continued for

then could be larger than . This is proven by
using the derivatives in (12) and the Taylor polynomial approximation of
degree m for expanded about :

(20) .
Figure 1. The values for the "linear search" obtained by using
formula (15)
near a "double root" p (of order ). Notice
that .

If is closer to p than then (19) and (20) imply that ,

hence we have:

(21) .

Therefore, the way to computationally determine m is to successively compute the

values using formula (13) for until we arrive
at .

The New Adaptive Newton-Raphson Algorithm

Start with , then we determine the next approximation as follows:

 Observe that the above iteration involves a linear search in either the
interval when or in the interval when . In the

algorithm, the value is stored so that unnecessary computations are

avoided. After the point has been found, it should replace
and the process is repeated.

Example 1. Use the value and compare Methods A,B and C for finding the
double root of the equation .
Solution 1.

Behavior at a Triple Root

When the function has a triple root, then one more iteration for the linear search in
(13) is necessary. The situation is shown in Figure 2.
Figure 2. The values for the "linear search" obtained by using
formula (15)
near a "triple root" p (of order ). Notice that
.

Example 2. Use the value and compare Methods A,B and C for finding the
triple root of the equation .

Example 3. Use the value and compare Methods A,B and C for finding the
quadruple root of the equation

Acknowledgement

This module uses Mathematica instead of Pascal, and the content is that of the article:

John Mathews, An Improved Newton's Method, The AMATYC Review, Vol. 10, No. 2,
Spring, 1989, pp. 9-14.

References

1. Dodes, 1. A., Numerical analysis for computer science, 1978, New York: North-
Holland.
2. Mathews, J. H., Numerical methods for computer science, engineering and
mathematics, 1987, Englewood Cliffs, NJ: Prentice-Hall.
3. Ralston, A. & Rabinowitz, P., A first course in numerical analysis, second ed., 1978,
New York: McGraw-Hill.
4. Rice, J. R., Numerical methods, software and analysis: IMSL reference edition,
1983, New York: McGraw-Hill.

Module

for

The Bisection Method

Background. The bisection method is one of the bracketing methods for finding roots
of equations.
Implementation. Given a function f(x) and an interval which might contain a root,
perform a predetermined number of iterations using the bisection method.
Limitations. Investigate the result of applying the bisection method over an interval
where there is a discontinuity. Apply the bisection method for a function using an
interval where there are distinct roots. Apply the bisection method over a "large"
interval.

Theorem (Bisection Theorem). Assume that and that there exists a number
such that .
If have opposite signs, and represents the sequence of midpoints
generated by the bisection process, then

for ,

and the sequence converges to the zero .

That is, .

Proof Bisection Method Bisection Method

Mathematica Subroutine (Bisection Method).

Example 1. Find all the real solutions to the cubic equation .

Example 2. Use the cubic equation in Example 1 and perform the

following call to the bisection method.

Reduce the volume of printout.

After you have debugged you program and it is working properly, delete the
unnecessary print statements.

Concise Program for the Bisection Method

Now test the example to see if it still works. Use the last case in Example 1 given above
and compare with the previous results.
Example 3. Convergence Find the solution to the cubic equation
. Use the starting interval .
Solution 3.

Example 4. Not a root located Find the solution to the equation . Use the
starting interval .
Solution 4.

Animations (Bisection Method Bisection Method). Internet hyperlinks to

animations.

Old Lab Project (Bisection Method Bisection Method). Internet hyperlinks to an old
lab project.

Module

for

The Secant Method

The Newton-Raphson algorithm requires two functions evaluations per iteration,

and . Historically, the calculation of a derivative could involve
considerable effort. But, with modern computer algebra software packages such as
Mathematica, this has become less of an issue. Moreover, many functions have non-
elementary forms (integrals, sums, discrete solution to an I.V.P.), and it is desirable to
have a method for finding a root that does not depend on the computation of a
derivative. The secant method does not need a formula for the derivative and it can be
coded so that only one new function evaluation is required per iteration.

The formula for the secant method is the same one that was used in the regula falsi
method, except that the logical decisions regarding how to define each succeeding term
are different.
Theorem (Secant Method). Assume that and there exists a number
, where . If , then there exists a such that the
sequence defined by the iteration

for

will converge to for certain initial approximations .

Proof Secant Method Secant Method

Algorithm (Secant Method). Find a root of given two initial

approximations using the iteration

for .

Computer Programs Secant Method Secant Method

Mathematica Subroutine (Secant Method).

Example 1. Use the secant method to find the three roots of the cubic
polynomial .

Determine the secant iteration formula that is used.

Show details of the computations for the starting value .
Solution 1.

Example 2. Use Newton's method to find the roots of the cubic

polynomial .
2 (a) Fast Convergence. Investigate quadratic convergence at the simple root
, using the starting value
2 (b) Slow Convergence. Investigate linear convergence at the double root
, using the starting value
Solution 2.

The following subroutine call uses a maximum of 20 iterations, just to make sure
enough iterations are performed.
However, it will terminate when the difference between consecutive iterations is less
than .
By interrogating k afterward we can see how many iterations were actually performed.
Example 3. Fast Convergence Find the solution to .
Use the Secant Method and the starting approximations and .
Solution 3.

Example 4. Slow Convergence Find the solution to .

Use the Secant Method and the starting approximations and .
Solution 4.

Example 5. Convergence, Oscillation Find the solution to .

Use the Secant Method and the starting approximations and .
Solution 5.

Example 6. Convergence Find the solution to .

Use the Secant Method and the starting approximations and .
Solution 6.

Example 7. NON Convergence, Diverging to Infinity Find the solution to

.
Use the Secant Method and the starting approximations and .
Solution 7.

Example 8. Convergence Find the solution to .

Use the Secant Method and the starting approximations and .
Solution 8.

Animations (Secant Method Secant Method) . Internet hyperlinks to animations.

Research Experience for Undergraduates

Secant Method Secant Method Internet hyperlinks to web sites and a bibliography of
articles.

Download this Mathematica NotebooSecant MethodReturn to Numerical Methods

- Numerical Analysis

Module

for

The Regula Falsi Method

Background. The Regula Falsi method is one of the bracketing methods for finding
roots of equations.
Implementation. Given a function f(x) and an interval which might contain a root,
perform a predetermined number of iterations using the Regula Falsi method.
Limitations. Investigate the result of applying the Regula Falsi method over an interval
where there is a discontinuity. Apply the Regula Falsi method for a function using an
interval where there are distinct roots. Apply the Regula Falsi method over a "large"
interval.

Theorem (Regula Falsi Theorem). Assume that and that there exists a
number such that .
If have opposite signs, and
represents the sequence of points generated by the Regula Falsi process, then the
sequence converges to the zero .

That is, .

Proof False Position or Regula Falsi Method False Position or Regula Falsi
Method

Computer Programs False Position or Regula Falsi Method False Position or

Regula Falsi Method

Mathematica Subroutine (Regula Falsi Method).

Example 1. Find all the real solutions to the cubic equation .

Solution 1.

Remember. The Regula Falsi method can only be used to find a real root in an
interval [a,b] in which f[x] changes sign.
Example 2. Use the cubic equation in Example 1 and perform the
following call to the Regula Falsi subroutine.
Solution 2.

Reduce the volume of printout.

After you have debugged you program and it is working properly, delete the
unnecessary print statements.

Now test the example to see if it still works. Use the last case in Example 1 given above
and compare with the previous results.

Reducing the Computational Load for the Regula Falsi Method

The following program uses fewer computations in the Regula Falsi method and is the
traditional way to do it. Can you determine how many fewer functional evaluations are
used ?

Various Scenarios and Animations for Regula Falsi Method.

Example 3. Convergence Find the solution to the cubic equation

. Use the starting interval .
Solution 3.

Animations (Regula Falsi Method Regula Falsi Method). Internet hyperlinks to

animations.

Research Experience for Undergraduates

Regula Falsi Method Regula Falsi Method Internet hyperlinks to web sites and a
bibliography of articles.
Download this Mathematica Notebook Regula Falsi Method

Return to Numerical Methods - Numerical Analysis

Module

for

Gauss-Jordan Elimination

In this module we develop a algorithm for solving a general linear system of

equations consisting of n equations and n unknowns where it is assumed that the
system has a unique solution. The method is attributed Johann Carl Friedrich Gauss
(1777-1855) and Wilhelm Jordan (1842 to 1899). The following theorem states the
sufficient conditions for the existence and uniqueness of solutions of a linear system
.

Theorem (Unique Solutions) Assume that is an matrix. The following

statements are equivalent.

(i) Given any matrix , the linear system has a unique solution.

(ii) The matrix is nonsingular (i.e., exists).

(iii) The system of equations has the unique solution .

(iv) The determinant of is nonzero, i.e. .

It is convenient to store all the coefficients of the linear system in one array of
dimension . The coefficients of are stored in column of the array (i.e.
). Row contains all the coefficients necessary to represent the equation
in the linear system. The augmented matrix is denoted and the linear system
is represented as follows:
 The system , with augmented matrix , can be solved by performing row
operations on . The variables are placeholders for the coefficients and cam be
omitted until the end of the computation.

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Theorem (Elementary Row Operations). The following operations applied to the

augmented matrix yield an equivalent linear system.

(i) Interchanges: The order of two rows can be interchanged.

(ii) Scaling: Multiplying a row by a nonzero constant.

(iii) Replacement: Row r can be replaced by the sum of that tow and a nonzero
multiple of any other row;
that is: .

It is common practice to implement (iii) by replacing a row with the difference of that
row and a multiple of another row.

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Definition (Pivot Element). The number in the coefficient matrix that is used to
eliminate where , is called the pivot element, and the
row is called the pivot row.

Theorem (Gaussian Elimination with Back Substitution). Assume that is an

nonsingular matrix. There exists a unique system that is equivalent to the given
system , where is an upper-triangular matrix with for
. After are constructed, back substitution can be used to solve for .

Proof Gauss-Jordan Elimination and Pivoting Gauss-Jordan Elimination and

Pivoting

Algorithm I. (Limited Gauss-Jordan Elimination). Construct the solution to the

linear system by using Gauss-Jordan elimination under the assumption that
row interchanges are not needed. The following subroutine uses row operations to
eliminate in column p for .
Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan
Elimination and Pivoting

Mathematica Subroutine (Limited Gauss-Jordan Elimination).

Example 1. Use the Gauss-Jordan elimination method to solve the linear

system .
Solution 1.

Example 2. Interchange rows 2 and 3 and try to use the Gauss-Jordan subroutine to

solve .
Use lists in the subscript to select rows 2 and 3 and perform the row interchange.
Solution 2.

Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan

Elimination and Pivoting

Mathematica Subroutine (Complete Gauss-Jordan Elimination).

Example 3. Use the improved Gauss-Jordan elimination subroutine with row

interchanges to solve .
Use the matrix A and vector B in Example 2.
Solution 3.

Example 4. Use the Gauss-Jordan elimination method to solve the linear

system .
Solution 4.
Use the subroutine "GaussJordan" to find the inverse of a matrix.

Theorem (Inverse Matrix) Assume that is an nonsingular matrix. Form the

augmented matrix of dimension . Use Gauss-Jordan elimination to
reduce the matrix so that the identity is in the first columns. Then the inverse
is located in columns .

Proof The Inverse Matrix The Inverse Matrix

Example 5. Use Gauss-Jordan elimination to find the inverse of the

matrix .
Solution 5.

Algorithm III. (Concise Gauss-Jordan Elimination). Construct the solution to the

linear system by using Gauss-Jordan elimination. The print statements are for
pedagogical purposes and are not needed.

Computer Programs Gauss-Jordan Elimination and Pivoting Gauss-Jordan

Elimination and Pivoting

Mathematica Subroutine (Concise Gauss-Jordan Elimination).

Remark. The Gauss-Jordan elimination method is the "heuristic" scheme found in most
linear algebra textbooks. The line of code

divides each entry in the pivot row by its leading coefficient . Is this step
necessary? A more computationally efficient algorithm will be studied which uses
upper-triangularization followed by back substitution. The partial pivoting strategy will
also be employed, which reduces propagated error and instability.

Application to Polynomial Interpolation

Consider a polynomial of degree n=5 that passes through the six points
;

.
For each point is used to an equation , which in turn are used to
write a system of six equations in six unknowns

=
=
=
=
=
=

The above system can be written in matrix form MC = B

Solve this linear system for the coefficients and then construct the
interpolating polynomial

Example 6. Find the polynomial p[x] that passes through the six
points using the following steps.
(a). Write down the linear system MC = B to be solved.
(b). Solve the linear system for the coefficients using our Gauss-Jordan subroutine.
(c). Construct the polynomial p[x].
Solution 6.

Module

forJacobi and Gauss-Seidel Iteration

Background

Iterative schemes require time to achieve sufficient accuracy and are reserved for
large systems of equations where there are a majority of zero elements in the matrix.
Often times the algorithms are taylor-made to take advantage of the special structure
such as band matrices. Practical uses include applications in circuit analysis, boundary
value problems and partial differential equations.

Iteration is a popular technique finding roots of equations. Generalization of fixed

point iteration can be applied to systems of linear equations to produce accurate
results. The method Jacobi iteration is attributed to Carl Jacobi (1804-1851) and Gauss-
Seidel iteration is attributed to Johann Carl Friedrich Gauss (1777-1855) and Philipp
Ludwig von Seidel (1821-1896).

Consider that the n×n square matrix A is split into three parts, the main diagonal D,
below diagonal L and above diagonal U. We have A = D - L - U.

-
Definition (Diagonally Dominant). The matrix is strictly diagonally dominant if

for .

Theorem (Jacobi Iteration). The solution to the linear system can be

obtained starting with , and using iteration scheme

where

and .

If is carefully chosen a sequence is generated which converges to the

solution P, i.e. .
A sufficient condition for the method to be applicable is that A is strictly diagonally
dominant or diagonally dominant and irreducible.

Proof Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration

Theorem (Gauss-Seidel Iteration). The solution to the linear system can be

obtained starting with , and using iteration scheme

where

and .

If is carefully chosen a sequence is generated which converges to the

solution P, i.e. .
A sufficient condition for the method to be applicable is that A is strictly diagonally
dominant or diagonally dominant and irreducible.

Proof Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration

Computer Programs Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel

Iteration

Mathematica Subroutine (Jacobi Iteration).

Example 1. Use Jacobi iteration to solve the linear

system .
Try 10, 20 and 30 iterations.

Example 2. Use Jacobi iteration to attempt solving the linear

system .
Try 10 iterations.
Observe that something is not working. In example 5 we will check to see if this matrix
is diagonally dominant.

Example 3. Use Gauss-Seidel iteration to solve the linear

system .
Try 10, 20 iterations. Compare the speed of convergence with Jacobi iteration.

Example 4 Use Gauss-Seidel iteration to attempt solving the linear

system .
Try 10 iterations.
Observe that something is not working. In example 5 we will check to see if this matrix
is diagonally dominant.

.
Mathematica Subroutine (Gauss-Seidel Iteration).

Example 6. Use Jacobi and Gauss-Seidel iteration to solve the linear

system .
Use a tolerance of and a maximum of 50 iterations.

version of Jacobi iteration to solve the linear system .

Research Experience for Undergraduates

Jacobi and Gauss-Seidel Iteration Jacobi and Gauss-Seidel Iteration Internet

hyperlinks to web sites and a bibliography of articles.

Download this Mathematica Notebook Jacobi and Gauss-Seidel Iteration

Return to Numerical Methods - Numerical Analysis

 Module

for

Numerical Differentiation, Part I

Background.

Numerical differentiation formulas formulas can be derived by first constructing the

Lagrange interpolating polynomial through three points, differentiating the
Lagrange polynomial, and finally evaluating at the desired point. In this
module the truncation error will be investigated, but round off error from computer
arithmetic using computer numbers will be studied in another module.

Theorem (Three point rule for ). The central difference formula for the first
derivative, based on three points is

and the remainder term is

Together they make the equation , and the truncation

error bound is

where . This gives rise to the Big "O" notation for the error
term for :

Proof Numerical Differentiation Numerical Differentiation

Theorem (Three point rule for ). The central difference formula for the
second derivative, based on three points is
 ,

and the remainder term is

Together they make the equation , and the truncation

error bound is

where . This gives rise to the Big "O" notation for the error
term for :

Proof Numerical Differentiation Numerical Differentiation

Animations (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to animations.

Computer Programs Numerical Differentiation Numerical Differentiation

Project I.

Investigate the numerical differentiation formula and

truncation error bound

where . The truncation error is investigated. The round off
error from computer arithmetic using computer numbers will be studied in another
module.

Enter the three point formula for numerical differentiation.

Aside. From a mathematical standpoint, we expect that the limit of the difference
quotient is the derivative. Such is the case, check it out.

Example 1. Consider the function . Find the formula for the third
derivative , it will be used in our explorations for the remainder term and the
truncation error bound. Graph . Find the bound
. Look at it's graph and estimate the value , be sure to take the absolute value if
necessary.

Example 2 (a). Compute numerical approximations for the derivative , using

step sizes , include the details.
2 (b). Compute numerical approximations for the derivatives
, using step sizes .
2 (c). Plot the numerical approximation over the interval
. Compare it with the graph of over the interval .

Example 3. Plot the absolute error over the interval

, and estimate the maximum absolute error over the interval.
3 (a). Compute the error bound and observe
that over .
3 (b). Since the function f[x] and its derivative is well known, and we have the graph
for , we can observe that the maximum error on the given
interval occurs at x=0. Thus we can do better that "theory", we see that
over .
Example 4. Investigate the behavior of . If the step size is reduced by a factor

of then the error bound is reduced by . This is the behavior.

Project II.

Investigate the numerical differentiation

formulae and truncation error

bound where . The

truncation error is investigated. The round off error from computer arithmetic using
computer numbers will be studied in another module.

Enter the formula for numerical differentiation.

Aside. It looks like the formula is a second divided difference, i.e. the difference
quotient of two difference quotients. Such is the case.

Aside. From a mathematical standpoint, we expect that the limit of the second divided
difference is the second derivative. Such is the case.

Example 5. Consider the function . Find the formula for the fourth
derivative , it will be used in our explorations for the remainder term and the
truncation error bound. Graph . Find the bound
. Look at it's graph and estimate the value , be sure to take the absolute value if
necessary.

Example 6 (a). Compute numerical approximations for the derivatives

, using step sizes .
6 (b). Plot the numerical approximation over the interval
. Compare it with the graph of over the interval

Example 7. Plot the absolute error over the interval

, and estimate the maximum absolute error over the interval.
7 (a). Compute the error bound and observe
that over .
7 (b). Since the function f[x] and its derivative is well known, and we have the graph
for , we can observe that the maximum error on the given
interval occurs at x=0. Thus we can do better that "theory", we see that
over .

Example 8. Investigate the behavior of . If the step size is reduced by a factor

of then the error bound is reduced by . This is the behavior.

Example 9. Given , find numerical approximations to the

derivative , using two points and the forward difference formula.

Example 10. Given , find numerical approximations to the

derivative , using two points and the backward difference formula.

Example 11. Given , find numerical approximations to the

derivative , using two points and the central difference formula.
Example 12. Given , find numerical approximations to the
derivative , using three points and the forward difference formula.

Example 13. Given , find numerical approximations to the

derivative , using three points and the backward difference formula.

Example 14. Given , find numerical approximations to the

derivative , using three points and the central difference formula.

Example 15. Given , find numerical approximations to the second

derivative , using three points and the forward difference formula.

Example 16. Given , find numerical approximations to the second

derivative , using three points and the backward difference formula.

Example 17. Given , find numerical approximations to the second

derivative , using three points and the central difference formula.

Animations (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to animations.

Old Lab Project (Numerical Differentiation Numerical Differentiation). Internet

hyperlinks to an old lab project.

Research Experience for Undergraduates

Numerical Differentiation Numerical Differentiation Internet hyperlinks to web

sites and a bibliography of articles.
 Module

for

The Trapezoidal Rule for Numerical Integration

Theorem (Trapezoidal Rule) Consider over , where .

The trapezoidal rule is

This is an numerical approximation to the integral of over and we have

the expression

The remainder term for the trapezoidal rule is , where

lies somewhere between , and have the equality

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical

Integration

Composite Trapezoidal Rule

An intuitive method of finding the area under a curve y = f(x) is by approximating

that area with a series of trapezoids that lie above the intervals . When
several trapezoids are used, we call it the composite trapezoidal rule.

Theorem (Composite Trapezoidal Rule) Consider over . Suppose

that the interval is subdivided into m subintervals of equal

width by using the equally spaced nodes

for . The composite trapezoidal rule for m subintervals is
 .

This is an numerical approximation to the integral of over and we write

Remainder term for the Composite Trapezoidal Rule

Corollary (Trapezoidal Rule: Remainder term) Suppose that is subdivided

into m subintervals of width . The composite trapezoidal

rule

is an numerical approximation to the integral, and

Furthermore, if , then there exists a value c with a < c < b so that the
error term has the form

This is expressed using the "big " notation .

Remark. When the step size is reduced by a factor of the error term

should be reduced by approximately .

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical

Integration

Animations (Trapezoidal Rule Trapezoidal Rule).

Computer Programs Trapezoidal Rule for Numerical Integration Trapezoidal

Rule for Numerical Integration
Algorithm Composite Trapezoidal Rule. To approximate the integral

by sampling at the equally spaced points for

, where . Notice that and .

Mathematica Subroutine (Trapezoidal Rule).

Or you can use the traditional program.

Mathematica Subroutine (Trapezoidal Rule).

Example 1. Numerically approximate the integral by using the

trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

Example 2. Numerically approximate the integral by using the

trapezoidal rule with m = 50, 100, 200, 400 and 800 subintervals.
Example 3. Find the analytic value of the integral (i.e. find the
"true value").

Example 4. Use the "true value" in example 3 and find the error for the trapezoidal rule
approximations in example 2.

Example 5. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Example 6. Numerically approximate the integral by using

the trapezoidal rule with m = 1, 2, 4, 8, and 16 subintervals.

Example 7. Numerically approximate the integral by using

the trapezoidal rule with m = 50, 100, 200, 400 and 800 subintervals.

Example 8. Find the analytic value of the integral (i.e. find

the "true value").

Example 9. Use the "true value" in example 8 and find the error for the trapezoidal rule
approximations in exercise 7.

Example 10. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Recursive Integration Rules

Theorem (Successive Trapezoidal Rules) Suppose that and the

points subdivide into subintervals equal

width . The trapezoidal rules obey the relationship

Definition (Sequence of Trapezoidal Rules) Define

, which is the trapezoidal rule with step size . Then for each

define is the trapezoidal rule with step size

.

Corollary (Recursive Trapezoidal Rule) Start with . Then

a sequence of trapezoidal rules is generated by the recursive formula

for .

where .

The recursive trapezoidal rule is used for the Romberg integration algorithm.

Various Scenarios and Animations for the Trapezoidal Rule.

Example 11. Let over . Use the Trapezoidal Rule to

approximate the value of the integral.

Animations (Trapezoidal Rule Trapezoidal Rule).

Research Experience for Undergraduates

Trapezoidal Rule for Numerical Integration Trapezoidal Rule for Numerical
Integration Internet hyperlinks to web sites and a bibliography of articles.

Trapezoidal Rule for Numerical Integration

Return to Numerical Methods - Numerical Analysis

Module

for

Simpson's Rule for Numerical Integration

The numerical integration technique known as "Simpson's Rule" is credited to the

mathematician Thomas Simpson (1710-1761) of Leicestershire, England. His also
worked in the areas of numerical interpolation and probability theory.

Theorem (Simpson's Rule) Consider over , where , and

. Simpson's rule is

This is an numerical approximation to the integral of over and we have

the expression

The remainder term for Simpson's rule is , where lies

somewhere between , and have the equality

.Simpson's Rule for

Numerical Integration Simpson's Rule for Numerical Integration

Composite Simpson Rule

 Our next method of finding the area under a curve is by approximating that
curve with a series of parabolic segments that lie above the intervals
. When several parabolas are used, we call it the composite Simpson rule.

Theorem (Composite Simpson's Rule) Consider over . Suppose that

the interval is subdivided into subintervals of equal

width by using the equally spaced sample points

for . The composite Simpson's rule for subintervals is

This is an numerical approximation to the integral of over and we write

Simpson's Rule for Numerical Integration Simpson's Rule for Numerical

Integration

Remainder term for the Composite Simpson Rule

Corollary (Simpson's Rule: Remainder term) Suppose that is subdivided

into subintervals of width . The composite Simpson's

rule

is an numerical approximation to the integral, and

Furthermore, if , then there exists a value with so that the

error term has the form

This is expressed using the "big " notation .

Remark. When the step size is reduced by a factor of the remainder term

should be reduced by approximately .

Algorithm Composite Simpson Rule. To approximate the integral

by sampling at the equally spaced sample points

for , where . Notice that and .

Example 1. Numerically approximate the integral by using

Simpson's rule with m = 1, 2, 4, and 8.

Example 2. Numerically approximate the integral by using

Simpson's rule with m = 10, 20, 40, 80, and 160.

Example 3. Find the analytic value of the integral (i.e. find the
"true value").

Example 4. Use the "true value" in example 3 and find the error for the Simpson rule
approximations in example 2.

Example 5. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Example 6. Numerically approximate the integral by using
Simpson's rule with m = 1, 2, 4, and 8.

Example 7. Numerically approximate the integral by using

Simpson's rule with m = 10, 20, 40, 80, and 160 subintervals.

Example 8. Find the analytic value of the integral (i.e. find

the "true value").

Example 9. Use the "true value" in example 8 and find the error for the Simpson rule
approximations in example 7.

Example 10. When the step size is reduced by a factor of the error term

should be reduced by approximately . Explore this phenomenon.

Various Scenarios and Animations for Simpson's Rule.

Example 11. Let over . Use Simpson's rule to

approximate the value of the integral.

Module

for

Riemann Sums, Midpoint Rule and Trapezoidal Rule

Definition. Definite Integral as a Limit of a Riemann Sum. Let be
continuous over the interval ,
and let be a partition, then the definite integral is given
by

where and the mesh size of the partition goes to zero in the "limit," i.e
. .

Riemann Sums Riemann Sums

Animations (Riemann Sums Riemann Sums).

Animations (Trapezoidal Rule Trapezoidal Rule).

Computer Programs Riemann Sums Riemann Sums

The following two Mathematica subroutines are used to illustrate this concept, which
was introduced in calculus.

Example 1. Let over . Use the left Riemann sum with n = 25, 50,
and 100 to approximate the value of the integral.

Example 2. Let over . Use the right Riemann sum with n = 25,
50, and 100 to approximate the value of the integral.

Example 3. Let over . Use the midpoint rule with n = 25, 50, and
100 to approximate the value of the integral.

Example 4. Let over . Use the trapezoidal rule with n = 25, 50,
and 100 to approximate the value of the integral.
.
Example 5. Let over . Compare the left Riemann sum, right
Riemann sum, midpoint rule and trapezoidal rule for n = 100 subintervals. Compare
them with the analytic solution.

Example 6. Let over .

6 (a) Find the formula for the left Riemann sum using n subintervals.
6 (b) Find the limit of the left Riemann sum in part (a).

Example 7. Let over .

7 (a) Find the formula for the right Riemann sum using n subintervals.
7 (b) Find the limit of the right Riemann sum in part (a).

Example 8. Let over .

8 (a) Find the formula for the midpoint rule sum using n subintervals.
8 (b) Find the limit of the midpoint rule sum in part (a).

Example 9. Let over .

9 (a) Find the formula for the trapezoidal rule sum using n subintervals.
9 (b) Find the limit of the trapezoidal rule sum in part (a).

Various Scenarios and Animations for Riemann Sums.

Example 10. Let over . Use the Left Riemann Sum to

approximate the value of the integral.

Example 11. Let over . Use the Right Riemann Sum

to approximate the value of the integral.
The following animations for the lower Riemann sums are included for illustration
purposes.

Example 12. Let over . Use the Lower Riemann Sum

to approximate the value of the integral.

The following animations for the upper Riemann sums are included for illustration
purposes.

Example 13. Let over . Use the Upper Riemann Sum

to approximate the value of the integral.

Example 14. Let over . Use the Midpoint Rule to

approximate the value of the integral.

Example 15. Let over . Use the Trapezoidal Rule to

approximate the value of the integral.

Module

for

Tri-Diagonal Linear Systems

Background

In applications, it is often the case that systems of equations arise where the
coefficient matrix has a special structure. Sparse matrices which contain a majority of
zeros occur are often encountered. It is usually more efficient to solve these systems
using a taylor-made algorithm which takes advantage of the special structure. Important
examples are band matrices, and the most common cases are the tridiagonal matrices.
Definition (Tridiagonal Matrix). An n×n matrix A is called a tridiagonal matrix
if whenever . A tridiagonal matrix has the for

(1) .

Theorem Suppose that the tri-diagonal linear system has

and for . If
, and , and , then
there exists a unique solution.

Proof Tri-Diagonal Matrices Tri-Diagonal Matrices

The enumeration scheme for the elements in (1) do not take advantage of the
overwhelming number of zeros. If the two subscripts were retained the computer would
reserve storage for elements. Since practical applications involve large values
of , it is important to devise a more efficient way to label only those elements
that will be used and conceptualize the existence of all those off diagonal zeros. The
idea that is often used is to call elements of the main diagonal , subdiagonal
elements are directly below the main diagonal, and the superdiagonal
elements are directly above the main diagonal. Taking advantage of the single
subscripts on we can reduce the storage requirement to merely
elements. This bizarre way of looking at a tridiagonal matrix is

Observe that A does not have the usual alphabetical connection with its new elements
named , and that the length of the subdiagonal and superdiagonal is n-1.
Definition (tri-diagonal system). If A is tridiagonal, then a tridiagonal system is

(2)

A tri-diagonal linear system can be written in the form

(3)

=
=
=

=
=

Goals

To understand the "Gaussian elimination process."

To understand solutions to"large matrices" that are contain mostly zero elements, called
"sparse matrices."
To be aware of the uses for advanced topics in numerical analysis:
(i) Used in the construction of cubic splines,
(ii) Used in the Finite difference solution of boundary value problems,
(iii) Used in the solution of parabolic P.D.E.'s.

Algorithm (tridiagonal linear system). Solve a tri-diagonal system given in

(2-3).
Below diagonal elements:

Main diagonal elements:

Above diagonal elements:

Column vector elements:

Computer Programs Tri-Diagonal Matrices Tri-Diagonal Matrices

Mathematica Subroutine (tri-diagonal linear system).

Example 1. Create a 31 by 31 tridiagonal matrix A where the main diagonal elements

are all "2" and all elements of the subdiagonal and superdiagonal elements are
"1." Then solve the linear system
where

. Compute the solution using decimal arithmetic and rational arithmetic.

Solution 1 (a).
Solution 1 (b).

Observation. The 31×31 matrix A in Example 1 has 961 elements, 870 of which are
zeros. This is an example of a "sparse" matrix. Do those zeros contribute to the
solution ? Since the matrix is tri-diagonal and diagonally dominant, there are other
algorithms which can be used to compute the solution. Our tri-diagonal subroutine will
work, and it requires only 12.3% as much space to save the necessary information to
solve the problem. In larger matrices the savings will be even more. It is more efficient
to use vectors instead of matrices when you have sparse matrices that are tri-diagonal.

Example 2. Solve the 31 by 31 system in Example 1 using our

TriDiagonal[a,d,c,b] subroutine. Compute the solution using decimal
arithmetic and rational arithmetic.

Example 3. Solve the 31 by 31 system in Example 1 using Mathematica's built

in TridiagonalSolve[a,d,c,b] procedure. Compute the solution using
decimal arithmetic and rational arithmetic.

Conclusion. If the matrix is tri-diagonal this structure should be considered because

the stored zeros do not contribute to the accuracy of the solution and just waste space in
the computer. Since tri-diagonal systems occur in the solution of boundary value
differential equations, practical problems could easily involve tri-diagonal matrices of
dimension 1000 x 1000. This would involve 1,000,000 entries if a square matrix was
used but only 2998 entries if the vector form of the tri-diagonal matrix is used.

Example 4. Solve the 31 by 31 system in Example 1 using the inverse matrix

and the computation .

Research Experience for Undergraduates

Tri-Diagonal Matrices Tri-Diagonal Matrices Internet hyperlinks to web sites and a

bibliography of articles.

Download this Mathematica Notebook Tri-Diagonal Linear Systems

Return to Numerical Methods - Numerical Analysis

(c) John H. Mathews 2004
(c) John H. Mathews 2004

(c) John H. Mathews 2004

(c) John H. Mathews 2004