Chapter 1

INTRODUCTION TO NUMERICAL METHODS

AND ERROR ANALISIS
Engineer is the one who converts problems into solutions.- zar98

Outcomes:

After completing Part One, students should be able to

1. describe numerical techniques as compared to analytical methods

2. use Taylor series expansion to approximate a function
3. perform error analysis associated with numerical methods

Scope and Preview: In this part, we will investigate how numerical methods are used to
approximate the true solution of a problem. An analytical method provides an exact solution.
The reliability of the numerical method will be analyzed by performing an error analysis. Taylor
series expansion (TSE) of a function in one variable will be discussed in details. This is
particularly important since TSE is used as a basis of approximation in the numerical analysis.

1.1 Introduction to numerical methods and mathematical modeling. Numerical

methods provide approximate solutions to many of the engineering problems. The results
obtained may used to assist decision-making process in engineering design. There are many
applications of numerical methods: finding root(s) of a function, solving linear algebraic
equations, fitting curves to data, interpolating the function, performing numerical differentiation
and integration, and solving ordinary and partial differential equations. Various algorithms are
available with each has its own advantages and limitations.

1.2 Error Analysis. Error analysis is very important especially when numerical methods are
implemented on computers. We will study sources of error in numerical methods and how the
error is quantified.

1.3 Approximation of a function. TSE will be used to approximate the value of the
function at any point of interest. The error associated with this approximation will be defined.
Application in numerical differentiation of a function will be introduced.
1

Introduction to Numerical Methods

and Error Analysis

1.0

Numerical Methods
with Applications

INTRODUCTION

In engineering and sciences, we rarely come across simple and straight forward situations which
can be solved analytically. Often the problems are complex that analytical solutions are either
extremely difficult or impossible to obtain. Furthermore when the constraints are imposed on the
problem, the complexity also increases. Therefore, numerical methods provide an alternative
means of approximating the true solution via computer implementation. They are sometimes
referred to as numerical analysis. Note, however, the numerical solution by no means should
replace the analytical solution if it can be easily and practically obtained. The analytical method
provides continuous solution of the problem while the numerical method only gives solution at
discrete points.
Numerical computations combine two important tools in engineering problem solving: 1)
Mathematics, and 2) computers. They influence the practices of engineering design and modeling
process in the present world. Therefore, in order to be competence in solving engineering
problems, a person must enhance his or her knowledge and skills of these tools.

1.1
1.1.1

MATHEMATICAL MODELING AND ENGINEERING SOLUTION

Mathematical Modeling:

Modeling Process: Model is referred to as a mathematical expression relating independent,

dependent variables, parameters, and forcing functions. The variables may be physical,
engineering, and economic variables and etc. In general, we may express the mathematical model
in the form

qo f (qi , c, u)
where

q0
qi
c
u

= dependent variable
= independent variable(s)
= parameter(s)
= forcing function(s)

Lets consider a body subjected to a force as shown in Figure 1.1,

x
F(t)

Figure 1-1: A body subjected to force.

Applying Newtons second law and neglecting the friction, we may write
2

(1-1)

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

F ma

(1-2)

and for one-dimensional problem (as in this case), we can write eq(1-2) as

or

dv
F
dt

(1-3)

d 2x
F
dt 2

(1-4)

We see that eq.(1-4) relates a linear position x (dependent variable) to the applied force F
(forcing function) and the time t (independent variable). This relation forms a mathematical
model of a body being acted upon by a force in one dimension. The mass m is the only parameter
in the above model. The complexity of the model increases if we decide to include the effect of
friction and other considerations such as wind resistance.
Once the model is obtained, numerical method is employed in predicting and obtaining the
information about the performance or behaviors of the model. Remember that the method only
provides approximate information of the model.

1.1.2

Analytical vs. Numerical Methods

Consider a free falling object when it is released from rest at time t = 0 s. Neglecting the air
resistance, the equation of motion is given by

y y0 v0t 21 gt 2
where

(1-5)

g = gravitational acceleration = 9.81 m/s2

v0 = initial velocity (= 0 in this case i.e. starts from rest)

Analytical solution of the velocity is given by

dy
v v0 gt
dt

(1-6)

v = 0 + 9.81t
= 9.81t
We can solve for the velocity at any time t, for example, at t = 2.0 s,
v=

9.81(2.0) = 19.61 m/s (True solution)

Numerical solution may be expressed using one of the standard formulas, such as forward finite
difference as follows.

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

vi

yi 1 yi
ti 1 ti

(1-7)

In order to use this formula, we need to have the values of the position y at the discrete time t (at
least at two different points with one being at ti and the other at ti+1). For example, the position y
may be tabulated at time t as shown below
t (s)
y (m)

0
0

2
19.62

3
44.145

Then, from eq.(1-7)

at ti = 2.0 s,

vi

44.145 19.62
= 12.2625 m/s
3 2

Repeat the calculation but with smaller time steps or intervals,

t (s)
y (m)

2
19.62

2.5
30.65625

v (t = 2) = 22.0725 m/s
t (s)
y (m)

2
19.62

2.1
21.63105

v (t = 2) = 20.1105 m/s
t (s)
y (m)

2
19.62

2.05
20.61326

v (t = 2) = 19.86525 m/s
We observe that the numerical solution approaches the true value when the step size is reduced.
However, there is a limit to how small could the step size be. This is especially important when a
computer is used in the solution procedures.

(iii) Advantages of Numerical Methods

1. Provide extremely powerful problem-solving tools with the ability to handle large system of
equations, nonlinearities in the models, and complicated geometry.
2. Reinforce understanding of mathematics in describing the physical phenomena with the use of
computers.
3. Provide an insight and opportunity to analyze solution(s) of the problem from many different
4

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

perspectives.
4. Reduce the amount of time used in arithmetic operations.
However, we need to understand the limitations and pitfalls of each numerical methods before we
can effectively and efficiently employ them to solve problems with acceptable accuracy. There
are several factors that influence a decision-making process in choosing one particular method
over the others.
1. complexity of the problem
2. societal importance of the problem
3. solutions are unbiased and precise
4. cost effective
5. minimal environmental impacts.

In general, engineering problem solving and analysis can be represented as shown in the diagram
below.
Problem
definition
Theory

Mathematical
model

Experimentation

Performance verification:
analytical/numerical methods,
statistics, computers, graphics, etc.

Economic and societal interfaces:

optimization, ,design , scheduling,
communication, environmental
impacts, etc.

Implementation

Figure 1-2: Engineering problem-solving process

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

1.2 Error Analysis

(i) Significant Digits (s.ds) - those digits (or numbers) that can be used with confident plus one
estimated digit. For example:
100

110

120

metal sheet

Figure 1-3: Length of a metal sheet

The length of the metal sheet is read as 116.5 units (4 s.ds) which contains one estimated digits.
If we were to read the length as 116.57 units, then the last digit which is 7 is not significant.
Suppose the electric meter indicates the electric power consumption as shown,

3
4

watts

Figure 1-4: Electrical power used.

The reading to the nearest watt is recorded as 52970.35 watts (7 s.ds). It would be incorrect to
read the meter as 52970.348 watts which now has two doubtful digits. The following table
illustrate more about the number of significant digits contain in the numbers.
Number
2.347
1.652 0
189.708
0.001 2
0.001 200

Significant Digits
4
5
6
2
4

The reasonable number of significant digits in calculations should be determined from the
relative error required in the final answer. *Rule does not apply to conversion factors and exact
numbers.

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

(ii) Accuracy and Precision. Precision in numerical method is referred to the consistency of the
estimates to be closed to each other. On the other hand, accuracy is the measure of closeness of
the estimate to the target.
Increasing Accuracy

(a)

(b)

(c)

(d)
Increasing Precision

Figure 1.5: Accuracy and precision

The accuracy of the numerical computation depends on a particular method used and on its ease
of implementation on the computer (i.e. accuracy and limitations of the computer). A computer
is used for two functions in regard to numerical computation: 1) to store numbers and 2) to
perform arithmetic operations. Computer can only store a finite number of digits. The remaining
digits are either chopped off or rounded off. This finite number representation in the computer
leads to an unavoidable error called a round-off error (REr).
Example of a computer program that supposedly reads and writes the same number:

C
C
C
C

Program Show Error

This program compares input and output number.
READ(11, 1) A
WRITE(10, 1) A
FORMAT(2X, F15.13)
STOP
END

Programming result,
input

0.1234567890123

output :

0.1234567910433
7

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

Figure 1-6: Round-off error in a computer.

In performing arithmetic operations, only certain digits are usually retained in order to get an
approximate solution. Several digits are truncated or omitted. These truncated digits lead to what
is called a truncation error (TEr).
Using the above programming result, the output A may be retained only up to 5 decimal places
(d.ps),
input

0.1234567890123

output :
Remark:

0.12346 (5 d.ps)

1. If any solution is sought with too much precision, REr will be dominant.
Otherwise, lack of mathematical accuracy (TEr) prevails.
2. REr and TEr constitute a limit to how accurately a solution can be obtained by
implementing numerical methods on computers.

Recall the approximation of the velocity v (which is essentially the first derivative of the
function y ), we may express the general form of the first derivative for any function f(x) at the
base point x0 using first finite forward difference formula as

mapprox f ' ( x0 )

f ( x0 h ) f ( x0 )
h

(1-8)

and the true value (analytical solution) is represented by a slope of the tangent line at x0 which
may be shown in the figure below,

mapprox

mtrue

f(x)

x0

x0 + h

Figure 1-7: First derivative of a function.

8

Numerical Methods
with Applications

We observe that when

Introduction to Numerical Methods

and Error Analysis

mapprox less accurate

mapprox more accurate

shows TEr.

when h is too small, REr dominates. This observation suggests that there must be an optimum
value of h .
Summary:

1. REr is an error introduced in representation of a given number by a computer.

It usually becomes a limiting factor when small changes in certain input
parameters result in relatively large deviations of output. It is likely to occur
when the relative magnitude of the numbers involved in the calculations
differs significantly and when two nearly identical numbers are subtracted
from each other.
2. TEr is an error introduced by approximating some ideal mathematical
functions in a simpler form. It usually stems from the computational method
used and is dictated by the desired accuracy in the computations.

(iii) Error Quantification. When the true value of the quantity is known, the error is quantified
in two different but related ways.
Let x denote an exact (or true) value and x* denote an approximate (or estimated) value or
simply an approximation. We may then express the relation,

or

True value = Approximation + Error

(1-9a)

Error = True value - Approximation

(1-9b)

which defines the error as the difference between the true value and the approximation.
(True) Absolute error:
(1-10)

Et x x *

Note that the absolute error does not reflect the degree of its influence.
For example,

Et of 0.01m in measurement of the distance to the moon and,

Et of 0.01m in diameter of a piston in a cylinder

do not suggest the significant or scale of the error, though, both have same the value of the
absolute error. In view of this limitation, the second way of quantifying the error is introduced.

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

(True) Relative error:

Et , rel

x x*

(1-11a)

Note that the absolute error is normalized or scaled with respect to (or relative to) the true value
so that the influence of the error can be significantly interpreted. Usually the relative error is
expressed in terms of percentage, that is

(True) Percent relative error :

x x*
x

x100%

(1-11b)

(iv) Error bound and Its Propagation. Any number (x*) satisfying the inequality

( x*) x x *

(1-12)

is called an error bound or an absolute error bound for the error of x* as an approximation of
x.
Remark: all numbers larger than (x*) also serve as error bounds.
Similarly, Any number (x*) satisfying the inequality
( x*)

x x*

(1-13)

is called a relative error bound .

Suppose an absolute error bound is known, we may select
( x*)

( x*)
x

(1-14)

( x*)
x*

(1-15)

and if (x*) <<< x*, then

( x*)

which is a good approximation of a relative error bound (x*).

10

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Error Propagation in a Function

Next we would like to estimate the effect of the discrepancy between the true value and the
approximate value on the value of the function. This is called error propagation.
If a function f is dependent on
(a)
a single independent variable x : f(x)
(b)
two independent variables x and y : f(x, y)
(c)
several independent variables x1 , x2 , x3 , ... ,xn : f(x1 , x2 ,..., xn ).
Lets consider a function of a single variable f(x).
Let

x be the true value and

x* be an approximate value of x

Then, TSE for f(x) computed near f(x*) is given by

f ( x) f ( x*) f ' ( x*)( x x*)

f ' ' ( x*)

( x x*) 2 ...
2

(1-16)

Truncating after the first derivative term and rearranging the remaining terms to give

f ( x) f ( x*) f ' ( x*)( x x*)

(1-17a)

f ( x*) f ' ( x*) x *

where

(1-17b)

f ( x*) f ( x) f ( x*) is an estimate of the error of the function.

x* x x * is an estimate of the error of independent variable x.

Eq.(1-17) provides 2 capabilities:

1. to approximate the error in f(x) knowing its derivative.
2. to approximate the error in the independent variable x.
Similarly, for a function of n independent variables f(x1 , x2 ,..., xn ), we may write
f ( x1 *, x2 *,..., xn *)

where

f f
f
,
,...,
x1 x 2
x n

f
f
f
x1 *
x2 * ...
xn *
x1
x2
xn

(1-18)

are partial derivatives of the function evaluated at points x1*, x2*,...,

and xn*.

11

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

Condition number (Nc ): It provides a measure of the extent to which a change in x affects a
change in f(x). A function is said to be ill-conditioned if a small change (i.e. perturbation) in x
causes a large change in f(x). Nc is the ratio of the relative error of f(x) and x
x * f ' ( x*)
Nc
f ( x*)

1
1

relative error for f ( x ) and x is identical.

relative error for f ( x) is amplified .
(1-19)
relative error for f ( x) is attenuated .

where estimate of the relative error of f(x) and x is given,repectively, by

f ( x)

( x)

and

f ( x) f ( x*) f ' ( x*)( x x*)

f ( x*)
f ( x*)

(1-20)

x x*
x*

(1-21)

Error estimate of the function.

Example 1.1:

Knowing a value of x* = 2.0 with an error of x* = 0.01, estimate the resulting

error in the function
f ( x ) 0.5 x 3 0.1x 2 0.8 x 0.7

Solution:

At x* = 2.0 , f (2.0) 0.5(2.0) 3 0.1(2.0) 2 0.8(2.0) 0.7 = 4.5

f ' ( x ) 1.5 x 2 0.2 x 0.8

Using eq.(1-17) with the first derivative,

f(x*) = f(2.0) [1.5(2.0)2 - 0.2(2.0) + 0.8](0.01) = 0.064

Therefore, we predict that the function value at x* = 2.0 would be in the range
f(2.0) = 4.5 0.064

or

4.436 < f(2.0) < 4.564 (error bounds)

12

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Error estimate of the function.

Example 1.2:

For a beam with one end fixed and the other end free (as shown below), the
deflection of the beam y is given by
F

FL4
8 EI

where E is modulus of elasticity and I is mass moment of

inertia. Knowing the approximate values of the variables and
their associated errors as

L
y

F* = 120 lbf/ft
L* = 27 ft
E* = 1.65(108) lbf/ft2
I* = 0.072 ft4

F* = 5 lbf/ft
L* = 0.1 ft
E* = 8.0(105) lbf/ft2
I* = 0.0005 ft4

estimate the resulting error in the deflection y.

Solution:

The deflection y is
y*

(120)(27) 4
8(1.65x108 )(0.072)

= 0.6710 ft (4 d.ps)

Using eq.(1-18) with the first partial derivative,

y(F*, L*, E*, I* )

y
y
y
y
F *
L *
E * I *
F
L
E
I

L4
FL3
FL4
FL4
F *
L *
E *
I *
2
8 EI
2 EI
8E I
8 EI 2

0.027959 + 0.009941 + 0.003253 + 0.004660

0.045813
Therefore, we predict that the deflection y to be in the range
y = 0.6710 0.045813 ft

13

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

(v) Error Quantification in Numerical Methods: In the absent of the true value (which is
usually the case in numerical methods), the following error calculation is employed.
Let

xnew denote
xold denote

a present (or new) approximation.

a previous (or old) approximation.

We may express the relation,

Present approximation = Previous approximation + Approximate Error

(1-22a)

or Approximate Error = Present approximation - Previous approximation

(1-22b)

(Approximate) Relative error:

x new x old
Ea

(1-23a)

x new

or in the percentage form, the approximate percent relative error is

x new x old

x new

x100%

(1-23b)

In employing the numerical methods, particularly the iterative approaches, we are basically
interested in whether the absolute value of the approximate percent relative error a is less than
the prespecified tolerance s (or desired accuracy) that is

a s

(1-24)

The desired accuracy or prespecified tolerance s is defined as

s (0.5x10 2 p )%

(1-25a)

E s (0.5x10 p )

(1-25b)

or the acceptable error:

If eq.(1-24) is satisfied, the approximation is said to be accurate at least to p decimal places

(d.ps)!!! For example, for
p=1
p=2

1 d.p accuracy
2 d.ps accuracy

a s
a s

= 0.5x101 = 5%
= 0.5x100 = 0.5%
14

Numerical Methods
with Applications

p=3

Introduction to Numerical Methods

and Error Analysis

a s

3 d.ps accuracy

= 0.5x10-1 = 0.05%

Note: It is the user who determines the desired accuracy or the acceptable error. This usually
depends on the importance of the problem and its intended ends.
Error estimates of series approximation.
Example 1-3:

Exponential function using the Maclaurin1 Series expansion is given by

x 2 x3
xn
e x 1 x

...
2! 3!
n!
Approximate the function at x = 0.5 with an accuracy to 3 d.ps and also calculate the true
percent relative error.

Solution:

e0.5 1.648721271 = 1.64...

True value,

#
Equivalent expression and value
Terms
1
ex 1
2

e x 1 x = 1 + 0.5 = 1.5

ex 1 x

4
5
6

x2
2!
x2
ex 1 x
2!
x2
ex 1 x
2!
x2
ex 1 x
2!
y

t %
1.64..1 100% 39.3
1.64..
1.64. .15
. 100% 9.02
1.64. .
1.64. .1.625 100% 1.44
1.64. .

= 1.5 + 0.52/2

x3
3!
x 3 x4

3! 4!
x 3 x 4 x5

3! 4! 5!

a %
15
. 1 100%
15
.

1.6251.5 100% 7.69

1.625

0.175

1.27

0.0172

0.158

0.00142

0.0158

Note: p = 3 => s = 0.05%

ex
3 terms

We stop calculation since a < s .

The approximate value is then

2 terms
1.0

e0.5 1.6487 (accurate to 4 d.ps)

1 terms

x
1

Colin Maclaurin (1698-1746, 48), Scottish mathematician.

15

33.3

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

Error estimate of the function.

Worksheet 1.3:

Maclaurin series for sin(x) is given by

sin( x) x

x 3 x5
x 2 n 1
. . . ( 1) n
3! 5!
(2n 1)!

n = 0,1,2,3,...

Approximate the function at x = /2 radians with an accuracy to 4 d.ps.

Solution:

True value,

sin(/2) = 1.0

For p = 4,

s (0.5x102 4 )% 0.005%

sin(/2) 1.570796

a = 57.1%

sin(/2) 1.570796 0.645964 = 0.924832

a = 7.52%

sin(/2) 0.924832 + 0.079693 = 1.004525

a = 0.453%

sin(/2) 1.004525 0.004682 = 0.999843

a = 0.0157%

sin(/2) 0.999843 + 0.000160 = 1.000003

a = 0.00003%

Since a < s , we stop the calculation.

16

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

1.3 Function Approximation

(i) Taylor2 series expansion (TSE)
If a function f and its first n+1 derivatives are continuous on the interval [x0, x], the value of
the function at x is then given by
f ( x ) f ( x0 )

f ' ( x0 )
f ' ' ( x0 )
f ( n ) ( x0 )
( x x0 )
( x x0 ) 2 ...
( x x0 ) n Rn
1!
2!
n!

(1-26)

which is known as the Taylor series for f about x = x0.

x x0 h

Defining

or

h x x0 step size,

we can express eq.(1-26) as

f ' ( x0 )
f ' ' ( x0 ) 2
f ( n ) ( x0 ) n
f ( x ) f ( x0 )
h
h ...
h Rn
1!
2!
n!
where

(1-27)

x0 the base point (or starting value)

h the step size (or step increment)
Rn the remainder (or error term)

The remainder is defined as

x

Rn

(x )
f
n!
x0

( n 1)

( ) d

(1-28)

where is a dummy variable. Note that Maclaurin series is a special form of Taylor series for
which x0 = 0.
2

Brook Taylor (1685-1731,46), English mathematician.

17

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

In a form convenient for computer implementation, eq.(1-27) can be written as

n

f ( x)

k 0

f ( k ) ( x0 ) k
h Rn
k!

(1-29)

Note that 0! = 1.
From the second theorem of mean for integrals, eq.(1-28) yields

Rn

f ( n 1) ( ) n 1
h
(n 1)!

where

x0 x

(1-30)

which is called the derivative or the Lagrange3 form of the remainder with being an unknown
value. Other formulas for Rn are also available. They are available in most advanced calculus
text.

(ii) Order of TSE:

zero-order

f ( x ) f ( x0 )

first-order

f ( x ) f ( x0 )

f '( x0 )
h
1!

(1-32)

second-order :

f ( x ) f ( x0 )

f '( x0 )
f ' ' ( x0 ) 2
h
h
1!
2!

(1-33)

nth-order

f '( x0 )
f '' (x0 ) 2
f ( n ) ( x0 ) n
f ( x ) f ( x0 )
h
h ...
h
1!
2!
n!

(1-31)

(1-34)

Joseph Louis Lagrange(1736-1813, 77), French-Italian mathematician and astronomer. A

successor of Leonhard Euler(1707-1783, 76)-born in Switzerland as director of Berlin Academy
in(1776).

18

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Taylor series approximation of a polynomial.

Example 1-4: Use zero- through third-order TSE to approximate the function

f ( x) 0.5x3 01
. x2 0.8x 0.7
at x = 1.0 and the base point (a) x0 = 0 and (b) x0 = 0.8. Also calculate both the true and
approximate percent relative errors for each order.

Solution:

True value,

(a) with h = x - x0 = 1.0,

order
0th
1st
2nd
3rd

f (1) 05
. (1)3 01
. (1) 2 08
. (1) 0.7 = 0.5
f(x) = 0.5x3 -0.1x2 +0.8x - 0.7 f(0) = -0.7
f(x) = 1.5x2 -0.2x +0.8
f(0) = 0.8
f(x) = 3.0x -0.2
f(0) = -0.2
f(x) = 3.0
f(0) = 3.0
(4)
f (x) = 0.0

Expansion and Value

f (1) f (0) 0.7

(0.8)
f (1) 0.7
(1.0) 01
.
1!
(0.2 )
f (1) 01
.
(1.0)2 0.0
2!
(3.0)
f (1) 0.0
(1.0)3 05
.
3!

t %
0.5 ( 0.7 )
100% 240
0.5
0.5 ( 01
.)
100% 80
0.5

a
01
. (0.7 )
100% 800
01
.

100

100

We observe that though, the true percent relative error is zero, the approximate percent relative
error is 100%. In practice, when using numerical method (in the absent of the true value), we
will continue with higher order approximation(s) that is
4th-order approximation:

f (1) 0.5

( 0)
(10
. ) 3 0.5
4!

19

and a = 0%.

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

(b) with h = x - x0 = 0.2,

Using the derivatives from part (a), we obtain
f(0.8) = 0.132
f(0.8) = 1.6
f(0.8) = 2.2
f(0.8) = 3.0
f(4)(x) = 0.0
order

Expansion and Value

0th

f (1) f (0.8) 0132

.

1st

f (1) 0132
.

2nd
3rd

(16
. )
( 0.2 ) 0.452
1!
( 2.2 )
f (1) 0.452
( 0.2 ) 2 0.496
2!
(3.0)
f (1) 0.496
( 0.2 ) 3 05
.
3!

t %

a %

73.6

9.6

70.8

0.8

8.9

0.8

In practice, we usually require the result to be within a specified tolerance. For example, we may
want the result to be accurate at least to 3 d.ps. If this is the case, then we will have to continue
with higher order approximation(s).
p = 3,

s = 0.05%

4th-order approximation:

f (1) 0.5

( 0)
(10
. ) 3 0.5
4!

Since a < s , we stop the calculation.

20

and a = 0%.

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Taylor series approximation of a polynomial.

Worksheet 1-4:

Use TSE to approximate the function

f ( x ) x 4 0.7 x 3 2 x 2 x 62.5
at x = 5.0 and the base point x0 = 4.5 with an accuracy to 3 d.ps. Also calculate the true percent
relative error for each of the orders.

Solution:

f (5) (5) 4 0.7(5)3 2(5)2 (5) 62.5 = 420

True value,

h = x - x0 = 0.5, f(x) = x4 - 0.7x3 -2x2 - x - 62.5 f(4.5) = 238.775

f(x) = 4x3 - 2.1x2 -4x - 1
f(4.5) = 302.975
f(x) = 12x2 - 4.2x - 4
f(4.5) = 220.1
f(x) = 24x - 4.2
f(4.5) = 103.8
(4)
f (x) = 24
f(4)(4.5) = 24
order

Expansion and Value

t %

43.1

( 302.975)
( 0.5)
1!

7.08

38.8

(2201
.)
( 0.5) 2
2!

0.530

6.59

(1038
. )
(0.5)3
3!

0.0149

0.515

( 24)
( 0.5) 4
4!

0.0

0.0149

0th

f (5) f ( 4.5) 238.775

1st

f (5) 238.775
= 390.2625

2nd

f (5) 390.2625
= 417.775

3rd

f (5) 417.775
= 419.9375

4th

f (5) 419.9375
= 420

21

Introduction to Numerical Methods

and Error Analysis

Remark:

Numerical Methods
with Applications

1. The function and its n+1 derivatives are assumed to be continuous in the
interval of interest [x0, x].
2. Order of approximation is the highest order of the derivatives.
3. Accuracy of approximation improves as order of TSE increases or when the
step size h decreases.
4. Higher order terms become apparent when the nonlinearities of the function
increases or when the step size increases.
5. When the function is linear (i.e. a polynomial), the first-order TSE with
reasonably small step size is usually adequate.
6. TSE is used when the function is not known or in order to obtain approximate
solutions for trigonometric and logarithmic functions.

In practice, we may use the known function to illustrate the practicality of the TSE.

(iii) Remainder, Rn
Recall the derivative form of the remainder, eq.(1-30),

Rn

f ( n 1) ( ) n 1
h
( n 1)!

There are two major drawbacks associated with this formula:

1. is not known exactly. It lies only in between x0 and x.
2. In order to evaluate Rn, we need to know the (n+1)th derivative of f(x) i.e. function f(x) must
be known. But if we knew f(x), it is no need to use TSE. We can use the function directly to
get the exact value.
Despite these two drawbacks, we do however have control over two things:
1. the term hn+1 i.e. we can choose how far away from x0 we want to evaluate f(x).
2. the number of terms to be included in TSE.

22

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Consequently, Rn can be expressed as

Rn ( h n 1 )

(1-35)

which implies that the truncation error is of the order of hn+1. In other words, it is proportional
to the step size h raised to the power of (n+1). It enables us to judge the comparative error of
numerical methods based on TSE.
For example,

Rn (h) : halving the step size h, will halve the error.

when

Rn ( h 2 ) : halving the step size h, will quarter the error.

Remark:

In general, we assume that the truncation error is decreased with addition of more
terms in the TSE and with smaller step size h.

We may also gain some other insights about the remainder. Lets define an upper bound on the
magnitude of the reminder Rn such that

Rn Rn ,u

(1-36)

with

Rn ,u
where

M
h n 1
( n 1)!

and

f ( n 1) () M

(1-37,38)

Rn,u is an upper bound on the magnitude of the reminder Rn

M is an upper bound on the magnitude of the (n+1)th derivative at .
Upper bound of the remainder.

Example 1-5:

Recall the exponential function using the Maclaurin series example 1-3,

ex 1

x2 x3
xn
...
2! 3!
n!

Define an upper bound of the remainder and determine the number of terms n so that the
approximation about x = 1.0 is accurate to 1 d.p.

23

Introduction to Numerical Methods

and Error Analysis

Solution:

Note

Numerical Methods
with Applications

x0 = 0.

The remainder

Rn

f ( n 1) () n 1
e
h
x n 1
( n 1)!
( n 1)!

For x = 1.0, we have

f (1) e1 1 1

1 1
1
e
...
2! 3!
n! ( n 1)!

where 0 < < 1

Defining,

Rn

e
e

( n 1)! ( n 1)!

For an upper bound,

< 1 and consequently e < e1 = e = M.

Therefore, the upper bound on the magnitude of the remainder is

Rn

e
e
Rn,u
(n 1)!
(n 1)!

To gain meaningful interpretation of eq.(E1.5-1), let e < 3, and for p = 1,

Rn

3
(0.5x10 p )
( n 1)!

By trial and error, we have n = 4.

24

(E1.5-1)

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

TSE for a function with an infinite number of derivatives.

Example 1.6:

Use TSE from zero- through fifth-order to approximate the function,

f ( x)

1
x

at x = 2.0 with x0 = 1.0. Evaluate the true percent relative error for all cases.

Solution:
Derivatives,

True value,

f(2.0) = 0.707107 (accurate up to 6 d.ps).

1 3
f '( x ) x 2
2
3 5 2
f ''( x ) x
4
15 7
f '''( x ) x 2
8
105 9 2
(4 )
f ( x)
x
16
945 112
f ( 5) ( x )
x
32

=> f(1.0) = - 0.5

=> f(1.0) = 0.75
=> f(1.0) = - 1.875
=> f (4)(1.0) = 6.5625
=> f(5)(1.0) = - 29.53125

with h = x - x0 = 2.0 - 1.0 = 1.0,

Order of approximations
0th:

f(2.0)

1st:

f(2.0) 1.0 + (-0.5)h

41.4

1.0

29.3

1.0 + (-0.5)(1.0) 0.5

2nd:

t %

f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2

0.5 + (0.375)(1.0)2 0.875

23.7

3rd: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3

0.875 + (-0.3125)(1.0)3 0.5625

25

20.5

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

4th: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4

0.5625+ (0.2734375)(1.0)4 0.8359375

18.22

5th: f(2.0) 1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4 +

(-29.53125/120) h 5
0.8359375 + (-0.24609375)(1.0)5 0.58984375
6th, 7th, 8th:

0.815427, 0.605954, 0.802335

16.58

15.32, 14.30,
13.47

Note: Although (in this example), the addition of more terms will further reduce the error, but
there is no significant improvement. It suggests that an infinite number of terms is required in
order to get the approximation closed enough to the true value.
This is true for most of the function with
(1) an infinite number of the derivatives and
(2) relatively large step size.
In addition, for some functions, the solution may indeed diverge. This depends on the interval of
validity of x for that function.

26

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

TSE for a function with an infinite number of derivatives.

Worksheet 1.5:
Solution:
Now

Repeat example 1.6 but with x0 = 1.5.

True value,

f(2.0) = 0.707107 (6 d.ps)

h = x - x0 = 2.0 - 1.5 = 0.5 (which is half of step size in the previous example)
=>

f(1.5) = 0.8165,
f(1.5) = - 0.4536,

f(1.5) = - 0.2722,
f (4)(1.5) = 1.0584,

f(1.5) = 0.2722
f(5)(1.5) = - 3.1753

Order of approximations

t %

0th: f(2.0) 0.8165

15.47

1st: f(2.0) 0.816 + (-0.2722)h

0.8165 + (-0.2722)(0.5) 0.6804

3.77

2nd: f(2.0) 0.8165 + (-0.5)h + (0.2722/2)h2

0.6804 + (0.1361)(0.5)2 0.7144

1.032

3rd: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3

0.7144 + (-0.0756)(0.5)3 0.7050

0.2.97

4th: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 +

(1.0584/24)h4
0.7050 + (0.0441)(0.5)4 0.7077
5th: f(2.0) 0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 +
(1.0584/24)h4 + (-3.1753/120)h5
0.7077 + (-0.02646)(0.5)5 0.7069

0.0849

0.0283
0.00745
0.00399

0.70715, 0.70707

Note: the approximate value approaches the true value faster when the step size is reduced. After
the fifth-order approximation, the result is accurate at least up to 3 d.ps.

27

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

(iv) Truncation Error Associated With TSE:

Denoting

xi = x0
xi+1 = x

and using the zero-order TSE, we write

f(xi+1) f(xi)

Ro

with the remainder,

(1-39)

f '( xi )
f ''( xi ) 2
h
h . . . Rn
1!
2!

(1-40)

We may approximate the zero-order remainder as

Ro f '( xi )h

(1-41)

Figure 1.8(a) illustrates the concept of error approximation in TSE.

f(xi+1)

f(x)

f(x)

f(x)

f(x)

f()
Exact

R0

slope = R0/h

f(xi)
0th-order

xi+1

xi

xi

(a)

xi+1

(b)
Figure 1.8: Graphical depiction of error in TSE

Using derivative mean-value theorem: if a function f(x) and its first derivative are continuous
over an interval from xi and xi+1, then there exists at least one point on the function that has a
slope, designated by f(), that is parallel to the line joining f(xi ) and f(xi+1 ).

28

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Thus, the slope f() is equal to the rise Ro divided by the run h, or

f '( )

Ro
h

=>

Ro f '( ) h

(1-42)

Similarly, the first-order and second-order remainders are, respectively

R1

f ''( ) 2
h
2!

(1-43)

R2

f '''( ) 3
h
3!

(1-44)

(v) Application: Truncation Error of the Derivatives

Using TSE and truncating after the first derivative term, we can write

f ( xi 1 ) f ( xi ) f '( xi )( xi 1 xi ) R1

(1-45)

and solving for the first derivative term yield

f '( xi )

f ( xi 1 ) f ( xi )
R1

xi 1 xi
xi 1 xi

st
Truncation
1 order
error

(1-46)

approximation

From eq.(1-43) and eq.(1-46), we can then estimate the truncation error associated with the
approximation of the first derivative using TSE as

R1
f ''( )

h
h
2!
or

R1
( h)
h

where

h = xi+1 - xi

(1-47)

We conclude that the estimate of the first derivative using TSE has a truncation error of order of
h i.e. the error of the first derivative approximation is proportional to the step size. It follows
that the error will be halved if the step size is halved.

29

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

(vi) Finite Difference Approximations of First Derivative

There are three different methods for the approximation of the first derivative.
a) Forward difference method (FDM):
From eq.(1-45) and eq.(1-47) with h = xi+1 - xi , we may write eq.(1-45) in the form

f (xi 1 ) f (xi ) f ' (xi ) h R1

(1-48)

Rearranging terms to give the finite difference formula for the first derivative as

f ' ( xi )

where

f ( xi 1 ) f ( xi )
(h)
h

or

f '( xi )

f i the first forward difference

and
f i
the first finite divided difference.
h

f i
(h)
h

(1-49)

h step size

b) Backward difference method (BDM):

TS can also be expanded backwards to approximate the previous value f(xi-1 ) based on the
present value f(xi ) as

f (xi 1 ) f ( xi ) f ' ( xi )h

f '' (xi ) 2 f ' '' (xi ) 3

h
h . . . (1-50)
2!
3!

Truncating all terms after the first derivative term and rearranging the equation yields

f ' ( xi )
where

f ( xi ) f ( xi 1 )
( h )
h

or

fi the first backward difference.

30

f ' ( xi )

f i
( h )
h

(1-51)

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

c) Centered (or central) difference method (CDM):

This can be derived by subtracting the backward TSE from the forward TSE which yields

f ( xi 1 ) f ( xi 1 ) 2 f ' ( xi ) h

f ' ' ' ( xi ) 3

h ...
3

(1-52)

Again solving for the first derivative and rearranging the terms to give

f '( xi )

f ( xi 1 ) f ( xi 1 ) f '' ' (xi ) 2

h
2h
6

f ' ( xi )

or

f ( xi 1 ) f ( xi 1 )
( h2 )
2h

(1-53)

Notice that the error for the central different method is of the order of h2 as compared to h for
the forward and backward difference methods. Therefore, central difference method gives better
approximation of the derivative than the other two methods.
The three methods can be depicted graphically as shown in figure 1.9 below.
f(xi+1)

f(xi+1)
f(xi)

f(xi)

f(xi )

exact

exact

exact

f(xi-1 )

f(xi-1)
xi

(a) FDM

xi+1

xi-1

xi

(b) BDM

xi-1

xi

xi+1

(c) CDM

Figure 1.9: Approximation of first derivative.

We will study numerical differentiation in more details in part Five. This includes the higher
accuracy formulas and also higher order derivatives.

31

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

Numerical Differentiation for First Derivative.

Example 1.7:

Use forward, backward and centered difference to estimate the first

derivative of the function

f (x ) 0.5x 3 0.1x2 0.8x 0.7

at x = 0.5 using a step size h = 0.5. Also calculate the true percent relative
error for each approximation.

Solution:

True value,

f ' (x ) 15
. x 2 0.2 x 0.8
f '(0.5) 15
. (0.5)2 0.2 (0.5) 0.8 = 1.075

With h = 0.5 and at x = 0.5, they imply that

xi = 0.5 => f(xi) = -0.2625
xi-1 = 0 => f(xi-1) = -0.7
xi+1 = 1.0 => f(xi+1) = 0.5
Using eqs.(1-43), (1-45), and (1-47), we obtain
t

FDM:

BDM:

CDM:

0.5 ( 0.2625)
1525
.
0.5

41.9

( 0.2625) ( 0.7 )
0.875
0.5

18.60

0.5 ( 0.7 )
1200
.
10
.

11.63

f ' (0.5)

f ' (0.5)

f ' (0.5)

32

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

Numerical Differentiation for First Derivative.

Worksheet 1.6:

Use forward, backward and centered difference to estimate the first

derivative of the function

f (x ) 0.5x 3 0.1x2 0.8x 0.7

at x = 0.5 using a step size h = 0.25. Also calculate the true percent relative error for each
approximation.

Solution:

True value,

f ' (x ) 15
. x 2 0.2 x 0.8
f '(0.5) 15
. (0.5)2 0.2 (0.5) 0.8 = 1.075

With h = 0.25 and at x = 0.5 imply that

xi = 0.5
xi-1 = 0.25
xi+1 = 0.75

=>
=>
=>

f(xi) = -0.2625
f(xi-1) = -0.4984375
f(xi+1) = 0.0546875
t

FDM

BDM

CDM

( 0.0546875) ( 0.2625)
126875
.
0.25

18.02

f '( 0.5)

( 0.2625) ( 0.4984375)
0.94375
0.25

12.21

f '(0.5)

0.0546875 ( 0.4984375)
110625
.
0.5

f '(0.5)

2.91

We observe that the CDM is more accurate than the FDM and BDM. Also, halving the step size
approximately halves the error of FDM and BDM, and quarters the error of CDM.
End of Part One

33

Introduction to Numerical Methods

and Error Analysis

Numerical Methods
with Applications

Checklist:
In part One, I have acquired the following knowledge and understanding:
Important formula/thing
Topics/Concepts
Y N
to remember
1. Objectives
2. Introduction to Numerical Methods:
a. Mathematical modeling

b. Differences between analytical &

numerical methods
3. Error analysis:
a. significant digits

b. accuracy and precision

c. Error quantification

d. Error bound and propagation

e. Error in numerical analysis

4. Approximation of function:
a. TSE
b. Remainder

c. Truncation error in TSE

d. Finite difference of the first

derivative

34

Need
Guide

Numerical Methods
with Applications

Introduction to Numerical Methods

and Error Analysis

35