(EE3006/3006A)

Unit 1. Numerical Analysis Basics

1. Errors
• Errors definitions
• The causes of errors
• Error analysis using Taylor series
• Error propagation

2. Numerical Differentiation
• Finite-divided difference approximation

3. Numerical Integration
• Trapezoidal rule

(EE3006/3006A)

1. Errors
=> Numerical errors arise from the use of approximations in computer-
based calculations to represent exact mathematical operations and
quantities:
True value = approximation + error

 The errors associated with calculations and measurements can be

qualitatively characterized by using accuracy and precision:

Accuracy: refers to how closely a computed or measured value agrees with the
true value.
Precision: refers to how closely individual computed or measured values agree
with each other.

1
 (EE3006/3006A)

Increasing accuracy

Increasing precision

True value

Accuracy: how closely a computed or measured value agrees with the true value.
Precision: how closely individual computed or measured values agree with each other. 3

(EE3006/3006A)

1.1 Error Definitions

 “True” error: Et  true value  approximation
true value  approximation
 Relative error:  t  100%
true value

 Relative approximation error:

As the true value is rarely available in many situations, we often use the
relative approximation error:
approximation error
a  100%
approximation
present approximation - previous approximation
 100%
present approximation
4

2
 (EE3006/3006A)

 Error bound s :
=> Determine how big the errors of the calculation (or measurement)

i. To define (calculation or iteration) stopping criterion:

|  a |  s (error bound , or acceptable level )

-> Usually not concern with sign, but with prefixed percent tolerance.

ii. To define a criterion assuring that a numerical result is correct

to n significant figures as:

 
Proof: the smallest percent tolerance for a
 a   s  0.5102n % number with n significant figures is
|  0.5 | 0.5
Biggest acceptable error  or 0.5  10 2  n %
Biggest integer with 99  9 9 1 0 n
n digital numbers
n
5

(EE3006/3006A)

Example 1:
Use Maclaurin series expansion of e x :

x 2 x3 xn
e  1  x    .... ,
x
n = 1, 2, ….
2! 3! n!
(use a polynomial to approximate exponential)

to calculate e0.5
until the prespecified error criterion
conforming to 3 significant figures.

3
 (EE3006/3006A)

Solution:
1. The error bound for the requirement of 3 significant figures is:
 s  (0.5 1023%)  0.05%
2. With the Maclaurin series expansion, we can calculate the approximation
term by term: x 2 x3
e x  1  x    ....
xn
2! 3! n!
n Result εa (%) εt (%)
1 39.3
1 1.5 33.3 9.02
2 1.625 7.69 1.44
3 1.645833333 1.27 0.175
4 1.648437500 0.158 0.0172

5 1.648697917 0.0158 0.00142

where the true relative error is calculated with the true value of e0.5, i.e.
1.648721. 7

(EE3006/3006A)

1.2 The Causes of Errors

 Round-off Errors
• Round-off errors arise because digital computers cannot exactly
represent some quantities, such as irrational numbers.
• Two major facets of round-off errors involved in numerical
calculations:
1) digital computers have size and precision limits on their ability to
represent numbers.
2) certain numerical manipulations are highly sensitive to round-
off errors. This can result from both mathematical consideration as
well as from the way in which computers perform operations.

4
 (EE3006/3006A)

 Truncation Errors
• A truncation error is caused by truncating an infinite number
and approximating it by a finite number;
• Truncation error also includes discretization error, which arises
from taking a finite number of steps to approximate an infinite
process;
• In some instances, round-off error is also called truncation error
especially when the number is rounded by truncation.

=>How
=> To gain
to useinsight,
a mathematical
a widely tool
usedto analyze theseformulation
mathematical errors arisen
forin
numerical analysis?
error analysis is: TAYLOR SERIES.

(EE3006/3006A)

1.3 Error Analysis using Taylor Series

 Taylor series:
f ''  xi  2 f '''  xi  3
f  xi 1   f  xi   f '  xi  h  h  h 
2! 3!
f n  xi  n
......  h  Rn
n!
 stepsize)
where hh(step size  xi 1  xi ; Rn 
f   hn1
n1

xi    xi 1
 n 1!

=> provides a means to predict the function value at one point (i.e., xi+1)
by using the function value at another point (i.e., xi) and its
derivatives.

10

5
 (EE3006/3006A)

 The n-th order Taylor series expansion

• Zero-order approximation:
f  xi 1   f  xi 

(It is fine only if the function is almost a constant)

• First-order approximation
f  x i 1   f  x i   f '  x i  x i 1  x i 
{
(slope multiplied by distance)

• Second-order approximation
f ' '  xi 
f  xi 1   f  xi   f '  xi  xi 1  xi    xi 1  xi 2
2!
(can captures some of the curvature) 11

(EE3006/3006A)
f ''  xi  2
f  xi 1   f  xi   f '  xi  h  h 
2!
 Truncation error ...... 
f n  xi  n
h  Rn
n!
f     n1
n 1

Rn  h ( h  0 ) (xi < ξ <xi+1)

 n  1!
abbreviation
n 1
Rn  O(h )
 The term O(hn+1) means the truncation error at the order of hn+1;
 The order of truncation error indicates the accuracy of the
approximation.

For example:
1) if the error is O(h), halving the step size will halve the error;
2) if the error is O(h2), halving the step size will quarter the error!
12

6
 (EE3006/3006A)

1.4 Error Propagation

=> The total numerical error results from a combination of the
errors of all steps of calculations or measurements.

 Error Propagation Theorem:

1) In addition and subtraction, an error bound for the result is
given by the sum of the error bounds of the individual terms.
2) In multiplication and division, an approximate error bound
for the relative error of the result is given by the sum of error
bounds for the relative errors of the given numbers.

13

(EE3006/3006A)

Proof (of 1.a, addition case):

If write the error bounds of e1 and e2 as ε1 and ε2, respectively, then we know
|e1| < ε1 and |e2| < ε2

Since the sum of the true values x and y can be written as

the error bound of the absolute error of the sum approximation x+y then is

14

7
 (EE3006/3006A)

2. Numerical Differentiation
The definition of differentiation is:

xi

: the
It isoperation of finding
widely used the derivative,
to handle or instantaneous
many different kinds ofrate of change
differential
of a function with respect to its variable.
equations in science and engineering:
dV
 Ohm’s law: J  
dx
dz (t )
 Speed: v(t ) 
dt
 Fourier’s law: q  k dT
dx
15

(EE3006/3006A)

f(x)

f  xi  x 

Δy

f  xi 
Δx

xi xi+Δx x

Discrete analogue:  y 
f  xi  x  f  xi 
x xi x
16

8
 (EE3006/3006A)

2.1 Finite-Divided Difference Approximation

=> Finite-divided difference approximation is one of the widely
used methods to calculate the differentiation numerically based
on the first derivative in Taylor series expansion:

f ( x0 ) f (3) ( x0 )
f  x 1   f  x0   f   x0  ( x 1  x0 )  ( x 1  x0 ) 2  ( x 1  x0 )3....
2! 3!

 Forward difference approximation

 Backward difference approximation
 Centered difference approximation

17

(EE3006/3006A)

 Forward Difference Approximation (FDA)

f   xi 
f  xi 1   f  xi   f   xi  ( xi 1  xi )  ( xi 1  xi ) 2  ...
2 h2
h

 f   xi  h f f x ixi1f f xix1i O(h2 )

f   xi    O(h )
h (error)
(approximation)

FDA-2 with higher-order error :

 f  xi  2   4 f ( xi 1 )  3 f  xi 
f '  xi    O (h 2 )
2h
The error is O(h2). 18

9
 (EE3006/3006A)

 Backward Difference Approximation (BDA)

f ( xi ) 2
f  xi 1   f  xi   f   xi  h  h  ....
2!

f  xi   f  xi 1 
f   xi    O ( h)
h
(error)
(approximation)

BDA-2 with higher-order error :

3 f  xi   4 f  xi 1   f  xi  2 
f   xi    O(h 2 )
2h
The error is O(h2). 19

(EE3006/3006A)

 Centered Difference Approximation (CDA)

f ( xi ) 2
f  xi 1   f  xi   f   xi  h  h  .... +…
2!
f ( xi ) 2
f  xi 1   f  xi   f   xi  h  h  .... +…
2!

f ( 3) ( xi ) 3
f ( xi 1 )  f ( xi 1 )  2 f ' ( xi )h  2 h 
…
3!
which can be solved for
f fx ( x  )f x f ( x (3)
)   i1 i 1  i1  i f1 
) (xi ) 2 2
ff ' (xx
i i 
Oh(h )...
2h 2h 6 (error)
(approximation)

20

10
 (EE3006/3006A)

*Example 2:
Use a forward, a backward, and a centered finite-divided
difference approximation to calculate the first derivative of

f ( x )  0.1x 4  0.15 x 3  0.5 x 2  0.25 x  1.2

at x=0.5 using the step size of h=0.25.

(Hint: the true value of f’(0.5) is -0.9125).

21

(EE3006/3006A)

f  xi 1   f  xi 
Solution: f   xi  
h
( f ( x)  0.1x 4  0.15x3  0.5x 2  0.25x  1.2 ) f  xi   f  xi 1 
f   xi  
h
The data needed for this example are: f  xi 1   f  xi 1 
f   xi  
xi-1=0.25 f(xi-1)=1.1035156 2h
xi=0.5 f(xi)=0.925
xi+1=0.75 f(xi+1)=0.6363281
0.6363281  0.925
FDA: f   0.5    -1.15469
0.25

0.925  1.1035156
BDA: f   0.5   -0.714062
0.25

0.6363281  1.1035156
CDA: f   0.5   -0.934375
2  0.25
22

11
 (EE3006/3006A)

3. Numerical Integration
 Dictionary definition of integration - “to bring together, as
parts, into a whole; to unite; to indicate the total amount”
 Mathematically, definite integration is represented by
b
I   f ( x)dx
a

which stands for the integral of the function f(x) with respect
to the independent variable x, evaluated between the limits
from a to b.

23

(EE3006/3006A)

f(x)

a b x
b
I   f ( x)dx
a

 How to calculate the integral numerically and efficiently?

24

12
 (EE3006/3006A)

 Newton-Cotes Formulas
=> it is based on the strategy of replacing a complicated function or
tabulated data with a polynomial that is easy to integrate.
b b

I   f  x dx   f n  x dx
a a

f n  x   a0  a1 x .... an x n fn(x) is an n-th order

polynomial

25

(EE3006/3006A)

Newton-Cotes Integration Formulas

• Trapezoidal rule Corresponds to the case where the

polynomial is the first order
• Simpson’s 1/3 Rules
Corresponds to the case where the
• Unequal Segments function is a second order polynomial

• Open Integration

26

13
 (EE3006/3006A)

3.1 Trapezoidal Rule f (x)

b b Error (depending on h)
I   f  x dx   f 1  x dx f1(x)

a a

  f1(x)=a0+a1x.
where Integral Estimate

A straight line can be represented as:

a b
f  b  f  a 
f1  x  f  a   x  a
ba
(=>the line goes through f(a) and f(b). )
The integral then is:
b
f (b)  f (a) f (a)  f (b)
I  [ f (a)  (x  a)]dx  h (h  b  a)
a
ba 2
27

(EE3006/3006A)

 Composite Trapezoidal Rule

 If the integral range is large, it can be equally divided by points
(x0, x1, x2… xn), i.e. n segments of
equal width:
ba
h
n
Then, the total integral can be
represented as:
x1 x2 xn

I  f ( x )dx   f ( x)dx  ...   f ( x)dx

x0 x1 xn 1

28

14
 (EE3006/3006A)

By substituting the trapezoidal rule for each integral yields

f (x0)  f (x1) f (x )  f (x2 ) f (x )  f (xn )

I h h 1 ... h n1
2 2 2

After grouping terms, it can be rewritten as:

h n 1

I 
2
f ( x0 )  2 
i 1
f ( xi )  f ( xn )


29

(EE3006/3006A)

*Example 3:
Use the two-segment trapezoidal rule to estimate the integral
of
f ( x)  0.2  25x  200x 2  675x3  900x 4  400x5

from a=0 to b=0.8.

Solution:
For n=2, one has the step size h=0.4
f (0)  0.2, f (0.4)  2.456, f (0.8)  0.232
0.2  2(2.456)  0.232
I  0.4  1.0688
2
30

15
 (EE3006/3006A)

Classwork 1:

31

(EE3006/3006A)

Reference:
(S.C. Chapra, Applied numerical methods with MATLAB for engineers and
scientists, McGraw Hill, 2008)

1. Errors (pp. 58-67)

2. Numerical Differentiation (pp.72-76)
3. Numerical Integration (pp.285-290)

32

16