Computational Methods

CHAPTER-ONE

NUMERICAL ERROR ANALYSIS AND NUMBER

SYSTEM

3/22/2024 1
 CONTENTS
• Introduction to Numerical Methods
• Measuring Errors
• Sources of Errors
• Taylor Theorem Revision
• Reading Assignment

✓ Binary Representation of
Numbers
✓ Floating Point Representation
✓ Propagation of Errors

3/22/2024 2
 Part I
Introduction to Numerical Methods
• A numerical method is a complete and unambiguous set of
procedures for the solution of a problem, together with computable
error estimates.
• It uses algorithms for approximating solutions to problems.
• Mathematical models are an integral part in solving engineering
problems.
• It results in need of using mathematical procedures
➢ Differentiation and integration
➢Non-linear equations,
➢simultaneous linear equations,
➢curve fitting by interpolation or regression,

3/22/2024 3
Why use Numerical Methods?

To solve problems that cannot be solved exactly

x u2
1 −

2
 e 2 du
−

3/22/2024 4
 Steps in Solving Engineering Problem
.
Problem Description

Mathematical Model

Solution of Mathematical Model

Using the Solution

3/22/2024 5
MEASURING ERRORS
Why do we measure errors?
1) To determine the accuracy of numerical results.
2) To develop stopping criteria for iterative algorithms.

3/22/2024 6
 True Error

• Defined as the difference between the true value in

a calculation and the approximate value found using
a numerical method etc.

True Error = True Value – Approximate Value

3/22/2024 7
 Example—True Error

The derivative, f (x) of a function f (x) can be

approximated by the equation,
f ( x + h) − f ( x)
f ' ( x) 
h

If f (x) = 7e and h = 0.3

0.5 x

a) Find the approximate value of f ' (2)

b) True value of f ' (2)
c) True error for part (a)

3/22/2024 8
 Example (cont.)

Solution:
a) For x = 2 and h = 0.3
f (2 + 0.3) − f (2)
f ' (2) 
0.3
f (2.3) − f (2)
=
0.3
7e0.5(2.3) − 7e0.5(2)
=
0.3
22.107 − 19.028
= = 10.263
0.3

3/22/2024 9
 Example (cont.)

Solution:
b) The exact value of f ' (2) can be found by using
our knowledge of differential calculus.
f (x) = 7e0.5 x
f ' (x) = 7  0.5  e0.5x
= 3.5e0.5x
So the true value of f ' (2) is
f ' (2) = 3.5e0.5(2)
= 9.5140
True error is calculated as
Et = True Value – Approximate Value
= 9.5140 −10.263 = −0.722
3/22/2024 10
 Relative True Error

• Defined as the ratio between the true error, and the true value.

True Error
Relative True Error ( t ) =
True Value

3/22/2024 11
 Example—Relative True Error

Following from the previous example for true error,

find the relative true error for f (x) = 7e0.5 x at f '(2)
with h = 0.3
From the previous example,
Et = −0.722
Relative True Error is defined as
True Error
t =
True Value
− 0.722
= = −0.075888
9.5140
as a percentage,
 t = −0.075888 100% = −7.5888%
3/22/2024 12
 Approximate Error

• What can be done if true values are not known or are very difficult to
obtain?
• Approximate error is defined as the difference between the present
approximation and the previous approximation.

Approximate Error ( E a ) = Present Approximation – Previous Approximation

3/22/2024 13
 Example—Approximate Error

For f (x) = 7e0.5 x at x = 2 find the following,

a) f (2) using h = 0.3
b) f (2) using h = 0.15
c) approximate error for the value of f (2) for part b)
Solution:
a) For x = 2 and h = 0.3
f ( x + h) − f ( x)
f ' ( x) 
h
f (2 + 0.3) − f (2)
f ' (2) 
0.3

3/22/2024 14
 Example (cont.)

Solution: (cont.)
f (2.3) − f (2)
=
0.3
7e0.5(2.3) − 7e0.5(2)
=
0.3
22.107 − 19.028
= = 10.263
0.3
b) For x = 2 and h = 0.15
f (2 + 0.15) − f (2)
f ' (2) 
0.15
f (2.15) − f (2)
=
0.15
3/22/2024 15
 Example (cont.)

Solution: (cont.)
7e 0.5(2.15) − 7e 0.5(2)
=
0.15
20.50 − 19.028
= = 9.8800
0.15

c) So the approximate error, Ea is

Ea = Present Approximation – Previous Approximation
= 9.8800 − 10.263
= −0.38300

3/22/2024 16
Relative Approximate Error

• Defined as the ratio between the approximate

error and the present approximation.

Approximate Error
Relative Approximate Error ( a) =
Present Approximation

3/22/2024 17
 Example—Relative Approximate Error

For f (x) = 7e0.5 x

at x = 2 , find the relative approximate
error using values from h = 0.3 and h = 0.15
Solution:
From Example 3, the approximate value of f (2) = 10.263
using h = 0.3 and f (2) = 9.8800 using h = 0.15
Ea = Present Approximation – Previous Approximation
= 9.8800 − 10.263
= −0.38300

3/22/2024 18
 Example (cont.)

Solution: (cont.)
Approximate Error
a =
Present Approximation
− 0.38300
= = −0.038765
9.8800
as a percentage,
 a = − 0 . 0 3 8 7 6 5  1 0 0 % = − 3. 8765%

Absolute relative approximate errors may also need to

be calculated,
 a =| −0.038765 | = 0.038765 or 3.8765%

3/22/2024 19
 How is Absolute Relative Error used as a
stopping criterion?

If |a |  s where  s is a pre-specified tolerance, then

no further iterations are necessary and the process is
stopped.

If at least m significant digits are required to be

correct in the final answer, then
|a | 0.510 2−m%

3/22/2024 20
 Table of Values

For f (x) = 7e at x = 2 with varying step size, h

0.5 x

h f (2) a m
0.3 10.263 N/A 0

0.15 9.8800 3.877% 1

0.10 9.7558 1.273% 1

0.01 9.5378 2.285% 1

0.001 9.5164 0.2249% 2

3/22/2024 21
 SOURCES OF ERROR
There are two sources of numerical error
1) Round off error
2) Truncation error
Round-off Error
▪ Caused by representing a number approximately
• Most decimal numbers cannot be represented with complete
accuracy in a computer.
• The numbers are stored in fixed-point format or floating-point
format.
1
 0.333333
3
2  1.4142...

3/22/2024 22
Truncation error
• Error caused by truncating or approximating a mathematical
procedure.
Example of Truncation Error
• Taking only a few terms of a Maclaurin series to approximate
ex 2 3
x x
e x = 1+ x + + +..................
• If only 3 terms are used, 2! 3!


x −  1+ x + x 2

Truncation Error = e 
 2! 

3/22/2024 23
 Another Example of Truncation Error

Using a finite x to approximate f (x)

f (x + x) − f (x)
f  (x) 
x

secant line
P

tangent line

3/22/2024 24
 Another Example of Truncation Error

Using finite rectangles to approximate an

integral.
y

90

y = x2
60

30

0 x
0 1.5 3 4.5 6 7.5 9 10.5 12

3/22/2024 25
 Example 1 —Maclaurin series

Calculate the value of e with an absolute

1.2

relative approximate error of less than 1%.

1.22 1.23
e 1.2
= 1+1.2 + + + .................
2! 3!
n
e1.2 Ea a %
1 1
2 2.2 1.2 54.545
3 2.92 0.72 24.658
4 3.208 0.288 8.9776
5 3.2944 0.0864 2.6226
6 3.3151 0.020736 0.62550

6 terms are required. How many are required to get at least 1

significant digit correct in your answer?
3/22/2024 26
 Example 2 —Differentiation

f (x + x) − f (x)
Find f (3)
for f (x) = x 2 using f  (x) 
x
and x = 0.2 =
3.2 2 − 32
f (3 + 0.2) − f (3)
f (3) =
'

0.2 0.2
f (3.2) − f (3) 10.24 − 9 1.24
= = = = 6.2
0.2 0.2 0.2

The actual value is

f ' (x) = 2x, f ' (3) = 2 3 = 6

Truncation error is then, 6 − 6.2 = −0.2

Can you find the truncation error with
3/22/2024 27
 Example 3 — Integration

Use two rectangles of equal width to

approximate the area under the curve for
f (x) = x 2 over the interval [3,9]
y
9
90
x 2
dx
3

y = x2
60

30

0 x
0 3 6 9 12

3/22/2024 28
 Integration example (cont.)
Choosing a width of 3, we have
9

 x 2
dx = (x 2
)
x=3
(6 − 3) + (x 2 )
x=6
(9 − 6)
3
Can you find the truncation error
= (3 )3 + (6 )3
2 2
with 4 rectangles?
= 27 +108 = 135 y

Actual value is given by 90

   9 3 − 33 
9 3 9
 x dx =  3  =  3  = 234
x y = x2
2
60

3  3

Truncation error is then

30

234 − 135 = 99 0 x
0 1.5 3 4.5 6 7.5 9 10.5 12

3/22/2024 29
TAYLOR SERIES REVISITED
What is a Taylor series?
Some examples of Taylor series which you must have seen

x 2 x 4 x6
cos(x) = 1− + − +
2! 4! 6!
x3 x5 x 7
sin(x) = x − + − +
3! 5! 7!
x2 x3
e = 1+ x +
x
+ +
2! 3!

3/22/2024 30
 General Taylor Series

The general form of the Taylor series is given by

f  (x) 2 + f (x) h3 +
f (x + h) = f (x)+ f (x)h + h
2! 3!
provided that all derivatives of f(x) are continuous and
exist in the interval [x,x+h]

What does this mean in plain English?

As Archimedes would have said, “Give me the value of the function
at a single point, and the value of all (first, second, and so on) its
derivatives at that single point, and I can give you the value of the
function at any other point” (fine print excluded)

3/22/2024 31
 Example—Taylor Series

Find the value of f (6) given that f (4) = 125, f (4) = 74,
f  (4) = 30, f (4) = 6 and all other higher order derivatives
of f (x) at x = 4 are zero.

Solution:
h2 h3
f (x + h ) = f (x )+ f (x)h+ f  (x ) + f (x) +
2! 3!
x=4
h = 6−4 = 2

3/22/2024 32
 Example (cont.)

Solution: (cont.)
Since the higher order derivatives are zero,
22 23
f (4+ 2 ) = f (4 )+ f (4)2+ f  (4) + f (4)
2! 3!
 2 2   23 
f (6) = 125 + 74 (2)+ 30  + 6 
 2!   3! 
= 125 + 148 + 60 + 8
= 341
Note that to find f (6) exactly, we only need the value
of the function and all its derivatives at some other
point, in this case x = 4
3/22/2024 33
 Derivation for Maclaurin Series for ex

Derive the Maclaurin series

x2 x3
e = 1+ x +
x
+ +
2! 3!
The Maclaurin series is simply the Taylor series about
the point x=0
h2 h3 h4 h5
f (x + h) = f (x )+ f (x )h + f  (x ) + f (x) + f (x) + f (x) +
2! 3! 4 5
h2 h3 h4 h5
f (0 + h )= f (0 )+ f (0)h + f  (0) + f (0) + f (0) + f (0) +
2! 3! 4 5

3/22/2024 34
 Derivation (cont.)

Since f (x) = e x , f (x) = e x , f  (x) = e x , ..., f n (x) = ex and

f n (0) = e 0 = 1

the Maclaurin series is then

(e0 ) 2 (e0 ) 3
f (h) = (e ) + (e )h +
0 0
h + h ...
2! 3!
1 1
= 1+ h + h 2 + h 3 ...
2! 3!
So,
x 2 x3
f (x) = 1+ x + + +...
2! 3!

3/22/2024 35
 Error in Taylor Series

The Taylor polynomial of order n of a function f(x)

with (n+1) continuous derivatives in the domain
[x,x+h] is given by
2 n

f (x + h) = f (x)+ f ( x)h + f '' (x ) ++ f (n)(x ) + R n (x)

h h
2! n!
where the remainder is given by
R n (x) =
(x − h)n+1 (n+1)
f (c )
(n + 1)!
where
x  c  x+h

that is, c is some point in the domain [x,x+h]

3/22/2024 36
 Example—error in Taylor series

The Taylor series for e x at point x = 0 is given by

x 2 x3 x 4 x5
e =1+ x +
x
+ + + +
2! 3! 4! 5!
It can be seen that as the number of terms used
increases, the error bound decreases and hence a
better estimate of the function can be found.
How many terms would it require to get an
approximation of e1 within a magnitude of
true error of less than 10-6.

3/22/2024 37
 Example—(cont.)

Solution:
Using (n + 1) terms of Taylor series gives error bound of
Rn (x) =
(x − h)n+1 (n+1)
f (c) x = 0, h = 1, f (x) = e x
(n + 1)!

Rn (0) =
(0 −1)n+1 (n+1)
(n + 1)! f (c)
=
( −1)n+1 c
(n +1)! e
Since
x  c  x+h
0  c  0 +1  Rn (0) 
1 e
0  c 1 (n +1)! (n +1)!
3/22/2024 38
 Example—(cont.)

Solution: (cont.)
So if we want to find out how many terms it would
require to get an approximation of e1 within a
magnitude of true error of less than 10−6 ,
e
 10−6
(n +1)!

(n +1)! 106 e
(n +1)! 106  3
n9
So 9 terms or more are needed to get a true error
less than 10−6
3/22/2024 39
 Part 2 Reading Assignment

BINARY REPRESENTATION
• In a binary system, we have the base made of only two digits 0
and 1. So it is a base 2 system.
• For example, look at the number 257.56. Each digit in 257.56
has a value of 0 through 9 and has a place value. It can be
written as

257.76 = 2  1 0 2 + 510 1 + 7  1 0 0 + 7  1 0 − 1 + 610 − 2

Base 2
 (1 23 + 0 22 +1 21 +1 20 ) 
(1011.0011)2 =  
 + (0 2 −1 + 0 2 −2 +1 2 −3 +1 2 −4 ) 
 10
= 11.1875
3/22/2024 40
 Convert Base 10 Integer to binary
representation
Table 1 Converting a base-10 integer to binary representation.
Quotient Remainder

11/2 5 1 = a0
5/2 2 1 = a1
2/2 1 0 = a2
1/2 0 1 = a3
Hence
(11)10 = (a3 a2 a1a0 ) 2
= (1011)2

3/22/2024 41
 Start

Integer N to be
Input (N) 10
converted to binary
format

i=0

Divide N by 2 to get
quotient Q & remainder R

i=i+1,N=Q
ai = R

No
Is Q = 0?

Yes

n=i
(N)10 = (an. . .a0)2

STOP

3/22/2024 42
 Fractional Decimal Number
to Binary
Table 2. Converting a base-10 fraction to binary representation.

Number Number after Number before

decimal decimal
0.1875 2 0.375 0.375 0 = a−1
0.375 2 0.75 0.75 0 = a−2
0.75 2 1.5 0.5 1 = a−3
0.5 2 1.0 0.0 1 = a−4

Hence
(0.1875)10 = (a−1a−2a−3a− 4 )2
= (0.0011)2

3/22/2024 43
 Start

Fraction F to be
Input (F) 10
converted to binary
format
i = −1

Multiply F by 2 to get
number before decimal,
S and after decimal, T

i = i −1, F = T
ai = R

No
Is T =0?

Yes

n=i
(F)10 = (a-1. . .a-n)2

STOP

3/22/2024 44
Decimal Number to Binary

(11.1875)10= ( ?.? )2
Since
(11)10 = (1011) 2
and
(0.1875)10 = (0.0011)2

we have
(11.1875)10 = (1011.0011)2

3/22/2024 45
 All Fractional Decimal Numbers Cannot
be Represented Exactly
Table 3. Converting a base-10 fraction to approximate binary representation.
Number Number
Number after before
decimal Decimal
0.3 2 0.6 0.6 0 = a−1
0.6  2 1.2 0.2 1 = a−2
0.2  2 0.4 0.4 0 = a−3
0.4  2 0.8 0.8 0 = a−4
0.8  2 1.6 0.6 1 = a−5

(0.3)10  (a−1a−2a−3a−4a−5 )2 = (0.01001)2 = 0.28125

3/22/2024 46
 Another Way to Look at Conversion

Convert (11.1875)10 to base 2

(11)10 = 23 + 3
= 23 + 21 +1
= 23 + 21 + 20
= 1 23 + 0 22 +1 21 +1 20
= (1011)2

3/22/2024 47
 (0.1875)10 = 2−3 + 0.0625
=2 +2
−3 −4

= 0 2−1 + 0 2−2 +1 2−3 +1 2−4

= (.0011)2

(11.1875)10= (1011.0011)2
3/22/2024 48
 FLOATING POINT REPRESENTATION

• To keep the relative error of similar order for all numbers,

one may use a floating-point representation of the number.
For example, in floating-point representation, a number
Scientific Form:

256.78 is written as + 2 . 5 6 7 8  1 0 2

0.003678 is written as + 3 . 6 7 8  1 0 − 3
− 256.78 is written as − 2 . 5 6 7 8  1 0 2

3/22/2024 49
Example

The form is   m10e

or
sign mantissa 10exponent

Example: For

− 2.5678102
 = −1
m = 2.5678
e=2
3/22/2024 50
Floating Point Format for Binary Numbers

y =   m 2e
 = sign of number (0for + ve,1for - ve)
m = mantissa (1)2  m  (10)2 

1 is not stored as it is always given to be 1.

e = integer exponent

3/22/2024 51
Example
9 bit-hypothetical word
▪ the first bit is used for the sign of the number,
▪ the second bit for the sign of the exponent,
▪ the next four bits for the mantissa, and
▪ the next three bits for the exponent
(54.75)10 = (110110.11)2 = (1.1011011)2  2 5
 (1.1011)2  (101)2
We have the representation as

0 0 1 0 1 1 1 0 1
mantissa exponent
Sign of the Sign of the
number exponent

3/22/2024 52
 Machine Epsilon
Defined as the measure of accuracy and found
by difference between 1 and the next number
that can be represented

3/22/2024 53
Example

Ten bit word

▪ Sign of number
▪ Sign of exponent
▪ Next four bits for exponent
▪ Next four bits for mantissa
0 0 0 0 0 0 0 0 0 0 = (1)10
Next
number
0 0 0 0 0 0 0 0 0 1 = (1.0001)2 = (1.0625)10

mach = 1.0625 −1 = 2−4

3/22/2024 54
Relative Error and Machine Epsilon

The absolute relative true error in representing

a number will be less then the machine epsilon
Example
(0.02832)10  (1.1100)2 2 −6 = (1.1100 )2 2 −(0110)2
10 bit word (sign, sign of exponent, 4 for exponent, 4 for mantissa)

0 1 0 1 1 0 1 1 0 0
Sign of the exponent mantissa
Sign of the
number
exponent

(1.1100)2  2 −(0110 ) 2
= 0.0274375
0.02832 − 0.0274375
a =
0.02832
3/22/2024 = 0.034472  2 −4 = 0.0625 55
 IEEE-754 Floating Point Standard
• Standardizes representation of floating point
numbers on different computers in single and double
precision.

• Standardizes representation of floating point

operations on different computers.

3/22/2024 56
 IEEE-754 Format Single Precision

32 bits for single precision

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Sign Biased Mantissa (m)

(s) Exponent (e’)

Value = (−1)  (1 s
m)2
 2.e'−127

3/22/2024 57
 Example#1
1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Sign Biased Mantissa (m)

(s) Exponent (e’)

Value = (−1)  (1.m)2  2 e'−127

s

= (−1)  (1.10100000)2  2
1 (10100010 ) 2 −127

= (−1) (1.625) 2162−127

= (−1) (1.625)235 = −5.58341010

3/22/2024 58
 Example#2

Represent -5.5834x1010 as a single precision floating point

number.

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

Sign Biased Mantissa (m)

(s) Exponent (e’)

− 5.5834 10 10
= ( − 1) 1
 ( 1. ? )  2  ?

3/22/2024 59
Exponent for 32 Bit IEEE-754
8 bits would represent
0  e  255
Bias is 127; so subtract 127 from representation

−127  e  128

3/22/2024 60
 Exponent for Special Cases
Actual range of e
1  e  254
e = 0 and e = 255 are reserved for special numbers

Actual range of e
− 126  e  127

3/22/2024 61
Special Exponents and Numbers

e = 0 all zeros
e = 255 all ones

s e m Represents
0 all zeros all zeros 0
1 all zeros all zeros -0
0 all ones all zeros
1 all ones all zeros −
0 or 1 all ones non-zero NaN

3/22/2024 62
 IEEE-754 Format

The largest number by magnitude

(1.1........1)2  2127 = 3.40 10 38

The smallest number by magnitude
(1.00......0)2  2 −126
= 2.1810 −38

Machine epsilon
mach = 2−23 = 1.19 10 −7

3/22/2024 63
 PROPAGATION OF ERRORS

In numerical methods, the calculations are not made with exact

numbers. How do these inaccuracies propagate through the
calculations?

3/22/2024 64
 Example 1:
Find the bounds for the propagation in adding two numbers. For example if one is
calculating X +Y where
X = 1.5 ± 0.05
Y = 3.4 ± 0.04
Solution
Maximum possible value of X = 1.55 and Y = 3.44

Maximum possible value of X + Y = 1.55 + 3.44 = 4.99

Minimum possible value of X = 1.45 and Y = 3.36.

Minimum possible value of X + Y = 1.45 + 3.36 = 4.81 Hence

4.81 ≤ X + Y ≤4.99.

3/22/2024 65
 Propagation of Errors In Formulas

If f is a function of several variables X 1 , X 2 , X 3 ,.......,X n−1 , X n

then the maximum possible value of the error in f is

f f f f
f  X 1 + X 2 + .......+ X n−1 + X n
X 1 X 2 X n−1 X n

3/22/2024 66
 Example 2:

The strain in an axial member of a square cross-

section is given by
F
= 2
hE
Given
F = 72  0.9 N
h = 4  0.1 mm
E = 70 1.5 GPa

Find the maximum possible error in the measured

strain.
3/22/2024 67
 Example 2:
Solution
= 72
−3 2 9
(4 10 ) (70 10 )
= 64.286 10−6
= 64.286
  
 = F + h + E
F h E
3/22/2024 68
 Example 2:
 1  
= =−
2F F
=− 2 2
F h 2 E h h3 E E h E

Thus
1 2F F
E = 2 F + 3
h + 2 2
E
h E h E h E
1 2 72
=  0.9 +  0.0001
(4 10 ) (70 10 )
−3 2 9
(410 ) (70 10 )
−3 3 9

72
+ 1.5109

(4 10−3 )2 (70 109 )2

= 5.3955
Hence
= (64.286  5.3955)
3/22/2024 69
Example 3:
Subtraction of numbers that are nearly equal can create unwanted
inaccuracies. Using the formula for error propagation, show that
this is true.

Solution
Let
z = x− y
Then
z z
z = x + y
x y
= (1)x + (−1)y
= x + y
So the relative change is
z x + y
=
z x− y
3/22/2024 70
 Example 3:

For example if

x = 2  0.001
y = 2.003  0.001

z 0.001 + 0.001
=
z | 2 − 2.003 |
= 0.6667
= 66.67%

3/22/2024 71