MATH2089

Numerical Methods
Lecture 1

Part 1 Computing with real numbers

Dr Victoria Timchenko
School of Mechanical and Manufacturing
Engineering (J17)
Part1 Overview
 What is MATH2089 about?
 What are Numerical Methods?
 How we approach “Problem solving”
 Digital representation of numbers
 Floating Point Arithmetic
 Error analysis
Engineering Grand Challenges

Prediction of Weather, Climate and

Global Change

Weather Balloon
Ozone Model

Tornado Machine
Engineering Grand Challenges

Improvements in
Vehicle Performance

Wind Tunnel
Objectives of the Course

 Introduce the basics of numerical analysis

 Develop skills to apply numerical methods for
solving practical engineering problems
Syllabus
Week Topic Relevant
reading
1 Data representation, error analysis, introduction to MATLAB 1.3-1.6 & class
notes
2 Applied MATLAB programming. class notes
3 Nonlinear equations: bisection method, fixed point iteration, Newton- 2.3 – 2.6,2.14
Raphson and secant methods
4-5 Systems of linear equations: elimination methods, LU factorization, 3.3 - 3.19
Iterative methods, special linear systems
6 Interpolation and polynomial approximation, curve fitting 5.5 - 5.6,5.8 -
5.10
7 Numerical differentiation 7.1 – 7.10
8 Numerical integration 8.1 - 8.12
9 Euler method, Predictor-corrector methods 9.5-9.6, 9.10
10 Runge-Kutta method. Boundary value problems 9.7, 10.5-10.6

11 Parabolic equations. Methods of solutions 11.5

12 Elliptic and hyperbolic equations. Methods of solution 11.4-11.5, 11.9
Assessments
Numerical Methods
ASSESSMENT
DETAILS MARKS DUE DATE
COMPONENT

Lab Test administrated during

Matlab computer Test 12 Week 6
labs periods

Quiz (30-35 minutes long)

Tutorial Quiz 13 Week 7
administered during tutorials

Matlab computer
Satisfactory participation in Throughout
laboratory 15
laboratory classes Semester
participation

During exam period. The formal

June 2016
Final examination exam scripts will not be 60
returned.

Total 100
Labs/Tuts MATH2089
Week2: Lab work Matlab fundamentals (3)
Week3: Tutorial work on data representation, error analysis
Week4: Matlab laboratory work on nonlinear equations (3)
Week5: Tutorial work on nonlinear equations
Week6: 30-35 min Matlab test/ Matlab laboratory work on systems of
equations
Week7: 30-35 min tutorial Quizz on nonlinear equations and systems of
equations.
Week8: Laboratory work on curve fitting and polynomial approximation
(3)
Week9: Tutorial work on numerical differentiation and integration
Week10: Laboratory work on ODE (Euler & Heun methods) (3)
Week11: Tutorial work on ODE
Week12: Laboratory work on PDE (3)
Week13: Tutorial work on PDE & review.
5 labs each 3 =15 marks in total assessment
 What are Numerical Methods and why to study them?

Numerical methods are techniques by which mathematical

problems are formulated so that they can be solved with
arithmetic and logical operations.

Because digital computers excel at performing such operations,

numerical methods are sometimes referred as computational
methods.
Numerical and Analytical Methods

ax 2  bx  c  0 Quadratic equation

b  b2  4ac analytical solution:

x general, can be used for any a, b, c
2a
can be solved iteratively:
x  (bx  c ) / ax gives approximate solution,
does not always work!
Try with a=1, b=-3 and c=2
Why use Numerical Methods?

ax  b  0
Can be solved analytically
ax 2  bx  c  0

ax 2  b sin x  c  0 Cannot be solved analytically

ax 2.1  bx  c  0

Real engineering problems are complex

and described by non-linear equations –
no analytical solutions,
Have to be solved by numerical methods!
When are Numerical Methods used?
Example: Discharge of water from reservoir

v (t ) t
 tanh( 2 gh )
2 gh 2L

Find the value of h necessary for achieving a velocity

of v = 5 m/s at time t = 3 s when L = 5m
When are Numerical Methods used?

Solution of transcendental equation

When are numerical methods used?

Example: Electrical Circuit Analysis

Solution of system of linear equations

When are Numerical Methods used?

Benchmark data for velocity profile (Schlichting, 1968)

When are Numerical Methods used?

r
2vmax 0 r 1/ n
vave  2  r (1  ) dr
r0 0 r0

Numerical evaluation of integral I2

Mathematical Modeling and Problem Solving
1. Problem definition: State the problem clearly, including any
simplifying assumptions

2. Mathematical model: Develop a mathematical statement of the

problem in a form that can be solved for a numerical answer

3. Problem solving tools: Solve the equation(s) using graphics,

statistics, computer, analytical methods, numerical methods,
etc.

4. Implementation: Interpret the numerical result to arrive at a

decision. This will require experience and an understanding of
the situation. This is the hardest part of solving problems and
must be learned on the job
Example: Problem Solving
A parachutist of mass 68 kg jumps out of a stationary hot
air balloon. Compute velocity prior to opening the
parachute. The drag coefficient c = 12.5 kg/s and the
gravitational constant g = 9.8 m/s2.
Example: Solution

Problem definition
 Determine the terminal velocity of a free-falling body near
the earth’s surface
 Air resistance is linearly proportional to velocity, FU  cv
 Initial condition of v = 0 at t = 0
 v is dependent variable, t is independent variable
Example: Solution (con’t)

Mathematical model
dv  F
 Newton’s second law: F  ma 
dt m
dv c
 F  FD  FU  mg  cv  dt
g v
m
dv m  c 
a) Analytical method:  c   dt ln  g  v   t  A
g v c  m 
m
where A is an integrating constant
m
Initial condition of v = 0 at t = 0; A   ln g
c
gm
Analytical solution: v(t ) 
c
1  e  ( c m ) t  or v (t )  53.311  e 0.1838t 
Example: Solution (con’t)
b) Numerical method : The time rate change of velocity can be
approximated as:
v (ti 1 )  v (ti ) c
 g  v (ti )
ti 1  ti m
Numerical solution:
 c 
v (ti 1 )  v (ti )   g  v (ti )  (ti 1  ti )
New value Old value
 m  step size t

v (ti 1 )  v (ti )  9.8  0.1838v (ti )  (ti 1  ti )

Example: Solution (con’t)
Implementation
a) Analytical solution:
 The parachutist accelerates rapidly. A velocity of 44.83 m/s
(161.4 km/h) is attained after 10 s.
 After a sufficient long time, the terminal velocity of 53.31
m/s (191.9 km/h) is reached. This shows that the force of
gravity will be in balance with the air resistance, FD  FU
or mg  cvterminal 60
Terminal velocity

40
v, m/s

20

0
0 4 t, s 8 12
Example: Solution (con’t)
b) Numerical solution:
Velocity of parachutist
60

50

40
v (m/s)

30

20 Analytical Solution
Euler solution with dt=2s
Euler solution with dt=1s
RK solution
10

0
0 2 4 6 8 10 12 14 16 18 20
t (sec)
Conservation laws in engineering and science

Change =increase –decrease

For example,
for steady state incompressible fluid
flow in pipes

Flow in = flow out

Types of balances commonly used in engineering
Data Representation
In a digital computer, all information is stored in binary form. A bit is a
binary digit (i.e., a zero or one). A byte is larger unit, usually consists of 8
bits. A word consists of bytes, for example 32-bit word consists of 4 bytes
Most computers do arithmetic using the binary (base 2) number system.
They convert input (which is in base 10 numbers) to base 2, then perform
base 2 arithmetic and finally translate the answer into base 10

In decimal form:

N  ( ak  10k )  ( ak 1  10k 1 )  ...  ( a1  101 )  ( a0  100 )

N  ak ak 1...a2a1a0ten

In binary form:
( N )10  (ak  2k )  (ak 1  2k 1 )  ...  ( a1  21 )  ( a0  20 )
N  ak ak 1...a2 a1a0two
Binary Storage of Integers
 Integers can be exactly represented by base 2
Limitations:
 A finite number of bits are used to store each value
in computer memory
 Limiting the number of bits limits the size of integer
that can be represented

Determine the number in binary system:

(111)2 ? =1 x 20 +1 x 21 +1 x 22 =7
(1111)2 ?
(11111)2 ?
Internal Representation of Integers in Memory
Can be pictured as an ordered sequence of storage locations called
memory cells

215  1 to  215 (32767 to  32767)

231  1 to  231
Floating Point Arithmetic
All computers designed since 1985 use IEEE floating-point
arithmetic.
Most nonzero floating-point numbers are normalized. This
means they can be expressed as
x = ±(1 + f) · 2e
The quantity f is the fraction or mantissa and e is the
exponent. The fraction satisfies: 0 ≤ f < 1 and uses at
most 52 bits.
0 ≤ 252f < 252
The exponent e is an integer in the interval
-1022 ≤ e ≤ 1023
Floating Point Binary Representation

A double precision (64 bit) floating point real number in

memory can be schematically represented as

The finite number of bits in the exponent limits the

magnitude or range of the floating point numbers.
The finite number of bits in the mantissa limits the
number of significant digits or the precision of the
floating point numbers.
Digital Storage of Floating Point Numbers
The floating point mantissa is expressed in powers of
1/2
0 1 2 3
1 1 1 1
   1 is not used    0.5    0.25    0.125
2 2 2 2

The binary mantissa for 0.1 is (00011 0011 . . .)2.

The decimal value of 0.1 cannot be represented by a

finite number of binary digits.
Consequences of Finite Storage

Most real numbers cannot be stored exactly - they do

not exist on the floating point number line
Floating Point Number Line

It is a subset of the real number line

Matlab:
Machine Precision
The magnitude of round-off errors is quantified by machine
precision εm.
There is a number, εm > 0, such that
1+δ=1 whenever |δ| < εm.
In exact arithmetic 1 + δ = 1 only when δ = 0, so in exact
arithmetic εm is identically zero.
Matlab uses double precision (64 bit) arithmetic. The built-in
variables eps stores the value of εm
eps = 2.2204 × 10−16
Underflow, Overflow and Not-a-Number
Any number that is less than the smallest floating point
number underflows to 0

Any number larger than the largest floating point

number is said to overflow and resulted in Inf

Any result that is mathematically not defined is

represented by NaN
Floating Point Arithmetic
Floating Point Arithmetic in MATLAB

>>x=29/1300
x=0.223
>>y=29-1300*x
y=3.5527e-015

In exact arithmetic the value of y should be zero.

What is going on?
The Patriot Missile Failure
On February 25, 1991, during the Gulf War, an American Patriot Missile
battery in Dharan, Saudi Arabia, failed to track and intercept an incoming
Iraqi Scud missile. The Scud struck an American Army barracks, killing 28
soldiers and injuring around 100 other people.

It turns out that the cause was an inaccurate calculation of the time since
boot due to computer arithmetic errors. Specifically, the time in tenths of
second as measured by the system's internal clock was multiplied by 1/10
to produce the time in seconds. This calculation was performed using a
24 bit fixed point register.
24 bit timer stored 0.00011001100110011001100
Error = 0.0000000000000000000000011001100 . . . binary
≈ 9.5 10−8
Timer running 100 hours : error = 9.5 10−8 × 100 × 60 × 60 × 10 = 0.34
secs
Scud 1, 676 m/sec : error > 0.5 km
Error Analysis

If x is an approximation to x

The absolute error is Ex  x  x

xx
The relative error is Rx  , x  0.
x
A number x is considered to be an approximation
to the true value x to d significant digits if d is the
largest positive integer for which

x  x 1 d
 10
x 2
Sources of Error
(1) Errors in mathematical modelling: the simplifying approximations
and assumptions made in representing a physical system by
mathematical equation introduce error.
(2) Blunders: the programming errors, if undetected, introduce errors to
the computed values.
(3) Errors in input: due to the errors in data transfer and the
uncertainties associated with measurements.
(4) Machine errors: the floating point representation of numbers involves
rounding and chopping errors, as well as underflow and overflow
errors. These errors are introduced at each arithmetic operation
during computation.
(5) Truncation errors associated with the mathematical process: for
example, the approximate evaluation of an infinite series or an integral
involving infinity as a limit of integration involve computational errors.
Round-off Error
Since only a finite number of digits are stored in a
computer, the actual numbers may undergo chopping or
rounding of the last digit.
Effects of round-off usually accumulate slowly, but . . .
Subtracting nearly equal numbers leads to severe loss of
precision.
A similar loss of precision occurs when two numbers of
very different magnitude are added.
Round-off is inevitable: good algorithms minimize the
effect of round-off!
Catastrophic Cancellation Errors

Evaluate c = a + b with
a = x.xxx . . . × 100 and
b = y.yyy . . . × 10−8

The most significant digits of a are retained, but the least

significant digits of b are lost because of the mismatch in
magnitude of a and b
Catastrophic Cancellation Errors (con’t)

Consider c = a − b with
a = x.xxxxxxxxxxx1ssssss
b = x.xxxxxxxxxxx0tttttt
where x, y, s and t are decimal digits.

The digits sss . . . and ttt . . . are lost when a and b are
stored in double-precision, floating point format
Roots of Quadratic Equation

The roots of the quadratic equation: ax  bx  c  0

2

b  b2  4ac
x
2a

When is this formula susceptible to catastrophic

cancellation?
Find a mathematically equivalent formula that removes
this difficulty
Taylor Series

For a continuous function f(x) defined on the interval x

x  [a, b] we define the nth order Taylor Series
approximation Pn(x)
df ( x  x0 ) 2 d 2 f ( x  x0 ) n d n f
Pn ( x )  f ( x0 )  ( x  x0 )   ... 
dx x  x0 2! dx 2 x  x0
n! dx n x  x0

Then there exists  with x0    x such that

( x  x0 )( n 1) d ( n 1) f
f ( x )  Pn ( x )  Rn ( x ) where Rn ( x ) 
( n  1)! dx ( n 1) x 
Big “O” Notation – Same Order of Magnitude
Truncation Error
Truncation error is the discrepancy introduced by the use
of an approximate expression in place of an exact
mathematical expression or formula. For example consider
the Taylor series expansion of the function ln(1+x)
 (1)i 1 i
y ( x)  ln(1  x)   x 
i 1 i
1 1 1 1 1
 x  x 2  x 3  x 4  x 5  x 6  ...
2 3 4 5 6

Let the function y(x) be approximated by the first four terms

of the Taylor’s series expansion. The resulting discrepancy
between the exact function y(x) and the approximate
function is called the truncation error
Truncation Error- Example
Find the truncation error in approximating the function
y(x) = ln(1+x) by
y1 ( x)  x
1
y2 ( x )  x  x 2
2
1 1
y3 ( x)  x  x 2  x 3
2 3
1 1 1
y4 ( x )  x  x 2  x 3  x 4
2 3 4

Over the range 0<x<1

Total Numerical Error
Practical Considerations

1. x has a relative error of 10−p- means that x has

approximately p correct decimal digits
2. Never give an answer to more accuracy than your input
data
3. What error is acceptable in your engineering
computation?
10% or relative error of 0.1
6 significant decimal digits ⇐⇒ relative error ≤ 0.5 × 10−6
Practical Considerations (con’t)

4. Avoid subtracting two nearly equal numbers.

5. When adding and subtracting numbers, sort the
numbers and work with the smallest numbers first
6. Use Taylor series to analyze total numerical errors.
7. Try to estimate the accuracy of your numerical results.
Check if the results satisfy some condition or equation.
8. Employ sensitivity analysis and compare the results
with a different step size, method, model parameter or
input values.
Part 2 Starting with MATLAB

Matlab stands for Matrix laboratory

Some Advantages:

 No need to define the class and size of a variable

before using it
 No special commands and/or procedures are
required for running programs with size over 655,360
bytes
 Fast and convenient for matrix operations.
 Lots of powerful functions and toolboxes for
calculation and presentation
 Very comprehensive online help
 Some Disadvantages:

 Looping is relatively slow in Matlab when compared

with FORTRAN, Pascal and C
 Matlab programs must be run in Matlab environment!
Matlab alone can NOT produce any standalone
executable (exe) program
 This problem may be solved by the Matlab Compiler
Toolbox! This toolbox allows users to translate simple
Matlab m-files to C (or C++) source code files. Then,
one can use existing C compiler to produce the
corresponding executable (exe) files
 Command Window
Current Directory Enter commands at the prompt

MATLAB Windows
Command History
Workspace Window
 Scalar

Vector

3x4 Matrix

When you define variables in the command window, they are listed in
the workspace window
If you double click on any variable in the workspace window
MATLAB launches a document window containing the array
editor. You can edit variables in the array editor
The semicolon
suppresses the output
from each command
Figure Window
Editing Window

 This window allows you to type and save a series of

commands without executing them
 There are several ways to open an editing window
 From the file menu
 With the new file icon
Open an editing window
from the file menu or
with the new file icon
 Save and Run

Write your code in the editing window,

then run it using the Save and Run icon